2024-09-13T09:56:18.581 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T09:56:18.602 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T09:56:18.778 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334 branch: wip-aclamk-cbt-combined description: rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-low-osd-mem-target} rados supported-random-distro$/{ubuntu_latest} thrashers/careful_host thrashosds-health workloads/dedup-io-mixed} email: akupczyk@redhat.com first_in_suite: false job_id: '7903334' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-aclamk-cbt-combined ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr1: false ms bind msgr2: true ms inject socket failures: 5000 ms type: async osd_max_pg_log_entries: 2 osd_min_pg_log_entries: 1 osd_pg_log_trim_min: 0 osd_pool_default_min_size: 2 osd_pool_default_size: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true bluestore write v2 random: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd backoff on degraded: true osd backoff on peering: true osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd heartbeat use min delay socket: true osd max backfills: 3 osd max markdown count: 1000 osd max scrubs: 5 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd memory target: 1610612736 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub - \(POOL_APP_NOT_ENABLED\) sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 min_in: 4 workunit: branch: wip-aclamk-cbt-combined sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 owner: scheduled_akupczyk@teuthology priority: 160 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.4 - osd.8 - osd.12 - - mon.b - osd.1 - osd.5 - osd.9 - osd.13 - - mon.c - osd.2 - osd.6 - osd.10 - osd.14 - - mgr.x - osd.3 - osd.7 - osd.11 - osd.15 - client.0 seed: 5394 sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 sleep_before_teardown: 0 subset: 582/12160 suite: rados suite_branch: wip-aclamk-cbt-combined suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d58f73069a8157a9ace12a0247ee3e8497707ea9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 targets: smithi073.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGFNcWHIuhr/Fl6vAUWbx9f8Bpo8uPUcDtmfmVT/sDyxn3P0l+crjZyXrhqpHoXOBL+bC2+Ry4HyFVM3vp0Oews= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNLZ9gQu9hew3jFbGCVslEhFZIziBy7it4+2KwVKFW9YkrsupCrtMk3iMYkNw/WvY20pthgxzIao8dYjXDK6MIQ= smithi181.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBARhPDFf8PGrBpIEomhEneW0jTgM5aFSKvodrxvRyzDg1hjyvD7VismBFk0sIJGUcUDbQOpSnjZGgdjPlwPwbPc= smithi195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNgq2tN2MKdKem/Lz1ezMkGn+Z4nW/jpem18wdb9rFK/J7jALXCtLLbBvuH/moSYDtw+t7Gee+uzYFWdKYVGC9A= tasks: - install: null - ceph: conf: osd: debug monc: 20 - exec: mon.a: - while ! ceph balancer status ; do sleep 1 ; done - ceph balancer mode crush-compat - ceph balancer on - thrashosds: aggressive_pg_num_changes: false chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 min_in: 2 thrash_hosts: true timeout: 1200 - exec: client.0: - sudo ceph osd pool create low_tier 4 - rados: clients: - client.0 dedup_chunk_algo: fastcdc dedup_chunk_size: '131072' enable_dedup: true low_tier_pool: low_tier objects: 50 op_weights: read: 100 set_chunk: 30 tier_evict: 10 tier_flush: 5 tier_promote: 10 write: 50 ops: 1500 set_chunk: true teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_09:37:56 tube: smithi user: akupczyk verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T09:56:18.778 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d58f73069a8157a9ace12a0247ee3e8497707ea9/qa; will attempt to use it 2024-09-13T09:56:18.779 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d58f73069a8157a9ace12a0247ee3e8497707ea9/qa/tasks 2024-09-13T09:56:18.779 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T09:56:18.781 INFO:teuthology.task.internal:Checking packages... 2024-09-13T09:56:18.802 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd58f73069a8157a9ace12a0247ee3e8497707ea9' 2024-09-13T09:56:18.803 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T09:56:18.803 INFO:teuthology.packaging:ref: None 2024-09-13T09:56:18.803 INFO:teuthology.packaging:tag: None 2024-09-13T09:56:18.803 INFO:teuthology.packaging:branch: wip-aclamk-cbt-combined 2024-09-13T09:56:18.803 INFO:teuthology.packaging:sha1: d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T09:56:18.803 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-aclamk-cbt-combined 2024-09-13T09:56:18.953 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4697-gd58f7306-1jammy 2024-09-13T09:56:18.955 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T09:56:18.963 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T09:56:18.963 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T09:56:19.019 INFO:teuthology.task.internal:Saving configuration 2024-09-13T09:56:19.036 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T09:56:19.043 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T09:56:19.070 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 09:50:37.484381', 'locked_by': 'scheduled_akupczyk@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJgRK7C73H+j1J5dWGg52t+GUqwweGrb+Ed9wtVSKHW5'} 2024-09-13T09:56:19.092 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 09:50:37.482929', 'locked_by': 'scheduled_akupczyk@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGYY1rXWoDTj8mcZ+9KI96sQzCL/i9joXPuie0wFPQvs'} 2024-09-13T09:56:19.113 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi181.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 09:50:37.485641', 'locked_by': 'scheduled_akupczyk@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICl3WSJv2k7KfPBXd4cYNNdujDzqGuUNkUGSoDtD7DNQ'} 2024-09-13T09:56:19.136 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 09:50:37.486886', 'locked_by': 'scheduled_akupczyk@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMJk3Gz8/3TYsjsfimK1dx4WG87W75wZd8WEcfnGvDeV'} 2024-09-13T09:56:19.136 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T09:56:19.143 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'] 2024-09-13T09:56:19.143 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'] 2024-09-13T09:56:19.143 INFO:teuthology.task.internal:roles: ubuntu@smithi181.front.sepia.ceph.com - ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'] 2024-09-13T09:56:19.143 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0'] 2024-09-13T09:56:19.144 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T09:56:19.277 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9f40de3be0>, signals=[15]) 2024-09-13T09:56:19.277 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T09:56:19.287 INFO:teuthology.task.internal:Opening connections... 2024-09-13T09:56:19.287 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi073.front.sepia.ceph.com 2024-09-13T09:56:19.289 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:56:19.371 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-09-13T09:56:19.371 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:56:19.447 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi181.front.sepia.ceph.com 2024-09-13T09:56:19.448 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:56:19.524 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi195.front.sepia.ceph.com 2024-09-13T09:56:19.524 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:56:19.598 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T09:56:19.606 DEBUG:teuthology.orchestra.run.smithi073:> uname -m 2024-09-13T09:56:19.612 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2024-09-13T09:56:19.612 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2024-09-13T09:56:19.658 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-13T09:56:19.658 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="Ubuntu" 2024-09-13T09:56:19.658 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="22.04" 2024-09-13T09:56:19.658 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-13T09:56:19.658 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_CODENAME=jammy 2024-09-13T09:56:19.659 INFO:teuthology.orchestra.run.smithi073.stdout:ID=ubuntu 2024-09-13T09:56:19.659 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE=debian 2024-09-13T09:56:19.659 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T09:56:19.659 INFO:teuthology.orchestra.run.smithi073.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T09:56:19.659 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T09:56:19.659 INFO:teuthology.orchestra.run.smithi073.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T09:56:19.659 INFO:teuthology.orchestra.run.smithi073.stdout:UBUNTU_CODENAME=jammy 2024-09-13T09:56:19.660 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2024-09-13T09:56:19.680 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-09-13T09:56:19.686 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-09-13T09:56:19.686 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-09-13T09:56:19.733 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-13T09:56:19.733 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="Ubuntu" 2024-09-13T09:56:19.733 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="22.04" 2024-09-13T09:56:19.733 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-13T09:56:19.734 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_CODENAME=jammy 2024-09-13T09:56:19.734 INFO:teuthology.orchestra.run.smithi136.stdout:ID=ubuntu 2024-09-13T09:56:19.734 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE=debian 2024-09-13T09:56:19.734 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T09:56:19.734 INFO:teuthology.orchestra.run.smithi136.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T09:56:19.734 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T09:56:19.734 INFO:teuthology.orchestra.run.smithi136.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T09:56:19.734 INFO:teuthology.orchestra.run.smithi136.stdout:UBUNTU_CODENAME=jammy 2024-09-13T09:56:19.734 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-09-13T09:56:19.755 DEBUG:teuthology.orchestra.run.smithi181:> uname -m 2024-09-13T09:56:19.761 INFO:teuthology.orchestra.run.smithi181.stdout:x86_64 2024-09-13T09:56:19.761 DEBUG:teuthology.orchestra.run.smithi181:> cat /etc/os-release 2024-09-13T09:56:19.808 INFO:teuthology.orchestra.run.smithi181.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-13T09:56:19.808 INFO:teuthology.orchestra.run.smithi181.stdout:NAME="Ubuntu" 2024-09-13T09:56:19.808 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_ID="22.04" 2024-09-13T09:56:19.808 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-13T09:56:19.808 INFO:teuthology.orchestra.run.smithi181.stdout:VERSION_CODENAME=jammy 2024-09-13T09:56:19.809 INFO:teuthology.orchestra.run.smithi181.stdout:ID=ubuntu 2024-09-13T09:56:19.809 INFO:teuthology.orchestra.run.smithi181.stdout:ID_LIKE=debian 2024-09-13T09:56:19.809 INFO:teuthology.orchestra.run.smithi181.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T09:56:19.809 INFO:teuthology.orchestra.run.smithi181.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T09:56:19.809 INFO:teuthology.orchestra.run.smithi181.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T09:56:19.809 INFO:teuthology.orchestra.run.smithi181.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T09:56:19.809 INFO:teuthology.orchestra.run.smithi181.stdout:UBUNTU_CODENAME=jammy 2024-09-13T09:56:19.809 INFO:teuthology.lock.ops:Updating smithi181.front.sepia.ceph.com on lock server 2024-09-13T09:56:19.839 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2024-09-13T09:56:19.845 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2024-09-13T09:56:19.845 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2024-09-13T09:56:19.891 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-13T09:56:19.892 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="Ubuntu" 2024-09-13T09:56:19.892 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="22.04" 2024-09-13T09:56:19.892 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-13T09:56:19.892 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_CODENAME=jammy 2024-09-13T09:56:19.892 INFO:teuthology.orchestra.run.smithi195.stdout:ID=ubuntu 2024-09-13T09:56:19.892 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE=debian 2024-09-13T09:56:19.892 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T09:56:19.892 INFO:teuthology.orchestra.run.smithi195.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T09:56:19.892 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T09:56:19.892 INFO:teuthology.orchestra.run.smithi195.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T09:56:19.892 INFO:teuthology.orchestra.run.smithi195.stdout:UBUNTU_CODENAME=jammy 2024-09-13T09:56:19.893 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2024-09-13T09:56:19.914 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T09:56:19.925 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T09:56:19.929 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T09:56:19.930 DEBUG:teuthology.orchestra.run.smithi073:> test '!' -e /home/ubuntu/cephtest 2024-09-13T09:56:19.932 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-09-13T09:56:19.934 DEBUG:teuthology.orchestra.run.smithi181:> test '!' -e /home/ubuntu/cephtest 2024-09-13T09:56:19.937 DEBUG:teuthology.orchestra.run.smithi195:> test '!' -e /home/ubuntu/cephtest 2024-09-13T09:56:19.942 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T09:56:19.948 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T09:56:19.948 DEBUG:teuthology.orchestra.run.smithi073:> test -z $(ls -A /var/lib/ceph) 2024-09-13T09:56:19.979 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-09-13T09:56:19.982 DEBUG:teuthology.orchestra.run.smithi181:> test -z $(ls -A /var/lib/ceph) 2024-09-13T09:56:19.985 DEBUG:teuthology.orchestra.run.smithi195:> test -z $(ls -A /var/lib/ceph) 2024-09-13T09:56:20.264 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T09:56:20.359 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T09:56:20.373 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T09:56:20.373 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T09:56:20.374 DEBUG:teuthology.orchestra.run.smithi073:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T09:56:20.374 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T09:56:20.374 DEBUG:teuthology.orchestra.run.smithi181:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T09:56:20.375 DEBUG:teuthology.orchestra.run.smithi195:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T09:56:20.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:56:20.380 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2024-09-13T09:56:20.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:56:20.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:56:20.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T09:56:20.383 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-09-13T09:56:20.384 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2024-09-13T09:56:20.384 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-09-13T09:56:20.428 INFO:teuthology.orchestra.run.smithi181.stdout:5.15.0-56-generic 2024-09-13T09:56:20.428 INFO:teuthology.task.kernel:Running kernel on smithi181: 5.15.0-56-generic 2024-09-13T09:56:20.428 DEBUG:teuthology.orchestra.run.smithi181:> sudo apt-get clean 2024-09-13T09:56:20.430 INFO:teuthology.orchestra.run.smithi136.stdout:5.15.0-56-generic 2024-09-13T09:56:20.430 INFO:teuthology.task.kernel:Running kernel on smithi136: 5.15.0-56-generic 2024-09-13T09:56:20.430 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get clean 2024-09-13T09:56:20.431 INFO:teuthology.orchestra.run.smithi073.stdout:5.15.0-56-generic 2024-09-13T09:56:20.431 INFO:teuthology.task.kernel:Running kernel on smithi073: 5.15.0-56-generic 2024-09-13T09:56:20.431 DEBUG:teuthology.orchestra.run.smithi073:> sudo apt-get clean 2024-09-13T09:56:20.432 INFO:teuthology.orchestra.run.smithi195.stdout:5.15.0-56-generic 2024-09-13T09:56:20.432 INFO:teuthology.task.kernel:Running kernel on smithi195: 5.15.0-56-generic 2024-09-13T09:56:20.433 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get clean 2024-09-13T09:56:20.564 DEBUG:teuthology.orchestra.run.smithi181:> sudo apt-get update 2024-09-13T09:56:20.565 DEBUG:teuthology.orchestra.run.smithi073:> sudo apt-get update 2024-09-13T09:56:20.566 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2024-09-13T09:56:20.574 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get update 2024-09-13T09:56:20.707 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T09:56:20.708 INFO:teuthology.orchestra.run.smithi073.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-13T09:56:20.708 INFO:teuthology.orchestra.run.smithi073.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-13T09:56:20.718 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T09:56:20.718 INFO:teuthology.orchestra.run.smithi181.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-13T09:56:20.718 INFO:teuthology.orchestra.run.smithi181.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-13T09:56:20.719 INFO:teuthology.orchestra.run.smithi136.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-13T09:56:20.721 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-13T09:56:20.782 INFO:teuthology.orchestra.run.smithi073.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-13T09:56:20.786 INFO:teuthology.orchestra.run.smithi181.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-13T09:56:20.830 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T09:56:20.831 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T09:56:20.912 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-13T09:56:20.912 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-13T09:56:20.924 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-13T09:56:20.938 INFO:teuthology.orchestra.run.smithi073.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-13T09:56:20.941 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-13T09:56:20.977 INFO:teuthology.orchestra.run.smithi181.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-13T09:56:20.991 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-13T09:56:21.011 INFO:teuthology.orchestra.run.smithi073.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-13T09:56:21.011 INFO:teuthology.orchestra.run.smithi073.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-13T09:56:21.030 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-13T09:56:21.032 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-13T09:56:21.032 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-13T09:56:21.032 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-13T09:56:21.043 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-13T09:56:21.047 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-13T09:56:21.049 INFO:teuthology.orchestra.run.smithi073.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-13T09:56:21.050 INFO:teuthology.orchestra.run.smithi181.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-13T09:56:21.050 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-13T09:56:21.051 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-13T09:56:21.051 INFO:teuthology.orchestra.run.smithi073.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-13T09:56:21.052 INFO:teuthology.orchestra.run.smithi073.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-13T09:56:21.052 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-13T09:56:21.056 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-13T09:56:21.056 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-13T09:56:21.064 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-13T09:56:21.069 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-13T09:56:21.071 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-13T09:56:21.071 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-13T09:56:21.071 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-13T09:56:21.072 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-13T09:56:21.072 INFO:teuthology.orchestra.run.smithi136.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-13T09:56:21.072 INFO:teuthology.orchestra.run.smithi073.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-13T09:56:21.072 INFO:teuthology.orchestra.run.smithi073.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-13T09:56:21.075 INFO:teuthology.orchestra.run.smithi136.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-13T09:56:21.075 INFO:teuthology.orchestra.run.smithi136.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-13T09:56:21.076 INFO:teuthology.orchestra.run.smithi073.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-13T09:56:21.076 INFO:teuthology.orchestra.run.smithi073.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-13T09:56:21.078 INFO:teuthology.orchestra.run.smithi181.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-13T09:56:21.081 INFO:teuthology.orchestra.run.smithi136.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-13T09:56:21.082 INFO:teuthology.orchestra.run.smithi181.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-13T09:56:21.082 INFO:teuthology.orchestra.run.smithi073.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-13T09:56:21.084 INFO:teuthology.orchestra.run.smithi181.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-13T09:56:21.084 INFO:teuthology.orchestra.run.smithi181.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-13T09:56:21.087 INFO:teuthology.orchestra.run.smithi073.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-13T09:56:21.088 INFO:teuthology.orchestra.run.smithi073.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-13T09:56:21.088 INFO:teuthology.orchestra.run.smithi136.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-13T09:56:21.090 INFO:teuthology.orchestra.run.smithi136.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-13T09:56:21.090 INFO:teuthology.orchestra.run.smithi136.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-13T09:56:21.090 INFO:teuthology.orchestra.run.smithi136.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-13T09:56:21.093 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-13T09:56:21.093 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-13T09:56:21.102 INFO:teuthology.orchestra.run.smithi136.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-13T09:56:21.107 INFO:teuthology.orchestra.run.smithi181.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-13T09:56:21.107 INFO:teuthology.orchestra.run.smithi181.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-13T09:56:21.110 INFO:teuthology.orchestra.run.smithi181.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-13T09:56:21.111 INFO:teuthology.orchestra.run.smithi181.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-13T09:56:21.116 INFO:teuthology.orchestra.run.smithi181.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-13T09:56:21.119 INFO:teuthology.orchestra.run.smithi073.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-13T09:56:21.119 INFO:teuthology.orchestra.run.smithi073.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-13T09:56:21.120 INFO:teuthology.orchestra.run.smithi073.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-13T09:56:21.120 INFO:teuthology.orchestra.run.smithi073.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-13T09:56:21.120 INFO:teuthology.orchestra.run.smithi073.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-13T09:56:21.121 INFO:teuthology.orchestra.run.smithi181.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-13T09:56:21.122 INFO:teuthology.orchestra.run.smithi181.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-13T09:56:21.125 INFO:teuthology.orchestra.run.smithi136.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-13T09:56:21.125 INFO:teuthology.orchestra.run.smithi073.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-13T09:56:21.127 INFO:teuthology.orchestra.run.smithi073.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-13T09:56:21.127 INFO:teuthology.orchestra.run.smithi073.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-13T09:56:21.128 INFO:teuthology.orchestra.run.smithi073.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-13T09:56:21.144 INFO:teuthology.orchestra.run.smithi073.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-13T09:56:21.147 INFO:teuthology.orchestra.run.smithi073.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-13T09:56:21.147 INFO:teuthology.orchestra.run.smithi073.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-13T09:56:21.157 INFO:teuthology.orchestra.run.smithi073.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-13T09:56:21.161 INFO:teuthology.orchestra.run.smithi181.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-13T09:56:21.161 INFO:teuthology.orchestra.run.smithi181.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-13T09:56:21.161 INFO:teuthology.orchestra.run.smithi181.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-13T09:56:21.162 INFO:teuthology.orchestra.run.smithi181.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-13T09:56:21.162 INFO:teuthology.orchestra.run.smithi181.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-13T09:56:21.165 INFO:teuthology.orchestra.run.smithi181.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-13T09:56:21.167 INFO:teuthology.orchestra.run.smithi073.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-13T09:56:21.168 INFO:teuthology.orchestra.run.smithi181.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-13T09:56:21.168 INFO:teuthology.orchestra.run.smithi181.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-13T09:56:21.169 INFO:teuthology.orchestra.run.smithi181.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-13T09:56:21.170 INFO:teuthology.orchestra.run.smithi073.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-13T09:56:21.170 INFO:teuthology.orchestra.run.smithi073.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-13T09:56:21.170 INFO:teuthology.orchestra.run.smithi073.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-13T09:56:21.171 INFO:teuthology.orchestra.run.smithi073.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-13T09:56:21.178 INFO:teuthology.orchestra.run.smithi073.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-13T09:56:21.179 INFO:teuthology.orchestra.run.smithi073.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-13T09:56:21.179 INFO:teuthology.orchestra.run.smithi073.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-13T09:56:21.180 INFO:teuthology.orchestra.run.smithi073.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-13T09:56:21.187 INFO:teuthology.orchestra.run.smithi181.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-13T09:56:21.189 INFO:teuthology.orchestra.run.smithi181.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-13T09:56:21.190 INFO:teuthology.orchestra.run.smithi181.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-13T09:56:21.196 INFO:teuthology.orchestra.run.smithi073.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-13T09:56:21.196 INFO:teuthology.orchestra.run.smithi073.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-13T09:56:21.197 INFO:teuthology.orchestra.run.smithi073.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-13T09:56:21.197 INFO:teuthology.orchestra.run.smithi073.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-13T09:56:21.198 INFO:teuthology.orchestra.run.smithi181.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-13T09:56:21.198 INFO:teuthology.orchestra.run.smithi073.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-13T09:56:21.205 INFO:teuthology.orchestra.run.smithi181.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-13T09:56:21.206 INFO:teuthology.orchestra.run.smithi181.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-13T09:56:21.206 INFO:teuthology.orchestra.run.smithi181.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-13T09:56:21.207 INFO:teuthology.orchestra.run.smithi181.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-13T09:56:21.207 INFO:teuthology.orchestra.run.smithi181.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-13T09:56:21.222 INFO:teuthology.orchestra.run.smithi181.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-13T09:56:21.222 INFO:teuthology.orchestra.run.smithi181.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-13T09:56:21.222 INFO:teuthology.orchestra.run.smithi181.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-13T09:56:21.223 INFO:teuthology.orchestra.run.smithi181.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-13T09:56:21.235 INFO:teuthology.orchestra.run.smithi181.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-13T09:56:21.236 INFO:teuthology.orchestra.run.smithi181.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-13T09:56:21.236 INFO:teuthology.orchestra.run.smithi181.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-13T09:56:21.236 INFO:teuthology.orchestra.run.smithi181.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-13T09:56:21.253 INFO:teuthology.orchestra.run.smithi181.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-13T09:56:21.267 INFO:teuthology.orchestra.run.smithi136.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-13T09:56:21.269 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-13T09:56:21.405 INFO:teuthology.orchestra.run.smithi136.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-13T09:56:21.408 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-13T09:56:21.722 INFO:teuthology.orchestra.run.smithi136.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-13T09:56:21.726 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-13T09:56:21.758 INFO:teuthology.orchestra.run.smithi136.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-13T09:56:21.762 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-13T09:56:21.773 INFO:teuthology.orchestra.run.smithi136.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-13T09:56:21.773 INFO:teuthology.orchestra.run.smithi136.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-13T09:56:21.775 INFO:teuthology.orchestra.run.smithi136.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-13T09:56:21.776 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-13T09:56:21.777 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-13T09:56:21.843 INFO:teuthology.orchestra.run.smithi136.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-13T09:56:21.846 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-13T09:56:21.847 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-13T09:56:21.859 INFO:teuthology.orchestra.run.smithi136.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-13T09:56:21.860 INFO:teuthology.orchestra.run.smithi136.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-13T09:56:21.863 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-13T09:56:21.863 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-13T09:56:21.881 INFO:teuthology.orchestra.run.smithi136.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-13T09:56:21.893 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-13T09:56:21.910 INFO:teuthology.orchestra.run.smithi136.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-13T09:56:21.913 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-13T09:56:21.916 INFO:teuthology.orchestra.run.smithi136.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-13T09:56:21.917 INFO:teuthology.orchestra.run.smithi136.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-13T09:56:21.919 INFO:teuthology.orchestra.run.smithi136.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-13T09:56:21.919 INFO:teuthology.orchestra.run.smithi136.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-13T09:56:21.919 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-13T09:56:21.920 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-13T09:56:21.920 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-13T09:56:21.967 INFO:teuthology.orchestra.run.smithi136.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-13T09:56:21.967 INFO:teuthology.orchestra.run.smithi136.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-13T09:56:21.969 INFO:teuthology.orchestra.run.smithi136.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-13T09:56:21.970 INFO:teuthology.orchestra.run.smithi136.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-13T09:56:21.971 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-13T09:56:21.972 INFO:teuthology.orchestra.run.smithi195.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-13T09:56:21.972 INFO:teuthology.orchestra.run.smithi195.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-13T09:56:21.973 INFO:teuthology.orchestra.run.smithi195.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-13T09:56:22.006 INFO:teuthology.orchestra.run.smithi195.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-13T09:56:22.006 INFO:teuthology.orchestra.run.smithi195.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-13T09:56:22.006 INFO:teuthology.orchestra.run.smithi195.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-13T09:56:22.006 INFO:teuthology.orchestra.run.smithi195.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-13T09:56:22.006 INFO:teuthology.orchestra.run.smithi195.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-13T09:56:22.048 INFO:teuthology.orchestra.run.smithi136.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-13T09:56:22.049 INFO:teuthology.orchestra.run.smithi136.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-13T09:56:22.049 INFO:teuthology.orchestra.run.smithi136.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-13T09:56:22.050 INFO:teuthology.orchestra.run.smithi136.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-13T09:56:22.050 INFO:teuthology.orchestra.run.smithi136.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-13T09:56:22.053 INFO:teuthology.orchestra.run.smithi195.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-13T09:56:27.330 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 16.0 MB in 2s (7,554 kB/s) 2024-09-13T09:56:27.390 INFO:teuthology.orchestra.run.smithi181.stdout:Fetched 16.0 MB in 2s (7,364 kB/s) 2024-09-13T09:56:27.440 INFO:teuthology.orchestra.run.smithi073.stdout:Fetched 16.0 MB in 2s (7,661 kB/s) 2024-09-13T09:56:27.456 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 16.0 MB in 2s (7,425 kB/s) 2024-09-13T09:56:28.768 INFO:teuthology.orchestra.run.smithi181.stdout:Reading package lists... 2024-09-13T09:56:28.769 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-09-13T09:56:28.786 DEBUG:teuthology.orchestra.run.smithi181:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T09:56:28.787 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T09:56:28.840 INFO:teuthology.orchestra.run.smithi181.stdout:Reading package lists... 2024-09-13T09:56:28.866 INFO:teuthology.orchestra.run.smithi073.stdout:Reading package lists... 2024-09-13T09:56:28.883 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-09-13T09:56:28.885 DEBUG:teuthology.orchestra.run.smithi073:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T09:56:28.887 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-13T09:56:28.904 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T09:56:28.948 INFO:teuthology.orchestra.run.smithi073.stdout:Reading package lists... 2024-09-13T09:56:28.957 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-13T09:56:29.026 INFO:teuthology.orchestra.run.smithi181.stdout:Building dependency tree... 2024-09-13T09:56:29.026 INFO:teuthology.orchestra.run.smithi181.stdout:Reading state information... 2024-09-13T09:56:29.070 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-09-13T09:56:29.071 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-09-13T09:56:29.143 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2024-09-13T09:56:29.144 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2024-09-13T09:56:29.157 INFO:teuthology.orchestra.run.smithi073.stdout:Building dependency tree... 2024-09-13T09:56:29.157 INFO:teuthology.orchestra.run.smithi073.stdout:Reading state information... 2024-09-13T09:56:29.191 INFO:teuthology.orchestra.run.smithi181.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:56:29.191 INFO:teuthology.orchestra.run.smithi181.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T09:56:29.191 INFO:teuthology.orchestra.run.smithi181.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T09:56:29.192 INFO:teuthology.orchestra.run.smithi181.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T09:56:29.192 INFO:teuthology.orchestra.run.smithi181.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-13T09:56:29.192 INFO:teuthology.orchestra.run.smithi181.stdout: python2.7-minimal 2024-09-13T09:56:29.192 INFO:teuthology.orchestra.run.smithi181.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:56:29.193 INFO:teuthology.orchestra.run.smithi181.stdout:The following additional packages will be installed: 2024-09-13T09:56:29.193 INFO:teuthology.orchestra.run.smithi181.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T09:56:29.193 INFO:teuthology.orchestra.run.smithi181.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-13T09:56:29.193 INFO:teuthology.orchestra.run.smithi181.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-13T09:56:29.195 INFO:teuthology.orchestra.run.smithi181.stdout:Suggested packages: 2024-09-13T09:56:29.195 INFO:teuthology.orchestra.run.smithi181.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-13T09:56:29.195 INFO:teuthology.orchestra.run.smithi181.stdout:Recommended packages: 2024-09-13T09:56:29.195 INFO:teuthology.orchestra.run.smithi181.stdout: thermald 2024-09-13T09:56:29.237 INFO:teuthology.orchestra.run.smithi181.stdout:The following NEW packages will be installed: 2024-09-13T09:56:29.237 INFO:teuthology.orchestra.run.smithi181.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T09:56:29.237 INFO:teuthology.orchestra.run.smithi181.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-13T09:56:29.238 INFO:teuthology.orchestra.run.smithi181.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-13T09:56:29.239 INFO:teuthology.orchestra.run.smithi181.stdout:The following packages will be upgraded: 2024-09-13T09:56:29.239 INFO:teuthology.orchestra.run.smithi181.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T09:56:29.241 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:56:29.241 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T09:56:29.241 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T09:56:29.241 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T09:56:29.241 INFO:teuthology.orchestra.run.smithi195.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-13T09:56:29.242 INFO:teuthology.orchestra.run.smithi195.stdout: python2.7-minimal 2024-09-13T09:56:29.242 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:56:29.243 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2024-09-13T09:56:29.243 INFO:teuthology.orchestra.run.smithi195.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T09:56:29.243 INFO:teuthology.orchestra.run.smithi195.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-13T09:56:29.244 INFO:teuthology.orchestra.run.smithi195.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-13T09:56:29.245 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2024-09-13T09:56:29.245 INFO:teuthology.orchestra.run.smithi195.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-13T09:56:29.245 INFO:teuthology.orchestra.run.smithi195.stdout:Recommended packages: 2024-09-13T09:56:29.245 INFO:teuthology.orchestra.run.smithi195.stdout: thermald 2024-09-13T09:56:29.301 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2024-09-13T09:56:29.301 INFO:teuthology.orchestra.run.smithi195.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T09:56:29.301 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-13T09:56:29.302 INFO:teuthology.orchestra.run.smithi195.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-13T09:56:29.303 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages will be upgraded: 2024-09-13T09:56:29.304 INFO:teuthology.orchestra.run.smithi195.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T09:56:29.307 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:56:29.307 INFO:teuthology.orchestra.run.smithi136.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T09:56:29.308 INFO:teuthology.orchestra.run.smithi136.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T09:56:29.308 INFO:teuthology.orchestra.run.smithi136.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T09:56:29.308 INFO:teuthology.orchestra.run.smithi136.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-13T09:56:29.308 INFO:teuthology.orchestra.run.smithi136.stdout: python2.7-minimal 2024-09-13T09:56:29.308 INFO:teuthology.orchestra.run.smithi136.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:56:29.310 INFO:teuthology.orchestra.run.smithi136.stdout:The following additional packages will be installed: 2024-09-13T09:56:29.310 INFO:teuthology.orchestra.run.smithi136.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T09:56:29.310 INFO:teuthology.orchestra.run.smithi136.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-13T09:56:29.310 INFO:teuthology.orchestra.run.smithi136.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-13T09:56:29.311 INFO:teuthology.orchestra.run.smithi136.stdout:Suggested packages: 2024-09-13T09:56:29.311 INFO:teuthology.orchestra.run.smithi136.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-13T09:56:29.312 INFO:teuthology.orchestra.run.smithi136.stdout:Recommended packages: 2024-09-13T09:56:29.312 INFO:teuthology.orchestra.run.smithi136.stdout: thermald 2024-09-13T09:56:29.330 INFO:teuthology.orchestra.run.smithi073.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:56:29.330 INFO:teuthology.orchestra.run.smithi073.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T09:56:29.330 INFO:teuthology.orchestra.run.smithi073.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T09:56:29.330 INFO:teuthology.orchestra.run.smithi073.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T09:56:29.330 INFO:teuthology.orchestra.run.smithi073.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-13T09:56:29.331 INFO:teuthology.orchestra.run.smithi073.stdout: python2.7-minimal 2024-09-13T09:56:29.331 INFO:teuthology.orchestra.run.smithi073.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:56:29.331 INFO:teuthology.orchestra.run.smithi073.stdout:The following additional packages will be installed: 2024-09-13T09:56:29.332 INFO:teuthology.orchestra.run.smithi073.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T09:56:29.332 INFO:teuthology.orchestra.run.smithi073.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-13T09:56:29.332 INFO:teuthology.orchestra.run.smithi073.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-13T09:56:29.333 INFO:teuthology.orchestra.run.smithi073.stdout:Suggested packages: 2024-09-13T09:56:29.333 INFO:teuthology.orchestra.run.smithi073.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-13T09:56:29.333 INFO:teuthology.orchestra.run.smithi073.stdout:Recommended packages: 2024-09-13T09:56:29.333 INFO:teuthology.orchestra.run.smithi073.stdout: thermald 2024-09-13T09:56:29.363 INFO:teuthology.orchestra.run.smithi136.stdout:The following NEW packages will be installed: 2024-09-13T09:56:29.363 INFO:teuthology.orchestra.run.smithi136.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T09:56:29.363 INFO:teuthology.orchestra.run.smithi136.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-13T09:56:29.364 INFO:teuthology.orchestra.run.smithi136.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-13T09:56:29.365 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages will be upgraded: 2024-09-13T09:56:29.365 INFO:teuthology.orchestra.run.smithi136.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T09:56:29.385 INFO:teuthology.orchestra.run.smithi073.stdout:The following NEW packages will be installed: 2024-09-13T09:56:29.385 INFO:teuthology.orchestra.run.smithi073.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T09:56:29.386 INFO:teuthology.orchestra.run.smithi073.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-13T09:56:29.389 INFO:teuthology.orchestra.run.smithi073.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-13T09:56:29.389 INFO:teuthology.orchestra.run.smithi073.stdout:The following packages will be upgraded: 2024-09-13T09:56:29.389 INFO:teuthology.orchestra.run.smithi073.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T09:56:29.416 INFO:teuthology.orchestra.run.smithi136.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T09:56:29.417 INFO:teuthology.orchestra.run.smithi136.stdout:Need to get 113 MB of archives. 2024-09-13T09:56:29.417 INFO:teuthology.orchestra.run.smithi136.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-13T09:56:29.417 INFO:teuthology.orchestra.run.smithi136.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-13T09:56:29.419 INFO:teuthology.orchestra.run.smithi181.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T09:56:29.419 INFO:teuthology.orchestra.run.smithi181.stdout:Need to get 113 MB of archives. 2024-09-13T09:56:29.419 INFO:teuthology.orchestra.run.smithi181.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-13T09:56:29.419 INFO:teuthology.orchestra.run.smithi181.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-13T09:56:29.438 INFO:teuthology.orchestra.run.smithi073.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T09:56:29.438 INFO:teuthology.orchestra.run.smithi073.stdout:Need to get 113 MB of archives. 2024-09-13T09:56:29.438 INFO:teuthology.orchestra.run.smithi073.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-13T09:56:29.438 INFO:teuthology.orchestra.run.smithi073.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-13T09:56:29.483 INFO:teuthology.orchestra.run.smithi195.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T09:56:29.483 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 113 MB of archives. 2024-09-13T09:56:29.484 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-13T09:56:29.484 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-13T09:56:29.834 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-13T09:56:29.863 INFO:teuthology.orchestra.run.smithi073.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-13T09:56:29.924 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-13T09:56:29.950 INFO:teuthology.orchestra.run.smithi073.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-13T09:56:30.731 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-13T09:56:30.732 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-13T09:56:30.732 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-13T09:56:30.742 INFO:teuthology.orchestra.run.smithi073.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-13T09:56:30.742 INFO:teuthology.orchestra.run.smithi073.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-13T09:56:30.742 INFO:teuthology.orchestra.run.smithi073.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-13T09:56:30.849 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-13T09:56:30.860 INFO:teuthology.orchestra.run.smithi073.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-13T09:56:30.869 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-13T09:56:30.881 INFO:teuthology.orchestra.run.smithi073.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-13T09:56:30.988 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-13T09:56:31.056 INFO:teuthology.orchestra.run.smithi181.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-13T09:56:31.258 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 113 MB in 1s (75.7 MB/s) 2024-09-13T09:56:31.291 INFO:teuthology.orchestra.run.smithi073.stdout:Fetched 113 MB in 1s (76.2 MB/s) 2024-09-13T09:56:31.336 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-13T09:56:31.424 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-13T09:56:31.453 INFO:teuthology.orchestra.run.smithi181.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-13T09:56:31.467 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-13T09:56:33.434 INFO:teuthology.orchestra.run.smithi136.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-13T09:56:33.438 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:56:33.489 INFO:teuthology.orchestra.run.smithi073.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-13T09:56:33.493 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:56:33.555 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:33.556 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:33.558 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-13T09:56:33.559 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-13T09:56:33.559 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-13T09:56:33.626 INFO:teuthology.orchestra.run.smithi181.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-13T09:56:33.626 INFO:teuthology.orchestra.run.smithi181.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-13T09:56:33.626 INFO:teuthology.orchestra.run.smithi181.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-13T09:56:33.897 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-13T09:56:33.976 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-13T09:56:33.980 INFO:teuthology.orchestra.run.smithi181.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-13T09:56:34.062 INFO:teuthology.orchestra.run.smithi181.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-13T09:56:34.380 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 113 MB in 5s (24.3 MB/s) 2024-09-13T09:56:34.476 INFO:teuthology.orchestra.run.smithi181.stdout:Fetched 113 MB in 5s (23.5 MB/s) 2024-09-13T09:56:34.562 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-13T09:56:34.641 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-13T09:56:35.700 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-13T09:56:35.721 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:56:35.817 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:36.274 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-13T09:56:36.287 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-13T09:56:36.295 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:56:36.307 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:56:36.345 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:36.376 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:36.570 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-13T09:56:36.574 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:56:36.617 INFO:teuthology.orchestra.run.smithi181.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-13T09:56:36.621 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:56:36.659 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:36.721 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:36.878 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-13T09:56:36.898 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:56:36.936 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:38.714 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-13T09:56:38.735 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:56:38.832 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:39.077 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-13T09:56:39.098 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:56:39.176 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:39.283 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-13T09:56:39.305 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:56:39.342 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:39.672 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-13T09:56:39.692 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:56:39.730 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:43.149 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:56:43.283 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:56:43.616 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:56:43.729 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:56:43.915 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:56:44.007 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-13T09:56:44.020 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-13T09:56:44.041 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:56:44.065 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T09:56:44.408 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:56:44.537 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:56:44.857 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-13T09:56:44.879 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-13T09:56:44.923 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T09:56:45.709 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:56:45.827 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:56:46.143 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:56:46.173 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:56:46.264 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:56:46.299 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:56:46.567 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-13T09:56:46.589 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-13T09:56:46.633 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T09:56:46.698 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:56:46.827 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:56:47.105 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-13T09:56:47.127 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-13T09:56:47.172 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T09:56:50.034 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-13T09:56:50.070 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:56:50.100 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:50.722 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-13T09:56:50.759 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:56:50.797 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:52.395 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-13T09:56:52.431 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:56:52.469 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:52.657 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:56:52.779 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:56:52.883 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-13T09:56:52.920 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T09:56:52.958 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:53.188 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T09:56:53.293 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:53.394 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-13T09:56:53.505 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:53.522 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:56:53.653 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:56:54.020 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T09:56:54.158 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:54.284 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-13T09:56:54.387 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:54.883 INFO:teuthology.orchestra.run.smithi073.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:56:54.883 INFO:teuthology.orchestra.run.smithi073.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-13T09:56:54.883 INFO:teuthology.orchestra.run.smithi073.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:56:54.883 INFO:teuthology.orchestra.run.smithi073.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-13T09:56:55.058 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:55.067 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:56:55.189 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:56:55.531 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T09:56:55.569 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T09:56:55.644 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:55.694 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T09:56:55.706 INFO:teuthology.orchestra.run.smithi136.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:56:55.706 INFO:teuthology.orchestra.run.smithi136.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-13T09:56:55.706 INFO:teuthology.orchestra.run.smithi136.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:56:55.706 INFO:teuthology.orchestra.run.smithi136.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-13T09:56:55.762 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-13T09:56:55.892 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:56.044 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T09:56:57.100 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:57.108 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:57.118 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:57.234 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-13T09:56:57.370 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:57.746 INFO:teuthology.orchestra.run.smithi181.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:56:57.746 INFO:teuthology.orchestra.run.smithi181.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-13T09:56:57.746 INFO:teuthology.orchestra.run.smithi181.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:56:57.746 INFO:teuthology.orchestra.run.smithi181.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-13T09:56:57.902 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:58.502 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-13T09:56:58.637 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-13T09:56:58.693 INFO:teuthology.orchestra.run.smithi195.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:56:58.694 INFO:teuthology.orchestra.run.smithi195.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-13T09:56:58.694 INFO:teuthology.orchestra.run.smithi195.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:56:58.694 INFO:teuthology.orchestra.run.smithi195.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-13T09:56:58.725 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:58.763 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:58.827 INFO:teuthology.orchestra.run.smithi073.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T09:56:58.827 INFO:teuthology.orchestra.run.smithi073.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:56:58.848 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:56:59.311 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:57:00.112 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-13T09:57:00.238 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-13T09:57:00.299 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:57:00.364 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:57:00.427 INFO:teuthology.orchestra.run.smithi136.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T09:57:00.428 INFO:teuthology.orchestra.run.smithi136.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:57:00.712 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-13T09:57:00.838 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-13T09:57:00.964 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:57:01.029 INFO:teuthology.orchestra.run.smithi181.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T09:57:01.029 INFO:teuthology.orchestra.run.smithi181.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:57:01.675 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-13T09:57:01.801 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-13T09:57:01.928 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T09:57:02.004 INFO:teuthology.orchestra.run.smithi195.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T09:57:02.004 INFO:teuthology.orchestra.run.smithi195.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:57:13.218 INFO:teuthology.orchestra.run.smithi073.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T09:57:13.218 INFO:teuthology.orchestra.run.smithi073.stdout:Sourcing file `/etc/default/grub' 2024-09-13T09:57:13.232 INFO:teuthology.orchestra.run.smithi073.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T09:57:13.256 INFO:teuthology.orchestra.run.smithi073.stdout:Generating grub configuration file ... 2024-09-13T09:57:13.622 INFO:teuthology.orchestra.run.smithi073.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:57:13.643 INFO:teuthology.orchestra.run.smithi073.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:57:14.031 INFO:teuthology.orchestra.run.smithi073.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:57:14.037 INFO:teuthology.orchestra.run.smithi073.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T09:57:14.119 INFO:teuthology.orchestra.run.smithi073.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T09:57:14.125 INFO:teuthology.orchestra.run.smithi073.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T09:57:14.308 INFO:teuthology.orchestra.run.smithi073.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T09:57:14.308 INFO:teuthology.orchestra.run.smithi073.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T09:57:14.308 INFO:teuthology.orchestra.run.smithi073.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T09:57:14.352 INFO:teuthology.orchestra.run.smithi073.stdout:done 2024-09-13T09:57:14.430 INFO:teuthology.orchestra.run.smithi136.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T09:57:14.431 INFO:teuthology.orchestra.run.smithi136.stdout:Sourcing file `/etc/default/grub' 2024-09-13T09:57:14.448 INFO:teuthology.orchestra.run.smithi136.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T09:57:14.470 INFO:teuthology.orchestra.run.smithi136.stdout:Generating grub configuration file ... 2024-09-13T09:57:14.824 INFO:teuthology.orchestra.run.smithi136.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:57:14.844 DEBUG:teuthology.orchestra.run.smithi073:> dpkg -s linux-image-generic 2024-09-13T09:57:14.845 INFO:teuthology.orchestra.run.smithi136.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:57:14.867 INFO:teuthology.orchestra.run.smithi181.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T09:57:14.868 INFO:teuthology.orchestra.run.smithi181.stdout:Sourcing file `/etc/default/grub' 2024-09-13T09:57:14.870 INFO:teuthology.orchestra.run.smithi073.stdout:Package: linux-image-generic 2024-09-13T09:57:14.870 INFO:teuthology.orchestra.run.smithi073.stdout:Status: install ok installed 2024-09-13T09:57:14.870 INFO:teuthology.orchestra.run.smithi073.stdout:Priority: optional 2024-09-13T09:57:14.870 INFO:teuthology.orchestra.run.smithi073.stdout:Section: kernel 2024-09-13T09:57:14.870 INFO:teuthology.orchestra.run.smithi073.stdout:Installed-Size: 21 2024-09-13T09:57:14.870 INFO:teuthology.orchestra.run.smithi073.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T09:57:14.870 INFO:teuthology.orchestra.run.smithi073.stdout:Architecture: amd64 2024-09-13T09:57:14.870 INFO:teuthology.orchestra.run.smithi073.stdout:Source: linux-meta 2024-09-13T09:57:14.870 INFO:teuthology.orchestra.run.smithi073.stdout:Version: 5.15.0.121.121 2024-09-13T09:57:14.870 INFO:teuthology.orchestra.run.smithi073.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T09:57:14.871 INFO:teuthology.orchestra.run.smithi073.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T09:57:14.871 INFO:teuthology.orchestra.run.smithi073.stdout:Recommends: thermald 2024-09-13T09:57:14.871 INFO:teuthology.orchestra.run.smithi073.stdout:Description: Generic Linux kernel image 2024-09-13T09:57:14.871 INFO:teuthology.orchestra.run.smithi073.stdout: This package will always depend on the latest generic kernel image 2024-09-13T09:57:14.871 INFO:teuthology.orchestra.run.smithi073.stdout: available. 2024-09-13T09:57:14.871 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-13T09:57:14.871 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T09:57:14.871 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T09:57:14.872 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T09:57:14.872 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi073.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T09:57:14.872 DEBUG:teuthology.orchestra.run.smithi073:> sudo apt-get clean 2024-09-13T09:57:14.885 INFO:teuthology.orchestra.run.smithi181.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T09:57:14.905 INFO:teuthology.orchestra.run.smithi181.stdout:Generating grub configuration file ... 2024-09-13T09:57:15.060 DEBUG:teuthology.orchestra.run.smithi073:> sudo apt-get update 2024-09-13T09:57:15.132 INFO:teuthology.orchestra.run.smithi136.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:57:15.138 INFO:teuthology.orchestra.run.smithi136.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T09:57:15.145 INFO:teuthology.orchestra.run.smithi181.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:57:15.165 INFO:teuthology.orchestra.run.smithi181.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:57:15.179 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T09:57:15.189 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T09:57:15.190 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T09:57:15.202 INFO:teuthology.orchestra.run.smithi136.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T09:57:15.206 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T09:57:15.207 INFO:teuthology.orchestra.run.smithi136.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T09:57:15.376 INFO:teuthology.orchestra.run.smithi136.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T09:57:15.377 INFO:teuthology.orchestra.run.smithi136.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T09:57:15.377 INFO:teuthology.orchestra.run.smithi136.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T09:57:15.427 INFO:teuthology.orchestra.run.smithi136.stdout:done 2024-09-13T09:57:15.445 INFO:teuthology.orchestra.run.smithi181.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:57:15.450 INFO:teuthology.orchestra.run.smithi181.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T09:57:15.511 INFO:teuthology.orchestra.run.smithi181.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T09:57:15.517 INFO:teuthology.orchestra.run.smithi181.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T09:57:15.659 INFO:teuthology.orchestra.run.smithi181.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T09:57:15.659 INFO:teuthology.orchestra.run.smithi181.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T09:57:15.659 INFO:teuthology.orchestra.run.smithi181.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T09:57:15.700 INFO:teuthology.orchestra.run.smithi181.stdout:done 2024-09-13T09:57:15.916 DEBUG:teuthology.orchestra.run.smithi136:> dpkg -s linux-image-generic 2024-09-13T09:57:15.937 INFO:teuthology.orchestra.run.smithi136.stdout:Package: linux-image-generic 2024-09-13T09:57:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:Status: install ok installed 2024-09-13T09:57:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:Priority: optional 2024-09-13T09:57:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:Section: kernel 2024-09-13T09:57:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:Installed-Size: 21 2024-09-13T09:57:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T09:57:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: amd64 2024-09-13T09:57:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:Source: linux-meta 2024-09-13T09:57:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:Version: 5.15.0.121.121 2024-09-13T09:57:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T09:57:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T09:57:15.938 INFO:teuthology.orchestra.run.smithi136.stdout:Recommends: thermald 2024-09-13T09:57:15.939 INFO:teuthology.orchestra.run.smithi136.stdout:Description: Generic Linux kernel image 2024-09-13T09:57:15.939 INFO:teuthology.orchestra.run.smithi136.stdout: This package will always depend on the latest generic kernel image 2024-09-13T09:57:15.939 INFO:teuthology.orchestra.run.smithi136.stdout: available. 2024-09-13T09:57:15.939 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-13T09:57:15.939 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T09:57:15.939 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T09:57:15.939 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T09:57:15.940 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi136.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T09:57:15.940 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get clean 2024-09-13T09:57:16.033 INFO:teuthology.orchestra.run.smithi195.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T09:57:16.033 INFO:teuthology.orchestra.run.smithi195.stdout:Sourcing file `/etc/default/grub' 2024-09-13T09:57:16.041 INFO:teuthology.orchestra.run.smithi195.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T09:57:16.057 INFO:teuthology.orchestra.run.smithi195.stdout:Generating grub configuration file ... 2024-09-13T09:57:16.102 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get update 2024-09-13T09:57:16.207 DEBUG:teuthology.orchestra.run.smithi181:> dpkg -s linux-image-generic 2024-09-13T09:57:16.227 INFO:teuthology.orchestra.run.smithi181.stdout:Package: linux-image-generic 2024-09-13T09:57:16.227 INFO:teuthology.orchestra.run.smithi181.stdout:Status: install ok installed 2024-09-13T09:57:16.228 INFO:teuthology.orchestra.run.smithi181.stdout:Priority: optional 2024-09-13T09:57:16.228 INFO:teuthology.orchestra.run.smithi181.stdout:Section: kernel 2024-09-13T09:57:16.228 INFO:teuthology.orchestra.run.smithi181.stdout:Installed-Size: 21 2024-09-13T09:57:16.228 INFO:teuthology.orchestra.run.smithi181.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T09:57:16.228 INFO:teuthology.orchestra.run.smithi181.stdout:Architecture: amd64 2024-09-13T09:57:16.228 INFO:teuthology.orchestra.run.smithi181.stdout:Source: linux-meta 2024-09-13T09:57:16.228 INFO:teuthology.orchestra.run.smithi181.stdout:Version: 5.15.0.121.121 2024-09-13T09:57:16.228 INFO:teuthology.orchestra.run.smithi181.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T09:57:16.228 INFO:teuthology.orchestra.run.smithi181.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T09:57:16.228 INFO:teuthology.orchestra.run.smithi181.stdout:Recommends: thermald 2024-09-13T09:57:16.228 INFO:teuthology.orchestra.run.smithi181.stdout:Description: Generic Linux kernel image 2024-09-13T09:57:16.229 INFO:teuthology.orchestra.run.smithi181.stdout: This package will always depend on the latest generic kernel image 2024-09-13T09:57:16.229 INFO:teuthology.orchestra.run.smithi181.stdout: available. 2024-09-13T09:57:16.229 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-13T09:57:16.229 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T09:57:16.229 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T09:57:16.229 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-09-13T09:57:16.230 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi181.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T09:57:16.230 DEBUG:teuthology.orchestra.run.smithi181:> sudo apt-get clean 2024-09-13T09:57:16.230 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T09:57:16.344 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:57:16.350 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T09:57:16.358 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:57:16.385 DEBUG:teuthology.orchestra.run.smithi181:> sudo apt-get update 2024-09-13T09:57:16.432 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T09:57:16.505 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T09:57:16.512 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T09:57:16.514 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T09:57:16.531 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T09:57:16.576 INFO:teuthology.orchestra.run.smithi073.stdout:Reading package lists... 2024-09-13T09:57:16.595 DEBUG:teuthology.orchestra.run.smithi073:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T09:57:16.624 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T09:57:16.651 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:57:16.655 INFO:teuthology.orchestra.run.smithi073.stdout:Reading package lists... 2024-09-13T09:57:16.659 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T09:57:16.722 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T09:57:16.728 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T09:57:16.865 INFO:teuthology.orchestra.run.smithi073.stdout:Building dependency tree... 2024-09-13T09:57:16.866 INFO:teuthology.orchestra.run.smithi073.stdout:Reading state information... 2024-09-13T09:57:16.884 INFO:teuthology.orchestra.run.smithi195.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T09:57:16.884 INFO:teuthology.orchestra.run.smithi195.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T09:57:16.884 INFO:teuthology.orchestra.run.smithi195.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T09:57:16.922 INFO:teuthology.orchestra.run.smithi195.stdout:done 2024-09-13T09:57:17.045 INFO:teuthology.orchestra.run.smithi073.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-13T09:57:17.046 INFO:teuthology.orchestra.run.smithi073.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:57:17.046 INFO:teuthology.orchestra.run.smithi073.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T09:57:17.046 INFO:teuthology.orchestra.run.smithi073.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T09:57:17.046 INFO:teuthology.orchestra.run.smithi073.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T09:57:17.046 INFO:teuthology.orchestra.run.smithi073.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T09:57:17.046 INFO:teuthology.orchestra.run.smithi073.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T09:57:17.046 INFO:teuthology.orchestra.run.smithi073.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:57:17.083 INFO:teuthology.orchestra.run.smithi073.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T09:57:17.085 DEBUG:teuthology.orchestra.run.smithi073:> dpkg -s linux-image-generic 2024-09-13T09:57:17.101 INFO:teuthology.orchestra.run.smithi073.stdout:Package: linux-image-generic 2024-09-13T09:57:17.101 INFO:teuthology.orchestra.run.smithi073.stdout:Status: install ok installed 2024-09-13T09:57:17.101 INFO:teuthology.orchestra.run.smithi073.stdout:Priority: optional 2024-09-13T09:57:17.102 INFO:teuthology.orchestra.run.smithi073.stdout:Section: kernel 2024-09-13T09:57:17.102 INFO:teuthology.orchestra.run.smithi073.stdout:Installed-Size: 21 2024-09-13T09:57:17.102 INFO:teuthology.orchestra.run.smithi073.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T09:57:17.102 INFO:teuthology.orchestra.run.smithi073.stdout:Architecture: amd64 2024-09-13T09:57:17.102 INFO:teuthology.orchestra.run.smithi073.stdout:Source: linux-meta 2024-09-13T09:57:17.102 INFO:teuthology.orchestra.run.smithi073.stdout:Version: 5.15.0.121.121 2024-09-13T09:57:17.102 INFO:teuthology.orchestra.run.smithi073.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T09:57:17.102 INFO:teuthology.orchestra.run.smithi073.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T09:57:17.102 INFO:teuthology.orchestra.run.smithi073.stdout:Recommends: thermald 2024-09-13T09:57:17.102 INFO:teuthology.orchestra.run.smithi073.stdout:Description: Generic Linux kernel image 2024-09-13T09:57:17.102 INFO:teuthology.orchestra.run.smithi073.stdout: This package will always depend on the latest generic kernel image 2024-09-13T09:57:17.103 INFO:teuthology.orchestra.run.smithi073.stdout: available. 2024-09-13T09:57:17.103 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-09-13T09:57:17.149 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.NFrTH8Liaa 2024-09-13T09:57:17.150 DEBUG:teuthology.orchestra.run.smithi073:> sudo cp /boot/grub/grub.cfg /tmp/tmp.NFrTH8Liaa 2024-09-13T09:57:17.205 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.NFrTH8Liaa 2024-09-13T09:57:17.324 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.NFrTH8Liaa is 10KB 2024-09-13T09:57:17.384 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.NFrTH8Liaa 2024-09-13T09:57:17.390 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T09:57:17.394 DEBUG:teuthology.orchestra.run.smithi195:> dpkg -s linux-image-generic 2024-09-13T09:57:17.415 INFO:teuthology.orchestra.run.smithi195.stdout:Package: linux-image-generic 2024-09-13T09:57:17.415 INFO:teuthology.orchestra.run.smithi195.stdout:Status: install ok installed 2024-09-13T09:57:17.415 INFO:teuthology.orchestra.run.smithi195.stdout:Priority: optional 2024-09-13T09:57:17.415 INFO:teuthology.orchestra.run.smithi195.stdout:Section: kernel 2024-09-13T09:57:17.415 INFO:teuthology.orchestra.run.smithi195.stdout:Installed-Size: 21 2024-09-13T09:57:17.415 INFO:teuthology.orchestra.run.smithi195.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T09:57:17.416 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: amd64 2024-09-13T09:57:17.416 INFO:teuthology.orchestra.run.smithi195.stdout:Source: linux-meta 2024-09-13T09:57:17.416 INFO:teuthology.orchestra.run.smithi195.stdout:Version: 5.15.0.121.121 2024-09-13T09:57:17.416 INFO:teuthology.orchestra.run.smithi195.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T09:57:17.416 INFO:teuthology.orchestra.run.smithi195.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T09:57:17.416 INFO:teuthology.orchestra.run.smithi195.stdout:Recommends: thermald 2024-09-13T09:57:17.416 INFO:teuthology.orchestra.run.smithi195.stdout:Description: Generic Linux kernel image 2024-09-13T09:57:17.416 INFO:teuthology.orchestra.run.smithi195.stdout: This package will always depend on the latest generic kernel image 2024-09-13T09:57:17.416 INFO:teuthology.orchestra.run.smithi195.stdout: available. 2024-09-13T09:57:17.416 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-13T09:57:17.417 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T09:57:17.417 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T09:57:17.417 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-09-13T09:57:17.417 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi195.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T09:57:17.417 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get clean 2024-09-13T09:57:17.447 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T09:57:17.448 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T09:57:17.449 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T09:57:17.547 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-13T09:57:17.547 DEBUG:teuthology.orchestra.run.smithi073:> sudo update-grub 2024-09-13T09:57:17.630 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2024-09-13T09:57:17.714 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-13T09:57:17.733 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T09:57:17.758 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T09:57:17.797 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-13T09:57:17.881 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T09:57:17.944 INFO:teuthology.orchestra.run.smithi181.stdout:Reading package lists... 2024-09-13T09:57:17.965 DEBUG:teuthology.orchestra.run.smithi181:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T09:57:17.972 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T09:57:18.009 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2024-09-13T09:57:18.009 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2024-09-13T09:57:18.025 INFO:teuthology.orchestra.run.smithi181.stdout:Reading package lists... 2024-09-13T09:57:18.045 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T09:57:18.202 INFO:teuthology.orchestra.run.smithi136.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-13T09:57:18.202 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:57:18.202 INFO:teuthology.orchestra.run.smithi136.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T09:57:18.202 INFO:teuthology.orchestra.run.smithi136.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T09:57:18.202 INFO:teuthology.orchestra.run.smithi136.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T09:57:18.203 INFO:teuthology.orchestra.run.smithi136.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T09:57:18.203 INFO:teuthology.orchestra.run.smithi136.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T09:57:18.203 INFO:teuthology.orchestra.run.smithi136.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:57:18.216 INFO:teuthology.orchestra.run.smithi181.stdout:Building dependency tree... 2024-09-13T09:57:18.216 INFO:teuthology.orchestra.run.smithi181.stdout:Reading state information... 2024-09-13T09:57:18.250 INFO:teuthology.orchestra.run.smithi136.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T09:57:18.250 DEBUG:teuthology.orchestra.run.smithi136:> dpkg -s linux-image-generic 2024-09-13T09:57:18.262 INFO:teuthology.orchestra.run.smithi136.stdout:Package: linux-image-generic 2024-09-13T09:57:18.263 INFO:teuthology.orchestra.run.smithi136.stdout:Status: install ok installed 2024-09-13T09:57:18.263 INFO:teuthology.orchestra.run.smithi136.stdout:Priority: optional 2024-09-13T09:57:18.263 INFO:teuthology.orchestra.run.smithi136.stdout:Section: kernel 2024-09-13T09:57:18.263 INFO:teuthology.orchestra.run.smithi136.stdout:Installed-Size: 21 2024-09-13T09:57:18.263 INFO:teuthology.orchestra.run.smithi136.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T09:57:18.263 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: amd64 2024-09-13T09:57:18.263 INFO:teuthology.orchestra.run.smithi136.stdout:Source: linux-meta 2024-09-13T09:57:18.263 INFO:teuthology.orchestra.run.smithi136.stdout:Version: 5.15.0.121.121 2024-09-13T09:57:18.264 INFO:teuthology.orchestra.run.smithi136.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T09:57:18.264 INFO:teuthology.orchestra.run.smithi136.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T09:57:18.264 INFO:teuthology.orchestra.run.smithi136.stdout:Recommends: thermald 2024-09-13T09:57:18.264 INFO:teuthology.orchestra.run.smithi136.stdout:Description: Generic Linux kernel image 2024-09-13T09:57:18.264 INFO:teuthology.orchestra.run.smithi136.stdout: This package will always depend on the latest generic kernel image 2024-09-13T09:57:18.264 INFO:teuthology.orchestra.run.smithi136.stdout: available. 2024-09-13T09:57:18.264 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-09-13T09:57:18.308 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.pGdpuhorMv 2024-09-13T09:57:18.308 DEBUG:teuthology.orchestra.run.smithi136:> sudo cp /boot/grub/grub.cfg /tmp/tmp.pGdpuhorMv 2024-09-13T09:57:18.368 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.pGdpuhorMv 2024-09-13T09:57:18.383 INFO:teuthology.orchestra.run.smithi181.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-13T09:57:18.383 INFO:teuthology.orchestra.run.smithi181.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:57:18.383 INFO:teuthology.orchestra.run.smithi181.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T09:57:18.383 INFO:teuthology.orchestra.run.smithi181.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T09:57:18.383 INFO:teuthology.orchestra.run.smithi181.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T09:57:18.384 INFO:teuthology.orchestra.run.smithi181.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T09:57:18.384 INFO:teuthology.orchestra.run.smithi181.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T09:57:18.384 INFO:teuthology.orchestra.run.smithi181.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:57:18.425 INFO:teuthology.orchestra.run.smithi181.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T09:57:18.429 DEBUG:teuthology.orchestra.run.smithi181:> dpkg -s linux-image-generic 2024-09-13T09:57:18.486 INFO:teuthology.orchestra.run.smithi181.stdout:Package: linux-image-generic 2024-09-13T09:57:18.486 INFO:teuthology.orchestra.run.smithi181.stdout:Status: install ok installed 2024-09-13T09:57:18.486 INFO:teuthology.orchestra.run.smithi181.stdout:Priority: optional 2024-09-13T09:57:18.486 INFO:teuthology.orchestra.run.smithi181.stdout:Section: kernel 2024-09-13T09:57:18.486 INFO:teuthology.orchestra.run.smithi181.stdout:Installed-Size: 21 2024-09-13T09:57:18.486 INFO:teuthology.orchestra.run.smithi181.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T09:57:18.486 INFO:teuthology.orchestra.run.smithi181.stdout:Architecture: amd64 2024-09-13T09:57:18.487 INFO:teuthology.orchestra.run.smithi181.stdout:Source: linux-meta 2024-09-13T09:57:18.487 INFO:teuthology.orchestra.run.smithi181.stdout:Version: 5.15.0.121.121 2024-09-13T09:57:18.487 INFO:teuthology.orchestra.run.smithi181.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T09:57:18.487 INFO:teuthology.orchestra.run.smithi181.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T09:57:18.487 INFO:teuthology.orchestra.run.smithi181.stdout:Recommends: thermald 2024-09-13T09:57:18.487 INFO:teuthology.orchestra.run.smithi181.stdout:Description: Generic Linux kernel image 2024-09-13T09:57:18.487 INFO:teuthology.orchestra.run.smithi181.stdout: This package will always depend on the latest generic kernel image 2024-09-13T09:57:18.487 INFO:teuthology.orchestra.run.smithi181.stdout: available. 2024-09-13T09:57:18.487 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2024-09-13T09:57:18.494 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.pGdpuhorMv is 10KB 2024-09-13T09:57:18.507 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.pGdpuhorMv 2024-09-13T09:57:18.514 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T09:57:18.530 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.Pai8vMh4PA 2024-09-13T09:57:18.531 DEBUG:teuthology.orchestra.run.smithi181:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Pai8vMh4PA 2024-09-13T09:57:18.567 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T09:57:18.567 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T09:57:18.567 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T09:57:18.588 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.Pai8vMh4PA 2024-09-13T09:57:18.664 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-13T09:57:18.665 DEBUG:teuthology.orchestra.run.smithi136:> sudo update-grub 2024-09-13T09:57:18.712 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.Pai8vMh4PA is 10KB 2024-09-13T09:57:18.725 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.Pai8vMh4PA 2024-09-13T09:57:18.730 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T09:57:18.783 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T09:57:18.783 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T09:57:18.783 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T09:57:18.840 INFO:teuthology.orchestra.run.smithi073.stderr:Sourcing file `/etc/default/grub' 2024-09-13T09:57:18.841 INFO:teuthology.orchestra.run.smithi073.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T09:57:18.844 INFO:teuthology.orchestra.run.smithi073.stderr:Generating grub configuration file ... 2024-09-13T09:57:18.874 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-13T09:57:18.874 DEBUG:teuthology.orchestra.run.smithi181:> sudo update-grub 2024-09-13T09:57:19.026 INFO:teuthology.orchestra.run.smithi073.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:57:19.032 INFO:teuthology.orchestra.run.smithi073.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:57:19.218 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-09-13T09:57:19.237 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T09:57:19.300 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-09-13T09:57:19.310 INFO:teuthology.orchestra.run.smithi073.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:57:19.316 INFO:teuthology.orchestra.run.smithi073.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T09:57:19.356 INFO:teuthology.orchestra.run.smithi073.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T09:57:19.361 INFO:teuthology.orchestra.run.smithi073.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T09:57:19.486 INFO:teuthology.orchestra.run.smithi073.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T09:57:19.486 INFO:teuthology.orchestra.run.smithi073.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T09:57:19.486 INFO:teuthology.orchestra.run.smithi073.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T09:57:19.500 INFO:teuthology.orchestra.run.smithi073.stderr:done 2024-09-13T09:57:19.509 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-09-13T09:57:19.510 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-09-13T09:57:19.510 DEBUG:teuthology.orchestra.run.smithi073:> sudo shutdown -r now 2024-09-13T09:57:19.700 INFO:teuthology.orchestra.run.smithi195.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-13T09:57:19.700 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T09:57:19.700 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T09:57:19.700 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T09:57:19.700 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T09:57:19.701 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T09:57:19.701 INFO:teuthology.orchestra.run.smithi195.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T09:57:19.701 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T09:57:19.743 INFO:teuthology.orchestra.run.smithi195.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T09:57:19.745 DEBUG:teuthology.orchestra.run.smithi195:> dpkg -s linux-image-generic 2024-09-13T09:57:19.760 INFO:teuthology.orchestra.run.smithi195.stdout:Package: linux-image-generic 2024-09-13T09:57:19.760 INFO:teuthology.orchestra.run.smithi195.stdout:Status: install ok installed 2024-09-13T09:57:19.760 INFO:teuthology.orchestra.run.smithi195.stdout:Priority: optional 2024-09-13T09:57:19.761 INFO:teuthology.orchestra.run.smithi195.stdout:Section: kernel 2024-09-13T09:57:19.761 INFO:teuthology.orchestra.run.smithi195.stdout:Installed-Size: 21 2024-09-13T09:57:19.761 INFO:teuthology.orchestra.run.smithi195.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T09:57:19.761 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: amd64 2024-09-13T09:57:19.761 INFO:teuthology.orchestra.run.smithi195.stdout:Source: linux-meta 2024-09-13T09:57:19.761 INFO:teuthology.orchestra.run.smithi195.stdout:Version: 5.15.0.121.121 2024-09-13T09:57:19.761 INFO:teuthology.orchestra.run.smithi195.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T09:57:19.761 INFO:teuthology.orchestra.run.smithi195.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T09:57:19.761 INFO:teuthology.orchestra.run.smithi195.stdout:Recommends: thermald 2024-09-13T09:57:19.761 INFO:teuthology.orchestra.run.smithi195.stdout:Description: Generic Linux kernel image 2024-09-13T09:57:19.761 INFO:teuthology.orchestra.run.smithi195.stdout: This package will always depend on the latest generic kernel image 2024-09-13T09:57:19.761 INFO:teuthology.orchestra.run.smithi195.stdout: available. 2024-09-13T09:57:19.762 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-09-13T09:57:19.806 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.HqkZbw885y 2024-09-13T09:57:19.807 DEBUG:teuthology.orchestra.run.smithi195:> sudo cp /boot/grub/grub.cfg /tmp/tmp.HqkZbw885y 2024-09-13T09:57:19.863 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.HqkZbw885y 2024-09-13T09:57:19.963 INFO:teuthology.orchestra.run.smithi136.stderr:Sourcing file `/etc/default/grub' 2024-09-13T09:57:19.965 INFO:teuthology.orchestra.run.smithi136.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T09:57:19.968 INFO:teuthology.orchestra.run.smithi136.stderr:Generating grub configuration file ... 2024-09-13T09:57:19.986 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.HqkZbw885y is 10KB 2024-09-13T09:57:20.002 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.HqkZbw885y 2024-09-13T09:57:20.007 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T09:57:20.062 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T09:57:20.062 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T09:57:20.063 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T09:57:20.070 INFO:teuthology.orchestra.run.smithi181.stderr:Sourcing file `/etc/default/grub' 2024-09-13T09:57:20.071 INFO:teuthology.orchestra.run.smithi181.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T09:57:20.074 INFO:teuthology.orchestra.run.smithi181.stderr:Generating grub configuration file ... 2024-09-13T09:57:20.130 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-13T09:57:20.130 DEBUG:teuthology.orchestra.run.smithi195:> sudo update-grub 2024-09-13T09:57:20.151 INFO:teuthology.orchestra.run.smithi136.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:57:20.156 INFO:teuthology.orchestra.run.smithi136.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:57:20.252 INFO:teuthology.orchestra.run.smithi181.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:57:20.258 INFO:teuthology.orchestra.run.smithi181.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:57:20.415 INFO:teuthology.orchestra.run.smithi136.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:57:20.419 INFO:teuthology.orchestra.run.smithi136.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T09:57:20.461 INFO:teuthology.orchestra.run.smithi136.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T09:57:20.466 INFO:teuthology.orchestra.run.smithi136.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T09:57:20.481 INFO:teuthology.orchestra.run.smithi181.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:57:20.486 INFO:teuthology.orchestra.run.smithi181.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T09:57:20.524 INFO:teuthology.orchestra.run.smithi181.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T09:57:20.529 INFO:teuthology.orchestra.run.smithi181.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T09:57:20.618 INFO:teuthology.orchestra.run.smithi136.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T09:57:20.618 INFO:teuthology.orchestra.run.smithi136.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T09:57:20.619 INFO:teuthology.orchestra.run.smithi136.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T09:57:20.631 INFO:teuthology.orchestra.run.smithi136.stderr:done 2024-09-13T09:57:20.633 INFO:teuthology.orchestra.run.smithi181.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T09:57:20.633 INFO:teuthology.orchestra.run.smithi181.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T09:57:20.633 INFO:teuthology.orchestra.run.smithi181.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T09:57:20.644 INFO:teuthology.orchestra.run.smithi181.stderr:done 2024-09-13T09:57:20.645 DEBUG:teuthology.orchestra.run.smithi136:> sudo shutdown -r now 2024-09-13T09:57:20.661 DEBUG:teuthology.orchestra.run.smithi181:> sudo shutdown -r now 2024-09-13T09:57:21.408 INFO:teuthology.orchestra.run.smithi195.stderr:Sourcing file `/etc/default/grub' 2024-09-13T09:57:21.409 INFO:teuthology.orchestra.run.smithi195.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T09:57:21.411 INFO:teuthology.orchestra.run.smithi195.stderr:Generating grub configuration file ... 2024-09-13T09:57:21.599 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T09:57:21.605 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T09:57:21.843 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T09:57:21.848 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T09:57:21.887 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T09:57:21.892 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T09:57:22.009 INFO:teuthology.orchestra.run.smithi195.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T09:57:22.009 INFO:teuthology.orchestra.run.smithi195.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T09:57:22.009 INFO:teuthology.orchestra.run.smithi195.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T09:57:22.020 INFO:teuthology.orchestra.run.smithi195.stderr:done 2024-09-13T09:57:22.023 DEBUG:teuthology.orchestra.run.smithi195:> sudo shutdown -r now 2024-09-13T09:57:49.541 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T09:57:49.542 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-13T09:57:49.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:57:50.655 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T09:57:50.655 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-13T09:57:50.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:57:50.671 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-09-13T09:57:50.671 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-09-13T09:57:50.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:57:52.027 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-09-13T09:57:52.027 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-13T09:57:52.027 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:58:07.933 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-09-13T09:58:10.493 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.195 2024-09-13T09:58:16.941 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-13T09:58:16.942 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:58:19.496 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-13T09:58:19.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:58:19.997 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-09-13T09:58:22.557 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.195 2024-09-13T09:58:25.341 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-09-13T09:58:32.003 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-13T09:58:32.004 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:58:34.344 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-13T09:58:34.345 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:58:34.557 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-13T09:58:34.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:58:50.433 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-09-13T09:58:50.674 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T09:58:52.733 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-09-13T09:58:52.989 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.195 2024-09-13T09:58:59.682 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-09-13T09:58:59.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:59:00.204 DEBUG:teuthology.orchestra.run.smithi181:> true 2024-09-13T09:59:01.144 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-09-13T09:59:01.144 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.15.0-121-generic"... 2024-09-13T09:59:01.144 DEBUG:teuthology.orchestra.run.smithi181:> uname -r 2024-09-13T09:59:01.191 INFO:teuthology.orchestra.run.smithi181.stdout:5.15.0-121-generic 2024-09-13T09:59:01.191 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-13T09:59:01.191 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T09:59:01.192 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T09:59:02.192 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-09-13T09:59:02.193 DEBUG:teuthology.orchestra.run.smithi181:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T09:59:02.426 INFO:teuthology.orchestra.run.smithi181.stdout:ttyS1 2024-09-13T09:59:02.449 DEBUG:teuthology.parallel:result is None 2024-09-13T09:59:04.734 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-13T09:59:04.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:59:05.264 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-09-13T09:59:05.435 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-13T09:59:05.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:59:05.806 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-09-13T09:59:06.151 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-13T09:59:06.152 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-121-generic"... 2024-09-13T09:59:06.152 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-09-13T09:59:06.157 INFO:teuthology.orchestra.run.smithi136.stdout:5.15.0-121-generic 2024-09-13T09:59:06.158 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-13T09:59:06.158 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T09:59:06.158 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T09:59:06.394 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-13T09:59:06.394 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-121-generic"... 2024-09-13T09:59:06.394 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2024-09-13T09:59:06.400 INFO:teuthology.orchestra.run.smithi073.stdout:5.15.0-121-generic 2024-09-13T09:59:06.400 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-13T09:59:06.400 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T09:59:06.400 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T09:59:07.159 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T09:59:07.159 DEBUG:teuthology.orchestra.run.smithi136:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T09:59:07.298 INFO:teuthology.orchestra.run.smithi136.stdout:ttyS1 2024-09-13T09:59:07.332 DEBUG:teuthology.parallel:result is None 2024-09-13T09:59:07.401 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T09:59:07.401 DEBUG:teuthology.orchestra.run.smithi073:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T09:59:07.488 INFO:teuthology.orchestra.run.smithi073.stdout:ttyS1 2024-09-13T09:59:07.512 DEBUG:teuthology.parallel:result is None 2024-09-13T09:59:07.990 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-13T09:59:07.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T09:59:08.266 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-09-13T09:59:08.915 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-13T09:59:08.915 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.15.0-121-generic"... 2024-09-13T09:59:08.915 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-09-13T09:59:08.962 INFO:teuthology.orchestra.run.smithi195.stdout:5.15.0-121-generic 2024-09-13T09:59:08.962 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-13T09:59:08.962 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T09:59:08.962 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T09:59:09.964 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-09-13T09:59:09.964 DEBUG:teuthology.orchestra.run.smithi195:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T09:59:10.033 INFO:teuthology.orchestra.run.smithi195.stdout:ttyS1 2024-09-13T09:59:10.068 DEBUG:teuthology.parallel:result is None 2024-09-13T09:59:10.068 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T09:59:10.078 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T09:59:10.078 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T09:59:10.081 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T09:59:10.083 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T09:59:10.088 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T09:59:10.093 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T09:59:10.103 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T09:59:10.108 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T09:59:10.108 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T09:59:10.131 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T09:59:10.133 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T09:59:10.136 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T09:59:10.224 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T09:59:10.230 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T09:59:10.230 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T09:59:10.232 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T09:59:10.234 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T09:59:10.237 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T09:59:10.252 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:59:10.253 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:59:10.254 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:59:10.260 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:59:10.261 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:59:10.263 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:59:10.264 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:59:10.268 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T09:59:10.270 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T09:59:10.276 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T09:59:10.276 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T09:59:10.311 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T09:59:10.313 DEBUG:teuthology.orchestra.run.smithi181:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T09:59:10.315 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T09:59:10.331 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T09:59:10.340 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T09:59:10.340 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T09:59:10.366 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T09:59:10.369 DEBUG:teuthology.orchestra.run.smithi181:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T09:59:10.372 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T09:59:10.378 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T09:59:10.417 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T09:59:10.465 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T09:59:10.465 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T09:59:10.559 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T09:59:10.565 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T09:59:10.611 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T09:59:10.611 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T09:59:10.672 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T09:59:10.677 DEBUG:teuthology.orchestra.run.smithi181:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T09:59:10.723 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T09:59:10.723 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T09:59:10.790 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T09:59:10.796 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T09:59:10.842 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T09:59:10.842 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T09:59:10.914 DEBUG:teuthology.orchestra.run.smithi073:> sudo service rsyslog restart 2024-09-13T09:59:10.916 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-09-13T09:59:10.921 DEBUG:teuthology.orchestra.run.smithi181:> sudo service rsyslog restart 2024-09-13T09:59:10.923 DEBUG:teuthology.orchestra.run.smithi195:> sudo service rsyslog restart 2024-09-13T09:59:10.987 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T09:59:10.998 INFO:teuthology.task.internal:Starting timer... 2024-09-13T09:59:10.998 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T09:59:11.013 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T09:59:11.026 DEBUG:teuthology.task.selinux:Excluding smithi073: OS 'ubuntu' does not support SELinux 2024-09-13T09:59:11.026 DEBUG:teuthology.task.selinux:Excluding smithi136: OS 'ubuntu' does not support SELinux 2024-09-13T09:59:11.027 DEBUG:teuthology.task.selinux:Excluding smithi181: OS 'ubuntu' does not support SELinux 2024-09-13T09:59:11.027 DEBUG:teuthology.task.selinux:Excluding smithi195: OS 'ubuntu' does not support SELinux 2024-09-13T09:59:11.027 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T09:59:11.027 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-13T09:59:11.027 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T09:59:11.027 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T09:59:11.039 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T09:59:11.051 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T09:59:11.052 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi073.front.sepia.ceph.com,smithi136.front.sepia.ceph.com,smithi181.front.sepia.ceph.com,smithi195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T10:06:43.740 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi073.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com'), Remote(name='ubuntu@smithi181.front.sepia.ceph.com'), Remote(name='ubuntu@smithi195.front.sepia.ceph.com')] 2024-09-13T10:06:43.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-13T10:06:43.744 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:06:43.827 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-09-13T10:06:43.905 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-13T10:06:43.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-13T10:06:43.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:06:43.987 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-09-13T10:06:44.061 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-09-13T10:06:44.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-09-13T10:06:44.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi181.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:06:44.142 DEBUG:teuthology.orchestra.run.smithi181:> true 2024-09-13T10:06:44.220 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi181.front.sepia.ceph.com' 2024-09-13T10:06:44.221 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-13T10:06:44.221 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T10:06:44.298 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-09-13T10:06:44.373 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-09-13T10:06:44.374 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T10:06:44.385 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T10:06:44.385 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T10:06:44.386 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T10:06:44.388 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T10:06:44.389 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T10:06:44.391 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T10:06:44.391 DEBUG:teuthology.orchestra.run.smithi181:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T10:06:44.393 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T10:06:44.393 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T10:06:44.417 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-13T10:06:44.417 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: Command line: ntpd -gq 2024-09-13T10:06:44.417 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: ---------------------------------------------------- 2024-09-13T10:06:44.417 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: ntp-4 is maintained by Network Time Foundation, 2024-09-13T10:06:44.418 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-13T10:06:44.418 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: corporation. Support and training for ntp-4 are 2024-09-13T10:06:44.418 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: available at https://www.nwtime.org/support 2024-09-13T10:06:44.418 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: ---------------------------------------------------- 2024-09-13T10:06:44.418 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: proto: precision = 0.046 usec (-24) 2024-09-13T10:06:44.418 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: basedate set to 2022-02-04 2024-09-13T10:06:44.418 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: gps base set to 2022-02-06 (week 2196) 2024-09-13T10:06:44.418 INFO:teuthology.orchestra.run.smithi073.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T10:06:44.419 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T10:06:44.419 INFO:teuthology.orchestra.run.smithi073.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T10:06:44.420 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T10:06:44.420 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T10:06:44.420 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T10:06:44.420 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T10:06:44.420 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: Listen normally on 3 enp3s0f1 172.21.15.73:123 2024-09-13T10:06:44.420 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: Listen normally on 4 lo [::1]:123 2024-09-13T10:06:44.420 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce7d%5]:123 2024-09-13T10:06:44.420 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:44 ntpd[18546]: Listening on routing socket on fd #22 for interface updates 2024-09-13T10:06:44.422 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-13T10:06:44.422 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: Command line: ntpd -gq 2024-09-13T10:06:44.422 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: ---------------------------------------------------- 2024-09-13T10:06:44.422 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: ntp-4 is maintained by Network Time Foundation, 2024-09-13T10:06:44.422 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-13T10:06:44.422 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: corporation. Support and training for ntp-4 are 2024-09-13T10:06:44.422 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: available at https://www.nwtime.org/support 2024-09-13T10:06:44.422 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: ---------------------------------------------------- 2024-09-13T10:06:44.422 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: proto: precision = 0.047 usec (-24) 2024-09-13T10:06:44.422 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: basedate set to 2022-02-04 2024-09-13T10:06:44.422 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: gps base set to 2022-02-06 (week 2196) 2024-09-13T10:06:44.423 INFO:teuthology.orchestra.run.smithi136.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T10:06:44.423 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-13T10:06:44.423 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: Command line: ntpd -gq 2024-09-13T10:06:44.423 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: ---------------------------------------------------- 2024-09-13T10:06:44.423 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: ntp-4 is maintained by Network Time Foundation, 2024-09-13T10:06:44.423 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-13T10:06:44.423 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: corporation. Support and training for ntp-4 are 2024-09-13T10:06:44.423 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: available at https://www.nwtime.org/support 2024-09-13T10:06:44.424 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: ---------------------------------------------------- 2024-09-13T10:06:44.424 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: proto: precision = 0.062 usec (-24) 2024-09-13T10:06:44.424 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: basedate set to 2022-02-04 2024-09-13T10:06:44.424 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: gps base set to 2022-02-06 (week 2196) 2024-09-13T10:06:44.424 INFO:teuthology.orchestra.run.smithi181.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T10:06:44.424 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T10:06:44.425 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T10:06:44.425 INFO:teuthology.orchestra.run.smithi136.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T10:06:44.425 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T10:06:44.425 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T10:06:44.426 INFO:teuthology.orchestra.run.smithi181.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T10:06:44.427 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T10:06:44.427 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T10:06:44.427 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T10:06:44.427 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: Listen normally on 3 enp3s0f1 172.21.15.136:123 2024-09-13T10:06:44.427 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: Listen normally on 4 lo [::1]:123 2024-09-13T10:06:44.428 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6e43%5]:123 2024-09-13T10:06:44.428 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:44 ntpd[18458]: Listening on routing socket on fd #22 for interface updates 2024-09-13T10:06:44.428 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T10:06:44.428 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T10:06:44.428 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T10:06:44.428 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: Listen normally on 3 enp3s0f1 172.21.15.181:123 2024-09-13T10:06:44.428 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: Listen normally on 4 lo [::1]:123 2024-09-13T10:06:44.428 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7b9d%5]:123 2024-09-13T10:06:44.428 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:44 ntpd[18469]: Listening on routing socket on fd #22 for interface updates 2024-09-13T10:06:44.442 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-13T10:06:44.442 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: Command line: ntpd -gq 2024-09-13T10:06:44.442 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: ---------------------------------------------------- 2024-09-13T10:06:44.442 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: ntp-4 is maintained by Network Time Foundation, 2024-09-13T10:06:44.442 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-13T10:06:44.442 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: corporation. Support and training for ntp-4 are 2024-09-13T10:06:44.442 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: available at https://www.nwtime.org/support 2024-09-13T10:06:44.442 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: ---------------------------------------------------- 2024-09-13T10:06:44.442 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: proto: precision = 0.062 usec (-24) 2024-09-13T10:06:44.442 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: basedate set to 2022-02-04 2024-09-13T10:06:44.442 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: gps base set to 2022-02-06 (week 2196) 2024-09-13T10:06:44.443 INFO:teuthology.orchestra.run.smithi195.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T10:06:44.443 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T10:06:44.444 INFO:teuthology.orchestra.run.smithi195.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T10:06:44.445 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T10:06:44.445 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T10:06:44.445 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T10:06:44.445 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T10:06:44.446 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: Listen normally on 3 enp3s0f1 172.21.15.195:123 2024-09-13T10:06:44.446 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: Listen normally on 4 lo [::1]:123 2024-09-13T10:06:44.446 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8081%5]:123 2024-09-13T10:06:44.446 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:44 ntpd[18465]: Listening on routing socket on fd #22 for interface updates 2024-09-13T10:06:45.417 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:45 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:45.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:45 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:45.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:45 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:45.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:45 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:45.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:45 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:45.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:45 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:45.419 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:45.419 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:45.419 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:45.419 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:45.420 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:45.420 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:45.420 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:45.420 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:45.420 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:45.420 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:45.420 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:45.420 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:45.420 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:45.420 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:45.420 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:45.420 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:45.421 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:45.421 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:45 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:45.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:45 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:45.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:45 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:45.422 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:45 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:45.422 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:45 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:45.422 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:45 ntpd[18469]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:45.422 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:45 ntpd[18469]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:46.418 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:46 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:46.418 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:46 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:46.420 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:46 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:46.420 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:46 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:46.420 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:46 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:46.420 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:46 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:46.420 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:46 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:46.420 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:46 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:46.420 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:46 ntpd[18469]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:46.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:46 ntpd[18469]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:46.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:46 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:46.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:46 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:46.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:46 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:46.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:46 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:46.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:46 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:46.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:46 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:46.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:46 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:46.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:46 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:47.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:47 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:47.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:47 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:47.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:47 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:47.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:47 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:47.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:47 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:47.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:47 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:47.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:47 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:47.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:47 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:47.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:47 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:47.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:47 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:47.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:47 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:47.419 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:47 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:47.420 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:47.421 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:47.421 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:47.421 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:47.421 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:47.421 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:47.421 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:47.421 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:47.421 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:47.421 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:47.421 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:47.422 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:47.422 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:47.422 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:47.422 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:47.422 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:47.422 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:47.422 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:47 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:47.423 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:47 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:47.423 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:47 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:47.424 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:47 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:47.424 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:47 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:47.424 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:47 ntpd[18469]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:47.424 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:47 ntpd[18469]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:47.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:47 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:47.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:47 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:47.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:47 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:47.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:47 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:47.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:47 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:47.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:47 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:48.417 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:48 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:48.417 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:48 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:48.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:48 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:48.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:48 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:48.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:48 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:48.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:48 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:48.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:48 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:48.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:48 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:48.420 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:48 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:48.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:48 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:48.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:48 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:48.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:48 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:48.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:48 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:48.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:48 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:48.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:48 ntpd[18469]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:48.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:48 ntpd[18469]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:48.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:48 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:48.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:48 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:48.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:48 ntpd[18469]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:48.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:48 ntpd[18469]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:48.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:48 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:48.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:48 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:48.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:48 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:48.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:48 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:48.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:48 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:48.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:48 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:49.419 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:49 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:49.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:49 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:49.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:49 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:49.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:49 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:49.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:49 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:49.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:49 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:49.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:49 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:49.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:49 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:49.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:49 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:49.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:49 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:49.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:49 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:49.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:49 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:49.422 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:49 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:49.422 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:49 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:49.422 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:49 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:49.422 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:49 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:49.422 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:49 ntpd[18469]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:49.422 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:49 ntpd[18469]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:49.424 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:49.424 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:49.424 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:49.424 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:49.424 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:49.424 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:49.424 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:49.424 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:49.424 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:49.425 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:49.425 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:49.425 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:49.425 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:49.425 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:49.425 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:49.425 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:49.425 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:49.425 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:49 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:49.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:49 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:49.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:49 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:49.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:49 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:49.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:49 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:49.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:49 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:49.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:49 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:49.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:49 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:49.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:49 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:49.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:49 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:49.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:49 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:49.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:49 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:49.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:49 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:50.417 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:50 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:50.417 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:50 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:50.417 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:50 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:50.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:50 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:50.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:50 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:50.418 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:50 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:50.420 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:50 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:50.420 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:50 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:50.420 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:50 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:50.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:50 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:50.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:50 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:50.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:50 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:50.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:50 ntpd[18469]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:50.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:50 ntpd[18469]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:50.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:50 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:50.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:50 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:50.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:50 ntpd[18469]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:50.421 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:50 ntpd[18469]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:50.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:50 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:50.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:50 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:50.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:50 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:50.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:50 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:50.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:50 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:50.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:50 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:51.419 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:51 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:51.419 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:51 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:51.419 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:51 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:51.419 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:51 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:51.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:51 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:51.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:51 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:51.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:51 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:51.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:51 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:51.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:51 ntpd[18546]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:51.420 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:51 ntpd[18546]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:51.421 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:51 ntpd[18546]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:51.421 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:51 ntpd[18546]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:51.421 INFO:teuthology.orchestra.run.smithi073.stdout:13 Sep 10:06:51 ntpd[18546]: ntpd: time slew -0.000977 s 2024-09-13T10:06:51.422 INFO:teuthology.orchestra.run.smithi073.stdout:ntpd: time slew -0.000977s 2024-09-13T10:06:51.422 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:51 ntpd[18546]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T10:06:51.422 INFO:teuthology.orchestra.run.smithi073.stderr:13 Sep 10:06:51 ntpd[18546]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T10:06:51.426 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:51 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:51.426 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:51 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:51.426 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:51 ntpd[18469]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:51.426 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:51 ntpd[18469]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:51.426 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:51 ntpd[18469]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:51.427 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:51 ntpd[18469]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:51.427 INFO:teuthology.orchestra.run.smithi181.stdout:13 Sep 10:06:51 ntpd[18469]: ntpd: time slew -0.000724 s 2024-09-13T10:06:51.428 INFO:teuthology.orchestra.run.smithi181.stdout:ntpd: time slew -0.000724s 2024-09-13T10:06:51.429 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:51 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:51.429 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:51 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:51.429 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:51 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:51.430 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:51 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:51.430 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:51 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:51.430 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:51 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:51.430 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:51 ntpd[18458]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:51.430 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:51 ntpd[18458]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:51.431 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:51 ntpd[18458]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:51.431 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:51 ntpd[18458]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:51.431 INFO:teuthology.orchestra.run.smithi136.stdout:13 Sep 10:06:51 ntpd[18458]: ntpd: time slew -0.001362 s 2024-09-13T10:06:51.432 INFO:teuthology.orchestra.run.smithi136.stdout:ntpd: time slew -0.001362s 2024-09-13T10:06:51.432 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:51 ntpd[18469]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T10:06:51.432 INFO:teuthology.orchestra.run.smithi181.stderr:13 Sep 10:06:51 ntpd[18469]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T10:06:51.433 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:51 ntpd[18458]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T10:06:51.433 INFO:teuthology.orchestra.run.smithi136.stderr:13 Sep 10:06:51 ntpd[18458]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T10:06:51.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:51 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:51.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:51 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:51.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:51 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:51.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:51 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:51.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:51 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:51.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:51 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:51.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:51 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:51.444 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:51 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:51.444 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:51 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:51.444 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:51 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:51.444 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:51 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:51.444 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:51 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T10:06:51.462 INFO:teuthology.orchestra.run.smithi136.stdout:============================================================================== 2024-09-13T10:06:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:06:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:06:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:06:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:06:51.468 INFO:teuthology.orchestra.run.smithi073.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T10:06:51.468 INFO:teuthology.orchestra.run.smithi073.stdout:============================================================================== 2024-09-13T10:06:51.468 INFO:teuthology.orchestra.run.smithi073.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:06:51.468 INFO:teuthology.orchestra.run.smithi073.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:06:51.468 INFO:teuthology.orchestra.run.smithi073.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:06:51.468 INFO:teuthology.orchestra.run.smithi073.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:06:51.475 INFO:teuthology.orchestra.run.smithi181.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T10:06:51.475 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================== 2024-09-13T10:06:51.475 INFO:teuthology.orchestra.run.smithi181.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:06:51.475 INFO:teuthology.orchestra.run.smithi181.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:06:51.476 INFO:teuthology.orchestra.run.smithi181.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:06:51.476 INFO:teuthology.orchestra.run.smithi181.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:06:52.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:52 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:52.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:52 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:52.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:52 ntpd[18465]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T10:06:52.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:52 ntpd[18465]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T10:06:52.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:52 ntpd[18465]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T10:06:52.442 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:52 ntpd[18465]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T10:06:52.443 INFO:teuthology.orchestra.run.smithi195.stdout:13 Sep 10:06:52 ntpd[18465]: ntpd: time slew -0.000776 s 2024-09-13T10:06:52.443 INFO:teuthology.orchestra.run.smithi195.stdout:ntpd: time slew -0.000776s 2024-09-13T10:06:52.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:52 ntpd[18465]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T10:06:52.443 INFO:teuthology.orchestra.run.smithi195.stderr:13 Sep 10:06:52 ntpd[18465]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T10:06:52.502 INFO:teuthology.orchestra.run.smithi195.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T10:06:52.503 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================== 2024-09-13T10:06:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:06:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:06:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:06:52.503 INFO:teuthology.orchestra.run.smithi195.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:06:52.504 INFO:teuthology.run_tasks:Running task install... 2024-09-13T10:06:52.512 DEBUG:teuthology.task.install:project ceph 2024-09-13T10:06:52.513 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'}} 2024-09-13T10:06:52.513 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'} 2024-09-13T10:06:52.513 INFO:teuthology.task.install:Using flavor: default 2024-09-13T10:06:52.519 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T10:06:52.519 INFO:teuthology.task.install:extra packages: [] 2024-09-13T10:06:52.520 DEBUG:teuthology.orchestra.run.smithi073:> sudo apt-key list | grep Ceph 2024-09-13T10:06:52.520 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-key list | grep Ceph 2024-09-13T10:06:52.520 DEBUG:teuthology.orchestra.run.smithi181:> sudo apt-key list | grep Ceph 2024-09-13T10:06:52.521 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-key list | grep Ceph 2024-09-13T10:06:52.582 INFO:teuthology.orchestra.run.smithi181.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-13T10:06:52.586 INFO:teuthology.orchestra.run.smithi073.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-13T10:06:52.590 INFO:teuthology.orchestra.run.smithi136.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-13T10:06:52.606 INFO:teuthology.orchestra.run.smithi195.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-13T10:06:52.612 INFO:teuthology.orchestra.run.smithi181.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T10:06:52.612 INFO:teuthology.orchestra.run.smithi181.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T10:06:52.613 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-13T10:06:52.614 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:06:52.620 INFO:teuthology.orchestra.run.smithi073.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T10:06:52.620 INFO:teuthology.orchestra.run.smithi073.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T10:06:52.620 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-13T10:06:52.620 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:06:52.624 INFO:teuthology.orchestra.run.smithi136.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T10:06:52.624 INFO:teuthology.orchestra.run.smithi136.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T10:06:52.624 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-13T10:06:52.624 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:06:52.633 INFO:teuthology.orchestra.run.smithi195.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T10:06:52.633 INFO:teuthology.orchestra.run.smithi195.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T10:06:52.634 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-13T10:06:52.634 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:06:52.766 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default/ 2024-09-13T10:06:52.766 INFO:teuthology.task.install.deb:Package version is 19.3.0-4697-gd58f7306-1jammy 2024-09-13T10:06:52.770 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default/ 2024-09-13T10:06:52.770 INFO:teuthology.task.install.deb:Package version is 19.3.0-4697-gd58f7306-1jammy 2024-09-13T10:06:52.771 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default/ 2024-09-13T10:06:52.771 INFO:teuthology.task.install.deb:Package version is 19.3.0-4697-gd58f7306-1jammy 2024-09-13T10:06:52.783 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default/ 2024-09-13T10:06:52.784 INFO:teuthology.task.install.deb:Package version is 19.3.0-4697-gd58f7306-1jammy 2024-09-13T10:06:52.877 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T10:06:52.878 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T10:06:52.885 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T10:06:52.885 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T10:06:52.888 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:06:52.889 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T10:06:52.892 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get update 2024-09-13T10:06:52.894 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:06:52.894 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T10:06:52.903 DEBUG:teuthology.orchestra.run.smithi073:> sudo apt-get update 2024-09-13T10:06:52.908 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2024-09-13T10:06:52.934 DEBUG:teuthology.orchestra.run.smithi181:> sudo apt-get update 2024-09-13T10:06:53.058 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T10:06:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T10:06:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T10:06:53.074 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T10:06:53.079 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T10:06:53.083 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T10:06:53.097 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T10:06:53.101 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T10:06:53.104 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T10:06:53.112 INFO:teuthology.orchestra.run.smithi136.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy InRelease 2024-09-13T10:06:53.113 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T10:06:53.124 INFO:teuthology.orchestra.run.smithi073.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy InRelease 2024-09-13T10:06:53.130 INFO:teuthology.orchestra.run.smithi195.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy InRelease 2024-09-13T10:06:53.130 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T10:06:53.135 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-13T10:06:53.145 INFO:teuthology.orchestra.run.smithi073.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-13T10:06:53.153 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-13T10:06:53.157 INFO:teuthology.orchestra.run.smithi136.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-13T10:06:53.158 INFO:teuthology.orchestra.run.smithi181.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy InRelease 2024-09-13T10:06:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-13T10:06:53.175 INFO:teuthology.orchestra.run.smithi195.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-13T10:06:53.179 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T10:06:53.179 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,426 B] 2024-09-13T10:06:53.179 INFO:teuthology.orchestra.run.smithi181.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-13T10:06:53.187 INFO:teuthology.orchestra.run.smithi073.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,426 B] 2024-09-13T10:06:53.195 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:7 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T10:06:53.196 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,426 B] 2024-09-13T10:06:53.202 INFO:teuthology.orchestra.run.smithi181.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-13T10:06:53.226 INFO:teuthology.orchestra.run.smithi181.stdout:Hit:7 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T10:06:53.227 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-13T10:06:53.228 INFO:teuthology.orchestra.run.smithi073.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-13T10:06:53.228 INFO:teuthology.orchestra.run.smithi181.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,426 B] 2024-09-13T10:06:53.228 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-13T10:06:53.254 INFO:teuthology.orchestra.run.smithi181.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-13T10:06:53.261 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T10:06:53.342 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T10:06:53.415 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 29.3 kB in 0s (88.8 kB/s) 2024-09-13T10:06:53.425 INFO:teuthology.orchestra.run.smithi073.stdout:Fetched 29.3 kB in 0s (85.0 kB/s) 2024-09-13T10:06:53.440 INFO:teuthology.orchestra.run.smithi181.stdout:Fetched 29.3 kB in 0s (89.5 kB/s) 2024-09-13T10:06:53.486 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 29.3 kB in 0s (70.9 kB/s) 2024-09-13T10:06:54.426 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-09-13T10:06:54.427 INFO:teuthology.orchestra.run.smithi073.stdout:Reading package lists... 2024-09-13T10:06:54.442 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4697-gd58f7306-1jammy cephadm=19.3.0-4697-gd58f7306-1jammy ceph-mds=19.3.0-4697-gd58f7306-1jammy ceph-mgr=19.3.0-4697-gd58f7306-1jammy ceph-common=19.3.0-4697-gd58f7306-1jammy ceph-fuse=19.3.0-4697-gd58f7306-1jammy ceph-test=19.3.0-4697-gd58f7306-1jammy ceph-volume=19.3.0-4697-gd58f7306-1jammy radosgw=19.3.0-4697-gd58f7306-1jammy python3-rados=19.3.0-4697-gd58f7306-1jammy python3-rgw=19.3.0-4697-gd58f7306-1jammy python3-cephfs=19.3.0-4697-gd58f7306-1jammy python3-rbd=19.3.0-4697-gd58f7306-1jammy libcephfs2=19.3.0-4697-gd58f7306-1jammy libcephfs-dev=19.3.0-4697-gd58f7306-1jammy librados2=19.3.0-4697-gd58f7306-1jammy librbd1=19.3.0-4697-gd58f7306-1jammy rbd-fuse=19.3.0-4697-gd58f7306-1jammy 2024-09-13T10:06:54.443 DEBUG:teuthology.orchestra.run.smithi073:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4697-gd58f7306-1jammy cephadm=19.3.0-4697-gd58f7306-1jammy ceph-mds=19.3.0-4697-gd58f7306-1jammy ceph-mgr=19.3.0-4697-gd58f7306-1jammy ceph-common=19.3.0-4697-gd58f7306-1jammy ceph-fuse=19.3.0-4697-gd58f7306-1jammy ceph-test=19.3.0-4697-gd58f7306-1jammy ceph-volume=19.3.0-4697-gd58f7306-1jammy radosgw=19.3.0-4697-gd58f7306-1jammy python3-rados=19.3.0-4697-gd58f7306-1jammy python3-rgw=19.3.0-4697-gd58f7306-1jammy python3-cephfs=19.3.0-4697-gd58f7306-1jammy python3-rbd=19.3.0-4697-gd58f7306-1jammy libcephfs2=19.3.0-4697-gd58f7306-1jammy libcephfs-dev=19.3.0-4697-gd58f7306-1jammy librados2=19.3.0-4697-gd58f7306-1jammy librbd1=19.3.0-4697-gd58f7306-1jammy rbd-fuse=19.3.0-4697-gd58f7306-1jammy 2024-09-13T10:06:54.444 INFO:teuthology.orchestra.run.smithi181.stdout:Reading package lists... 2024-09-13T10:06:54.462 DEBUG:teuthology.orchestra.run.smithi181:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4697-gd58f7306-1jammy cephadm=19.3.0-4697-gd58f7306-1jammy ceph-mds=19.3.0-4697-gd58f7306-1jammy ceph-mgr=19.3.0-4697-gd58f7306-1jammy ceph-common=19.3.0-4697-gd58f7306-1jammy ceph-fuse=19.3.0-4697-gd58f7306-1jammy ceph-test=19.3.0-4697-gd58f7306-1jammy ceph-volume=19.3.0-4697-gd58f7306-1jammy radosgw=19.3.0-4697-gd58f7306-1jammy python3-rados=19.3.0-4697-gd58f7306-1jammy python3-rgw=19.3.0-4697-gd58f7306-1jammy python3-cephfs=19.3.0-4697-gd58f7306-1jammy python3-rbd=19.3.0-4697-gd58f7306-1jammy libcephfs2=19.3.0-4697-gd58f7306-1jammy libcephfs-dev=19.3.0-4697-gd58f7306-1jammy librados2=19.3.0-4697-gd58f7306-1jammy librbd1=19.3.0-4697-gd58f7306-1jammy rbd-fuse=19.3.0-4697-gd58f7306-1jammy 2024-09-13T10:06:54.493 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-13T10:06:54.498 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-09-13T10:06:54.499 INFO:teuthology.orchestra.run.smithi073.stdout:Reading package lists... 2024-09-13T10:06:54.509 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4697-gd58f7306-1jammy cephadm=19.3.0-4697-gd58f7306-1jammy ceph-mds=19.3.0-4697-gd58f7306-1jammy ceph-mgr=19.3.0-4697-gd58f7306-1jammy ceph-common=19.3.0-4697-gd58f7306-1jammy ceph-fuse=19.3.0-4697-gd58f7306-1jammy ceph-test=19.3.0-4697-gd58f7306-1jammy ceph-volume=19.3.0-4697-gd58f7306-1jammy radosgw=19.3.0-4697-gd58f7306-1jammy python3-rados=19.3.0-4697-gd58f7306-1jammy python3-rgw=19.3.0-4697-gd58f7306-1jammy python3-cephfs=19.3.0-4697-gd58f7306-1jammy python3-rbd=19.3.0-4697-gd58f7306-1jammy libcephfs2=19.3.0-4697-gd58f7306-1jammy libcephfs-dev=19.3.0-4697-gd58f7306-1jammy librados2=19.3.0-4697-gd58f7306-1jammy librbd1=19.3.0-4697-gd58f7306-1jammy rbd-fuse=19.3.0-4697-gd58f7306-1jammy 2024-09-13T10:06:54.518 INFO:teuthology.orchestra.run.smithi181.stdout:Reading package lists... 2024-09-13T10:06:54.575 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-09-13T10:06:54.684 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-09-13T10:06:54.685 INFO:teuthology.orchestra.run.smithi073.stdout:Building dependency tree... 2024-09-13T10:06:54.685 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-09-13T10:06:54.685 INFO:teuthology.orchestra.run.smithi073.stdout:Reading state information... 2024-09-13T10:06:54.704 INFO:teuthology.orchestra.run.smithi181.stdout:Building dependency tree... 2024-09-13T10:06:54.705 INFO:teuthology.orchestra.run.smithi181.stdout:Reading state information... 2024-09-13T10:06:54.777 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2024-09-13T10:06:54.777 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2024-09-13T10:06:54.872 INFO:teuthology.orchestra.run.smithi073.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T10:06:54.873 INFO:teuthology.orchestra.run.smithi073.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T10:06:54.873 INFO:teuthology.orchestra.run.smithi073.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T10:06:54.873 INFO:teuthology.orchestra.run.smithi073.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T10:06:54.875 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T10:06:54.875 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T10:06:54.875 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T10:06:54.875 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T10:06:54.875 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T10:06:54.875 INFO:teuthology.orchestra.run.smithi195.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T10:06:54.875 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T10:06:54.876 INFO:teuthology.orchestra.run.smithi073.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T10:06:54.876 INFO:teuthology.orchestra.run.smithi073.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T10:06:54.876 INFO:teuthology.orchestra.run.smithi073.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T10:06:54.876 INFO:teuthology.orchestra.run.smithi073.stdout:The following additional packages will be installed: 2024-09-13T10:06:54.876 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T10:06:54.876 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-13T10:06:54.876 INFO:teuthology.orchestra.run.smithi073.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-13T10:06:54.876 INFO:teuthology.orchestra.run.smithi073.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-13T10:06:54.877 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-13T10:06:54.877 INFO:teuthology.orchestra.run.smithi073.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-13T10:06:54.877 INFO:teuthology.orchestra.run.smithi073.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-13T10:06:54.877 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-13T10:06:54.877 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-13T10:06:54.877 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-13T10:06:54.877 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T10:06:54.877 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T10:06:54.877 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T10:06:54.877 INFO:teuthology.orchestra.run.smithi073.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T10:06:54.878 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T10:06:54.878 INFO:teuthology.orchestra.run.smithi073.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T10:06:54.878 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-13T10:06:54.878 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T10:06:54.878 INFO:teuthology.orchestra.run.smithi073.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T10:06:54.878 INFO:teuthology.orchestra.run.smithi073.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T10:06:54.878 INFO:teuthology.orchestra.run.smithi073.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T10:06:54.878 INFO:teuthology.orchestra.run.smithi073.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T10:06:54.878 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-13T10:06:54.878 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet zip 2024-09-13T10:06:54.879 INFO:teuthology.orchestra.run.smithi073.stdout:Suggested packages: 2024-09-13T10:06:54.879 INFO:teuthology.orchestra.run.smithi073.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-13T10:06:54.879 INFO:teuthology.orchestra.run.smithi073.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-13T10:06:54.879 INFO:teuthology.orchestra.run.smithi073.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-13T10:06:54.879 INFO:teuthology.orchestra.run.smithi073.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-13T10:06:54.879 INFO:teuthology.orchestra.run.smithi073.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-13T10:06:54.880 INFO:teuthology.orchestra.run.smithi073.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T10:06:54.880 INFO:teuthology.orchestra.run.smithi073.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-13T10:06:54.880 INFO:teuthology.orchestra.run.smithi073.stdout:Recommended packages: 2024-09-13T10:06:54.880 INFO:teuthology.orchestra.run.smithi073.stdout: btrfs-tools 2024-09-13T10:06:54.881 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2024-09-13T10:06:54.881 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T10:06:54.881 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-13T10:06:54.881 INFO:teuthology.orchestra.run.smithi195.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-13T10:06:54.881 INFO:teuthology.orchestra.run.smithi195.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-13T10:06:54.881 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-13T10:06:54.881 INFO:teuthology.orchestra.run.smithi195.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-13T10:06:54.882 INFO:teuthology.orchestra.run.smithi195.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-13T10:06:54.882 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-13T10:06:54.882 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-13T10:06:54.882 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-13T10:06:54.882 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T10:06:54.882 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T10:06:54.882 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T10:06:54.882 INFO:teuthology.orchestra.run.smithi195.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T10:06:54.882 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T10:06:54.882 INFO:teuthology.orchestra.run.smithi195.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T10:06:54.882 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-13T10:06:54.882 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T10:06:54.882 INFO:teuthology.orchestra.run.smithi195.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T10:06:54.883 INFO:teuthology.orchestra.run.smithi195.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T10:06:54.883 INFO:teuthology.orchestra.run.smithi195.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T10:06:54.883 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T10:06:54.883 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-13T10:06:54.883 INFO:teuthology.orchestra.run.smithi195.stdout: xmlstarlet zip 2024-09-13T10:06:54.883 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2024-09-13T10:06:54.883 INFO:teuthology.orchestra.run.smithi195.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-13T10:06:54.883 INFO:teuthology.orchestra.run.smithi195.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-13T10:06:54.883 INFO:teuthology.orchestra.run.smithi195.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-13T10:06:54.883 INFO:teuthology.orchestra.run.smithi195.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-13T10:06:54.883 INFO:teuthology.orchestra.run.smithi195.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-13T10:06:54.883 INFO:teuthology.orchestra.run.smithi195.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T10:06:54.883 INFO:teuthology.orchestra.run.smithi195.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-13T10:06:54.883 INFO:teuthology.orchestra.run.smithi195.stdout:Recommended packages: 2024-09-13T10:06:54.884 INFO:teuthology.orchestra.run.smithi195.stdout: btrfs-tools 2024-09-13T10:06:54.890 INFO:teuthology.orchestra.run.smithi181.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T10:06:54.890 INFO:teuthology.orchestra.run.smithi181.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T10:06:54.890 INFO:teuthology.orchestra.run.smithi181.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T10:06:54.890 INFO:teuthology.orchestra.run.smithi181.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T10:06:54.891 INFO:teuthology.orchestra.run.smithi181.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T10:06:54.891 INFO:teuthology.orchestra.run.smithi181.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T10:06:54.891 INFO:teuthology.orchestra.run.smithi181.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T10:06:54.892 INFO:teuthology.orchestra.run.smithi181.stdout:The following additional packages will be installed: 2024-09-13T10:06:54.892 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T10:06:54.892 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-13T10:06:54.892 INFO:teuthology.orchestra.run.smithi181.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-13T10:06:54.892 INFO:teuthology.orchestra.run.smithi181.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-13T10:06:54.893 INFO:teuthology.orchestra.run.smithi181.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-13T10:06:54.893 INFO:teuthology.orchestra.run.smithi181.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-13T10:06:54.893 INFO:teuthology.orchestra.run.smithi181.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-13T10:06:54.893 INFO:teuthology.orchestra.run.smithi181.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-13T10:06:54.893 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-13T10:06:54.893 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-13T10:06:54.893 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T10:06:54.893 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T10:06:54.894 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T10:06:54.894 INFO:teuthology.orchestra.run.smithi181.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T10:06:54.894 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T10:06:54.894 INFO:teuthology.orchestra.run.smithi181.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T10:06:54.894 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-13T10:06:54.894 INFO:teuthology.orchestra.run.smithi181.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T10:06:54.894 INFO:teuthology.orchestra.run.smithi181.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T10:06:54.894 INFO:teuthology.orchestra.run.smithi181.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T10:06:54.894 INFO:teuthology.orchestra.run.smithi181.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T10:06:54.894 INFO:teuthology.orchestra.run.smithi181.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T10:06:54.894 INFO:teuthology.orchestra.run.smithi181.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-13T10:06:54.894 INFO:teuthology.orchestra.run.smithi181.stdout: xmlstarlet zip 2024-09-13T10:06:54.895 INFO:teuthology.orchestra.run.smithi181.stdout:Suggested packages: 2024-09-13T10:06:54.895 INFO:teuthology.orchestra.run.smithi181.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-13T10:06:54.895 INFO:teuthology.orchestra.run.smithi181.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-13T10:06:54.895 INFO:teuthology.orchestra.run.smithi181.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-13T10:06:54.895 INFO:teuthology.orchestra.run.smithi181.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-13T10:06:54.895 INFO:teuthology.orchestra.run.smithi181.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-13T10:06:54.895 INFO:teuthology.orchestra.run.smithi181.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T10:06:54.895 INFO:teuthology.orchestra.run.smithi181.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-13T10:06:54.895 INFO:teuthology.orchestra.run.smithi181.stdout:Recommended packages: 2024-09-13T10:06:54.896 INFO:teuthology.orchestra.run.smithi181.stdout: btrfs-tools 2024-09-13T10:06:54.981 INFO:teuthology.orchestra.run.smithi073.stdout:The following NEW packages will be installed: 2024-09-13T10:06:54.981 INFO:teuthology.orchestra.run.smithi073.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T10:06:54.981 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T10:06:54.981 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-13T10:06:54.981 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-13T10:06:54.981 INFO:teuthology.orchestra.run.smithi073.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-13T10:06:54.982 INFO:teuthology.orchestra.run.smithi073.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-13T10:06:54.983 INFO:teuthology.orchestra.run.smithi073.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-13T10:06:54.983 INFO:teuthology.orchestra.run.smithi073.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-13T10:06:54.983 INFO:teuthology.orchestra.run.smithi073.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-13T10:06:54.983 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-13T10:06:54.983 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-13T10:06:54.983 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-13T10:06:54.983 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T10:06:54.983 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T10:06:54.983 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T10:06:54.983 INFO:teuthology.orchestra.run.smithi073.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T10:06:54.984 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T10:06:54.984 INFO:teuthology.orchestra.run.smithi073.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T10:06:54.984 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-13T10:06:54.984 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-13T10:06:54.984 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-13T10:06:54.984 INFO:teuthology.orchestra.run.smithi073.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-13T10:06:54.984 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-13T10:06:54.984 INFO:teuthology.orchestra.run.smithi073.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-13T10:06:54.984 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-13T10:06:54.984 INFO:teuthology.orchestra.run.smithi073.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-13T10:06:54.986 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2024-09-13T10:06:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T10:06:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T10:06:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-13T10:06:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-13T10:06:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-13T10:06:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-13T10:06:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-13T10:06:54.986 INFO:teuthology.orchestra.run.smithi195.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-13T10:06:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-13T10:06:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-13T10:06:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-13T10:06:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-13T10:06:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T10:06:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T10:06:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T10:06:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T10:06:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T10:06:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T10:06:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-13T10:06:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-13T10:06:54.987 INFO:teuthology.orchestra.run.smithi195.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-13T10:06:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-13T10:06:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-13T10:06:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-13T10:06:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-13T10:06:54.988 INFO:teuthology.orchestra.run.smithi195.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-13T10:06:54.989 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T10:06:54.989 INFO:teuthology.orchestra.run.smithi136.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T10:06:54.989 INFO:teuthology.orchestra.run.smithi136.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T10:06:54.989 INFO:teuthology.orchestra.run.smithi136.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T10:06:54.989 INFO:teuthology.orchestra.run.smithi136.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T10:06:54.990 INFO:teuthology.orchestra.run.smithi136.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T10:06:54.990 INFO:teuthology.orchestra.run.smithi136.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T10:06:54.990 INFO:teuthology.orchestra.run.smithi136.stdout:The following additional packages will be installed: 2024-09-13T10:06:54.990 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T10:06:54.990 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-13T10:06:54.990 INFO:teuthology.orchestra.run.smithi136.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-13T10:06:54.990 INFO:teuthology.orchestra.run.smithi136.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-13T10:06:54.991 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-13T10:06:54.991 INFO:teuthology.orchestra.run.smithi136.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-13T10:06:54.991 INFO:teuthology.orchestra.run.smithi136.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-13T10:06:54.991 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-13T10:06:54.992 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-13T10:06:54.992 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-13T10:06:54.992 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T10:06:54.992 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T10:06:54.992 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T10:06:54.992 INFO:teuthology.orchestra.run.smithi136.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T10:06:54.992 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T10:06:54.992 INFO:teuthology.orchestra.run.smithi136.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T10:06:54.992 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-09-13T10:06:54.992 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T10:06:54.992 INFO:teuthology.orchestra.run.smithi136.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T10:06:54.992 INFO:teuthology.orchestra.run.smithi136.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T10:06:54.992 INFO:teuthology.orchestra.run.smithi136.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T10:06:54.993 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T10:06:54.993 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-13T10:06:54.993 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet zip 2024-09-13T10:06:54.993 INFO:teuthology.orchestra.run.smithi136.stdout:Suggested packages: 2024-09-13T10:06:54.993 INFO:teuthology.orchestra.run.smithi136.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-13T10:06:54.993 INFO:teuthology.orchestra.run.smithi136.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-13T10:06:54.993 INFO:teuthology.orchestra.run.smithi136.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-13T10:06:54.993 INFO:teuthology.orchestra.run.smithi136.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-13T10:06:54.993 INFO:teuthology.orchestra.run.smithi136.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-13T10:06:54.993 INFO:teuthology.orchestra.run.smithi136.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T10:06:54.994 INFO:teuthology.orchestra.run.smithi136.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-13T10:06:54.994 INFO:teuthology.orchestra.run.smithi136.stdout:Recommended packages: 2024-09-13T10:06:54.994 INFO:teuthology.orchestra.run.smithi136.stdout: btrfs-tools 2024-09-13T10:06:54.999 INFO:teuthology.orchestra.run.smithi181.stdout:The following NEW packages will be installed: 2024-09-13T10:06:54.999 INFO:teuthology.orchestra.run.smithi181.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T10:06:54.999 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T10:06:55.001 INFO:teuthology.orchestra.run.smithi181.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-13T10:06:55.001 INFO:teuthology.orchestra.run.smithi181.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-13T10:06:55.001 INFO:teuthology.orchestra.run.smithi181.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-13T10:06:55.001 INFO:teuthology.orchestra.run.smithi181.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-13T10:06:55.001 INFO:teuthology.orchestra.run.smithi181.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-13T10:06:55.001 INFO:teuthology.orchestra.run.smithi181.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-13T10:06:55.001 INFO:teuthology.orchestra.run.smithi181.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-13T10:06:55.001 INFO:teuthology.orchestra.run.smithi181.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-13T10:06:55.001 INFO:teuthology.orchestra.run.smithi181.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-13T10:06:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-13T10:06:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T10:06:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T10:06:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T10:06:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T10:06:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T10:06:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T10:06:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-13T10:06:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-13T10:06:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-13T10:06:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-13T10:06:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-13T10:06:55.002 INFO:teuthology.orchestra.run.smithi181.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-13T10:06:55.003 INFO:teuthology.orchestra.run.smithi181.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-13T10:06:55.003 INFO:teuthology.orchestra.run.smithi181.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-13T10:06:55.045 INFO:teuthology.orchestra.run.smithi195.stdout:0 upgraded, 118 newly installed, 0 to remove and 331 not upgraded. 2024-09-13T10:06:55.045 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 251 MB of archives. 2024-09-13T10:06:55.046 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 933 MB of additional disk space will be used. 2024-09-13T10:06:55.046 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-13T10:06:55.096 INFO:teuthology.orchestra.run.smithi136.stdout:The following NEW packages will be installed: 2024-09-13T10:06:55.096 INFO:teuthology.orchestra.run.smithi136.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T10:06:55.096 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T10:06:55.096 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-13T10:06:55.096 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-13T10:06:55.096 INFO:teuthology.orchestra.run.smithi136.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-13T10:06:55.096 INFO:teuthology.orchestra.run.smithi136.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-13T10:06:55.097 INFO:teuthology.orchestra.run.smithi136.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-13T10:06:55.097 INFO:teuthology.orchestra.run.smithi136.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-13T10:06:55.097 INFO:teuthology.orchestra.run.smithi136.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-13T10:06:55.098 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-13T10:06:55.098 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-13T10:06:55.098 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-13T10:06:55.098 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T10:06:55.098 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T10:06:55.098 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T10:06:55.098 INFO:teuthology.orchestra.run.smithi136.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T10:06:55.098 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T10:06:55.098 INFO:teuthology.orchestra.run.smithi136.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T10:06:55.098 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-13T10:06:55.098 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-13T10:06:55.098 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-13T10:06:55.098 INFO:teuthology.orchestra.run.smithi136.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-13T10:06:55.098 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-13T10:06:55.099 INFO:teuthology.orchestra.run.smithi136.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-13T10:06:55.099 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-13T10:06:55.099 INFO:teuthology.orchestra.run.smithi136.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-13T10:06:55.126 INFO:teuthology.orchestra.run.smithi073.stdout:0 upgraded, 118 newly installed, 0 to remove and 331 not upgraded. 2024-09-13T10:06:55.126 INFO:teuthology.orchestra.run.smithi073.stdout:Need to get 251 MB of archives. 2024-09-13T10:06:55.127 INFO:teuthology.orchestra.run.smithi073.stdout:After this operation, 933 MB of additional disk space will be used. 2024-09-13T10:06:55.127 INFO:teuthology.orchestra.run.smithi073.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4697-gd58f7306-1jammy [3,631 kB] 2024-09-13T10:06:55.132 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-13T10:06:55.135 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-13T10:06:55.137 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4697-gd58f7306-1jammy [3,631 kB] 2024-09-13T10:06:55.139 INFO:teuthology.orchestra.run.smithi181.stdout:0 upgraded, 118 newly installed, 0 to remove and 331 not upgraded. 2024-09-13T10:06:55.140 INFO:teuthology.orchestra.run.smithi181.stdout:Need to get 251 MB of archives. 2024-09-13T10:06:55.140 INFO:teuthology.orchestra.run.smithi181.stdout:After this operation, 933 MB of additional disk space will be used. 2024-09-13T10:06:55.140 INFO:teuthology.orchestra.run.smithi181.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4697-gd58f7306-1jammy [3,631 kB] 2024-09-13T10:06:55.153 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-13T10:06:55.169 INFO:teuthology.orchestra.run.smithi073.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-13T10:06:55.190 INFO:teuthology.orchestra.run.smithi181.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-13T10:06:55.226 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-13T10:06:55.232 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-13T10:06:55.242 INFO:teuthology.orchestra.run.smithi136.stdout:0 upgraded, 118 newly installed, 0 to remove and 331 not upgraded. 2024-09-13T10:06:55.242 INFO:teuthology.orchestra.run.smithi136.stdout:Need to get 251 MB of archives. 2024-09-13T10:06:55.242 INFO:teuthology.orchestra.run.smithi136.stdout:After this operation, 933 MB of additional disk space will be used. 2024-09-13T10:06:55.242 INFO:teuthology.orchestra.run.smithi136.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4697-gd58f7306-1jammy [3,631 kB] 2024-09-13T10:06:55.244 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-13T10:06:55.249 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-13T10:06:55.249 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-13T10:06:55.250 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-13T10:06:55.259 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-13T10:06:55.262 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-13T10:06:55.264 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-13T10:06:55.265 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-13T10:06:55.265 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-13T10:06:55.281 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-13T10:06:55.284 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-13T10:06:55.285 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-13T10:06:55.286 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-13T10:06:55.286 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-13T10:06:55.287 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-13T10:06:55.298 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-13T10:06:55.298 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-13T10:06:55.298 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-13T10:06:55.299 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-13T10:06:55.299 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-13T10:06:55.315 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-13T10:06:55.315 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-13T10:06:55.318 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-13T10:06:55.318 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-13T10:06:55.319 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-13T10:06:55.332 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-13T10:06:55.333 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T10:06:55.333 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-13T10:06:55.333 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-13T10:06:55.335 INFO:teuthology.orchestra.run.smithi195.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-13T10:06:55.349 INFO:teuthology.orchestra.run.smithi195.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-13T10:06:55.349 INFO:teuthology.orchestra.run.smithi195.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-13T10:06:55.355 INFO:teuthology.orchestra.run.smithi195.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-13T10:06:55.355 INFO:teuthology.orchestra.run.smithi195.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-13T10:06:55.355 INFO:teuthology.orchestra.run.smithi195.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-13T10:06:55.366 INFO:teuthology.orchestra.run.smithi195.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-13T10:06:55.367 INFO:teuthology.orchestra.run.smithi195.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-13T10:06:55.369 INFO:teuthology.orchestra.run.smithi195.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-13T10:06:55.370 INFO:teuthology.orchestra.run.smithi195.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-13T10:06:55.371 INFO:teuthology.orchestra.run.smithi195.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-13T10:06:55.382 INFO:teuthology.orchestra.run.smithi195.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-13T10:06:55.385 INFO:teuthology.orchestra.run.smithi195.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-13T10:06:55.385 INFO:teuthology.orchestra.run.smithi195.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-13T10:06:55.386 INFO:teuthology.orchestra.run.smithi195.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-13T10:06:55.389 INFO:teuthology.orchestra.run.smithi195.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-13T10:06:55.423 INFO:teuthology.orchestra.run.smithi073.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4697-gd58f7306-1jammy [3,320 kB] 2024-09-13T10:06:55.441 INFO:teuthology.orchestra.run.smithi195.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-13T10:06:55.441 INFO:teuthology.orchestra.run.smithi195.stdout:Get:53 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4697-gd58f7306-1jammy [3,320 kB] 2024-09-13T10:06:55.441 INFO:teuthology.orchestra.run.smithi195.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-13T10:06:55.442 INFO:teuthology.orchestra.run.smithi195.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T10:06:55.443 INFO:teuthology.orchestra.run.smithi195.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-13T10:06:55.444 INFO:teuthology.orchestra.run.smithi195.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-13T10:06:55.444 INFO:teuthology.orchestra.run.smithi181.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4697-gd58f7306-1jammy [3,320 kB] 2024-09-13T10:06:55.461 INFO:teuthology.orchestra.run.smithi195.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-13T10:06:55.462 INFO:teuthology.orchestra.run.smithi195.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-13T10:06:55.462 INFO:teuthology.orchestra.run.smithi195.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-13T10:06:55.477 INFO:teuthology.orchestra.run.smithi195.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-13T10:06:55.477 INFO:teuthology.orchestra.run.smithi195.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-13T10:06:55.484 INFO:teuthology.orchestra.run.smithi195.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-13T10:06:55.485 INFO:teuthology.orchestra.run.smithi195.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-13T10:06:55.485 INFO:teuthology.orchestra.run.smithi195.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-13T10:06:55.486 INFO:teuthology.orchestra.run.smithi195.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-13T10:06:55.486 INFO:teuthology.orchestra.run.smithi195.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-13T10:06:55.486 INFO:teuthology.orchestra.run.smithi195.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-13T10:06:55.489 INFO:teuthology.orchestra.run.smithi195.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-13T10:06:55.501 INFO:teuthology.orchestra.run.smithi195.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-13T10:06:55.503 INFO:teuthology.orchestra.run.smithi195.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-13T10:06:55.503 INFO:teuthology.orchestra.run.smithi195.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-13T10:06:55.518 INFO:teuthology.orchestra.run.smithi195.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-13T10:06:55.520 INFO:teuthology.orchestra.run.smithi195.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-13T10:06:55.521 INFO:teuthology.orchestra.run.smithi195.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-13T10:06:55.521 INFO:teuthology.orchestra.run.smithi195.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-13T10:06:55.525 INFO:teuthology.orchestra.run.smithi195.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-13T10:06:55.526 INFO:teuthology.orchestra.run.smithi195.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-13T10:06:55.529 INFO:teuthology.orchestra.run.smithi195.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-13T10:06:55.535 INFO:teuthology.orchestra.run.smithi195.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T10:06:55.536 INFO:teuthology.orchestra.run.smithi195.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-13T10:06:55.536 INFO:teuthology.orchestra.run.smithi195.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-13T10:06:55.552 INFO:teuthology.orchestra.run.smithi195.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-13T10:06:55.553 INFO:teuthology.orchestra.run.smithi195.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-13T10:06:55.559 INFO:teuthology.orchestra.run.smithi195.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-13T10:06:55.559 INFO:teuthology.orchestra.run.smithi195.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-13T10:06:55.560 INFO:teuthology.orchestra.run.smithi195.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-13T10:06:55.561 INFO:teuthology.orchestra.run.smithi195.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-13T10:06:55.561 INFO:teuthology.orchestra.run.smithi195.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-13T10:06:55.573 INFO:teuthology.orchestra.run.smithi073.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4697-gd58f7306-1jammy [722 kB] 2024-09-13T10:06:55.592 INFO:teuthology.orchestra.run.smithi073.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-13T10:06:55.592 INFO:teuthology.orchestra.run.smithi073.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4697-gd58f7306-1jammy [356 kB] 2024-09-13T10:06:55.603 INFO:teuthology.orchestra.run.smithi073.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4697-gd58f7306-1jammy [32.8 kB] 2024-09-13T10:06:55.604 INFO:teuthology.orchestra.run.smithi073.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4697-gd58f7306-1jammy [180 kB] 2024-09-13T10:06:55.604 INFO:teuthology.orchestra.run.smithi073.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-13T10:06:55.608 INFO:teuthology.orchestra.run.smithi073.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4697-gd58f7306-1jammy [71.2 kB] 2024-09-13T10:06:55.613 INFO:teuthology.orchestra.run.smithi181.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-13T10:06:55.626 INFO:teuthology.orchestra.run.smithi181.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-13T10:06:55.630 INFO:teuthology.orchestra.run.smithi073.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4697-gd58f7306-1jammy [338 kB] 2024-09-13T10:06:55.686 INFO:teuthology.orchestra.run.smithi073.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-13T10:06:55.707 INFO:teuthology.orchestra.run.smithi181.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-13T10:06:55.710 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-13T10:06:55.711 INFO:teuthology.orchestra.run.smithi073.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4697-gd58f7306-1jammy [7,979 kB] 2024-09-13T10:06:55.723 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-13T10:06:55.804 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-13T10:06:55.914 INFO:teuthology.orchestra.run.smithi073.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-13T10:06:55.925 INFO:teuthology.orchestra.run.smithi073.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-13T10:06:55.936 INFO:teuthology.orchestra.run.smithi181.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-13T10:06:55.947 INFO:teuthology.orchestra.run.smithi181.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-13T10:06:55.950 INFO:teuthology.orchestra.run.smithi181.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4697-gd58f7306-1jammy [722 kB] 2024-09-13T10:06:55.954 INFO:teuthology.orchestra.run.smithi073.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-13T10:06:55.963 INFO:teuthology.orchestra.run.smithi195.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4697-gd58f7306-1jammy [722 kB] 2024-09-13T10:06:55.963 INFO:teuthology.orchestra.run.smithi073.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-13T10:06:55.964 INFO:teuthology.orchestra.run.smithi073.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-13T10:06:55.965 INFO:teuthology.orchestra.run.smithi073.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-13T10:06:55.978 INFO:teuthology.orchestra.run.smithi181.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-13T10:06:55.983 INFO:teuthology.orchestra.run.smithi073.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-13T10:06:55.987 INFO:teuthology.orchestra.run.smithi181.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-13T10:06:55.988 INFO:teuthology.orchestra.run.smithi181.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-13T10:06:55.988 INFO:teuthology.orchestra.run.smithi073.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-13T10:06:55.989 INFO:teuthology.orchestra.run.smithi181.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-13T10:06:55.992 INFO:teuthology.orchestra.run.smithi073.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-13T10:06:55.994 INFO:teuthology.orchestra.run.smithi181.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4697-gd58f7306-1jammy [356 kB] 2024-09-13T10:06:55.996 INFO:teuthology.orchestra.run.smithi073.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-13T10:06:56.005 INFO:teuthology.orchestra.run.smithi195.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4697-gd58f7306-1jammy [356 kB] 2024-09-13T10:06:56.007 INFO:teuthology.orchestra.run.smithi181.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-13T10:06:56.011 INFO:teuthology.orchestra.run.smithi181.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-13T10:06:56.015 INFO:teuthology.orchestra.run.smithi181.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-13T10:06:56.021 INFO:teuthology.orchestra.run.smithi181.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-13T10:06:56.027 INFO:teuthology.orchestra.run.smithi181.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4697-gd58f7306-1jammy [32.8 kB] 2024-09-13T10:06:56.029 INFO:teuthology.orchestra.run.smithi181.stdout:Get:20 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4697-gd58f7306-1jammy [180 kB] 2024-09-13T10:06:56.033 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-13T10:06:56.033 INFO:teuthology.orchestra.run.smithi181.stdout:Get:21 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4697-gd58f7306-1jammy [71.2 kB] 2024-09-13T10:06:56.035 INFO:teuthology.orchestra.run.smithi181.stdout:Get:22 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4697-gd58f7306-1jammy [338 kB] 2024-09-13T10:06:56.042 INFO:teuthology.orchestra.run.smithi195.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4697-gd58f7306-1jammy [32.8 kB] 2024-09-13T10:06:56.044 INFO:teuthology.orchestra.run.smithi195.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4697-gd58f7306-1jammy [180 kB] 2024-09-13T10:06:56.044 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-13T10:06:56.048 INFO:teuthology.orchestra.run.smithi195.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4697-gd58f7306-1jammy [71.2 kB] 2024-09-13T10:06:56.049 INFO:teuthology.orchestra.run.smithi195.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4697-gd58f7306-1jammy [338 kB] 2024-09-13T10:06:56.074 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-13T10:06:56.074 INFO:teuthology.orchestra.run.smithi195.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4697-gd58f7306-1jammy [7,979 kB] 2024-09-13T10:06:56.076 INFO:teuthology.orchestra.run.smithi073.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-13T10:06:56.081 INFO:teuthology.orchestra.run.smithi073.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-13T10:06:56.082 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-13T10:06:56.083 INFO:teuthology.orchestra.run.smithi136.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-13T10:06:56.084 INFO:teuthology.orchestra.run.smithi073.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-13T10:06:56.085 INFO:teuthology.orchestra.run.smithi073.stdout:Get:27 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4697-gd58f7306-1jammy [112 kB] 2024-09-13T10:06:56.085 INFO:teuthology.orchestra.run.smithi136.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-13T10:06:56.088 INFO:teuthology.orchestra.run.smithi073.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-13T10:06:56.089 INFO:teuthology.orchestra.run.smithi073.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-13T10:06:56.089 INFO:teuthology.orchestra.run.smithi073.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-13T10:06:56.091 INFO:teuthology.orchestra.run.smithi073.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-13T10:06:56.092 INFO:teuthology.orchestra.run.smithi073.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-13T10:06:56.100 INFO:teuthology.orchestra.run.smithi181.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-13T10:06:56.103 INFO:teuthology.orchestra.run.smithi136.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-13T10:06:56.104 INFO:teuthology.orchestra.run.smithi181.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-13T10:06:56.107 INFO:teuthology.orchestra.run.smithi136.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-13T10:06:56.108 INFO:teuthology.orchestra.run.smithi181.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-13T10:06:56.111 INFO:teuthology.orchestra.run.smithi136.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-13T10:06:56.112 INFO:teuthology.orchestra.run.smithi181.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-13T10:06:56.113 INFO:teuthology.orchestra.run.smithi181.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-13T10:06:56.114 INFO:teuthology.orchestra.run.smithi181.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-13T10:06:56.115 INFO:teuthology.orchestra.run.smithi136.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-13T10:06:56.116 INFO:teuthology.orchestra.run.smithi181.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-13T10:06:56.117 INFO:teuthology.orchestra.run.smithi181.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-13T10:06:56.117 INFO:teuthology.orchestra.run.smithi181.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-13T10:06:56.119 INFO:teuthology.orchestra.run.smithi073.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-13T10:06:56.119 INFO:teuthology.orchestra.run.smithi073.stdout:Get:34 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4697-gd58f7306-1jammy [473 kB] 2024-09-13T10:06:56.128 INFO:teuthology.orchestra.run.smithi073.stdout:Get:35 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4697-gd58f7306-1jammy [28.2 MB] 2024-09-13T10:06:56.159 INFO:teuthology.orchestra.run.smithi073.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-13T10:06:56.184 INFO:teuthology.orchestra.run.smithi181.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-13T10:06:56.196 INFO:teuthology.orchestra.run.smithi136.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-13T10:06:56.199 INFO:teuthology.orchestra.run.smithi136.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-13T10:06:56.200 INFO:teuthology.orchestra.run.smithi181.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-13T10:06:56.200 INFO:teuthology.orchestra.run.smithi181.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-13T10:06:56.201 INFO:teuthology.orchestra.run.smithi181.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-13T10:06:56.201 INFO:teuthology.orchestra.run.smithi073.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-13T10:06:56.202 INFO:teuthology.orchestra.run.smithi073.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-13T10:06:56.203 INFO:teuthology.orchestra.run.smithi073.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-13T10:06:56.204 INFO:teuthology.orchestra.run.smithi136.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-13T10:06:56.206 INFO:teuthology.orchestra.run.smithi181.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-13T10:06:56.207 INFO:teuthology.orchestra.run.smithi136.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-13T10:06:56.208 INFO:teuthology.orchestra.run.smithi181.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-13T10:06:56.208 INFO:teuthology.orchestra.run.smithi136.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-13T10:06:56.209 INFO:teuthology.orchestra.run.smithi073.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-13T10:06:56.209 INFO:teuthology.orchestra.run.smithi136.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-13T10:06:56.210 INFO:teuthology.orchestra.run.smithi073.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-13T10:06:56.211 INFO:teuthology.orchestra.run.smithi136.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-13T10:06:56.211 INFO:teuthology.orchestra.run.smithi136.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-13T10:06:56.212 INFO:teuthology.orchestra.run.smithi136.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-13T10:06:56.241 INFO:teuthology.orchestra.run.smithi073.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-13T10:06:56.242 INFO:teuthology.orchestra.run.smithi073.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-13T10:06:56.243 INFO:teuthology.orchestra.run.smithi073.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T10:06:56.244 INFO:teuthology.orchestra.run.smithi073.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-13T10:06:56.266 INFO:teuthology.orchestra.run.smithi181.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-13T10:06:56.267 INFO:teuthology.orchestra.run.smithi181.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-13T10:06:56.269 INFO:teuthology.orchestra.run.smithi181.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T10:06:56.269 INFO:teuthology.orchestra.run.smithi181.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-13T10:06:56.278 INFO:teuthology.orchestra.run.smithi136.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-13T10:06:56.295 INFO:teuthology.orchestra.run.smithi136.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-13T10:06:56.295 INFO:teuthology.orchestra.run.smithi136.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-13T10:06:56.296 INFO:teuthology.orchestra.run.smithi136.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-13T10:06:56.301 INFO:teuthology.orchestra.run.smithi136.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-13T10:06:56.302 INFO:teuthology.orchestra.run.smithi136.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-13T10:06:56.322 INFO:teuthology.orchestra.run.smithi073.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-13T10:06:56.324 INFO:teuthology.orchestra.run.smithi073.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-13T10:06:56.326 INFO:teuthology.orchestra.run.smithi073.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-13T10:06:56.327 INFO:teuthology.orchestra.run.smithi073.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-13T10:06:56.338 INFO:teuthology.orchestra.run.smithi073.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-13T10:06:56.347 INFO:teuthology.orchestra.run.smithi181.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-13T10:06:56.350 INFO:teuthology.orchestra.run.smithi181.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-13T10:06:56.351 INFO:teuthology.orchestra.run.smithi181.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-13T10:06:56.352 INFO:teuthology.orchestra.run.smithi181.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-13T10:06:56.360 INFO:teuthology.orchestra.run.smithi136.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-13T10:06:56.360 INFO:teuthology.orchestra.run.smithi136.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-13T10:06:56.363 INFO:teuthology.orchestra.run.smithi181.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-13T10:06:56.363 INFO:teuthology.orchestra.run.smithi136.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T10:06:56.363 INFO:teuthology.orchestra.run.smithi136.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-13T10:06:56.403 INFO:teuthology.orchestra.run.smithi073.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-13T10:06:56.404 INFO:teuthology.orchestra.run.smithi073.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-13T10:06:56.405 INFO:teuthology.orchestra.run.smithi073.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-13T10:06:56.408 INFO:teuthology.orchestra.run.smithi073.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-13T10:06:56.412 INFO:teuthology.orchestra.run.smithi073.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-13T10:06:56.429 INFO:teuthology.orchestra.run.smithi181.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-13T10:06:56.430 INFO:teuthology.orchestra.run.smithi181.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-13T10:06:56.431 INFO:teuthology.orchestra.run.smithi181.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-13T10:06:56.434 INFO:teuthology.orchestra.run.smithi181.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-13T10:06:56.438 INFO:teuthology.orchestra.run.smithi181.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-13T10:06:56.441 INFO:teuthology.orchestra.run.smithi136.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-13T10:06:56.444 INFO:teuthology.orchestra.run.smithi136.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-13T10:06:56.445 INFO:teuthology.orchestra.run.smithi136.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-13T10:06:56.446 INFO:teuthology.orchestra.run.smithi136.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-13T10:06:56.457 INFO:teuthology.orchestra.run.smithi136.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-13T10:06:56.485 INFO:teuthology.orchestra.run.smithi073.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-13T10:06:56.487 INFO:teuthology.orchestra.run.smithi073.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-13T10:06:56.489 INFO:teuthology.orchestra.run.smithi073.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-13T10:06:56.493 INFO:teuthology.orchestra.run.smithi073.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-13T10:06:56.494 INFO:teuthology.orchestra.run.smithi073.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-13T10:06:56.511 INFO:teuthology.orchestra.run.smithi181.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-13T10:06:56.513 INFO:teuthology.orchestra.run.smithi181.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-13T10:06:56.515 INFO:teuthology.orchestra.run.smithi181.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-13T10:06:56.519 INFO:teuthology.orchestra.run.smithi181.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-13T10:06:56.520 INFO:teuthology.orchestra.run.smithi181.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-13T10:06:56.522 INFO:teuthology.orchestra.run.smithi136.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-13T10:06:56.523 INFO:teuthology.orchestra.run.smithi136.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-13T10:06:56.524 INFO:teuthology.orchestra.run.smithi136.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-13T10:06:56.526 INFO:teuthology.orchestra.run.smithi136.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-13T10:06:56.531 INFO:teuthology.orchestra.run.smithi136.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-13T10:06:56.566 INFO:teuthology.orchestra.run.smithi073.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-13T10:06:56.571 INFO:teuthology.orchestra.run.smithi073.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-13T10:06:56.592 INFO:teuthology.orchestra.run.smithi181.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-13T10:06:56.597 INFO:teuthology.orchestra.run.smithi181.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-13T10:06:56.604 INFO:teuthology.orchestra.run.smithi136.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-13T10:06:56.606 INFO:teuthology.orchestra.run.smithi136.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-13T10:06:56.607 INFO:teuthology.orchestra.run.smithi136.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-13T10:06:56.612 INFO:teuthology.orchestra.run.smithi136.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-13T10:06:56.613 INFO:teuthology.orchestra.run.smithi136.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-13T10:06:56.685 INFO:teuthology.orchestra.run.smithi136.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-13T10:06:56.690 INFO:teuthology.orchestra.run.smithi136.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-13T10:06:56.734 INFO:teuthology.orchestra.run.smithi073.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-13T10:06:56.736 INFO:teuthology.orchestra.run.smithi073.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-13T10:06:56.738 INFO:teuthology.orchestra.run.smithi073.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T10:06:56.739 INFO:teuthology.orchestra.run.smithi073.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-13T10:06:56.741 INFO:teuthology.orchestra.run.smithi073.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-13T10:06:56.758 INFO:teuthology.orchestra.run.smithi181.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-13T10:06:56.760 INFO:teuthology.orchestra.run.smithi181.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-13T10:06:56.762 INFO:teuthology.orchestra.run.smithi181.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T10:06:56.763 INFO:teuthology.orchestra.run.smithi181.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-13T10:06:56.765 INFO:teuthology.orchestra.run.smithi181.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-13T10:06:56.776 INFO:teuthology.orchestra.run.smithi073.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-13T10:06:56.779 INFO:teuthology.orchestra.run.smithi073.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-13T10:06:56.780 INFO:teuthology.orchestra.run.smithi073.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-13T10:06:56.801 INFO:teuthology.orchestra.run.smithi181.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-13T10:06:56.804 INFO:teuthology.orchestra.run.smithi181.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-13T10:06:56.804 INFO:teuthology.orchestra.run.smithi181.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-13T10:06:56.836 INFO:teuthology.orchestra.run.smithi073.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-13T10:06:56.837 INFO:teuthology.orchestra.run.smithi073.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-13T10:06:56.853 INFO:teuthology.orchestra.run.smithi136.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-13T10:06:56.854 INFO:teuthology.orchestra.run.smithi136.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-13T10:06:56.857 INFO:teuthology.orchestra.run.smithi136.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T10:06:56.857 INFO:teuthology.orchestra.run.smithi136.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-13T10:06:56.860 INFO:teuthology.orchestra.run.smithi136.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-13T10:06:56.861 INFO:teuthology.orchestra.run.smithi181.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-13T10:06:56.861 INFO:teuthology.orchestra.run.smithi181.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-13T10:06:56.897 INFO:teuthology.orchestra.run.smithi136.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-13T10:06:56.900 INFO:teuthology.orchestra.run.smithi136.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-13T10:06:56.900 INFO:teuthology.orchestra.run.smithi136.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-13T10:06:56.900 INFO:teuthology.orchestra.run.smithi073.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-13T10:06:56.902 INFO:teuthology.orchestra.run.smithi073.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-13T10:06:56.902 INFO:teuthology.orchestra.run.smithi073.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-13T10:06:56.904 INFO:teuthology.orchestra.run.smithi073.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-13T10:06:56.904 INFO:teuthology.orchestra.run.smithi073.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-13T10:06:56.905 INFO:teuthology.orchestra.run.smithi073.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-13T10:06:56.912 INFO:teuthology.orchestra.run.smithi073.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-13T10:06:56.925 INFO:teuthology.orchestra.run.smithi181.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-13T10:06:56.926 INFO:teuthology.orchestra.run.smithi181.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-13T10:06:56.927 INFO:teuthology.orchestra.run.smithi181.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-13T10:06:56.928 INFO:teuthology.orchestra.run.smithi181.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-13T10:06:56.928 INFO:teuthology.orchestra.run.smithi181.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-13T10:06:56.930 INFO:teuthology.orchestra.run.smithi181.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-13T10:06:56.936 INFO:teuthology.orchestra.run.smithi181.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-13T10:06:56.957 INFO:teuthology.orchestra.run.smithi136.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-13T10:06:56.957 INFO:teuthology.orchestra.run.smithi136.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-13T10:06:56.982 INFO:teuthology.orchestra.run.smithi073.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-13T10:06:56.991 INFO:teuthology.orchestra.run.smithi073.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-13T10:06:56.993 INFO:teuthology.orchestra.run.smithi073.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-13T10:06:57.006 INFO:teuthology.orchestra.run.smithi181.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-13T10:06:57.010 INFO:teuthology.orchestra.run.smithi181.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-13T10:06:57.011 INFO:teuthology.orchestra.run.smithi181.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-13T10:06:57.019 INFO:teuthology.orchestra.run.smithi136.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-13T10:06:57.020 INFO:teuthology.orchestra.run.smithi136.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-13T10:06:57.020 INFO:teuthology.orchestra.run.smithi136.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-13T10:06:57.021 INFO:teuthology.orchestra.run.smithi136.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-13T10:06:57.022 INFO:teuthology.orchestra.run.smithi136.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-13T10:06:57.023 INFO:teuthology.orchestra.run.smithi136.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-13T10:06:57.030 INFO:teuthology.orchestra.run.smithi136.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-13T10:06:57.057 INFO:teuthology.orchestra.run.smithi073.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4697-gd58f7306-1jammy [5,206 kB] 2024-09-13T10:06:57.063 INFO:teuthology.orchestra.run.smithi073.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-13T10:06:57.072 INFO:teuthology.orchestra.run.smithi073.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-13T10:06:57.072 INFO:teuthology.orchestra.run.smithi073.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-13T10:06:57.073 INFO:teuthology.orchestra.run.smithi073.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-13T10:06:57.084 INFO:teuthology.orchestra.run.smithi073.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-13T10:06:57.086 INFO:teuthology.orchestra.run.smithi073.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-13T10:06:57.088 INFO:teuthology.orchestra.run.smithi181.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-13T10:06:57.094 INFO:teuthology.orchestra.run.smithi073.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-13T10:06:57.094 INFO:teuthology.orchestra.run.smithi181.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-13T10:06:57.096 INFO:teuthology.orchestra.run.smithi181.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-13T10:06:57.097 INFO:teuthology.orchestra.run.smithi181.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-13T10:06:57.100 INFO:teuthology.orchestra.run.smithi136.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-13T10:06:57.104 INFO:teuthology.orchestra.run.smithi136.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-13T10:06:57.105 INFO:teuthology.orchestra.run.smithi136.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-13T10:06:57.108 INFO:teuthology.orchestra.run.smithi181.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-13T10:06:57.110 INFO:teuthology.orchestra.run.smithi181.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-13T10:06:57.117 INFO:teuthology.orchestra.run.smithi181.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-13T10:06:57.121 INFO:teuthology.orchestra.run.smithi073.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4697-gd58f7306-1jammy [251 kB] 2024-09-13T10:06:57.124 INFO:teuthology.orchestra.run.smithi073.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4697-gd58f7306-1jammy [124 kB] 2024-09-13T10:06:57.125 INFO:teuthology.orchestra.run.smithi073.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4697-gd58f7306-1jammy [1,561 kB] 2024-09-13T10:06:57.144 INFO:teuthology.orchestra.run.smithi073.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T10:06:57.146 INFO:teuthology.orchestra.run.smithi073.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-13T10:06:57.147 INFO:teuthology.orchestra.run.smithi073.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-13T10:06:57.162 INFO:teuthology.orchestra.run.smithi073.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4697-gd58f7306-1jammy [6,309 kB] 2024-09-13T10:06:57.169 INFO:teuthology.orchestra.run.smithi181.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T10:06:57.171 INFO:teuthology.orchestra.run.smithi181.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-13T10:06:57.172 INFO:teuthology.orchestra.run.smithi181.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-13T10:06:57.182 INFO:teuthology.orchestra.run.smithi136.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-13T10:06:57.188 INFO:teuthology.orchestra.run.smithi136.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-13T10:06:57.190 INFO:teuthology.orchestra.run.smithi136.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-13T10:06:57.192 INFO:teuthology.orchestra.run.smithi136.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-13T10:06:57.203 INFO:teuthology.orchestra.run.smithi136.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-13T10:06:57.204 INFO:teuthology.orchestra.run.smithi136.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-13T10:06:57.211 INFO:teuthology.orchestra.run.smithi136.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-13T10:06:57.226 INFO:teuthology.orchestra.run.smithi073.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-13T10:06:57.228 INFO:teuthology.orchestra.run.smithi073.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-13T10:06:57.245 INFO:teuthology.orchestra.run.smithi073.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-13T10:06:57.246 INFO:teuthology.orchestra.run.smithi073.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-13T10:06:57.247 INFO:teuthology.orchestra.run.smithi073.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-13T10:06:57.251 INFO:teuthology.orchestra.run.smithi181.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-13T10:06:57.252 INFO:teuthology.orchestra.run.smithi073.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-13T10:06:57.253 INFO:teuthology.orchestra.run.smithi181.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-13T10:06:57.253 INFO:teuthology.orchestra.run.smithi073.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-13T10:06:57.263 INFO:teuthology.orchestra.run.smithi136.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T10:06:57.265 INFO:teuthology.orchestra.run.smithi136.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-13T10:06:57.265 INFO:teuthology.orchestra.run.smithi136.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-13T10:06:57.269 INFO:teuthology.orchestra.run.smithi181.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-13T10:06:57.270 INFO:teuthology.orchestra.run.smithi181.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-13T10:06:57.271 INFO:teuthology.orchestra.run.smithi181.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-13T10:06:57.276 INFO:teuthology.orchestra.run.smithi181.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-13T10:06:57.278 INFO:teuthology.orchestra.run.smithi181.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-13T10:06:57.345 INFO:teuthology.orchestra.run.smithi136.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-13T10:06:57.346 INFO:teuthology.orchestra.run.smithi136.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-13T10:06:57.363 INFO:teuthology.orchestra.run.smithi136.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-13T10:06:57.364 INFO:teuthology.orchestra.run.smithi136.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-13T10:06:57.365 INFO:teuthology.orchestra.run.smithi136.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-13T10:06:57.370 INFO:teuthology.orchestra.run.smithi136.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-13T10:06:57.371 INFO:teuthology.orchestra.run.smithi136.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-13T10:06:57.573 INFO:teuthology.orchestra.run.smithi073.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4697-gd58f7306-1jammy [23.1 MB] 2024-09-13T10:06:58.281 INFO:teuthology.orchestra.run.smithi073.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4697-gd58f7306-1jammy [14.1 kB] 2024-09-13T10:06:58.282 INFO:teuthology.orchestra.run.smithi073.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4697-gd58f7306-1jammy [880 kB] 2024-09-13T10:06:58.299 INFO:teuthology.orchestra.run.smithi073.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4697-gd58f7306-1jammy [2,433 kB] 2024-09-13T10:06:58.341 INFO:teuthology.orchestra.run.smithi073.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4697-gd58f7306-1jammy [812 kB] 2024-09-13T10:06:58.349 INFO:teuthology.orchestra.run.smithi073.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4697-gd58f7306-1jammy [160 kB] 2024-09-13T10:06:58.351 INFO:teuthology.orchestra.run.smithi073.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4697-gd58f7306-1jammy [61.4 MB] 2024-09-13T10:06:58.357 INFO:teuthology.orchestra.run.smithi136.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4697-gd58f7306-1jammy [3,320 kB] 2024-09-13T10:06:58.409 INFO:teuthology.orchestra.run.smithi181.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4697-gd58f7306-1jammy [7,979 kB] 2024-09-13T10:07:00.777 INFO:teuthology.orchestra.run.smithi073.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4697-gd58f7306-1jammy [8,625 kB] 2024-09-13T10:07:00.912 INFO:teuthology.orchestra.run.smithi073.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4697-gd58f7306-1jammy [14.2 kB] 2024-09-13T10:07:00.932 INFO:teuthology.orchestra.run.smithi073.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4697-gd58f7306-1jammy [55.7 MB] 2024-09-13T10:07:03.025 INFO:teuthology.orchestra.run.smithi073.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4697-gd58f7306-1jammy [130 kB] 2024-09-13T10:07:03.028 INFO:teuthology.orchestra.run.smithi073.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4697-gd58f7306-1jammy [38.9 kB] 2024-09-13T10:07:03.029 INFO:teuthology.orchestra.run.smithi073.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4697-gd58f7306-1jammy [15.9 MB] 2024-09-13T10:07:03.819 INFO:teuthology.orchestra.run.smithi073.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4697-gd58f7306-1jammy [91.3 kB] 2024-09-13T10:07:03.848 INFO:teuthology.orchestra.run.smithi136.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4697-gd58f7306-1jammy [722 kB] 2024-09-13T10:07:03.913 INFO:teuthology.orchestra.run.smithi136.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4697-gd58f7306-1jammy [356 kB] 2024-09-13T10:07:03.918 INFO:teuthology.orchestra.run.smithi181.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4697-gd58f7306-1jammy [112 kB] 2024-09-13T10:07:03.921 INFO:teuthology.orchestra.run.smithi181.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4697-gd58f7306-1jammy [473 kB] 2024-09-13T10:07:03.931 INFO:teuthology.orchestra.run.smithi181.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4697-gd58f7306-1jammy [28.2 MB] 2024-09-13T10:07:03.934 INFO:teuthology.orchestra.run.smithi195.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4697-gd58f7306-1jammy [112 kB] 2024-09-13T10:07:03.937 INFO:teuthology.orchestra.run.smithi195.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4697-gd58f7306-1jammy [473 kB] 2024-09-13T10:07:03.944 INFO:teuthology.orchestra.run.smithi136.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4697-gd58f7306-1jammy [32.8 kB] 2024-09-13T10:07:03.945 INFO:teuthology.orchestra.run.smithi136.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4697-gd58f7306-1jammy [180 kB] 2024-09-13T10:07:03.946 INFO:teuthology.orchestra.run.smithi195.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4697-gd58f7306-1jammy [28.2 MB] 2024-09-13T10:07:03.976 INFO:teuthology.orchestra.run.smithi136.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4697-gd58f7306-1jammy [71.2 kB] 2024-09-13T10:07:03.978 INFO:teuthology.orchestra.run.smithi136.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4697-gd58f7306-1jammy [338 kB] 2024-09-13T10:07:04.013 INFO:teuthology.orchestra.run.smithi136.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4697-gd58f7306-1jammy [7,979 kB] 2024-09-13T10:07:04.393 INFO:teuthology.orchestra.run.smithi073.stdout:Fetched 251 MB in 9s (28.4 MB/s) 2024-09-13T10:07:04.553 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-13T10:07:04.652 INFO:teuthology.orchestra.run.smithi073.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-13T10:07:04.658 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-13T10:07:04.751 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T10:07:04.878 INFO:teuthology.orchestra.run.smithi136.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4697-gd58f7306-1jammy [112 kB] 2024-09-13T10:07:04.889 INFO:teuthology.orchestra.run.smithi136.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4697-gd58f7306-1jammy [473 kB] 2024-09-13T10:07:04.903 INFO:teuthology.orchestra.run.smithi136.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4697-gd58f7306-1jammy [28.2 MB] 2024-09-13T10:07:05.063 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T10:07:05.089 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-13T10:07:05.129 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T10:07:05.176 INFO:teuthology.orchestra.run.smithi195.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4697-gd58f7306-1jammy [5,206 kB] 2024-09-13T10:07:05.276 INFO:teuthology.orchestra.run.smithi195.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4697-gd58f7306-1jammy [251 kB] 2024-09-13T10:07:05.287 INFO:teuthology.orchestra.run.smithi195.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4697-gd58f7306-1jammy [124 kB] 2024-09-13T10:07:05.290 INFO:teuthology.orchestra.run.smithi195.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4697-gd58f7306-1jammy [1,561 kB] 2024-09-13T10:07:05.334 INFO:teuthology.orchestra.run.smithi195.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4697-gd58f7306-1jammy [6,309 kB] 2024-09-13T10:07:05.365 INFO:teuthology.orchestra.run.smithi181.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4697-gd58f7306-1jammy [5,206 kB] 2024-09-13T10:07:05.441 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T10:07:05.444 INFO:teuthology.orchestra.run.smithi181.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4697-gd58f7306-1jammy [251 kB] 2024-09-13T10:07:05.446 INFO:teuthology.orchestra.run.smithi181.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4697-gd58f7306-1jammy [124 kB] 2024-09-13T10:07:05.453 INFO:teuthology.orchestra.run.smithi181.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4697-gd58f7306-1jammy [1,561 kB] 2024-09-13T10:07:05.468 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-13T10:07:05.505 INFO:teuthology.orchestra.run.smithi181.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4697-gd58f7306-1jammy [6,309 kB] 2024-09-13T10:07:05.516 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T10:07:05.605 INFO:teuthology.orchestra.run.smithi195.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4697-gd58f7306-1jammy [23.1 MB] 2024-09-13T10:07:05.749 INFO:teuthology.orchestra.run.smithi181.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4697-gd58f7306-1jammy [23.1 MB] 2024-09-13T10:07:05.953 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T10:07:05.980 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T10:07:06.028 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:07:06.373 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T10:07:06.399 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T10:07:06.431 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:07:06.741 INFO:teuthology.orchestra.run.smithi181.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4697-gd58f7306-1jammy [14.1 kB] 2024-09-13T10:07:06.742 INFO:teuthology.orchestra.run.smithi181.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4697-gd58f7306-1jammy [880 kB] 2024-09-13T10:07:06.759 INFO:teuthology.orchestra.run.smithi181.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4697-gd58f7306-1jammy [2,433 kB] 2024-09-13T10:07:06.768 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T10:07:06.794 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T10:07:06.799 INFO:teuthology.orchestra.run.smithi136.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4697-gd58f7306-1jammy [5,206 kB] 2024-09-13T10:07:06.800 INFO:teuthology.orchestra.run.smithi181.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4697-gd58f7306-1jammy [812 kB] 2024-09-13T10:07:06.810 INFO:teuthology.orchestra.run.smithi181.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4697-gd58f7306-1jammy [160 kB] 2024-09-13T10:07:06.817 INFO:teuthology.orchestra.run.smithi181.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4697-gd58f7306-1jammy [61.4 MB] 2024-09-13T10:07:06.824 INFO:teuthology.orchestra.run.smithi195.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4697-gd58f7306-1jammy [14.1 kB] 2024-09-13T10:07:06.825 INFO:teuthology.orchestra.run.smithi195.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4697-gd58f7306-1jammy [880 kB] 2024-09-13T10:07:06.834 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:07:06.842 INFO:teuthology.orchestra.run.smithi195.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4697-gd58f7306-1jammy [2,433 kB] 2024-09-13T10:07:06.884 INFO:teuthology.orchestra.run.smithi195.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4697-gd58f7306-1jammy [812 kB] 2024-09-13T10:07:06.923 INFO:teuthology.orchestra.run.smithi195.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4697-gd58f7306-1jammy [160 kB] 2024-09-13T10:07:06.925 INFO:teuthology.orchestra.run.smithi195.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4697-gd58f7306-1jammy [61.4 MB] 2024-09-13T10:07:07.058 INFO:teuthology.orchestra.run.smithi136.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4697-gd58f7306-1jammy [251 kB] 2024-09-13T10:07:07.101 INFO:teuthology.orchestra.run.smithi136.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4697-gd58f7306-1jammy [124 kB] 2024-09-13T10:07:07.104 INFO:teuthology.orchestra.run.smithi136.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4697-gd58f7306-1jammy [1,561 kB] 2024-09-13T10:07:07.171 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-13T10:07:07.197 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-13T10:07:07.216 INFO:teuthology.orchestra.run.smithi136.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4697-gd58f7306-1jammy [6,309 kB] 2024-09-13T10:07:07.237 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T10:07:07.557 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package librados2. 2024-09-13T10:07:07.584 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../007-librados2_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:07.640 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking librados2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:07.723 INFO:teuthology.orchestra.run.smithi136.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4697-gd58f7306-1jammy [23.1 MB] 2024-09-13T10:07:08.085 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package librbd1. 2024-09-13T10:07:08.112 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../008-librbd1_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:08.143 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking librbd1 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:08.596 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T10:07:08.623 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../009-libcephfs2_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:08.663 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libcephfs2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:08.923 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-rados. 2024-09-13T10:07:08.940 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../010-python3-rados_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:08.982 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-rados (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:09.285 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T10:07:09.312 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:09.351 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-ceph-argparse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:09.518 INFO:teuthology.orchestra.run.smithi181.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4697-gd58f7306-1jammy [8,625 kB] 2024-09-13T10:07:09.545 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T10:07:09.561 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../012-python3-cephfs_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:09.604 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-cephfs (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:09.605 INFO:teuthology.orchestra.run.smithi136.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4697-gd58f7306-1jammy [14.1 kB] 2024-09-13T10:07:09.606 INFO:teuthology.orchestra.run.smithi136.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4697-gd58f7306-1jammy [880 kB] 2024-09-13T10:07:09.623 INFO:teuthology.orchestra.run.smithi136.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4697-gd58f7306-1jammy [2,433 kB] 2024-09-13T10:07:09.683 INFO:teuthology.orchestra.run.smithi181.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4697-gd58f7306-1jammy [14.2 kB] 2024-09-13T10:07:09.683 INFO:teuthology.orchestra.run.smithi181.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4697-gd58f7306-1jammy [55.7 MB] 2024-09-13T10:07:09.744 INFO:teuthology.orchestra.run.smithi136.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4697-gd58f7306-1jammy [812 kB] 2024-09-13T10:07:09.768 INFO:teuthology.orchestra.run.smithi195.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4697-gd58f7306-1jammy [8,625 kB] 2024-09-13T10:07:09.787 INFO:teuthology.orchestra.run.smithi136.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4697-gd58f7306-1jammy [160 kB] 2024-09-13T10:07:09.790 INFO:teuthology.orchestra.run.smithi136.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4697-gd58f7306-1jammy [61.4 MB] 2024-09-13T10:07:10.014 INFO:teuthology.orchestra.run.smithi195.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4697-gd58f7306-1jammy [14.2 kB] 2024-09-13T10:07:10.014 INFO:teuthology.orchestra.run.smithi195.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4697-gd58f7306-1jammy [55.7 MB] 2024-09-13T10:07:10.099 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T10:07:10.125 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../013-python3-ceph-common_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:10.168 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-ceph-common (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:10.452 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T10:07:10.467 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-13T10:07:10.502 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T10:07:10.797 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T10:07:10.823 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-13T10:07:10.863 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-13T10:07:11.124 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T10:07:11.139 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../016-python3-rbd_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:11.182 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-rbd (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:11.502 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T10:07:11.529 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-13T10:07:11.569 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T10:07:11.846 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-13T10:07:11.859 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-13T10:07:11.896 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-13T10:07:12.002 INFO:teuthology.orchestra.run.smithi181.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4697-gd58f7306-1jammy [130 kB] 2024-09-13T10:07:12.004 INFO:teuthology.orchestra.run.smithi181.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4697-gd58f7306-1jammy [38.9 kB] 2024-09-13T10:07:12.005 INFO:teuthology.orchestra.run.smithi181.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4697-gd58f7306-1jammy [15.9 MB] 2024-09-13T10:07:12.208 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-13T10:07:12.223 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-13T10:07:12.266 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-13T10:07:12.521 INFO:teuthology.orchestra.run.smithi195.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4697-gd58f7306-1jammy [130 kB] 2024-09-13T10:07:12.523 INFO:teuthology.orchestra.run.smithi181.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4697-gd58f7306-1jammy [91.3 kB] 2024-09-13T10:07:12.524 INFO:teuthology.orchestra.run.smithi195.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4697-gd58f7306-1jammy [38.9 kB] 2024-09-13T10:07:12.525 INFO:teuthology.orchestra.run.smithi195.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4697-gd58f7306-1jammy [15.9 MB] 2024-09-13T10:07:12.587 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package lua5.1. 2024-09-13T10:07:12.612 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-13T10:07:12.653 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-13T10:07:12.916 INFO:teuthology.orchestra.run.smithi195.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4697-gd58f7306-1jammy [91.3 kB] 2024-09-13T10:07:12.931 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package lua-any. 2024-09-13T10:07:12.946 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-13T10:07:12.989 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-13T10:07:13.085 INFO:teuthology.orchestra.run.smithi181.stdout:Fetched 251 MB in 18s (14.3 MB/s) 2024-09-13T10:07:13.222 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-13T10:07:13.233 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package zip. 2024-09-13T10:07:13.248 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-13T10:07:13.300 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking zip (3.0-12build2) ... 2024-09-13T10:07:13.321 INFO:teuthology.orchestra.run.smithi181.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-13T10:07:13.327 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-13T10:07:13.396 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T10:07:13.474 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 251 MB in 18s (14.0 MB/s) 2024-09-13T10:07:13.604 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-13T10:07:13.612 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package unzip. 2024-09-13T10:07:13.638 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-13T10:07:13.678 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-13T10:07:13.702 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-13T10:07:13.707 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-13T10:07:13.716 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T10:07:13.743 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-13T10:07:13.758 INFO:teuthology.orchestra.run.smithi136.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4697-gd58f7306-1jammy [8,625 kB] 2024-09-13T10:07:13.782 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T10:07:13.794 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T10:07:13.906 INFO:teuthology.orchestra.run.smithi136.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4697-gd58f7306-1jammy [14.2 kB] 2024-09-13T10:07:13.906 INFO:teuthology.orchestra.run.smithi136.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4697-gd58f7306-1jammy [55.7 MB] 2024-09-13T10:07:14.016 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package luarocks. 2024-09-13T10:07:14.042 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-13T10:07:14.074 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-13T10:07:14.136 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T10:07:14.162 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-13T10:07:14.202 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T10:07:14.273 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T10:07:14.299 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-13T10:07:14.331 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T10:07:14.432 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package librgw2. 2024-09-13T10:07:14.458 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../025-librgw2_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:14.498 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking librgw2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:14.573 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T10:07:14.599 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T10:07:14.639 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:07:14.667 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T10:07:14.694 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-13T10:07:14.734 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T10:07:15.034 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T10:07:15.061 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T10:07:15.100 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:07:15.109 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T10:07:15.125 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../026-python3-rgw_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:15.179 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T10:07:15.206 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T10:07:15.253 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:07:15.310 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-rgw (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:15.445 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T10:07:15.472 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T10:07:15.512 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:07:15.604 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-13T10:07:15.611 INFO:teuthology.orchestra.run.smithi136.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4697-gd58f7306-1jammy [130 kB] 2024-09-13T10:07:15.614 INFO:teuthology.orchestra.run.smithi136.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4697-gd58f7306-1jammy [38.9 kB] 2024-09-13T10:07:15.614 INFO:teuthology.orchestra.run.smithi136.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4697-gd58f7306-1jammy [15.9 MB] 2024-09-13T10:07:15.624 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T10:07:15.631 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-13T10:07:15.639 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T10:07:15.674 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:07:15.687 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T10:07:15.982 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T10:07:16.009 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../028-libradosstriper1_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:16.010 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T10:07:16.023 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-13T10:07:16.032 INFO:teuthology.orchestra.run.smithi136.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-aclamk-cbt-combined/d58f73069a8157a9ace12a0247ee3e8497707ea9/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4697-gd58f7306-1jammy [91.3 kB] 2024-09-13T10:07:16.036 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T10:07:16.039 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-13T10:07:16.049 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libradosstriper1 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:16.073 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T10:07:16.076 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:07:16.376 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package librados2. 2024-09-13T10:07:16.402 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../007-librados2_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:16.438 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-13T10:07:16.443 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking librados2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:16.452 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-common. 2024-09-13T10:07:16.464 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-13T10:07:16.478 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../029-ceph-common_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:16.505 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T10:07:16.519 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-common (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:16.598 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 251 MB in 21s (12.0 MB/s) 2024-09-13T10:07:16.731 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-13T10:07:16.799 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librados2. 2024-09-13T10:07:16.826 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../007-librados2_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:16.830 INFO:teuthology.orchestra.run.smithi136.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-13T10:07:16.835 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-13T10:07:16.857 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librados2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:16.896 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package librbd1. 2024-09-13T10:07:16.904 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T10:07:16.922 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../008-librbd1_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:16.962 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking librbd1 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:17.249 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T10:07:17.276 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-13T10:07:17.285 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librbd1. 2024-09-13T10:07:17.311 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../008-librbd1_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:17.316 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T10:07:17.352 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librbd1 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:17.381 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T10:07:17.408 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../009-libcephfs2_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:17.440 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libcephfs2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:17.653 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T10:07:17.679 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-13T10:07:17.692 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-rados. 2024-09-13T10:07:17.708 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../010-python3-rados_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:17.719 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T10:07:17.742 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-rados (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:17.821 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T10:07:17.848 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../009-libcephfs2_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:17.888 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcephfs2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:18.024 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-base. 2024-09-13T10:07:18.050 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../030-ceph-base_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:18.053 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T10:07:18.080 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:18.089 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T10:07:18.099 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-base (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:18.116 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T10:07:18.120 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-ceph-argparse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:18.155 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:07:18.182 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rados. 2024-09-13T10:07:18.197 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../010-python3-rados_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:18.240 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rados (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:18.372 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T10:07:18.387 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../012-python3-cephfs_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:18.430 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-cephfs (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:18.535 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T10:07:18.559 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T10:07:18.561 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:18.586 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T10:07:18.593 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-ceph-argparse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:18.626 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:07:18.687 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-13T10:07:18.713 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-13T10:07:18.750 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T10:07:18.753 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-13T10:07:18.777 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../013-python3-ceph-common_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:18.816 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-ceph-common (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:18.829 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T10:07:18.852 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../012-python3-cephfs_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:18.895 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cephfs (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:18.962 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T10:07:18.988 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T10:07:19.029 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:07:19.057 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-cheroot. 2024-09-13T10:07:19.083 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-13T10:07:19.123 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T10:07:19.153 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T10:07:19.179 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-13T10:07:19.215 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T10:07:19.219 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T10:07:19.242 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../013-python3-ceph-common_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:19.284 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-ceph-common (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:19.365 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-13T10:07:19.391 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-13T10:07:19.410 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-13T10:07:19.423 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T10:07:19.436 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-13T10:07:19.492 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-13T10:07:19.539 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T10:07:19.565 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-13T10:07:19.605 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-13T10:07:19.743 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librados2. 2024-09-13T10:07:19.751 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T10:07:19.769 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../007-librados2_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:19.777 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-13T10:07:19.779 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-13T10:07:19.806 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-13T10:07:19.809 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librados2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:19.818 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T10:07:19.833 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T10:07:19.846 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-13T10:07:19.859 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../016-python3-rbd_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:19.891 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-rbd (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:20.087 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T10:07:20.113 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-13T10:07:20.145 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-13T10:07:20.149 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-13T10:07:20.176 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-13T10:07:20.215 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-13T10:07:20.236 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T10:07:20.262 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librbd1. 2024-09-13T10:07:20.263 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-13T10:07:20.288 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../008-librbd1_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:20.302 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T10:07:20.329 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librbd1 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:20.398 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T10:07:20.413 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../016-python3-rbd_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:20.457 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rbd (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:20.518 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-tempora. 2024-09-13T10:07:20.545 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-13T10:07:20.580 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-13T10:07:20.585 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-13T10:07:20.607 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-13T10:07:20.647 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-13T10:07:20.740 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T10:07:20.766 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../009-libcephfs2_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:20.793 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T10:07:20.806 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libcephfs2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:20.819 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-13T10:07:20.859 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T10:07:20.888 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-portend. 2024-09-13T10:07:20.915 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-13T10:07:20.954 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-13T10:07:21.062 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rados. 2024-09-13T10:07:21.085 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-13T10:07:21.087 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../010-python3-rados_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:21.111 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-13T10:07:21.128 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rados (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:21.145 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-13T10:07:21.158 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-13T10:07:21.160 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-13T10:07:21.203 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-13T10:07:21.207 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-13T10:07:21.231 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-13T10:07:21.265 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-13T10:07:21.462 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package lua5.1. 2024-09-13T10:07:21.488 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-13T10:07:21.490 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-13T10:07:21.505 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-13T10:07:21.528 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-13T10:07:21.544 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T10:07:21.556 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-13T10:07:21.571 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-13T10:07:21.597 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T10:07:21.610 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-13T10:07:21.623 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:21.664 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-ceph-argparse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:21.832 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package lua-any. 2024-09-13T10:07:21.846 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-13T10:07:21.889 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-13T10:07:21.899 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T10:07:21.915 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../012-python3-cephfs_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:21.939 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T10:07:21.943 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package lua5.1. 2024-09-13T10:07:21.949 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cephfs (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:21.962 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-13T10:07:21.969 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-13T10:07:21.997 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-13T10:07:22.018 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-13T10:07:22.142 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package zip. 2024-09-13T10:07:22.168 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-13T10:07:22.208 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking zip (3.0-12build2) ... 2024-09-13T10:07:22.227 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T10:07:22.254 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../013-python3-ceph-common_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:22.267 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T10:07:22.293 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-13T10:07:22.294 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-ceph-common (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:22.334 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-13T10:07:22.346 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package lua-any. 2024-09-13T10:07:22.372 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-13T10:07:22.413 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-13T10:07:22.503 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package unzip. 2024-09-13T10:07:22.530 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-13T10:07:22.570 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-13T10:07:22.589 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T10:07:22.615 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-13T10:07:22.620 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T10:07:22.646 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-13T10:07:22.655 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T10:07:22.687 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-13T10:07:22.692 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package zip. 2024-09-13T10:07:22.707 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-13T10:07:22.759 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking zip (3.0-12build2) ... 2024-09-13T10:07:22.890 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package luarocks. 2024-09-13T10:07:22.917 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-13T10:07:22.948 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-13T10:07:22.967 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T10:07:22.990 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-mako. 2024-09-13T10:07:22.993 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-09-13T10:07:23.016 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-13T10:07:23.033 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-13T10:07:23.056 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T10:07:23.096 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package unzip. 2024-09-13T10:07:23.122 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-13T10:07:23.170 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-13T10:07:23.290 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package librgw2. 2024-09-13T10:07:23.294 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T10:07:23.305 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../025-librgw2_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:23.309 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../016-python3-rbd_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:23.339 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking librgw2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:23.344 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rbd (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:23.401 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T10:07:23.428 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T10:07:23.468 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T10:07:23.583 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package luarocks. 2024-09-13T10:07:23.609 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-13T10:07:23.622 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T10:07:23.648 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-13T10:07:23.666 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-13T10:07:23.680 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T10:07:23.737 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T10:07:23.753 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-13T10:07:23.787 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-13T10:07:23.900 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T10:07:23.915 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../026-python3-rgw_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:23.958 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-rgw (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:23.975 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-13T10:07:24.001 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-13T10:07:24.041 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-13T10:07:24.074 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librgw2. 2024-09-13T10:07:24.099 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-webob. 2024-09-13T10:07:24.100 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../025-librgw2_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:24.125 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-13T10:07:24.148 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librgw2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:24.165 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T10:07:24.304 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-13T10:07:24.330 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-13T10:07:24.353 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-13T10:07:24.368 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-13T10:07:24.369 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T10:07:24.411 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-13T10:07:24.519 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T10:07:24.545 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-13T10:07:24.588 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T10:07:24.681 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T10:07:24.707 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../028-libradosstriper1_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:24.739 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package lua5.1. 2024-09-13T10:07:24.765 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-13T10:07:24.768 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libradosstriper1 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:24.797 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-13T10:07:24.897 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T10:07:24.923 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-13T10:07:24.959 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T10:07:24.963 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T10:07:24.975 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../026-python3-rgw_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:25.026 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rgw (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:25.084 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-common. 2024-09-13T10:07:25.110 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../029-ceph-common_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:25.117 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package lua-any. 2024-09-13T10:07:25.143 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-09-13T10:07:25.150 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-common (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:25.192 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-13T10:07:25.241 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-paste. 2024-09-13T10:07:25.267 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-13T10:07:25.299 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T10:07:25.362 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-13T10:07:25.389 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-13T10:07:25.429 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T10:07:25.453 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package zip. 2024-09-13T10:07:25.468 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-09-13T10:07:25.511 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking zip (3.0-12build2) ... 2024-09-13T10:07:25.740 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T10:07:25.766 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../028-libradosstriper1_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:25.806 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libradosstriper1 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:25.823 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package unzip. 2024-09-13T10:07:25.836 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T10:07:25.849 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-13T10:07:25.851 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-13T10:07:25.889 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-13T10:07:25.894 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T10:07:26.138 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T10:07:26.154 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-13T10:07:26.189 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-13T10:07:26.201 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package luarocks. 2024-09-13T10:07:26.228 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-13T10:07:26.268 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-13T10:07:26.343 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-common. 2024-09-13T10:07:26.370 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../029-ceph-common_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:26.418 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-common (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:26.458 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T10:07:26.485 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-13T10:07:26.525 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-13T10:07:26.616 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-base. 2024-09-13T10:07:26.642 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../030-ceph-base_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:26.643 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librgw2. 2024-09-13T10:07:26.669 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../025-librgw2_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:26.692 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-base (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:26.714 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librgw2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:26.836 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T10:07:27.003 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-13T10:07:27.003 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T10:07:27.283 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-certifi. 2024-09-13T10:07:27.286 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-13T10:07:27.309 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-13T10:07:27.313 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-13T10:07:27.349 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-13T10:07:27.703 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-13T10:07:28.012 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T10:07:28.039 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../026-python3-rgw_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:28.048 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-base. 2024-09-13T10:07:28.070 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rgw (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:28.075 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../030-ceph-base_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:28.133 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-base (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:28.390 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-13T10:07:28.403 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-idna. 2024-09-13T10:07:28.416 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-13T10:07:28.418 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-13T10:07:28.432 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-cheroot. 2024-09-13T10:07:28.453 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T10:07:28.456 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T10:07:28.458 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-13T10:07:28.498 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T10:07:28.731 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-urllib3. 2024-09-13T10:07:28.757 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-13T10:07:28.793 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T10:07:28.797 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T10:07:28.819 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../028-libradosstriper1_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:28.826 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-13T10:07:28.852 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-13T10:07:28.859 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libradosstriper1 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:28.892 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-13T10:07:29.093 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-requests. 2024-09-13T10:07:29.119 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-13T10:07:29.159 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T10:07:29.204 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-13T10:07:29.230 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-common. 2024-09-13T10:07:29.230 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-13T10:07:29.256 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../029-ceph-common_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:29.279 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-13T10:07:29.288 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-common (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:29.453 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-13T10:07:29.472 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T10:07:29.480 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-13T10:07:29.499 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-13T10:07:29.519 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-13T10:07:29.539 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T10:07:29.573 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-13T10:07:29.600 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-13T10:07:29.640 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-13T10:07:29.822 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cheroot. 2024-09-13T10:07:29.825 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T10:07:29.841 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:29.849 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-13T10:07:29.883 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:29.888 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T10:07:29.892 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-tempora. 2024-09-13T10:07:29.919 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-13T10:07:29.959 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-13T10:07:30.213 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T10:07:30.217 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-13T10:07:30.240 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:30.244 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-13T10:07:30.262 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-portend. 2024-09-13T10:07:30.271 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:30.288 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-13T10:07:30.292 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-13T10:07:30.328 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-13T10:07:30.574 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T10:07:30.601 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-13T10:07:30.640 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-13T10:07:30.641 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-13T10:07:30.661 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-13T10:07:30.666 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-13T10:07:30.688 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-13T10:07:30.706 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-13T10:07:30.736 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-13T10:07:30.785 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-base. 2024-09-13T10:07:30.811 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../030-ceph-base_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:30.861 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-base (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:30.943 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T10:07:30.970 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-13T10:07:30.984 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T10:07:31.010 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-13T10:07:31.036 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-13T10:07:31.042 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-13T10:07:31.056 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-13T10:07:31.083 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-13T10:07:31.122 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-13T10:07:31.338 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T10:07:31.365 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../063-ceph-mgr_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:31.405 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:31.423 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-13T10:07:31.425 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-tempora. 2024-09-13T10:07:31.449 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-13T10:07:31.452 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-13T10:07:31.489 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-13T10:07:31.492 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-13T10:07:31.570 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T10:07:31.597 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-13T10:07:31.645 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-13T10:07:31.742 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cheroot. 2024-09-13T10:07:31.757 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-13T10:07:31.758 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T10:07:31.784 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../064-ceph-mon_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:31.800 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T10:07:31.824 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mon (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:31.857 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-portend. 2024-09-13T10:07:31.884 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-13T10:07:31.936 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-13T10:07:31.974 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T10:07:32.000 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-13T10:07:32.040 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-13T10:07:32.301 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T10:07:32.306 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-13T10:07:32.312 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-13T10:07:32.316 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-13T10:07:32.333 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-13T10:07:32.338 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-13T10:07:32.351 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-13T10:07:32.372 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-13T10:07:32.378 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-13T10:07:32.428 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T10:07:32.454 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../065-ceph-osd_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:32.494 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-osd (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:32.646 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-mako. 2024-09-13T10:07:32.664 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-13T10:07:32.672 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-13T10:07:32.684 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T10:07:32.690 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-13T10:07:32.711 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-13T10:07:32.712 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T10:07:32.731 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-13T10:07:32.750 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-13T10:07:32.975 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-13T10:07:32.988 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-13T10:07:33.025 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-13T10:07:33.040 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T10:07:33.067 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T10:07:33.104 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T10:07:33.107 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T10:07:33.130 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-13T10:07:33.170 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-13T10:07:33.370 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-tempora. 2024-09-13T10:07:33.397 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-09-13T10:07:33.409 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T10:07:33.436 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-13T10:07:33.437 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-13T10:07:33.448 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T10:07:33.464 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-13T10:07:33.476 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-13T10:07:33.498 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-13T10:07:33.565 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph. 2024-09-13T10:07:33.591 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../066-ceph_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:33.631 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:33.739 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-portend. 2024-09-13T10:07:33.745 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-webob. 2024-09-13T10:07:33.766 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-09-13T10:07:33.772 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-13T10:07:33.801 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T10:07:33.806 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-13T10:07:33.812 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T10:07:33.828 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-13T10:07:33.868 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-13T10:07:33.917 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T10:07:33.943 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../067-ceph-fuse_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:33.975 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-fuse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:34.083 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-13T10:07:34.106 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-13T10:07:34.123 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T10:07:34.142 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-13T10:07:34.150 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-13T10:07:34.154 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-mako. 2024-09-13T10:07:34.180 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-13T10:07:34.193 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T10:07:34.220 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T10:07:34.328 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T10:07:34.355 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../068-ceph-mds_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:34.394 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mds (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:34.429 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T10:07:34.455 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-13T10:07:34.501 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T10:07:34.503 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-13T10:07:34.515 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T10:07:34.528 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-13T10:07:34.542 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T10:07:34.568 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T10:07:34.573 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T10:07:34.789 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package cephadm. 2024-09-13T10:07:34.816 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../069-cephadm_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:34.832 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T10:07:34.855 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-09-13T10:07:34.870 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-paste. 2024-09-13T10:07:34.876 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T10:07:34.880 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking cephadm (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:34.890 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-13T10:07:34.897 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-13T10:07:34.903 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-13T10:07:34.946 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T10:07:35.143 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-13T10:07:35.160 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T10:07:35.167 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T10:07:35.186 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-09-13T10:07:35.193 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-13T10:07:35.226 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-13T10:07:35.232 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T10:07:35.233 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-13T10:07:35.251 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-13T10:07:35.290 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T10:07:35.446 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-webob. 2024-09-13T10:07:35.473 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-13T10:07:35.512 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T10:07:35.538 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T10:07:35.561 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-13T10:07:35.564 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-13T10:07:35.587 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T10:07:35.604 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-13T10:07:35.618 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T10:07:35.627 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T10:07:35.645 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-13T10:07:35.684 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-13T10:07:35.874 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T10:07:35.900 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-13T10:07:35.907 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-mako. 2024-09-13T10:07:35.934 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-13T10:07:35.960 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T10:07:35.973 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T10:07:35.987 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T10:07:36.015 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-13T10:07:36.054 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-13T10:07:36.252 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T10:07:36.278 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-13T10:07:36.302 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T10:07:36.318 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T10:07:36.328 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T10:07:36.348 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T10:07:36.369 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T10:07:36.375 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-13T10:07:36.407 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T10:07:36.588 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-paste. 2024-09-13T10:07:36.614 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-13T10:07:36.654 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T10:07:36.655 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T10:07:36.681 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-13T10:07:36.721 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-13T10:07:36.745 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-certifi. 2024-09-13T10:07:36.757 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-babel. 2024-09-13T10:07:36.768 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-13T10:07:36.785 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T10:07:36.803 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-13T10:07:36.824 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T10:07:36.965 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T10:07:36.981 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-13T10:07:36.999 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-webob. 2024-09-13T10:07:37.024 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T10:07:37.025 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-13T10:07:37.065 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T10:07:37.114 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-idna. 2024-09-13T10:07:37.141 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-13T10:07:37.143 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T10:07:37.171 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-13T10:07:37.181 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T10:07:37.210 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T10:07:37.310 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T10:07:37.337 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-13T10:07:37.377 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-13T10:07:37.460 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T10:07:37.486 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-13T10:07:37.500 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-urllib3. 2024-09-13T10:07:37.527 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-13T10:07:37.539 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T10:07:37.555 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T10:07:37.567 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T10:07:37.582 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:37.621 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:37.680 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T10:07:37.706 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-13T10:07:37.746 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-13T10:07:37.839 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T10:07:37.865 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-13T10:07:37.887 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-requests. 2024-09-13T10:07:37.905 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T10:07:37.913 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-13T10:07:37.942 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T10:07:37.953 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T10:07:37.968 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T10:07:38.008 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T10:07:38.057 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T10:07:38.084 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-13T10:07:38.124 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T10:07:38.208 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-paste. 2024-09-13T10:07:38.234 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-13T10:07:38.273 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T10:07:38.274 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T10:07:38.299 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-13T10:07:38.319 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-routes. 2024-09-13T10:07:38.339 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T10:07:38.346 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-13T10:07:38.385 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T10:07:38.487 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-certifi. 2024-09-13T10:07:38.514 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-13T10:07:38.553 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-13T10:07:38.569 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T10:07:38.582 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-13T10:07:38.617 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T10:07:38.619 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T10:07:38.633 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:38.675 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:38.747 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T10:07:38.774 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:38.813 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:38.890 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-idna. 2024-09-13T10:07:38.916 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-13T10:07:38.922 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T10:07:38.949 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-13T10:07:38.956 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T10:07:38.989 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-13T10:07:39.047 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T10:07:39.074 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:39.138 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:39.226 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-urllib3. 2024-09-13T10:07:39.253 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-13T10:07:39.292 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T10:07:39.292 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T10:07:39.318 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-09-13T10:07:39.358 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-13T10:07:39.450 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T10:07:39.476 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-13T10:07:39.516 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-13T10:07:39.588 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-requests. 2024-09-13T10:07:39.615 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-13T10:07:39.653 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T10:07:39.654 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T10:07:39.679 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-13T10:07:39.711 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T10:07:39.802 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T10:07:39.829 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-13T10:07:39.869 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-13T10:07:39.924 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T10:07:39.950 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-13T10:07:39.982 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T10:07:40.033 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-certifi. 2024-09-13T10:07:40.048 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-09-13T10:07:40.083 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-13T10:07:40.172 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T10:07:40.199 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../063-ceph-mgr_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:40.230 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:40.310 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T10:07:40.326 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:40.368 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:40.386 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-idna. 2024-09-13T10:07:40.412 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../055-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-13T10:07:40.461 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T10:07:40.616 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T10:07:40.643 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../064-ceph-mon_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:40.683 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mon (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:40.731 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T10:07:40.759 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:40.780 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-urllib3. 2024-09-13T10:07:40.798 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:40.806 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-13T10:07:40.847 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T10:07:41.118 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T10:07:41.144 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-13T10:07:41.175 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-requests. 2024-09-13T10:07:41.184 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-13T10:07:41.201 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-13T10:07:41.241 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T10:07:41.278 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T10:07:41.305 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../065-ceph-osd_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:41.344 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-osd (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:41.487 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T10:07:41.514 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-13T10:07:41.553 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T10:07:41.553 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-13T10:07:41.576 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-13T10:07:41.579 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-13T10:07:41.594 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-13T10:07:41.635 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T10:07:41.636 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T10:07:41.840 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T10:07:41.855 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../063-ceph-mgr_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:41.889 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:41.922 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T10:07:41.938 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:41.981 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:42.114 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T10:07:42.141 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-13T10:07:42.180 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T10:07:42.284 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T10:07:42.312 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../064-ceph-mon_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:42.351 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mon (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:42.369 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T10:07:42.395 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:42.440 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph. 2024-09-13T10:07:42.444 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:42.466 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../066-ceph_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:42.506 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:42.600 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-13T10:07:42.628 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-13T10:07:42.666 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-13T10:07:42.763 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T10:07:42.790 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-13T10:07:42.800 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T10:07:42.827 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../067-ceph-fuse_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:42.830 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-13T10:07:42.867 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-fuse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:42.927 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T10:07:42.946 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T10:07:42.955 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-13T10:07:42.973 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../065-ceph-osd_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:42.985 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T10:07:43.012 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-osd (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:43.133 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T10:07:43.159 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-09-13T10:07:43.199 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-13T10:07:43.228 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T10:07:43.256 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../068-ceph-mds_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:43.295 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mds (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:43.503 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T10:07:43.529 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../063-ceph-mgr_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:43.561 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:43.619 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T10:07:43.647 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:43.698 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package cephadm. 2024-09-13T10:07:43.724 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../069-cephadm_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:43.764 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking cephadm (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:43.771 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:43.939 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T10:07:43.966 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../064-ceph-mon_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:44.006 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mon (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:44.042 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T10:07:44.057 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph. 2024-09-13T10:07:44.069 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-13T10:07:44.073 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../066-ceph_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:44.100 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-13T10:07:44.107 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:44.411 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-13T10:07:44.438 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T10:07:44.452 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T10:07:44.477 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T10:07:44.478 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../067-ceph-fuse_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:44.518 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-fuse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:44.617 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T10:07:44.644 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../065-ceph-osd_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:44.684 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-osd (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:44.708 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T10:07:44.736 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-13T10:07:44.766 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-13T10:07:44.888 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T10:07:44.916 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../068-ceph-mds_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:44.955 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mds (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:45.044 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T10:07:45.060 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-13T10:07:45.094 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-13T10:07:45.341 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephadm. 2024-09-13T10:07:45.368 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../069-cephadm_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:45.422 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T10:07:45.450 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-13T10:07:45.461 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephadm (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:45.474 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-babel. 2024-09-13T10:07:45.489 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T10:07:45.501 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T10:07:45.532 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T10:07:45.737 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph. 2024-09-13T10:07:45.753 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../066-ceph_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:45.787 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:45.800 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T10:07:45.802 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T10:07:45.828 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-13T10:07:45.829 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-13T10:07:45.844 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T10:07:45.860 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T10:07:45.866 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-13T10:07:45.870 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-13T10:07:45.910 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-13T10:07:46.090 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T10:07:46.116 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../067-ceph-fuse_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:46.154 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T10:07:46.156 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-fuse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:46.181 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T10:07:46.181 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-13T10:07:46.207 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:46.212 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-13T10:07:46.229 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-13T10:07:46.245 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T10:07:46.246 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:46.279 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T10:07:46.510 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T10:07:46.532 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T10:07:46.536 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../068-ceph-mds_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:46.547 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-13T10:07:46.576 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mds (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:46.590 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T10:07:46.608 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T10:07:46.635 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T10:07:46.674 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T10:07:46.952 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-routes. 2024-09-13T10:07:46.968 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-13T10:07:47.002 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T10:07:47.029 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package cephadm. 2024-09-13T10:07:47.056 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../069-cephadm_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:47.087 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking cephadm (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:47.143 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T10:07:47.170 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-13T10:07:47.209 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-13T10:07:47.284 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-babel. 2024-09-13T10:07:47.312 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T10:07:47.339 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T10:07:47.351 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T10:07:47.357 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T10:07:47.366 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:47.383 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-13T10:07:47.405 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:47.423 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-13T10:07:47.521 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T10:07:47.548 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-13T10:07:47.587 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T10:07:47.679 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T10:07:47.705 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-13T10:07:47.745 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T10:07:47.843 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-13T10:07:47.869 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T10:07:47.909 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T10:07:48.098 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T10:07:48.125 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:48.165 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:48.260 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T10:07:48.276 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:48.318 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:48.527 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T10:07:48.553 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T10:07:48.593 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T10:07:48.687 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T10:07:48.715 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-13T10:07:48.754 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T10:07:48.879 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-routes. 2024-09-13T10:07:48.906 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-13T10:07:48.945 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T10:07:49.064 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-babel. 2024-09-13T10:07:49.065 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T10:07:49.080 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T10:07:49.093 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T10:07:49.114 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T10:07:49.132 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T10:07:49.291 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T10:07:49.318 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:49.357 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:49.384 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package jq. 2024-09-13T10:07:49.400 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T10:07:49.401 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T10:07:49.428 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-13T10:07:49.442 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-13T10:07:49.458 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T10:07:49.695 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package socat. 2024-09-13T10:07:49.719 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-13T10:07:49.761 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-13T10:07:49.778 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T10:07:49.794 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:49.828 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:50.018 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-13T10:07:50.049 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-13T10:07:50.056 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T10:07:50.072 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-13T10:07:50.093 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T10:07:50.106 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-13T10:07:50.123 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T10:07:50.139 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T10:07:50.173 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T10:07:50.443 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-routes. 2024-09-13T10:07:50.452 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-test. 2024-09-13T10:07:50.469 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-13T10:07:50.480 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../097-ceph-test_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:50.509 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T10:07:50.519 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-test (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:50.538 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T10:07:50.566 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-13T10:07:50.596 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T10:07:50.863 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T10:07:50.892 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:50.929 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:50.950 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-13T10:07:50.978 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-13T10:07:51.016 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-13T10:07:51.310 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T10:07:51.338 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-13T10:07:51.377 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T10:07:52.044 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T10:07:52.054 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-13T10:07:52.072 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:52.083 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-13T10:07:52.120 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T10:07:52.195 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:52.632 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T10:07:52.659 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-13T10:07:52.698 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T10:07:53.065 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T10:07:53.068 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-13T10:07:53.093 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../098-ceph-volume_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:53.095 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-13T10:07:53.131 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-volume (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:53.135 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-13T10:07:53.283 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T10:07:53.311 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-13T10:07:53.349 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-13T10:07:53.410 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T10:07:53.426 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../099-libcephfs-dev_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:53.438 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T10:07:53.465 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-13T10:07:53.468 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libcephfs-dev (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:53.504 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T10:07:53.667 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-13T10:07:53.677 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T10:07:53.684 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-13T10:07:53.705 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-13T10:07:53.725 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T10:07:53.743 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-13T10:07:53.779 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-13T10:07:53.807 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-13T10:07:53.846 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-13T10:07:54.072 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T10:07:54.099 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-13T10:07:54.138 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T10:07:54.184 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-13T10:07:54.196 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T10:07:54.204 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T10:07:54.211 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-13T10:07:54.223 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-13T10:07:54.232 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:54.250 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-13T10:07:54.262 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T10:07:54.381 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:54.450 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T10:07:54.477 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-13T10:07:54.516 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-13T10:07:54.570 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T10:07:54.591 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-13T10:07:54.599 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-13T10:07:54.618 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-13T10:07:54.637 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T10:07:54.648 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-13T10:07:54.811 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T10:07:54.839 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-13T10:07:54.870 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-13T10:07:54.951 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T10:07:54.979 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-13T10:07:55.018 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T10:07:55.034 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package pkg-config. 2024-09-13T10:07:55.062 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-13T10:07:55.092 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-13T10:07:55.139 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T10:07:55.156 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-13T10:07:55.198 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T10:07:55.293 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T10:07:55.310 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-13T10:07:55.336 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T10:07:55.343 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-13T10:07:55.353 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-13T10:07:55.395 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-13T10:07:55.484 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T10:07:55.511 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-13T10:07:55.550 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-13T10:07:55.638 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T10:07:55.669 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-13T10:07:55.677 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T10:07:55.705 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:55.782 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-13T10:07:55.810 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-13T10:07:55.837 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:55.848 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-13T10:07:55.862 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T10:07:55.888 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-13T10:07:55.890 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-13T10:07:55.928 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T10:07:56.151 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T10:07:56.179 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T10:07:56.200 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T10:07:56.217 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T10:07:56.228 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-13T10:07:56.266 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T10:07:56.517 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T10:07:56.534 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:56.537 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T10:07:56.565 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-13T10:07:56.575 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:56.603 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T10:07:56.603 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-13T10:07:56.631 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-13T10:07:56.661 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-13T10:07:56.757 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T10:07:56.773 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-09-13T10:07:56.815 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-13T10:07:56.837 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T10:07:56.864 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-13T10:07:56.903 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T10:07:56.915 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T10:07:56.943 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-13T10:07:56.973 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T10:07:56.981 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-13T10:07:57.001 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-13T10:07:57.039 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-13T10:07:57.118 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T10:07:57.146 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-13T10:07:57.194 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-13T10:07:57.273 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T10:07:57.301 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T10:07:57.301 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-py. 2024-09-13T10:07:57.329 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-13T10:07:57.339 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T10:07:57.343 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T10:07:57.367 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-13T10:07:57.371 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-13T10:07:57.401 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T10:07:57.497 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T10:07:57.525 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-13T10:07:57.563 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T10:07:57.600 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package jq. 2024-09-13T10:07:57.628 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T10:07:57.663 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T10:07:57.668 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-13T10:07:57.687 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pygments. 2024-09-13T10:07:57.690 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-13T10:07:57.716 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-13T10:07:57.728 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-13T10:07:57.754 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T10:07:57.875 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T10:07:57.902 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-09-13T10:07:57.936 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package socat. 2024-09-13T10:07:57.944 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-13T10:07:57.964 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-13T10:07:58.003 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-13T10:07:58.049 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T10:07:58.077 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-13T10:07:58.115 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T10:07:58.216 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T10:07:58.233 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-13T10:07:58.266 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-13T10:07:58.298 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T10:07:58.314 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-13T10:07:58.347 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-13T10:07:58.404 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T10:07:58.431 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-09-13T10:07:58.462 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-13T10:07:58.544 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-toml. 2024-09-13T10:07:58.572 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-13T10:07:58.610 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-13T10:07:58.697 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-test. 2024-09-13T10:07:58.709 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../097-ceph-test_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:07:58.751 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-test (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:58.757 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T10:07:58.785 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-13T10:07:58.788 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T10:07:58.816 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:07:58.823 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T10:07:58.854 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:07:58.913 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T10:07:58.941 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-13T10:07:58.979 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T10:07:59.123 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T10:07:59.135 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T10:07:59.151 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-13T10:07:59.162 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-09-13T10:07:59.181 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T10:07:59.201 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-13T10:07:59.291 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T10:07:59.320 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-13T10:07:59.358 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-13T10:07:59.513 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T10:07:59.518 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T10:07:59.540 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-13T10:07:59.546 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T10:07:59.579 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T10:07:59.584 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T10:07:59.635 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T10:07:59.652 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-13T10:07:59.693 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-13T10:07:59.837 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package jq. 2024-09-13T10:07:59.865 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T10:07:59.903 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-13T10:08:00.148 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package socat. 2024-09-13T10:08:00.176 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-13T10:08:00.210 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T10:08:00.214 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-13T10:08:00.226 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:08:00.268 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:00.381 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package radosgw. 2024-09-13T10:08:00.410 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../116-radosgw_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:08:00.447 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking radosgw (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:00.501 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T10:08:00.517 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-13T10:08:00.559 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-13T10:08:00.590 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T10:08:00.618 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-13T10:08:00.656 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T10:08:00.922 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-test. 2024-09-13T10:08:00.938 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../097-ceph-test_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:08:00.980 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-test (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:01.009 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T10:08:01.037 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T10:08:01.075 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T10:08:01.281 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T10:08:01.309 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../098-ceph-volume_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:08:01.311 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package jq. 2024-09-13T10:08:01.325 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T10:08:01.327 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T10:08:01.342 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../117-rbd-fuse_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:08:01.347 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking ceph-volume (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:01.360 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-13T10:08:01.383 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking rbd-fuse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:01.605 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package socat. 2024-09-13T10:08:01.621 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-13T10:08:01.659 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T10:08:01.663 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-13T10:08:01.676 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../099-libcephfs-dev_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:08:01.688 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-13T10:08:01.718 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking libcephfs-dev (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:01.963 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T10:08:01.966 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T10:08:01.983 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-13T10:08:02.003 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-13T10:08:02.019 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-13T10:08:02.024 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-13T10:08:02.053 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-13T10:08:02.064 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T10:08:02.306 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-13T10:08:02.329 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-test. 2024-09-13T10:08:02.356 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../097-ceph-test_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:08:02.395 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-test (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:02.400 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-13T10:08:02.427 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-13T10:08:02.466 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-13T10:08:02.667 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-13T10:08:02.829 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T10:08:02.856 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-13T10:08:02.895 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T10:08:03.093 INFO:teuthology.orchestra.run.smithi073.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-13T10:08:03.157 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up cephadm (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:03.334 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package pkg-config. 2024-09-13T10:08:03.362 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-13T10:08:03.400 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-13T10:08:03.478 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T10:08:03.506 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../098-ceph-volume_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:08:03.544 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-volume (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:03.636 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T10:08:03.665 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-13T10:08:03.666 INFO:teuthology.orchestra.run.smithi073.stdout:Adding system user cephadm....done 2024-09-13T10:08:03.694 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-13T10:08:03.805 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T10:08:03.840 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T10:08:03.856 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../099-libcephfs-dev_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:08:03.898 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcephfs-dev (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:04.082 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-13T10:08:04.091 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-13T10:08:04.119 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-13T10:08:04.157 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-13T10:08:04.217 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-13T10:08:04.245 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-13T10:08:04.284 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-13T10:08:04.350 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-13T10:08:04.476 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-13T10:08:04.485 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T10:08:04.513 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T10:08:04.551 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T10:08:04.639 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-13T10:08:04.666 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-13T10:08:04.705 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-13T10:08:04.727 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T10:08:04.880 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T10:08:04.908 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-13T10:08:04.946 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-13T10:08:04.949 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T10:08:04.977 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../098-ceph-volume_19.3.0-4697-gd58f7306-1jammy_all.deb ... 2024-09-13T10:08:04.978 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T10:08:05.016 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-volume (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:05.017 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T10:08:05.045 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-13T10:08:05.075 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T10:08:05.087 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-13T10:08:05.275 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T10:08:05.303 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-13T10:08:05.319 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T10:08:05.341 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-13T10:08:05.347 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../099-libcephfs-dev_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:08:05.381 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T10:08:05.419 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libcephfs-dev (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:05.456 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package pkg-config. 2024-09-13T10:08:05.484 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-13T10:08:05.522 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-13T10:08:05.669 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-py. 2024-09-13T10:08:05.697 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-13T10:08:05.730 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-13T10:08:05.732 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-13T10:08:05.735 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-13T10:08:05.758 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-13T10:08:05.775 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T10:08:05.797 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-13T10:08:05.803 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-13T10:08:05.841 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-13T10:08:05.984 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-13T10:08:06.064 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pygments. 2024-09-13T10:08:06.092 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-13T10:08:06.130 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T10:08:06.134 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-13T10:08:06.162 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-09-13T10:08:06.201 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-13T10:08:06.251 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-13T10:08:06.253 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-13T10:08:06.281 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-13T10:08:06.319 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-13T10:08:06.486 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-13T10:08:06.571 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T10:08:06.581 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T10:08:06.592 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T10:08:06.599 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-13T10:08:06.608 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T10:08:06.621 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-13T10:08:06.637 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T10:08:06.638 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T10:08:06.659 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-13T10:08:06.729 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-13T10:08:06.900 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T10:08:06.916 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-13T10:08:06.958 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-13T10:08:06.961 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-toml. 2024-09-13T10:08:06.980 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-13T10:08:06.990 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-13T10:08:07.020 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-13T10:08:07.026 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package pkg-config. 2024-09-13T10:08:07.054 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-13T10:08:07.092 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-13T10:08:07.231 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-ceph-argparse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:07.278 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T10:08:07.298 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T10:08:07.306 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-13T10:08:07.320 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T10:08:07.326 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-13T10:08:07.336 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-13T10:08:07.345 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-13T10:08:07.364 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T10:08:07.378 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-13T10:08:07.676 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T10:08:07.681 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-py. 2024-09-13T10:08:07.692 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-13T10:08:07.709 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-13T10:08:07.724 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T10:08:07.734 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-13T10:08:07.747 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-13T10:08:07.773 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-13T10:08:07.801 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-13T10:08:07.825 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-13T10:08:07.831 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-13T10:08:07.942 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-13T10:08:08.011 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T10:08:08.026 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pygments. 2024-09-13T10:08:08.028 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-13T10:08:08.054 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-13T10:08:08.068 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-13T10:08:08.069 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-13T10:08:08.084 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T10:08:08.100 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T10:08:08.128 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T10:08:08.166 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T10:08:08.336 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-13T10:08:08.487 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T10:08:08.514 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-13T10:08:08.553 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-13T10:08:08.555 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T10:08:08.583 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-13T10:08:08.629 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-13T10:08:08.753 INFO:teuthology.orchestra.run.smithi073.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-13T10:08:08.784 INFO:teuthology.orchestra.run.smithi073.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-13T10:08:08.848 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T10:08:08.864 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T10:08:08.892 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-13T10:08:08.931 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-13T10:08:08.949 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-toml. 2024-09-13T10:08:08.974 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-13T10:08:08.977 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-13T10:08:09.015 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-13T10:08:09.016 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package radosgw. 2024-09-13T10:08:09.044 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../116-radosgw_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:08:09.082 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking radosgw (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:09.275 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-py. 2024-09-13T10:08:09.303 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-09-13T10:08:09.318 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T10:08:09.325 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T10:08:09.341 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-13T10:08:09.346 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-13T10:08:09.384 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T10:08:09.451 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-13T10:08:09.645 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pygments. 2024-09-13T10:08:09.673 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-13T10:08:09.711 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T10:08:09.713 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T10:08:09.718 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T10:08:09.741 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-13T10:08:09.780 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-13T10:08:09.951 INFO:teuthology.orchestra.run.smithi181.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T10:08:09.968 INFO:teuthology.orchestra.run.smithi181.stdout:Preparing to unpack .../117-rbd-fuse_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:08:09.986 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-13T10:08:10.009 INFO:teuthology.orchestra.run.smithi181.stdout:Unpacking rbd-fuse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:10.058 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T10:08:10.086 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-13T10:08:10.124 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-13T10:08:10.199 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T10:08:10.227 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-13T10:08:10.265 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-13T10:08:10.287 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up zip (3.0-12build2) ... 2024-09-13T10:08:10.303 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-13T10:08:10.413 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T10:08:10.559 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-toml. 2024-09-13T10:08:10.564 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T10:08:10.587 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-09-13T10:08:10.625 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-13T10:08:10.699 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T10:08:10.886 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package radosgw. 2024-09-13T10:08:10.903 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T10:08:10.914 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../116-radosgw_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:08:10.931 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-13T10:08:10.952 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking radosgw (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:10.961 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-13T10:08:10.969 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T10:08:11.090 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T10:08:11.286 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-13T10:08:11.315 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T10:08:11.341 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T10:08:11.343 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-13T10:08:11.381 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-13T10:08:11.459 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-13T10:08:11.585 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T10:08:11.658 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T10:08:11.686 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-13T10:08:11.724 INFO:teuthology.orchestra.run.smithi181.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-13T10:08:11.725 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-13T10:08:11.792 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up cephadm (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:11.839 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T10:08:11.855 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../117-rbd-fuse_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:08:11.894 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-13T10:08:11.897 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking rbd-fuse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:12.201 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-13T10:08:12.295 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-13T10:08:12.300 INFO:teuthology.orchestra.run.smithi181.stdout:Adding system user cephadm....done 2024-09-13T10:08:12.438 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package radosgw. 2024-09-13T10:08:12.440 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T10:08:12.460 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T10:08:12.465 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../116-radosgw_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:08:12.503 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking radosgw (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:12.548 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T10:08:12.620 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T10:08:12.725 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-13T10:08:12.898 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-13T10:08:12.933 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T10:08:12.976 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-13T10:08:13.102 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-13T10:08:13.211 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-13T10:08:13.226 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T10:08:13.353 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T10:08:13.590 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T10:08:13.596 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T10:08:13.618 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../117-rbd-fuse_19.3.0-4697-gd58f7306-1jammy_amd64.deb ... 2024-09-13T10:08:13.628 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-13T10:08:13.633 INFO:teuthology.orchestra.run.smithi195.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-13T10:08:13.656 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking rbd-fuse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:13.722 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-13T10:08:13.730 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephadm (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:13.878 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-13T10:08:13.944 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-13T10:08:13.988 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-ceph-common (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:14.124 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T10:08:14.194 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T10:08:14.306 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T10:08:14.320 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T10:08:14.509 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-13T10:08:14.581 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-13T10:08:14.607 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T10:08:14.769 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-13T10:08:14.798 INFO:teuthology.orchestra.run.smithi195.stdout:Adding system user cephadm....done 2024-09-13T10:08:14.905 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-13T10:08:14.954 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T10:08:15.028 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-13T10:08:15.230 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-13T10:08:15.271 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-13T10:08:15.329 INFO:teuthology.orchestra.run.smithi136.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-13T10:08:15.389 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up cephadm (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:15.481 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-13T10:08:15.522 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-13T10:08:15.582 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-13T10:08:15.721 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-13T10:08:15.773 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-13T10:08:15.825 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T10:08:15.897 INFO:teuthology.orchestra.run.smithi136.stdout:Adding system user cephadm....done 2024-09-13T10:08:15.964 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:08:16.049 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-ceph-argparse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:16.053 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T10:08:16.074 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-13T10:08:16.177 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T10:08:16.309 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T10:08:16.325 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T10:08:16.362 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-13T10:08:16.434 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-13T10:08:16.451 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-13T10:08:16.560 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-13T10:08:16.577 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T10:08:16.603 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-13T10:08:16.622 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-13T10:08:16.686 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-13T10:08:16.748 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-13T10:08:16.864 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-13T10:08:16.921 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-13T10:08:16.938 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T10:08:16.999 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T10:08:17.121 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-13T10:08:17.250 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T10:08:17.313 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-13T10:08:17.323 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-13T10:08:17.352 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-13T10:08:17.372 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:08:17.376 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-13T10:08:17.416 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T10:08:17.498 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-13T10:08:17.542 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-13T10:08:17.574 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-13T10:08:17.678 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T10:08:17.791 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-13T10:08:17.842 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-13T10:08:17.901 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T10:08:18.011 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-13T10:08:18.030 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-13T10:08:18.051 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-13T10:08:18.144 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-13T10:08:18.262 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T10:08:18.281 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-13T10:08:18.318 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-13T10:08:18.412 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-13T10:08:18.541 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-13T10:08:18.569 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T10:08:18.632 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-13T10:08:18.688 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-13T10:08:18.812 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-13T10:08:18.839 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up zip (3.0-12build2) ... 2024-09-13T10:08:18.900 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-13T10:08:18.947 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-ceph-argparse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:18.981 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T10:08:19.063 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T10:08:19.153 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-13T10:08:19.189 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-13T10:08:19.198 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T10:08:19.299 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-13T10:08:19.408 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-13T10:08:19.443 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-13T10:08:19.448 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T10:08:19.534 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-13T10:08:19.549 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T10:08:19.675 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T10:08:19.737 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-ceph-argparse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:19.794 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-13T10:08:19.825 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T10:08:19.926 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T10:08:20.035 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-13T10:08:20.104 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T10:08:20.118 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-13T10:08:20.145 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T10:08:20.190 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-13T10:08:20.213 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-13T10:08:20.225 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-13T10:08:20.289 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T10:08:20.339 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-13T10:08:20.378 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-13T10:08:20.415 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-13T10:08:20.446 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-13T10:08:20.465 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-13T10:08:20.487 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-13T10:08:20.605 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T10:08:20.694 INFO:teuthology.orchestra.run.smithi073.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-13T10:08:20.732 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-13T10:08:20.766 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T10:08:20.838 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-13T10:08:20.884 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-13T10:08:20.948 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T10:08:21.057 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T10:08:21.098 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T10:08:21.125 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-13T10:08:21.155 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-13T10:08:21.159 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T10:08:21.219 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T10:08:21.354 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-13T10:08:21.434 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-13T10:08:21.444 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-13T10:08:21.492 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T10:08:21.693 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-13T10:08:21.721 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T10:08:21.771 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up zip (3.0-12build2) ... 2024-09-13T10:08:21.817 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T10:08:21.819 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-13T10:08:21.847 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-13T10:08:21.897 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T10:08:22.078 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:08:22.140 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T10:08:22.160 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-13T10:08:22.188 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T10:08:22.407 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-13T10:08:22.420 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-13T10:08:22.505 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T10:08:22.546 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-ceph-common (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:22.598 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T10:08:22.708 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up zip (3.0-12build2) ... 2024-09-13T10:08:22.790 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T10:08:22.834 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T10:08:22.841 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T10:08:22.847 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T10:08:22.984 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-13T10:08:23.058 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-13T10:08:23.135 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T10:08:23.173 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T10:08:23.184 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-13T10:08:23.452 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-13T10:08:23.486 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-13T10:08:23.528 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T10:08:23.728 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T10:08:23.795 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T10:08:23.871 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-13T10:08:23.938 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-13T10:08:24.056 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-13T10:08:24.064 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T10:08:24.121 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T10:08:24.365 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-13T10:08:24.433 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T10:08:24.504 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-13T10:08:24.506 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T10:08:24.717 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T10:08:24.741 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-13T10:08:24.780 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:08:24.791 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T10:08:24.914 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-13T10:08:24.992 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T10:08:25.103 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T10:08:25.159 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-13T10:08:25.175 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T10:08:25.204 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up librados2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:25.300 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-13T10:08:25.330 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:25.377 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T10:08:25.426 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T10:08:25.456 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-13T10:08:25.460 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-13T10:08:25.620 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-ceph-common (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:25.678 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T10:08:25.727 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-13T10:08:25.801 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libcephfs2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:25.921 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T10:08:25.927 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libradosstriper1 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:25.961 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-13T10:08:26.004 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-13T10:08:26.045 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-13T10:08:26.196 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:08:26.239 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-13T10:08:26.247 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T10:08:26.314 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-13T10:08:26.329 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up librbd1 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:26.364 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-ceph-common (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:26.455 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr-modules-core (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:26.581 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-fuse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:26.606 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-13T10:08:26.657 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T10:08:26.803 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T10:08:26.803 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T10:08:26.866 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-13T10:08:26.967 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T10:08:27.125 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-13T10:08:27.299 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libcephfs-dev (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:27.376 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T10:08:27.426 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-rados (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:27.453 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-13T10:08:27.551 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up librgw2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:27.636 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-13T10:08:27.677 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T10:08:27.738 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:08:27.872 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-13T10:08:27.896 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T10:08:28.021 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-13T10:08:28.064 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-13T10:08:28.140 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T10:08:28.249 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-13T10:08:28.293 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:08:28.442 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T10:08:28.442 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T10:08:28.442 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T10:08:28.453 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-13T10:08:28.676 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-13T10:08:28.709 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T10:08:28.733 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T10:08:28.835 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-13T10:08:28.881 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-rbd (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:28.953 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-13T10:08:28.969 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T10:08:28.990 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up rbd-fuse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:29.034 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-13T10:08:29.108 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-rgw (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:29.229 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:08:29.234 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-cephfs (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:29.262 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-13T10:08:29.310 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-13T10:08:29.355 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-13T10:08:29.360 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-common (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:29.428 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-13T10:08:29.521 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-13T10:08:29.528 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T10:08:29.601 INFO:teuthology.orchestra.run.smithi181.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-13T10:08:29.648 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-13T10:08:29.773 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:08:29.852 INFO:teuthology.orchestra.run.smithi073.stdout:Adding group ceph....done 2024-09-13T10:08:29.857 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T10:08:29.893 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-13T10:08:29.902 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-13T10:08:29.985 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T10:08:30.159 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-13T10:08:30.209 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-13T10:08:30.241 INFO:teuthology.orchestra.run.smithi073.stdout:Adding system user ceph....done 2024-09-13T10:08:30.374 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-13T10:08:30.401 INFO:teuthology.orchestra.run.smithi073.stdout:Setting system user ceph properties....done 2024-09-13T10:08:30.417 INFO:teuthology.orchestra.run.smithi073.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T10:08:30.437 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T10:08:30.461 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-13T10:08:30.553 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T10:08:30.642 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-13T10:08:30.703 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-13T10:08:30.718 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-13T10:08:30.768 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-13T10:08:30.883 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T10:08:30.970 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T10:08:30.986 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T10:08:31.027 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:08:31.096 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-13T10:08:31.128 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T10:08:31.245 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-13T10:08:31.356 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T10:08:31.370 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-test (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:31.429 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T10:08:31.482 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T10:08:31.487 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up radosgw (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:31.504 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T10:08:31.622 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-13T10:08:31.730 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T10:08:31.775 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T10:08:31.890 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T10:08:31.951 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T10:08:31.951 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T10:08:32.007 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-13T10:08:32.016 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T10:08:32.084 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-13T10:08:32.133 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-13T10:08:32.300 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T10:08:32.343 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-13T10:08:32.409 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-13T10:08:32.422 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-base (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:32.495 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-13T10:08:32.601 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-13T10:08:32.621 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T10:08:32.685 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T10:08:32.709 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-13T10:08:32.768 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T10:08:32.861 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-13T10:08:32.956 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T10:08:32.987 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-13T10:08:33.046 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-13T10:08:33.065 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T10:08:33.121 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T10:08:33.193 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-13T10:08:33.241 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mds (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:33.423 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T10:08:33.448 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T10:08:33.450 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-13T10:08:33.472 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T10:08:33.472 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T10:08:33.549 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T10:08:33.736 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-13T10:08:33.737 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T10:08:33.852 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-13T10:08:33.942 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-13T10:08:33.968 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:34.061 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T10:08:34.137 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:08:34.186 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up librados2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:34.198 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T10:08:34.198 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T10:08:34.200 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-13T10:08:34.263 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T10:08:34.312 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:34.327 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-13T10:08:34.438 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-13T10:08:34.586 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T10:08:34.588 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T10:08:34.695 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T10:08:34.702 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-osd (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:34.791 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libcephfs2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:34.890 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T10:08:34.917 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libradosstriper1 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:34.972 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T10:08:34.973 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T10:08:35.013 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T10:08:35.027 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-13T10:08:35.174 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-13T10:08:35.275 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-13T10:08:35.295 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up librbd1 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:35.348 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T10:08:35.396 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr-modules-core (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:35.479 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:35.522 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-fuse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:35.543 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-13T10:08:35.588 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:35.632 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-13T10:08:35.731 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mon (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:35.735 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T10:08:35.735 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T10:08:35.758 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-13T10:08:35.828 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T10:08:35.944 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T10:08:35.944 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T10:08:36.034 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-13T10:08:36.173 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-13T10:08:36.198 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up libcephfs-dev (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:36.301 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T10:08:36.324 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-rados (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:36.465 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr-cephadm (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:36.467 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up librgw2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:36.549 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T10:08:36.576 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T10:08:36.616 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:36.629 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-13T10:08:36.742 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr-dashboard (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:36.851 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T10:08:36.894 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-volume (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:37.023 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T10:08:37.178 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T10:08:37.246 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-13T10:08:37.304 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librados2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:37.307 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T10:08:37.355 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T10:08:37.455 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:37.499 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T10:08:37.582 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-13T10:08:37.660 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T10:08:37.771 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-rbd (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:37.786 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librados2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:37.895 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:37.897 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up rbd-fuse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:37.951 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcephfs2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:37.996 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-13T10:08:38.023 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-rgw (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:38.102 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libradosstriper1 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:38.158 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up python3-cephfs (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:38.253 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-13T10:08:38.284 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-common (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:38.307 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libcephfs2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:38.433 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libradosstriper1 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:38.538 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librbd1 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:38.559 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-13T10:08:38.664 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-modules-core (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:38.759 INFO:teuthology.orchestra.run.smithi181.stdout:Adding group ceph....done 2024-09-13T10:08:38.790 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-fuse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:38.870 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librbd1 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:38.959 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-13T10:08:38.985 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T10:08:38.985 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T10:08:38.995 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-modules-core (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:39.096 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-fuse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:39.128 INFO:teuthology.orchestra.run.smithi181.stdout:Adding system user ceph....done 2024-09-13T10:08:39.288 INFO:teuthology.orchestra.run.smithi181.stdout:Setting system user ceph properties....done 2024-09-13T10:08:39.304 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T10:08:39.305 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T10:08:39.305 INFO:teuthology.orchestra.run.smithi181.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T10:08:39.445 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T10:08:39.458 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcephfs-dev (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:39.567 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rados (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T10:08:39.644 DEBUG:teuthology.parallel:result is None 2024-09-13T10:08:39.685 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librgw2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:39.773 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T10:08:39.819 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T10:08:39.840 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libcephfs-dev (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:39.965 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rados (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:40.091 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librgw2 (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:40.217 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T10:08:40.260 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-test (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:40.394 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up radosgw (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:40.843 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T10:08:40.843 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T10:08:41.015 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rbd (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:41.141 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up rbd-fuse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:41.276 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rgw (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:41.312 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-base (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:41.426 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cephfs (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:41.437 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rbd (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:41.538 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up rbd-fuse (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:41.594 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-common (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:41.639 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rgw (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:41.655 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T10:08:41.765 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cephfs (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:41.892 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-common (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:42.139 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mds (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:42.154 INFO:teuthology.orchestra.run.smithi195.stdout:Adding group ceph....done 2024-09-13T10:08:42.374 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T10:08:42.374 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T10:08:42.458 INFO:teuthology.orchestra.run.smithi136.stdout:Adding group ceph....done 2024-09-13T10:08:42.547 INFO:teuthology.orchestra.run.smithi195.stdout:Adding system user ceph....done 2024-09-13T10:08:42.715 INFO:teuthology.orchestra.run.smithi195.stdout:Setting system user ceph properties....done 2024-09-13T10:08:42.730 INFO:teuthology.orchestra.run.smithi195.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T10:08:42.785 INFO:teuthology.orchestra.run.smithi136.stdout:Adding system user ceph....done 2024-09-13T10:08:42.857 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:42.869 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T10:08:42.903 INFO:teuthology.orchestra.run.smithi136.stdout:Setting system user ceph properties....done 2024-09-13T10:08:42.919 INFO:teuthology.orchestra.run.smithi136.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T10:08:43.055 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T10:08:43.077 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T10:08:43.077 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T10:08:43.236 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T10:08:43.390 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T10:08:43.559 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-osd (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:43.696 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-test (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:43.822 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up radosgw (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:43.858 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-test (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:43.858 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T10:08:43.858 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T10:08:43.984 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up radosgw (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:44.281 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T10:08:44.281 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T10:08:44.360 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:44.455 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T10:08:44.455 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T10:08:44.461 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:44.587 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mon (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:44.748 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-base (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:44.790 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T10:08:44.791 INFO:teuthology.orchestra.run.smithi181.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T10:08:45.070 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T10:08:45.237 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-base (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:45.264 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr-cephadm (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:45.424 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:45.550 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-mgr-dashboard (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:45.550 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mds (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:45.588 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T10:08:45.701 INFO:teuthology.orchestra.run.smithi181.stdout:Setting up ceph-volume (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:45.781 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T10:08:45.781 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T10:08:46.063 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mds (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:46.087 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-13T10:08:46.179 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T10:08:46.219 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:46.285 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T10:08:46.285 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T10:08:46.323 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T10:08:46.446 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T10:08:46.446 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T10:08:46.782 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:46.903 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-osd (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:47.009 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T10:08:47.009 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T10:08:47.159 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T10:08:47.159 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T10:08:47.465 INFO:teuthology.orchestra.run.smithi181.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-13T10:08:47.516 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-osd (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:47.673 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:47.789 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T10:08:47.789 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T10:08:47.790 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:47.916 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mon (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:48.111 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T10:08:48.111 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T10:08:48.127 INFO:teuthology.orchestra.run.smithi181.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T10:08:48.133 DEBUG:teuthology.parallel:result is None 2024-09-13T10:08:48.309 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:48.435 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:48.584 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-cephadm (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:48.586 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mon (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:48.735 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:48.799 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T10:08:48.799 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T10:08:48.861 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-dashboard (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:49.020 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-volume (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:49.390 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-13T10:08:49.499 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T10:08:49.500 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-cephadm (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:49.639 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:49.643 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T10:08:49.740 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-dashboard (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:49.874 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-volume (19.3.0-4697-gd58f7306-1jammy) ... 2024-09-13T10:08:50.236 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-13T10:08:50.344 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T10:08:50.497 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T10:08:50.852 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-13T10:08:51.495 INFO:teuthology.orchestra.run.smithi195.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T10:08:51.500 DEBUG:teuthology.parallel:result is None 2024-09-13T10:08:51.505 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-13T10:08:52.133 INFO:teuthology.orchestra.run.smithi136.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T10:08:52.138 DEBUG:teuthology.parallel:result is None 2024-09-13T10:08:52.139 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:08:52.287 DEBUG:teuthology.orchestra.run.smithi073:> dpkg-query -W -f '${Version}' ceph 2024-09-13T10:08:52.307 INFO:teuthology.orchestra.run.smithi073.stdout:19.3.0-4697-gd58f7306-1jammy 2024-09-13T10:08:52.307 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4697-gd58f7306-1jammy 2024-09-13T10:08:52.307 INFO:teuthology.task.install:The correct ceph version 19.3.0-4697-gd58f7306-1jammy is installed. 2024-09-13T10:08:52.309 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:08:52.452 DEBUG:teuthology.orchestra.run.smithi136:> dpkg-query -W -f '${Version}' ceph 2024-09-13T10:08:52.471 INFO:teuthology.orchestra.run.smithi136.stdout:19.3.0-4697-gd58f7306-1jammy 2024-09-13T10:08:52.471 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4697-gd58f7306-1jammy 2024-09-13T10:08:52.471 INFO:teuthology.task.install:The correct ceph version 19.3.0-4697-gd58f7306-1jammy is installed. 2024-09-13T10:08:52.473 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:08:52.609 DEBUG:teuthology.orchestra.run.smithi181:> dpkg-query -W -f '${Version}' ceph 2024-09-13T10:08:52.628 INFO:teuthology.orchestra.run.smithi181.stdout:19.3.0-4697-gd58f7306-1jammy 2024-09-13T10:08:52.628 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4697-gd58f7306-1jammy 2024-09-13T10:08:52.629 INFO:teuthology.task.install:The correct ceph version 19.3.0-4697-gd58f7306-1jammy is installed. 2024-09-13T10:08:52.630 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d58f73069a8157a9ace12a0247ee3e8497707ea9 2024-09-13T10:08:52.766 DEBUG:teuthology.orchestra.run.smithi195:> dpkg-query -W -f '${Version}' ceph 2024-09-13T10:08:52.785 INFO:teuthology.orchestra.run.smithi195.stdout:19.3.0-4697-gd58f7306-1jammy 2024-09-13T10:08:52.785 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4697-gd58f7306-1jammy 2024-09-13T10:08:52.786 INFO:teuthology.task.install:The correct ceph version 19.3.0-4697-gd58f7306-1jammy is installed. 2024-09-13T10:08:52.788 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T10:08:52.788 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:08:52.788 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T10:08:52.802 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T10:08:52.802 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T10:08:52.817 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T10:08:52.817 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T10:08:52.832 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:08:52.832 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T10:08:52.847 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T10:08:52.847 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:08:52.847 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T10:08:52.860 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T10:08:52.912 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T10:08:52.913 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T10:08:52.926 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T10:08:52.981 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T10:08:52.981 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T10:08:52.995 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T10:08:53.051 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:08:53.052 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T10:08:53.066 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T10:08:53.119 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T10:08:53.119 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:08:53.119 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T10:08:53.133 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T10:08:53.188 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T10:08:53.189 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T10:08:53.202 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T10:08:53.257 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T10:08:53.257 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T10:08:53.269 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T10:08:53.323 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:08:53.323 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T10:08:53.337 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T10:08:53.395 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T10:08:53.395 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:08:53.395 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T10:08:53.409 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T10:08:53.464 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T10:08:53.464 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T10:08:53.478 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T10:08:53.533 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T10:08:53.533 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T10:08:53.547 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T10:08:53.603 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:08:53.604 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T10:08:53.618 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T10:08:53.674 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T10:08:53.759 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T10:08:53.759 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 777 /var/log/ceph 2024-09-13T10:08:53.762 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /var/log/ceph 2024-09-13T10:08:53.764 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 777 /var/log/ceph 2024-09-13T10:08:53.771 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 777 /var/log/ceph 2024-09-13T10:08:53.783 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T10:08:53.783 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T10:08:53.818 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T10:08:53.821 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T10:08:53.824 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T10:08:53.837 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T10:08:53.838 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T10:08:53.870 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T10:08:53.875 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T10:08:53.880 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T10:08:53.905 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T10:08:53.905 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 5, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T10:08:53.905 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334', 'branch': 'wip-aclamk-cbt-combined', 'description': 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-low-osd-mem-target} rados supported-random-distro$/{ubuntu_latest} thrashers/careful_host thrashosds-health workloads/dedup-io-mixed}', 'email': 'akupczyk@redhat.com', 'first_in_suite': False, 'job_id': '7903334', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-aclamk-cbt-combined'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 5, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-aclamk-cbt-combined', 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'}}, 'owner': 'scheduled_akupczyk@teuthology', 'priority': 160, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'], ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'], ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'], ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0']], 'seed': 5394, 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9', 'sleep_before_teardown': 0, 'subset': '582/12160', 'suite': 'rados', 'suite_branch': 'wip-aclamk-cbt-combined', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_d58f73069a8157a9ace12a0247ee3e8497707ea9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9', 'targets': {'smithi073.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGFNcWHIuhr/Fl6vAUWbx9f8Bpo8uPUcDtmfmVT/sDyxn3P0l+crjZyXrhqpHoXOBL+bC2+Ry4HyFVM3vp0Oews=', 'smithi136.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNLZ9gQu9hew3jFbGCVslEhFZIziBy7it4+2KwVKFW9YkrsupCrtMk3iMYkNw/WvY20pthgxzIao8dYjXDK6MIQ=', 'smithi181.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBARhPDFf8PGrBpIEomhEneW0jTgM5aFSKvodrxvRyzDg1hjyvD7VismBFk0sIJGUcUDbQOpSnjZGgdjPlwPwbPc=', 'smithi195.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNgq2tN2MKdKem/Lz1ezMkGn+Z4nW/jpem18wdb9rFK/J7jALXCtLLbBvuH/moSYDtw+t7Gee+uzYFWdKYVGC9A='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 5, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9', 'cluster': 'ceph'}}, {'exec': {'mon.a': ['while ! ceph balancer status ; do sleep 1 ; done', 'ceph balancer mode crush-compat', 'ceph balancer on']}}, {'thrashosds': {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 2, 'thrash_hosts': True, 'timeout': 1200}}, {'exec': {'client.0': ['sudo ceph osd pool create low_tier 4']}}, {'rados': {'clients': ['client.0'], 'dedup_chunk_algo': 'fastcdc', 'dedup_chunk_size': '131072', 'enable_dedup': True, 'low_tier_pool': 'low_tier', 'objects': 50, 'op_weights': {'read': 100, 'set_chunk': 30, 'tier_evict': 10, 'tier_flush': 5, 'tier_promote': 10, 'write': 50}, 'ops': 1500, 'set_chunk': True}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-13_09:37:56', 'tube': 'smithi', 'user': 'akupczyk', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T10:08:53.906 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T10:08:53.926 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T10:08:53.928 DEBUG:teuthology.orchestra.run.smithi181:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T10:08:53.937 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T10:08:53.942 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T10:08:53.977 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T10:08:53.979 DEBUG:teuthology.orchestra.run.smithi181:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T10:08:53.984 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T10:08:53.998 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:08:53.998 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T10:08:54.030 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T10:08:54.030 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_1 2024-09-13T10:08:54.077 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T10:08:54.078 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:08:54.078 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 892 Links: 1 2024-09-13T10:08:54.078 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:08:54.078 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-09-13 10:03:11.340386790 +0000 2024-09-13T10:08:54.078 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-09-13 10:03:11.076389148 +0000 2024-09-13T10:08:54.078 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-09-13 10:03:11.076389148 +0000 2024-09-13T10:08:54.078 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2024-09-13T10:08:54.078 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T10:08:54.132 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-09-13T10:08:54.132 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-09-13T10:08:54.132 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000372614 s, 1.4 MB/s 2024-09-13T10:08:54.133 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T10:08:54.182 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_2 2024-09-13T10:08:54.229 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T10:08:54.229 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:08:54.229 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 909 Links: 1 2024-09-13T10:08:54.229 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:08:54.229 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-09-13 10:03:11.848382253 +0000 2024-09-13T10:08:54.230 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-09-13 10:03:11.600384468 +0000 2024-09-13T10:08:54.230 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-09-13 10:03:11.600384468 +0000 2024-09-13T10:08:54.230 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2024-09-13T10:08:54.230 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T10:08:54.283 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-09-13T10:08:54.284 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-09-13T10:08:54.284 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000361641 s, 1.4 MB/s 2024-09-13T10:08:54.285 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T10:08:54.335 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_3 2024-09-13T10:08:54.381 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T10:08:54.381 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:08:54.382 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 921 Links: 1 2024-09-13T10:08:54.382 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:08:54.382 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-09-13 10:03:12.316378072 +0000 2024-09-13T10:08:54.382 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-09-13 10:03:12.084380144 +0000 2024-09-13T10:08:54.382 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-09-13 10:03:12.084380144 +0000 2024-09-13T10:08:54.382 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2024-09-13T10:08:54.382 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T10:08:54.435 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-09-13T10:08:54.435 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-09-13T10:08:54.435 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000343383 s, 1.5 MB/s 2024-09-13T10:08:54.437 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T10:08:54.486 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_4 2024-09-13T10:08:54.533 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T10:08:54.533 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:08:54.534 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 941 Links: 1 2024-09-13T10:08:54.534 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:08:54.534 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-09-13 10:03:12.820373570 +0000 2024-09-13T10:08:54.534 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-09-13 10:03:12.560375892 +0000 2024-09-13T10:08:54.534 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-09-13 10:03:12.560375892 +0000 2024-09-13T10:08:54.534 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2024-09-13T10:08:54.534 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T10:08:54.588 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-09-13T10:08:54.589 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-09-13T10:08:54.589 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000280518 s, 1.8 MB/s 2024-09-13T10:08:54.593 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T10:08:54.642 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-13T10:08:54.643 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T10:08:54.643 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T10:08:54.649 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T10:08:54.649 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-09-13T10:08:54.695 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T10:08:54.695 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:08:54.695 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 889 Links: 1 2024-09-13T10:08:54.695 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:08:54.695 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-13 10:03:10.492075487 +0000 2024-09-13T10:08:54.695 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-13 10:03:10.252077629 +0000 2024-09-13T10:08:54.695 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-13 10:03:10.252077629 +0000 2024-09-13T10:08:54.695 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-09-13T10:08:54.696 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T10:08:54.749 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-13T10:08:54.750 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-13T10:08:54.750 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000384359 s, 1.3 MB/s 2024-09-13T10:08:54.751 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T10:08:54.800 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-09-13T10:08:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T10:08:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:08:54.847 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 900 Links: 1 2024-09-13T10:08:54.847 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:08:54.847 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-13 10:03:11.004070913 +0000 2024-09-13T10:08:54.847 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-13 10:03:10.768073021 +0000 2024-09-13T10:08:54.847 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-13 10:03:10.768073021 +0000 2024-09-13T10:08:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-09-13T10:08:54.847 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T10:08:54.900 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-13T10:08:54.901 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-13T10:08:54.901 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000413637 s, 1.2 MB/s 2024-09-13T10:08:54.902 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T10:08:54.952 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-09-13T10:08:55.001 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T10:08:55.002 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:08:55.002 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 915 Links: 1 2024-09-13T10:08:55.002 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:08:55.002 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-13 10:03:11.260068625 +0000 2024-09-13T10:08:55.002 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-13 10:03:11.256068662 +0000 2024-09-13T10:08:55.002 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-13 10:03:11.256068662 +0000 2024-09-13T10:08:55.003 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-09-13T10:08:55.003 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T10:08:55.061 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-13T10:08:55.061 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-13T10:08:55.061 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.00040119 s, 1.3 MB/s 2024-09-13T10:08:55.062 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T10:08:55.111 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-09-13T10:08:55.158 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T10:08:55.158 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:08:55.158 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 937 Links: 1 2024-09-13T10:08:55.159 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:08:55.159 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-09-13 10:03:11.752064232 +0000 2024-09-13T10:08:55.159 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-09-13 10:03:11.748064268 +0000 2024-09-13T10:08:55.159 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-09-13 10:03:11.748064268 +0000 2024-09-13T10:08:55.159 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-09-13T10:08:55.159 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T10:08:55.213 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-09-13T10:08:55.213 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-09-13T10:08:55.213 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000423706 s, 1.2 MB/s 2024-09-13T10:08:55.214 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T10:08:55.264 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-13T10:08:55.265 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T10:08:55.265 DEBUG:teuthology.orchestra.run.smithi181:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T10:08:55.271 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T10:08:55.271 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_1 2024-09-13T10:08:55.320 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T10:08:55.320 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:08:55.321 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 879 Links: 1 2024-09-13T10:08:55.321 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:08:55.321 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-09-13 10:03:12.989366703 +0000 2024-09-13T10:08:55.321 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-09-13 10:03:12.741368919 +0000 2024-09-13T10:08:55.321 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-09-13 10:03:12.741368919 +0000 2024-09-13T10:08:55.321 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2024-09-13T10:08:55.321 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T10:08:55.379 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-09-13T10:08:55.379 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-09-13T10:08:55.379 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000428077 s, 1.2 MB/s 2024-09-13T10:08:55.380 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T10:08:55.433 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_2 2024-09-13T10:08:55.483 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T10:08:55.484 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:08:55.484 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 896 Links: 1 2024-09-13T10:08:55.484 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:08:55.484 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-09-13 10:03:13.489362236 +0000 2024-09-13T10:08:55.484 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-09-13 10:03:13.237364488 +0000 2024-09-13T10:08:55.484 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-09-13 10:03:13.237364488 +0000 2024-09-13T10:08:55.484 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2024-09-13T10:08:55.484 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T10:08:55.538 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-09-13T10:08:55.538 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-09-13T10:08:55.538 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.00043687 s, 1.2 MB/s 2024-09-13T10:08:55.539 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T10:08:55.591 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_3 2024-09-13T10:08:55.643 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T10:08:55.644 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:08:55.644 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 915 Links: 1 2024-09-13T10:08:55.644 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:08:55.644 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-09-13 10:03:13.737360021 +0000 2024-09-13T10:08:55.644 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-09-13 10:03:13.733360058 +0000 2024-09-13T10:08:55.644 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-09-13 10:03:13.733360058 +0000 2024-09-13T10:08:55.644 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2024-09-13T10:08:55.644 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T10:08:55.702 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-09-13T10:08:55.703 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-09-13T10:08:55.703 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000426417 s, 1.2 MB/s 2024-09-13T10:08:55.704 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T10:08:55.754 DEBUG:teuthology.orchestra.run.smithi181:> stat /dev/vg_nvme/lv_4 2024-09-13T10:08:55.800 INFO:teuthology.orchestra.run.smithi181.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T10:08:55.800 INFO:teuthology.orchestra.run.smithi181.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:08:55.801 INFO:teuthology.orchestra.run.smithi181.stdout:Device: 5h/5d Inode: 931 Links: 1 2024-09-13T10:08:55.801 INFO:teuthology.orchestra.run.smithi181.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:08:55.801 INFO:teuthology.orchestra.run.smithi181.stdout:Access: 2024-09-13 10:03:14.525352984 +0000 2024-09-13T10:08:55.801 INFO:teuthology.orchestra.run.smithi181.stdout:Modify: 2024-09-13 10:03:14.249355449 +0000 2024-09-13T10:08:55.801 INFO:teuthology.orchestra.run.smithi181.stdout:Change: 2024-09-13 10:03:14.249355449 +0000 2024-09-13T10:08:55.801 INFO:teuthology.orchestra.run.smithi181.stdout: Birth: - 2024-09-13T10:08:55.801 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T10:08:55.854 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records in 2024-09-13T10:08:55.854 INFO:teuthology.orchestra.run.smithi181.stderr:1+0 records out 2024-09-13T10:08:55.854 INFO:teuthology.orchestra.run.smithi181.stderr:512 bytes copied, 0.000380846 s, 1.3 MB/s 2024-09-13T10:08:55.856 DEBUG:teuthology.orchestra.run.smithi181:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T10:08:55.905 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-13T10:08:55.905 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:08:55.905 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T10:08:55.911 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T10:08:55.911 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_1 2024-09-13T10:08:55.959 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T10:08:55.959 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:08:55.959 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 880 Links: 1 2024-09-13T10:08:55.959 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:08:55.960 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-13 10:03:12.780820622 +0000 2024-09-13T10:08:55.960 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-13 10:03:12.516822980 +0000 2024-09-13T10:08:55.960 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-13 10:03:12.516822980 +0000 2024-09-13T10:08:55.960 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-09-13T10:08:55.960 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T10:08:56.014 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-13T10:08:56.014 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-13T10:08:56.014 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000314324 s, 1.6 MB/s 2024-09-13T10:08:56.015 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T10:08:56.064 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_2 2024-09-13T10:08:56.111 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T10:08:56.111 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:08:56.111 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 897 Links: 1 2024-09-13T10:08:56.111 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:08:56.111 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-13 10:03:13.260816335 +0000 2024-09-13T10:08:56.112 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-13 10:03:13.008818584 +0000 2024-09-13T10:08:56.112 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-13 10:03:13.008818584 +0000 2024-09-13T10:08:56.112 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-09-13T10:08:56.112 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T10:08:56.164 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-13T10:08:56.165 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-13T10:08:56.165 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000210613 s, 2.4 MB/s 2024-09-13T10:08:56.166 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T10:08:56.212 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_3 2024-09-13T10:08:56.259 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T10:08:56.259 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:08:56.259 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 914 Links: 1 2024-09-13T10:08:56.259 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:08:56.259 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-13 10:03:13.484814334 +0000 2024-09-13T10:08:56.259 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-13 10:03:13.480814370 +0000 2024-09-13T10:08:56.259 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-13 10:03:13.480814370 +0000 2024-09-13T10:08:56.260 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-09-13T10:08:56.260 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T10:08:56.313 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-13T10:08:56.313 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-13T10:08:56.313 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000244887 s, 2.1 MB/s 2024-09-13T10:08:56.314 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T10:08:56.365 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_4 2024-09-13T10:08:56.411 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T10:08:56.412 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T10:08:56.412 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 934 Links: 1 2024-09-13T10:08:56.412 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T10:08:56.412 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-09-13 10:03:14.244807545 +0000 2024-09-13T10:08:56.412 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-09-13 10:03:13.976809939 +0000 2024-09-13T10:08:56.412 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-09-13 10:03:13.976809939 +0000 2024-09-13T10:08:56.412 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-09-13T10:08:56.413 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T10:08:56.464 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-09-13T10:08:56.464 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-09-13T10:08:56.464 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000199484 s, 2.6 MB/s 2024-09-13T10:08:56.465 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T10:08:56.512 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-13T10:08:56.512 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi073.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi136.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-09-13T10:08:56.512 INFO:tasks.ceph:Generating config... 2024-09-13T10:08:56.514 INFO:tasks.ceph:[osd] debug monc = 20 2024-09-13T10:08:56.514 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-13T10:08:56.514 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-13T10:08:56.514 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T10:08:56.514 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T10:08:56.514 INFO:tasks.ceph:[osd] bluestore write v2 random = True 2024-09-13T10:08:56.514 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-13T10:08:56.514 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-13T10:08:56.514 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-13T10:08:56.514 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T10:08:56.515 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T10:08:56.515 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-13T10:08:56.515 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T10:08:56.515 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T10:08:56.515 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T10:08:56.515 INFO:tasks.ceph:[osd] osd backoff on degraded = True 2024-09-13T10:08:56.515 INFO:tasks.ceph:[osd] osd backoff on peering = True 2024-09-13T10:08:56.515 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-13T10:08:56.515 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2024-09-13T10:08:56.515 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-13T10:08:56.515 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-13T10:08:56.515 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T10:08:56.515 INFO:tasks.ceph:[osd] osd heartbeat use min delay socket = True 2024-09-13T10:08:56.516 INFO:tasks.ceph:[osd] osd max backfills = 3 2024-09-13T10:08:56.516 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-13T10:08:56.516 INFO:tasks.ceph:[osd] osd max scrubs = 5 2024-09-13T10:08:56.516 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-13T10:08:56.516 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-13T10:08:56.516 INFO:tasks.ceph:[osd] osd memory target = 1610612736 2024-09-13T10:08:56.516 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T10:08:56.516 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-13T10:08:56.516 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-13T10:08:56.516 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-09-13T10:08:56.516 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-09-13T10:08:56.516 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-13T10:08:56.516 INFO:tasks.ceph:[osd] osd snap trim sleep = 2 2024-09-13T10:08:56.517 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-13T10:08:56.517 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-09-13T10:08:56.517 INFO:tasks.ceph:[global] ms bind msgr1 = False 2024-09-13T10:08:56.517 INFO:tasks.ceph:[global] ms bind msgr2 = True 2024-09-13T10:08:56.517 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-13T10:08:56.517 INFO:tasks.ceph:[global] ms type = async 2024-09-13T10:08:56.517 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 2 2024-09-13T10:08:56.517 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 1 2024-09-13T10:08:56.517 INFO:tasks.ceph:[global] osd_pg_log_trim_min = 0 2024-09-13T10:08:56.517 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2024-09-13T10:08:56.517 INFO:tasks.ceph:[global] osd_pool_default_size = 3 2024-09-13T10:08:56.517 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T10:08:56.517 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T10:08:56.518 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T10:08:56.518 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T10:08:56.518 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T10:08:56.518 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-09-13T10:08:56.518 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-09-13T10:08:56.518 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-09-13T10:08:56.518 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-09-13T10:08:56.518 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-13T10:08:56.518 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-09-13T10:08:56.518 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T10:08:56.518 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T10:08:56.553 INFO:teuthology.orchestra.run.smithi073.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T10:08:56.565 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T10:08:56.643 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T10:08:56.696 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.73'), ('mon.b', '172.21.15.136'), ('mon.c', '172.21.15.181')] 2024-09-13T10:08:56.696 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.73,172.21.15.136,172.21.15.181', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 5, 'osd mclock override recovery settings': True, 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-13T10:08:56.697 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:08:56.698 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T10:08:56.741 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.73 --add b 172.21.15.136 --add c 172.21.15.181 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:08:56.826 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:08:56.827 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: generated fsid c1e91d47-9492-4461-a043-8698fd74ccf3 2024-09-13T10:08:56.827 INFO:teuthology.orchestra.run.smithi073.stdout:setting min_mon_release = quincy 2024-09-13T10:08:56.827 INFO:teuthology.orchestra.run.smithi073.stdout:epoch 0 2024-09-13T10:08:56.827 INFO:teuthology.orchestra.run.smithi073.stdout:fsid c1e91d47-9492-4461-a043-8698fd74ccf3 2024-09-13T10:08:56.827 INFO:teuthology.orchestra.run.smithi073.stdout:last_changed 2024-09-13T10:08:56.828605+0000 2024-09-13T10:08:56.827 INFO:teuthology.orchestra.run.smithi073.stdout:created 2024-09-13T10:08:56.828605+0000 2024-09-13T10:08:56.827 INFO:teuthology.orchestra.run.smithi073.stdout:min_mon_release 17 (quincy) 2024-09-13T10:08:56.827 INFO:teuthology.orchestra.run.smithi073.stdout:election_strategy: 3 2024-09-13T10:08:56.828 INFO:teuthology.orchestra.run.smithi073.stdout:0: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.a 2024-09-13T10:08:56.828 INFO:teuthology.orchestra.run.smithi073.stdout:1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2024-09-13T10:08:56.828 INFO:teuthology.orchestra.run.smithi073.stdout:2: [v2:172.21.15.181:3300/0,v1:172.21.15.181:6789/0] mon.c 2024-09-13T10:08:56.828 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T10:08:56.829 DEBUG:teuthology.orchestra.run.smithi073:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T10:08:56.877 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c1e91d47-9492-4461-a043-8698fd74ccf3... 2024-09-13T10:08:56.878 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T10:08:56.922 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T10:08:56.925 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T10:08:56.928 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T10:08:56.950 INFO:teuthology.orchestra.run.smithi073.stdout:[global] 2024-09-13T10:08:56.951 INFO:teuthology.orchestra.run.smithi073.stdout: chdir = "" 2024-09-13T10:08:56.951 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T10:08:56.951 INFO:teuthology.orchestra.run.smithi073.stdout: auth supported = cephx 2024-09-13T10:08:56.951 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.951 INFO:teuthology.orchestra.run.smithi073.stdout: filestore xattr use omap = true 2024-09-13T10:08:56.951 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.952 INFO:teuthology.orchestra.run.smithi073.stdout: mon clock drift allowed = 1.000 2024-09-13T10:08:56.952 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.952 INFO:teuthology.orchestra.run.smithi073.stdout: osd crush chooseleaf type = 0 2024-09-13T10:08:56.952 INFO:teuthology.orchestra.run.smithi073.stdout: auth debug = true 2024-09-13T10:08:56.952 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.952 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on old message = true 2024-09-13T10:08:56.952 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on bug = true 2024-09-13T10:08:56.953 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.953 INFO:teuthology.orchestra.run.smithi073.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T10:08:56.953 INFO:teuthology.orchestra.run.smithi073.stdout: mon pg warn max object skew = 0 2024-09-13T10:08:56.953 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.953 INFO:teuthology.orchestra.run.smithi073.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T10:08:56.953 INFO:teuthology.orchestra.run.smithi073.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T10:08:56.954 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.954 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default size = 2 2024-09-13T10:08:56.954 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.954 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow primary affinity = true 2024-09-13T10:08:56.954 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow pg remap = true 2024-09-13T10:08:56.954 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on legacy crush tunables = false 2024-09-13T10:08:56.954 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on crush straw calc version zero = false 2024-09-13T10:08:56.954 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on no sortbitwise = false 2024-09-13T10:08:56.954 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on osd down out interval zero = false 2024-09-13T10:08:56.955 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on too few osds = false 2024-09-13T10:08:56.955 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T10:08:56.955 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T10:08:56.955 INFO:teuthology.orchestra.run.smithi073.stdout: mon_allow_pool_size_one = true 2024-09-13T10:08:56.955 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.955 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T10:08:56.955 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.955 INFO:teuthology.orchestra.run.smithi073.stdout: osd default data pool replay window = 5 2024-09-13T10:08:56.955 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.955 INFO:teuthology.orchestra.run.smithi073.stdout: mon allow pool delete = true 2024-09-13T10:08:56.955 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.955 INFO:teuthology.orchestra.run.smithi073.stdout: mon cluster log file level = debug 2024-09-13T10:08:56.955 INFO:teuthology.orchestra.run.smithi073.stdout: debug asserts on shutdown = true 2024-09-13T10:08:56.956 INFO:teuthology.orchestra.run.smithi073.stdout: mon health detail to clog = false 2024-09-13T10:08:56.956 INFO:teuthology.orchestra.run.smithi073.stdout: mon host = "172.21.15.73,172.21.15.136,172.21.15.181" 2024-09-13T10:08:56.956 INFO:teuthology.orchestra.run.smithi073.stdout: mon client directed command retry = 5 2024-09-13T10:08:56.956 INFO:teuthology.orchestra.run.smithi073.stdout: mon election default strategy = 3 2024-09-13T10:08:56.956 INFO:teuthology.orchestra.run.smithi073.stdout: ms bind msgr1 = False 2024-09-13T10:08:56.956 INFO:teuthology.orchestra.run.smithi073.stdout: ms bind msgr2 = True 2024-09-13T10:08:56.956 INFO:teuthology.orchestra.run.smithi073.stdout: ms inject socket failures = 5000 2024-09-13T10:08:56.956 INFO:teuthology.orchestra.run.smithi073.stdout: ms type = async 2024-09-13T10:08:56.956 INFO:teuthology.orchestra.run.smithi073.stdout: osd_max_pg_log_entries = 2 2024-09-13T10:08:56.956 INFO:teuthology.orchestra.run.smithi073.stdout: osd_min_pg_log_entries = 1 2024-09-13T10:08:56.956 INFO:teuthology.orchestra.run.smithi073.stdout: osd_pg_log_trim_min = 0 2024-09-13T10:08:56.956 INFO:teuthology.orchestra.run.smithi073.stdout: osd_pool_default_min_size = 2 2024-09-13T10:08:56.956 INFO:teuthology.orchestra.run.smithi073.stdout: osd_pool_default_size = 3 2024-09-13T10:08:56.957 INFO:teuthology.orchestra.run.smithi073.stdout: fsid = c1e91d47-9492-4461-a043-8698fd74ccf3 2024-09-13T10:08:56.957 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.957 INFO:teuthology.orchestra.run.smithi073.stdout:[osd] 2024-09-13T10:08:56.957 INFO:teuthology.orchestra.run.smithi073.stdout: osd journal size = 100 2024-09-13T10:08:56.957 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.957 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub load threshold = 5.0 2024-09-13T10:08:56.957 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub max interval = 120 2024-09-13T10:08:56.957 INFO:teuthology.orchestra.run.smithi073.stdout: osd mclock profile = high_recovery_ops 2024-09-13T10:08:56.957 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.957 INFO:teuthology.orchestra.run.smithi073.stdout: osd recover clone overlap = true 2024-09-13T10:08:56.957 INFO:teuthology.orchestra.run.smithi073.stdout: osd recovery max chunk = 1048576 2024-09-13T10:08:56.957 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.958 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug shutdown = true 2024-09-13T10:08:56.958 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug op order = true 2024-09-13T10:08:56.958 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify stray on activate = true 2024-09-13T10:08:56.958 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug trim objects = true 2024-09-13T10:08:56.958 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.958 INFO:teuthology.orchestra.run.smithi073.stdout: osd open classes on start = true 2024-09-13T10:08:56.958 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug pg log writeout = true 2024-09-13T10:08:56.958 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.958 INFO:teuthology.orchestra.run.smithi073.stdout: osd deep scrub update digest min age = 30 2024-09-13T10:08:56.958 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.958 INFO:teuthology.orchestra.run.smithi073.stdout: osd map max advance = 10 2024-09-13T10:08:56.958 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.958 INFO:teuthology.orchestra.run.smithi073.stdout: journal zero on create = true 2024-09-13T10:08:56.959 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.959 INFO:teuthology.orchestra.run.smithi073.stdout: filestore ondisk finisher threads = 3 2024-09-13T10:08:56.959 INFO:teuthology.orchestra.run.smithi073.stdout: filestore apply finisher threads = 3 2024-09-13T10:08:56.959 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.959 INFO:teuthology.orchestra.run.smithi073.stdout: bdev debug aio = true 2024-09-13T10:08:56.959 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug misdirected ops = true 2024-09-13T10:08:56.959 INFO:teuthology.orchestra.run.smithi073.stdout: debug monc = 20 2024-09-13T10:08:56.959 INFO:teuthology.orchestra.run.smithi073.stdout: bdev async discard = True 2024-09-13T10:08:56.959 INFO:teuthology.orchestra.run.smithi073.stdout: bdev enable discard = True 2024-09-13T10:08:56.960 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore block size = 96636764160 2024-09-13T10:08:56.960 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore fsck on mount = True 2024-09-13T10:08:56.960 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore write v2 random = True 2024-09-13T10:08:56.960 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore zero block detection = True 2024-09-13T10:08:56.960 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluefs = 20 2024-09-13T10:08:56.960 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluestore = 20 2024-09-13T10:08:56.960 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-09-13T10:08:56.960 INFO:teuthology.orchestra.run.smithi073.stdout: debug osd = 20 2024-09-13T10:08:56.960 INFO:teuthology.orchestra.run.smithi073.stdout: debug rocksdb = 10 2024-09-13T10:08:56.960 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T10:08:56.960 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd full ratio = 0.9 2024-09-13T10:08:56.960 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd nearfull ratio = 0.8 2024-09-13T10:08:56.960 INFO:teuthology.orchestra.run.smithi073.stdout: osd backoff on degraded = True 2024-09-13T10:08:56.961 INFO:teuthology.orchestra.run.smithi073.stdout: osd backoff on peering = True 2024-09-13T10:08:56.961 INFO:teuthology.orchestra.run.smithi073.stdout: osd blocked scrub grace period = 3600 2024-09-13T10:08:56.961 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug reject backfill probability = 0.3 2024-09-13T10:08:56.961 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify cached snaps = True 2024-09-13T10:08:56.961 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify missing on start = True 2024-09-13T10:08:56.961 INFO:teuthology.orchestra.run.smithi073.stdout: osd failsafe full ratio = 0.95 2024-09-13T10:08:56.961 INFO:teuthology.orchestra.run.smithi073.stdout: osd heartbeat use min delay socket = True 2024-09-13T10:08:56.961 INFO:teuthology.orchestra.run.smithi073.stdout: osd max backfills = 3 2024-09-13T10:08:56.961 INFO:teuthology.orchestra.run.smithi073.stdout: osd max markdown count = 1000 2024-09-13T10:08:56.961 INFO:teuthology.orchestra.run.smithi073.stdout: osd max scrubs = 5 2024-09-13T10:08:56.962 INFO:teuthology.orchestra.run.smithi073.stdout: osd mclock override recovery settings = True 2024-09-13T10:08:56.962 INFO:teuthology.orchestra.run.smithi073.stdout: osd memory target = 1610612736 2024-09-13T10:08:56.962 INFO:teuthology.orchestra.run.smithi073.stdout: osd objectstore = bluestore 2024-09-13T10:08:56.962 INFO:teuthology.orchestra.run.smithi073.stdout: osd op queue = debug_random 2024-09-13T10:08:56.962 INFO:teuthology.orchestra.run.smithi073.stdout: osd op queue cut off = debug_random 2024-09-13T10:08:56.962 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub min interval = 60 2024-09-13T10:08:56.962 INFO:teuthology.orchestra.run.smithi073.stdout: osd shutdown pgref assert = True 2024-09-13T10:08:56.962 INFO:teuthology.orchestra.run.smithi073.stdout: osd snap trim sleep = 2 2024-09-13T10:08:56.962 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.962 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr] 2024-09-13T10:08:56.962 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-09-13T10:08:56.962 INFO:teuthology.orchestra.run.smithi073.stdout: debug mgr = 20 2024-09-13T10:08:56.963 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-09-13T10:08:56.963 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-09-13T10:08:56.963 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-09-13T10:08:56.963 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-09-13T10:08:56.963 INFO:teuthology.orchestra.run.smithi073.stdout: mgr/telemetry/nag = false 2024-09-13T10:08:56.963 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.963 INFO:teuthology.orchestra.run.smithi073.stdout:[mon] 2024-09-13T10:08:56.963 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-09-13T10:08:56.963 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-09-13T10:08:56.963 INFO:teuthology.orchestra.run.smithi073.stdout: debug paxos = 20 2024-09-13T10:08:56.963 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-09-13T10:08:56.963 INFO:teuthology.orchestra.run.smithi073.stdout: mon data avail warn = 5 2024-09-13T10:08:56.964 INFO:teuthology.orchestra.run.smithi073.stdout: mon mgr mkfs grace = 240 2024-09-13T10:08:56.964 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-09-13T10:08:56.964 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd reporter subtree level = osd 2024-09-13T10:08:56.964 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd prime pg temp = true 2024-09-13T10:08:56.964 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-09-13T10:08:56.964 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.964 INFO:teuthology.orchestra.run.smithi073.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T10:08:56.964 INFO:teuthology.orchestra.run.smithi073.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T10:08:56.964 INFO:teuthology.orchestra.run.smithi073.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T10:08:56.964 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.964 INFO:teuthology.orchestra.run.smithi073.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T10:08:56.964 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T10:08:56.964 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T10:08:56.964 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.965 INFO:teuthology.orchestra.run.smithi073.stdout: # 1m isn't quite enough 2024-09-13T10:08:56.965 INFO:teuthology.orchestra.run.smithi073.stdout: mon_down_mkfs_grace = 2m 2024-09-13T10:08:56.965 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.965 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_filestore_osds = false 2024-09-13T10:08:56.965 INFO:teuthology.orchestra.run.smithi073.stdout: mon min osdmap epochs = 50 2024-09-13T10:08:56.965 INFO:teuthology.orchestra.run.smithi073.stdout: mon osdmap full prune interval = 2 2024-09-13T10:08:56.965 INFO:teuthology.orchestra.run.smithi073.stdout: mon osdmap full prune min = 15 2024-09-13T10:08:56.965 INFO:teuthology.orchestra.run.smithi073.stdout: mon osdmap full prune txsize = 2 2024-09-13T10:08:56.965 INFO:teuthology.orchestra.run.smithi073.stdout: mon scrub interval = 300 2024-09-13T10:08:56.965 INFO:teuthology.orchestra.run.smithi073.stdout: paxos service trim min = 10 2024-09-13T10:08:56.965 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:08:56.965 INFO:teuthology.orchestra.run.smithi073.stdout:[client] 2024-09-13T10:08:56.965 INFO:teuthology.orchestra.run.smithi073.stdout: rgw cache enabled = true 2024-09-13T10:08:56.966 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable ops log = true 2024-09-13T10:08:56.966 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable usage log = true 2024-09-13T10:08:56.966 INFO:teuthology.orchestra.run.smithi073.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T10:08:56.966 INFO:teuthology.orchestra.run.smithi073.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T10:08:56.966 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.a] 2024-09-13T10:08:56.966 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.b] 2024-09-13T10:08:56.966 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.c] 2024-09-13T10:08:56.967 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2024-09-13T10:08:56.967 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2024-09-13T10:08:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T10:08:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2024-09-13T10:08:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2024-09-13T10:08:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2024-09-13T10:08:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2024-09-13T10:08:56.968 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2024-09-13T10:08:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2024-09-13T10:08:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2024-09-13T10:08:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T10:08:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2024-09-13T10:08:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.969 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T10:08:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T10:08:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2024-09-13T10:08:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2024-09-13T10:08:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2024-09-13T10:08:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2024-09-13T10:08:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2024-09-13T10:08:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2024-09-13T10:08:56.970 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2024-09-13T10:08:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2024-09-13T10:08:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T10:08:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T10:08:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2024-09-13T10:08:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T10:08:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2024-09-13T10:08:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2024-09-13T10:08:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2024-09-13T10:08:56.971 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2024-09-13T10:08:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2024-09-13T10:08:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.73,172.21.15.136,172.21.15.181" 2024-09-13T10:08:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: mon client directed command retry = 5 2024-09-13T10:08:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: mon election default strategy = 3 2024-09-13T10:08:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: ms bind msgr1 = False 2024-09-13T10:08:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: ms bind msgr2 = True 2024-09-13T10:08:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: ms inject socket failures = 5000 2024-09-13T10:08:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: ms type = async 2024-09-13T10:08:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: osd_max_pg_log_entries = 2 2024-09-13T10:08:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: osd_min_pg_log_entries = 1 2024-09-13T10:08:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pg_log_trim_min = 0 2024-09-13T10:08:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_min_size = 2 2024-09-13T10:08:56.972 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_size = 3 2024-09-13T10:08:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = c1e91d47-9492-4461-a043-8698fd74ccf3 2024-09-13T10:08:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.973 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2024-09-13T10:08:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2024-09-13T10:08:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2024-09-13T10:08:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 120 2024-09-13T10:08:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2024-09-13T10:08:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2024-09-13T10:08:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2024-09-13T10:08:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.973 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2024-09-13T10:08:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2024-09-13T10:08:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2024-09-13T10:08:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug trim objects = true 2024-09-13T10:08:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2024-09-13T10:08:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2024-09-13T10:08:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2024-09-13T10:08:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2024-09-13T10:08:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2024-09-13T10:08:56.974 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2024-09-13T10:08:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2024-09-13T10:08:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2024-09-13T10:08:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2024-09-13T10:08:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: debug monc = 20 2024-09-13T10:08:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: bdev async discard = True 2024-09-13T10:08:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: bdev enable discard = True 2024-09-13T10:08:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2024-09-13T10:08:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2024-09-13T10:08:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore write v2 random = True 2024-09-13T10:08:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore zero block detection = True 2024-09-13T10:08:56.975 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 20 2024-09-13T10:08:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 20 2024-09-13T10:08:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-09-13T10:08:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2024-09-13T10:08:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 10 2024-09-13T10:08:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T10:08:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2024-09-13T10:08:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2024-09-13T10:08:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: osd backoff on degraded = True 2024-09-13T10:08:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: osd backoff on peering = True 2024-09-13T10:08:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: osd blocked scrub grace period = 3600 2024-09-13T10:08:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug reject backfill probability = 0.3 2024-09-13T10:08:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify cached snaps = True 2024-09-13T10:08:56.976 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify missing on start = True 2024-09-13T10:08:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2024-09-13T10:08:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: osd heartbeat use min delay socket = True 2024-09-13T10:08:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: osd max backfills = 3 2024-09-13T10:08:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: osd max markdown count = 1000 2024-09-13T10:08:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: osd max scrubs = 5 2024-09-13T10:08:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock override recovery settings = True 2024-09-13T10:08:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: osd memory target = 1610612736 2024-09-13T10:08:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2024-09-13T10:08:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: osd op queue = debug_random 2024-09-13T10:08:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: osd op queue cut off = debug_random 2024-09-13T10:08:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub min interval = 60 2024-09-13T10:08:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: osd shutdown pgref assert = True 2024-09-13T10:08:56.977 INFO:teuthology.orchestra.run.smithi195.stdout: osd snap trim sleep = 2 2024-09-13T10:08:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.978 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2024-09-13T10:08:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-09-13T10:08:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2024-09-13T10:08:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-09-13T10:08:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-09-13T10:08:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-09-13T10:08:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-09-13T10:08:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2024-09-13T10:08:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.978 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2024-09-13T10:08:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-09-13T10:08:56.978 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-09-13T10:08:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2024-09-13T10:08:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-09-13T10:08:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2024-09-13T10:08:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2024-09-13T10:08:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-09-13T10:08:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2024-09-13T10:08:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2024-09-13T10:08:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-09-13T10:08:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T10:08:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T10:08:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T10:08:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.979 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T10:08:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T10:08:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T10:08:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2024-09-13T10:08:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2024-09-13T10:08:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.980 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2024-09-13T10:08:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: mon min osdmap epochs = 50 2024-09-13T10:08:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: mon osdmap full prune interval = 2 2024-09-13T10:08:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: mon osdmap full prune min = 15 2024-09-13T10:08:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: mon osdmap full prune txsize = 2 2024-09-13T10:08:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: mon scrub interval = 300 2024-09-13T10:08:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: paxos service trim min = 10 2024-09-13T10:08:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-09-13T10:08:56.981 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2024-09-13T10:08:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2024-09-13T10:08:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2024-09-13T10:08:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2024-09-13T10:08:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T10:08:56.981 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T10:08:56.982 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2024-09-13T10:08:56.982 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2024-09-13T10:08:56.982 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2024-09-13T10:08:56.983 INFO:teuthology.orchestra.run.smithi181.stdout:[global] 2024-09-13T10:08:56.983 INFO:teuthology.orchestra.run.smithi181.stdout: chdir = "" 2024-09-13T10:08:56.984 INFO:teuthology.orchestra.run.smithi181.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T10:08:56.984 INFO:teuthology.orchestra.run.smithi181.stdout: auth supported = cephx 2024-09-13T10:08:56.984 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.984 INFO:teuthology.orchestra.run.smithi181.stdout: filestore xattr use omap = true 2024-09-13T10:08:56.984 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.984 INFO:teuthology.orchestra.run.smithi181.stdout: mon clock drift allowed = 1.000 2024-09-13T10:08:56.984 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.984 INFO:teuthology.orchestra.run.smithi181.stdout: osd crush chooseleaf type = 0 2024-09-13T10:08:56.984 INFO:teuthology.orchestra.run.smithi181.stdout: auth debug = true 2024-09-13T10:08:56.984 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.984 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on old message = true 2024-09-13T10:08:56.984 INFO:teuthology.orchestra.run.smithi181.stdout: ms die on bug = true 2024-09-13T10:08:56.984 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.985 INFO:teuthology.orchestra.run.smithi181.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T10:08:56.985 INFO:teuthology.orchestra.run.smithi181.stdout: mon pg warn max object skew = 0 2024-09-13T10:08:56.985 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.985 INFO:teuthology.orchestra.run.smithi181.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T10:08:56.985 INFO:teuthology.orchestra.run.smithi181.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T10:08:56.985 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.985 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default size = 2 2024-09-13T10:08:56.985 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.985 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow primary affinity = true 2024-09-13T10:08:56.985 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd allow pg remap = true 2024-09-13T10:08:56.986 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on legacy crush tunables = false 2024-09-13T10:08:56.986 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on crush straw calc version zero = false 2024-09-13T10:08:56.986 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on no sortbitwise = false 2024-09-13T10:08:56.986 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on osd down out interval zero = false 2024-09-13T10:08:56.986 INFO:teuthology.orchestra.run.smithi181.stdout: mon warn on too few osds = false 2024-09-13T10:08:56.986 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T10:08:56.986 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T10:08:56.986 INFO:teuthology.orchestra.run.smithi181.stdout: mon_allow_pool_size_one = true 2024-09-13T10:08:56.986 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.986 INFO:teuthology.orchestra.run.smithi181.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T10:08:56.986 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.986 INFO:teuthology.orchestra.run.smithi181.stdout: osd default data pool replay window = 5 2024-09-13T10:08:56.987 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.987 INFO:teuthology.orchestra.run.smithi181.stdout: mon allow pool delete = true 2024-09-13T10:08:56.987 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.987 INFO:teuthology.orchestra.run.smithi181.stdout: mon cluster log file level = debug 2024-09-13T10:08:56.987 INFO:teuthology.orchestra.run.smithi181.stdout: debug asserts on shutdown = true 2024-09-13T10:08:56.987 INFO:teuthology.orchestra.run.smithi181.stdout: mon health detail to clog = false 2024-09-13T10:08:56.987 INFO:teuthology.orchestra.run.smithi181.stdout: mon host = "172.21.15.73,172.21.15.136,172.21.15.181" 2024-09-13T10:08:56.987 INFO:teuthology.orchestra.run.smithi181.stdout: mon client directed command retry = 5 2024-09-13T10:08:56.987 INFO:teuthology.orchestra.run.smithi181.stdout: mon election default strategy = 3 2024-09-13T10:08:56.987 INFO:teuthology.orchestra.run.smithi181.stdout: ms bind msgr1 = False 2024-09-13T10:08:56.987 INFO:teuthology.orchestra.run.smithi181.stdout: ms bind msgr2 = True 2024-09-13T10:08:56.987 INFO:teuthology.orchestra.run.smithi181.stdout: ms inject socket failures = 5000 2024-09-13T10:08:56.987 INFO:teuthology.orchestra.run.smithi181.stdout: ms type = async 2024-09-13T10:08:56.988 INFO:teuthology.orchestra.run.smithi181.stdout: osd_max_pg_log_entries = 2 2024-09-13T10:08:56.988 INFO:teuthology.orchestra.run.smithi181.stdout: osd_min_pg_log_entries = 1 2024-09-13T10:08:56.988 INFO:teuthology.orchestra.run.smithi181.stdout: osd_pg_log_trim_min = 0 2024-09-13T10:08:56.988 INFO:teuthology.orchestra.run.smithi181.stdout: osd_pool_default_min_size = 2 2024-09-13T10:08:56.988 INFO:teuthology.orchestra.run.smithi181.stdout: osd_pool_default_size = 3 2024-09-13T10:08:56.988 INFO:teuthology.orchestra.run.smithi181.stdout: fsid = c1e91d47-9492-4461-a043-8698fd74ccf3 2024-09-13T10:08:56.988 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.988 INFO:teuthology.orchestra.run.smithi181.stdout:[osd] 2024-09-13T10:08:56.988 INFO:teuthology.orchestra.run.smithi181.stdout: osd journal size = 100 2024-09-13T10:08:56.988 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.989 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub load threshold = 5.0 2024-09-13T10:08:56.989 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub max interval = 120 2024-09-13T10:08:56.989 INFO:teuthology.orchestra.run.smithi181.stdout: osd mclock profile = high_recovery_ops 2024-09-13T10:08:56.989 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.989 INFO:teuthology.orchestra.run.smithi181.stdout: osd recover clone overlap = true 2024-09-13T10:08:56.989 INFO:teuthology.orchestra.run.smithi181.stdout: osd recovery max chunk = 1048576 2024-09-13T10:08:56.989 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.989 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug shutdown = true 2024-09-13T10:08:56.989 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug op order = true 2024-09-13T10:08:56.990 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug verify stray on activate = true 2024-09-13T10:08:56.990 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug trim objects = true 2024-09-13T10:08:56.990 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.990 INFO:teuthology.orchestra.run.smithi181.stdout: osd open classes on start = true 2024-09-13T10:08:56.990 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug pg log writeout = true 2024-09-13T10:08:56.990 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.990 INFO:teuthology.orchestra.run.smithi181.stdout: osd deep scrub update digest min age = 30 2024-09-13T10:08:56.990 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.990 INFO:teuthology.orchestra.run.smithi181.stdout: osd map max advance = 10 2024-09-13T10:08:56.990 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.991 INFO:teuthology.orchestra.run.smithi181.stdout: journal zero on create = true 2024-09-13T10:08:56.991 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.991 INFO:teuthology.orchestra.run.smithi181.stdout: filestore ondisk finisher threads = 3 2024-09-13T10:08:56.991 INFO:teuthology.orchestra.run.smithi181.stdout: filestore apply finisher threads = 3 2024-09-13T10:08:56.991 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.991 INFO:teuthology.orchestra.run.smithi181.stdout: bdev debug aio = true 2024-09-13T10:08:56.991 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug misdirected ops = true 2024-09-13T10:08:56.991 INFO:teuthology.orchestra.run.smithi181.stdout: debug monc = 20 2024-09-13T10:08:56.991 INFO:teuthology.orchestra.run.smithi181.stdout: bdev async discard = True 2024-09-13T10:08:56.992 INFO:teuthology.orchestra.run.smithi181.stdout: bdev enable discard = True 2024-09-13T10:08:56.992 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore block size = 96636764160 2024-09-13T10:08:56.992 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore fsck on mount = True 2024-09-13T10:08:56.992 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore write v2 random = True 2024-09-13T10:08:56.992 INFO:teuthology.orchestra.run.smithi181.stdout: bluestore zero block detection = True 2024-09-13T10:08:56.992 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluefs = 20 2024-09-13T10:08:56.992 INFO:teuthology.orchestra.run.smithi181.stdout: debug bluestore = 20 2024-09-13T10:08:56.992 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2024-09-13T10:08:56.992 INFO:teuthology.orchestra.run.smithi181.stdout: debug osd = 20 2024-09-13T10:08:56.992 INFO:teuthology.orchestra.run.smithi181.stdout: debug rocksdb = 10 2024-09-13T10:08:56.992 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T10:08:56.992 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd full ratio = 0.9 2024-09-13T10:08:56.993 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd nearfull ratio = 0.8 2024-09-13T10:08:56.993 INFO:teuthology.orchestra.run.smithi181.stdout: osd backoff on degraded = True 2024-09-13T10:08:56.993 INFO:teuthology.orchestra.run.smithi181.stdout: osd backoff on peering = True 2024-09-13T10:08:56.993 INFO:teuthology.orchestra.run.smithi181.stdout: osd blocked scrub grace period = 3600 2024-09-13T10:08:56.993 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug reject backfill probability = 0.3 2024-09-13T10:08:56.993 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug verify cached snaps = True 2024-09-13T10:08:56.993 INFO:teuthology.orchestra.run.smithi181.stdout: osd debug verify missing on start = True 2024-09-13T10:08:56.993 INFO:teuthology.orchestra.run.smithi181.stdout: osd failsafe full ratio = 0.95 2024-09-13T10:08:56.993 INFO:teuthology.orchestra.run.smithi181.stdout: osd heartbeat use min delay socket = True 2024-09-13T10:08:56.993 INFO:teuthology.orchestra.run.smithi181.stdout: osd max backfills = 3 2024-09-13T10:08:56.993 INFO:teuthology.orchestra.run.smithi181.stdout: osd max markdown count = 1000 2024-09-13T10:08:56.993 INFO:teuthology.orchestra.run.smithi181.stdout: osd max scrubs = 5 2024-09-13T10:08:56.993 INFO:teuthology.orchestra.run.smithi181.stdout: osd mclock override recovery settings = True 2024-09-13T10:08:56.994 INFO:teuthology.orchestra.run.smithi181.stdout: osd memory target = 1610612736 2024-09-13T10:08:56.994 INFO:teuthology.orchestra.run.smithi181.stdout: osd objectstore = bluestore 2024-09-13T10:08:56.994 INFO:teuthology.orchestra.run.smithi181.stdout: osd op queue = debug_random 2024-09-13T10:08:56.994 INFO:teuthology.orchestra.run.smithi181.stdout: osd op queue cut off = debug_random 2024-09-13T10:08:56.994 INFO:teuthology.orchestra.run.smithi181.stdout: osd scrub min interval = 60 2024-09-13T10:08:56.994 INFO:teuthology.orchestra.run.smithi181.stdout: osd shutdown pgref assert = True 2024-09-13T10:08:56.994 INFO:teuthology.orchestra.run.smithi181.stdout: osd snap trim sleep = 2 2024-09-13T10:08:56.994 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.994 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr] 2024-09-13T10:08:56.994 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2024-09-13T10:08:56.994 INFO:teuthology.orchestra.run.smithi181.stdout: debug mgr = 20 2024-09-13T10:08:56.994 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2024-09-13T10:08:56.994 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2024-09-13T10:08:56.995 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2024-09-13T10:08:56.995 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2024-09-13T10:08:56.995 INFO:teuthology.orchestra.run.smithi181.stdout: mgr/telemetry/nag = false 2024-09-13T10:08:56.995 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.995 INFO:teuthology.orchestra.run.smithi181.stdout:[mon] 2024-09-13T10:08:56.995 INFO:teuthology.orchestra.run.smithi181.stdout: debug ms = 1 2024-09-13T10:08:56.995 INFO:teuthology.orchestra.run.smithi181.stdout: debug mon = 20 2024-09-13T10:08:56.995 INFO:teuthology.orchestra.run.smithi181.stdout: debug paxos = 20 2024-09-13T10:08:56.995 INFO:teuthology.orchestra.run.smithi181.stdout: debug auth = 20 2024-09-13T10:08:56.995 INFO:teuthology.orchestra.run.smithi181.stdout: mon data avail warn = 5 2024-09-13T10:08:56.995 INFO:teuthology.orchestra.run.smithi181.stdout: mon mgr mkfs grace = 240 2024-09-13T10:08:56.995 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min pgs per osd = 4 2024-09-13T10:08:56.995 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd reporter subtree level = osd 2024-09-13T10:08:56.996 INFO:teuthology.orchestra.run.smithi181.stdout: mon osd prime pg temp = true 2024-09-13T10:08:56.996 INFO:teuthology.orchestra.run.smithi181.stdout: mon reweight min bytes per osd = 10 2024-09-13T10:08:56.996 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.996 INFO:teuthology.orchestra.run.smithi181.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T10:08:56.996 INFO:teuthology.orchestra.run.smithi181.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T10:08:56.996 INFO:teuthology.orchestra.run.smithi181.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T10:08:56.996 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.996 INFO:teuthology.orchestra.run.smithi181.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T10:08:56.996 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T10:08:56.996 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T10:08:56.996 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.996 INFO:teuthology.orchestra.run.smithi181.stdout: # 1m isn't quite enough 2024-09-13T10:08:56.996 INFO:teuthology.orchestra.run.smithi181.stdout: mon_down_mkfs_grace = 2m 2024-09-13T10:08:56.997 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.997 INFO:teuthology.orchestra.run.smithi181.stdout: mon_warn_on_filestore_osds = false 2024-09-13T10:08:56.997 INFO:teuthology.orchestra.run.smithi181.stdout: mon min osdmap epochs = 50 2024-09-13T10:08:56.997 INFO:teuthology.orchestra.run.smithi181.stdout: mon osdmap full prune interval = 2 2024-09-13T10:08:56.997 INFO:teuthology.orchestra.run.smithi181.stdout: mon osdmap full prune min = 15 2024-09-13T10:08:56.997 INFO:teuthology.orchestra.run.smithi181.stdout: mon osdmap full prune txsize = 2 2024-09-13T10:08:56.997 INFO:teuthology.orchestra.run.smithi181.stdout: mon scrub interval = 300 2024-09-13T10:08:56.997 INFO:teuthology.orchestra.run.smithi181.stdout: paxos service trim min = 10 2024-09-13T10:08:56.997 INFO:teuthology.orchestra.run.smithi181.stdout: 2024-09-13T10:08:56.997 INFO:teuthology.orchestra.run.smithi181.stdout:[client] 2024-09-13T10:08:56.997 INFO:teuthology.orchestra.run.smithi181.stdout: rgw cache enabled = true 2024-09-13T10:08:56.997 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable ops log = true 2024-09-13T10:08:56.997 INFO:teuthology.orchestra.run.smithi181.stdout: rgw enable usage log = true 2024-09-13T10:08:56.998 INFO:teuthology.orchestra.run.smithi181.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T10:08:56.998 INFO:teuthology.orchestra.run.smithi181.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T10:08:56.998 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.a] 2024-09-13T10:08:56.998 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.b] 2024-09-13T10:08:56.998 INFO:teuthology.orchestra.run.smithi181.stdout:[mon.c] 2024-09-13T10:08:56.998 INFO:teuthology.orchestra.run.smithi136.stdout:[global] 2024-09-13T10:08:56.998 INFO:teuthology.orchestra.run.smithi136.stdout: chdir = "" 2024-09-13T10:08:56.998 INFO:teuthology.orchestra.run.smithi136.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T10:08:56.998 INFO:teuthology.orchestra.run.smithi136.stdout: auth supported = cephx 2024-09-13T10:08:56.998 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:56.998 INFO:teuthology.orchestra.run.smithi136.stdout: filestore xattr use omap = true 2024-09-13T10:08:56.998 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:56.999 INFO:teuthology.orchestra.run.smithi136.stdout: mon clock drift allowed = 1.000 2024-09-13T10:08:56.999 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:56.999 INFO:teuthology.orchestra.run.smithi136.stdout: osd crush chooseleaf type = 0 2024-09-13T10:08:56.999 INFO:teuthology.orchestra.run.smithi136.stdout: auth debug = true 2024-09-13T10:08:56.999 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:56.999 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on old message = true 2024-09-13T10:08:56.999 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on bug = true 2024-09-13T10:08:56.999 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:56.999 INFO:teuthology.orchestra.run.smithi136.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T10:08:56.999 INFO:teuthology.orchestra.run.smithi136.stdout: mon pg warn max object skew = 0 2024-09-13T10:08:56.999 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:56.999 INFO:teuthology.orchestra.run.smithi136.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T10:08:56.999 INFO:teuthology.orchestra.run.smithi136.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T10:08:57.000 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.000 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default size = 2 2024-09-13T10:08:57.000 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.000 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow primary affinity = true 2024-09-13T10:08:57.000 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow pg remap = true 2024-09-13T10:08:57.000 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on legacy crush tunables = false 2024-09-13T10:08:57.000 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on crush straw calc version zero = false 2024-09-13T10:08:57.000 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on no sortbitwise = false 2024-09-13T10:08:57.000 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on osd down out interval zero = false 2024-09-13T10:08:57.000 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on too few osds = false 2024-09-13T10:08:57.000 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T10:08:57.000 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T10:08:57.001 INFO:teuthology.orchestra.run.smithi136.stdout: mon_allow_pool_size_one = true 2024-09-13T10:08:57.001 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.001 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T10:08:57.001 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.001 INFO:teuthology.orchestra.run.smithi136.stdout: osd default data pool replay window = 5 2024-09-13T10:08:57.001 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.001 INFO:teuthology.orchestra.run.smithi136.stdout: mon allow pool delete = true 2024-09-13T10:08:57.001 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.001 INFO:teuthology.orchestra.run.smithi136.stdout: mon cluster log file level = debug 2024-09-13T10:08:57.002 INFO:teuthology.orchestra.run.smithi136.stdout: debug asserts on shutdown = true 2024-09-13T10:08:57.002 INFO:teuthology.orchestra.run.smithi136.stdout: mon health detail to clog = false 2024-09-13T10:08:57.002 INFO:teuthology.orchestra.run.smithi136.stdout: mon host = "172.21.15.73,172.21.15.136,172.21.15.181" 2024-09-13T10:08:57.002 INFO:teuthology.orchestra.run.smithi136.stdout: mon client directed command retry = 5 2024-09-13T10:08:57.002 INFO:teuthology.orchestra.run.smithi136.stdout: mon election default strategy = 3 2024-09-13T10:08:57.002 INFO:teuthology.orchestra.run.smithi136.stdout: ms bind msgr1 = False 2024-09-13T10:08:57.002 INFO:teuthology.orchestra.run.smithi136.stdout: ms bind msgr2 = True 2024-09-13T10:08:57.002 INFO:teuthology.orchestra.run.smithi136.stdout: ms inject socket failures = 5000 2024-09-13T10:08:57.002 INFO:teuthology.orchestra.run.smithi136.stdout: ms type = async 2024-09-13T10:08:57.002 INFO:teuthology.orchestra.run.smithi136.stdout: osd_max_pg_log_entries = 2 2024-09-13T10:08:57.002 INFO:teuthology.orchestra.run.smithi136.stdout: osd_min_pg_log_entries = 1 2024-09-13T10:08:57.002 INFO:teuthology.orchestra.run.smithi136.stdout: osd_pg_log_trim_min = 0 2024-09-13T10:08:57.002 INFO:teuthology.orchestra.run.smithi136.stdout: osd_pool_default_min_size = 2 2024-09-13T10:08:57.003 INFO:teuthology.orchestra.run.smithi136.stdout: osd_pool_default_size = 3 2024-09-13T10:08:57.003 INFO:teuthology.orchestra.run.smithi136.stdout: fsid = c1e91d47-9492-4461-a043-8698fd74ccf3 2024-09-13T10:08:57.003 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.003 INFO:teuthology.orchestra.run.smithi136.stdout:[osd] 2024-09-13T10:08:57.003 INFO:teuthology.orchestra.run.smithi136.stdout: osd journal size = 100 2024-09-13T10:08:57.003 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.003 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub load threshold = 5.0 2024-09-13T10:08:57.003 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub max interval = 120 2024-09-13T10:08:57.003 INFO:teuthology.orchestra.run.smithi136.stdout: osd mclock profile = high_recovery_ops 2024-09-13T10:08:57.003 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.003 INFO:teuthology.orchestra.run.smithi136.stdout: osd recover clone overlap = true 2024-09-13T10:08:57.003 INFO:teuthology.orchestra.run.smithi136.stdout: osd recovery max chunk = 1048576 2024-09-13T10:08:57.003 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.003 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug shutdown = true 2024-09-13T10:08:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug op order = true 2024-09-13T10:08:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify stray on activate = true 2024-09-13T10:08:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug trim objects = true 2024-09-13T10:08:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: osd open classes on start = true 2024-09-13T10:08:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug pg log writeout = true 2024-09-13T10:08:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: osd deep scrub update digest min age = 30 2024-09-13T10:08:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: osd map max advance = 10 2024-09-13T10:08:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: journal zero on create = true 2024-09-13T10:08:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.004 INFO:teuthology.orchestra.run.smithi136.stdout: filestore ondisk finisher threads = 3 2024-09-13T10:08:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: filestore apply finisher threads = 3 2024-09-13T10:08:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: bdev debug aio = true 2024-09-13T10:08:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug misdirected ops = true 2024-09-13T10:08:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: debug monc = 20 2024-09-13T10:08:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: bdev async discard = True 2024-09-13T10:08:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: bdev enable discard = True 2024-09-13T10:08:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore block size = 96636764160 2024-09-13T10:08:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore fsck on mount = True 2024-09-13T10:08:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore write v2 random = True 2024-09-13T10:08:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore zero block detection = True 2024-09-13T10:08:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluefs = 20 2024-09-13T10:08:57.005 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluestore = 20 2024-09-13T10:08:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-09-13T10:08:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: debug osd = 20 2024-09-13T10:08:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: debug rocksdb = 10 2024-09-13T10:08:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T10:08:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd full ratio = 0.9 2024-09-13T10:08:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd nearfull ratio = 0.8 2024-09-13T10:08:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: osd backoff on degraded = True 2024-09-13T10:08:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: osd backoff on peering = True 2024-09-13T10:08:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: osd blocked scrub grace period = 3600 2024-09-13T10:08:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug reject backfill probability = 0.3 2024-09-13T10:08:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify cached snaps = True 2024-09-13T10:08:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify missing on start = True 2024-09-13T10:08:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: osd failsafe full ratio = 0.95 2024-09-13T10:08:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: osd heartbeat use min delay socket = True 2024-09-13T10:08:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: osd max backfills = 3 2024-09-13T10:08:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: osd max markdown count = 1000 2024-09-13T10:08:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: osd max scrubs = 5 2024-09-13T10:08:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: osd mclock override recovery settings = True 2024-09-13T10:08:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: osd memory target = 1610612736 2024-09-13T10:08:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: osd objectstore = bluestore 2024-09-13T10:08:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: osd op queue = debug_random 2024-09-13T10:08:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: osd op queue cut off = debug_random 2024-09-13T10:08:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub min interval = 60 2024-09-13T10:08:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: osd shutdown pgref assert = True 2024-09-13T10:08:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: osd snap trim sleep = 2 2024-09-13T10:08:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.008 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr] 2024-09-13T10:08:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-09-13T10:08:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: debug mgr = 20 2024-09-13T10:08:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2024-09-13T10:08:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2024-09-13T10:08:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2024-09-13T10:08:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2024-09-13T10:08:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: mgr/telemetry/nag = false 2024-09-13T10:08:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.008 INFO:teuthology.orchestra.run.smithi136.stdout:[mon] 2024-09-13T10:08:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-09-13T10:08:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2024-09-13T10:08:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: debug paxos = 20 2024-09-13T10:08:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2024-09-13T10:08:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: mon data avail warn = 5 2024-09-13T10:08:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: mon mgr mkfs grace = 240 2024-09-13T10:08:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2024-09-13T10:08:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd reporter subtree level = osd 2024-09-13T10:08:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd prime pg temp = true 2024-09-13T10:08:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2024-09-13T10:08:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T10:08:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T10:08:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T10:08:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T10:08:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T10:08:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T10:08:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: # 1m isn't quite enough 2024-09-13T10:08:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: mon_down_mkfs_grace = 2m 2024-09-13T10:08:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_filestore_osds = false 2024-09-13T10:08:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: mon min osdmap epochs = 50 2024-09-13T10:08:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: mon osdmap full prune interval = 2 2024-09-13T10:08:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: mon osdmap full prune min = 15 2024-09-13T10:08:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: mon osdmap full prune txsize = 2 2024-09-13T10:08:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: mon scrub interval = 300 2024-09-13T10:08:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: paxos service trim min = 10 2024-09-13T10:08:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-09-13T10:08:57.010 INFO:teuthology.orchestra.run.smithi136.stdout:[client] 2024-09-13T10:08:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: rgw cache enabled = true 2024-09-13T10:08:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable ops log = true 2024-09-13T10:08:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable usage log = true 2024-09-13T10:08:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T10:08:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T10:08:57.011 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.a] 2024-09-13T10:08:57.011 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.b] 2024-09-13T10:08:57.011 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.c] 2024-09-13T10:08:57.012 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T10:08:57.012 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T10:08:57.073 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T10:08:57.073 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:08:57.073 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T10:08:57.078 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:08:57.078 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T10:08:57.125 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi073.front.sepia.ceph.com 2024-09-13T10:08:57.125 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:08:57.125 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T10:08:57.126 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T10:08:57.189 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:08:57.189 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:08:57.238 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi136.front.sepia.ceph.com 2024-09-13T10:08:57.238 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T10:08:57.238 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T10:08:57.238 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T10:08:57.260 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T10:08:57.260 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:08:57.307 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi181.front.sepia.ceph.com 2024-09-13T10:08:57.308 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T10:08:57.308 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T10:08:57.308 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T10:08:57.333 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T10:08:57.333 DEBUG:teuthology.orchestra.run.smithi181:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:08:57.380 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi195.front.sepia.ceph.com 2024-09-13T10:08:57.380 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:08:57.380 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T10:08:57.381 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T10:08:57.404 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:08:57.404 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:08:57.454 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T10:08:57.454 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T10:08:57.454 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T10:08:57.499 INFO:teuthology.orchestra.run.smithi073.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T10:08:57.501 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T10:08:57.545 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T10:08:57.561 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T10:08:57.562 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T10:08:57.562 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T10:08:57.636 INFO:teuthology.orchestra.run.smithi195.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T10:08:57.646 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T10:08:57.646 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi073.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi136.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi181.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-09-13T10:08:57.646 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T10:08:57.662 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-13T10:08:57.662 INFO:tasks.ceph:role: osd.0 2024-09-13T10:08:57.662 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-13T10:08:57.662 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T10:08:57.725 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T10:08:57.725 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T10:08:57.725 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T10:08:57.725 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T10:08:57.725 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T10:08:57.725 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-09-13T10:08:57.725 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T10:08:57.725 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T10:08:57.725 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T10:08:57.725 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T10:08:58.176 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-09-13T10:08:58.179 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-09-13T10:08:58.179 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T10:08:58.291 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T10:08:58.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:08:58.303 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T10:08:58.304 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-13T10:08:58.356 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-13T10:08:58.356 INFO:tasks.ceph:role: osd.4 2024-09-13T10:08:58.356 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-13T10:08:58.356 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T10:08:58.415 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T10:08:58.415 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T10:08:58.416 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T10:08:58.416 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T10:08:58.416 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T10:08:58.416 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-09-13T10:08:58.416 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T10:08:58.416 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T10:08:58.416 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T10:08:58.416 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T10:08:58.858 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-09-13T10:08:58.860 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-09-13T10:08:58.860 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-13T10:08:58.933 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-13T10:08:58.986 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T10:08:58.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:08:58.988 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2024-09-13T10:08:59.040 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-13T10:08:59.041 INFO:tasks.ceph:role: osd.8 2024-09-13T10:08:59.041 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-13T10:08:59.041 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T10:08:59.108 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T10:08:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T10:08:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T10:08:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T10:08:59.108 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T10:08:59.108 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-09-13T10:08:59.108 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T10:08:59.108 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T10:08:59.109 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T10:08:59.109 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T10:08:59.570 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-09-13T10:08:59.572 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-09-13T10:08:59.572 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-8 2024-09-13T10:08:59.663 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-8 2024-09-13T10:08:59.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:08:59.673 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T10:08:59.673 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-12 2024-09-13T10:08:59.728 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-09-13T10:08:59.728 INFO:tasks.ceph:role: osd.12 2024-09-13T10:08:59.728 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-13T10:08:59.728 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T10:08:59.788 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T10:08:59.788 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T10:08:59.789 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T10:08:59.789 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T10:08:59.789 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T10:08:59.789 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-09-13T10:08:59.789 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T10:08:59.789 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T10:08:59.789 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T10:08:59.790 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T10:09:00.244 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-09-13T10:09:00.246 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-09-13T10:09:00.246 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-12 2024-09-13T10:09:00.379 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-12 2024-09-13T10:09:00.390 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T10:09:00.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:09:00.391 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:09:00.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:09:00.497+0000 7f35307567c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T10:09:00.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:09:00.501+0000 7f35307567c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T10:09:00.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:09:00.501+0000 7f35307567c0 -1 bdev(0x55f8b6503800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-13T10:09:00.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:09:00.501+0000 7f35307567c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T10:09:03.657 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T10:09:03.670 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:09:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:09:03.749+0000 7fdafe3597c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-13T10:09:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:09:03.749+0000 7fdafe3597c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-13T10:09:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:09:03.749+0000 7fdafe3597c0 -1 bdev(0x55dd6f949800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-13T10:09:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:09:03.749+0000 7fdafe3597c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-13T10:09:06.898 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-13T10:09:06.954 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:09:07.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:09:07.033+0000 7f5877a247c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2024-09-13T10:09:07.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:09:07.033+0000 7f5877a247c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2024-09-13T10:09:07.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:09:07.033+0000 7f5877a247c0 -1 bdev(0x55c5c24ed800 /var/lib/ceph/osd/ceph-8/block) open stat got: (1) Operation not permitted 2024-09-13T10:09:07.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:09:07.033+0000 7f5877a247c0 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2024-09-13T10:09:10.160 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2024-09-13T10:09:10.221 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 12 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:09:10.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:09:10.301+0000 7f1c278cc7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-12/keyring: can't open /var/lib/ceph/osd/ceph-12/keyring: (2) No such file or directory 2024-09-13T10:09:10.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:09:10.301+0000 7f1c278cc7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-12/keyring 2024-09-13T10:09:10.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:09:10.301+0000 7f1c278cc7c0 -1 bdev(0x55d752525800 /var/lib/ceph/osd/ceph-12/block) open stat got: (1) Operation not permitted 2024-09-13T10:09:10.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:09:10.301+0000 7f1c278cc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-12) _read_fsid unparsable uuid 2024-09-13T10:09:13.451 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-12 2024-09-13T10:09:13.506 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T10:09:13.522 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-13T10:09:13.522 INFO:tasks.ceph:role: osd.1 2024-09-13T10:09:13.522 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-13T10:09:13.522 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T10:09:13.590 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T10:09:13.591 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T10:09:13.591 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T10:09:13.591 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T10:09:13.591 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T10:09:13.591 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-09-13T10:09:13.591 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T10:09:13.591 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T10:09:13.591 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T10:09:13.591 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T10:09:14.098 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2024-09-13T10:09:14.100 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-09-13T10:09:14.100 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-09-13T10:09:14.220 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T10:09:14.232 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T10:09:14.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:09:14.233 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-13T10:09:14.289 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-13T10:09:14.289 INFO:tasks.ceph:role: osd.5 2024-09-13T10:09:14.289 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-13T10:09:14.289 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T10:09:14.354 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T10:09:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T10:09:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T10:09:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T10:09:14.355 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T10:09:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-09-13T10:09:14.355 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T10:09:14.355 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T10:09:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T10:09:14.355 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T10:09:14.744 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2024-09-13T10:09:14.746 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-09-13T10:09:14.746 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-13T10:09:14.809 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-13T10:09:14.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:09:14.863 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T10:09:14.864 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2024-09-13T10:09:14.916 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-13T10:09:14.916 INFO:tasks.ceph:role: osd.9 2024-09-13T10:09:14.916 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-13T10:09:14.917 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T10:09:14.976 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T10:09:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T10:09:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T10:09:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T10:09:14.977 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T10:09:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-09-13T10:09:14.977 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T10:09:14.977 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T10:09:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T10:09:14.977 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T10:09:15.471 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2024-09-13T10:09:15.473 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-09-13T10:09:15.473 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-9 2024-09-13T10:09:15.575 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-9 2024-09-13T10:09:15.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:09:15.585 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T10:09:15.585 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-13 2024-09-13T10:09:15.639 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-09-13T10:09:15.639 INFO:tasks.ceph:role: osd.13 2024-09-13T10:09:15.639 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-13T10:09:15.639 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T10:09:15.702 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T10:09:15.703 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T10:09:15.703 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T10:09:15.703 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T10:09:15.703 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T10:09:15.703 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-09-13T10:09:15.703 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T10:09:15.703 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T10:09:15.703 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T10:09:15.703 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T10:09:16.221 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2024-09-13T10:09:16.223 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-09-13T10:09:16.223 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-13 2024-09-13T10:09:16.349 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-13 2024-09-13T10:09:16.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:09:16.362 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T10:09:16.362 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:09:16.485 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-13T10:09:16.480+0000 7f62b791f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T10:09:16.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-13T10:09:16.484+0000 7f62b791f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T10:09:16.490 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-13T10:09:16.488+0000 7f62b791f7c0 -1 bdev(0x5587ec187800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-13T10:09:16.490 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-13T10:09:16.488+0000 7f62b791f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T10:09:19.664 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T10:09:19.678 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:09:19.764 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-13T10:09:19.760+0000 7f182b7347c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-13T10:09:19.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-13T10:09:19.760+0000 7f182b7347c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-13T10:09:19.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-13T10:09:19.760+0000 7f182b7347c0 -1 bdev(0x55a99432d800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-13T10:09:19.765 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-13T10:09:19.764+0000 7f182b7347c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-13T10:09:22.946 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-13T10:09:23.002 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:09:23.082 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-13T10:09:23.080+0000 7f9ad9d777c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2024-09-13T10:09:23.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-13T10:09:23.080+0000 7f9ad9d777c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2024-09-13T10:09:23.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-13T10:09:23.080+0000 7f9ad9d777c0 -1 bdev(0x55bdada5f800 /var/lib/ceph/osd/ceph-9/block) open stat got: (1) Operation not permitted 2024-09-13T10:09:23.083 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-13T10:09:23.080+0000 7f9ad9d777c0 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2024-09-13T10:09:26.229 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2024-09-13T10:09:26.286 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 13 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:09:26.368 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-13T10:09:26.364+0000 7fe47cd147c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-13/keyring: can't open /var/lib/ceph/osd/ceph-13/keyring: (2) No such file or directory 2024-09-13T10:09:26.369 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-13T10:09:26.364+0000 7fe47cd147c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-13/keyring 2024-09-13T10:09:26.369 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-13T10:09:26.368+0000 7fe47cd147c0 -1 bdev(0x5653a67b3800 /var/lib/ceph/osd/ceph-13/block) open stat got: (1) Operation not permitted 2024-09-13T10:09:26.369 INFO:teuthology.orchestra.run.smithi136.stderr:2024-09-13T10:09:26.368+0000 7fe47cd147c0 -1 bluestore(/var/lib/ceph/osd/ceph-13) _read_fsid unparsable uuid 2024-09-13T10:09:29.506 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-13 2024-09-13T10:09:29.563 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T10:09:29.577 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-13T10:09:29.577 INFO:tasks.ceph:role: osd.2 2024-09-13T10:09:29.577 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com 2024-09-13T10:09:29.577 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T10:09:29.638 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T10:09:29.639 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T10:09:29.639 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T10:09:29.639 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T10:09:29.639 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T10:09:29.639 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2024-09-13T10:09:29.639 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T10:09:29.639 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T10:09:29.639 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T10:09:29.639 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T10:09:30.104 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2024-09-13T10:09:30.105 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2024-09-13T10:09:30.106 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-09-13T10:09:30.200 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T10:09:30.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:09:30.256 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T10:09:30.257 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-13T10:09:30.311 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-13T10:09:30.311 INFO:tasks.ceph:role: osd.6 2024-09-13T10:09:30.312 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com 2024-09-13T10:09:30.312 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T10:09:30.374 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T10:09:30.374 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T10:09:30.374 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T10:09:30.374 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T10:09:30.374 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T10:09:30.374 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2024-09-13T10:09:30.375 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T10:09:30.375 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T10:09:30.375 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T10:09:30.375 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T10:09:30.850 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2024-09-13T10:09:30.853 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2024-09-13T10:09:30.853 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2024-09-13T10:09:31.014 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-13T10:09:31.025 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T10:09:31.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:09:31.027 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-10 2024-09-13T10:09:31.084 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-13T10:09:31.084 INFO:tasks.ceph:role: osd.10 2024-09-13T10:09:31.084 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com 2024-09-13T10:09:31.084 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T10:09:31.146 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T10:09:31.146 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T10:09:31.146 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T10:09:31.146 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T10:09:31.146 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T10:09:31.146 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2024-09-13T10:09:31.147 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T10:09:31.147 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T10:09:31.147 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T10:09:31.147 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T10:09:31.626 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2024-09-13T10:09:31.628 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2024-09-13T10:09:31.629 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-10 2024-09-13T10:09:31.774 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-10 2024-09-13T10:09:31.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:09:31.785 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T10:09:31.786 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/osd/ceph-14 2024-09-13T10:09:31.839 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-09-13T10:09:31.839 INFO:tasks.ceph:role: osd.14 2024-09-13T10:09:31.839 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com 2024-09-13T10:09:31.839 DEBUG:teuthology.orchestra.run.smithi181:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T10:09:31.902 INFO:teuthology.orchestra.run.smithi181.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T10:09:31.902 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T10:09:31.902 INFO:teuthology.orchestra.run.smithi181.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T10:09:31.902 INFO:teuthology.orchestra.run.smithi181.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T10:09:31.903 INFO:teuthology.orchestra.run.smithi181.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T10:09:31.903 INFO:teuthology.orchestra.run.smithi181.stdout: = sunit=0 swidth=0 blks 2024-09-13T10:09:31.903 INFO:teuthology.orchestra.run.smithi181.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T10:09:31.903 INFO:teuthology.orchestra.run.smithi181.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T10:09:31.903 INFO:teuthology.orchestra.run.smithi181.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T10:09:31.903 INFO:teuthology.orchestra.run.smithi181.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T10:09:32.351 INFO:teuthology.orchestra.run.smithi181.stdout:Discarding blocks...Done. 2024-09-13T10:09:32.353 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi181.front.sepia.ceph.com -o noatime 2024-09-13T10:09:32.353 DEBUG:teuthology.orchestra.run.smithi181:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-14 2024-09-13T10:09:32.442 DEBUG:teuthology.orchestra.run.smithi181:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-14 2024-09-13T10:09:32.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:09:32.494 INFO:teuthology.orchestra.run.smithi181.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T10:09:32.495 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:09:32.644 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-13T10:09:32.642+0000 7ff783b547c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T10:09:32.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-13T10:09:32.642+0000 7ff783b547c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T10:09:32.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-13T10:09:32.642+0000 7ff783b547c0 -1 bdev(0x5628afccd800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-13T10:09:32.645 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-13T10:09:32.642+0000 7ff783b547c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T10:09:35.773 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T10:09:35.787 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:09:35.869 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-13T10:09:35.866+0000 7f68ff9257c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-13T10:09:35.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-13T10:09:35.866+0000 7f68ff9257c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-13T10:09:35.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-13T10:09:35.866+0000 7f68ff9257c0 -1 bdev(0x55faa0719800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-13T10:09:35.870 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-13T10:09:35.866+0000 7f68ff9257c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-13T10:09:38.982 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-13T10:09:39.035 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 10 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:09:39.118 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-13T10:09:39.114+0000 7f3314f5c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-10/keyring: can't open /var/lib/ceph/osd/ceph-10/keyring: (2) No such file or directory 2024-09-13T10:09:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-13T10:09:39.118+0000 7f3314f5c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-10/keyring 2024-09-13T10:09:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-13T10:09:39.118+0000 7f3314f5c7c0 -1 bdev(0x55a3ffbaf800 /var/lib/ceph/osd/ceph-10/block) open stat got: (1) Operation not permitted 2024-09-13T10:09:39.119 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-13T10:09:39.118+0000 7f3314f5c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-10) _read_fsid unparsable uuid 2024-09-13T10:09:42.251 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2024-09-13T10:09:42.308 DEBUG:teuthology.orchestra.run.smithi181:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 14 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:09:42.389 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-13T10:09:42.386+0000 7f4d300a37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-14/keyring: can't open /var/lib/ceph/osd/ceph-14/keyring: (2) No such file or directory 2024-09-13T10:09:42.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-13T10:09:42.386+0000 7f4d300a37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-14/keyring 2024-09-13T10:09:42.390 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-13T10:09:42.386+0000 7f4d300a37c0 -1 bdev(0x56377232b800 /var/lib/ceph/osd/ceph-14/block) open stat got: (1) Operation not permitted 2024-09-13T10:09:42.391 INFO:teuthology.orchestra.run.smithi181.stderr:2024-09-13T10:09:42.386+0000 7f4d300a37c0 -1 bluestore(/var/lib/ceph/osd/ceph-14) _read_fsid unparsable uuid 2024-09-13T10:09:45.519 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-14 2024-09-13T10:09:45.575 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T10:09:45.590 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-13T10:09:45.590 INFO:tasks.ceph:role: osd.3 2024-09-13T10:09:45.590 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-13T10:09:45.591 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T10:09:45.646 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T10:09:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T10:09:45.646 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T10:09:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T10:09:45.647 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T10:09:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-09-13T10:09:45.647 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T10:09:45.647 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T10:09:45.647 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T10:09:45.647 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T10:09:45.670 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-09-13T10:09:45.672 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-09-13T10:09:45.672 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-13T10:09:45.731 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T10:09:45.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:09:45.784 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T10:09:45.784 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-13T10:09:45.837 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-13T10:09:45.838 INFO:tasks.ceph:role: osd.7 2024-09-13T10:09:45.838 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-13T10:09:45.838 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T10:09:45.889 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T10:09:45.889 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T10:09:45.889 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T10:09:45.889 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T10:09:45.889 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T10:09:45.889 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-09-13T10:09:45.890 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T10:09:45.890 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T10:09:45.890 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T10:09:45.890 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T10:09:45.913 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-09-13T10:09:45.915 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-09-13T10:09:45.915 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-7 2024-09-13T10:09:45.973 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-13T10:09:46.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:09:46.023 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T10:09:46.023 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-11 2024-09-13T10:09:46.075 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-13T10:09:46.076 INFO:tasks.ceph:role: osd.11 2024-09-13T10:09:46.076 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-13T10:09:46.076 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T10:09:46.129 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T10:09:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T10:09:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T10:09:46.129 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T10:09:46.130 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T10:09:46.130 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-09-13T10:09:46.130 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T10:09:46.130 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T10:09:46.130 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T10:09:46.130 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T10:09:46.153 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-09-13T10:09:46.155 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-09-13T10:09:46.155 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-11 2024-09-13T10:09:46.213 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-11 2024-09-13T10:09:46.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:09:46.264 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T10:09:46.264 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-15 2024-09-13T10:09:46.317 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-09-13T10:09:46.317 INFO:tasks.ceph:role: osd.15 2024-09-13T10:09:46.317 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-13T10:09:46.317 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T10:09:46.370 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T10:09:46.370 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T10:09:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T10:09:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T10:09:46.371 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T10:09:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-09-13T10:09:46.371 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T10:09:46.371 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T10:09:46.371 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T10:09:46.371 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T10:09:46.394 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-09-13T10:09:46.395 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-09-13T10:09:46.396 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-15 2024-09-13T10:09:46.458 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-15 2024-09-13T10:09:46.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:09:46.514 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T10:09:46.515 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:09:46.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-13T10:09:46.634+0000 7f1d354d57c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T10:09:46.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-13T10:09:46.634+0000 7f1d354d57c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T10:09:46.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-13T10:09:46.634+0000 7f1d354d57c0 -1 bdev(0x5643f301d800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-13T10:09:46.636 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-13T10:09:46.634+0000 7f1d354d57c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T10:09:49.786 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T10:09:49.800 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:09:49.880 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-13T10:09:49.878+0000 7f0ceb1d37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-13T10:09:49.880 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-13T10:09:49.878+0000 7f0ceb1d37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-13T10:09:49.880 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-13T10:09:49.878+0000 7f0ceb1d37c0 -1 bdev(0x555c388c5800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-13T10:09:49.881 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-13T10:09:49.878+0000 7f0ceb1d37c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-13T10:09:53.009 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-13T10:09:53.067 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 11 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:09:53.148 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-13T10:09:53.146+0000 7fc64fa4a7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-11/keyring: can't open /var/lib/ceph/osd/ceph-11/keyring: (2) No such file or directory 2024-09-13T10:09:53.149 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-13T10:09:53.146+0000 7fc64fa4a7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-11/keyring 2024-09-13T10:09:53.149 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-13T10:09:53.146+0000 7fc64fa4a7c0 -1 bdev(0x56043f8d1800 /var/lib/ceph/osd/ceph-11/block) open stat got: (1) Operation not permitted 2024-09-13T10:09:53.149 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-13T10:09:53.146+0000 7fc64fa4a7c0 -1 bluestore(/var/lib/ceph/osd/ceph-11) _read_fsid unparsable uuid 2024-09-13T10:09:56.290 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2024-09-13T10:09:56.346 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 15 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:09:56.428 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-13T10:09:56.426+0000 7fc2cef5d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-15/keyring: can't open /var/lib/ceph/osd/ceph-15/keyring: (2) No such file or directory 2024-09-13T10:09:56.429 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-13T10:09:56.426+0000 7fc2cef5d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-15/keyring 2024-09-13T10:09:56.429 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-13T10:09:56.426+0000 7fc2cef5d7c0 -1 bdev(0x563215c6f800 /var/lib/ceph/osd/ceph-15/block) open stat got: (1) Operation not permitted 2024-09-13T10:09:56.429 INFO:teuthology.orchestra.run.smithi195.stderr:2024-09-13T10:09:56.426+0000 7fc2cef5d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-15) _read_fsid unparsable uuid 2024-09-13T10:09:59.574 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-15 2024-09-13T10:09:59.631 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T10:09:59.632 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:09:59.632 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T10:09:59.647 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:09:59.647 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T10:09:59.700 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:09:59.700 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-13T10:09:59.755 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:09:59.755 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2024-09-13T10:09:59.811 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:09:59.811 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-12/keyring of=/dev/stdout 2024-09-13T10:09:59.867 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T10:09:59.867 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T10:09:59.881 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T10:09:59.881 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-13T10:09:59.937 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T10:09:59.937 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2024-09-13T10:09:59.993 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T10:09:59.993 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-13/keyring of=/dev/stdout 2024-09-13T10:10:00.053 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T10:10:00.053 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T10:10:00.067 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T10:10:00.067 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-13T10:10:00.122 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T10:10:00.122 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-10/keyring of=/dev/stdout 2024-09-13T10:10:00.174 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T10:10:00.175 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-14/keyring of=/dev/stdout 2024-09-13T10:10:00.230 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:10:00.230 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T10:10:00.244 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:10:00.245 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T10:10:00.297 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:10:00.297 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-13T10:10:00.349 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:10:00.349 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-11/keyring of=/dev/stdout 2024-09-13T10:10:00.401 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:10:00.401 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-15/keyring of=/dev/stdout 2024-09-13T10:10:00.454 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:10:00.454 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T10:10:00.502 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T10:10:00.503 DEBUG:teuthology.orchestra.run.smithi073:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T10:10:00.505 DEBUG:teuthology.orchestra.run.smithi136:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T10:10:00.507 DEBUG:teuthology.orchestra.run.smithi181:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T10:10:00.515 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr.y] 2024-09-13T10:10:00.515 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQA5D+Rmj8fWHRAA7NVyZyPzyXZaP6Yjz3NhZg== 2024-09-13T10:10:00.515 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.0] 2024-09-13T10:10:00.515 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQA8D+RmaHnGHRAA44+S8r8FsNkaOGh0pRphTA== 2024-09-13T10:10:00.515 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.4] 2024-09-13T10:10:00.516 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQA/D+RmyvvmLBAAyYqHZ4MDx8wFr8bjT7SgSA== 2024-09-13T10:10:00.516 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.8] 2024-09-13T10:10:00.516 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBDD+Rma/FAAhAATy6ea3sPYe3+b57PDFvYiA== 2024-09-13T10:10:00.516 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.12] 2024-09-13T10:10:00.516 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBGD+Rmo4EtEhAAsoNclUOJUGPgPvsqdr4IDA== 2024-09-13T10:10:00.516 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.1] 2024-09-13T10:10:00.516 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBMD+RmmK8JHRAA2y+ZKEosIYDRLYOjDluhzg== 2024-09-13T10:10:00.516 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.5] 2024-09-13T10:10:00.516 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBPD+RmrtCtLRAAsx3ti151k4KHEoDIH+byEQ== 2024-09-13T10:10:00.516 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.9] 2024-09-13T10:10:00.517 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBTD+Rm1w0HBRAAiqnOpevYaMHQbDi/4AVDtQ== 2024-09-13T10:10:00.517 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.13] 2024-09-13T10:10:00.517 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBWD+Rmv7gVFhAASt2pGkg7CbR3Zh4W6TWE/Q== 2024-09-13T10:10:00.517 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.2] 2024-09-13T10:10:00.517 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBcD+RmySJ+JhAAeDn0+xuGmdJGm1d1QPP/Zw== 2024-09-13T10:10:00.517 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.6] 2024-09-13T10:10:00.517 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBfD+RmSx3yMxAAwbPxuPJK84Hs+rLXaFoSvQ== 2024-09-13T10:10:00.517 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.10] 2024-09-13T10:10:00.517 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBjD+RmV+AtBxAAM3OGg4L7RiHtEUf7wWZFQw== 2024-09-13T10:10:00.517 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.14] 2024-09-13T10:10:00.518 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBmD+Rm2DpYFxAAzml5uia5twxFMYs0zPU3uA== 2024-09-13T10:10:00.518 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr.x] 2024-09-13T10:10:00.518 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQA5D+RmsEGQIBAAtxryI6AMa+mKQBRSW/2XSg== 2024-09-13T10:10:00.518 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.3] 2024-09-13T10:10:00.518 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBqD+RmOjf9JRAAzjwfP39RPlq+fb8uQ82bhw== 2024-09-13T10:10:00.518 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.7] 2024-09-13T10:10:00.518 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBtD+Rm/k6SNBAA7j/2vXrE1MFupLsTvbpjAQ== 2024-09-13T10:10:00.518 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.11] 2024-09-13T10:10:00.518 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBxD+Rmt/L0CBAAV1XXnlWNbL0sc9wUU76uBA== 2024-09-13T10:10:00.518 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.15] 2024-09-13T10:10:00.519 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQB0D+RmDjmiGRAAXrkGwFzsx/vxTesGU+/R+w== 2024-09-13T10:10:00.519 INFO:teuthology.orchestra.run.smithi073.stdout:[client.0] 2024-09-13T10:10:00.519 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQA5D+RmJZMDJhAAzhQsGeBKI4xf/bbmzVtAMA== 2024-09-13T10:10:00.520 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.y] 2024-09-13T10:10:00.520 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQA5D+Rmj8fWHRAA7NVyZyPzyXZaP6Yjz3NhZg== 2024-09-13T10:10:00.520 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.0] 2024-09-13T10:10:00.520 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQA8D+RmaHnGHRAA44+S8r8FsNkaOGh0pRphTA== 2024-09-13T10:10:00.520 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.4] 2024-09-13T10:10:00.520 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQA/D+RmyvvmLBAAyYqHZ4MDx8wFr8bjT7SgSA== 2024-09-13T10:10:00.520 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.8] 2024-09-13T10:10:00.520 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBDD+Rma/FAAhAATy6ea3sPYe3+b57PDFvYiA== 2024-09-13T10:10:00.520 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.12] 2024-09-13T10:10:00.520 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBGD+Rmo4EtEhAAsoNclUOJUGPgPvsqdr4IDA== 2024-09-13T10:10:00.521 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.1] 2024-09-13T10:10:00.521 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBMD+RmmK8JHRAA2y+ZKEosIYDRLYOjDluhzg== 2024-09-13T10:10:00.521 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.5] 2024-09-13T10:10:00.521 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBPD+RmrtCtLRAAsx3ti151k4KHEoDIH+byEQ== 2024-09-13T10:10:00.521 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.9] 2024-09-13T10:10:00.521 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBTD+Rm1w0HBRAAiqnOpevYaMHQbDi/4AVDtQ== 2024-09-13T10:10:00.521 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.13] 2024-09-13T10:10:00.521 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBWD+Rmv7gVFhAASt2pGkg7CbR3Zh4W6TWE/Q== 2024-09-13T10:10:00.521 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.2] 2024-09-13T10:10:00.521 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBcD+RmySJ+JhAAeDn0+xuGmdJGm1d1QPP/Zw== 2024-09-13T10:10:00.521 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.6] 2024-09-13T10:10:00.521 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBfD+RmSx3yMxAAwbPxuPJK84Hs+rLXaFoSvQ== 2024-09-13T10:10:00.521 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.10] 2024-09-13T10:10:00.521 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBjD+RmV+AtBxAAM3OGg4L7RiHtEUf7wWZFQw== 2024-09-13T10:10:00.522 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.14] 2024-09-13T10:10:00.522 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBmD+Rm2DpYFxAAzml5uia5twxFMYs0zPU3uA== 2024-09-13T10:10:00.522 INFO:teuthology.orchestra.run.smithi181.stdout:[mgr.x] 2024-09-13T10:10:00.522 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQA5D+RmsEGQIBAAtxryI6AMa+mKQBRSW/2XSg== 2024-09-13T10:10:00.522 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.3] 2024-09-13T10:10:00.522 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBqD+RmOjf9JRAAzjwfP39RPlq+fb8uQ82bhw== 2024-09-13T10:10:00.522 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.7] 2024-09-13T10:10:00.522 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBtD+Rm/k6SNBAA7j/2vXrE1MFupLsTvbpjAQ== 2024-09-13T10:10:00.522 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.11] 2024-09-13T10:10:00.522 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQBxD+Rmt/L0CBAAV1XXnlWNbL0sc9wUU76uBA== 2024-09-13T10:10:00.522 INFO:teuthology.orchestra.run.smithi181.stdout:[osd.15] 2024-09-13T10:10:00.522 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQB0D+RmDjmiGRAAXrkGwFzsx/vxTesGU+/R+w== 2024-09-13T10:10:00.522 INFO:teuthology.orchestra.run.smithi181.stdout:[client.0] 2024-09-13T10:10:00.523 INFO:teuthology.orchestra.run.smithi181.stdout: key = AQA5D+RmJZMDJhAAzhQsGeBKI4xf/bbmzVtAMA== 2024-09-13T10:10:00.524 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr.y] 2024-09-13T10:10:00.524 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQA5D+Rmj8fWHRAA7NVyZyPzyXZaP6Yjz3NhZg== 2024-09-13T10:10:00.524 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.0] 2024-09-13T10:10:00.524 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQA8D+RmaHnGHRAA44+S8r8FsNkaOGh0pRphTA== 2024-09-13T10:10:00.524 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.4] 2024-09-13T10:10:00.524 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQA/D+RmyvvmLBAAyYqHZ4MDx8wFr8bjT7SgSA== 2024-09-13T10:10:00.524 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.8] 2024-09-13T10:10:00.524 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBDD+Rma/FAAhAATy6ea3sPYe3+b57PDFvYiA== 2024-09-13T10:10:00.524 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.12] 2024-09-13T10:10:00.525 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBGD+Rmo4EtEhAAsoNclUOJUGPgPvsqdr4IDA== 2024-09-13T10:10:00.525 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.1] 2024-09-13T10:10:00.525 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBMD+RmmK8JHRAA2y+ZKEosIYDRLYOjDluhzg== 2024-09-13T10:10:00.525 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.5] 2024-09-13T10:10:00.525 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBPD+RmrtCtLRAAsx3ti151k4KHEoDIH+byEQ== 2024-09-13T10:10:00.525 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.9] 2024-09-13T10:10:00.525 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBTD+Rm1w0HBRAAiqnOpevYaMHQbDi/4AVDtQ== 2024-09-13T10:10:00.525 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.13] 2024-09-13T10:10:00.525 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBWD+Rmv7gVFhAASt2pGkg7CbR3Zh4W6TWE/Q== 2024-09-13T10:10:00.525 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.2] 2024-09-13T10:10:00.525 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBcD+RmySJ+JhAAeDn0+xuGmdJGm1d1QPP/Zw== 2024-09-13T10:10:00.525 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.6] 2024-09-13T10:10:00.526 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBfD+RmSx3yMxAAwbPxuPJK84Hs+rLXaFoSvQ== 2024-09-13T10:10:00.526 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.10] 2024-09-13T10:10:00.526 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBjD+RmV+AtBxAAM3OGg4L7RiHtEUf7wWZFQw== 2024-09-13T10:10:00.526 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.14] 2024-09-13T10:10:00.526 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBmD+Rm2DpYFxAAzml5uia5twxFMYs0zPU3uA== 2024-09-13T10:10:00.526 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr.x] 2024-09-13T10:10:00.526 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQA5D+RmsEGQIBAAtxryI6AMa+mKQBRSW/2XSg== 2024-09-13T10:10:00.526 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.3] 2024-09-13T10:10:00.526 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBqD+RmOjf9JRAAzjwfP39RPlq+fb8uQ82bhw== 2024-09-13T10:10:00.526 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.7] 2024-09-13T10:10:00.526 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBtD+Rm/k6SNBAA7j/2vXrE1MFupLsTvbpjAQ== 2024-09-13T10:10:00.526 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.11] 2024-09-13T10:10:00.526 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBxD+Rmt/L0CBAAV1XXnlWNbL0sc9wUU76uBA== 2024-09-13T10:10:00.527 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.15] 2024-09-13T10:10:00.527 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQB0D+RmDjmiGRAAXrkGwFzsx/vxTesGU+/R+w== 2024-09-13T10:10:00.527 INFO:teuthology.orchestra.run.smithi136.stdout:[client.0] 2024-09-13T10:10:00.527 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQA5D+RmJZMDJhAAzhQsGeBKI4xf/bbmzVtAMA== 2024-09-13T10:10:00.528 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T10:10:00.570 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T10:10:00.573 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T10:10:00.619 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:00.657 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:00.662 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:00.709 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:00.741 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:00.746 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:00.790 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:00.821 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:00.830 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:00.870 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:00.901 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:00.914 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:00.954 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:00.981 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:00.998 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.040 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.069 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.086 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.128 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.157 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.170 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.210 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.241 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.250 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.294 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.321 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.334 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.377 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.405 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.423 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.517 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.520 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.523 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.567 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.610 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.612 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.654 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T10:10:01.693 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T10:10:01.698 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T10:10:01.738 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.777 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.782 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.825 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.862 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.864 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.906 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.908 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.950 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.992 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:01.994 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:02.034 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T10:10:02.076 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T10:10:02.082 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T10:10:02.119 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T10:10:02.167 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T10:10:02.168 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T10:10:02.180 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T10:10:02.279 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T10:10:02.290 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T10:10:02.304 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T10:10:02.401 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T10:10:02.412 DEBUG:teuthology.orchestra.run.smithi181:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T10:10:02.426 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T10:10:02.522 DEBUG:teuthology.orchestra.run.smithi181:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T10:10:02.534 DEBUG:teuthology.orchestra.run.smithi073:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:10:02.536 DEBUG:teuthology.orchestra.run.smithi136:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:10:02.539 DEBUG:teuthology.orchestra.run.smithi181:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T10:10:02.578 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T10:10:02.579 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T10:10:02.579 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T10:10:02.585 INFO:tasks.ceph.mon.a:Started 2024-09-13T10:10:02.585 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T10:10:02.585 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T10:10:02.588 INFO:tasks.ceph.mon.b:Started 2024-09-13T10:10:02.588 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T10:10:02.588 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T10:10:02.624 INFO:tasks.ceph.mon.c:Started 2024-09-13T10:10:02.625 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T10:10:02.625 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T10:10:02.625 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T10:10:02.628 INFO:tasks.ceph.mgr.y:Started 2024-09-13T10:10:02.628 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T10:10:02.629 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T10:10:02.631 INFO:tasks.ceph.mgr.x:Started 2024-09-13T10:10:02.631 DEBUG:tasks.ceph:set 0 configs 2024-09-13T10:10:02.631 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T10:10:02.632 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T10:10:14.233 INFO:teuthology.orchestra.run.smithi073.stderr:adjusted tunables profile to default 2024-09-13T10:10:14.252 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T10:10:14.252 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T10:10:14.253 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:10:14.253 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T10:10:14.264 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:10:14.264 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-13T10:10:14.322 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:10:14.322 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2024-09-13T10:10:14.379 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-13T10:10:14.379 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-12/fsid of=/dev/stdout 2024-09-13T10:10:14.431 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T10:10:14.431 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T10:10:14.446 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T10:10:14.446 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-13T10:10:14.500 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T10:10:14.501 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2024-09-13T10:10:14.556 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-09-13T10:10:14.556 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-13/fsid of=/dev/stdout 2024-09-13T10:10:14.608 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T10:10:14.609 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T10:10:14.623 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T10:10:14.624 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-13T10:10:14.679 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T10:10:14.679 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-10/fsid of=/dev/stdout 2024-09-13T10:10:14.734 DEBUG:teuthology.orchestra.run.smithi181:> set -ex 2024-09-13T10:10:14.735 DEBUG:teuthology.orchestra.run.smithi181:> sudo dd if=/var/lib/ceph/osd/ceph-14/fsid of=/dev/stdout 2024-09-13T10:10:14.790 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:10:14.790 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T10:10:14.803 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:10:14.803 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-13T10:10:14.858 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:10:14.858 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-11/fsid of=/dev/stdout 2024-09-13T10:10:14.912 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-09-13T10:10:14.912 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-15/fsid of=/dev/stdout 2024-09-13T10:10:14.965 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new f7d92188-c27c-4744-a330-4b7105e0f2d6 0 2024-09-13T10:10:15.212 INFO:teuthology.orchestra.run.smithi195.stdout:0 2024-09-13T10:10:15.230 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new e900b771-7c96-4b5d-93d7-35a8ea6c026a 1 2024-09-13T10:10:15.438 INFO:teuthology.orchestra.run.smithi195.stdout:1 2024-09-13T10:10:15.455 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 28b7aa59-33b2-4e24-8f3d-758f90e25f68 2 2024-09-13T10:10:15.665 INFO:teuthology.orchestra.run.smithi195.stdout:2 2024-09-13T10:10:15.682 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new a93d2df5-781e-49e1-a507-1718b6f954ed 3 2024-09-13T10:10:15.890 INFO:teuthology.orchestra.run.smithi195.stdout:3 2024-09-13T10:10:15.907 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new e69053fd-f6d9-437e-a9d0-8896de6b927f 4 2024-09-13T10:10:16.115 INFO:teuthology.orchestra.run.smithi195.stdout:4 2024-09-13T10:10:16.131 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 3667ec10-561e-4a23-8197-6357db4af315 5 2024-09-13T10:10:16.325 INFO:teuthology.orchestra.run.smithi195.stdout:5 2024-09-13T10:10:16.342 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 30d4d63c-f04f-4423-91a6-816a7cbc0287 6 2024-09-13T10:10:16.538 INFO:teuthology.orchestra.run.smithi195.stdout:6 2024-09-13T10:10:16.553 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 48812f05-82f1-4263-8ec5-4861a6e91b61 7 2024-09-13T10:10:16.794 INFO:teuthology.orchestra.run.smithi195.stdout:7 2024-09-13T10:10:16.810 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 72056e61-d04b-4fd3-94bf-57ee3df4b466 8 2024-09-13T10:10:16.976 INFO:tasks.ceph.mgr.x.smithi195.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T10:10:16.976 INFO:tasks.ceph.mgr.x.smithi195.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T10:10:16.976 INFO:tasks.ceph.mgr.x.smithi195.stderr: from numpy import show_config as show_numpy_config 2024-09-13T10:10:16.994 INFO:tasks.ceph.mgr.y.smithi073.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T10:10:16.994 INFO:tasks.ceph.mgr.y.smithi073.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T10:10:16.994 INFO:tasks.ceph.mgr.y.smithi073.stderr: from numpy import show_config as show_numpy_config 2024-09-13T10:10:17.003 INFO:teuthology.orchestra.run.smithi195.stdout:8 2024-09-13T10:10:17.019 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 20e7f118-7148-4c27-9ec4-a33954709515 9 2024-09-13T10:10:17.224 INFO:teuthology.orchestra.run.smithi195.stdout:9 2024-09-13T10:10:17.241 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 012409e8-51c2-4584-aba7-95996f0d72ad 10 2024-09-13T10:10:17.436 INFO:teuthology.orchestra.run.smithi195.stdout:10 2024-09-13T10:10:17.453 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 88b137b1-7d30-45f7-ac5c-ba026704ef24 11 2024-09-13T10:10:17.651 INFO:teuthology.orchestra.run.smithi195.stdout:11 2024-09-13T10:10:17.667 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new b54732a2-9dff-42e1-b4ef-281cbca3ad7c 12 2024-09-13T10:10:17.871 INFO:teuthology.orchestra.run.smithi195.stdout:12 2024-09-13T10:10:17.887 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 20b48864-44dc-4732-8dff-d0d21cc08032 13 2024-09-13T10:10:18.081 INFO:teuthology.orchestra.run.smithi195.stdout:13 2024-09-13T10:10:18.097 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 9fac3ef4-e9be-4e8b-9a21-0de35b543d65 14 2024-09-13T10:10:18.285 INFO:teuthology.orchestra.run.smithi195.stdout:14 2024-09-13T10:10:18.301 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 16b34f90-3148-4d41-a403-ba0f900991e1 15 2024-09-13T10:10:18.490 INFO:teuthology.orchestra.run.smithi195.stdout:15 2024-09-13T10:10:18.506 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T10:10:18.506 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T10:10:18.509 INFO:tasks.ceph.osd.0:Started 2024-09-13T10:10:18.509 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T10:10:18.509 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T10:10:18.512 INFO:tasks.ceph.osd.4:Started 2024-09-13T10:10:18.512 INFO:tasks.ceph.osd.8:Restarting daemon 2024-09-13T10:10:18.512 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2024-09-13T10:10:18.515 INFO:tasks.ceph.osd.8:Started 2024-09-13T10:10:18.515 INFO:tasks.ceph.osd.12:Restarting daemon 2024-09-13T10:10:18.515 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 12 2024-09-13T10:10:18.517 INFO:tasks.ceph.osd.12:Started 2024-09-13T10:10:18.517 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T10:10:18.517 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T10:10:18.520 INFO:tasks.ceph.osd.1:Started 2024-09-13T10:10:18.520 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T10:10:18.520 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T10:10:18.523 INFO:tasks.ceph.osd.5:Started 2024-09-13T10:10:18.523 INFO:tasks.ceph.osd.9:Restarting daemon 2024-09-13T10:10:18.523 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2024-09-13T10:10:18.525 INFO:tasks.ceph.osd.9:Started 2024-09-13T10:10:18.526 INFO:tasks.ceph.osd.13:Restarting daemon 2024-09-13T10:10:18.526 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 13 2024-09-13T10:10:18.529 INFO:tasks.ceph.osd.13:Started 2024-09-13T10:10:18.529 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T10:10:18.529 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T10:10:18.532 INFO:tasks.ceph.osd.2:Started 2024-09-13T10:10:18.532 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-13T10:10:18.532 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-13T10:10:18.535 INFO:tasks.ceph.osd.6:Started 2024-09-13T10:10:18.535 INFO:tasks.ceph.osd.10:Restarting daemon 2024-09-13T10:10:18.535 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 10 2024-09-13T10:10:18.537 INFO:tasks.ceph.osd.10:Started 2024-09-13T10:10:18.537 INFO:tasks.ceph.osd.14:Restarting daemon 2024-09-13T10:10:18.538 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 14 2024-09-13T10:10:18.540 INFO:tasks.ceph.osd.14:Started 2024-09-13T10:10:18.540 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T10:10:18.540 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T10:10:18.542 INFO:tasks.ceph.osd.3:Started 2024-09-13T10:10:18.542 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T10:10:18.542 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T10:10:18.544 INFO:tasks.ceph.osd.7:Started 2024-09-13T10:10:18.544 INFO:tasks.ceph.osd.11:Restarting daemon 2024-09-13T10:10:18.544 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 11 2024-09-13T10:10:18.546 INFO:tasks.ceph.osd.11:Started 2024-09-13T10:10:18.546 INFO:tasks.ceph.osd.15:Restarting daemon 2024-09-13T10:10:18.546 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 15 2024-09-13T10:10:18.548 INFO:tasks.ceph.osd.15:Started 2024-09-13T10:10:18.548 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T10:10:18.734 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:10:18.735 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":18,"fsid":"c1e91d47-9492-4461-a043-8698fd74ccf3","created":"2024-09-13T10:10:14.019912+0000","modified":"2024-09-13T10:10:18.489546+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T10:10:18.489546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f7d92188-c27c-4744-a330-4b7105e0f2d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e900b771-7c96-4b5d-93d7-35a8ea6c026a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"28b7aa59-33b2-4e24-8f3d-758f90e25f68","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a93d2df5-781e-49e1-a507-1718b6f954ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e69053fd-f6d9-437e-a9d0-8896de6b927f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3667ec10-561e-4a23-8197-6357db4af315","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"30d4d63c-f04f-4423-91a6-816a7cbc0287","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"48812f05-82f1-4263-8ec5-4861a6e91b61","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"72056e61-d04b-4fd3-94bf-57ee3df4b466","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"20e7f118-7148-4c27-9ec4-a33954709515","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"012409e8-51c2-4584-aba7-95996f0d72ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"88b137b1-7d30-45f7-ac5c-ba026704ef24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"b54732a2-9dff-42e1-b4ef-281cbca3ad7c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"20b48864-44dc-4732-8dff-d0d21cc08032","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"9fac3ef4-e9be-4e8b-9a21-0de35b543d65","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"16b34f90-3148-4d41-a403-ba0f900991e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T10:10:18.751 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T10:10:18.751 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T10:10:19.763 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:10:19.759+0000 7f3e1d7597c0 -1 Falling back to public interface 2024-09-13T10:10:19.787 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:19.783+0000 7f79a40a37c0 -1 Falling back to public interface 2024-09-13T10:10:19.875 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:10:19.871+0000 7f8dee6b97c0 -1 Falling back to public interface 2024-09-13T10:10:19.886 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:10:19.883+0000 7f7ce1c787c0 -1 Falling back to public interface 2024-09-13T10:10:19.894 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:10:19.891+0000 7f3210ed97c0 -1 Falling back to public interface 2024-09-13T10:10:19.935 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-13T10:10:19.933+0000 7fc6a0050640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T10:10:19.936 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-13T10:10:19.933+0000 7fc6a0050640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T10:10:19.937 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:10:19.933+0000 7ff0faf027c0 -1 Falling back to public interface 2024-09-13T10:10:19.943 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:19.941+0000 7fbb09dc67c0 -1 Falling back to public interface 2024-09-13T10:10:19.944 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:10:19.941+0000 7fafb87bd7c0 -1 Falling back to public interface 2024-09-13T10:10:19.946 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:10:19.943+0000 7faceb42a7c0 -1 Falling back to public interface 2024-09-13T10:10:20.118 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:10:20.115+0000 7fbdf87dc7c0 -1 Falling back to public interface 2024-09-13T10:10:20.132 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:20.129+0000 7fd5dfd307c0 -1 Falling back to public interface 2024-09-13T10:10:20.167 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:10:20.165+0000 7f3e2ad447c0 -1 Falling back to public interface 2024-09-13T10:10:20.177 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:10:20.173+0000 7f57525a37c0 -1 Falling back to public interface 2024-09-13T10:10:20.191 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:10:20.189+0000 7f443049f7c0 -1 Falling back to public interface 2024-09-13T10:10:20.346 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:20.343+0000 7f3ed87117c0 -1 Falling back to public interface 2024-09-13T10:10:20.361 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:20.357+0000 7ff215be97c0 -1 Falling back to public interface 2024-09-13T10:10:20.361 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T10:10:20.662 INFO:teuthology.misc.health.smithi073.stdout: 2024-09-13T10:10:20.663 INFO:teuthology.misc.health.smithi073.stdout:{"epoch":18,"fsid":"c1e91d47-9492-4461-a043-8698fd74ccf3","created":"2024-09-13T10:10:14.019912+0000","modified":"2024-09-13T10:10:18.489546+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T10:10:18.489546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f7d92188-c27c-4744-a330-4b7105e0f2d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e900b771-7c96-4b5d-93d7-35a8ea6c026a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"28b7aa59-33b2-4e24-8f3d-758f90e25f68","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a93d2df5-781e-49e1-a507-1718b6f954ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e69053fd-f6d9-437e-a9d0-8896de6b927f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3667ec10-561e-4a23-8197-6357db4af315","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"30d4d63c-f04f-4423-91a6-816a7cbc0287","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"48812f05-82f1-4263-8ec5-4861a6e91b61","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"72056e61-d04b-4fd3-94bf-57ee3df4b466","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"20e7f118-7148-4c27-9ec4-a33954709515","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"012409e8-51c2-4584-aba7-95996f0d72ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"88b137b1-7d30-45f7-ac5c-ba026704ef24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"b54732a2-9dff-42e1-b4ef-281cbca3ad7c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"20b48864-44dc-4732-8dff-d0d21cc08032","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"9fac3ef4-e9be-4e8b-9a21-0de35b543d65","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"16b34f90-3148-4d41-a403-ba0f900991e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T10:10:20.679 DEBUG:teuthology.misc:0 of 16 OSDs are up 2024-09-13T10:10:23.722 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:10:23.719+0000 7f3e1d7597c0 -1 osd.9 0 log_to_monitors true 2024-09-13T10:10:23.902 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:10:23.899+0000 7f7ce1c787c0 -1 osd.13 0 log_to_monitors true 2024-09-13T10:10:23.937 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:10:23.935+0000 7faceb42a7c0 -1 osd.8 0 log_to_monitors true 2024-09-13T10:10:23.957 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:23.953+0000 7fbb09dc67c0 -1 osd.3 0 log_to_monitors true 2024-09-13T10:10:23.980 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:10:23.977+0000 7fafb87bd7c0 -1 osd.14 0 log_to_monitors true 2024-09-13T10:10:24.031 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:24.027+0000 7f79a40a37c0 -1 osd.5 0 log_to_monitors true 2024-09-13T10:10:24.077 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:10:24.075+0000 7f3210ed97c0 -1 osd.4 0 log_to_monitors true 2024-09-13T10:10:24.092 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:24.089+0000 7fd5dfd307c0 -1 osd.15 0 log_to_monitors true 2024-09-13T10:10:24.227 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:10:24.225+0000 7ff0faf027c0 -1 osd.2 0 log_to_monitors true 2024-09-13T10:10:24.289 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:10:24.285+0000 7f3e2ad447c0 -1 osd.7 0 log_to_monitors true 2024-09-13T10:10:24.359 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:24.355+0000 7f3ed87117c0 -1 osd.0 0 log_to_monitors true 2024-09-13T10:10:24.393 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:24.389+0000 7ff215be97c0 -1 osd.6 0 log_to_monitors true 2024-09-13T10:10:24.416 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:10:24.411+0000 7fbdf87dc7c0 -1 osd.12 0 log_to_monitors true 2024-09-13T10:10:24.453 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:10:24.449+0000 7f57525a37c0 -1 osd.10 0 log_to_monitors true 2024-09-13T10:10:24.501 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:10:24.497+0000 7f443049f7c0 -1 osd.11 0 log_to_monitors true 2024-09-13T10:10:24.600 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:10:24.595+0000 7f8dee6b97c0 -1 osd.1 0 log_to_monitors true 2024-09-13T10:10:24.953 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:10:24.947+0000 7f3e19702640 -1 osd.9 0 waiting for initial osdmap 2024-09-13T10:10:24.958 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:10:24.955+0000 7f7cddc21640 -1 osd.13 0 waiting for initial osdmap 2024-09-13T10:10:24.965 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:10:24.959+0000 7f3e1553c640 -1 osd.9 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:10:24.969 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:10:24.963+0000 7f7cd9a5b640 -1 osd.13 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:10:25.956 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:25.955+0000 7f79a004c640 -1 osd.5 0 waiting for initial osdmap 2024-09-13T10:10:25.957 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:25.952+0000 7fbb05d6f640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T10:10:25.957 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:10:25.952+0000 7f3e26ced640 -1 osd.7 0 waiting for initial osdmap 2024-09-13T10:10:25.959 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:25.956+0000 7fd5dbcd9640 -1 osd.15 0 waiting for initial osdmap 2024-09-13T10:10:25.962 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:10:25.957+0000 7f574e54c640 -1 osd.10 0 waiting for initial osdmap 2024-09-13T10:10:25.962 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:25.963+0000 7f799be86640 -1 osd.5 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:10:25.966 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:25.960+0000 7fbb01ba9640 -1 osd.3 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:10:25.968 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:10:25.964+0000 7f3e22b27640 -1 osd.7 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:10:25.972 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:25.968+0000 7fd5d7b13640 -1 osd.15 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:10:25.977 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:10:25.975+0000 7f320ce82640 -1 osd.4 0 waiting for initial osdmap 2024-09-13T10:10:26.117 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:10:26.115+0000 7f3208cbc640 -1 osd.4 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:10:26.177 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:10:26.173+0000 7f574a386640 -1 osd.10 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:10:26.555 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:10:26.552+0000 7f442c448640 -1 osd.11 0 waiting for initial osdmap 2024-09-13T10:10:26.565 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:10:26.560+0000 7f442427a640 -1 osd.11 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:10:26.747 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:10:26.747+0000 7f8dea662640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T10:10:26.756 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:10:26.755+0000 7f8de2494640 -1 osd.1 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:10:27.602 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:10:27.593+0000 7fafb4766640 -1 osd.14 0 waiting for initial osdmap 2024-09-13T10:10:27.610 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:10:27.605+0000 7fafac598640 -1 osd.14 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:10:27.647 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:10:27.643+0000 7face7be5640 -1 osd.8 0 waiting for initial osdmap 2024-09-13T10:10:27.658 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:10:27.655+0000 7facdf205640 -1 osd.8 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:10:27.664 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:10:27.661+0000 7ff0f6eab640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T10:10:27.671 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:10:27.665+0000 7ff0eecdd640 -1 osd.2 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:10:27.691 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:27.685+0000 7ff2123a4640 -1 osd.6 0 waiting for initial osdmap 2024-09-13T10:10:27.691 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:27.685+0000 7ff211b92640 -1 osd.6 0 waiting for initial osdmap 2024-09-13T10:10:27.695 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:27.689+0000 7ff2099c4640 -1 osd.6 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:10:27.710 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:10:27.707+0000 7fbdf4785640 -1 osd.12 0 waiting for initial osdmap 2024-09-13T10:10:27.716 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:10:27.715+0000 7fbdec5b7640 -1 osd.12 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:10:27.738 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:27.735+0000 7f3ed4ecc640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T10:10:27.744 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:27.743+0000 7f3ecc4ec640 -1 osd.0 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T10:10:28.289 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T10:10:28.586 INFO:teuthology.misc.health.smithi073.stdout: 2024-09-13T10:10:28.586 INFO:teuthology.misc.health.smithi073.stdout:{"epoch":23,"fsid":"c1e91d47-9492-4461-a043-8698fd74ccf3","created":"2024-09-13T10:10:14.019912+0000","modified":"2024-09-13T10:10:28.089885+0000","last_up_change":"2024-09-13T10:10:28.089885+0000","last_in_change":"2024-09-13T10:10:18.489546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f7d92188-c27c-4744-a330-4b7105e0f2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1977392321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":1977392321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6815","nonce":1977392321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1977392321}]},"public_addr":"172.21.15.73:6812/1977392321","cluster_addr":"172.21.15.73:6813/1977392321","heartbeat_back_addr":"172.21.15.73:6815/1977392321","heartbeat_front_addr":"172.21.15.73:6814/1977392321","state":["exists","up"]},{"osd":1,"uuid":"e900b771-7c96-4b5d-93d7-35a8ea6c026a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1228102864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6809","nonce":1228102864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6811","nonce":1228102864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1228102864}]},"public_addr":"172.21.15.136:6808/1228102864","cluster_addr":"172.21.15.136:6809/1228102864","heartbeat_back_addr":"172.21.15.136:6811/1228102864","heartbeat_front_addr":"172.21.15.136:6810/1228102864","state":["exists","up"]},{"osd":2,"uuid":"28b7aa59-33b2-4e24-8f3d-758f90e25f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3314820936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":3314820936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6803","nonce":3314820936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3314820936}]},"public_addr":"172.21.15.181:6800/3314820936","cluster_addr":"172.21.15.181:6801/3314820936","heartbeat_back_addr":"172.21.15.181:6803/3314820936","heartbeat_front_addr":"172.21.15.181:6802/3314820936","state":["exists","up"]},{"osd":3,"uuid":"a93d2df5-781e-49e1-a507-1718b6f954ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6801","nonce":2558565460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2558565460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2558565460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":2558565460}]},"public_addr":"172.21.15.195:6801/2558565460","cluster_addr":"172.21.15.195:6802/2558565460","heartbeat_back_addr":"172.21.15.195:6804/2558565460","heartbeat_front_addr":"172.21.15.195:6803/2558565460","state":["exists","up"]},{"osd":4,"uuid":"e69053fd-f6d9-437e-a9d0-8896de6b927f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":858330493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6801","nonce":858330493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6803","nonce":858330493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":858330493}]},"public_addr":"172.21.15.73:6800/858330493","cluster_addr":"172.21.15.73:6801/858330493","heartbeat_back_addr":"172.21.15.73:6803/858330493","heartbeat_front_addr":"172.21.15.73:6802/858330493","state":["exists","up"]},{"osd":5,"uuid":"3667ec10-561e-4a23-8197-6357db4af315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":984313481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6805","nonce":984313481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6807","nonce":984313481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":984313481}]},"public_addr":"172.21.15.136:6804/984313481","cluster_addr":"172.21.15.136:6805/984313481","heartbeat_back_addr":"172.21.15.136:6807/984313481","heartbeat_front_addr":"172.21.15.136:6806/984313481","state":["exists","up"]},{"osd":6,"uuid":"30d4d63c-f04f-4423-91a6-816a7cbc0287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1505939978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":1505939978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":1505939978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1505939978}]},"public_addr":"172.21.15.181:6812/1505939978","cluster_addr":"172.21.15.181:6813/1505939978","heartbeat_back_addr":"172.21.15.181:6815/1505939978","heartbeat_front_addr":"172.21.15.181:6814/1505939978","state":["exists","up"]},{"osd":7,"uuid":"48812f05-82f1-4263-8ec5-4861a6e91b61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":3679292757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3679292757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3679292757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6811","nonce":3679292757}]},"public_addr":"172.21.15.195:6809/3679292757","cluster_addr":"172.21.15.195:6810/3679292757","heartbeat_back_addr":"172.21.15.195:6812/3679292757","heartbeat_front_addr":"172.21.15.195:6811/3679292757","state":["exists","up"]},{"osd":8,"uuid":"72056e61-d04b-4fd3-94bf-57ee3df4b466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4052106885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6805","nonce":4052106885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":4052106885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4052106885}]},"public_addr":"172.21.15.73:6804/4052106885","cluster_addr":"172.21.15.73:6805/4052106885","heartbeat_back_addr":"172.21.15.73:6807/4052106885","heartbeat_front_addr":"172.21.15.73:6806/4052106885","state":["exists","up"]},{"osd":9,"uuid":"20e7f118-7148-4c27-9ec4-a33954709515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4266571820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6801","nonce":4266571820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6803","nonce":4266571820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4266571820}]},"public_addr":"172.21.15.136:6800/4266571820","cluster_addr":"172.21.15.136:6801/4266571820","heartbeat_back_addr":"172.21.15.136:6803/4266571820","heartbeat_front_addr":"172.21.15.136:6802/4266571820","state":["exists","up"]},{"osd":10,"uuid":"012409e8-51c2-4584-aba7-95996f0d72ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":958255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":958255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":958255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":958255259}]},"public_addr":"172.21.15.181:6808/958255259","cluster_addr":"172.21.15.181:6809/958255259","heartbeat_back_addr":"172.21.15.181:6811/958255259","heartbeat_front_addr":"172.21.15.181:6810/958255259","state":["exists","up"]},{"osd":11,"uuid":"88b137b1-7d30-45f7-ac5c-ba026704ef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":281884267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":281884267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":281884267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6815","nonce":281884267}]},"public_addr":"172.21.15.195:6813/281884267","cluster_addr":"172.21.15.195:6814/281884267","heartbeat_back_addr":"172.21.15.195:6816/281884267","heartbeat_front_addr":"172.21.15.195:6815/281884267","state":["exists","up"]},{"osd":12,"uuid":"b54732a2-9dff-42e1-b4ef-281cbca3ad7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3532043806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6809","nonce":3532043806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6811","nonce":3532043806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3532043806}]},"public_addr":"172.21.15.73:6808/3532043806","cluster_addr":"172.21.15.73:6809/3532043806","heartbeat_back_addr":"172.21.15.73:6811/3532043806","heartbeat_front_addr":"172.21.15.73:6810/3532043806","state":["exists","up"]},{"osd":13,"uuid":"20b48864-44dc-4732-8dff-d0d21cc08032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":4083982131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":4083982131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6815","nonce":4083982131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":4083982131}]},"public_addr":"172.21.15.136:6812/4083982131","cluster_addr":"172.21.15.136:6813/4083982131","heartbeat_back_addr":"172.21.15.136:6815/4083982131","heartbeat_front_addr":"172.21.15.136:6814/4083982131","state":["exists","up"]},{"osd":14,"uuid":"9fac3ef4-e9be-4e8b-9a21-0de35b543d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1173379175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":1173379175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":1173379175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1173379175}]},"public_addr":"172.21.15.181:6804/1173379175","cluster_addr":"172.21.15.181:6805/1173379175","heartbeat_back_addr":"172.21.15.181:6807/1173379175","heartbeat_front_addr":"172.21.15.181:6806/1173379175","state":["exists","up"]},{"osd":15,"uuid":"16b34f90-3148-4d41-a403-ba0f900991e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6805","nonce":848686366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":848686366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":848686366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":848686366}]},"public_addr":"172.21.15.195:6805/848686366","cluster_addr":"172.21.15.195:6806/848686366","heartbeat_back_addr":"172.21.15.195:6808/848686366","heartbeat_front_addr":"172.21.15.195:6807/848686366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.641263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:24.991965+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.066350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.003317+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.263941+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:24.743520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.407426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.542266+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:24.923863+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.070162+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T10:10:28.602 DEBUG:teuthology.misc:16 of 16 OSDs are up 2024-09-13T10:10:28.603 INFO:tasks.ceph:Creating RBD pool 2024-09-13T10:10:28.603 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-13T10:10:29.108 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'rbd' created 2024-09-13T10:10:29.136 DEBUG:teuthology.orchestra.run.smithi073:> rbd --cluster ceph pool init rbd 2024-09-13T10:10:32.217 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T10:10:32.218 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-13T10:10:32.218 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T10:10:32.535 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:10:32.553 INFO:teuthology.orchestra.run.smithi073.stdout:[{"version":9,"timestamp":"2024-09-13T10:10:27.741079+0000","name":"","changes":[{"name":"osd.0/osd_mclock_max_capacity_iops_ssd","new_value":"4881.182603"}]}] 2024-09-13T10:10:32.554 INFO:tasks.ceph_manager:config epoch is 9 2024-09-13T10:10:32.554 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T10:10:32.554 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T10:10:32.554 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T10:10:32.916 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:10:32.935 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":5,"flags":0,"active_gid":4114,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":3870791665}]},"active_addr":"172.21.15.195:6800/3870791665","active_change":"2024-09-13T10:10:18.915813+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":965773991}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":2444672215}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":4292031379}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":1745821161}]}]} 2024-09-13T10:10:32.942 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T10:10:32.942 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T10:10:32.942 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T10:10:33.249 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:10:33.249 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":27,"fsid":"c1e91d47-9492-4461-a043-8698fd74ccf3","created":"2024-09-13T10:10:14.019912+0000","modified":"2024-09-13T10:10:32.128991+0000","last_up_change":"2024-09-13T10:10:28.089885+0000","last_in_change":"2024-09-13T10:10:18.489546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T10:10:28.924860+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":27,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":0.68999999761581421,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T10:10:28.972519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.789999961853027,"score_stable":15.789999961853027,"optimal_score":0.18999999761581421,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7d92188-c27c-4744-a330-4b7105e0f2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1977392321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":1977392321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6815","nonce":1977392321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1977392321}]},"public_addr":"172.21.15.73:6812/1977392321","cluster_addr":"172.21.15.73:6813/1977392321","heartbeat_back_addr":"172.21.15.73:6815/1977392321","heartbeat_front_addr":"172.21.15.73:6814/1977392321","state":["exists","up"]},{"osd":1,"uuid":"e900b771-7c96-4b5d-93d7-35a8ea6c026a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1228102864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6809","nonce":1228102864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6811","nonce":1228102864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1228102864}]},"public_addr":"172.21.15.136:6808/1228102864","cluster_addr":"172.21.15.136:6809/1228102864","heartbeat_back_addr":"172.21.15.136:6811/1228102864","heartbeat_front_addr":"172.21.15.136:6810/1228102864","state":["exists","up"]},{"osd":2,"uuid":"28b7aa59-33b2-4e24-8f3d-758f90e25f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3314820936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":3314820936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6803","nonce":3314820936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3314820936}]},"public_addr":"172.21.15.181:6800/3314820936","cluster_addr":"172.21.15.181:6801/3314820936","heartbeat_back_addr":"172.21.15.181:6803/3314820936","heartbeat_front_addr":"172.21.15.181:6802/3314820936","state":["exists","up"]},{"osd":3,"uuid":"a93d2df5-781e-49e1-a507-1718b6f954ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6801","nonce":2558565460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2558565460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2558565460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":2558565460}]},"public_addr":"172.21.15.195:6801/2558565460","cluster_addr":"172.21.15.195:6802/2558565460","heartbeat_back_addr":"172.21.15.195:6804/2558565460","heartbeat_front_addr":"172.21.15.195:6803/2558565460","state":["exists","up"]},{"osd":4,"uuid":"e69053fd-f6d9-437e-a9d0-8896de6b927f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":858330493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6801","nonce":858330493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6803","nonce":858330493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":858330493}]},"public_addr":"172.21.15.73:6800/858330493","cluster_addr":"172.21.15.73:6801/858330493","heartbeat_back_addr":"172.21.15.73:6803/858330493","heartbeat_front_addr":"172.21.15.73:6802/858330493","state":["exists","up"]},{"osd":5,"uuid":"3667ec10-561e-4a23-8197-6357db4af315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":984313481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6805","nonce":984313481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6807","nonce":984313481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":984313481}]},"public_addr":"172.21.15.136:6804/984313481","cluster_addr":"172.21.15.136:6805/984313481","heartbeat_back_addr":"172.21.15.136:6807/984313481","heartbeat_front_addr":"172.21.15.136:6806/984313481","state":["exists","up"]},{"osd":6,"uuid":"30d4d63c-f04f-4423-91a6-816a7cbc0287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1505939978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":1505939978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":1505939978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1505939978}]},"public_addr":"172.21.15.181:6812/1505939978","cluster_addr":"172.21.15.181:6813/1505939978","heartbeat_back_addr":"172.21.15.181:6815/1505939978","heartbeat_front_addr":"172.21.15.181:6814/1505939978","state":["exists","up"]},{"osd":7,"uuid":"48812f05-82f1-4263-8ec5-4861a6e91b61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":3679292757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3679292757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3679292757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6811","nonce":3679292757}]},"public_addr":"172.21.15.195:6809/3679292757","cluster_addr":"172.21.15.195:6810/3679292757","heartbeat_back_addr":"172.21.15.195:6812/3679292757","heartbeat_front_addr":"172.21.15.195:6811/3679292757","state":["exists","up"]},{"osd":8,"uuid":"72056e61-d04b-4fd3-94bf-57ee3df4b466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4052106885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6805","nonce":4052106885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":4052106885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4052106885}]},"public_addr":"172.21.15.73:6804/4052106885","cluster_addr":"172.21.15.73:6805/4052106885","heartbeat_back_addr":"172.21.15.73:6807/4052106885","heartbeat_front_addr":"172.21.15.73:6806/4052106885","state":["exists","up"]},{"osd":9,"uuid":"20e7f118-7148-4c27-9ec4-a33954709515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4266571820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6801","nonce":4266571820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6803","nonce":4266571820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4266571820}]},"public_addr":"172.21.15.136:6800/4266571820","cluster_addr":"172.21.15.136:6801/4266571820","heartbeat_back_addr":"172.21.15.136:6803/4266571820","heartbeat_front_addr":"172.21.15.136:6802/4266571820","state":["exists","up"]},{"osd":10,"uuid":"012409e8-51c2-4584-aba7-95996f0d72ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":958255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":958255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":958255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":958255259}]},"public_addr":"172.21.15.181:6808/958255259","cluster_addr":"172.21.15.181:6809/958255259","heartbeat_back_addr":"172.21.15.181:6811/958255259","heartbeat_front_addr":"172.21.15.181:6810/958255259","state":["exists","up"]},{"osd":11,"uuid":"88b137b1-7d30-45f7-ac5c-ba026704ef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":281884267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":281884267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":281884267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6815","nonce":281884267}]},"public_addr":"172.21.15.195:6813/281884267","cluster_addr":"172.21.15.195:6814/281884267","heartbeat_back_addr":"172.21.15.195:6816/281884267","heartbeat_front_addr":"172.21.15.195:6815/281884267","state":["exists","up"]},{"osd":12,"uuid":"b54732a2-9dff-42e1-b4ef-281cbca3ad7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3532043806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6809","nonce":3532043806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6811","nonce":3532043806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3532043806}]},"public_addr":"172.21.15.73:6808/3532043806","cluster_addr":"172.21.15.73:6809/3532043806","heartbeat_back_addr":"172.21.15.73:6811/3532043806","heartbeat_front_addr":"172.21.15.73:6810/3532043806","state":["exists","up"]},{"osd":13,"uuid":"20b48864-44dc-4732-8dff-d0d21cc08032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":4083982131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":4083982131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6815","nonce":4083982131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":4083982131}]},"public_addr":"172.21.15.136:6812/4083982131","cluster_addr":"172.21.15.136:6813/4083982131","heartbeat_back_addr":"172.21.15.136:6815/4083982131","heartbeat_front_addr":"172.21.15.136:6814/4083982131","state":["exists","up"]},{"osd":14,"uuid":"9fac3ef4-e9be-4e8b-9a21-0de35b543d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1173379175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":1173379175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":1173379175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1173379175}]},"public_addr":"172.21.15.181:6804/1173379175","cluster_addr":"172.21.15.181:6805/1173379175","heartbeat_back_addr":"172.21.15.181:6807/1173379175","heartbeat_front_addr":"172.21.15.181:6806/1173379175","state":["exists","up"]},{"osd":15,"uuid":"16b34f90-3148-4d41-a403-ba0f900991e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6805","nonce":848686366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":848686366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":848686366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":848686366}]},"public_addr":"172.21.15.195:6805/848686366","cluster_addr":"172.21.15.195:6806/848686366","heartbeat_back_addr":"172.21.15.195:6808/848686366","heartbeat_front_addr":"172.21.15.195:6807/848686366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.372908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.641263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.237394+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:24.991965+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.066350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.003317+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.353521+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.263941+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:24.922976+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:24.743520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.407426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.542266+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.457347+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:24.923863+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:24.980077+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.070162+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T10:10:33.268 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T10:10:33.268 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T10:10:33.564 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:10:33.565 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":27,"fsid":"c1e91d47-9492-4461-a043-8698fd74ccf3","created":"2024-09-13T10:10:14.019912+0000","modified":"2024-09-13T10:10:32.128991+0000","last_up_change":"2024-09-13T10:10:28.089885+0000","last_in_change":"2024-09-13T10:10:18.489546+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T10:10:28.924860+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":27,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":0.68999999761581421,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T10:10:28.972519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.789999961853027,"score_stable":15.789999961853027,"optimal_score":0.18999999761581421,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7d92188-c27c-4744-a330-4b7105e0f2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1977392321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":1977392321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6815","nonce":1977392321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1977392321}]},"public_addr":"172.21.15.73:6812/1977392321","cluster_addr":"172.21.15.73:6813/1977392321","heartbeat_back_addr":"172.21.15.73:6815/1977392321","heartbeat_front_addr":"172.21.15.73:6814/1977392321","state":["exists","up"]},{"osd":1,"uuid":"e900b771-7c96-4b5d-93d7-35a8ea6c026a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1228102864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6809","nonce":1228102864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6811","nonce":1228102864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1228102864}]},"public_addr":"172.21.15.136:6808/1228102864","cluster_addr":"172.21.15.136:6809/1228102864","heartbeat_back_addr":"172.21.15.136:6811/1228102864","heartbeat_front_addr":"172.21.15.136:6810/1228102864","state":["exists","up"]},{"osd":2,"uuid":"28b7aa59-33b2-4e24-8f3d-758f90e25f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3314820936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":3314820936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6803","nonce":3314820936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3314820936}]},"public_addr":"172.21.15.181:6800/3314820936","cluster_addr":"172.21.15.181:6801/3314820936","heartbeat_back_addr":"172.21.15.181:6803/3314820936","heartbeat_front_addr":"172.21.15.181:6802/3314820936","state":["exists","up"]},{"osd":3,"uuid":"a93d2df5-781e-49e1-a507-1718b6f954ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6801","nonce":2558565460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2558565460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2558565460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":2558565460}]},"public_addr":"172.21.15.195:6801/2558565460","cluster_addr":"172.21.15.195:6802/2558565460","heartbeat_back_addr":"172.21.15.195:6804/2558565460","heartbeat_front_addr":"172.21.15.195:6803/2558565460","state":["exists","up"]},{"osd":4,"uuid":"e69053fd-f6d9-437e-a9d0-8896de6b927f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":858330493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6801","nonce":858330493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6803","nonce":858330493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":858330493}]},"public_addr":"172.21.15.73:6800/858330493","cluster_addr":"172.21.15.73:6801/858330493","heartbeat_back_addr":"172.21.15.73:6803/858330493","heartbeat_front_addr":"172.21.15.73:6802/858330493","state":["exists","up"]},{"osd":5,"uuid":"3667ec10-561e-4a23-8197-6357db4af315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":984313481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6805","nonce":984313481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6807","nonce":984313481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":984313481}]},"public_addr":"172.21.15.136:6804/984313481","cluster_addr":"172.21.15.136:6805/984313481","heartbeat_back_addr":"172.21.15.136:6807/984313481","heartbeat_front_addr":"172.21.15.136:6806/984313481","state":["exists","up"]},{"osd":6,"uuid":"30d4d63c-f04f-4423-91a6-816a7cbc0287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1505939978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":1505939978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":1505939978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1505939978}]},"public_addr":"172.21.15.181:6812/1505939978","cluster_addr":"172.21.15.181:6813/1505939978","heartbeat_back_addr":"172.21.15.181:6815/1505939978","heartbeat_front_addr":"172.21.15.181:6814/1505939978","state":["exists","up"]},{"osd":7,"uuid":"48812f05-82f1-4263-8ec5-4861a6e91b61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":3679292757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3679292757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3679292757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6811","nonce":3679292757}]},"public_addr":"172.21.15.195:6809/3679292757","cluster_addr":"172.21.15.195:6810/3679292757","heartbeat_back_addr":"172.21.15.195:6812/3679292757","heartbeat_front_addr":"172.21.15.195:6811/3679292757","state":["exists","up"]},{"osd":8,"uuid":"72056e61-d04b-4fd3-94bf-57ee3df4b466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4052106885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6805","nonce":4052106885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":4052106885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4052106885}]},"public_addr":"172.21.15.73:6804/4052106885","cluster_addr":"172.21.15.73:6805/4052106885","heartbeat_back_addr":"172.21.15.73:6807/4052106885","heartbeat_front_addr":"172.21.15.73:6806/4052106885","state":["exists","up"]},{"osd":9,"uuid":"20e7f118-7148-4c27-9ec4-a33954709515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4266571820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6801","nonce":4266571820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6803","nonce":4266571820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4266571820}]},"public_addr":"172.21.15.136:6800/4266571820","cluster_addr":"172.21.15.136:6801/4266571820","heartbeat_back_addr":"172.21.15.136:6803/4266571820","heartbeat_front_addr":"172.21.15.136:6802/4266571820","state":["exists","up"]},{"osd":10,"uuid":"012409e8-51c2-4584-aba7-95996f0d72ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":958255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":958255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":958255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":958255259}]},"public_addr":"172.21.15.181:6808/958255259","cluster_addr":"172.21.15.181:6809/958255259","heartbeat_back_addr":"172.21.15.181:6811/958255259","heartbeat_front_addr":"172.21.15.181:6810/958255259","state":["exists","up"]},{"osd":11,"uuid":"88b137b1-7d30-45f7-ac5c-ba026704ef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":281884267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":281884267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":281884267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6815","nonce":281884267}]},"public_addr":"172.21.15.195:6813/281884267","cluster_addr":"172.21.15.195:6814/281884267","heartbeat_back_addr":"172.21.15.195:6816/281884267","heartbeat_front_addr":"172.21.15.195:6815/281884267","state":["exists","up"]},{"osd":12,"uuid":"b54732a2-9dff-42e1-b4ef-281cbca3ad7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3532043806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6809","nonce":3532043806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6811","nonce":3532043806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3532043806}]},"public_addr":"172.21.15.73:6808/3532043806","cluster_addr":"172.21.15.73:6809/3532043806","heartbeat_back_addr":"172.21.15.73:6811/3532043806","heartbeat_front_addr":"172.21.15.73:6810/3532043806","state":["exists","up"]},{"osd":13,"uuid":"20b48864-44dc-4732-8dff-d0d21cc08032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":4083982131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":4083982131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6815","nonce":4083982131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":4083982131}]},"public_addr":"172.21.15.136:6812/4083982131","cluster_addr":"172.21.15.136:6813/4083982131","heartbeat_back_addr":"172.21.15.136:6815/4083982131","heartbeat_front_addr":"172.21.15.136:6814/4083982131","state":["exists","up"]},{"osd":14,"uuid":"9fac3ef4-e9be-4e8b-9a21-0de35b543d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1173379175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":1173379175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":1173379175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1173379175}]},"public_addr":"172.21.15.181:6804/1173379175","cluster_addr":"172.21.15.181:6805/1173379175","heartbeat_back_addr":"172.21.15.181:6807/1173379175","heartbeat_front_addr":"172.21.15.181:6806/1173379175","state":["exists","up"]},{"osd":15,"uuid":"16b34f90-3148-4d41-a403-ba0f900991e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6805","nonce":848686366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":848686366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":848686366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":848686366}]},"public_addr":"172.21.15.195:6805/848686366","cluster_addr":"172.21.15.195:6806/848686366","heartbeat_back_addr":"172.21.15.195:6808/848686366","heartbeat_front_addr":"172.21.15.195:6807/848686366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.372908+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.641263+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.237394+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:24.991965+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.066350+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.003317+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.353521+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.263941+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:24.922976+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:24.743520+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.407426+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.542266+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.457347+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:24.923863+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:24.980077+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:10:25.070162+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T10:10:33.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T10:10:33.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T10:10:33.584 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T10:10:33.584 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T10:10:33.584 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T10:10:33.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T10:10:33.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T10:10:33.586 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T10:10:33.586 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-13T10:10:33.587 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-13T10:10:33.587 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-09-13T10:10:33.588 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-09-13T10:10:33.588 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-09-13T10:10:33.588 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-09-13T10:10:33.589 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-09-13T10:10:33.589 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-09-13T10:10:33.985 INFO:teuthology.orchestra.run.smithi073.stdout:94489280517 2024-09-13T10:10:33.986 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T10:10:34.038 INFO:teuthology.orchestra.run.smithi073.stdout:98784247812 2024-09-13T10:10:34.038 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T10:10:34.046 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-09-13T10:10:34.046 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-09-13T10:10:34.047 INFO:teuthology.orchestra.run.smithi073.stdout:98784247811 2024-09-13T10:10:34.048 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T10:10:34.067 INFO:teuthology.orchestra.run.smithi073.stdout:98784247814 2024-09-13T10:10:34.067 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-13T10:10:34.084 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-09-13T10:10:34.085 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T10:10:34.086 INFO:teuthology.orchestra.run.smithi073.stdout:90194313220 2024-09-13T10:10:34.086 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-09-13T10:10:34.096 INFO:teuthology.orchestra.run.smithi073.stdout:94489280515 2024-09-13T10:10:34.096 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T10:10:34.126 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-09-13T10:10:34.126 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-13T10:10:34.127 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-09-13T10:10:34.127 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T10:10:34.141 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-09-13T10:10:34.141 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T10:10:34.147 INFO:teuthology.orchestra.run.smithi073.stdout:98784247812 2024-09-13T10:10:34.147 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-09-13T10:10:34.165 INFO:teuthology.orchestra.run.smithi073.stdout:90194313221 2024-09-13T10:10:34.166 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-13T10:10:34.167 INFO:teuthology.orchestra.run.smithi073.stdout:98784247812 2024-09-13T10:10:34.168 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T10:10:34.185 INFO:teuthology.orchestra.run.smithi073.stdout:98784247814 2024-09-13T10:10:34.186 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-09-13T10:10:34.232 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-09-13T10:10:34.232 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-13T10:10:34.863 INFO:teuthology.orchestra.run.smithi073.stdout:94489280515 2024-09-13T10:10:34.928 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280515 for osd.1 2024-09-13T10:10:34.928 INFO:teuthology.orchestra.run.smithi073.stdout:90194313219 2024-09-13T10:10:34.991 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313219 for osd.13 2024-09-13T10:10:35.044 INFO:teuthology.orchestra.run.smithi073.stdout:94489280514 2024-09-13T10:10:35.053 INFO:teuthology.orchestra.run.smithi073.stdout:94489280514 2024-09-13T10:10:35.092 INFO:teuthology.orchestra.run.smithi073.stdout:98784247810 2024-09-13T10:10:35.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280514 for osd.15 2024-09-13T10:10:35.112 INFO:teuthology.orchestra.run.smithi073.stdout:94489280513 2024-09-13T10:10:35.115 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280514 for osd.3 2024-09-13T10:10:35.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247810 for osd.2 2024-09-13T10:10:35.146 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280513 for osd.5 2024-09-13T10:10:35.165 INFO:teuthology.orchestra.run.smithi073.stdout:98784247809 2024-09-13T10:10:35.173 INFO:teuthology.orchestra.run.smithi073.stdout:94489280517 2024-09-13T10:10:35.206 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-09-13T10:10:35.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247811 got 98784247809 for osd.6 2024-09-13T10:10:35.212 INFO:teuthology.orchestra.run.smithi073.stdout:98784247812 2024-09-13T10:10:35.224 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-09-13T10:10:35.225 INFO:teuthology.orchestra.run.smithi073.stdout:90194313221 2024-09-13T10:10:35.231 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.4 2024-09-13T10:10:35.231 DEBUG:teuthology.parallel:result is None 2024-09-13T10:10:35.238 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.10 2024-09-13T10:10:35.238 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-09-13T10:10:35.238 DEBUG:teuthology.parallel:result is None 2024-09-13T10:10:35.243 INFO:teuthology.orchestra.run.smithi073.stdout:98784247812 2024-09-13T10:10:35.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.7 2024-09-13T10:10:35.255 DEBUG:teuthology.parallel:result is None 2024-09-13T10:10:35.260 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.0 2024-09-13T10:10:35.260 INFO:teuthology.orchestra.run.smithi073.stdout:98784247814 2024-09-13T10:10:35.260 DEBUG:teuthology.parallel:result is None 2024-09-13T10:10:35.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.9 2024-09-13T10:10:35.261 DEBUG:teuthology.parallel:result is None 2024-09-13T10:10:35.263 INFO:teuthology.orchestra.run.smithi073.stdout:98784247814 2024-09-13T10:10:35.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.11 2024-09-13T10:10:35.272 DEBUG:teuthology.parallel:result is None 2024-09-13T10:10:35.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.14 2024-09-13T10:10:35.277 DEBUG:teuthology.parallel:result is None 2024-09-13T10:10:35.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247814 got 98784247814 for osd.12 2024-09-13T10:10:35.284 DEBUG:teuthology.parallel:result is None 2024-09-13T10:10:35.287 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247814 got 98784247814 for osd.8 2024-09-13T10:10:35.288 DEBUG:teuthology.parallel:result is None 2024-09-13T10:10:35.929 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T10:10:35.992 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-09-13T10:10:36.093 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-09-13T10:10:36.116 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T10:10:36.135 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T10:10:36.147 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T10:10:36.212 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T10:10:36.262 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-09-13T10:10:36.287 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.1 2024-09-13T10:10:36.287 DEBUG:teuthology.parallel:result is None 2024-09-13T10:10:36.387 INFO:teuthology.orchestra.run.smithi073.stdout:90194313220 2024-09-13T10:10:36.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.13 2024-09-13T10:10:36.414 DEBUG:teuthology.parallel:result is None 2024-09-13T10:10:36.491 INFO:teuthology.orchestra.run.smithi073.stdout:98784247812 2024-09-13T10:10:36.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247812 got 98784247812 for osd.2 2024-09-13T10:10:36.509 DEBUG:teuthology.parallel:result is None 2024-09-13T10:10:36.520 INFO:teuthology.orchestra.run.smithi073.stdout:94489280515 2024-09-13T10:10:36.524 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-09-13T10:10:36.541 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280515 got 94489280515 for osd.5 2024-09-13T10:10:36.542 DEBUG:teuthology.parallel:result is None 2024-09-13T10:10:36.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.3 2024-09-13T10:10:36.544 DEBUG:teuthology.parallel:result is None 2024-09-13T10:10:36.559 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-09-13T10:10:36.576 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.15 2024-09-13T10:10:36.576 DEBUG:teuthology.parallel:result is None 2024-09-13T10:10:36.654 INFO:teuthology.orchestra.run.smithi073.stdout:98784247811 2024-09-13T10:10:36.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247811 got 98784247811 for osd.6 2024-09-13T10:10:36.672 DEBUG:teuthology.parallel:result is None 2024-09-13T10:10:36.672 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T10:10:36.672 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:10:36.946 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:10:36.947 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:10:36.963 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-09-13T10:10:36.925393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":11,"kb":1509949440,"kb_used":442292,"kb_used_data":5708,"kb_used_omap":24,"kb_used_meta":436263,"kb_avail":1509507148,"statfs":{"total":1546188226560,"available":1545735319552,"internally_reserved":0,"allocated":5844992,"data_stored":3267327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":446733391},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":590387,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.796151"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.135708+0000","last_change":"2024-09-13T10:10:32.135708+0000","last_active":"2024-09-13T10:10:32.135708+0000","last_peered":"2024-09-13T10:10:32.135708+0000","last_clean":"2024-09-13T10:10:32.135708+0000","last_became_active":"2024-09-13T10:10:30.125785+0000","last_became_peered":"2024-09-13T10:10:30.125785+0000","last_unstale":"2024-09-13T10:10:32.135708+0000","last_undegraded":"2024-09-13T10:10:32.135708+0000","last_fullsized":"2024-09-13T10:10:32.135708+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:35.380273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010905629999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.932079+0000","last_change":"2024-09-13T10:10:32.932079+0000","last_active":"2024-09-13T10:10:32.932079+0000","last_peered":"2024-09-13T10:10:32.932079+0000","last_clean":"2024-09-13T10:10:32.932079+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:10:32.932079+0000","last_undegraded":"2024-09-13T10:10:32.932079+0000","last_fullsized":"2024-09-13T10:10:32.932079+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:57.533673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.177488+0000","last_change":"2024-09-13T10:10:32.177488+0000","last_active":"2024-09-13T10:10:32.177488+0000","last_peered":"2024-09-13T10:10:32.177488+0000","last_clean":"2024-09-13T10:10:32.177488+0000","last_became_active":"2024-09-13T10:10:30.128658+0000","last_became_peered":"2024-09-13T10:10:30.128658+0000","last_unstale":"2024-09-13T10:10:32.177488+0000","last_undegraded":"2024-09-13T10:10:32.177488+0000","last_fullsized":"2024-09-13T10:10:32.177488+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:50.278429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.701581+0000","last_change":"2024-09-13T10:10:32.701581+0000","last_active":"2024-09-13T10:10:32.701581+0000","last_peered":"2024-09-13T10:10:32.701581+0000","last_clean":"2024-09-13T10:10:32.701581+0000","last_became_active":"2024-09-13T10:10:30.126588+0000","last_became_peered":"2024-09-13T10:10:30.126588+0000","last_unstale":"2024-09-13T10:10:32.701581+0000","last_undegraded":"2024-09-13T10:10:32.701581+0000","last_fullsized":"2024-09-13T10:10:32.701581+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:40.095103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089379399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"26'112","reported_seq":152,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.134592+0000","last_change":"2024-09-13T10:10:30.127365+0000","last_active":"2024-09-13T10:10:32.134592+0000","last_peered":"2024-09-13T10:10:32.134592+0000","last_clean":"2024-09-13T10:10:32.134592+0000","last_became_active":"2024-09-13T10:10:30.126496+0000","last_became_peered":"2024-09-13T10:10:30.126496+0000","last_unstale":"2024-09-13T10:10:32.134592+0000","last_undegraded":"2024-09-13T10:10:32.134592+0000","last_fullsized":"2024-09-13T10:10:32.134592+0000","mapping_epoch":24,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:53.401784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.701711+0000","last_change":"2024-09-13T10:10:32.702214+0000","last_active":"2024-09-13T10:10:32.701711+0000","last_peered":"2024-09-13T10:10:32.701711+0000","last_clean":"2024-09-13T10:10:32.701711+0000","last_became_active":"2024-09-13T10:10:30.124759+0000","last_became_peered":"2024-09-13T10:10:30.124759+0000","last_unstale":"2024-09-13T10:10:32.701711+0000","last_undegraded":"2024-09-13T10:10:32.701711+0000","last_fullsized":"2024-09-13T10:10:32.701711+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:45.386731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090474700000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"27'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.147518+0000","last_change":"2024-09-13T10:10:32.146042+0000","last_active":"2024-09-13T10:10:32.147518+0000","last_peered":"2024-09-13T10:10:32.147518+0000","last_clean":"2024-09-13T10:10:32.147518+0000","last_became_active":"2024-09-13T10:10:30.127232+0000","last_became_peered":"2024-09-13T10:10:30.127232+0000","last_unstale":"2024-09-13T10:10:32.147518+0000","last_undegraded":"2024-09-13T10:10:32.147518+0000","last_fullsized":"2024-09-13T10:10:32.147518+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:29.955802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034622780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:33.223188+0000","last_change":"2024-09-13T10:10:33.223611+0000","last_active":"2024-09-13T10:10:33.223188+0000","last_peered":"2024-09-13T10:10:33.223188+0000","last_clean":"2024-09-13T10:10:33.223188+0000","last_became_active":"2024-09-13T10:10:30.128310+0000","last_became_peered":"2024-09-13T10:10:30.128310+0000","last_unstale":"2024-09-13T10:10:33.223188+0000","last_undegraded":"2024-09-13T10:10:33.223188+0000","last_fullsized":"2024-09-13T10:10:33.223188+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:46.008600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081844899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.149042+0000","last_change":"2024-09-13T10:10:32.149042+0000","last_active":"2024-09-13T10:10:32.149042+0000","last_peered":"2024-09-13T10:10:32.149042+0000","last_clean":"2024-09-13T10:10:32.149042+0000","last_became_active":"2024-09-13T10:10:30.272007+0000","last_became_peered":"2024-09-13T10:10:30.272007+0000","last_unstale":"2024-09-13T10:10:32.149042+0000","last_undegraded":"2024-09-13T10:10:32.149042+0000","last_fullsized":"2024-09-13T10:10:32.149042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:49.552306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000987604,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":11}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247812,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247814,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247814,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":847872,"data_stored":684335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313221,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":93986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,8,9,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247812,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":847872,"data_stored":684335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,4,5,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[3,4,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280516,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":835584,"data_stored":677029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,7,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31432,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,5,7,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280516,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247812,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":93986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247811,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":258048,"data_stored":93986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:10:36.965 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:10:37.254 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:10:37.254 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:10:37.271 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-09-13T10:10:36.925393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":11,"kb":1509949440,"kb_used":442292,"kb_used_data":5708,"kb_used_omap":24,"kb_used_meta":436263,"kb_avail":1509507148,"statfs":{"total":1546188226560,"available":1545735319552,"internally_reserved":0,"allocated":5844992,"data_stored":3267327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":446733391},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":590387,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.796151"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.135708+0000","last_change":"2024-09-13T10:10:32.135708+0000","last_active":"2024-09-13T10:10:32.135708+0000","last_peered":"2024-09-13T10:10:32.135708+0000","last_clean":"2024-09-13T10:10:32.135708+0000","last_became_active":"2024-09-13T10:10:30.125785+0000","last_became_peered":"2024-09-13T10:10:30.125785+0000","last_unstale":"2024-09-13T10:10:32.135708+0000","last_undegraded":"2024-09-13T10:10:32.135708+0000","last_fullsized":"2024-09-13T10:10:32.135708+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:35.380273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010905629999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.932079+0000","last_change":"2024-09-13T10:10:32.932079+0000","last_active":"2024-09-13T10:10:32.932079+0000","last_peered":"2024-09-13T10:10:32.932079+0000","last_clean":"2024-09-13T10:10:32.932079+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:10:32.932079+0000","last_undegraded":"2024-09-13T10:10:32.932079+0000","last_fullsized":"2024-09-13T10:10:32.932079+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:57.533673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.177488+0000","last_change":"2024-09-13T10:10:32.177488+0000","last_active":"2024-09-13T10:10:32.177488+0000","last_peered":"2024-09-13T10:10:32.177488+0000","last_clean":"2024-09-13T10:10:32.177488+0000","last_became_active":"2024-09-13T10:10:30.128658+0000","last_became_peered":"2024-09-13T10:10:30.128658+0000","last_unstale":"2024-09-13T10:10:32.177488+0000","last_undegraded":"2024-09-13T10:10:32.177488+0000","last_fullsized":"2024-09-13T10:10:32.177488+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:50.278429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.701581+0000","last_change":"2024-09-13T10:10:32.701581+0000","last_active":"2024-09-13T10:10:32.701581+0000","last_peered":"2024-09-13T10:10:32.701581+0000","last_clean":"2024-09-13T10:10:32.701581+0000","last_became_active":"2024-09-13T10:10:30.126588+0000","last_became_peered":"2024-09-13T10:10:30.126588+0000","last_unstale":"2024-09-13T10:10:32.701581+0000","last_undegraded":"2024-09-13T10:10:32.701581+0000","last_fullsized":"2024-09-13T10:10:32.701581+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:40.095103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089379399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"26'112","reported_seq":152,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.134592+0000","last_change":"2024-09-13T10:10:30.127365+0000","last_active":"2024-09-13T10:10:32.134592+0000","last_peered":"2024-09-13T10:10:32.134592+0000","last_clean":"2024-09-13T10:10:32.134592+0000","last_became_active":"2024-09-13T10:10:30.126496+0000","last_became_peered":"2024-09-13T10:10:30.126496+0000","last_unstale":"2024-09-13T10:10:32.134592+0000","last_undegraded":"2024-09-13T10:10:32.134592+0000","last_fullsized":"2024-09-13T10:10:32.134592+0000","mapping_epoch":24,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:53.401784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.701711+0000","last_change":"2024-09-13T10:10:32.702214+0000","last_active":"2024-09-13T10:10:32.701711+0000","last_peered":"2024-09-13T10:10:32.701711+0000","last_clean":"2024-09-13T10:10:32.701711+0000","last_became_active":"2024-09-13T10:10:30.124759+0000","last_became_peered":"2024-09-13T10:10:30.124759+0000","last_unstale":"2024-09-13T10:10:32.701711+0000","last_undegraded":"2024-09-13T10:10:32.701711+0000","last_fullsized":"2024-09-13T10:10:32.701711+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:45.386731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090474700000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"27'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.147518+0000","last_change":"2024-09-13T10:10:32.146042+0000","last_active":"2024-09-13T10:10:32.147518+0000","last_peered":"2024-09-13T10:10:32.147518+0000","last_clean":"2024-09-13T10:10:32.147518+0000","last_became_active":"2024-09-13T10:10:30.127232+0000","last_became_peered":"2024-09-13T10:10:30.127232+0000","last_unstale":"2024-09-13T10:10:32.147518+0000","last_undegraded":"2024-09-13T10:10:32.147518+0000","last_fullsized":"2024-09-13T10:10:32.147518+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:29.955802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034622780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:33.223188+0000","last_change":"2024-09-13T10:10:33.223611+0000","last_active":"2024-09-13T10:10:33.223188+0000","last_peered":"2024-09-13T10:10:33.223188+0000","last_clean":"2024-09-13T10:10:33.223188+0000","last_became_active":"2024-09-13T10:10:30.128310+0000","last_became_peered":"2024-09-13T10:10:30.128310+0000","last_unstale":"2024-09-13T10:10:33.223188+0000","last_undegraded":"2024-09-13T10:10:33.223188+0000","last_fullsized":"2024-09-13T10:10:33.223188+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:46.008600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081844899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.149042+0000","last_change":"2024-09-13T10:10:32.149042+0000","last_active":"2024-09-13T10:10:32.149042+0000","last_peered":"2024-09-13T10:10:32.149042+0000","last_clean":"2024-09-13T10:10:32.149042+0000","last_became_active":"2024-09-13T10:10:30.272007+0000","last_became_peered":"2024-09-13T10:10:30.272007+0000","last_unstale":"2024-09-13T10:10:32.149042+0000","last_undegraded":"2024-09-13T10:10:32.149042+0000","last_fullsized":"2024-09-13T10:10:32.149042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:49.552306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000987604,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":11}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247812,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247814,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247814,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":847872,"data_stored":684335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313221,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":93986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,8,9,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247812,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":847872,"data_stored":684335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,4,5,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280515,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[3,4,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280516,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":835584,"data_stored":677029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,7,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31432,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,5,7,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280516,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247812,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":93986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247811,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":258048,"data_stored":93986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:10:37.272 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T10:10:37.273 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T10:10:37.273 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T10:10:37.273 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T10:10:37.586 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:10:37.586 INFO:teuthology.orchestra.run.smithi073.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T10:10:37.604 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T10:10:37.604 INFO:teuthology.run_tasks:Running task exec... 2024-09-13T10:10:37.614 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T10:10:37.614 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi073.front.sepia.ceph.com 2024-09-13T10:10:37.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'while ! ceph balancer status ; do sleep 1 ; done' 2024-09-13T10:10:37.892 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:10:37.892 INFO:teuthology.orchestra.run.smithi073.stdout: "active": true, 2024-09-13T10:10:37.892 INFO:teuthology.orchestra.run.smithi073.stdout: "last_optimize_duration": "0:00:00.000202", 2024-09-13T10:10:37.892 INFO:teuthology.orchestra.run.smithi073.stdout: "last_optimize_started": "Fri Sep 13 10:10:18 2024", 2024-09-13T10:10:37.893 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": "upmap", 2024-09-13T10:10:37.893 INFO:teuthology.orchestra.run.smithi073.stdout: "no_optimization_needed": false, 2024-09-13T10:10:37.893 INFO:teuthology.orchestra.run.smithi073.stdout: "optimize_result": "No pools available", 2024-09-13T10:10:37.893 INFO:teuthology.orchestra.run.smithi073.stdout: "plans": [] 2024-09-13T10:10:37.893 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:10:37.911 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer mode crush-compat' 2024-09-13T10:10:39.215 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer on' 2024-09-13T10:10:39.524 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-13T10:10:39.535 INFO:tasks.thrashosds:config is {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 2, 'thrash_hosts': True, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-13T10:10:39.535 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-aclamk-cbt-combined'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 5, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-aclamk-cbt-combined', 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'}} 2024-09-13T10:10:39.536 INFO:tasks.thrashosds:config is {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 4, 'thrash_hosts': True, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-13T10:10:39.536 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-13T10:10:39.536 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T10:10:39.536 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:10:39.839 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:10:39.839 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:10:39.857 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-13T10:10:39.178925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":11,"kb":1509949440,"kb_used":442328,"kb_used_data":5744,"kb_used_omap":24,"kb_used_meta":436263,"kb_avail":1509507112,"statfs":{"total":1546188226560,"available":1545735282688,"internally_reserved":0,"allocated":5881856,"data_stored":3288513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":446733391},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":590387,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.046983"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-13T10:10:38.029456+0000","last_change":"2024-09-13T10:10:32.135708+0000","last_active":"2024-09-13T10:10:38.029456+0000","last_peered":"2024-09-13T10:10:38.029456+0000","last_clean":"2024-09-13T10:10:38.029456+0000","last_became_active":"2024-09-13T10:10:30.125785+0000","last_became_peered":"2024-09-13T10:10:30.125785+0000","last_unstale":"2024-09-13T10:10:38.029456+0000","last_undegraded":"2024-09-13T10:10:38.029456+0000","last_fullsized":"2024-09-13T10:10:38.029456+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:35.380273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010905629999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.932079+0000","last_change":"2024-09-13T10:10:32.932079+0000","last_active":"2024-09-13T10:10:32.932079+0000","last_peered":"2024-09-13T10:10:32.932079+0000","last_clean":"2024-09-13T10:10:32.932079+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:10:32.932079+0000","last_undegraded":"2024-09-13T10:10:32.932079+0000","last_fullsized":"2024-09-13T10:10:32.932079+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:57.533673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.177488+0000","last_change":"2024-09-13T10:10:32.177488+0000","last_active":"2024-09-13T10:10:32.177488+0000","last_peered":"2024-09-13T10:10:32.177488+0000","last_clean":"2024-09-13T10:10:32.177488+0000","last_became_active":"2024-09-13T10:10:30.128658+0000","last_became_peered":"2024-09-13T10:10:30.128658+0000","last_unstale":"2024-09-13T10:10:32.177488+0000","last_undegraded":"2024-09-13T10:10:32.177488+0000","last_fullsized":"2024-09-13T10:10:32.177488+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:50.278429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-13T10:10:38.488846+0000","last_change":"2024-09-13T10:10:32.701581+0000","last_active":"2024-09-13T10:10:38.488846+0000","last_peered":"2024-09-13T10:10:38.488846+0000","last_clean":"2024-09-13T10:10:38.488846+0000","last_became_active":"2024-09-13T10:10:30.126588+0000","last_became_peered":"2024-09-13T10:10:30.126588+0000","last_unstale":"2024-09-13T10:10:38.488846+0000","last_undegraded":"2024-09-13T10:10:38.488846+0000","last_fullsized":"2024-09-13T10:10:38.488846+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:40.095103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089379399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":154,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-13T10:10:38.029391+0000","last_change":"2024-09-13T10:10:30.127365+0000","last_active":"2024-09-13T10:10:38.029391+0000","last_peered":"2024-09-13T10:10:38.029391+0000","last_clean":"2024-09-13T10:10:38.029391+0000","last_became_active":"2024-09-13T10:10:30.126496+0000","last_became_peered":"2024-09-13T10:10:30.126496+0000","last_unstale":"2024-09-13T10:10:38.029391+0000","last_undegraded":"2024-09-13T10:10:38.029391+0000","last_fullsized":"2024-09-13T10:10:38.029391+0000","mapping_epoch":24,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:53.401784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.701711+0000","last_change":"2024-09-13T10:10:32.702214+0000","last_active":"2024-09-13T10:10:32.701711+0000","last_peered":"2024-09-13T10:10:32.701711+0000","last_clean":"2024-09-13T10:10:32.701711+0000","last_became_active":"2024-09-13T10:10:30.124759+0000","last_became_peered":"2024-09-13T10:10:30.124759+0000","last_unstale":"2024-09-13T10:10:32.701711+0000","last_undegraded":"2024-09-13T10:10:32.701711+0000","last_fullsized":"2024-09-13T10:10:32.701711+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:45.386731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090474700000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"27'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.147518+0000","last_change":"2024-09-13T10:10:32.146042+0000","last_active":"2024-09-13T10:10:32.147518+0000","last_peered":"2024-09-13T10:10:32.147518+0000","last_clean":"2024-09-13T10:10:32.147518+0000","last_became_active":"2024-09-13T10:10:30.127232+0000","last_became_peered":"2024-09-13T10:10:30.127232+0000","last_unstale":"2024-09-13T10:10:32.147518+0000","last_undegraded":"2024-09-13T10:10:32.147518+0000","last_fullsized":"2024-09-13T10:10:32.147518+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:29.955802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034622780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:33.223188+0000","last_change":"2024-09-13T10:10:33.223611+0000","last_active":"2024-09-13T10:10:33.223188+0000","last_peered":"2024-09-13T10:10:33.223188+0000","last_clean":"2024-09-13T10:10:33.223188+0000","last_became_active":"2024-09-13T10:10:30.128310+0000","last_became_peered":"2024-09-13T10:10:30.128310+0000","last_unstale":"2024-09-13T10:10:33.223188+0000","last_undegraded":"2024-09-13T10:10:33.223188+0000","last_fullsized":"2024-09-13T10:10:33.223188+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:46.008600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081844899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.149042+0000","last_change":"2024-09-13T10:10:32.149042+0000","last_active":"2024-09-13T10:10:32.149042+0000","last_peered":"2024-09-13T10:10:32.149042+0000","last_clean":"2024-09-13T10:10:32.149042+0000","last_became_active":"2024-09-13T10:10:30.272007+0000","last_became_peered":"2024-09-13T10:10:30.272007+0000","last_unstale":"2024-09-13T10:10:32.149042+0000","last_undegraded":"2024-09-13T10:10:32.149042+0000","last_fullsized":"2024-09-13T10:10:32.149042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:49.552306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000987604,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":11}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247815,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247815,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":847872,"data_stored":684335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313222,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":93986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,8,9,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280517,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":847872,"data_stored":684335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[3,4,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":847872,"data_stored":684335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,7,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31432,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,5,7,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280517,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":93986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247812,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":258048,"data_stored":93986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:10:39.859 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:10:40.150 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:10:40.151 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:10:40.167 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-13T10:10:39.178925+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":11,"kb":1509949440,"kb_used":442328,"kb_used_data":5744,"kb_used_omap":24,"kb_used_meta":436263,"kb_avail":1509507112,"statfs":{"total":1546188226560,"available":1545735282688,"internally_reserved":0,"allocated":5881856,"data_stored":3288513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":446733391},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":590387,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.046983"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-13T10:10:38.029456+0000","last_change":"2024-09-13T10:10:32.135708+0000","last_active":"2024-09-13T10:10:38.029456+0000","last_peered":"2024-09-13T10:10:38.029456+0000","last_clean":"2024-09-13T10:10:38.029456+0000","last_became_active":"2024-09-13T10:10:30.125785+0000","last_became_peered":"2024-09-13T10:10:30.125785+0000","last_unstale":"2024-09-13T10:10:38.029456+0000","last_undegraded":"2024-09-13T10:10:38.029456+0000","last_fullsized":"2024-09-13T10:10:38.029456+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:35.380273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010905629999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.932079+0000","last_change":"2024-09-13T10:10:32.932079+0000","last_active":"2024-09-13T10:10:32.932079+0000","last_peered":"2024-09-13T10:10:32.932079+0000","last_clean":"2024-09-13T10:10:32.932079+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:10:32.932079+0000","last_undegraded":"2024-09-13T10:10:32.932079+0000","last_fullsized":"2024-09-13T10:10:32.932079+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:57.533673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.177488+0000","last_change":"2024-09-13T10:10:32.177488+0000","last_active":"2024-09-13T10:10:32.177488+0000","last_peered":"2024-09-13T10:10:32.177488+0000","last_clean":"2024-09-13T10:10:32.177488+0000","last_became_active":"2024-09-13T10:10:30.128658+0000","last_became_peered":"2024-09-13T10:10:30.128658+0000","last_unstale":"2024-09-13T10:10:32.177488+0000","last_undegraded":"2024-09-13T10:10:32.177488+0000","last_fullsized":"2024-09-13T10:10:32.177488+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:50.278429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":23,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-13T10:10:38.488846+0000","last_change":"2024-09-13T10:10:32.701581+0000","last_active":"2024-09-13T10:10:38.488846+0000","last_peered":"2024-09-13T10:10:38.488846+0000","last_clean":"2024-09-13T10:10:38.488846+0000","last_became_active":"2024-09-13T10:10:30.126588+0000","last_became_peered":"2024-09-13T10:10:30.126588+0000","last_unstale":"2024-09-13T10:10:38.488846+0000","last_undegraded":"2024-09-13T10:10:38.488846+0000","last_fullsized":"2024-09-13T10:10:38.488846+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:40.095103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089379399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":154,"reported_epoch":28,"state":"active+clean","last_fresh":"2024-09-13T10:10:38.029391+0000","last_change":"2024-09-13T10:10:30.127365+0000","last_active":"2024-09-13T10:10:38.029391+0000","last_peered":"2024-09-13T10:10:38.029391+0000","last_clean":"2024-09-13T10:10:38.029391+0000","last_became_active":"2024-09-13T10:10:30.126496+0000","last_became_peered":"2024-09-13T10:10:30.126496+0000","last_unstale":"2024-09-13T10:10:38.029391+0000","last_undegraded":"2024-09-13T10:10:38.029391+0000","last_fullsized":"2024-09-13T10:10:38.029391+0000","mapping_epoch":24,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:53.401784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.701711+0000","last_change":"2024-09-13T10:10:32.702214+0000","last_active":"2024-09-13T10:10:32.701711+0000","last_peered":"2024-09-13T10:10:32.701711+0000","last_clean":"2024-09-13T10:10:32.701711+0000","last_became_active":"2024-09-13T10:10:30.124759+0000","last_became_peered":"2024-09-13T10:10:30.124759+0000","last_unstale":"2024-09-13T10:10:32.701711+0000","last_undegraded":"2024-09-13T10:10:32.701711+0000","last_fullsized":"2024-09-13T10:10:32.701711+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:45.386731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090474700000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"27'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.147518+0000","last_change":"2024-09-13T10:10:32.146042+0000","last_active":"2024-09-13T10:10:32.147518+0000","last_peered":"2024-09-13T10:10:32.147518+0000","last_clean":"2024-09-13T10:10:32.147518+0000","last_became_active":"2024-09-13T10:10:30.127232+0000","last_became_peered":"2024-09-13T10:10:30.127232+0000","last_unstale":"2024-09-13T10:10:32.147518+0000","last_undegraded":"2024-09-13T10:10:32.147518+0000","last_fullsized":"2024-09-13T10:10:32.147518+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:29.955802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034622780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:33.223188+0000","last_change":"2024-09-13T10:10:33.223611+0000","last_active":"2024-09-13T10:10:33.223188+0000","last_peered":"2024-09-13T10:10:33.223188+0000","last_clean":"2024-09-13T10:10:33.223188+0000","last_became_active":"2024-09-13T10:10:30.128310+0000","last_became_peered":"2024-09-13T10:10:30.128310+0000","last_unstale":"2024-09-13T10:10:33.223188+0000","last_undegraded":"2024-09-13T10:10:33.223188+0000","last_fullsized":"2024-09-13T10:10:33.223188+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:46.008600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081844899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-13T10:10:32.149042+0000","last_change":"2024-09-13T10:10:32.149042+0000","last_active":"2024-09-13T10:10:32.149042+0000","last_peered":"2024-09-13T10:10:32.149042+0000","last_clean":"2024-09-13T10:10:32.149042+0000","last_became_active":"2024-09-13T10:10:30.272007+0000","last_became_peered":"2024-09-13T10:10:30.272007+0000","last_unstale":"2024-09-13T10:10:32.149042+0000","last_undegraded":"2024-09-13T10:10:32.149042+0000","last_fullsized":"2024-09-13T10:10:32.149042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:49.552306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000987604,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":11}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247815,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247815,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":847872,"data_stored":684335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313222,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":93986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313221,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,8,9,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280517,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":847872,"data_stored":684335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280516,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":100907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[3,4,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":847872,"data_stored":684335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,7,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31432,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,5,7,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280517,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":93967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":93986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247812,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":258048,"data_stored":93986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:10:40.168 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T10:10:40.168 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-13T10:10:40.470 INFO:teuthology.orchestra.run.smithi073.stdout:epoch 29 2024-09-13T10:10:40.470 INFO:teuthology.orchestra.run.smithi073.stdout:fsid c1e91d47-9492-4461-a043-8698fd74ccf3 2024-09-13T10:10:40.471 INFO:teuthology.orchestra.run.smithi073.stdout:created 2024-09-13T10:10:14.019912+0000 2024-09-13T10:10:40.471 INFO:teuthology.orchestra.run.smithi073.stdout:modified 2024-09-13T10:10:39.170861+0000 2024-09-13T10:10:40.471 INFO:teuthology.orchestra.run.smithi073.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-13T10:10:40.471 INFO:teuthology.orchestra.run.smithi073.stdout:crush_version 6 2024-09-13T10:10:40.471 INFO:teuthology.orchestra.run.smithi073.stdout:full_ratio 0.95 2024-09-13T10:10:40.471 INFO:teuthology.orchestra.run.smithi073.stdout:backfillfull_ratio 0.9 2024-09-13T10:10:40.471 INFO:teuthology.orchestra.run.smithi073.stdout:nearfull_ratio 0.85 2024-09-13T10:10:40.471 INFO:teuthology.orchestra.run.smithi073.stdout:require_min_compat_client luminous 2024-09-13T10:10:40.471 INFO:teuthology.orchestra.run.smithi073.stdout:min_compat_client jewel 2024-09-13T10:10:40.471 INFO:teuthology.orchestra.run.smithi073.stdout:require_osd_release squid 2024-09-13T10:10:40.471 INFO:teuthology.orchestra.run.smithi073.stdout:stretch_mode_enabled false 2024-09-13T10:10:40.471 INFO:teuthology.orchestra.run.smithi073.stdout:pool 1 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 27 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.00 2024-09-13T10:10:40.471 INFO:teuthology.orchestra.run.smithi073.stdout:pool 2 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 26 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 15.79 2024-09-13T10:10:40.472 INFO:teuthology.orchestra.run.smithi073.stdout:max_osd 16 2024-09-13T10:10:40.472 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.73:6812/1977392321 v2:172.21.15.73:6813/1977392321 exists,up f7d92188-c27c-4744-a330-4b7105e0f2d6 2024-09-13T10:10:40.472 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.136:6808/1228102864 v2:172.21.15.136:6809/1228102864 exists,up e900b771-7c96-4b5d-93d7-35a8ea6c026a 2024-09-13T10:10:40.472 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2 up in weight 1 up_from 23 up_thru 24 down_at 0 last_clean_interval [0,0) v2:172.21.15.181:6800/3314820936 v2:172.21.15.181:6801/3314820936 exists,up 28b7aa59-33b2-4e24-8f3d-758f90e25f68 2024-09-13T10:10:40.472 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3 up in weight 1 up_from 22 up_thru 24 down_at 0 last_clean_interval [0,0) v2:172.21.15.195:6801/2558565460 v2:172.21.15.195:6802/2558565460 exists,up a93d2df5-781e-49e1-a507-1718b6f954ed 2024-09-13T10:10:40.472 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.73:6800/858330493 v2:172.21.15.73:6801/858330493 exists,up e69053fd-f6d9-437e-a9d0-8896de6b927f 2024-09-13T10:10:40.472 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.136:6804/984313481 v2:172.21.15.136:6805/984313481 exists,up 3667ec10-561e-4a23-8197-6357db4af315 2024-09-13T10:10:40.472 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6 up in weight 1 up_from 23 up_thru 24 down_at 0 last_clean_interval [0,0) v2:172.21.15.181:6812/1505939978 v2:172.21.15.181:6813/1505939978 exists,up 30d4d63c-f04f-4423-91a6-816a7cbc0287 2024-09-13T10:10:40.472 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7 up in weight 1 up_from 22 up_thru 24 down_at 0 last_clean_interval [0,0) v2:172.21.15.195:6809/3679292757 v2:172.21.15.195:6810/3679292757 exists,up 48812f05-82f1-4263-8ec5-4861a6e91b61 2024-09-13T10:10:40.472 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8 up in weight 1 up_from 23 up_thru 24 down_at 0 last_clean_interval [0,0) v2:172.21.15.73:6804/4052106885 v2:172.21.15.73:6805/4052106885 exists,up 72056e61-d04b-4fd3-94bf-57ee3df4b466 2024-09-13T10:10:40.472 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.136:6800/4266571820 v2:172.21.15.136:6801/4266571820 exists,up 20e7f118-7148-4c27-9ec4-a33954709515 2024-09-13T10:10:40.472 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.181:6808/958255259 v2:172.21.15.181:6809/958255259 exists,up 012409e8-51c2-4584-aba7-95996f0d72ad 2024-09-13T10:10:40.473 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.195:6813/281884267 v2:172.21.15.195:6814/281884267 exists,up 88b137b1-7d30-45f7-ac5c-ba026704ef24 2024-09-13T10:10:40.473 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.73:6808/3532043806 v2:172.21.15.73:6809/3532043806 exists,up b54732a2-9dff-42e1-b4ef-281cbca3ad7c 2024-09-13T10:10:40.473 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13 up in weight 1 up_from 21 up_thru 24 down_at 0 last_clean_interval [0,0) v2:172.21.15.136:6812/4083982131 v2:172.21.15.136:6813/4083982131 exists,up 20b48864-44dc-4732-8dff-d0d21cc08032 2024-09-13T10:10:40.473 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14 up in weight 1 up_from 23 up_thru 24 down_at 0 last_clean_interval [0,0) v2:172.21.15.181:6804/1173379175 v2:172.21.15.181:6805/1173379175 exists,up 9fac3ef4-e9be-4e8b-9a21-0de35b543d65 2024-09-13T10:10:40.473 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15 up in weight 1 up_from 22 up_thru 24 down_at 0 last_clean_interval [0,0) v2:172.21.15.195:6805/848686366 v2:172.21.15.195:6806/848686366 exists,up 16b34f90-3148-4d41-a403-ba0f900991e1 2024-09-13T10:10:40.488 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.73:6812/1977392321 v2:172.21.15.73:6813/1977392321 exists,up f7d92188-c27c-4744-a330-4b7105e0f2d6', 'osd.1 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.136:6808/1228102864 v2:172.21.15.136:6809/1228102864 exists,up e900b771-7c96-4b5d-93d7-35a8ea6c026a', 'osd.2 up in weight 1 up_from 23 up_thru 24 down_at 0 last_clean_interval [0,0) v2:172.21.15.181:6800/3314820936 v2:172.21.15.181:6801/3314820936 exists,up 28b7aa59-33b2-4e24-8f3d-758f90e25f68', 'osd.3 up in weight 1 up_from 22 up_thru 24 down_at 0 last_clean_interval [0,0) v2:172.21.15.195:6801/2558565460 v2:172.21.15.195:6802/2558565460 exists,up a93d2df5-781e-49e1-a507-1718b6f954ed', 'osd.4 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.73:6800/858330493 v2:172.21.15.73:6801/858330493 exists,up e69053fd-f6d9-437e-a9d0-8896de6b927f', 'osd.5 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.136:6804/984313481 v2:172.21.15.136:6805/984313481 exists,up 3667ec10-561e-4a23-8197-6357db4af315', 'osd.6 up in weight 1 up_from 23 up_thru 24 down_at 0 last_clean_interval [0,0) v2:172.21.15.181:6812/1505939978 v2:172.21.15.181:6813/1505939978 exists,up 30d4d63c-f04f-4423-91a6-816a7cbc0287', 'osd.7 up in weight 1 up_from 22 up_thru 24 down_at 0 last_clean_interval [0,0) v2:172.21.15.195:6809/3679292757 v2:172.21.15.195:6810/3679292757 exists,up 48812f05-82f1-4263-8ec5-4861a6e91b61', 'osd.8 up in weight 1 up_from 23 up_thru 24 down_at 0 last_clean_interval [0,0) v2:172.21.15.73:6804/4052106885 v2:172.21.15.73:6805/4052106885 exists,up 72056e61-d04b-4fd3-94bf-57ee3df4b466', 'osd.9 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.136:6800/4266571820 v2:172.21.15.136:6801/4266571820 exists,up 20e7f118-7148-4c27-9ec4-a33954709515', 'osd.10 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.181:6808/958255259 v2:172.21.15.181:6809/958255259 exists,up 012409e8-51c2-4584-aba7-95996f0d72ad', 'osd.11 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.195:6813/281884267 v2:172.21.15.195:6814/281884267 exists,up 88b137b1-7d30-45f7-ac5c-ba026704ef24', 'osd.12 up in weight 1 up_from 23 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.73:6808/3532043806 v2:172.21.15.73:6809/3532043806 exists,up b54732a2-9dff-42e1-b4ef-281cbca3ad7c', 'osd.13 up in weight 1 up_from 21 up_thru 24 down_at 0 last_clean_interval [0,0) v2:172.21.15.136:6812/4083982131 v2:172.21.15.136:6813/4083982131 exists,up 20b48864-44dc-4732-8dff-d0d21cc08032', 'osd.14 up in weight 1 up_from 23 up_thru 24 down_at 0 last_clean_interval [0,0) v2:172.21.15.181:6804/1173379175 v2:172.21.15.181:6805/1173379175 exists,up 9fac3ef4-e9be-4e8b-9a21-0de35b543d65', 'osd.15 up in weight 1 up_from 22 up_thru 24 down_at 0 last_clean_interval [0,0) v2:172.21.15.195:6805/848686366 v2:172.21.15.195:6806/848686366 exists,up 16b34f90-3148-4d41-a403-ba0f900991e1'] 2024-09-13T10:10:40.489 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-13T10:10:40.598 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:10:40.599 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mon.a", 2024-09-13T10:10:40.599 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster": "ceph", 2024-09-13T10:10:40.599 INFO:teuthology.orchestra.run.smithi073.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-13T10:10:40.599 INFO:teuthology.orchestra.run.smithi073.stdout: "admin_socket_mode": "", 2024-09-13T10:10:40.599 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-13T10:10:40.599 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_client_required": "cephx, none", 2024-09-13T10:10:40.599 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_cluster_required": "cephx", 2024-09-13T10:10:40.599 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_debug": "true", 2024-09-13T10:10:40.599 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-13T10:10:40.600 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-13T10:10:40.600 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_service_required": "cephx", 2024-09-13T10:10:40.600 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-13T10:10:40.600 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_supported": "cephx", 2024-09-13T10:10:40.600 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_aio": "true", 2024-09-13T10:10:40.600 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-13T10:10:40.600 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_aio_poll_ms": "250", 2024-09-13T10:10:40.600 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_aio_reap_max": "16", 2024-09-13T10:10:40.600 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_async_discard_threads": "0", 2024-09-13T10:10:40.600 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_block_size": "4096", 2024-09-13T10:10:40.600 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_debug_aio": "false", 2024-09-13T10:10:40.601 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-13T10:10:40.601 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-13T10:10:40.601 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_debug_inflight_ios": "false", 2024-09-13T10:10:40.601 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_enable_discard": "false", 2024-09-13T10:10:40.601 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_flock_retry": "3", 2024-09-13T10:10:40.601 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-13T10:10:40.601 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_inject_crash": "0", 2024-09-13T10:10:40.601 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-13T10:10:40.601 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_ioring": "false", 2024-09-13T10:10:40.601 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_ioring_hipri": "false", 2024-09-13T10:10:40.601 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-13T10:10:40.601 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-13T10:10:40.602 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-13T10:10:40.602 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-13T10:10:40.602 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-13T10:10:40.602 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-13T10:10:40.602 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_type": "", 2024-09-13T10:10:40.602 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_alloc_size": "1048576", 2024-09-13T10:10:40.602 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_allocator": "hybrid", 2024-09-13T10:10:40.602 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_buffered_io": "true", 2024-09-13T10:10:40.602 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_check_for_zeros": "false", 2024-09-13T10:10:40.602 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-13T10:10:40.602 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-13T10:10:40.602 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_compact_log_sync": "false", 2024-09-13T10:10:40.603 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-13T10:10:40.603 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-13T10:10:40.603 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-13T10:10:40.603 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-13T10:10:40.603 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_max_log_runway": "4194304", 2024-09-13T10:10:40.603 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_max_prefetch": "1048576", 2024-09-13T10:10:40.603 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_min_flush_size": "524288", 2024-09-13T10:10:40.603 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_min_log_runway": "1048576", 2024-09-13T10:10:40.603 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_replay_recovery": "false", 2024-09-13T10:10:40.604 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-13T10:10:40.604 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-13T10:10:40.604 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_sync_write": "false", 2024-09-13T10:10:40.604 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-13T10:10:40.604 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-13T10:10:40.604 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-13T10:10:40.604 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_allocation_from_file": "true", 2024-09-13T10:10:40.604 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_allocator": "hybrid", 2024-09-13T10:10:40.604 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_async_db_compaction": "true", 2024-09-13T10:10:40.604 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-13T10:10:40.604 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-13T10:10:40.605 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-13T10:10:40.605 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-13T10:10:40.605 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bdev_label_multi": "true", 2024-09-13T10:10:40.605 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-13T10:10:40.605 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-13T10:10:40.605 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-13T10:10:40.605 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-13T10:10:40.605 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-13T10:10:40.605 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_create": "true", 2024-09-13T10:10:40.605 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_db_create": "false", 2024-09-13T10:10:40.605 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_db_path": "", 2024-09-13T10:10:40.605 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_db_size": "0", 2024-09-13T10:10:40.606 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_path": "", 2024-09-13T10:10:40.606 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_preallocate_file": "false", 2024-09-13T10:10:40.606 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_size": "107374182400", 2024-09-13T10:10:40.606 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_wal_create": "false", 2024-09-13T10:10:40.606 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_wal_path": "", 2024-09-13T10:10:40.606 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_wal_size": "100663296", 2024-09-13T10:10:40.606 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bluefs": "true", 2024-09-13T10:10:40.606 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-13T10:10:40.606 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-13T10:10:40.606 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-13T10:10:40.606 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-13T10:10:40.606 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-13T10:10:40.607 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T10:10:40.607 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T10:10:40.607 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-13T10:10:40.607 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T10:10:40.607 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_autotune": "true", 2024-09-13T10:10:40.607 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-13T10:10:40.607 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-13T10:10:40.607 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-13T10:10:40.607 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-13T10:10:40.607 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_size": "0", 2024-09-13T10:10:40.607 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-13T10:10:40.608 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-13T10:10:40.608 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-13T10:10:40.608 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-13T10:10:40.608 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_type": "2q", 2024-09-13T10:10:40.608 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-13T10:10:40.608 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_clone_cow": "true", 2024-09-13T10:10:40.608 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-13T10:10:40.608 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-13T10:10:40.608 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-13T10:10:40.608 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-13T10:10:40.608 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-13T10:10:40.608 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-13T10:10:40.608 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-13T10:10:40.609 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_mode": "none", 2024-09-13T10:10:40.609 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-13T10:10:40.609 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_csum_type": "crc32c", 2024-09-13T10:10:40.609 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-13T10:10:40.609 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-13T10:10:40.609 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_freelist": "false", 2024-09-13T10:10:40.609 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-13T10:10:40.609 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-13T10:10:40.609 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-13T10:10:40.609 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-13T10:10:40.609 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-13T10:10:40.609 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-13T10:10:40.610 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-13T10:10:40.610 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-13T10:10:40.610 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-13T10:10:40.610 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-13T10:10:40.610 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-13T10:10:40.610 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-13T10:10:40.610 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_small_allocations": "0", 2024-09-13T10:10:40.610 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-13T10:10:40.610 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_default_buffered_read": "true", 2024-09-13T10:10:40.610 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_default_buffered_write": "false", 2024-09-13T10:10:40.610 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-13T10:10:40.610 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-13T10:10:40.610 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-13T10:10:40.610 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-13T10:10:40.611 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-13T10:10:40.611 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-13T10:10:40.611 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-13T10:10:40.611 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-13T10:10:40.611 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-13T10:10:40.611 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fail_eio": "false", 2024-09-13T10:10:40.611 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-13T10:10:40.611 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-13T10:10:40.611 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-13T10:10:40.611 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-13T10:10:40.611 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-13T10:10:40.611 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-13T10:10:40.611 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_mount": "false", 2024-09-13T10:10:40.611 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-13T10:10:40.612 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_umount": "false", 2024-09-13T10:10:40.612 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-13T10:10:40.612 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-13T10:10:40.612 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-13T10:10:40.612 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-13T10:10:40.612 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-13T10:10:40.612 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-13T10:10:40.612 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-13T10:10:40.612 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-13T10:10:40.612 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_ignore_data_csum": "false", 2024-09-13T10:10:40.612 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-13T10:10:40.612 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-13T10:10:40.612 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-13T10:10:40.613 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-13T10:10:40.613 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_log_op_age": "5.000000", 2024-09-13T10:10:40.613 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_alloc_size": "0", 2024-09-13T10:10:40.613 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_blob_size": "0", 2024-09-13T10:10:40.613 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-13T10:10:40.613 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-13T10:10:40.613 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-13T10:10:40.613 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_deferred_txc": "32", 2024-09-13T10:10:40.613 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_min_alloc_size": "0", 2024-09-13T10:10:40.613 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-13T10:10:40.613 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-13T10:10:40.613 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_nid_prealloc": "1024", 2024-09-13T10:10:40.613 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-13T10:10:40.614 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-13T10:10:40.614 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-13T10:10:40.614 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-13T10:10:40.614 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_retry_disk_reads": "3", 2024-09-13T10:10:40.614 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_rocksdb_cf": "true", 2024-09-13T10:10:40.614 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-13T10:10:40.614 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-13T10:10:40.614 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-13T10:10:40.614 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-13T10:10:40.614 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-13T10:10:40.614 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-13T10:10:40.614 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-13T10:10:40.614 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-13T10:10:40.614 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_spdk_mem": "512", 2024-09-13T10:10:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-13T10:10:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-13T10:10:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-13T10:10:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-13T10:10:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-13T10:10:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-13T10:10:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-13T10:10:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_tracing": "false", 2024-09-13T10:10:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-13T10:10:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-13T10:10:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-13T10:10:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-13T10:10:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-13T10:10:40.615 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-13T10:10:40.616 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-13T10:10:40.616 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-13T10:10:40.616 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-13T10:10:40.616 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_write_v2": "false", 2024-09-13T10:10:40.616 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_write_v2_random": "false", 2024-09-13T10:10:40.616 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_zero_block_detection": "false", 2024-09-13T10:10:40.616 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-13T10:10:40.616 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-13T10:10:40.616 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-13T10:10:40.616 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-13T10:10:40.616 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-13T10:10:40.616 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-13T10:10:40.616 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-13T10:10:40.617 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-13T10:10:40.617 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-13T10:10:40.617 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-13T10:10:40.617 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-13T10:10:40.617 INFO:teuthology.orchestra.run.smithi073.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-13T10:10:40.617 INFO:teuthology.orchestra.run.smithi073.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-13T10:10:40.617 INFO:teuthology.orchestra.run.smithi073.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-13T10:10:40.617 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_cluster_require_signatures": "false", 2024-09-13T10:10:40.617 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_cluster_require_version": "2", 2024-09-13T10:10:40.617 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_require_signatures": "false", 2024-09-13T10:10:40.617 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_require_version": "2", 2024-09-13T10:10:40.617 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_service_require_signatures": "false", 2024-09-13T10:10:40.617 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_service_require_version": "2", 2024-09-13T10:10:40.618 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_sign_messages": "true", 2024-09-13T10:10:40.618 INFO:teuthology.orchestra.run.smithi073.stdout: "chdir": "", 2024-09-13T10:10:40.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_acl_type": "", 2024-09-13T10:10:40.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_asio_thread_count": "2", 2024-09-13T10:10:40.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_cache_mid": "0.750000", 2024-09-13T10:10:40.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_cache_size": "16384", 2024-09-13T10:10:40.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_caps_release_delay": "5", 2024-09-13T10:10:40.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_check_pool_perm": "true", 2024-09-13T10:10:40.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-13T10:10:40.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_debug_force_sync_read": "false", 2024-09-13T10:10:40.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_debug_getattr_caps": "false", 2024-09-13T10:10:40.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_debug_inject_features": "", 2024-09-13T10:10:40.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_debug_inject_tick_delay": "0", 2024-09-13T10:10:40.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-13T10:10:40.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_die_on_failed_remount": "false", 2024-09-13T10:10:40.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_dirsize_rbytes": "true", 2024-09-13T10:10:40.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_force_lazyio": "false", 2024-09-13T10:10:40.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_fs": "", 2024-09-13T10:10:40.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-13T10:10:40.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_inject_release_failure": "false", 2024-09-13T10:10:40.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_max_inline_size": "4096", 2024-09-13T10:10:40.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-13T10:10:40.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_mds_namespace": "", 2024-09-13T10:10:40.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_metadata": "", 2024-09-13T10:10:40.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_mount_gid": "-1", 2024-09-13T10:10:40.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_mount_timeout": "300", 2024-09-13T10:10:40.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_mount_uid": "-1", 2024-09-13T10:10:40.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_mountpoint": "/", 2024-09-13T10:10:40.620 INFO:teuthology.orchestra.run.smithi073.stdout: "client_notify_timeout": "10", 2024-09-13T10:10:40.620 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc": "true", 2024-09-13T10:10:40.620 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc_max_dirty": "104857600", 2024-09-13T10:10:40.620 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-13T10:10:40.620 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc_max_objects": "1000", 2024-09-13T10:10:40.620 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc_size": "209715200", 2024-09-13T10:10:40.620 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc_target_dirty": "8388608", 2024-09-13T10:10:40.620 INFO:teuthology.orchestra.run.smithi073.stdout: "client_permissions": "true", 2024-09-13T10:10:40.620 INFO:teuthology.orchestra.run.smithi073.stdout: "client_quota": "true", 2024-09-13T10:10:40.620 INFO:teuthology.orchestra.run.smithi073.stdout: "client_quota_df": "true", 2024-09-13T10:10:40.620 INFO:teuthology.orchestra.run.smithi073.stdout: "client_readahead_max_bytes": "0", 2024-09-13T10:10:40.620 INFO:teuthology.orchestra.run.smithi073.stdout: "client_readahead_max_periods": "4", 2024-09-13T10:10:40.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_readahead_min": "131072", 2024-09-13T10:10:40.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_reconnect_stale": "false", 2024-09-13T10:10:40.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_shutdown_timeout": "30", 2024-09-13T10:10:40.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_snapdir": ".snap", 2024-09-13T10:10:40.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_tick_interval": "1", 2024-09-13T10:10:40.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_trace": "", 2024-09-13T10:10:40.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_try_dentry_invalidate": "false", 2024-09-13T10:10:40.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_use_faked_inos": "false", 2024-09-13T10:10:40.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_use_random_mds": "false", 2024-09-13T10:10:40.621 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_graylog": "false", 2024-09-13T10:10:40.621 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-13T10:10:40.621 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_graylog_port": "12201", 2024-09-13T10:10:40.621 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_monitors": "default=true", 2024-09-13T10:10:40.621 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_syslog": "false", 2024-09-13T10:10:40.622 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-13T10:10:40.622 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_syslog_level": "info", 2024-09-13T10:10:40.622 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "-", 2024-09-13T10:10:40.622 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_network": "", 2024-09-13T10:10:40.622 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_network_interface": "", 2024-09-13T10:10:40.622 INFO:teuthology.orchestra.run.smithi073.stdout: "compressor_zlib_isal": "false", 2024-09-13T10:10:40.622 INFO:teuthology.orchestra.run.smithi073.stdout: "compressor_zlib_level": "5", 2024-09-13T10:10:40.622 INFO:teuthology.orchestra.run.smithi073.stdout: "compressor_zlib_winsize": "-15", 2024-09-13T10:10:40.622 INFO:teuthology.orchestra.run.smithi073.stdout: "compressor_zstd_level": "1", 2024-09-13T10:10:40.622 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-13T10:10:40.622 INFO:teuthology.orchestra.run.smithi073.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-13T10:10:40.622 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_alien_op_num_threads": "6", 2024-09-13T10:10:40.623 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-13T10:10:40.623 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-13T10:10:40.623 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-13T10:10:40.623 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_osd_stat_interval": "0", 2024-09-13T10:10:40.623 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_seastar_cpu_cores": "", 2024-09-13T10:10:40.623 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_seastar_num_threads": "0", 2024-09-13T10:10:40.623 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_location": "", 2024-09-13T10:10:40.623 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_location_hook": "", 2024-09-13T10:10:40.623 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_location_hook_timeout": "10", 2024-09-13T10:10:40.623 INFO:teuthology.orchestra.run.smithi073.stdout: "daemonize": "false", 2024-09-13T10:10:40.623 INFO:teuthology.orchestra.run.smithi073.stdout: "daos_pool": "tank", 2024-09-13T10:10:40.623 INFO:teuthology.orchestra.run.smithi073.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-13T10:10:40.623 INFO:teuthology.orchestra.run.smithi073.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-13T10:10:40.624 INFO:teuthology.orchestra.run.smithi073.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-13T10:10:40.624 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_alienstore": "0/5", 2024-09-13T10:10:40.624 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_allow_any_pool_priority": "false", 2024-09-13T10:10:40.624 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_asok": "1/5", 2024-09-13T10:10:40.624 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_asok_assert_abort": "false", 2024-09-13T10:10:40.624 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_asserts_on_shutdown": "true", 2024-09-13T10:10:40.624 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_auth": "20/20", 2024-09-13T10:10:40.624 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_bdev": "1/3", 2024-09-13T10:10:40.624 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_bluefs": "1/5", 2024-09-13T10:10:40.624 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_bluestore": "1/5", 2024-09-13T10:10:40.624 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_buffer": "0/1", 2024-09-13T10:10:40.624 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_ceph_dedup": "0/5", 2024-09-13T10:10:40.624 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_ceph_exporter": "1/5", 2024-09-13T10:10:40.625 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_cephfs_mirror": "0/5", 2024-09-13T10:10:40.625 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_cephsqlite": "0/5", 2024-09-13T10:10:40.625 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_client": "0/5", 2024-09-13T10:10:40.625 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_compressor": "1/5", 2024-09-13T10:10:40.625 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_context": "0/1", 2024-09-13T10:10:40.625 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_crimson_interrupt": "0/5", 2024-09-13T10:10:40.625 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_crush": "1/1", 2024-09-13T10:10:40.625 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_crypto": "1/5", 2024-09-13T10:10:40.625 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_cyanstore": "0/5", 2024-09-13T10:10:40.625 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_deliberately_leak_memory": "false", 2024-09-13T10:10:40.625 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_disable_randomized_ping": "false", 2024-09-13T10:10:40.625 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_dpdk": "1/5", 2024-09-13T10:10:40.625 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_eventtrace": "1/5", 2024-09-13T10:10:40.626 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_filer": "0/1", 2024-09-13T10:10:40.626 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_filestore": "1/3", 2024-09-13T10:10:40.626 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_finisher": "1/1", 2024-09-13T10:10:40.626 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_fuse": "1/5", 2024-09-13T10:10:40.626 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_heartbeat_testing_span": "0", 2024-09-13T10:10:40.626 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_heartbeatmap": "1/5", 2024-09-13T10:10:40.626 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-13T10:10:40.626 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_javaclient": "1/5", 2024-09-13T10:10:40.626 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_journal": "1/3", 2024-09-13T10:10:40.626 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_journaler": "0/5", 2024-09-13T10:10:40.626 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_kstore": "1/5", 2024-09-13T10:10:40.626 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_lockdep": "0/1", 2024-09-13T10:10:40.626 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mclock": "1/5", 2024-09-13T10:10:40.627 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds": "1/5", 2024-09-13T10:10:40.627 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_balancer": "1/5", 2024-09-13T10:10:40.627 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_locker": "1/5", 2024-09-13T10:10:40.627 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_log": "1/5", 2024-09-13T10:10:40.627 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_log_expire": "1/5", 2024-09-13T10:10:40.627 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_migrator": "1/5", 2024-09-13T10:10:40.627 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_quiesce": "3/5", 2024-09-13T10:10:40.627 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_memstore": "1/5", 2024-09-13T10:10:40.627 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mgr": "2/5", 2024-09-13T10:10:40.627 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mgrc": "1/5", 2024-09-13T10:10:40.627 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mon": "20/20", 2024-09-13T10:10:40.627 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_monc": "0/10", 2024-09-13T10:10:40.627 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_ms": "1/1", 2024-09-13T10:10:40.628 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_none": "0/5", 2024-09-13T10:10:40.628 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_objclass": "0/5", 2024-09-13T10:10:40.628 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_objectcacher": "0/5", 2024-09-13T10:10:40.628 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_objecter": "0/1", 2024-09-13T10:10:40.628 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_optracker": "0/5", 2024-09-13T10:10:40.628 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_osd": "1/5", 2024-09-13T10:10:40.628 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_paxos": "20/20", 2024-09-13T10:10:40.628 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_perfcounter": "1/5", 2024-09-13T10:10:40.628 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_prioritycache": "1/5", 2024-09-13T10:10:40.628 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rados": "0/5", 2024-09-13T10:10:40.628 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rbd": "0/5", 2024-09-13T10:10:40.628 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rbd_mirror": "0/5", 2024-09-13T10:10:40.628 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rbd_pwl": "0/5", 2024-09-13T10:10:40.629 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rbd_replay": "0/5", 2024-09-13T10:10:40.629 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_refs": "0/0", 2024-09-13T10:10:40.629 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_reserver": "1/1", 2024-09-13T10:10:40.629 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw": "1/5", 2024-09-13T10:10:40.629 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_access": "1/5", 2024-09-13T10:10:40.629 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_datacache": "1/5", 2024-09-13T10:10:40.629 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_dbstore": "1/5", 2024-09-13T10:10:40.629 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_flight": "1/5", 2024-09-13T10:10:40.629 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-13T10:10:40.629 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_notification": "1/5", 2024-09-13T10:10:40.629 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_sync": "1/5", 2024-09-13T10:10:40.629 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rocksdb": "4/5", 2024-09-13T10:10:40.630 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore": "0/5", 2024-09-13T10:10:40.630 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_backref": "0/5", 2024-09-13T10:10:40.630 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_cache": "0/5", 2024-09-13T10:10:40.630 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_cleaner": "0/5", 2024-09-13T10:10:40.630 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_device": "0/5", 2024-09-13T10:10:40.630 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_epm": "0/5", 2024-09-13T10:10:40.630 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-13T10:10:40.630 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_journal": "0/5", 2024-09-13T10:10:40.630 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_lba": "0/5", 2024-09-13T10:10:40.630 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_odata": "0/5", 2024-09-13T10:10:40.630 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_omap": "0/5", 2024-09-13T10:10:40.630 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_onode": "0/5", 2024-09-13T10:10:40.630 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_t": "0/5", 2024-09-13T10:10:40.630 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_tm": "0/5", 2024-09-13T10:10:40.631 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_striper": "0/1", 2024-09-13T10:10:40.631 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_test": "0/5", 2024-09-13T10:10:40.631 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_throttle": "1/1", 2024-09-13T10:10:40.631 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_timer": "0/1", 2024-09-13T10:10:40.631 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_tp": "0/5", 2024-09-13T10:10:40.631 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_trace": "1/5", 2024-09-13T10:10:40.631 INFO:teuthology.orchestra.run.smithi073.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-13T10:10:40.631 INFO:teuthology.orchestra.run.smithi073.stdout: "device_failure_prediction_mode": "none", 2024-09-13T10:10:40.631 INFO:teuthology.orchestra.run.smithi073.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-13T10:10:40.631 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-09-13T10:10:40.631 INFO:teuthology.orchestra.run.smithi073.stdout: "err_to_graylog": "false", 2024-09-13T10:10:40.631 INFO:teuthology.orchestra.run.smithi073.stdout: "err_to_journald": "false", 2024-09-13T10:10:40.631 INFO:teuthology.orchestra.run.smithi073.stdout: "err_to_stderr": "true", 2024-09-13T10:10:40.632 INFO:teuthology.orchestra.run.smithi073.stdout: "err_to_syslog": "false", 2024-09-13T10:10:40.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event_tracing": "false", 2024-09-13T10:10:40.632 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_addr": "0.0.0.0", 2024-09-13T10:10:40.632 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_cert_file": "", 2024-09-13T10:10:40.632 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_http_port": "9926", 2024-09-13T10:10:40.632 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_key_file": "", 2024-09-13T10:10:40.632 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_prio_limit": "5", 2024-09-13T10:10:40.632 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-13T10:10:40.632 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_sort_metrics": "true", 2024-09-13T10:10:40.632 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_stats_period": "5", 2024-09-13T10:10:40.632 INFO:teuthology.orchestra.run.smithi073.stdout: "fake_statfs_for_testing": "0", 2024-09-13T10:10:40.632 INFO:teuthology.orchestra.run.smithi073.stdout: "fatal_signal_handlers": "true", 2024-09-13T10:10:40.633 INFO:teuthology.orchestra.run.smithi073.stdout: "filer_max_purge_ops": "10", 2024-09-13T10:10:40.633 INFO:teuthology.orchestra.run.smithi073.stdout: "filer_max_truncate_ops": "128", 2024-09-13T10:10:40.633 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_apply_finisher_threads": "1", 2024-09-13T10:10:40.633 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_blackhole": "false", 2024-09-13T10:10:40.633 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_btrfs_clone_range": "true", 2024-09-13T10:10:40.633 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_btrfs_snap": "true", 2024-09-13T10:10:40.633 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_caller_concurrency": "10", 2024-09-13T10:10:40.633 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_collect_device_partition_information": "true", 2024-09-13T10:10:40.633 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_commit_timeout": "600.000000", 2024-09-13T10:10:40.633 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_debug_inject_read_err": "false", 2024-09-13T10:10:40.633 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_debug_omap_check": "false", 2024-09-13T10:10:40.633 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_debug_verify_split": "false", 2024-09-13T10:10:40.633 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_dump_file": "", 2024-09-13T10:10:40.633 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-13T10:10:40.634 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-13T10:10:40.634 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fadvise": "true", 2024-09-13T10:10:40.634 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fail_eio": "true", 2024-09-13T10:10:40.634 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fd_cache_shards": "16", 2024-09-13T10:10:40.634 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fd_cache_size": "128", 2024-09-13T10:10:40.634 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fiemap": "false", 2024-09-13T10:10:40.634 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fiemap_threshold": "4096", 2024-09-13T10:10:40.634 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-13T10:10:40.634 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-13T10:10:40.634 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_inject_stall": "0", 2024-09-13T10:10:40.634 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_journal_parallel": "false", 2024-09-13T10:10:40.634 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_journal_trailing": "false", 2024-09-13T10:10:40.634 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_journal_writeahead": "false", 2024-09-13T10:10:40.634 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_kill_at": "0", 2024-09-13T10:10:40.635 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-13T10:10:40.635 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-13T10:10:40.635 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-13T10:10:40.635 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-13T10:10:40.635 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-13T10:10:40.635 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattrs": "0", 2024-09-13T10:10:40.635 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-13T10:10:40.635 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-13T10:10:40.635 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-13T10:10:40.635 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-13T10:10:40.635 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_xattr_value_size": "0", 2024-09-13T10:10:40.636 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-13T10:10:40.636 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-13T10:10:40.636 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-13T10:10:40.636 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_merge_threshold": "-10", 2024-09-13T10:10:40.636 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-13T10:10:40.636 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_odsync_write": "false", 2024-09-13T10:10:40.636 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_omap_backend": "rocksdb", 2024-09-13T10:10:40.636 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_omap_backend_path": "", 2024-09-13T10:10:40.636 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-13T10:10:40.636 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-13T10:10:40.636 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-13T10:10:40.636 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_op_thread_timeout": "60", 2024-09-13T10:10:40.637 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_op_threads": "2", 2024-09-13T10:10:40.637 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_punch_hole": "false", 2024-09-13T10:10:40.637 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-13T10:10:40.637 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-13T10:10:40.637 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-13T10:10:40.637 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-13T10:10:40.637 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-13T10:10:40.637 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-13T10:10:40.637 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-13T10:10:40.637 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-13T10:10:40.637 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-13T10:10:40.638 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_max_ops": "50", 2024-09-13T10:10:40.638 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-13T10:10:40.638 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_seek_data_hole": "false", 2024-09-13T10:10:40.638 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_sloppy_crc": "false", 2024-09-13T10:10:40.638 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-13T10:10:40.638 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_splice": "false", 2024-09-13T10:10:40.638 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_split_multiple": "2", 2024-09-13T10:10:40.638 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_split_rand_factor": "20", 2024-09-13T10:10:40.638 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_update_to": "1000", 2024-09-13T10:10:40.638 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-13T10:10:40.638 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-13T10:10:40.638 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-13T10:10:40.638 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-13T10:10:40.639 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-13T10:10:40.639 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-13T10:10:40.639 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_enable": "true", 2024-09-13T10:10:40.639 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-13T10:10:40.639 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-13T10:10:40.639 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-13T10:10:40.639 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-13T10:10:40.639 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-13T10:10:40.639 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-13T10:10:40.639 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_xfs_extsize": "false", 2024-09-13T10:10:40.639 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_zfs_snap": "false", 2024-09-13T10:10:40.639 INFO:teuthology.orchestra.run.smithi073.stdout: "fio_dir": "/tmp/fio", 2024-09-13T10:10:40.639 INFO:teuthology.orchestra.run.smithi073.stdout: "fsid": "c1e91d47-9492-4461-a043-8698fd74ccf3", 2024-09-13T10:10:40.640 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_allow_other": "true", 2024-09-13T10:10:40.640 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_atomic_o_trunc": "true", 2024-09-13T10:10:40.640 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_big_writes": "true", 2024-09-13T10:10:40.640 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_debug": "false", 2024-09-13T10:10:40.640 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_default_permissions": "false", 2024-09-13T10:10:40.640 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_disable_pagecache": "false", 2024-09-13T10:10:40.640 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_max_write": "0", 2024-09-13T10:10:40.640 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_multithreaded": "true", 2024-09-13T10:10:40.640 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_require_active_mds": "true", 2024-09-13T10:10:40.640 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_set_user_groups": "true", 2024-09-13T10:10:40.640 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_splice_move": "true", 2024-09-13T10:10:40.640 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_splice_read": "true", 2024-09-13T10:10:40.641 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_splice_write": "true", 2024-09-13T10:10:40.641 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-13T10:10:40.641 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_use_invalidate_cb": "true", 2024-09-13T10:10:40.641 INFO:teuthology.orchestra.run.smithi073.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-13T10:10:40.641 INFO:teuthology.orchestra.run.smithi073.stdout: "gss_target_name": "ceph", 2024-09-13T10:10:40.641 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_file": "", 2024-09-13T10:10:40.641 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_inject_failure": "0", 2024-09-13T10:10:40.641 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_interval": "5", 2024-09-13T10:10:40.641 INFO:teuthology.orchestra.run.smithi073.stdout: "host": "smithi073", 2024-09-13T10:10:40.641 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-13T10:10:40.641 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-13T10:10:40.641 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-13T10:10:40.641 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-13T10:10:40.641 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-13T10:10:40.641 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-13T10:10:40.642 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-13T10:10:40.642 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-13T10:10:40.642 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-13T10:10:40.642 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-13T10:10:40.642 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-13T10:10:40.642 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-13T10:10:40.642 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-13T10:10:40.642 INFO:teuthology.orchestra.run.smithi073.stdout: "inject_early_sigterm": "false", 2024-09-13T10:10:40.642 INFO:teuthology.orchestra.run.smithi073.stdout: "jaeger_agent_port": "6799", 2024-09-13T10:10:40.642 INFO:teuthology.orchestra.run.smithi073.stdout: "jaeger_tracing_enable": "false", 2024-09-13T10:10:40.642 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_aio": "true", 2024-09-13T10:10:40.642 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_align_min_size": "65536", 2024-09-13T10:10:40.642 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_block_align": "true", 2024-09-13T10:10:40.643 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_block_size": "4096", 2024-09-13T10:10:40.643 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_dio": "true", 2024-09-13T10:10:40.643 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_discard": "false", 2024-09-13T10:10:40.643 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_force_aio": "false", 2024-09-13T10:10:40.643 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_ignore_corruption": "false", 2024-09-13T10:10:40.643 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_max_write_bytes": "10485760", 2024-09-13T10:10:40.643 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_max_write_entries": "100", 2024-09-13T10:10:40.643 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_replay_from": "0", 2024-09-13T10:10:40.643 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-13T10:10:40.643 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-13T10:10:40.643 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-13T10:10:40.643 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-13T10:10:40.643 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_write_header_frequency": "0", 2024-09-13T10:10:40.643 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_zero_on_create": "false", 2024-09-13T10:10:40.643 INFO:teuthology.orchestra.run.smithi073.stdout: "journaler_prefetch_periods": "10", 2024-09-13T10:10:40.644 INFO:teuthology.orchestra.run.smithi073.stdout: "journaler_prezero_periods": "5", 2024-09-13T10:10:40.644 INFO:teuthology.orchestra.run.smithi073.stdout: "journaler_write_head_interval": "15", 2024-09-13T10:10:40.644 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-13T10:10:40.644 INFO:teuthology.orchestra.run.smithi073.stdout: "keyfile": "", 2024-09-13T10:10:40.644 INFO:teuthology.orchestra.run.smithi073.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-13T10:10:40.644 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_backend": "rocksdb", 2024-09-13T10:10:40.644 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_default_stripe_size": "65536", 2024-09-13T10:10:40.644 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_fsck_on_mount": "false", 2024-09-13T10:10:40.644 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-13T10:10:40.644 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_max_bytes": "67108864", 2024-09-13T10:10:40.644 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_max_ops": "512", 2024-09-13T10:10:40.644 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_nid_prealloc": "1024", 2024-09-13T10:10:40.644 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_onode_map_size": "1024", 2024-09-13T10:10:40.645 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-13T10:10:40.645 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_sync_submit_transaction": "false", 2024-09-13T10:10:40.645 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_sync_transaction": "false", 2024-09-13T10:10:40.645 INFO:teuthology.orchestra.run.smithi073.stdout: "librados_thread_count": "2", 2024-09-13T10:10:40.645 INFO:teuthology.orchestra.run.smithi073.stdout: "lockdep": "false", 2024-09-13T10:10:40.645 INFO:teuthology.orchestra.run.smithi073.stdout: "lockdep_force_backtrace": "false", 2024-09-13T10:10:40.645 INFO:teuthology.orchestra.run.smithi073.stdout: "log_coarse_timestamps": "true", 2024-09-13T10:10:40.645 INFO:teuthology.orchestra.run.smithi073.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-13T10:10:40.645 INFO:teuthology.orchestra.run.smithi073.stdout: "log_flush_on_exit": "false", 2024-09-13T10:10:40.645 INFO:teuthology.orchestra.run.smithi073.stdout: "log_graylog_host": "127.0.0.1", 2024-09-13T10:10:40.646 INFO:teuthology.orchestra.run.smithi073.stdout: "log_graylog_port": "12201", 2024-09-13T10:10:40.646 INFO:teuthology.orchestra.run.smithi073.stdout: "log_max_new": "1000", 2024-09-13T10:10:40.646 INFO:teuthology.orchestra.run.smithi073.stdout: "log_max_recent": "10000", 2024-09-13T10:10:40.646 INFO:teuthology.orchestra.run.smithi073.stdout: "log_stderr_prefix": "", 2024-09-13T10:10:40.646 INFO:teuthology.orchestra.run.smithi073.stdout: "log_stop_at_utilization": "0.970000", 2024-09-13T10:10:40.646 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": "true", 2024-09-13T10:10:40.646 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_graylog": "false", 2024-09-13T10:10:40.646 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_journald": "false", 2024-09-13T10:10:40.646 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_stderr": "false", 2024-09-13T10:10:40.646 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_syslog": "false", 2024-09-13T10:10:40.646 INFO:teuthology.orchestra.run.smithi073.stdout: "max_rotating_auth_attempts": "10", 2024-09-13T10:10:40.646 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-13T10:10:40.646 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_action_on_write_error": "1", 2024-09-13T10:10:40.647 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_alternate_name_max": "8192", 2024-09-13T10:10:40.647 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_asio_thread_count": "2", 2024-09-13T10:10:40.647 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_export_pin": "true", 2024-09-13T10:10:40.647 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_fragment_dirs": "true", 2024-09-13T10:10:40.647 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-13T10:10:40.647 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_fragment_interval": "5", 2024-09-13T10:10:40.647 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-13T10:10:40.647 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-13T10:10:40.647 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_interval": "10", 2024-09-13T10:10:40.647 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_max": "-1", 2024-09-13T10:10:40.647 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_max_until": "-1", 2024-09-13T10:10:40.647 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_merge_size": "50", 2024-09-13T10:10:40.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_midchunk": "0.300000", 2024-09-13T10:10:40.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-13T10:10:40.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_min_start": "0.200000", 2024-09-13T10:10:40.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_minchunk": "0.001000", 2024-09-13T10:10:40.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_mode": "0", 2024-09-13T10:10:40.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_need_max": "1.200000", 2024-09-13T10:10:40.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_need_min": "0.800000", 2024-09-13T10:10:40.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_overload_epochs": "2", 2024-09-13T10:10:40.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-13T10:10:40.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-13T10:10:40.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_split_bits": "3", 2024-09-13T10:10:40.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-13T10:10:40.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_split_size": "10000", 2024-09-13T10:10:40.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-13T10:10:40.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_target_decay": "10.000000", 2024-09-13T10:10:40.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-13T10:10:40.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_beacon_grace": "15.000000", 2024-09-13T10:10:40.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_beacon_interval": "4.000000", 2024-09-13T10:10:40.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-13T10:10:40.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-13T10:10:40.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_mid": "0.700000", 2024-09-13T10:10:40.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-13T10:10:40.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_quiesce_delay": "0", 2024-09-13T10:10:40.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-13T10:10:40.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-13T10:10:40.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-13T10:10:40.650 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_release_free_interval": "10", 2024-09-13T10:10:40.650 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_reservation": "0.050000", 2024-09-13T10:10:40.650 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-13T10:10:40.650 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_trim_interval": "1", 2024-09-13T10:10:40.650 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_trim_threshold": "262144", 2024-09-13T10:10:40.650 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-13T10:10:40.650 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-13T10:10:40.650 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-13T10:10:40.650 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_client_prealloc_inos": "1000", 2024-09-13T10:10:40.650 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-13T10:10:40.650 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_connect_bootstrapping": "false", 2024-09-13T10:10:40.650 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_damage_table_max_entries": "10000", 2024-09-13T10:10:40.650 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-13T10:10:40.651 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_debug_auth_pins": "false", 2024-09-13T10:10:40.651 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_debug_frag": "false", 2024-09-13T10:10:40.651 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_debug_scatterstat": "false", 2024-09-13T10:10:40.651 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_debug_subtrees": "false", 2024-09-13T10:10:40.651 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_decay_halflife": "5.000000", 2024-09-13T10:10:40.651 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_default_dir_hash": "2", 2024-09-13T10:10:40.651 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_defer_session_stale": "true", 2024-09-13T10:10:40.651 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_deny_all_reconnect": "false", 2024-09-13T10:10:40.651 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dir_keys_per_op": "16384", 2024-09-13T10:10:40.651 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dir_max_commit_size": "10", 2024-09-13T10:10:40.651 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dir_max_entries": "0", 2024-09-13T10:10:40.651 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dir_prefetch": "true", 2024-09-13T10:10:40.652 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-13T10:10:40.652 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-13T10:10:40.652 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dump_cache_on_map": "false", 2024-09-13T10:10:40.652 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-13T10:10:40.652 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-13T10:10:40.652 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_early_reply": "true", 2024-09-13T10:10:40.652 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_enable_op_tracker": "true", 2024-09-13T10:10:40.652 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_enforce_unique_name": "true", 2024-09-13T10:10:40.652 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-13T10:10:40.652 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-13T10:10:40.652 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_export_ephemeral_random": "true", 2024-09-13T10:10:40.652 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-13T10:10:40.652 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-13T10:10:40.653 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-13T10:10:40.653 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-13T10:10:40.653 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-13T10:10:40.653 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-13T10:10:40.653 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-13T10:10:40.653 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-13T10:10:40.653 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_health_summarize_threshold": "10", 2024-09-13T10:10:40.653 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-13T10:10:40.653 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-13T10:10:40.653 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_health_dummy": "false", 2024-09-13T10:10:40.653 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-13T10:10:40.653 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_migrator_session_race": "false", 2024-09-13T10:10:40.653 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-13T10:10:40.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-13T10:10:40.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-13T10:10:40.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_join_fs": "", 2024-09-13T10:10:40.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_journal_format": "1", 2024-09-13T10:10:40.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-13T10:10:40.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_create_at": "0", 2024-09-13T10:10:40.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_dirfrag_at": "0", 2024-09-13T10:10:40.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_export_at": "0", 2024-09-13T10:10:40.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_import_at": "0", 2024-09-13T10:10:40.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_journal_at": "0", 2024-09-13T10:10:40.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_journal_expire_at": "0", 2024-09-13T10:10:40.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_journal_replay_at": "0", 2024-09-13T10:10:40.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_link_at": "0", 2024-09-13T10:10:40.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_mdstable_at": "0", 2024-09-13T10:10:40.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_openc_at": "0", 2024-09-13T10:10:40.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_rename_at": "0", 2024-09-13T10:10:40.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_shutdown_at": "0", 2024-09-13T10:10:40.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_event_large_threshold": "524288", 2024-09-13T10:10:40.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_events_per_segment": "1024", 2024-09-13T10:10:40.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-13T10:10:40.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_max_events": "-1", 2024-09-13T10:10:40.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_max_segments": "128", 2024-09-13T10:10:40.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_pause": "false", 2024-09-13T10:10:40.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_segment_size": "0", 2024-09-13T10:10:40.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-13T10:10:40.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-13T10:10:40.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-13T10:10:40.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_trim_threshold": "128", 2024-09-13T10:10:40.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-13T10:10:40.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_warn_factor": "2.000000", 2024-09-13T10:10:40.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_caps_per_client": "1048576", 2024-09-13T10:10:40.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_completed_flushes": "100000", 2024-09-13T10:10:40.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_completed_requests": "100000", 2024-09-13T10:10:40.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_export_size": "20971520", 2024-09-13T10:10:40.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_file_recover": "32", 2024-09-13T10:10:40.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_purge_files": "64", 2024-09-13T10:10:40.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_purge_ops": "8192", 2024-09-13T10:10:40.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-13T10:10:40.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-13T10:10:40.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_snaps_per_dir": "100", 2024-09-13T10:10:40.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_metrics_update_interval": "2", 2024-09-13T10:10:40.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_min_caps_per_client": "100", 2024-09-13T10:10:40.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_min_caps_working_set": "10000", 2024-09-13T10:10:40.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-13T10:10:40.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_numa_node": "-1", 2024-09-13T10:10:40.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-13T10:10:40.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_complaint_time": "30.000000", 2024-09-13T10:10:40.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_history_duration": "600", 2024-09-13T10:10:40.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_history_size": "20", 2024-09-13T10:10:40.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_history_slow_op_size": "20", 2024-09-13T10:10:40.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-13T10:10:40.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_log_threshold": "5", 2024-09-13T10:10:40.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_ping_grace": "15", 2024-09-13T10:10:40.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_ping_interval": "5", 2024-09-13T10:10:40.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-13T10:10:40.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-13T10:10:40.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_max_caps": "30000", 2024-09-13T10:10:40.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-13T10:10:40.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-13T10:10:40.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-13T10:10:40.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_warning_threshold": "262144", 2024-09-13T10:10:40.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-13T10:10:40.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_replay_interval": "1.000000", 2024-09-13T10:10:40.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-13T10:10:40.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-13T10:10:40.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_root_ino_gid": "0", 2024-09-13T10:10:40.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_root_ino_uid": "0", 2024-09-13T10:10:40.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-13T10:10:40.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-13T10:10:40.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-13T10:10:40.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-13T10:10:40.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-13T10:10:40.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-13T10:10:40.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-13T10:10:40.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-13T10:10:40.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-13T10:10:40.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-13T10:10:40.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-13T10:10:40.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-13T10:10:40.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_shutdown_check": "0", 2024-09-13T10:10:40.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_skip_ino": "0", 2024-09-13T10:10:40.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-13T10:10:40.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_snap_max_uid": "4294967294", 2024-09-13T10:10:40.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_snap_min_uid": "0", 2024-09-13T10:10:40.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_snap_rstat": "false", 2024-09-13T10:10:40.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_standby_replay_damaged": "false", 2024-09-13T10:10:40.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_symlink_recovery": "true", 2024-09-13T10:10:40.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-13T10:10:40.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_thrash_exports": "0", 2024-09-13T10:10:40.661 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_thrash_fragments": "0", 2024-09-13T10:10:40.661 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_tick_interval": "5.000000", 2024-09-13T10:10:40.661 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_valgrind_exit": "false", 2024-09-13T10:10:40.661 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_verify_backtrace": "1", 2024-09-13T10:10:40.661 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_verify_scatter": "false", 2024-09-13T10:10:40.661 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-13T10:10:40.661 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_wipe_sessions": "false", 2024-09-13T10:10:40.661 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool_debug": "false", 2024-09-13T10:10:40.661 INFO:teuthology.orchestra.run.smithi073.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-13T10:10:40.661 INFO:teuthology.orchestra.run.smithi073.stdout: "memstore_device_bytes": "1073741824", 2024-09-13T10:10:40.661 INFO:teuthology.orchestra.run.smithi073.stdout: "memstore_page_set": "false", 2024-09-13T10:10:40.662 INFO:teuthology.orchestra.run.smithi073.stdout: "memstore_page_size": "65536", 2024-09-13T10:10:40.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_client_bytes": "134217728", 2024-09-13T10:10:40.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_client_messages": "512", 2024-09-13T10:10:40.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-13T10:10:40.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-13T10:10:40.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-13T10:10:40.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-13T10:10:40.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_disabled_modules": "", 2024-09-13T10:10:40.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_enable_op_tracker": "true", 2024-09-13T10:10:40.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-13T10:10:40.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_max_pg_creating": "1024", 2024-09-13T10:10:40.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_max_pg_num_change": "128", 2024-09-13T10:10:40.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_mds_bytes": "134217728", 2024-09-13T10:10:40.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_mds_messages": "128", 2024-09-13T10:10:40.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-13T10:10:40.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_mon_bytes": "134217728", 2024-09-13T10:10:40.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_mon_messages": "128", 2024-09-13T10:10:40.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-13T10:10:40.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-13T10:10:40.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_op_history_duration": "600", 2024-09-13T10:10:40.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_op_history_size": "20", 2024-09-13T10:10:40.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-13T10:10:40.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-13T10:10:40.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_op_log_threshold": "5", 2024-09-13T10:10:40.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_osd_bytes": "536870912", 2024-09-13T10:10:40.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_osd_messages": "8192", 2024-09-13T10:10:40.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_pool": "true", 2024-09-13T10:10:40.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-13T10:10:40.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_standby_modules": "true", 2024-09-13T10:10:40.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_stats_period": "5", 2024-09-13T10:10:40.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_stats_threshold": "5", 2024-09-13T10:10:40.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_tick_period": "2", 2024-09-13T10:10:40.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-13T10:10:40.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-13T10:10:40.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_allow_pool_delete": "true", 2024-09-13T10:10:40.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_allow_pool_size_one": "true", 2024-09-13T10:10:40.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_auth_validate_all_caps": "true", 2024-09-13T10:10:40.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-13T10:10:40.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-13T10:10:40.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_bytes": "104857600", 2024-09-13T10:10:40.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_directed_command_retry": "5", 2024-09-13T10:10:40.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-13T10:10:40.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-13T10:10:40.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-13T10:10:40.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-13T10:10:40.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_hunt_parallel": "3", 2024-09-13T10:10:40.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_log_interval": "1.000000", 2024-09-13T10:10:40.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-13T10:10:40.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_ping_interval": "10.000000", 2024-09-13T10:10:40.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-13T10:10:40.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_target_rank": "-1", 2024-09-13T10:10:40.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-13T10:10:40.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-13T10:10:40.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-13T10:10:40.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_level": "debug", 2024-09-13T10:10:40.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_file": "true", 2024-09-13T10:10:40.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-13T10:10:40.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-13T10:10:40.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-13T10:10:40.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_journald": "false", 2024-09-13T10:10:40.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-13T10:10:40.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-13T10:10:40.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-13T10:10:40.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_compact_on_bootstrap": "false", 2024-09-13T10:10:40.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_compact_on_start": "false", 2024-09-13T10:10:40.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_compact_on_trim": "true", 2024-09-13T10:10:40.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-13T10:10:40.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-13T10:10:40.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-13T10:10:40.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cpu_threads": "4", 2024-09-13T10:10:40.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_crush_min_required_version": "hammer", 2024-09-13T10:10:40.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_daemon_bytes": "419430400", 2024-09-13T10:10:40.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-13T10:10:40.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_data_avail_crit": "5", 2024-09-13T10:10:40.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_data_avail_warn": "5", 2024-09-13T10:10:40.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_data_size_warn": "16106127360", 2024-09-13T10:10:40.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-13T10:10:40.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-13T10:10:40.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_dump_json": "false", 2024-09-13T10:10:40.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-13T10:10:40.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_dump_transactions": "false", 2024-09-13T10:10:40.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_extra_checks": "false", 2024-09-13T10:10:40.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-13T10:10:40.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-13T10:10:40.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_no_require_reef": "false", 2024-09-13T10:10:40.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_no_require_squid": "false", 2024-09-13T10:10:40.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-13T10:10:40.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-13T10:10:40.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-13T10:10:40.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_down_mkfs_grace": "120", 2024-09-13T10:10:40.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_down_uptime_grace": "60", 2024-09-13T10:10:40.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_election_default_strategy": "3", 2024-09-13T10:10:40.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_election_timeout": "5.000000", 2024-09-13T10:10:40.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-13T10:10:40.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_elector_ping_divisor": "2", 2024-09-13T10:10:40.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-13T10:10:40.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_enable_op_tracker": "true", 2024-09-13T10:10:40.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_fake_pool_delete": "false", 2024-09-13T10:10:40.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_force_quorum_join": "false", 2024-09-13T10:10:40.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-13T10:10:40.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_globalid_prealloc": "10000", 2024-09-13T10:10:40.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_detail_to_clog": "false", 2024-09-13T10:10:40.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_log_update_period": "5", 2024-09-13T10:10:40.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_max_detail": "50", 2024-09-13T10:10:40.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_to_clog": "true", 2024-09-13T10:10:40.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_to_clog_interval": "600", 2024-09-13T10:10:40.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-13T10:10:40.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_host": "172.21.15.73,172.21.15.136,172.21.15.181", 2024-09-13T10:10:40.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_host_override": "", 2024-09-13T10:10:40.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_initial_members": "", 2024-09-13T10:10:40.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-13T10:10:40.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-13T10:10:40.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-13T10:10:40.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-13T10:10:40.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-13T10:10:40.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_lease": "5.000000", 2024-09-13T10:10:40.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-13T10:10:40.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-13T10:10:40.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_log_full_interval": "50", 2024-09-13T10:10:40.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_log_max": "10000", 2024-09-13T10:10:40.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_log_max_summary": "50", 2024-09-13T10:10:40.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-13T10:10:40.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_log_epochs": "500", 2024-09-13T10:10:40.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-13T10:10:40.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-13T10:10:40.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-13T10:10:40.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_osd": "10000", 2024-09-13T10:10:40.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_pg_per_osd": "10000", 2024-09-13T10:10:40.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_pool_pg_num": "65536", 2024-09-13T10:10:40.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-13T10:10:40.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-13T10:10:40.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mds_force_trim_to": "0", 2024-09-13T10:10:40.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mds_skip_sanity": "false", 2024-09-13T10:10:40.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_memory_autotune": "true", 2024-09-13T10:10:40.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_memory_target": "2147483648", 2024-09-13T10:10:40.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_beacon_grace": "30", 2024-09-13T10:10:40.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-13T10:10:40.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_digest_period": "5", 2024-09-13T10:10:40.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_inactive_grace": "60", 2024-09-13T10:10:40.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-13T10:10:40.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-13T10:10:40.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_min_osdmap_epochs": "50", 2024-09-13T10:10:40.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-13T10:10:40.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-13T10:10:40.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_complaint_time": "30", 2024-09-13T10:10:40.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_history_duration": "600", 2024-09-13T10:10:40.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_history_size": "20", 2024-09-13T10:10:40.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_history_slow_op_size": "20", 2024-09-13T10:10:40.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-13T10:10:40.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_log_threshold": "5", 2024-09-13T10:10:40.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-13T10:10:40.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-13T10:10:40.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-13T10:10:40.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_auto_mark_in": "false", 2024-09-13T10:10:40.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-13T10:10:40.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-13T10:10:40.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-13T10:10:40.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_cache_size": "500", 2024-09-13T10:10:40.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-13T10:10:40.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-13T10:10:40.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-13T10:10:40.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_down_out_interval": "600", 2024-09-13T10:10:40.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-13T10:10:40.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-13T10:10:40.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_force_trim_to": "0", 2024-09-13T10:10:40.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-13T10:10:40.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-13T10:10:40.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-13T10:10:40.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-13T10:10:40.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-13T10:10:40.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-13T10:10:40.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-13T10:10:40.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_min_down_reporters": "2", 2024-09-13T10:10:40.675 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-13T10:10:40.676 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-13T10:10:40.676 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-13T10:10:40.676 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-13T10:10:40.676 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-13T10:10:40.676 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-13T10:10:40.676 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_report_timeout": "900", 2024-09-13T10:10:40.676 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-13T10:10:40.676 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-13T10:10:40.676 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-13T10:10:40.676 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-13T10:10:40.676 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-13T10:10:40.676 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osdmap_full_prune_interval": "2", 2024-09-13T10:10:40.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osdmap_full_prune_min": "15", 2024-09-13T10:10:40.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-09-13T10:10:40.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-13T10:10:40.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_stuck_threshold": "60", 2024-09-13T10:10:40.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-13T10:10:40.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-13T10:10:40.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-13T10:10:40.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-13T10:10:40.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-13T10:10:40.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-13T10:10:40.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_probe_timeout": "2.000000", 2024-09-13T10:10:40.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_reweight_max_change": "0.050000", 2024-09-13T10:10:40.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_reweight_max_osds": "4", 2024-09-13T10:10:40.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-13T10:10:40.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-13T10:10:40.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-13T10:10:40.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-13T10:10:40.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-13T10:10:40.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_scrub_interval": "300", 2024-09-13T10:10:40.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_scrub_max_keys": "100", 2024-09-13T10:10:40.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_scrub_timeout": "300", 2024-09-13T10:10:40.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_session_timeout": "300", 2024-09-13T10:10:40.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_smart_report_timeout": "5", 2024-09-13T10:10:40.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_stat_smooth_intervals": "6", 2024-09-13T10:10:40.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-13T10:10:40.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_stretch_pool_min_size": "2", 2024-09-13T10:10:40.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_stretch_pool_size": "4", 2024-09-13T10:10:40.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-13T10:10:40.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-13T10:10:40.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_debug": "false", 2024-09-13T10:10:40.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-13T10:10:40.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-13T10:10:40.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_provider_kill_at": "0", 2024-09-13T10:10:40.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_requester_kill_at": "0", 2024-09-13T10:10:40.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_timeout": "60.000000", 2024-09-13T10:10:40.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_target_pg_per_osd": "100", 2024-09-13T10:10:40.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_tick_interval": "5", 2024-09-13T10:10:40.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_timecheck_interval": "300.000000", 2024-09-13T10:10:40.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-13T10:10:40.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_use_min_delay_socket": "false", 2024-09-13T10:10:40.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_older_version_delay": "604800", 2024-09-13T10:10:40.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-13T10:10:40.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-13T10:10:40.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-13T10:10:40.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-13T10:10:40.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-13T10:10:40.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-13T10:10:40.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-13T10:10:40.681 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_misplaced": "false", 2024-09-13T10:10:40.681 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-13T10:10:40.681 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_older_version": "true", 2024-09-13T10:10:40.681 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-13T10:10:40.681 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-13T10:10:40.681 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-13T10:10:40.681 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-13T10:10:40.681 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-13T10:10:40.681 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-13T10:10:40.681 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-13T10:10:40.681 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-13T10:10:40.682 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-13T10:10:40.682 INFO:teuthology.orchestra.run.smithi073.stdout: "monmap": "", 2024-09-13T10:10:40.682 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T10:10:40.682 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-13T10:10:40.682 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-13T10:10:40.682 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T10:10:40.682 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-13T10:10:40.682 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-13T10:10:40.682 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_tracing_enabled": "false", 2024-09-13T10:10:40.682 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_op_threads": "3", 2024-09-13T10:10:40.682 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-13T10:10:40.682 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_cm": "false", 2024-09-13T10:10:40.683 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_device_name": "", 2024-09-13T10:10:40.683 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_dscp": "96", 2024-09-13T10:10:40.683 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-13T10:10:40.683 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-13T10:10:40.683 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_local_gid": "", 2024-09-13T10:10:40.683 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-13T10:10:40.683 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_port_num": "1", 2024-09-13T10:10:40.683 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-13T10:10:40.683 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-13T10:10:40.683 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-13T10:10:40.683 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-13T10:10:40.683 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_sl": "3", 2024-09-13T10:10:40.683 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_support_srq": "true", 2024-09-13T10:10:40.683 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_type": "ib", 2024-09-13T10:10:40.684 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_reap_threshold": "5", 2024-09-13T10:10:40.684 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_before_connect": "false", 2024-09-13T10:10:40.684 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_ipv4": "true", 2024-09-13T10:10:40.684 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_ipv6": "false", 2024-09-13T10:10:40.684 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_msgr1": "false", 2024-09-13T10:10:40.684 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_msgr2": "true", 2024-09-13T10:10:40.684 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_port_max": "7568", 2024-09-13T10:10:40.684 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_port_min": "6800", 2024-09-13T10:10:40.684 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-13T10:10:40.684 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_retry_count": "3", 2024-09-13T10:10:40.684 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_retry_delay": "5", 2024-09-13T10:10:40.684 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_blackhole_client": "false", 2024-09-13T10:10:40.685 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_blackhole_mds": "false", 2024-09-13T10:10:40.685 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_blackhole_mgr": "false", 2024-09-13T10:10:40.685 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_blackhole_mon": "false", 2024-09-13T10:10:40.685 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_blackhole_osd": "false", 2024-09-13T10:10:40.685 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_client_mode": "crc secure", 2024-09-13T10:10:40.685 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-13T10:10:40.685 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_cluster_mode": "crc secure", 2024-09-13T10:10:40.685 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_cluster_type": "", 2024-09-13T10:10:40.685 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_compress_secure": "false", 2024-09-13T10:10:40.685 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_connection_idle_timeout": "900", 2024-09-13T10:10:40.685 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_connection_ready_timeout": "10", 2024-09-13T10:10:40.685 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_crc_data": "true", 2024-09-13T10:10:40.686 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_crc_header": "true", 2024-09-13T10:10:40.686 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_die_on_bad_msg": "false", 2024-09-13T10:10:40.686 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_die_on_bug": "true", 2024-09-13T10:10:40.686 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_die_on_old_message": "true", 2024-09-13T10:10:40.686 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_die_on_skipped_message": "false", 2024-09-13T10:10:40.686 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-13T10:10:40.686 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-13T10:10:40.686 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_coremask": "0xF", 2024-09-13T10:10:40.686 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-13T10:10:40.686 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-13T10:10:40.686 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_enable_tso": "true", 2024-09-13T10:10:40.686 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-13T10:10:40.687 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-13T10:10:40.687 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_hugepages": "", 2024-09-13T10:10:40.687 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-13T10:10:40.687 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-13T10:10:40.687 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_lro": "true", 2024-09-13T10:10:40.687 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_memory_channel": "4", 2024-09-13T10:10:40.687 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-13T10:10:40.687 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_pmd": "", 2024-09-13T10:10:40.687 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_port_id": "0", 2024-09-13T10:10:40.687 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-13T10:10:40.687 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-13T10:10:40.687 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dump_on_send": "false", 2024-09-13T10:10:40.688 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_initial_backoff": "0.200000", 2024-09-13T10:10:40.688 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_delay_max": "1.000000", 2024-09-13T10:10:40.688 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-13T10:10:40.688 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_delay_type": "", 2024-09-13T10:10:40.688 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-13T10:10:40.688 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_network_congestion": "0", 2024-09-13T10:10:40.688 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_socket_failures": "5000", 2024-09-13T10:10:40.688 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_learn_addr_from_peer": "true", 2024-09-13T10:10:40.688 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_max_accept_failures": "4", 2024-09-13T10:10:40.688 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_max_backoff": "15.000000", 2024-09-13T10:10:40.688 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_mon_client_mode": "secure crc", 2024-09-13T10:10:40.688 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-13T10:10:40.688 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_mon_service_mode": "secure crc", 2024-09-13T10:10:40.689 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_osd_compress_min_size": "1024", 2024-09-13T10:10:40.689 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_osd_compress_mode": "none", 2024-09-13T10:10:40.689 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-13T10:10:40.689 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-13T10:10:40.689 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_pq_min_cost": "65536", 2024-09-13T10:10:40.689 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_public_type": "", 2024-09-13T10:10:40.689 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_service_mode": "crc secure", 2024-09-13T10:10:40.689 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_tcp_listen_backlog": "512", 2024-09-13T10:10:40.689 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_tcp_nodelay": "true", 2024-09-13T10:10:40.689 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-13T10:10:40.689 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_tcp_rcvbuf": "0", 2024-09-13T10:10:40.689 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-13T10:10:40.690 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_type": "async", 2024-09-13T10:10:40.690 INFO:teuthology.orchestra.run.smithi073.stdout: "no_config_file": "false", 2024-09-13T10:10:40.690 INFO:teuthology.orchestra.run.smithi073.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-13T10:10:40.690 INFO:teuthology.orchestra.run.smithi073.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-13T10:10:40.690 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_completion_locks_per_session": "32", 2024-09-13T10:10:40.690 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-13T10:10:40.690 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-13T10:10:40.690 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_inflight_ops": "1024", 2024-09-13T10:10:40.690 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-13T10:10:40.690 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-13T10:10:40.690 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_tick_interval": "5.000000", 2024-09-13T10:10:40.691 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_timeout": "10.000000", 2024-09-13T10:10:40.691 INFO:teuthology.orchestra.run.smithi073.stdout: "objectstore_blackhole": "false", 2024-09-13T10:10:40.691 INFO:teuthology.orchestra.run.smithi073.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-13T10:10:40.691 INFO:teuthology.orchestra.run.smithi073.stdout: "openssl_engine_opts": "", 2024-09-13T10:10:40.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_delay_time": "5.000000", 2024-09-13T10:10:40.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_hist_halflife": "1000", 2024-09-13T10:10:40.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_max_low_ops": "2", 2024-09-13T10:10:40.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_max_ops": "4", 2024-09-13T10:10:40.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-13T10:10:40.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-13T10:10:40.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_slop": "0.020000", 2024-09-13T10:10:40.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-13T10:10:40.692 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-13T10:10:40.692 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_asio_thread_count": "2", 2024-09-13T10:10:40.692 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_async_recovery_min_cost": "100", 2024-09-13T10:10:40.692 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-13T10:10:40.692 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-13T10:10:40.692 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backfill_scan_max": "512", 2024-09-13T10:10:40.692 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backfill_scan_min": "64", 2024-09-13T10:10:40.692 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backoff_on_degraded": "false", 2024-09-13T10:10:40.692 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backoff_on_peering": "false", 2024-09-13T10:10:40.692 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backoff_on_unfound": "true", 2024-09-13T10:10:40.692 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_beacon_report_interval": "300", 2024-09-13T10:10:40.693 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bench_duration": "30", 2024-09-13T10:10:40.693 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-13T10:10:40.693 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bench_max_block_size": "67108864", 2024-09-13T10:10:40.693 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-13T10:10:40.693 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_blkin_trace_all": "false", 2024-09-13T10:10:40.693 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-13T10:10:40.693 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-13T10:10:40.693 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-13T10:10:40.693 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-13T10:10:40.693 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_check_for_log_corruption": "false", 2024-09-13T10:10:40.693 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-13T10:10:40.694 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T10:10:40.694 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-09-13T10:10:40.694 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T10:10:40.694 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_class_update_on_start": "true", 2024-09-13T10:10:40.694 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_client_message_cap": "256", 2024-09-13T10:10:40.694 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_client_message_size_cap": "524288000", 2024-09-13T10:10:40.694 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_client_op_priority": "63", 2024-09-13T10:10:40.694 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_client_watch_timeout": "30", 2024-09-13T10:10:40.694 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_command_max_records": "256", 2024-09-13T10:10:40.694 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_compact_on_start": "false", 2024-09-13T10:10:40.694 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-13T10:10:40.695 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-13T10:10:40.695 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-13T10:10:40.695 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_crush_update_on_start": "true", 2024-09-13T10:10:40.695 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_crush_update_weight_set": "true", 2024-09-13T10:10:40.695 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-13T10:10:40.695 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-13T10:10:40.695 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-13T10:10:40.695 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-13T10:10:40.695 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-13T10:10:40.695 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_feed_pullee": "-1", 2024-09-13T10:10:40.695 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-13T10:10:40.695 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-13T10:10:40.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-13T10:10:40.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_misdirected_ops": "false", 2024-09-13T10:10:40.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_no_acting_change": "false", 2024-09-13T10:10:40.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_no_purge_strays": "false", 2024-09-13T10:10:40.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_op_order": "false", 2024-09-13T10:10:40.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-13T10:10:40.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-13T10:10:40.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-13T10:10:40.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-13T10:10:40.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_shutdown": "false", 2024-09-13T10:10:40.696 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-13T10:10:40.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-13T10:10:40.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_trim_objects": "false", 2024-09-13T10:10:40.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-13T10:10:40.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-13T10:10:40.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_verify_snaps": "false", 2024-09-13T10:10:40.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-13T10:10:40.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-13T10:10:40.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_keys": "1024", 2024-09-13T10:10:40.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-13T10:10:40.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-13T10:10:40.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-13T10:10:40.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_stride": "524288", 2024-09-13T10:10:40.697 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-13T10:10:40.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-13T10:10:40.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_default_notify_timeout": "30", 2024-09-13T10:10:40.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_delete_sleep": "0.000000", 2024-09-13T10:10:40.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-13T10:10:40.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-13T10:10:40.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-13T10:10:40.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_discard_disconnected_ops": "true", 2024-09-13T10:10:40.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_ec_partial_reads": "true", 2024-09-13T10:10:40.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_enable_op_tracker": "true", 2024-09-13T10:10:40.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-13T10:10:40.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-13T10:10:40.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-13T10:10:40.698 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-13T10:10:40.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_fast_info": "true", 2024-09-13T10:10:40.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_fast_shutdown": "true", 2024-09-13T10:10:40.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-13T10:10:40.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-13T10:10:40.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-13T10:10:40.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-13T10:10:40.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-13T10:10:40.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_function_tracing": "false", 2024-09-13T10:10:40.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_grace": "20", 2024-09-13T10:10:40.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_interval": "6", 2024-09-13T10:10:40.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-13T10:10:40.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_min_peers": "10", 2024-09-13T10:10:40.699 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_min_size": "2000", 2024-09-13T10:10:40.700 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_stale": "600", 2024-09-13T10:10:40.700 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-13T10:10:40.700 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_hit_set_max_size": "100000", 2024-09-13T10:10:40.700 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_hit_set_min_size": "1000", 2024-09-13T10:10:40.700 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-13T10:10:40.700 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-13T10:10:40.700 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-13T10:10:40.700 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-13T10:10:40.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-13T10:10:40.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-13T10:10:40.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_journal_size": "5120", 2024-09-13T10:10:40.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_kill_backfill_at": "0", 2024-09-13T10:10:40.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-13T10:10:40.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_map_cache_size": "50", 2024-09-13T10:10:40.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_map_dedup": "true", 2024-09-13T10:10:40.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_map_message_max": "40", 2024-09-13T10:10:40.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-13T10:10:40.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_map_share_max_epochs": "40", 2024-09-13T10:10:40.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_attr_name_len": "100", 2024-09-13T10:10:40.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_attr_size": "0", 2024-09-13T10:10:40.701 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_backfills": "1", 2024-09-13T10:10:40.702 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_markdown_count": "5", 2024-09-13T10:10:40.702 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_markdown_period": "600", 2024-09-13T10:10:40.702 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_object_name_len": "2048", 2024-09-13T10:10:40.702 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_object_namespace_len": "256", 2024-09-13T10:10:40.702 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_object_size": "134217728", 2024-09-13T10:10:40.702 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-13T10:10:40.702 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-13T10:10:40.702 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_pg_blocked_by": "16", 2024-09-13T10:10:40.702 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_pg_log_entries": "2", 2024-09-13T10:10:40.702 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-13T10:10:40.702 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_pgls": "1024", 2024-09-13T10:10:40.702 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_push_cost": "8388608", 2024-09-13T10:10:40.702 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_push_objects": "10", 2024-09-13T10:10:40.703 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_scrubs": "3", 2024-09-13T10:10:40.703 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-13T10:10:40.703 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_trimming_pgs": "2", 2024-09-13T10:10:40.703 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_write_op_reply_len": "64", 2024-09-13T10:10:40.703 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_write_size": "90", 2024-09-13T10:10:40.703 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-13T10:10:40.703 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-13T10:10:40.703 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-13T10:10:40.703 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-13T10:10:40.703 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-13T10:10:40.703 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-13T10:10:40.703 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-13T10:10:40.703 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-13T10:10:40.704 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_profile": "balanced", 2024-09-13T10:10:40.704 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-13T10:10:40.704 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-13T10:10:40.704 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-13T10:10:40.704 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-13T10:10:40.704 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-13T10:10:40.704 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-13T10:10:40.704 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-13T10:10:40.704 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-13T10:10:40.704 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-13T10:10:40.704 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-13T10:10:40.704 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-13T10:10:40.705 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_base": "805306368", 2024-09-13T10:10:40.705 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_cache_min": "134217728", 2024-09-13T10:10:40.705 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-13T10:10:40.705 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-13T10:10:40.705 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_target": "4294967296", 2024-09-13T10:10:40.705 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_target_autotune": "false", 2024-09-13T10:10:40.705 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-13T10:10:40.705 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_min_pg_log_entries": "1", 2024-09-13T10:10:40.705 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_min_recovery_priority": "0", 2024-09-13T10:10:40.705 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-13T10:10:40.705 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-13T10:10:40.705 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mon_report_interval": "5", 2024-09-13T10:10:40.706 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-13T10:10:40.706 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-13T10:10:40.706 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_num_cache_shards": "32", 2024-09-13T10:10:40.706 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_num_op_tracker_shard": "32", 2024-09-13T10:10:40.706 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_numa_auto_affinity": "true", 2024-09-13T10:10:40.706 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_numa_node": "-1", 2024-09-13T10:10:40.706 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_numa_prefer_iface": "true", 2024-09-13T10:10:40.706 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-13T10:10:40.706 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_objecter_finishers": "1", 2024-09-13T10:10:40.706 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_objectstore": "bluestore", 2024-09-13T10:10:40.706 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_objectstore_fuse": "false", 2024-09-13T10:10:40.706 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-13T10:10:40.706 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_objectstore_tracing": "false", 2024-09-13T10:10:40.707 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_complaint_time": "30.000000", 2024-09-13T10:10:40.707 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_history_duration": "600", 2024-09-13T10:10:40.707 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_history_size": "20", 2024-09-13T10:10:40.707 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_history_slow_op_size": "20", 2024-09-13T10:10:40.707 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-13T10:10:40.707 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_log_threshold": "5", 2024-09-13T10:10:40.707 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_shards": "0", 2024-09-13T10:10:40.707 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_shards_hdd": "5", 2024-09-13T10:10:40.707 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_shards_ssd": "8", 2024-09-13T10:10:40.707 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-13T10:10:40.707 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-13T10:10:40.707 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-13T10:10:40.707 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-13T10:10:40.708 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_pq_min_cost": "65536", 2024-09-13T10:10:40.708 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-13T10:10:40.708 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_queue_cut_off": "high", 2024-09-13T10:10:40.708 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-13T10:10:40.708 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_thread_timeout": "15", 2024-09-13T10:10:40.708 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_open_classes_on_start": "true", 2024-09-13T10:10:40.708 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_os_flags": "0", 2024-09-13T10:10:40.708 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_peering_op_priority": "255", 2024-09-13T10:10:40.708 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_delete_cost": "1048576", 2024-09-13T10:10:40.708 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_delete_priority": "5", 2024-09-13T10:10:40.708 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-13T10:10:40.708 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-13T10:10:40.709 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-13T10:10:40.709 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_log_trim_max": "10000", 2024-09-13T10:10:40.709 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_log_trim_min": "0", 2024-09-13T10:10:40.709 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-13T10:10:40.709 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-13T10:10:40.709 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-13T10:10:40.709 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-13T10:10:40.709 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-13T10:10:40.709 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-13T10:10:40.709 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-13T10:10:40.709 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-13T10:10:40.709 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-13T10:10:40.709 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-13T10:10:40.710 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_crimson": "false", 2024-09-13T10:10:40.710 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-13T10:10:40.710 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-13T10:10:40.710 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-13T10:10:40.710 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-13T10:10:40.710 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-13T10:10:40.710 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-13T10:10:40.710 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-13T10:10:40.710 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-13T10:10:40.710 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flags": "0", 2024-09-13T10:10:40.710 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-13T10:10:40.710 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_min_size": "2", 2024-09-13T10:10:40.710 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-13T10:10:40.711 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_pg_num": "32", 2024-09-13T10:10:40.711 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_pgp_num": "0", 2024-09-13T10:10:40.711 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-13T10:10:40.711 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_read_ratio": "70", 2024-09-13T10:10:40.711 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_size": "3", 2024-09-13T10:10:40.711 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_type": "replicated", 2024-09-13T10:10:40.711 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-13T10:10:40.711 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-13T10:10:40.711 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_push_per_object_cost": "1000", 2024-09-13T10:10:40.711 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-13T10:10:40.711 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recover_clone_overlap": "true", 2024-09-13T10:10:40.711 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-13T10:10:40.712 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_cost": "20971520", 2024-09-13T10:10:40.712 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-13T10:10:40.712 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_active": "0", 2024-09-13T10:10:40.712 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-13T10:10:40.712 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-13T10:10:40.712 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-13T10:10:40.712 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-13T10:10:40.712 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_single_start": "1", 2024-09-13T10:10:40.712 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_op_priority": "3", 2024-09-13T10:10:40.712 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-13T10:10:40.712 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_priority": "5", 2024-09-13T10:10:40.712 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-13T10:10:40.713 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_sleep": "0.000000", 2024-09-13T10:10:40.713 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-13T10:10:40.713 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-13T10:10:40.713 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-13T10:10:40.713 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_repair_during_recovery": "false", 2024-09-13T10:10:40.713 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_requested_scrub_priority": "120", 2024-09-13T10:10:40.713 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-13T10:10:40.713 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-13T10:10:40.713 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_auto_repair": "false", 2024-09-13T10:10:40.713 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-13T10:10:40.713 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-13T10:10:40.713 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_begin_hour": "0", 2024-09-13T10:10:40.714 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_begin_week_day": "0", 2024-09-13T10:10:40.714 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_chunk_max": "25", 2024-09-13T10:10:40.714 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_chunk_min": "5", 2024-09-13T10:10:40.714 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_cost": "52428800", 2024-09-13T10:10:40.714 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-13T10:10:40.714 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_during_recovery": "false", 2024-09-13T10:10:40.714 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_end_hour": "0", 2024-09-13T10:10:40.714 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_end_week_day": "0", 2024-09-13T10:10:40.714 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_event_cost": "4096", 2024-09-13T10:10:40.714 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-13T10:10:40.714 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-13T10:10:40.714 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_invalid_stats": "true", 2024-09-13T10:10:40.714 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-13T10:10:40.715 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-13T10:10:40.715 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_max_preemptions": "5", 2024-09-13T10:10:40.715 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-13T10:10:40.715 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_priority": "5", 2024-09-13T10:10:40.715 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_sleep": "0.000000", 2024-09-13T10:10:40.715 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-13T10:10:40.715 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-13T10:10:40.715 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-13T10:10:40.715 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-13T10:10:40.715 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_skip_data_digest": "false", 2024-09-13T10:10:40.715 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_smart_report_timeout": "5", 2024-09-13T10:10:40.715 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_cost": "1048576", 2024-09-13T10:10:40.716 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_priority": "5", 2024-09-13T10:10:40.716 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-13T10:10:40.716 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-13T10:10:40.716 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-13T10:10:40.716 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-13T10:10:40.716 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-13T10:10:40.716 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-13T10:10:40.716 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-13T10:10:40.716 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_target_transaction_size": "30", 2024-09-13T10:10:40.716 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-13T10:10:40.716 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-13T10:10:40.716 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-13T10:10:40.716 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-13T10:10:40.717 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-13T10:10:40.717 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-13T10:10:40.717 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-13T10:10:40.717 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-13T10:10:40.717 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-13T10:10:40.717 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-13T10:10:40.717 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tracing": "false", 2024-09-13T10:10:40.717 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_use_stale_snap": "false", 2024-09-13T10:10:40.717 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-13T10:10:40.717 INFO:teuthology.orchestra.run.smithi073.stdout: "osdc_blkin_trace_all": "false", 2024-09-13T10:10:40.717 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_kill_at": "0", 2024-09-13T10:10:40.717 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_max_join_drift": "10", 2024-09-13T10:10:40.717 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_min": "500", 2024-09-13T10:10:40.718 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_min_wait": "0.050000", 2024-09-13T10:10:40.718 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_propose_interval": "1.000000", 2024-09-13T10:10:40.718 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_service_trim_max": "500", 2024-09-13T10:10:40.718 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-13T10:10:40.718 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_service_trim_min": "10", 2024-09-13T10:10:40.718 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_stash_full_interval": "25", 2024-09-13T10:10:40.718 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_trim_max": "500", 2024-09-13T10:10:40.718 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_trim_min": "250", 2024-09-13T10:10:40.718 INFO:teuthology.orchestra.run.smithi073.stdout: "perf": "true", 2024-09-13T10:10:40.718 INFO:teuthology.orchestra.run.smithi073.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-13T10:10:40.718 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-13T10:10:40.718 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin_dir": "/usr/lib/ceph", 2024-09-13T10:10:40.718 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "-", 2024-09-13T10:10:40.719 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrv": "", 2024-09-13T10:10:40.719 INFO:teuthology.orchestra.run.smithi073.stdout: "public_bind_addr": "-", 2024-09-13T10:10:40.719 INFO:teuthology.orchestra.run.smithi073.stdout: "public_network": "", 2024-09-13T10:10:40.719 INFO:teuthology.orchestra.run.smithi073.stdout: "public_network_interface": "", 2024-09-13T10:10:40.719 INFO:teuthology.orchestra.run.smithi073.stdout: "qat_compressor_busy_polling": "false", 2024-09-13T10:10:40.719 INFO:teuthology.orchestra.run.smithi073.stdout: "qat_compressor_enabled": "false", 2024-09-13T10:10:40.719 INFO:teuthology.orchestra.run.smithi073.stdout: "qat_compressor_session_max_number": "256", 2024-09-13T10:10:40.719 INFO:teuthology.orchestra.run.smithi073.stdout: "rados_mon_op_timeout": "0", 2024-09-13T10:10:40.719 INFO:teuthology.orchestra.run.smithi073.stdout: "rados_osd_op_timeout": "0", 2024-09-13T10:10:40.719 INFO:teuthology.orchestra.run.smithi073.stdout: "rados_tracing": "false", 2024-09-13T10:10:40.719 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_atime_update_interval": "60", 2024-09-13T10:10:40.719 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-13T10:10:40.720 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_balance_parent_reads": "false", 2024-09-13T10:10:40.720 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_balance_snap_reads": "false", 2024-09-13T10:10:40.720 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_blkin_trace_all": "false", 2024-09-13T10:10:40.720 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-13T10:10:40.720 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-13T10:10:40.720 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache": "true", 2024-09-13T10:10:40.720 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-13T10:10:40.720 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-13T10:10:40.720 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-13T10:10:40.720 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-13T10:10:40.720 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_policy": "writearound", 2024-09-13T10:10:40.720 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_size": "33554432", 2024-09-13T10:10:40.720 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-13T10:10:40.721 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-13T10:10:40.721 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_clone_copy_on_read": "false", 2024-09-13T10:10:40.721 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_compression_hint": "none", 2024-09-13T10:10:40.721 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_concurrent_management_ops": "10", 2024-09-13T10:10:40.721 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-13T10:10:40.721 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_clone_format": "auto", 2024-09-13T10:10:40.721 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_data_pool": "", 2024-09-13T10:10:40.721 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_features": "61", 2024-09-13T10:10:40.721 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_format": "2", 2024-09-13T10:10:40.721 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_map_options": "", 2024-09-13T10:10:40.721 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_order": "22", 2024-09-13T10:10:40.721 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_pool": "rbd", 2024-09-13T10:10:40.721 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-13T10:10:40.721 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_stripe_count": "0", 2024-09-13T10:10:40.722 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_stripe_unit": "0", 2024-09-13T10:10:40.722 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-13T10:10:40.722 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-13T10:10:40.722 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-13T10:10:40.722 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_enable_alloc_hint": "true", 2024-09-13T10:10:40.722 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-13T10:10:40.722 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_io_scheduler": "simple", 2024-09-13T10:10:40.722 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-13T10:10:40.722 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-13T10:10:40.722 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-13T10:10:40.722 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-13T10:10:40.722 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-13T10:10:40.723 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-13T10:10:40.723 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-13T10:10:40.723 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-13T10:10:40.723 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-13T10:10:40.723 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_order": "24", 2024-09-13T10:10:40.723 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_pool": "", 2024-09-13T10:10:40.723 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_splay_width": "4", 2024-09-13T10:10:40.723 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_localize_parent_reads": "false", 2024-09-13T10:10:40.723 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_localize_snap_reads": "false", 2024-09-13T10:10:40.723 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-13T10:10:40.723 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-13T10:10:40.723 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-13T10:10:40.723 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-13T10:10:40.724 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-13T10:10:40.724 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-13T10:10:40.724 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-13T10:10:40.724 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-13T10:10:40.724 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-13T10:10:40.724 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-13T10:10:40.724 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-13T10:10:40.724 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-13T10:10:40.724 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-13T10:10:40.724 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-13T10:10:40.724 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-13T10:10:40.724 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-13T10:10:40.725 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-13T10:10:40.725 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-13T10:10:40.725 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-13T10:10:40.725 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-13T10:10:40.725 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-13T10:10:40.725 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-13T10:10:40.725 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-13T10:10:40.725 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-13T10:10:40.725 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-13T10:10:40.725 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-13T10:10:40.725 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-13T10:10:40.725 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-13T10:10:40.725 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-13T10:10:40.726 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-13T10:10:40.726 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-13T10:10:40.726 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mtime_update_interval": "60", 2024-09-13T10:10:40.726 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_non_blocking_aio": "true", 2024-09-13T10:10:40.726 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_op_thread_timeout": "60", 2024-09-13T10:10:40.726 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_op_threads": "1", 2024-09-13T10:10:40.726 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_parent_cache_enabled": "false", 2024-09-13T10:10:40.726 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-13T10:10:40.726 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-13T10:10:40.726 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-13T10:10:40.726 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_plugins": "", 2024-09-13T10:10:40.726 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_bps_burst": "0", 2024-09-13T10:10:40.726 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-13T10:10:40.727 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_bps_limit": "0", 2024-09-13T10:10:40.727 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_exclude_ops": "0", 2024-09-13T10:10:40.727 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_iops_burst": "0", 2024-09-13T10:10:40.727 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-13T10:10:40.727 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_iops_limit": "0", 2024-09-13T10:10:40.727 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-13T10:10:40.727 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-13T10:10:40.727 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-13T10:10:40.727 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-13T10:10:40.727 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-13T10:10:40.727 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-13T10:10:40.727 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-13T10:10:40.727 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-13T10:10:40.728 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-13T10:10:40.728 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-13T10:10:40.728 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-13T10:10:40.728 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-13T10:10:40.728 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-13T10:10:40.728 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-13T10:10:40.728 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_read_from_replica_policy": "default", 2024-09-13T10:10:40.728 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-13T10:10:40.728 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-13T10:10:40.728 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-13T10:10:40.728 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-13T10:10:40.729 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_skip_partial_discard": "true", 2024-09-13T10:10:40.729 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-13T10:10:40.729 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_tracing": "false", 2024-09-13T10:10:40.729 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_validate_names": "true", 2024-09-13T10:10:40.729 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_validate_pool": "true", 2024-09-13T10:10:40.729 INFO:teuthology.orchestra.run.smithi073.stdout: "restapi_base_url": "", 2024-09-13T10:10:40.729 INFO:teuthology.orchestra.run.smithi073.stdout: "restapi_log_level": "", 2024-09-13T10:10:40.729 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-13T10:10:40.729 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-13T10:10:40.729 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_acl_grants_max_num": "100", 2024-09-13T10:10:40.729 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_admin_entry": "admin", 2024-09-13T10:10:40.729 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-13T10:10:40.730 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_asio_assert_yielding": "false", 2024-09-13T10:10:40.730 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_backend_store": "rados", 2024-09-13T10:10:40.730 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_barbican_url": "", 2024-09-13T10:10:40.730 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_beast_enable_async": "true", 2024-09-13T10:10:40.730 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_counters_cache": "false", 2024-09-13T10:10:40.730 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-13T10:10:40.730 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-13T10:10:40.730 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-13T10:10:40.730 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-13T10:10:40.730 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-13T10:10:40.730 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-13T10:10:40.730 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-13T10:10:40.731 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-13T10:10:40.731 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_cache_enabled": "true", 2024-09-13T10:10:40.731 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_cache_expiry_interval": "900", 2024-09-13T10:10:40.731 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_cache_lru_size": "25000", 2024-09-13T10:10:40.731 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_config_store": "rados", 2024-09-13T10:10:40.731 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_content_length_compat": "false", 2024-09-13T10:10:40.731 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_copy_obj_progress": "true", 2024-09-13T10:10:40.731 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-13T10:10:40.731 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_cors_rules_max_num": "100", 2024-09-13T10:10:40.731 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_cross_domain_policy": "", 2024-09-13T10:10:40.731 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-13T10:10:40.731 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_addr": "", 2024-09-13T10:10:40.732 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-13T10:10:40.732 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-13T10:10:40.732 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-13T10:10:40.732 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-13T10:10:40.732 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_password": "", 2024-09-13T10:10:40.732 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-13T10:10:40.732 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_username": "", 2024-09-13T10:10:40.732 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_require_ssl": "true", 2024-09-13T10:10:40.732 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-13T10:10:40.732 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-13T10:10:40.732 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-13T10:10:40.732 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-13T10:10:40.732 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-13T10:10:40.733 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-13T10:10:40.733 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-13T10:10:40.733 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-13T10:10:40.733 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-13T10:10:40.733 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-13T10:10:40.733 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-13T10:10:40.733 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-13T10:10:40.733 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-13T10:10:40.733 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-13T10:10:40.733 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-13T10:10:40.733 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_addr": "", 2024-09-13T10:10:40.733 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_auth": "token", 2024-09-13T10:10:40.734 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_namespace": "", 2024-09-13T10:10:40.734 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_prefix": "", 2024-09-13T10:10:40.734 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-13T10:10:40.734 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-13T10:10:40.734 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-13T10:10:40.734 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-13T10:10:40.734 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_token_file": "", 2024-09-13T10:10:40.734 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-13T10:10:40.734 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_curl_buffersize": "524288", 2024-09-13T10:10:40.734 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-13T10:10:40.734 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_curl_low_speed_time": "300", 2024-09-13T10:10:40.734 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-13T10:10:40.734 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-13T10:10:40.735 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-13T10:10:40.735 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-13T10:10:40.735 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-13T10:10:40.735 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-13T10:10:40.735 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-13T10:10:40.735 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-13T10:10:40.735 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-13T10:10:40.735 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-13T10:10:40.735 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-13T10:10:40.735 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-13T10:10:40.735 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-13T10:10:40.735 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-13T10:10:40.736 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-13T10:10:40.736 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-13T10:10:40.736 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-13T10:10:40.736 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-13T10:10:40.736 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-13T10:10:40.736 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_log_changes_size": "1000", 2024-09-13T10:10:40.736 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_log_num_shards": "128", 2024-09-13T10:10:40.736 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-13T10:10:40.736 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_log_window": "30", 2024-09-13T10:10:40.736 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-13T10:10:40.736 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-13T10:10:40.736 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-13T10:10:40.737 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-13T10:10:40.737 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-13T10:10:40.737 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-13T10:10:40.737 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-13T10:10:40.737 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-13T10:10:40.737 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-13T10:10:40.737 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-13T10:10:40.737 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-13T10:10:40.737 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-13T10:10:40.737 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-13T10:10:40.737 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_disable_s3select": "false", 2024-09-13T10:10:40.737 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-13T10:10:40.737 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-13T10:10:40.738 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-13T10:10:40.738 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-13T10:10:40.738 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-13T10:10:40.738 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-13T10:10:40.738 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-13T10:10:40.738 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-13T10:10:40.738 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-13T10:10:40.738 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-13T10:10:40.738 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-13T10:10:40.738 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-13T10:10:40.738 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dns_name": "", 2024-09-13T10:10:40.738 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dns_s3website_name": "", 2024-09-13T10:10:40.738 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dynamic_resharding": "true", 2024-09-13T10:10:40.739 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-13T10:10:40.739 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-13T10:10:40.739 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-13T10:10:40.739 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_gc_threads": "true", 2024-09-13T10:10:40.739 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_lc_threads": "true", 2024-09-13T10:10:40.739 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_mdsearch": "true", 2024-09-13T10:10:40.739 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_ops_log": "false", 2024-09-13T10:10:40.739 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_quota_threads": "true", 2024-09-13T10:10:40.739 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_static_website": "false", 2024-09-13T10:10:40.739 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_usage_log": "false", 2024-09-13T10:10:40.739 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enforce_swift_acls": "true", 2024-09-13T10:10:40.739 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_exit_timeout_secs": "120", 2024-09-13T10:10:40.739 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_expose_bucket": "false", 2024-09-13T10:10:40.740 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_extended_http_attrs": "", 2024-09-13T10:10:40.740 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_filter": "none", 2024-09-13T10:10:40.740 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-13T10:10:40.740 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_frontends": "beast port=7480", 2024-09-13T10:10:40.740 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-13T10:10:40.740 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_deferred": "50", 2024-09-13T10:10:40.740 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-13T10:10:40.740 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_objs": "32", 2024-09-13T10:10:40.740 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-13T10:10:40.740 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-13T10:10:40.740 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-13T10:10:40.741 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-13T10:10:40.741 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_processor_period": "3600", 2024-09-13T10:10:40.741 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-13T10:10:40.741 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-13T10:10:40.741 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-13T10:10:40.741 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-13T10:10:40.741 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_init_timeout": "300", 2024-09-13T10:10:40.741 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-13T10:10:40.741 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-13T10:10:40.741 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_kafka_connection_idle": "300", 2024-09-13T10:10:40.741 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-13T10:10:40.741 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-13T10:10:40.741 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-13T10:10:40.742 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-13T10:10:40.742 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-13T10:10:40.742 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_domain": "", 2024-09-13T10:10:40.742 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_password": "", 2024-09-13T10:10:40.742 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_password_path": "", 2024-09-13T10:10:40.742 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_project": "", 2024-09-13T10:10:40.742 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_tenant": "", 2024-09-13T10:10:40.742 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_token": "", 2024-09-13T10:10:40.742 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_token_path": "", 2024-09-13T10:10:40.742 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_user": "", 2024-09-13T10:10:40.742 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_barbican_domain": "", 2024-09-13T10:10:40.742 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_barbican_password": "", 2024-09-13T10:10:40.742 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_barbican_project": "", 2024-09-13T10:10:40.742 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-13T10:10:40.743 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_barbican_user": "", 2024-09-13T10:10:40.743 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-13T10:10:40.743 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-13T10:10:40.743 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-13T10:10:40.743 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-13T10:10:40.743 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-13T10:10:40.743 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_url": "", 2024-09-13T10:10:40.743 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-13T10:10:40.743 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_debug_interval": "-1", 2024-09-13T10:10:40.743 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_lock_max_time": "90", 2024-09-13T10:10:40.743 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_max_objs": "32", 2024-09-13T10:10:40.743 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_max_rules": "1000", 2024-09-13T10:10:40.743 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_max_worker": "3", 2024-09-13T10:10:40.743 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-13T10:10:40.744 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_thread_delay": "0", 2024-09-13T10:10:40.744 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-13T10:10:40.744 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_dnattr": "uid", 2024-09-13T10:10:40.744 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-13T10:10:40.744 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_searchfilter": "", 2024-09-13T10:10:40.744 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-13T10:10:40.744 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_uri": "", 2024-09-13T10:10:40.744 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-13T10:10:40.744 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-13T10:10:40.744 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-13T10:10:40.744 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-13T10:10:40.744 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_log_http_headers": "", 2024-09-13T10:10:40.744 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-13T10:10:40.745 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-13T10:10:40.745 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_log_object_name_utc": "false", 2024-09-13T10:10:40.745 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-13T10:10:40.745 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-13T10:10:40.745 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_attr_name_len": "0", 2024-09-13T10:10:40.745 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_attr_size": "0", 2024-09-13T10:10:40.745 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-13T10:10:40.745 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_chunk_size": "4194304", 2024-09-13T10:10:40.745 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-13T10:10:40.745 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-13T10:10:40.745 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-13T10:10:40.745 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_listing_results": "1000", 2024-09-13T10:10:40.745 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_notify_retries": "10", 2024-09-13T10:10:40.746 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-13T10:10:40.746 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_put_param_size": "1048576", 2024-09-13T10:10:40.746 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_put_size": "5368709120", 2024-09-13T10:10:40.746 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_slo_entries": "1000", 2024-09-13T10:10:40.746 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_md_log_max_shards": "64", 2024-09-13T10:10:40.746 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-13T10:10:40.746 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-13T10:10:40.746 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-13T10:10:40.746 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-13T10:10:40.746 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_mp_lock_max_time": "600", 2024-09-13T10:10:40.747 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-13T10:10:40.747 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-13T10:10:40.747 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-13T10:10:40.747 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-13T10:10:40.747 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-13T10:10:40.747 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-13T10:10:40.747 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-13T10:10:40.747 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-13T10:10:40.747 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_max_gc": "300", 2024-09-13T10:10:40.747 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-13T10:10:40.747 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-13T10:10:40.747 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-13T10:10:40.748 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-13T10:10:40.748 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-13T10:10:40.748 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-13T10:10:40.748 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-13T10:10:40.748 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_num_async_rados_threads": "32", 2024-09-13T10:10:40.748 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_num_control_oids": "8", 2024-09-13T10:10:40.748 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_numa_node": "-1", 2024-09-13T10:10:40.748 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-13T10:10:40.748 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-13T10:10:40.748 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_objexp_chunk_size": "100", 2024-09-13T10:10:40.748 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_objexp_gc_interval": "600", 2024-09-13T10:10:40.748 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-13T10:10:40.748 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-13T10:10:40.749 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-13T10:10:40.749 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_op_thread_timeout": "600", 2024-09-13T10:10:40.749 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_op_tracing": "false", 2024-09-13T10:10:40.749 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_opa_token": "", 2024-09-13T10:10:40.749 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_opa_url": "", 2024-09-13T10:10:40.749 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_opa_verify_ssl": "true", 2024-09-13T10:10:40.749 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-13T10:10:40.749 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-13T10:10:40.749 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ops_log_rados": "false", 2024-09-13T10:10:40.749 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ops_log_socket_path": "", 2024-09-13T10:10:40.749 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-13T10:10:40.749 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-13T10:10:40.749 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-13T10:10:40.749 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_period_push_interval": "2.000000", 2024-09-13T10:10:40.750 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-13T10:10:40.750 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-13T10:10:40.750 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-13T10:10:40.750 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-13T10:10:40.750 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_posix_cache_lanes": "3", 2024-09-13T10:10:40.750 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-13T10:10:40.750 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-13T10:10:40.750 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_posix_cache_partitions": "3", 2024-09-13T10:10:40.750 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-13T10:10:40.750 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_print_continue": "true", 2024-09-13T10:10:40.750 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-13T10:10:40.750 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-13T10:10:40.750 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-13T10:10:40.750 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-13T10:10:40.751 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-13T10:10:40.751 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_rados_tracing": "false", 2024-09-13T10:10:40.751 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_realm": "", 2024-09-13T10:10:40.751 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_realm_id": "", 2024-09-13T10:10:40.751 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-13T10:10:40.751 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_region": "", 2024-09-13T10:10:40.751 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-13T10:10:40.751 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-13T10:10:40.751 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-13T10:10:40.751 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-13T10:10:40.751 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_request_uri": "", 2024-09-13T10:10:40.751 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_reshard_batch_size": "64", 2024-09-13T10:10:40.751 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-13T10:10:40.752 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-13T10:10:40.752 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_reshard_max_aio": "128", 2024-09-13T10:10:40.752 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_reshard_num_logs": "16", 2024-09-13T10:10:40.752 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_reshard_thread_interval": "600", 2024-09-13T10:10:40.752 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_resolve_cname": "false", 2024-09-13T10:10:40.752 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-13T10:10:40.752 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_run_sync_thread": "true", 2024-09-13T10:10:40.752 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-13T10:10:40.752 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-13T10:10:40.752 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-13T10:10:40.752 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-13T10:10:40.752 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-13T10:10:40.752 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-13T10:10:40.752 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-13T10:10:40.752 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-13T10:10:40.753 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-13T10:10:40.753 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_scheduler_type": "throttler", 2024-09-13T10:10:40.753 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_script_uri": "", 2024-09-13T10:10:40.753 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_service_provider_name": "", 2024-09-13T10:10:40.753 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-13T10:10:40.753 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_client_id": "", 2024-09-13T10:10:40.753 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_client_secret": "", 2024-09-13T10:10:40.753 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_entry": "sts", 2024-09-13T10:10:40.753 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_key": "sts", 2024-09-13T10:10:40.753 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-13T10:10:40.753 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_min_session_duration": "900", 2024-09-13T10:10:40.753 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_token_introspection_url": "", 2024-09-13T10:10:40.754 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_account_in_url": "false", 2024-09-13T10:10:40.754 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_auth_entry": "auth", 2024-09-13T10:10:40.754 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_auth_url": "", 2024-09-13T10:10:40.754 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_custom_header": "", 2024-09-13T10:10:40.754 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-13T10:10:40.754 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_need_stats": "true", 2024-09-13T10:10:40.754 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_tenant_name": "", 2024-09-13T10:10:40.754 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_token_expiration": "86400", 2024-09-13T10:10:40.754 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_url": "", 2024-09-13T10:10:40.754 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_url_prefix": "swift", 2024-09-13T10:10:40.754 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-13T10:10:40.755 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-13T10:10:40.755 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-13T10:10:40.755 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_lease_period": "120", 2024-09-13T10:10:40.755 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-13T10:10:40.755 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-13T10:10:40.755 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-13T10:10:40.755 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-13T10:10:40.755 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-13T10:10:40.755 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-13T10:10:40.755 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-13T10:10:40.755 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-13T10:10:40.755 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-13T10:10:40.756 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_thread_pool_size": "512", 2024-09-13T10:10:40.756 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-13T10:10:40.756 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-13T10:10:40.756 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-13T10:10:40.756 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-13T10:10:40.756 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_comment": "", 2024-09-13T10:10:40.756 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_createby": "", 2024-09-13T10:10:40.756 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_encoding": "", 2024-09-13T10:10:40.756 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_flag": "false", 2024-09-13T10:10:40.756 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-13T10:10:40.756 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_origin": "", 2024-09-13T10:10:40.756 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-13T10:10:40.756 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_tracker": "", 2024-09-13T10:10:40.757 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_trust_forwarded_https": "false", 2024-09-13T10:10:40.757 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-13T10:10:40.757 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-13T10:10:40.757 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_usage_max_shards": "32", 2024-09-13T10:10:40.757 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_usage_max_user_shards": "1", 2024-09-13T10:10:40.757 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_use_opa_authz": "false", 2024-09-13T10:10:40.757 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_counters_cache": "false", 2024-09-13T10:10:40.757 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-13T10:10:40.757 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-13T10:10:40.757 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-13T10:10:40.757 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_max_buckets": "1000", 2024-09-13T10:10:40.757 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_policies_max_num": "100", 2024-09-13T10:10:40.758 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-13T10:10:40.758 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-13T10:10:40.758 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-13T10:10:40.758 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-13T10:10:40.758 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_unique_email": "true", 2024-09-13T10:10:40.758 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_verify_ssl": "true", 2024-09-13T10:10:40.758 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-13T10:10:40.758 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zone": "", 2024-09-13T10:10:40.758 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zone_id": "", 2024-09-13T10:10:40.758 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-13T10:10:40.758 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zonegroup": "", 2024-09-13T10:10:40.758 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zonegroup_id": "", 2024-09-13T10:10:40.758 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-13T10:10:40.759 INFO:teuthology.orchestra.run.smithi073.stdout: "rgwlc_auto_session_clear": "true", 2024-09-13T10:10:40.759 INFO:teuthology.orchestra.run.smithi073.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-13T10:10:40.759 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_block_size": "4096", 2024-09-13T10:10:40.759 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-13T10:10:40.759 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-13T10:10:40.759 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-13T10:10:40.759 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-13T10:10:40.759 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-13T10:10:40.759 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_size": "536870912", 2024-09-13T10:10:40.759 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-13T10:10:40.759 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-13T10:10:40.759 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-13T10:10:40.759 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-13T10:10:40.760 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-13T10:10:40.760 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-13T10:10:40.760 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-13T10:10:40.760 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-13T10:10:40.760 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_index_type": "binary_search", 2024-09-13T10:10:40.760 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-13T10:10:40.760 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-13T10:10:40.760 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_partition_filters": "false", 2024-09-13T10:10:40.760 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_perf": "false", 2024-09-13T10:10:40.760 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-13T10:10:40.760 INFO:teuthology.orchestra.run.smithi073.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-13T10:10:40.760 INFO:teuthology.orchestra.run.smithi073.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-13T10:10:40.760 INFO:teuthology.orchestra.run.smithi073.stdout: "run_dir": "/var/run/ceph", 2024-09-13T10:10:40.760 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_block_create": "true", 2024-09-13T10:10:40.761 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_cache_lru_size": "67108864", 2024-09-13T10:10:40.761 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-13T10:10:40.761 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-13T10:10:40.761 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_default_max_object_size": "16777216", 2024-09-13T10:10:40.761 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-13T10:10:40.761 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_device_size": "53687091200", 2024-09-13T10:10:40.761 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-13T10:10:40.761 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_full_integrity_check": "false", 2024-09-13T10:10:40.761 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_journal_batch_capacity": "16", 2024-09-13T10:10:40.761 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-13T10:10:40.761 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-13T10:10:40.761 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-13T10:10:40.761 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_main_device_type": "SSD", 2024-09-13T10:10:40.762 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-13T10:10:40.762 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-13T10:10:40.762 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-13T10:10:40.762 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-13T10:10:40.762 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-13T10:10:40.762 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-13T10:10:40.762 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_segment_size": "67108864", 2024-09-13T10:10:40.762 INFO:teuthology.orchestra.run.smithi073.stdout: "set_keepcaps": "false", 2024-09-13T10:10:40.762 INFO:teuthology.orchestra.run.smithi073.stdout: "setgroup": "", 2024-09-13T10:10:40.762 INFO:teuthology.orchestra.run.smithi073.stdout: "setuser": "", 2024-09-13T10:10:40.762 INFO:teuthology.orchestra.run.smithi073.stdout: "setuser_match_path": "", 2024-09-13T10:10:40.762 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-13T10:10:40.762 INFO:teuthology.orchestra.run.smithi073.stdout: "thp": "false", 2024-09-13T10:10:40.763 INFO:teuthology.orchestra.run.smithi073.stdout: "threadpool_default_timeout": "60", 2024-09-13T10:10:40.763 INFO:teuthology.orchestra.run.smithi073.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-13T10:10:40.763 INFO:teuthology.orchestra.run.smithi073.stdout: "throttler_perf_counter": "true", 2024-09-13T10:10:40.763 INFO:teuthology.orchestra.run.smithi073.stdout: "tmp_dir": "/tmp", 2024-09-13T10:10:40.763 INFO:teuthology.orchestra.run.smithi073.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-13T10:10:40.763 INFO:teuthology.orchestra.run.smithi073.stdout: "uadk_compressor_enabled": "false", 2024-09-13T10:10:40.763 INFO:teuthology.orchestra.run.smithi073.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-13T10:10:40.763 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:10:40.766 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-13T10:10:40.904 INFO:teuthology.orchestra.run.smithi073.stdout:mon.a: {} 2024-09-13T10:10:40.905 INFO:teuthology.orchestra.run.smithi073.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T10:10:40.915 INFO:teuthology.orchestra.run.smithi073.stdout:mon.b: {} 2024-09-13T10:10:40.915 INFO:teuthology.orchestra.run.smithi073.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T10:10:40.929 INFO:teuthology.orchestra.run.smithi073.stdout:mon.c: {} 2024-09-13T10:10:40.930 INFO:teuthology.orchestra.run.smithi073.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T10:10:40.944 DEBUG:teuthology.orchestra.run.smithi195:> type ceph-objectstore-tool 2024-09-13T10:10:40.950 INFO:teuthology.orchestra.run.smithi195.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T10:10:40.950 DEBUG:teuthology.orchestra.run.smithi181:> type ceph-objectstore-tool 2024-09-13T10:10:40.956 INFO:teuthology.orchestra.run.smithi181.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T10:10:40.956 DEBUG:teuthology.orchestra.run.smithi136:> type ceph-objectstore-tool 2024-09-13T10:10:40.962 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T10:10:40.962 DEBUG:teuthology.orchestra.run.smithi073:> type ceph-objectstore-tool 2024-09-13T10:10:40.966 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T10:10:40.967 INFO:teuthology.run_tasks:Running task exec... 2024-09-13T10:10:40.975 INFO:teuthology.task.exec:Executing custom commands... 2024-09-13T10:10:40.975 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi195.front.sepia.ceph.com 2024-09-13T10:10:40.975 DEBUG:teuthology.orchestra.run.smithi195:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool create low_tier 4' 2024-09-13T10:10:40.976 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-13T10:10:40.976 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:10:40.976 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:10:40.976 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:10:40.976 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-13T10:10:40.976 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-13T10:10:40.977 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:10:40.977 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-13T10:10:40.977 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T10:10:40.977 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-13T10:10:40.978 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T10:10:41.013 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:10:41.014+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:10:41.079 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:10:41.078+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:10:41.123 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:10:41.124 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:10:41.124 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:10:41.124 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:10:41.140 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T10:10:41.146 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:10:41.146 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:10:41.167 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:10:41.167 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:10:41.180 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:41.178+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:10:41.188 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:10:41.188 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:10:41.247 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:41.247 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:41.259 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T10:10:41.280 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:41.276+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:41.381 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:10:41.382+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:10:41.407 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:41.407 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:41.417 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:10:41.417 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:10:41.420 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T10:10:41.433 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:10:41.434 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:10:41.462 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:10:41.462 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:10:41.482 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:10:41.480+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:10:41.489 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:10:41.489 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:10:41.515 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:10:41.515 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:10:41.538 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:10:41.538 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:10:41.544 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:41.544 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:41.557 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T10:10:41.561 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:10:41.561 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:10:41.583 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:41.580+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:41.586 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:10:41.586 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:10:41.615 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:10:41.615 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:10:41.633 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:10:41.633 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:10:41.656 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:10:41.656 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:10:41.669 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:41.670 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:41.679 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:10:41.680 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:10:41.681 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:10:41.682+0000 7f1c418ca640 0 -- 172.21.15.73:0/119487708 >> v2:172.21.15.195:6805/848686366 conn(0x7f1c3c227fb0 msgr2=0x7f1c3c22a3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-13T10:10:41.683 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T10:10:41.683 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:10:41.682+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:10:41.784 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:41.784+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:41.845 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:41.846 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:41.860 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-13T10:10:41.885 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:10:41.882+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:10:41.906 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:10:41.906 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:10:41.978 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:41.978 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:41.986 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:10:41.980+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:10:41.991 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-13T10:10:42.087 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:10:42.086+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:10:42.110 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:42.110 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:42.124 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-13T10:10:42.188 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:10:42.184+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:10:42.194 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-13T10:10:42.195 INFO:teuthology.orchestra.run.smithi195.stderr:pool 'low_tier' created 2024-09-13T10:10:42.223 INFO:teuthology.run_tasks:Running task rados... 2024-09-13T10:10:42.234 INFO:tasks.rados:Beginning rados... 2024-09-13T10:10:42.234 INFO:tasks.rados:config is {'clients': ['client.0'], 'dedup_chunk_algo': 'fastcdc', 'dedup_chunk_size': '131072', 'enable_dedup': True, 'low_tier_pool': 'low_tier', 'objects': 50, 'op_weights': {'read': 100, 'set_chunk': 30, 'tier_evict': 10, 'tier_flush': 5, 'tier_promote': 10, 'write': 50}, 'ops': 1500, 'set_chunk': True} 2024-09-13T10:10:42.234 INFO:tasks.rados:overrides is {'admin_socket': {'branch': 'wip-aclamk-cbt-combined'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_max_pg_log_entries': 2, 'osd_min_pg_log_entries': 1, 'osd_pg_log_trim_min': 0, 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on degraded': True, 'osd backoff on peering': True, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd max scrubs': 5, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd memory target': 1610612736, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5, 'min_in': 4}, 'workunit': {'branch': 'wip-aclamk-cbt-combined', 'sha1': 'd58f73069a8157a9ace12a0247ee3e8497707ea9'}} 2024-09-13T10:10:42.234 INFO:tasks.rados:config is {'clients': ['client.0'], 'dedup_chunk_algo': 'fastcdc', 'dedup_chunk_size': '131072', 'enable_dedup': True, 'low_tier_pool': 'low_tier', 'objects': 50, 'op_weights': {'read': 100, 'set_chunk': 30, 'tier_evict': 10, 'tier_flush': 5, 'tier_promote': 10, 'write': 50}, 'ops': 1500, 'set_chunk': True} 2024-09-13T10:10:42.234 DEBUG:teuthology.run_tasks:Unwinding manager rados 2024-09-13T10:10:42.243 INFO:tasks.rados:joining rados 2024-09-13T10:10:42.243 INFO:tasks.rados:clients are ['client.0'] 2024-09-13T10:10:42.243 INFO:tasks.rados:starting run 0 out of 1 2024-09-13T10:10:42.243 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-13T10:10:42.243 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2024-09-13T10:10:42.289 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:42.288+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:10:42.290 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:42.290 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:42.305 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-13T10:10:42.390 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:42.388+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:42.476 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:42.476 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:42.491 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-13T10:10:42.492 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:42.490+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:10:42.592 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:42.588+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:42.607 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:42.607 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:42.621 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-13T10:10:42.693 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:10:42.688+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:10:42.744 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:42.744 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:42.759 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T10:10:42.794 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:42.788+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:42.891 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:42.891 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:42.894 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:10:42.894+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:10:42.905 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T10:10:42.996 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:42.996+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:43.023 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:43.023 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:43.038 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T10:10:43.097 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:10:43.094+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:10:43.161 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:43.161 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:43.175 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T10:10:43.204 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'unique_pool_0' created 2024-09-13T10:10:43.218 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:10:43.218+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:10:43.236 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-13T10:10:43.299 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:10:43.298+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:10:43.340 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:43.340 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:43.354 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T10:10:43.399 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:43.398+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:10:43.501 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:10:43.500+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:10:43.523 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:43.523 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:43.538 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T10:10:43.602 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:43.600+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:10:43.655 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:43.656 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:43.670 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-13T10:10:43.703 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:43.696+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:10:43.793 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:43.794 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:43.804 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:43.800+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:43.807 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-13T10:10:43.905 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:10:43.902+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:10:43.924 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:10:43.925 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:43.925 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:43.939 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-13T10:10:44.006 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:44.006+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:10:44.079 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:10:44.080 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:10:44.099 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:44.099 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:44.100 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:10:44.100 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:10:44.107 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:10:44.106+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:10:44.114 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-13T10:10:44.121 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:10:44.121 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:10:44.143 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:10:44.144 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:10:44.163 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:10:44.163 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:10:44.180 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:10:44.181 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:10:44.203 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:10:44.203 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:10:44.206 INFO:teuthology.orchestra.run.smithi073.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-13T10:10:44.208 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:10:44.206+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:10:44.218 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T10:10:44.239 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:10:44.240 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:10:44.241 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:44.241 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:44.255 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-13T10:10:44.257 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:10:44.257 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:10:44.276 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:10:44.276 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:10:44.297 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:10:44.297 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:10:44.309 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:10:44.306+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:10:44.317 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:10:44.317 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:10:44.334 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:10:44.334 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:10:44.353 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:10:44.353 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:10:44.374 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:10:44.374 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:10:44.374 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:10:44.374 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:10:44.374 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:10:44.375 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:10:44.375 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:10:44.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:10:44.374+0000 7f863ed9d640 0 -- 172.21.15.73:0/3585566370 >> v2:172.21.15.195:6805/848686366 conn(0x7f8640222e50 msgr2=0x7f8640225240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-13T10:10:44.388 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-13T10:10:44.431 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:44.430+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:10:44.511 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:44.510+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:10:44.552 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:10:44.552 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:10:44.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:10:44.552 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:10:44.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.552 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4365.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:44.552 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:30.119900+0000", 2024-09-13T10:10:44.553 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.432432735000001, 2024-09-13T10:10:44.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020673093999999999, 2024-09-13T10:10:44.553 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:44.553 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:44.553 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:44.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:44.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4365", 2024-09-13T10:10:44.553 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.73:0/186187721", 2024-09-13T10:10:44.553 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-13T10:10:44.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.553 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:44.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:44.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119900+0000", 2024-09-13T10:10:44.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:44.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:44.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119900+0000", 2024-09-13T10:10:44.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:44.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:44.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119903+0000", 2024-09-13T10:10:44.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0929999999999999e-06 2024-09-13T10:10:44.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:44.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119919+0000", 2024-09-13T10:10:44.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6082999999999998e-05 2024-09-13T10:10:44.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:44.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119954+0000", 2024-09-13T10:10:44.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4944000000000003e-05 2024-09-13T10:10:44.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:44.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119983+0000", 2024-09-13T10:10:44.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8932e-05 2024-09-13T10:10:44.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:44.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.120157+0000", 2024-09-13T10:10:44.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017421 2024-09-13T10:10:44.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-13T10:10:44.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.120180+0000", 2024-09-13T10:10:44.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3039000000000001e-05 2024-09-13T10:10:44.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:44.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.121820+0000", 2024-09-13T10:10:44.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016397479999999999 2024-09-13T10:10:44.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:10:44.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.121842+0000", 2024-09-13T10:10:44.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2612e-05 2024-09-13T10:10:44.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:44.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.125236+0000", 2024-09-13T10:10:44.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003394181 2024-09-13T10:10:44.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:44.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.125597+0000", 2024-09-13T10:10:44.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036106199999999999 2024-09-13T10:10:44.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:10:44.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.125908+0000", 2024-09-13T10:10:44.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031034200000000002 2024-09-13T10:10:44.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:44.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.139886+0000", 2024-09-13T10:10:44.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013977824 2024-09-13T10:10:44.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:44.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140020+0000", 2024-09-13T10:10:44.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013476199999999999 2024-09-13T10:10:44.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:44.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140357+0000", 2024-09-13T10:10:44.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033711 2024-09-13T10:10:44.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:44.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140464+0000", 2024-09-13T10:10:44.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106028 2024-09-13T10:10:44.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:44.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140573+0000", 2024-09-13T10:10:44.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010912399999999999 2024-09-13T10:10:44.560 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:44.560 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:44.561 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:44.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.561 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop_reply(client.4365.0:1 1.3 e25/24 ondisk, result = 0)", 2024-09-13T10:10:44.561 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:30.139765+0000", 2024-09-13T10:10:44.561 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.412567175, 2024-09-13T10:10:44.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027974400000000001, 2024-09-13T10:10:44.561 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:44.561 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:44.561 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:10:44.561 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:44.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:44.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.139765+0000", 2024-09-13T10:10:44.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:44.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:44.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.139765+0000", 2024-09-13T10:10:44.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:44.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:44.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.139767+0000", 2024-09-13T10:10:44.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.443e-06 2024-09-13T10:10:44.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:44.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.139775+0000", 2024-09-13T10:10:44.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.507e-06 2024-09-13T10:10:44.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:44.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.139796+0000", 2024-09-13T10:10:44.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0805000000000002e-05 2024-09-13T10:10:44.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:44.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.139810+0000", 2024-09-13T10:10:44.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3956e-05 2024-09-13T10:10:44.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:44.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.139983+0000", 2024-09-13T10:10:44.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017291 2024-09-13T10:10:44.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:44.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140004+0000", 2024-09-13T10:10:44.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1662e-05 2024-09-13T10:10:44.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:44.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140045+0000", 2024-09-13T10:10:44.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0460999999999998e-05 2024-09-13T10:10:44.565 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:44.565 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:44.565 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:44.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.565 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop_reply(client.4365.0:1 1.3 e25/24 ondisk, result = 0)", 2024-09-13T10:10:44.565 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:30.140172+0000", 2024-09-13T10:10:44.565 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.412159985000001, 2024-09-13T10:10:44.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041405899999999999, 2024-09-13T10:10:44.565 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:44.565 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:44.565 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:10:44.566 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:44.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:44.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140172+0000", 2024-09-13T10:10:44.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:44.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:44.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140172+0000", 2024-09-13T10:10:44.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:44.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:44.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140174+0000", 2024-09-13T10:10:44.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8470000000000001e-06 2024-09-13T10:10:44.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:44.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140186+0000", 2024-09-13T10:10:44.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1824999999999999e-05 2024-09-13T10:10:44.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:44.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140217+0000", 2024-09-13T10:10:44.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0694999999999997e-05 2024-09-13T10:10:44.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:44.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140239+0000", 2024-09-13T10:10:44.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2765999999999999e-05 2024-09-13T10:10:44.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:44.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140339+0000", 2024-09-13T10:10:44.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9097000000000003e-05 2024-09-13T10:10:44.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:44.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140350+0000", 2024-09-13T10:10:44.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.111e-05 2024-09-13T10:10:44.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:44.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:44.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:44.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140586+0000", 2024-09-13T10:10:44.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023671900000000001 2024-09-13T10:10:44.569 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:44.569 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:44.569 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:44.569 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:44.569 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:44.569 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:10:44.571 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T10:10:44.599 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:10:44.599 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:10:44.613 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:44.614+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:10:44.703 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:10:44.704 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:10:44.704 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:10:44.704 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:10:44.714 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:10:44.714+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:10:44.718 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T10:10:44.816 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:10:44.814+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:10:44.840 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:10:44.841 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:10:44.841 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:10:44.841 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:10:44.841 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:10:44.854 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T10:10:44.917 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:44.912+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:44.977 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:10:44.978 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:10:44.978 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:10:44.978 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:10:44.978 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.978 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4365.0:2 1.2 1:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:44.978 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:30.145458+0000", 2024-09-13T10:10:44.978 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 14.831619328, 2024-09-13T10:10:44.978 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039448400000000001, 2024-09-13T10:10:44.978 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:10:44.978 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:10:44.978 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "reached pg", 2024-09-13T10:10:44.979 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:10:44.979 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4365", 2024-09-13T10:10:44.979 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.73:0/186187721", 2024-09-13T10:10:44.979 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2 2024-09-13T10:10:44.979 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.979 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:10:44.979 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.979 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:10:44.979 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:30.145458+0000", 2024-09-13T10:10:44.979 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:44.979 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.980 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.980 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:10:44.980 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:30.145458+0000", 2024-09-13T10:10:44.980 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:44.980 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.980 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.980 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:10:44.980 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:30.145461+0000", 2024-09-13T10:10:44.980 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9469999999999999e-06 2024-09-13T10:10:44.980 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.980 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.980 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:10:44.980 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:30.145473+0000", 2024-09-13T10:10:44.981 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.216e-05 2024-09-13T10:10:44.981 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.981 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.981 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:10:44.981 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:30.145495+0000", 2024-09-13T10:10:44.981 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2048999999999998e-05 2024-09-13T10:10:44.981 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.981 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.981 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:10:44.982 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:30.145511+0000", 2024-09-13T10:10:44.982 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6039e-05 2024-09-13T10:10:44.982 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.982 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.982 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:10:44.982 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:30.145646+0000", 2024-09-13T10:10:44.982 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013476100000000001 2024-09-13T10:10:44.982 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.982 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:10:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:30.145853+0000", 2024-09-13T10:10:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00020652800000000001 2024-09-13T10:10:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:10:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4365.0:3 1.2 1:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:30.146270+0000", 2024-09-13T10:10:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 14.830807754, 2024-09-13T10:10:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000460175, 2024-09-13T10:10:44.983 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:10:44.984 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:10:44.984 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "reached pg", 2024-09-13T10:10:44.984 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:10:44.984 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4365", 2024-09-13T10:10:44.984 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.73:0/186187721", 2024-09-13T10:10:44.984 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3 2024-09-13T10:10:44.984 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.984 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:10:44.984 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.984 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:10:44.984 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:30.146270+0000", 2024-09-13T10:10:44.984 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:44.984 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.985 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.985 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:10:44.985 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:30.146270+0000", 2024-09-13T10:10:44.985 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:44.985 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.985 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.985 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:10:44.985 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:30.146272+0000", 2024-09-13T10:10:44.985 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2400000000000002e-06 2024-09-13T10:10:44.985 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.985 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.985 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:10:44.985 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:30.146282+0000", 2024-09-13T10:10:44.986 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0366e-05 2024-09-13T10:10:44.986 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.986 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.986 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:10:44.986 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:30.146312+0000", 2024-09-13T10:10:44.986 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9859e-05 2024-09-13T10:10:44.986 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.986 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.986 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:10:44.986 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:30.146328+0000", 2024-09-13T10:10:44.986 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5512999999999999e-05 2024-09-13T10:10:44.986 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.986 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.987 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:10:44.987 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:30.146463+0000", 2024-09-13T10:10:44.987 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013549100000000001 2024-09-13T10:10:44.987 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.987 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.987 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:10:44.987 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:30.146730+0000", 2024-09-13T10:10:44.987 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026670600000000002 2024-09-13T10:10:44.988 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:44.988 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:10:44.988 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:44.988 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.988 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.988 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4365.0:5 1.2 1:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T10:10:44.988 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:31.135075+0000", 2024-09-13T10:10:44.988 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 13.842001936999999, 2024-09-13T10:10:44.988 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010617898000000001, 2024-09-13T10:10:44.988 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:10:44.988 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:10:44.988 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:44.988 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:10:44.989 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4365", 2024-09-13T10:10:44.989 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.73:0/186187721", 2024-09-13T10:10:44.989 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 5 2024-09-13T10:10:44.989 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.989 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:10:44.989 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.989 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:10:44.989 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135075+0000", 2024-09-13T10:10:44.989 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:44.989 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.989 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.989 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:10:44.989 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135075+0000", 2024-09-13T10:10:44.990 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:44.990 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.990 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.990 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:10:44.990 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135089+0000", 2024-09-13T10:10:44.990 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3512e-05 2024-09-13T10:10:44.990 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.990 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.990 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:10:44.990 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135113+0000", 2024-09-13T10:10:44.990 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4006e-05 2024-09-13T10:10:44.990 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.990 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.991 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:10:44.991 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135150+0000", 2024-09-13T10:10:44.991 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6838999999999997e-05 2024-09-13T10:10:44.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.991 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.991 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:10:44.991 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135169+0000", 2024-09-13T10:10:44.991 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9499000000000001e-05 2024-09-13T10:10:44.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.991 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.991 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:10:44.991 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135393+0000", 2024-09-13T10:10:44.991 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000224081 2024-09-13T10:10:44.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.992 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.992 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op must wait for map", 2024-09-13T10:10:44.992 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135412+0000", 2024-09-13T10:10:44.993 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8989000000000001e-05 2024-09-13T10:10:44.993 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.993 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.993 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:10:44.993 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.138668+0000", 2024-09-13T10:10:44.993 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0032558909999999999 2024-09-13T10:10:44.993 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.993 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.993 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:10:44.993 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.138896+0000", 2024-09-13T10:10:44.993 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022737699999999999 2024-09-13T10:10:44.993 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.994 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.994 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 6,9", 2024-09-13T10:10:44.994 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.139159+0000", 2024-09-13T10:10:44.994 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026282399999999999 2024-09-13T10:10:44.994 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.994 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.994 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:10:44.994 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.143646+0000", 2024-09-13T10:10:44.994 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044877249999999997 2024-09-13T10:10:44.994 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.994 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.994 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:44.995 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.144479+0000", 2024-09-13T10:10:44.995 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00083277000000000004 2024-09-13T10:10:44.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.995 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.995 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:44.995 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145487+0000", 2024-09-13T10:10:44.995 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001008246 2024-09-13T10:10:44.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.995 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.995 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:10:44.995 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145571+0000", 2024-09-13T10:10:44.995 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.4040000000000002e-05 2024-09-13T10:10:44.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.996 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.996 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:10:44.996 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145693+0000", 2024-09-13T10:10:44.996 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000122099 2024-09-13T10:10:44.996 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:44.996 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:10:44.996 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:44.996 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:44.996 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:44.996 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_repop_reply(client.4365.0:5 1.2 e26/24 ondisk, result = 0)", 2024-09-13T10:10:44.996 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:31.144292+0000", 2024-09-13T10:10:44.996 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 13.832785167000001, 2024-09-13T10:10:44.997 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019974599999999999, 2024-09-13T10:10:44.997 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:10:45.001 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:10:45.001 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:10:45.001 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:10:45.001 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.001 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:10:45.001 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.144292+0000", 2024-09-13T10:10:45.002 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:45.002 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.002 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.002 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:10:45.002 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.144292+0000", 2024-09-13T10:10:45.002 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:45.002 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.002 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.003 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:10:45.003 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.144294+0000", 2024-09-13T10:10:45.003 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6670000000000001e-06 2024-09-13T10:10:45.003 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.003 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.003 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:10:45.004 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.144309+0000", 2024-09-13T10:10:45.004 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5211999999999999e-05 2024-09-13T10:10:45.004 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.004 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.004 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:10:45.004 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.144334+0000", 2024-09-13T10:10:45.005 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4510000000000001e-05 2024-09-13T10:10:45.005 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.005 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.005 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:10:45.005 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.144347+0000", 2024-09-13T10:10:45.006 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3816e-05 2024-09-13T10:10:45.006 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.006 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.006 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:10:45.006 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.144454+0000", 2024-09-13T10:10:45.006 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010616900000000001 2024-09-13T10:10:45.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.007 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.007 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:10:45.007 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.144467+0000", 2024-09-13T10:10:45.007 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3526000000000001e-05 2024-09-13T10:10:45.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.008 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.008 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:10:45.008 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.144492+0000", 2024-09-13T10:10:45.008 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4845999999999999e-05 2024-09-13T10:10:45.008 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:45.008 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:10:45.009 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:45.009 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.009 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.009 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_repop_reply(client.4365.0:5 1.2 e25/24 ondisk, result = 0)", 2024-09-13T10:10:45.009 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:31.145337+0000", 2024-09-13T10:10:45.009 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 13.831740448, 2024-09-13T10:10:45.010 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00038430799999999998, 2024-09-13T10:10:45.010 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:10:45.010 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:10:45.010 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:10:45.010 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:10:45.011 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.011 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:10:45.011 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145337+0000", 2024-09-13T10:10:45.011 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:45.011 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.011 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.012 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:10:45.012 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145337+0000", 2024-09-13T10:10:45.012 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:45.012 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.012 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.013 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:10:45.013 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145339+0000", 2024-09-13T10:10:45.013 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7090000000000001e-06 2024-09-13T10:10:45.013 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.013 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.013 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:10:45.014 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145349+0000", 2024-09-13T10:10:45.014 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0484e-05 2024-09-13T10:10:45.014 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.014 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.014 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:10:45.014 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145376+0000", 2024-09-13T10:10:45.015 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6574999999999999e-05 2024-09-13T10:10:45.015 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.015 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.015 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:10:45.015 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145395+0000", 2024-09-13T10:10:45.016 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9496000000000001e-05 2024-09-13T10:10:45.016 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.016 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.016 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:10:45.016 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145471+0000", 2024-09-13T10:10:45.016 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5797000000000006e-05 2024-09-13T10:10:45.017 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.017 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.017 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:10:45.017 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145480+0000", 2024-09-13T10:10:45.017 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.2799999999999992e-06 2024-09-13T10:10:45.017 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.017 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.017 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:10:45.017 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145721+0000", 2024-09-13T10:10:45.017 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000240967 2024-09-13T10:10:45.017 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:45.017 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:10:45.018 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:45.018 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.018 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.018 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4365.0:7 1.2 1:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T10:10:45.018 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:32.139311+0000", 2024-09-13T10:10:45.018 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 12.837766781999999, 2024-09-13T10:10:45.018 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0083110230000000007, 2024-09-13T10:10:45.018 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:10:45.018 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:10:45.018 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:45.018 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:10:45.018 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4365", 2024-09-13T10:10:45.018 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.73:0/186187721", 2024-09-13T10:10:45.019 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 7 2024-09-13T10:10:45.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.019 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:10:45.019 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.019 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:10:45.019 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.139311+0000", 2024-09-13T10:10:45.019 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:45.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.019 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.019 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:10:45.019 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.139311+0000", 2024-09-13T10:10:45.019 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:45.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.020 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.020 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:10:45.020 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.139314+0000", 2024-09-13T10:10:45.020 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2100000000000002e-06 2024-09-13T10:10:45.020 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.020 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.020 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:10:45.020 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.139330+0000", 2024-09-13T10:10:45.020 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5732000000000001e-05 2024-09-13T10:10:45.020 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.020 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.020 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:10:45.020 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.139363+0000", 2024-09-13T10:10:45.021 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3506000000000001e-05 2024-09-13T10:10:45.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.021 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.021 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:10:45.021 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.139383+0000", 2024-09-13T10:10:45.021 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9661999999999999e-05 2024-09-13T10:10:45.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.021 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.021 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:10:45.021 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.139518+0000", 2024-09-13T10:10:45.021 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013519899999999999 2024-09-13T10:10:45.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.021 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.022 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op must wait for map", 2024-09-13T10:10:45.022 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.139528+0000", 2024-09-13T10:10:45.022 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0385e-05 2024-09-13T10:10:45.022 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.022 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.022 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:10:45.022 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.143156+0000", 2024-09-13T10:10:45.022 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0036272829999999998 2024-09-13T10:10:45.022 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.022 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.022 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:10:45.022 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.143311+0000", 2024-09-13T10:10:45.022 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000155554 2024-09-13T10:10:45.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.023 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 6,9", 2024-09-13T10:10:45.023 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.143540+0000", 2024-09-13T10:10:45.023 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000228908 2024-09-13T10:10:45.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.023 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:10:45.023 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.146400+0000", 2024-09-13T10:10:45.023 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0028597499999999999 2024-09-13T10:10:45.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.023 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:45.024 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147244+0000", 2024-09-13T10:10:45.024 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00084438899999999997 2024-09-13T10:10:45.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.024 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.024 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:45.024 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147428+0000", 2024-09-13T10:10:45.024 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000183623 2024-09-13T10:10:45.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.024 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.024 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:10:45.025 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147516+0000", 2024-09-13T10:10:45.025 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8005999999999997e-05 2024-09-13T10:10:45.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.025 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.025 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:10:45.025 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147622+0000", 2024-09-13T10:10:45.025 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000105816 2024-09-13T10:10:45.025 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:45.025 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:10:45.025 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:45.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.026 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.026 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_repop_reply(client.4365.0:7 1.2 e26/24 ondisk, result = 0)", 2024-09-13T10:10:45.026 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:32.147038+0000", 2024-09-13T10:10:45.026 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 12.830039786, 2024-09-13T10:10:45.026 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021988800000000001, 2024-09-13T10:10:45.026 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:10:45.026 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:10:45.026 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:10:45.026 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:10:45.027 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.027 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:10:45.027 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147038+0000", 2024-09-13T10:10:45.027 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:45.027 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.027 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.027 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:10:45.027 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147038+0000", 2024-09-13T10:10:45.027 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:45.027 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.027 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.028 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:10:45.028 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147039+0000", 2024-09-13T10:10:45.028 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.784e-06 2024-09-13T10:10:45.028 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.028 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.028 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:10:45.028 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147049+0000", 2024-09-13T10:10:45.028 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1200000000000008e-06 2024-09-13T10:10:45.028 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.028 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.028 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:10:45.028 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147076+0000", 2024-09-13T10:10:45.029 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7821999999999999e-05 2024-09-13T10:10:45.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.029 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.029 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:10:45.029 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147090+0000", 2024-09-13T10:10:45.029 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3631999999999999e-05 2024-09-13T10:10:45.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.029 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.029 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:10:45.029 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147220+0000", 2024-09-13T10:10:45.029 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012975899999999999 2024-09-13T10:10:45.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.030 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.030 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:10:45.030 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147234+0000", 2024-09-13T10:10:45.030 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4686e-05 2024-09-13T10:10:45.030 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.030 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.030 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:10:45.030 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147258+0000", 2024-09-13T10:10:45.030 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3085e-05 2024-09-13T10:10:45.030 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:45.031 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:10:45.031 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:45.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.031 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.031 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_repop_reply(client.4365.0:7 1.2 e26/24 ondisk, result = 0)", 2024-09-13T10:10:45.031 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:32.147238+0000", 2024-09-13T10:10:45.031 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 12.829839667, 2024-09-13T10:10:45.031 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000413127, 2024-09-13T10:10:45.031 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:10:45.031 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:10:45.031 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:10:45.031 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:10:45.031 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.032 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:10:45.032 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147238+0000", 2024-09-13T10:10:45.032 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:45.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.032 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.032 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:10:45.032 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147238+0000", 2024-09-13T10:10:45.032 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:45.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.032 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.032 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:10:45.032 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147240+0000", 2024-09-13T10:10:45.032 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8700000000000001e-06 2024-09-13T10:10:45.033 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.033 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.033 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:10:45.033 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147252+0000", 2024-09-13T10:10:45.033 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1877e-05 2024-09-13T10:10:45.033 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.033 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.033 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:10:45.033 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147278+0000", 2024-09-13T10:10:45.033 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6089e-05 2024-09-13T10:10:45.033 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.034 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.034 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:10:45.034 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147299+0000", 2024-09-13T10:10:45.034 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1679e-05 2024-09-13T10:10:45.034 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.034 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.034 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:10:45.034 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147409+0000", 2024-09-13T10:10:45.034 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110035 2024-09-13T10:10:45.034 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.034 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.035 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:10:45.035 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147420+0000", 2024-09-13T10:10:45.035 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0896e-05 2024-09-13T10:10:45.035 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.035 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.035 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:10:45.035 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.147651+0000", 2024-09-13T10:10:45.035 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000230681 2024-09-13T10:10:45.036 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:45.036 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:10:45.036 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:45.036 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:45.036 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:10:45.036 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:10:45.038 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T10:10:45.038 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:45.032+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:45.139 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:45.138+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:10:45.162 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:10:45.162 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:10:45.162 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:10:45.162 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:10:45.175 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T10:10:45.217 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-13T10:10:45.239 DEBUG:teuthology.orchestra.run.smithi195:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --set_chunk --enable_dedup --low_tier_pool low_tier --dedup_chunk_size 131072 --dedup_chunk_algo fastcdc --max-ops 1500 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 25 --op delete 10 --op set_chunk 30 --op tier_promote 10 --op tier_evict 10 --op tier_flush 5 --op write_excl 25 --pool unique_pool_0 2024-09-13T10:10:45.240 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:10:45.238+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:10:45.261 INFO:tasks.rados.rados.0.smithi195.stdout:adding op weight read -> 100 2024-09-13T10:10:45.262 INFO:tasks.rados.rados.0.smithi195.stdout:adding op weight write -> 25 2024-09-13T10:10:45.262 INFO:tasks.rados.rados.0.smithi195.stdout:adding op weight delete -> 10 2024-09-13T10:10:45.262 INFO:tasks.rados.rados.0.smithi195.stdout:adding op weight set_chunk -> 30 2024-09-13T10:10:45.262 INFO:tasks.rados.rados.0.smithi195.stdout:adding op weight tier_promote -> 10 2024-09-13T10:10:45.262 INFO:tasks.rados.rados.0.smithi195.stdout:adding op weight tier_evict -> 10 2024-09-13T10:10:45.262 INFO:tasks.rados.rados.0.smithi195.stdout:adding op weight tier_flush -> 5 2024-09-13T10:10:45.262 INFO:tasks.rados.rados.0.smithi195.stdout:adding op weight write_excl -> 25 2024-09-13T10:10:45.262 INFO:tasks.rados.rados.0.smithi195.stdout:ceph version 19.3.0-4697-gd58f7306 (d58f73069a8157a9ace12a0247ee3e8497707ea9) squid (dev) 2024-09-13T10:10:45.262 INFO:tasks.rados.rados.0.smithi195.stdout:Configuration: 2024-09-13T10:10:45.262 INFO:tasks.rados.rados.0.smithi195.stdout: Number of operations: 1500 2024-09-13T10:10:45.262 INFO:tasks.rados.rados.0.smithi195.stdout: Number of objects: 50 2024-09-13T10:10:45.262 INFO:tasks.rados.rados.0.smithi195.stdout: Max in flight operations: 16 2024-09-13T10:10:45.263 INFO:tasks.rados.rados.0.smithi195.stdout: Object size (in bytes): 4000000 2024-09-13T10:10:45.263 INFO:tasks.rados.rados.0.smithi195.stdout: Write stride min: 400000 2024-09-13T10:10:45.263 INFO:tasks.rados.rados.0.smithi195.stdout: Write stride max: 800000 2024-09-13T10:10:45.293 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:10:45.293 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:10:45.293 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:10:45.293 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:10:45.293 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:10:45.307 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T10:10:45.341 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:10:45.338+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:10:45.431 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:10:45.431 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:10:45.431 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:10:45.431 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:10:45.431 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.431 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_repop(client.4365.0:5 1.2 e26/24 1:5cea7035:::rbd_info:head v 26'1, mlcod=0'0)", 2024-09-13T10:10:45.431 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:31.142541+0000", 2024-09-13T10:10:45.431 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 14.288653219, 2024-09-13T10:10:45.431 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001666004, 2024-09-13T10:10:45.431 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:10:45.432 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:10:45.432 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:45.432 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:10:45.432 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.432 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:10:45.432 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.142541+0000", 2024-09-13T10:10:45.432 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:45.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.432 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.432 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:10:45.432 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.142541+0000", 2024-09-13T10:10:45.432 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:45.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.433 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.433 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:10:45.433 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.142542+0000", 2024-09-13T10:10:45.433 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.0699999999999997e-07 2024-09-13T10:10:45.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.433 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.433 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:10:45.433 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.142560+0000", 2024-09-13T10:10:45.433 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8096000000000001e-05 2024-09-13T10:10:45.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.433 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.433 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:10:45.433 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.142600+0000", 2024-09-13T10:10:45.434 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0677999999999999e-05 2024-09-13T10:10:45.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.434 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.434 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:10:45.434 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.142629+0000", 2024-09-13T10:10:45.434 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8858999999999999e-05 2024-09-13T10:10:45.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.434 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.434 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:10:45.434 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.142775+0000", 2024-09-13T10:10:45.434 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014547099999999999 2024-09-13T10:10:45.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.434 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.434 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:10:45.434 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.142819+0000", 2024-09-13T10:10:45.435 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4490999999999998e-05 2024-09-13T10:10:45.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.435 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.435 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:10:45.435 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.144139+0000", 2024-09-13T10:10:45.435 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001320381 2024-09-13T10:10:45.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.435 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.435 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:10:45.435 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.144207+0000", 2024-09-13T10:10:45.435 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.7421000000000004e-05 2024-09-13T10:10:45.435 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:45.435 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:10:45.436 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:45.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.436 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.436 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_repop(client.4365.0:7 1.2 e27/24 1:5cea7035:::rbd_info:head v 27'2, mlcod=0'0)", 2024-09-13T10:10:45.436 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:32.144875+0000", 2024-09-13T10:10:45.436 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 13.286319141, 2024-09-13T10:10:45.436 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020636780000000002, 2024-09-13T10:10:45.436 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:10:45.436 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:10:45.436 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:45.436 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:10:45.436 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.436 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:10:45.437 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.144875+0000", 2024-09-13T10:10:45.437 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:45.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.437 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.437 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:10:45.437 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.144875+0000", 2024-09-13T10:10:45.437 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:10:45.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.437 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.437 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:10:45.437 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.144876+0000", 2024-09-13T10:10:45.437 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.2999999999999999e-07 2024-09-13T10:10:45.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.438 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.438 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:10:45.438 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.144902+0000", 2024-09-13T10:10:45.438 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5874999999999999e-05 2024-09-13T10:10:45.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.438 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.438 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:10:45.438 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.144932+0000", 2024-09-13T10:10:45.438 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9893e-05 2024-09-13T10:10:45.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.438 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.438 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:10:45.438 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.144954+0000", 2024-09-13T10:10:45.438 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2739e-05 2024-09-13T10:10:45.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.439 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.439 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:10:45.439 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.145117+0000", 2024-09-13T10:10:45.439 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016301999999999999 2024-09-13T10:10:45.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.439 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.439 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:10:45.439 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.145178+0000", 2024-09-13T10:10:45.439 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.0300999999999997e-05 2024-09-13T10:10:45.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.439 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.439 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:10:45.439 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.146827+0000", 2024-09-13T10:10:45.439 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0016495640000000001 2024-09-13T10:10:45.440 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:10:45.440 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:10:45.440 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:10:45.440 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:32.146939+0000", 2024-09-13T10:10:45.440 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000111456 2024-09-13T10:10:45.440 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:45.440 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:10:45.440 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:45.440 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:10:45.440 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:10:45.440 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:10:45.442 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:10:45.442+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:10:45.444 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-13T10:10:45.543 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:10:45.536+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:10:45.566 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:10:45.566 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:10:45.566 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:10:45.566 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:10:45.580 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-13T10:10:45.644 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:45.640+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:45.745 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:10:45.744+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:10:45.746 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:10:45.746 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:10:45.746 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:10:45.746 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:10:45.746 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:10:45.760 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-13T10:10:45.847 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:10:45.840+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:10:45.878 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:10:45.879 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:10:45.879 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:10:45.879 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [] 2024-09-13T10:10:45.879 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:10:45.893 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-13T10:10:45.948 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:45.946+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:10:45.977 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:10:45.977 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:10:45.977 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:10:45.978 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_0 2024-09-13T10:10:45.978 INFO:tasks.ceph.ceph_manager.ceph:contract_pool unique_pool_0 by 10 min 16 2024-09-13T10:10:45.978 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:10:46.049 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:10:46.050+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:10:46.062 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:10:46.062 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:10:46.062 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:10:46.062 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:10:46.076 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-13T10:10:46.150 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:46.144+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:10:46.200 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:10:46.201 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:10:46.201 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:10:46.201 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:10:46.201 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:10:46.214 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-13T10:10:46.251 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:46.248+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:46.295 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:10:46.296 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:10:46.315 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-13T10:10:46.222668+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":147,"num_write_kb":2674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":69,"acting":69,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":30,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":12,"kb":1509949440,"kb_used":442864,"kb_used_data":6296,"kb_used_omap":24,"kb_used_meta":436263,"kb_avail":1509506576,"statfs":{"total":1546188226560,"available":1545734733824,"internally_reserved":0,"allocated":6447104,"data_stored":3660027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":446733391},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.026924"},"pg_stats":[{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T10:10:45.543648+0000","last_change":"2024-09-13T10:10:44.224347+0000","last_active":"2024-09-13T10:10:45.543648+0000","last_peered":"2024-09-13T10:10:45.543648+0000","last_clean":"2024-09-13T10:10:45.543648+0000","last_became_active":"2024-09-13T10:10:44.223700+0000","last_became_peered":"2024-09-13T10:10:44.223700+0000","last_unstale":"2024-09-13T10:10:45.543648+0000","last_undegraded":"2024-09-13T10:10:45.543648+0000","last_fullsized":"2024-09-13T10:10:45.543648+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:50.844782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,7],"acting":[13,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T10:10:43.197988+0000","last_change":"2024-09-13T10:10:43.197988+0000","last_active":"2024-09-13T10:10:43.197988+0000","last_peered":"2024-09-13T10:10:43.197988+0000","last_clean":"2024-09-13T10:10:43.197988+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:43.197988+0000","last_undegraded":"2024-09-13T10:10:43.197988+0000","last_fullsized":"2024-09-13T10:10:43.197988+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-13T10:10:43.238471+0000","last_change":"2024-09-13T10:10:43.238471+0000","last_active":"2024-09-13T10:10:43.197988+0000","last_peered":"2024-09-13T10:10:43.197988+0000","last_clean":"2024-09-13T10:10:43.197988+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:43.238471+0000","last_undegraded":"2024-09-13T10:10:43.238471+0000","last_fullsized":"2024-09-13T10:10:43.238471+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,15,7],"acting":[12,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-13T10:10:43.238419+0000","last_change":"2024-09-13T10:10:43.238419+0000","last_active":"2024-09-13T10:10:43.197988+0000","last_peered":"2024-09-13T10:10:43.197988+0000","last_clean":"2024-09-13T10:10:43.197988+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:43.238419+0000","last_undegraded":"2024-09-13T10:10:43.238419+0000","last_fullsized":"2024-09-13T10:10:43.238419+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,6,3],"acting":[12,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-13T10:10:43.213516+0000","last_change":"2024-09-13T10:10:43.213516+0000","last_active":"2024-09-13T10:10:43.197988+0000","last_peered":"2024-09-13T10:10:43.197988+0000","last_clean":"2024-09-13T10:10:43.197988+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:43.213516+0000","last_undegraded":"2024-09-13T10:10:43.213516+0000","last_fullsized":"2024-09-13T10:10:43.213516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,4],"acting":[14,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-13T10:10:43.211762+0000","last_change":"2024-09-13T10:10:43.211762+0000","last_active":"2024-09-13T10:10:43.197988+0000","last_peered":"2024-09-13T10:10:43.197988+0000","last_clean":"2024-09-13T10:10:43.197988+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:43.211762+0000","last_undegraded":"2024-09-13T10:10:43.211762+0000","last_fullsized":"2024-09-13T10:10:43.211762+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,13],"acting":[8,6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T10:10:43.197988+0000","last_change":"2024-09-13T10:10:43.197988+0000","last_active":"2024-09-13T10:10:43.197988+0000","last_peered":"2024-09-13T10:10:43.197988+0000","last_clean":"2024-09-13T10:10:43.197988+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:43.197988+0000","last_undegraded":"2024-09-13T10:10:43.197988+0000","last_fullsized":"2024-09-13T10:10:43.197988+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T10:10:43.197988+0000","last_change":"2024-09-13T10:10:43.197988+0000","last_active":"2024-09-13T10:10:43.197988+0000","last_peered":"2024-09-13T10:10:43.197988+0000","last_clean":"2024-09-13T10:10:43.197988+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:43.197988+0000","last_undegraded":"2024-09-13T10:10:43.197988+0000","last_fullsized":"2024-09-13T10:10:43.197988+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-13T10:10:43.219912+0000","last_change":"2024-09-13T10:10:43.219912+0000","last_active":"2024-09-13T10:10:43.197988+0000","last_peered":"2024-09-13T10:10:43.197988+0000","last_clean":"2024-09-13T10:10:43.197988+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:43.219912+0000","last_undegraded":"2024-09-13T10:10:43.219912+0000","last_fullsized":"2024-09-13T10:10:43.219912+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,0],"acting":[6,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T10:10:43.220740+0000","last_change":"2024-09-13T10:10:32.149042+0000","last_active":"2024-09-13T10:10:43.220740+0000","last_peered":"2024-09-13T10:10:43.220740+0000","last_clean":"2024-09-13T10:10:43.220740+0000","last_became_active":"2024-09-13T10:10:30.272007+0000","last_became_peered":"2024-09-13T10:10:30.272007+0000","last_unstale":"2024-09-13T10:10:43.220740+0000","last_undegraded":"2024-09-13T10:10:43.220740+0000","last_fullsized":"2024-09-13T10:10:43.220740+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:49.552306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000987604,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T10:10:43.232167+0000","last_change":"2024-09-13T10:10:43.230767+0000","last_active":"2024-09-13T10:10:43.232167+0000","last_peered":"2024-09-13T10:10:43.232167+0000","last_clean":"2024-09-13T10:10:43.232167+0000","last_became_active":"2024-09-13T10:10:43.229961+0000","last_became_peered":"2024-09-13T10:10:43.229961+0000","last_unstale":"2024-09-13T10:10:43.232167+0000","last_undegraded":"2024-09-13T10:10:43.232167+0000","last_fullsized":"2024-09-13T10:10:43.232167+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:00.827218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-13T10:10:43.221300+0000","last_change":"2024-09-13T10:10:43.221300+0000","last_active":"2024-09-13T10:10:43.197988+0000","last_peered":"2024-09-13T10:10:43.197988+0000","last_clean":"2024-09-13T10:10:43.197988+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:43.221300+0000","last_undegraded":"2024-09-13T10:10:43.221300+0000","last_fullsized":"2024-09-13T10:10:43.221300+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10],"acting":[6,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T10:10:42.207958+0000","last_change":"2024-09-13T10:10:33.223611+0000","last_active":"2024-09-13T10:10:42.207958+0000","last_peered":"2024-09-13T10:10:42.207958+0000","last_clean":"2024-09-13T10:10:42.207958+0000","last_became_active":"2024-09-13T10:10:30.128310+0000","last_became_peered":"2024-09-13T10:10:30.128310+0000","last_unstale":"2024-09-13T10:10:42.207958+0000","last_undegraded":"2024-09-13T10:10:42.207958+0000","last_fullsized":"2024-09-13T10:10:42.207958+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:46.008600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081844899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T10:10:42.623650+0000","last_change":"2024-09-13T10:10:42.623650+0000","last_active":"2024-09-13T10:10:42.187897+0000","last_peered":"2024-09-13T10:10:42.187897+0000","last_clean":"2024-09-13T10:10:42.187897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:42.623650+0000","last_undegraded":"2024-09-13T10:10:42.623650+0000","last_fullsized":"2024-09-13T10:10:42.623650+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T10:10:43.197988+0000","last_change":"2024-09-13T10:10:43.197988+0000","last_active":"2024-09-13T10:10:43.197988+0000","last_peered":"2024-09-13T10:10:43.197988+0000","last_clean":"2024-09-13T10:10:43.197988+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:43.197988+0000","last_undegraded":"2024-09-13T10:10:43.197988+0000","last_fullsized":"2024-09-13T10:10:43.197988+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":31,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T10:10:43.229719+0000","last_change":"2024-09-13T10:10:32.146042+0000","last_active":"2024-09-13T10:10:43.229719+0000","last_peered":"2024-09-13T10:10:43.229719+0000","last_clean":"2024-09-13T10:10:43.229719+0000","last_became_active":"2024-09-13T10:10:30.127232+0000","last_became_peered":"2024-09-13T10:10:30.127232+0000","last_unstale":"2024-09-13T10:10:43.229719+0000","last_undegraded":"2024-09-13T10:10:43.229719+0000","last_fullsized":"2024-09-13T10:10:43.229719+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:29.955802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034622780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":30,"state":"creating+peering","last_fresh":"2024-09-13T10:10:42.205657+0000","last_change":"2024-09-13T10:10:42.205657+0000","last_active":"2024-09-13T10:10:42.187897+0000","last_peered":"2024-09-13T10:10:42.187897+0000","last_clean":"2024-09-13T10:10:42.187897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:42.205657+0000","last_undegraded":"2024-09-13T10:10:42.205657+0000","last_fullsized":"2024-09-13T10:10:42.205657+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T10:10:43.197988+0000","last_change":"2024-09-13T10:10:43.197988+0000","last_active":"2024-09-13T10:10:43.197988+0000","last_peered":"2024-09-13T10:10:43.197988+0000","last_clean":"2024-09-13T10:10:43.197988+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:43.197988+0000","last_undegraded":"2024-09-13T10:10:43.197988+0000","last_fullsized":"2024-09-13T10:10:43.197988+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":34,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T10:10:45.543130+0000","last_change":"2024-09-13T10:10:32.702214+0000","last_active":"2024-09-13T10:10:45.543130+0000","last_peered":"2024-09-13T10:10:45.543130+0000","last_clean":"2024-09-13T10:10:45.543130+0000","last_became_active":"2024-09-13T10:10:30.124759+0000","last_became_peered":"2024-09-13T10:10:30.124759+0000","last_unstale":"2024-09-13T10:10:45.543130+0000","last_undegraded":"2024-09-13T10:10:45.543130+0000","last_fullsized":"2024-09-13T10:10:45.543130+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:45.386731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090474700000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":160,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T10:10:43.208734+0000","last_change":"2024-09-13T10:10:30.127365+0000","last_active":"2024-09-13T10:10:43.208734+0000","last_peered":"2024-09-13T10:10:43.208734+0000","last_clean":"2024-09-13T10:10:43.208734+0000","last_became_active":"2024-09-13T10:10:30.126496+0000","last_became_peered":"2024-09-13T10:10:30.126496+0000","last_unstale":"2024-09-13T10:10:43.208734+0000","last_undegraded":"2024-09-13T10:10:43.208734+0000","last_fullsized":"2024-09-13T10:10:43.208734+0000","mapping_epoch":24,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:53.401784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-13T10:10:45.543181+0000","last_change":"2024-09-13T10:10:43.231291+0000","last_active":"2024-09-13T10:10:45.543181+0000","last_peered":"2024-09-13T10:10:45.543181+0000","last_clean":"2024-09-13T10:10:45.543181+0000","last_became_active":"2024-09-13T10:10:43.230866+0000","last_became_peered":"2024-09-13T10:10:43.230866+0000","last_unstale":"2024-09-13T10:10:45.543181+0000","last_undegraded":"2024-09-13T10:10:45.543181+0000","last_fullsized":"2024-09-13T10:10:45.543181+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:54.564637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,4,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-13T10:10:43.235609+0000","last_change":"2024-09-13T10:10:43.235609+0000","last_active":"2024-09-13T10:10:43.197988+0000","last_peered":"2024-09-13T10:10:43.197988+0000","last_clean":"2024-09-13T10:10:43.197988+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:43.235609+0000","last_undegraded":"2024-09-13T10:10:43.235609+0000","last_fullsized":"2024-09-13T10:10:43.235609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,12],"acting":[2,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T10:10:43.218236+0000","last_change":"2024-09-13T10:10:32.701581+0000","last_active":"2024-09-13T10:10:43.218236+0000","last_peered":"2024-09-13T10:10:43.218236+0000","last_clean":"2024-09-13T10:10:43.218236+0000","last_became_active":"2024-09-13T10:10:30.126588+0000","last_became_peered":"2024-09-13T10:10:30.126588+0000","last_unstale":"2024-09-13T10:10:43.218236+0000","last_undegraded":"2024-09-13T10:10:43.218236+0000","last_fullsized":"2024-09-13T10:10:43.218236+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:40.095103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089379399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T10:10:43.197988+0000","last_change":"2024-09-13T10:10:43.197988+0000","last_active":"2024-09-13T10:10:43.197988+0000","last_peered":"2024-09-13T10:10:43.197988+0000","last_clean":"2024-09-13T10:10:43.197988+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:43.197988+0000","last_undegraded":"2024-09-13T10:10:43.197988+0000","last_fullsized":"2024-09-13T10:10:43.197988+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T10:10:42.546336+0000","last_change":"2024-09-13T10:10:32.177488+0000","last_active":"2024-09-13T10:10:42.546336+0000","last_peered":"2024-09-13T10:10:42.546336+0000","last_clean":"2024-09-13T10:10:42.546336+0000","last_became_active":"2024-09-13T10:10:30.128658+0000","last_became_peered":"2024-09-13T10:10:30.128658+0000","last_unstale":"2024-09-13T10:10:42.546336+0000","last_undegraded":"2024-09-13T10:10:42.546336+0000","last_fullsized":"2024-09-13T10:10:42.546336+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:50.278429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-13T10:10:43.230337+0000","last_change":"2024-09-13T10:10:43.230337+0000","last_active":"2024-09-13T10:10:43.197988+0000","last_peered":"2024-09-13T10:10:43.197988+0000","last_clean":"2024-09-13T10:10:43.197988+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:43.230337+0000","last_undegraded":"2024-09-13T10:10:43.230337+0000","last_fullsized":"2024-09-13T10:10:43.230337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,8],"acting":[2,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":27,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T10:10:42.206365+0000","last_change":"2024-09-13T10:10:32.932079+0000","last_active":"2024-09-13T10:10:42.206365+0000","last_peered":"2024-09-13T10:10:42.206365+0000","last_clean":"2024-09-13T10:10:42.206365+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:10:42.206365+0000","last_undegraded":"2024-09-13T10:10:42.206365+0000","last_fullsized":"2024-09-13T10:10:42.206365+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:57.533673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-13T10:10:43.218991+0000","last_change":"2024-09-13T10:10:43.218991+0000","last_active":"2024-09-13T10:10:43.197988+0000","last_peered":"2024-09-13T10:10:43.197988+0000","last_clean":"2024-09-13T10:10:43.197988+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T10:10:43.218991+0000","last_undegraded":"2024-09-13T10:10:43.218991+0000","last_fullsized":"2024-09-13T10:10:43.218991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,14],"acting":[6,8,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-13T10:10:43.208754+0000","last_change":"2024-09-13T10:10:32.135708+0000","last_active":"2024-09-13T10:10:43.208754+0000","last_peered":"2024-09-13T10:10:43.208754+0000","last_clean":"2024-09-13T10:10:43.208754+0000","last_became_active":"2024-09-13T10:10:30.125785+0000","last_became_peered":"2024-09-13T10:10:30.125785+0000","last_unstale":"2024-09-13T10:10:43.208754+0000","last_undegraded":"2024-09-13T10:10:43.208754+0000","last_fullsized":"2024-09-13T10:10:43.208754+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:35.380273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010905629999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":30,"acting":30,"num_store_stats":6},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":6},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":11}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":117885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247816,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27320,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344520,"statfs":{"total":96636764160,"available":96608788480,"internally_reserved":0,"allocated":303104,"data_stored":126204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247816,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":884736,"data_stored":708253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313223,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":109974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313223,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":315392,"data_stored":134895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,8,9,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247814,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":303104,"data_stored":126204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":109955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280518,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":884736,"data_stored":708253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2,4,6,7,8,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280519,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":109955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":290816,"data_stored":117885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[3,4,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280518,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":884736,"data_stored":708253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":109955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,4,5,7,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31456,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":278528,"data_stored":109955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,5,7,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280518,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":109955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247814,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":307200,"data_stored":126223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247813,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27332,"kb_used_data":300,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344508,"statfs":{"total":96636764160,"available":96608776192,"internally_reserved":0,"allocated":307200,"data_stored":126223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,3,5,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:10:46.317 INFO:tasks.ceph.ceph_manager.ceph:too many creating 2024-09-13T10:10:46.352 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:46.348+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:10:46.381 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:10:46.381 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:10:46.381 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:10:46.381 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [] 2024-09-13T10:10:46.381 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:10:46.395 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T10:10:46.453 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:10:46.448+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:10:46.525 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:10:46.526 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:10:46.526 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:10:46.526 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:10:46.540 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T10:10:46.583 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:10:46.580+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:10:46.622 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:10:46.655 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:46.652+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:10:46.658 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:10:46.659 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:10:46.659 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:10:46.659 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:10:46.659 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:10:46.672 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T10:10:46.755 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:10:46.754+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:10:46.778 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:10:46.779 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:10:46.800 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:10:46.800 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:10:46.822 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:10:46.822 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:10:46.840 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:10:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:10:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:10:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:10:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:40 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'22, mlcod=25'19)", 2024-09-13T10:10:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.284257+0000", 2024-09-13T10:10:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.554269358999999, 2024-09-13T10:10:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032447280000000001, 2024-09-13T10:10:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.840 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.284257+0000", 2024-09-13T10:10:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.284257+0000", 2024-09-13T10:10:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.284260+0000", 2024-09-13T10:10:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3530000000000002e-06 2024-09-13T10:10:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.284274+0000", 2024-09-13T10:10:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4293e-05 2024-09-13T10:10:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.284301+0000", 2024-09-13T10:10:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7528000000000001e-05 2024-09-13T10:10:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.284320+0000", 2024-09-13T10:10:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8552000000000001e-05 2024-09-13T10:10:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.284448+0000", 2024-09-13T10:10:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012842600000000001 2024-09-13T10:10:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.284496+0000", 2024-09-13T10:10:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7580000000000002e-05 2024-09-13T10:10:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287410+0000", 2024-09-13T10:10:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029135039999999999 2024-09-13T10:10:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287502+0000", 2024-09-13T10:10:46.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2491999999999995e-05 2024-09-13T10:10:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.845 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:41 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'23, mlcod=25'19)", 2024-09-13T10:10:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.284835+0000", 2024-09-13T10:10:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.55369134, 2024-09-13T10:10:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004654498, 2024-09-13T10:10:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.284835+0000", 2024-09-13T10:10:46.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.284835+0000", 2024-09-13T10:10:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.284836+0000", 2024-09-13T10:10:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0589999999999999e-06 2024-09-13T10:10:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.284911+0000", 2024-09-13T10:10:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4484e-05 2024-09-13T10:10:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.284925+0000", 2024-09-13T10:10:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3822000000000001e-05 2024-09-13T10:10:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.284934+0000", 2024-09-13T10:10:46.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.594e-06 2024-09-13T10:10:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285451+0000", 2024-09-13T10:10:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00051642799999999998 2024-09-13T10:10:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285495+0000", 2024-09-13T10:10:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3900999999999999e-05 2024-09-13T10:10:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289359+0000", 2024-09-13T10:10:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0038643969999999999 2024-09-13T10:10:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289490+0000", 2024-09-13T10:10:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000130813 2024-09-13T10:10:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:42 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'24, mlcod=25'19)", 2024-09-13T10:10:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.285265+0000", 2024-09-13T10:10:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.553262069999999, 2024-09-13T10:10:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0079129720000000008, 2024-09-13T10:10:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285265+0000", 2024-09-13T10:10:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285265+0000", 2024-09-13T10:10:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285266+0000", 2024-09-13T10:10:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.015e-06 2024-09-13T10:10:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285275+0000", 2024-09-13T10:10:46.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0229999999999998e-06 2024-09-13T10:10:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285286+0000", 2024-09-13T10:10:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1314999999999999e-05 2024-09-13T10:10:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285296+0000", 2024-09-13T10:10:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0305e-05 2024-09-13T10:10:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286603+0000", 2024-09-13T10:10:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001306967 2024-09-13T10:10:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286654+0000", 2024-09-13T10:10:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1128000000000002e-05 2024-09-13T10:10:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.293091+0000", 2024-09-13T10:10:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0064362320000000001 2024-09-13T10:10:46.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.293178+0000", 2024-09-13T10:10:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6986999999999999e-05 2024-09-13T10:10:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:43 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'25, mlcod=25'19)", 2024-09-13T10:10:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.285599+0000", 2024-09-13T10:10:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.552927136000001, 2024-09-13T10:10:46.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0076391799999999998, 2024-09-13T10:10:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285599+0000", 2024-09-13T10:10:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285599+0000", 2024-09-13T10:10:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285600+0000", 2024-09-13T10:10:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6299999999999993e-07 2024-09-13T10:10:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285608+0000", 2024-09-13T10:10:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2219999999999996e-06 2024-09-13T10:10:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285621+0000", 2024-09-13T10:10:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3548e-05 2024-09-13T10:10:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285631+0000", 2024-09-13T10:10:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0234000000000001e-05 2024-09-13T10:10:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287590+0000", 2024-09-13T10:10:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019584469999999999 2024-09-13T10:10:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287648+0000", 2024-09-13T10:10:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8393999999999997e-05 2024-09-13T10:10:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.293182+0000", 2024-09-13T10:10:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0055336120000000003 2024-09-13T10:10:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.293239+0000", 2024-09-13T10:10:46.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6759999999999999e-05 2024-09-13T10:10:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:44 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'26, mlcod=25'19)", 2024-09-13T10:10:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.286062+0000", 2024-09-13T10:10:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.552464688000001, 2024-09-13T10:10:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0080326949999999994, 2024-09-13T10:10:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286062+0000", 2024-09-13T10:10:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286062+0000", 2024-09-13T10:10:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286063+0000", 2024-09-13T10:10:46.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.409e-06 2024-09-13T10:10:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286168+0000", 2024-09-13T10:10:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104666 2024-09-13T10:10:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286188+0000", 2024-09-13T10:10:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9953e-05 2024-09-13T10:10:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286203+0000", 2024-09-13T10:10:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5038999999999999e-05 2024-09-13T10:10:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288563+0000", 2024-09-13T10:10:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0023604260000000001 2024-09-13T10:10:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288604+0000", 2024-09-13T10:10:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0308000000000001e-05 2024-09-13T10:10:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.294003+0000", 2024-09-13T10:10:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0053990180000000002 2024-09-13T10:10:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.294095+0000", 2024-09-13T10:10:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1875999999999999e-05 2024-09-13T10:10:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:45 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'27, mlcod=25'19)", 2024-09-13T10:10:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.286765+0000", 2024-09-13T10:10:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.551761197000001, 2024-09-13T10:10:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0073987970000000004, 2024-09-13T10:10:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286765+0000", 2024-09-13T10:10:46.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286765+0000", 2024-09-13T10:10:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286768+0000", 2024-09-13T10:10:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1629999999999999e-06 2024-09-13T10:10:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286782+0000", 2024-09-13T10:10:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4496e-05 2024-09-13T10:10:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286808+0000", 2024-09-13T10:10:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6211999999999999e-05 2024-09-13T10:10:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286827+0000", 2024-09-13T10:10:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8352999999999999e-05 2024-09-13T10:10:46.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289596+0000", 2024-09-13T10:10:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027694960000000002 2024-09-13T10:10:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289675+0000", 2024-09-13T10:10:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8635999999999997e-05 2024-09-13T10:10:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.294099+0000", 2024-09-13T10:10:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0044240420000000004 2024-09-13T10:10:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.294164+0000", 2024-09-13T10:10:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5399000000000002e-05 2024-09-13T10:10:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:46 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'28, mlcod=25'19)", 2024-09-13T10:10:46.871 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.286967+0000", 2024-09-13T10:10:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.551559406999999, 2024-09-13T10:10:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0072572360000000002, 2024-09-13T10:10:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286967+0000", 2024-09-13T10:10:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286967+0000", 2024-09-13T10:10:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286969+0000", 2024-09-13T10:10:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6500000000000001e-06 2024-09-13T10:10:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286981+0000", 2024-09-13T10:10:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1846e-05 2024-09-13T10:10:46.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287002+0000", 2024-09-13T10:10:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1424999999999999e-05 2024-09-13T10:10:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287018+0000", 2024-09-13T10:10:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5532999999999998e-05 2024-09-13T10:10:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.290568+0000", 2024-09-13T10:10:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035500940000000002 2024-09-13T10:10:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.290626+0000", 2024-09-13T10:10:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8127000000000001e-05 2024-09-13T10:10:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.294167+0000", 2024-09-13T10:10:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035410979999999999 2024-09-13T10:10:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.294224+0000", 2024-09-13T10:10:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7463000000000002e-05 2024-09-13T10:10:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:47 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'29, mlcod=25'20)", 2024-09-13T10:10:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.287512+0000", 2024-09-13T10:10:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.551015048, 2024-09-13T10:10:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0067743509999999996, 2024-09-13T10:10:46.876 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287512+0000", 2024-09-13T10:10:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287512+0000", 2024-09-13T10:10:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287513+0000", 2024-09-13T10:10:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8300000000000001e-06 2024-09-13T10:10:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287676+0000", 2024-09-13T10:10:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016265799999999999 2024-09-13T10:10:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287697+0000", 2024-09-13T10:10:46.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1336e-05 2024-09-13T10:10:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287710+0000", 2024-09-13T10:10:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2232000000000001e-05 2024-09-13T10:10:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.291471+0000", 2024-09-13T10:10:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0037614499999999999 2024-09-13T10:10:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.291502+0000", 2024-09-13T10:10:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1338999999999999e-05 2024-09-13T10:10:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.294229+0000", 2024-09-13T10:10:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027261030000000001 2024-09-13T10:10:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.294286+0000", 2024-09-13T10:10:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7402999999999999e-05 2024-09-13T10:10:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.880 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.881 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.881 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:48 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'30, mlcod=25'20)", 2024-09-13T10:10:46.881 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.287932+0000", 2024-09-13T10:10:46.881 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.550594412999999, 2024-09-13T10:10:46.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0073708640000000004, 2024-09-13T10:10:46.881 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.881 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.881 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.881 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287932+0000", 2024-09-13T10:10:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287932+0000", 2024-09-13T10:10:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287934+0000", 2024-09-13T10:10:46.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.44e-06 2024-09-13T10:10:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287944+0000", 2024-09-13T10:10:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8959999999999998e-06 2024-09-13T10:10:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287964+0000", 2024-09-13T10:10:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0366000000000001e-05 2024-09-13T10:10:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287986+0000", 2024-09-13T10:10:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1665e-05 2024-09-13T10:10:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.292397+0000", 2024-09-13T10:10:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0044110390000000003 2024-09-13T10:10:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.292442+0000", 2024-09-13T10:10:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5692000000000002e-05 2024-09-13T10:10:46.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.295221+0000", 2024-09-13T10:10:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027789099999999999 2024-09-13T10:10:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.295303+0000", 2024-09-13T10:10:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1855999999999997e-05 2024-09-13T10:10:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:49 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'31, mlcod=25'21)", 2024-09-13T10:10:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.288557+0000", 2024-09-13T10:10:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.549969638, 2024-09-13T10:10:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0076013529999999999, 2024-09-13T10:10:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288557+0000", 2024-09-13T10:10:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288557+0000", 2024-09-13T10:10:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288559+0000", 2024-09-13T10:10:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.877e-06 2024-09-13T10:10:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288607+0000", 2024-09-13T10:10:46.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8013999999999998e-05 2024-09-13T10:10:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288630+0000", 2024-09-13T10:10:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3013000000000001e-05 2024-09-13T10:10:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288661+0000", 2024-09-13T10:10:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0824000000000002e-05 2024-09-13T10:10:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.293306+0000", 2024-09-13T10:10:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0046447939999999998 2024-09-13T10:10:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.293350+0000", 2024-09-13T10:10:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4603e-05 2024-09-13T10:10:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296071+0000", 2024-09-13T10:10:46.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027208639999999999 2024-09-13T10:10:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296158+0000", 2024-09-13T10:10:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7363999999999999e-05 2024-09-13T10:10:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:50 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'32, mlcod=25'21)", 2024-09-13T10:10:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.289006+0000", 2024-09-13T10:10:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.549520402999999, 2024-09-13T10:10:46.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0072323309999999998, 2024-09-13T10:10:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289006+0000", 2024-09-13T10:10:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289006+0000", 2024-09-13T10:10:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289008+0000", 2024-09-13T10:10:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9190000000000002e-06 2024-09-13T10:10:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289195+0000", 2024-09-13T10:10:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018727699999999999 2024-09-13T10:10:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289221+0000", 2024-09-13T10:10:46.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5585999999999999e-05 2024-09-13T10:10:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289233+0000", 2024-09-13T10:10:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1888999999999999e-05 2024-09-13T10:10:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.294346+0000", 2024-09-13T10:10:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0051129859999999999 2024-09-13T10:10:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.294390+0000", 2024-09-13T10:10:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3946000000000002e-05 2024-09-13T10:10:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296166+0000", 2024-09-13T10:10:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017759710000000001 2024-09-13T10:10:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296239+0000", 2024-09-13T10:10:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2756999999999995e-05 2024-09-13T10:10:46.894 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:51 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'33, mlcod=25'21)", 2024-09-13T10:10:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.289469+0000", 2024-09-13T10:10:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.549057774000001, 2024-09-13T10:10:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0082726669999999992, 2024-09-13T10:10:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289469+0000", 2024-09-13T10:10:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289469+0000", 2024-09-13T10:10:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289470+0000", 2024-09-13T10:10:46.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.463e-06 2024-09-13T10:10:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289483+0000", 2024-09-13T10:10:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.287e-05 2024-09-13T10:10:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289504+0000", 2024-09-13T10:10:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1146e-05 2024-09-13T10:10:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289520+0000", 2024-09-13T10:10:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5889e-05 2024-09-13T10:10:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.295380+0000", 2024-09-13T10:10:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058598160000000003 2024-09-13T10:10:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.295428+0000", 2024-09-13T10:10:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8128999999999999e-05 2024-09-13T10:10:46.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.297633+0000", 2024-09-13T10:10:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0022053049999999999 2024-09-13T10:10:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.297742+0000", 2024-09-13T10:10:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010804899999999999 2024-09-13T10:10:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.899 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:52 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'34, mlcod=25'21)", 2024-09-13T10:10:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.289917+0000", 2024-09-13T10:10:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.548609789, 2024-09-13T10:10:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0087424740000000001, 2024-09-13T10:10:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289917+0000", 2024-09-13T10:10:46.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289917+0000", 2024-09-13T10:10:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289918+0000", 2024-09-13T10:10:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.519e-06 2024-09-13T10:10:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289930+0000", 2024-09-13T10:10:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1216000000000001e-05 2024-09-13T10:10:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289956+0000", 2024-09-13T10:10:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6526e-05 2024-09-13T10:10:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289975+0000", 2024-09-13T10:10:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8471999999999999e-05 2024-09-13T10:10:46.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296563+0000", 2024-09-13T10:10:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0065881100000000003 2024-09-13T10:10:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296616+0000", 2024-09-13T10:10:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2913e-05 2024-09-13T10:10:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.298559+0000", 2024-09-13T10:10:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019437600000000001 2024-09-13T10:10:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.298659+0000", 2024-09-13T10:10:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9957999999999994e-05 2024-09-13T10:10:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:53 2.0 e25/24 2:30306672:devicehealth::main.db.0000000000000000:head v 25'35, mlcod=25'33)", 2024-09-13T10:10:46.904 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.300083+0000", 2024-09-13T10:10:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.538444102, 2024-09-13T10:10:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034380859999999999, 2024-09-13T10:10:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300083+0000", 2024-09-13T10:10:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300083+0000", 2024-09-13T10:10:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300088+0000", 2024-09-13T10:10:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9329999999999998e-06 2024-09-13T10:10:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300146+0000", 2024-09-13T10:10:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7442000000000003e-05 2024-09-13T10:10:46.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300171+0000", 2024-09-13T10:10:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4638999999999999e-05 2024-09-13T10:10:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300185+0000", 2024-09-13T10:10:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4418e-05 2024-09-13T10:10:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300304+0000", 2024-09-13T10:10:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119426 2024-09-13T10:10:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300345+0000", 2024-09-13T10:10:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0568999999999997e-05 2024-09-13T10:10:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.303445+0000", 2024-09-13T10:10:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0031000699999999999 2024-09-13T10:10:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.303521+0000", 2024-09-13T10:10:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5588999999999997e-05 2024-09-13T10:10:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:54 2.0 e25/24 2:30306672:devicehealth::main.db.0000000000000000:head v 25'36, mlcod=25'33)", 2024-09-13T10:10:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.300435+0000", 2024-09-13T10:10:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.538091308999999, 2024-09-13T10:10:46.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0031453900000000001, 2024-09-13T10:10:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300435+0000", 2024-09-13T10:10:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300435+0000", 2024-09-13T10:10:46.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300437+0000", 2024-09-13T10:10:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.347e-06 2024-09-13T10:10:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300534+0000", 2024-09-13T10:10:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6869999999999999e-05 2024-09-13T10:10:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300546+0000", 2024-09-13T10:10:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2767e-05 2024-09-13T10:10:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300556+0000", 2024-09-13T10:10:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5240000000000006e-06 2024-09-13T10:10:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.301379+0000", 2024-09-13T10:10:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00082360899999999999 2024-09-13T10:10:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.301414+0000", 2024-09-13T10:10:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4075999999999997e-05 2024-09-13T10:10:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.303525+0000", 2024-09-13T10:10:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021119620000000002 2024-09-13T10:10:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.303581+0000", 2024-09-13T10:10:46.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5235000000000002e-05 2024-09-13T10:10:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:55 2.0 e25/24 2:30306672:devicehealth::main.db.0000000000000000:head v 25'37, mlcod=25'33)", 2024-09-13T10:10:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.300928+0000", 2024-09-13T10:10:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.537598630000002, 2024-09-13T10:10:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029424490000000002, 2024-09-13T10:10:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.914 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300928+0000", 2024-09-13T10:10:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300928+0000", 2024-09-13T10:10:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.300940+0000", 2024-09-13T10:10:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2034e-05 2024-09-13T10:10:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.301047+0000", 2024-09-13T10:10:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107403 2024-09-13T10:10:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.301061+0000", 2024-09-13T10:10:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3164e-05 2024-09-13T10:10:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.301069+0000", 2024-09-13T10:10:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3620000000000006e-06 2024-09-13T10:10:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.302090+0000", 2024-09-13T10:10:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010206379999999999 2024-09-13T10:10:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.302139+0000", 2024-09-13T10:10:46.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9292000000000002e-05 2024-09-13T10:10:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.303769+0000", 2024-09-13T10:10:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00163031 2024-09-13T10:10:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.303870+0000", 2024-09-13T10:10:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000101246 2024-09-13T10:10:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.918 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:89 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'63, mlcod=25'60)", 2024-09-13T10:10:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.523928+0000", 2024-09-13T10:10:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.314599049000002, 2024-09-13T10:10:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030692079999999999, 2024-09-13T10:10:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.919 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.523928+0000", 2024-09-13T10:10:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.523928+0000", 2024-09-13T10:10:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.523929+0000", 2024-09-13T10:10:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.46e-06 2024-09-13T10:10:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.523940+0000", 2024-09-13T10:10:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1416e-05 2024-09-13T10:10:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.523962+0000", 2024-09-13T10:10:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2014999999999998e-05 2024-09-13T10:10:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.921 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.523977+0000", 2024-09-13T10:10:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5023e-05 2024-09-13T10:10:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.524564+0000", 2024-09-13T10:10:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00058669699999999996 2024-09-13T10:10:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.524614+0000", 2024-09-13T10:10:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0223999999999999e-05 2024-09-13T10:10:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.526895+0000", 2024-09-13T10:10:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0022810479999999999 2024-09-13T10:10:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.526997+0000", 2024-09-13T10:10:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000101325 2024-09-13T10:10:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:90 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'64, mlcod=25'60)", 2024-09-13T10:10:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.525495+0000", 2024-09-13T10:10:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.313031777999999, 2024-09-13T10:10:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002929095, 2024-09-13T10:10:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.525495+0000", 2024-09-13T10:10:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.525495+0000", 2024-09-13T10:10:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.525497+0000", 2024-09-13T10:10:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9259999999999999e-06 2024-09-13T10:10:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.525618+0000", 2024-09-13T10:10:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000121468 2024-09-13T10:10:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.525644+0000", 2024-09-13T10:10:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5485000000000001e-05 2024-09-13T10:10:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.525662+0000", 2024-09-13T10:10:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8292e-05 2024-09-13T10:10:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.525804+0000", 2024-09-13T10:10:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014158000000000001 2024-09-13T10:10:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.525865+0000", 2024-09-13T10:10:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1270000000000001e-05 2024-09-13T10:10:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.528348+0000", 2024-09-13T10:10:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0024835339999999999 2024-09-13T10:10:46.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.528424+0000", 2024-09-13T10:10:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5539999999999998e-05 2024-09-13T10:10:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:102 2.0 e25/24 2:30306672:devicehealth::main.db.0000000000000000:head v 25'76, mlcod=25'73)", 2024-09-13T10:10:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.564962+0000", 2024-09-13T10:10:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.273564846999999, 2024-09-13T10:10:46.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035915280000000001, 2024-09-13T10:10:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.564962+0000", 2024-09-13T10:10:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.564962+0000", 2024-09-13T10:10:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.564963+0000", 2024-09-13T10:10:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2869999999999999e-06 2024-09-13T10:10:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.565067+0000", 2024-09-13T10:10:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000103621 2024-09-13T10:10:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.565084+0000", 2024-09-13T10:10:46.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7385999999999999e-05 2024-09-13T10:10:46.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.565097+0000", 2024-09-13T10:10:46.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3145999999999999e-05 2024-09-13T10:10:46.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.565280+0000", 2024-09-13T10:10:46.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018305700000000001 2024-09-13T10:10:46.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.565360+0000", 2024-09-13T10:10:46.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9389999999999997e-05 2024-09-13T10:10:46.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.568410+0000", 2024-09-13T10:10:46.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030508110000000001 2024-09-13T10:10:46.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.568553+0000", 2024-09-13T10:10:46.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014283000000000001 2024-09-13T10:10:46.933 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.933 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.933 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.933 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:137 2.0 e26/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 26'103, mlcod=25'100)", 2024-09-13T10:10:46.933 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:31.499552+0000", 2024-09-13T10:10:46.933 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.338974272, 2024-09-13T10:10:46.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029914709999999999, 2024-09-13T10:10:46.933 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:46.933 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:46.933 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:46.934 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:46.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:46.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:31.499552+0000", 2024-09-13T10:10:46.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:46.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:31.499552+0000", 2024-09-13T10:10:46.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:46.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:46.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:31.499554+0000", 2024-09-13T10:10:46.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0370000000000001e-06 2024-09-13T10:10:46.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:46.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:31.499568+0000", 2024-09-13T10:10:46.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3256e-05 2024-09-13T10:10:46.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:46.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:31.499591+0000", 2024-09-13T10:10:46.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3048999999999999e-05 2024-09-13T10:10:46.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:46.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:31.499604+0000", 2024-09-13T10:10:46.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3149000000000001e-05 2024-09-13T10:10:46.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:46.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:31.499737+0000", 2024-09-13T10:10:46.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013338099999999999 2024-09-13T10:10:46.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:46.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:31.499798+0000", 2024-09-13T10:10:46.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0877e-05 2024-09-13T10:10:46.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:46.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:31.502435+0000", 2024-09-13T10:10:46.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026371039999999999 2024-09-13T10:10:46.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:46.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:46.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:46.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:31.502544+0000", 2024-09-13T10:10:46.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000108618 2024-09-13T10:10:46.937 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.937 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.938 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.938 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:46.938 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:46.938 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:10:46.939 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:10:46.939 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:10:46.940 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:10:46.940 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:10:46.941 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T10:10:46.941 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:46.936+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:46.941 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:10:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:10:46.942 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:10:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:10:46.942 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:10:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:10:46.943 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:10:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:10:46.964 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:10:46.964 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:10:46.985 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:10:46.985 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:10:47.005 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:10:47.005 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:10:47.022 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:10:47.022 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:10:47.039 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:47.036+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:47.044 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:10:47.044 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:10:47.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:10:47.046+0000 7feab9d9b640 0 -- 172.21.15.73:0/1452451918 >> v2:172.21.15.181:6804/1173379175 conn(0x7feabc222cd0 msgr2=0x7feabc225100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:10:47.112 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:47.112 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:47.126 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T10:10:47.141 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:47.138+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:10:47.241 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:10:47.238+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:10:47.242 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T10:10:47.271 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:10:47.272 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:10:47.287 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:47.288 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:47.300 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:10:47.301 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:10:47.302 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T10:10:47.342 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:10:47.336+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:10:47.424 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:47.425 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:47.438 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-13T10:10:47.443 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:10:47.442+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:10:47.544 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:10:47.542+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:10:47.560 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:47.560 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:47.574 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-13T10:10:47.646 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:10:47.644+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:10:47.696 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:47.696 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:47.710 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-13T10:10:47.746 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:10:47.740+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:10:47.833 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:47.833 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:47.847 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-13T10:10:47.848 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:10:47.844+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:10:47.948 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:10:47.944+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:10:47.968 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:47.968 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:47.983 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-13T10:10:48.048 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:48.043+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:48.105 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:48.105 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:48.120 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-13T10:10:48.164 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:10:48.159+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:10:48.243 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-13T10:10:48.250 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:48.244+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:10:48.286 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:48.286 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:48.300 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T10:10:48.351 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:48.350+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:10:48.438 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:48.438 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:48.452 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:48.451+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:48.453 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T10:10:48.554 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:48.551+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:48.574 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:48.574 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:48.588 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T10:10:48.655 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:10:48.651+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:10:48.710 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:48.710 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:48.724 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T10:10:48.755 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:10:48.751+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:10:48.848 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:48.848 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:48.856 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:48.854+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:10:48.865 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T10:10:48.957 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:10:48.958+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:10:49.032 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:49.032 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:49.046 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T10:10:49.058 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:10:49.052+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:10:49.159 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:49.159+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:49.219 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:49.219 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:49.233 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-13T10:10:49.260 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:49.255+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:49.326 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:10:49.358 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:49.358 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:49.361 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:49.358+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:10:49.373 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-13T10:10:49.462 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:10:49.456+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:10:49.470 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:10:49.470 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:10:49.491 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:10:49.491 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:10:49.512 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:10:49.512 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:10:49.533 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:10:49.533 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:10:49.540 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:49.540 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:49.552 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:10:49.552 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:10:49.555 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-13T10:10:49.563 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:49.559+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:49.573 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:10:49.574 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:10:49.594 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:10:49.595 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:10:49.615 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:10:49.616 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:10:49.633 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:10:49.633 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:10:49.653 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:10:49.653 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:10:49.664 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:10:49.660+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:10:49.672 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:10:49.672 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:10:49.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:10:49.673 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:10:49.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.673 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:68 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.673 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381353+0000", 2024-09-13T10:10:49.673 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.289680669999999, 2024-09-13T10:10:49.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.026093358000000001, 2024-09-13T10:10:49.673 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.673 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.673 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.673 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.673 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.673 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.674 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 68 2024-09-13T10:10:49.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.674 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381353+0000", 2024-09-13T10:10:49.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381353+0000", 2024-09-13T10:10:49.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381355+0000", 2024-09-13T10:10:49.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1930000000000001e-06 2024-09-13T10:10:49.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381362+0000", 2024-09-13T10:10:49.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8120000000000008e-06 2024-09-13T10:10:49.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381376+0000", 2024-09-13T10:10:49.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3263e-05 2024-09-13T10:10:49.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381481+0000", 2024-09-13T10:10:49.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000105004 2024-09-13T10:10:49.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.394395+0000", 2024-09-13T10:10:49.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012914518 2024-09-13T10:10:49.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.394568+0000", 2024-09-13T10:10:49.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017261300000000001 2024-09-13T10:10:49.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.394845+0000", 2024-09-13T10:10:49.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00027720700000000002 2024-09-13T10:10:49.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.397778+0000", 2024-09-13T10:10:49.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029325549999999999 2024-09-13T10:10:49.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.398084+0000", 2024-09-13T10:10:49.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030603200000000003 2024-09-13T10:10:49.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407244+0000", 2024-09-13T10:10:49.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0091600259999999999 2024-09-13T10:10:49.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407329+0000", 2024-09-13T10:10:49.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4927999999999997e-05 2024-09-13T10:10:49.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407447+0000", 2024-09-13T10:10:49.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000118207 2024-09-13T10:10:49.679 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.679 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.679 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.680 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:69 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.680 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381505+0000", 2024-09-13T10:10:49.680 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.289529129000002, 2024-09-13T10:10:49.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.026307490999999999, 2024-09-13T10:10:49.680 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.680 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.680 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.680 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.680 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.680 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.680 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 69 2024-09-13T10:10:49.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.681 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381505+0000", 2024-09-13T10:10:49.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381505+0000", 2024-09-13T10:10:49.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381506+0000", 2024-09-13T10:10:49.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0299999999999997e-07 2024-09-13T10:10:49.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381676+0000", 2024-09-13T10:10:49.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016962699999999999 2024-09-13T10:10:49.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381692+0000", 2024-09-13T10:10:49.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6099e-05 2024-09-13T10:10:49.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381701+0000", 2024-09-13T10:10:49.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5789999999999993e-06 2024-09-13T10:10:49.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.398159+0000", 2024-09-13T10:10:49.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016457690000000001 2024-09-13T10:10:49.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.398308+0000", 2024-09-13T10:10:49.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014868800000000001 2024-09-13T10:10:49.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.398531+0000", 2024-09-13T10:10:49.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000223124 2024-09-13T10:10:49.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407109+0000", 2024-09-13T10:10:49.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0085783860000000003 2024-09-13T10:10:49.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407221+0000", 2024-09-13T10:10:49.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011213999999999999 2024-09-13T10:10:49.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407645+0000", 2024-09-13T10:10:49.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00042394900000000002 2024-09-13T10:10:49.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407711+0000", 2024-09-13T10:10:49.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5419000000000005e-05 2024-09-13T10:10:49.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407812+0000", 2024-09-13T10:10:49.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101887 2024-09-13T10:10:49.686 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.686 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.686 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.687 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.687 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:10:49.687 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.289311655999999, 2024-09-13T10:10:49.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034624456999999997, 2024-09-13T10:10:49.687 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.687 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.687 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.687 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.687 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.687 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.687 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 70 2024-09-13T10:10:49.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.687 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:10:49.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:10:49.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381723+0000", 2024-09-13T10:10:49.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.540000000000001e-07 2024-09-13T10:10:49.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381731+0000", 2024-09-13T10:10:49.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.486e-06 2024-09-13T10:10:49.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381742+0000", 2024-09-13T10:10:49.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1269e-05 2024-09-13T10:10:49.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381750+0000", 2024-09-13T10:10:49.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8830000000000006e-06 2024-09-13T10:10:49.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401255+0000", 2024-09-13T10:10:49.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019505110999999999 2024-09-13T10:10:49.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401388+0000", 2024-09-13T10:10:49.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013253200000000001 2024-09-13T10:10:49.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401609+0000", 2024-09-13T10:10:49.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000220981 2024-09-13T10:10:49.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407538+0000", 2024-09-13T10:10:49.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059291220000000002 2024-09-13T10:10:49.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407626+0000", 2024-09-13T10:10:49.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8361000000000003e-05 2024-09-13T10:10:49.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416154+0000", 2024-09-13T10:10:49.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0085281000000000003 2024-09-13T10:10:49.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416234+0000", 2024-09-13T10:10:49.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9594999999999999e-05 2024-09-13T10:10:49.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416347+0000", 2024-09-13T10:10:49.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113063 2024-09-13T10:10:49.693 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.693 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.693 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.693 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:71 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.693 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:10:49.693 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.289271073999998, 2024-09-13T10:10:49.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035004, 2024-09-13T10:10:49.694 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.694 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.694 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.694 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.694 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.694 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.694 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 71 2024-09-13T10:10:49.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.694 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:10:49.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:10:49.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381764+0000", 2024-09-13T10:10:49.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4600000000000004e-07 2024-09-13T10:10:49.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381769+0000", 2024-09-13T10:10:49.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.48e-06 2024-09-13T10:10:49.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381780+0000", 2024-09-13T10:10:49.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0893000000000001e-05 2024-09-13T10:10:49.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381788+0000", 2024-09-13T10:10:49.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6389999999999992e-06 2024-09-13T10:10:49.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407896+0000", 2024-09-13T10:10:49.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.026107807 2024-09-13T10:10:49.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.408029+0000", 2024-09-13T10:10:49.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013383899999999999 2024-09-13T10:10:49.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.408250+0000", 2024-09-13T10:10:49.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022105399999999999 2024-09-13T10:10:49.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416130+0000", 2024-09-13T10:10:49.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0078798500000000007 2024-09-13T10:10:49.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416488+0000", 2024-09-13T10:10:49.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00035819299999999998 2024-09-13T10:10:49.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416595+0000", 2024-09-13T10:10:49.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001064 2024-09-13T10:10:49.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416662+0000", 2024-09-13T10:10:49.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6775999999999999e-05 2024-09-13T10:10:49.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416767+0000", 2024-09-13T10:10:49.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000105423 2024-09-13T10:10:49.699 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.699 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.699 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.700 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:72 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.700 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:10:49.700 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.28923404, 2024-09-13T10:10:49.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039399762999999997, 2024-09-13T10:10:49.700 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.700 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.700 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.700 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.700 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.700 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.700 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 72 2024-09-13T10:10:49.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.701 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:10:49.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:10:49.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381801+0000", 2024-09-13T10:10:49.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7700000000000004e-07 2024-09-13T10:10:49.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381898+0000", 2024-09-13T10:10:49.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7066999999999997e-05 2024-09-13T10:10:49.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381908+0000", 2024-09-13T10:10:49.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0336000000000001e-05 2024-09-13T10:10:49.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381917+0000", 2024-09-13T10:10:49.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5169999999999999e-06 2024-09-13T10:10:49.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.411900+0000", 2024-09-13T10:10:49.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029983381999999999 2024-09-13T10:10:49.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.412028+0000", 2024-09-13T10:10:49.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127916 2024-09-13T10:10:49.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.412248+0000", 2024-09-13T10:10:49.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022014699999999999 2024-09-13T10:10:49.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416576+0000", 2024-09-13T10:10:49.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004327549 2024-09-13T10:10:49.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416860+0000", 2024-09-13T10:10:49.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028391000000000001 2024-09-13T10:10:49.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421014+0000", 2024-09-13T10:10:49.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041539330000000003 2024-09-13T10:10:49.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421093+0000", 2024-09-13T10:10:49.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9097999999999994e-05 2024-09-13T10:10:49.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421200+0000", 2024-09-13T10:10:49.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000107231 2024-09-13T10:10:49.706 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.706 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.706 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.706 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:73 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.706 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:10:49.706 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.289100328, 2024-09-13T10:10:49.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044034528000000003, 2024-09-13T10:10:49.707 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.707 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.707 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.707 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.707 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.707 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.707 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 73 2024-09-13T10:10:49.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.707 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:10:49.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:10:49.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381935+0000", 2024-09-13T10:10:49.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0999999999999997e-07 2024-09-13T10:10:49.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381940+0000", 2024-09-13T10:10:49.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7030000000000003e-06 2024-09-13T10:10:49.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381949+0000", 2024-09-13T10:10:49.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5359999999999992e-06 2024-09-13T10:10:49.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381961+0000", 2024-09-13T10:10:49.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2539e-05 2024-09-13T10:10:49.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416942+0000", 2024-09-13T10:10:49.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034980270000000001 2024-09-13T10:10:49.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.417074+0000", 2024-09-13T10:10:49.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013246199999999999 2024-09-13T10:10:49.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.417298+0000", 2024-09-13T10:10:49.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000223823 2024-09-13T10:10:49.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425459+0000", 2024-09-13T10:10:49.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081613750000000002 2024-09-13T10:10:49.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425575+0000", 2024-09-13T10:10:49.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115513 2024-09-13T10:10:49.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425774+0000", 2024-09-13T10:10:49.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019959399999999999 2024-09-13T10:10:49.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425854+0000", 2024-09-13T10:10:49.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9532999999999999e-05 2024-09-13T10:10:49.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425968+0000", 2024-09-13T10:10:49.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011437 2024-09-13T10:10:49.713 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.713 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.713 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.713 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:74 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.713 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:10:49.713 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.289057454000002, 2024-09-13T10:10:49.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052701496, 2024-09-13T10:10:49.713 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.713 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.713 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.714 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.714 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.714 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.714 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 74 2024-09-13T10:10:49.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.714 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:10:49.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:10:49.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:10:49.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7999999999999995e-07 2024-09-13T10:10:49.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381983+0000", 2024-09-13T10:10:49.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4430000000000002e-06 2024-09-13T10:10:49.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381991+0000", 2024-09-13T10:10:49.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8960000000000003e-06 2024-09-13T10:10:49.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381998+0000", 2024-09-13T10:10:49.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7729999999999999e-06 2024-09-13T10:10:49.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421291+0000", 2024-09-13T10:10:49.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039292753 2024-09-13T10:10:49.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421424+0000", 2024-09-13T10:10:49.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013291900000000001 2024-09-13T10:10:49.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421646+0000", 2024-09-13T10:10:49.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000221843 2024-09-13T10:10:49.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425667+0000", 2024-09-13T10:10:49.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040206779999999998 2024-09-13T10:10:49.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425755+0000", 2024-09-13T10:10:49.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8848e-05 2024-09-13T10:10:49.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434472+0000", 2024-09-13T10:10:49.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0087164879999999997 2024-09-13T10:10:49.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434544+0000", 2024-09-13T10:10:49.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2273000000000004e-05 2024-09-13T10:10:49.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434678+0000", 2024-09-13T10:10:49.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013390200000000001 2024-09-13T10:10:49.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.723 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.723 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:75 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.723 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:10:49.723 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.289020589, 2024-09-13T10:10:49.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052864585999999998, 2024-09-13T10:10:49.723 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.723 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.723 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.723 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.723 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.724 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 75 2024-09-13T10:10:49.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.724 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:10:49.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:10:49.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:10:49.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8299999999999996e-07 2024-09-13T10:10:49.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382120+0000", 2024-09-13T10:10:49.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010627400000000001 2024-09-13T10:10:49.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382131+0000", 2024-09-13T10:10:49.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0796e-05 2024-09-13T10:10:49.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382140+0000", 2024-09-13T10:10:49.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5560000000000008e-06 2024-09-13T10:10:49.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426014+0000", 2024-09-13T10:10:49.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.043873843000000003 2024-09-13T10:10:49.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426205+0000", 2024-09-13T10:10:49.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019154600000000001 2024-09-13T10:10:49.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426429+0000", 2024-09-13T10:10:49.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022344699999999999 2024-09-13T10:10:49.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434352+0000", 2024-09-13T10:10:49.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007923355 2024-09-13T10:10:49.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434452+0000", 2024-09-13T10:10:49.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9961000000000001e-05 2024-09-13T10:10:49.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434700+0000", 2024-09-13T10:10:49.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024790899999999999 2024-09-13T10:10:49.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434774+0000", 2024-09-13T10:10:49.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4457999999999998e-05 2024-09-13T10:10:49.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434878+0000", 2024-09-13T10:10:49.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010375800000000001 2024-09-13T10:10:49.729 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.729 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.730 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.730 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:76 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.730 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:10:49.730 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.288853609, 2024-09-13T10:10:49.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053104138000000002, 2024-09-13T10:10:49.730 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.730 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.730 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.730 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.730 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.731 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.731 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 76 2024-09-13T10:10:49.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.731 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:10:49.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:10:49.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:10:49.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4600000000000003e-07 2024-09-13T10:10:49.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382188+0000", 2024-09-13T10:10:49.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4869999999999999e-06 2024-09-13T10:10:49.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382197+0000", 2024-09-13T10:10:49.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5519999999999993e-06 2024-09-13T10:10:49.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382207+0000", 2024-09-13T10:10:49.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7240000000000004e-06 2024-09-13T10:10:49.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430285+0000", 2024-09-13T10:10:49.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.048078093000000002 2024-09-13T10:10:49.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430429+0000", 2024-09-13T10:10:49.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000143981 2024-09-13T10:10:49.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430717+0000", 2024-09-13T10:10:49.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028802 2024-09-13T10:10:49.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435015+0000", 2024-09-13T10:10:49.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042973150000000003 2024-09-13T10:10:49.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435083+0000", 2024-09-13T10:10:49.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8448999999999995e-05 2024-09-13T10:10:49.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435109+0000", 2024-09-13T10:10:49.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6064999999999999e-05 2024-09-13T10:10:49.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435176+0000", 2024-09-13T10:10:49.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7362999999999999e-05 2024-09-13T10:10:49.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435285+0000", 2024-09-13T10:10:49.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108243 2024-09-13T10:10:49.736 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.736 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.737 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.737 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:95 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.737 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523740+0000", 2024-09-13T10:10:49.737 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.147294201000001, 2024-09-13T10:10:49.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025713889, 2024-09-13T10:10:49.737 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.737 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.737 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.737 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.737 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.737 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.738 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 95 2024-09-13T10:10:49.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.738 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523740+0000", 2024-09-13T10:10:49.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523740+0000", 2024-09-13T10:10:49.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523741+0000", 2024-09-13T10:10:49.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.06e-07 2024-09-13T10:10:49.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523746+0000", 2024-09-13T10:10:49.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3900000000000001e-06 2024-09-13T10:10:49.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523755+0000", 2024-09-13T10:10:49.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8990000000000007e-06 2024-09-13T10:10:49.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523766+0000", 2024-09-13T10:10:49.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0869999999999999e-05 2024-09-13T10:10:49.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.540435+0000", 2024-09-13T10:10:49.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016669033999999999 2024-09-13T10:10:49.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.540561+0000", 2024-09-13T10:10:49.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012601600000000001 2024-09-13T10:10:49.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.540779+0000", 2024-09-13T10:10:49.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021843399999999999 2024-09-13T10:10:49.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549095+0000", 2024-09-13T10:10:49.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0083153749999999998 2024-09-13T10:10:49.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549193+0000", 2024-09-13T10:10:49.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8301000000000006e-05 2024-09-13T10:10:49.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549308+0000", 2024-09-13T10:10:49.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000114886 2024-09-13T10:10:49.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549385+0000", 2024-09-13T10:10:49.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7576000000000004e-05 2024-09-13T10:10:49.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549454+0000", 2024-09-13T10:10:49.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8402e-05 2024-09-13T10:10:49.743 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.743 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.743 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.743 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:96 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.743 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:10:49.743 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.147222179, 2024-09-13T10:10:49.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032701419000000002, 2024-09-13T10:10:49.744 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.744 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.744 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.744 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.744 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.744 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.744 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 96 2024-09-13T10:10:49.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.744 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:10:49.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:10:49.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523813+0000", 2024-09-13T10:10:49.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.364e-06 2024-09-13T10:10:49.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523913+0000", 2024-09-13T10:10:49.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9827000000000005e-05 2024-09-13T10:10:49.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523927+0000", 2024-09-13T10:10:49.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4029000000000001e-05 2024-09-13T10:10:49.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523936+0000", 2024-09-13T10:10:49.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1030000000000008e-06 2024-09-13T10:10:49.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.544707+0000", 2024-09-13T10:10:49.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0207706 2024-09-13T10:10:49.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.544834+0000", 2024-09-13T10:10:49.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012683200000000001 2024-09-13T10:10:49.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.545057+0000", 2024-09-13T10:10:49.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022338100000000001 2024-09-13T10:10:49.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549279+0000", 2024-09-13T10:10:49.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042223080000000001 2024-09-13T10:10:49.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549556+0000", 2024-09-13T10:10:49.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000276307 2024-09-13T10:10:49.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556364+0000", 2024-09-13T10:10:49.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0068082510000000004 2024-09-13T10:10:49.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556445+0000", 2024-09-13T10:10:49.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1081000000000005e-05 2024-09-13T10:10:49.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556513+0000", 2024-09-13T10:10:49.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8336000000000004e-05 2024-09-13T10:10:49.750 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.750 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.750 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.750 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:97 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.750 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:10:49.750 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.147079294000001, 2024-09-13T10:10:49.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035410807000000002, 2024-09-13T10:10:49.750 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.750 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.751 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.751 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.751 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.751 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.751 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 97 2024-09-13T10:10:49.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.751 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:10:49.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:10:49.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523956+0000", 2024-09-13T10:10:49.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8599999999999997e-07 2024-09-13T10:10:49.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523962+0000", 2024-09-13T10:10:49.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1240000000000002e-06 2024-09-13T10:10:49.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523971+0000", 2024-09-13T10:10:49.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7320000000000004e-06 2024-09-13T10:10:49.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523978+0000", 2024-09-13T10:10:49.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0899999999999999e-06 2024-09-13T10:10:49.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549637+0000", 2024-09-13T10:10:49.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025658876000000001 2024-09-13T10:10:49.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549757+0000", 2024-09-13T10:10:49.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000120837 2024-09-13T10:10:49.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549971+0000", 2024-09-13T10:10:49.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021356000000000001 2024-09-13T10:10:49.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556335+0000", 2024-09-13T10:10:49.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0063639329999999996 2024-09-13T10:10:49.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556725+0000", 2024-09-13T10:10:49.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000390221 2024-09-13T10:10:49.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559219+0000", 2024-09-13T10:10:49.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0024943500000000002 2024-09-13T10:10:49.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559296+0000", 2024-09-13T10:10:49.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7065999999999997e-05 2024-09-13T10:10:49.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559366+0000", 2024-09-13T10:10:49.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9132000000000002e-05 2024-09-13T10:10:49.756 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.757 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.757 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.757 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:98 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.757 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:10:49.757 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.147044686000001, 2024-09-13T10:10:49.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035839604999999997, 2024-09-13T10:10:49.757 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.757 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.757 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.757 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.757 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.757 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.758 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 98 2024-09-13T10:10:49.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.758 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:10:49.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:10:49.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523990+0000", 2024-09-13T10:10:49.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6300000000000005e-07 2024-09-13T10:10:49.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523995+0000", 2024-09-13T10:10:49.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2669999999999997e-06 2024-09-13T10:10:49.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524007+0000", 2024-09-13T10:10:49.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2002e-05 2024-09-13T10:10:49.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524015+0000", 2024-09-13T10:10:49.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4869999999999996e-06 2024-09-13T10:10:49.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.553854+0000", 2024-09-13T10:10:49.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029839633000000001 2024-09-13T10:10:49.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.553985+0000", 2024-09-13T10:10:49.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013032899999999999 2024-09-13T10:10:49.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.554212+0000", 2024-09-13T10:10:49.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022723400000000001 2024-09-13T10:10:49.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559523+0000", 2024-09-13T10:10:49.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053113509999999997 2024-09-13T10:10:49.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559662+0000", 2024-09-13T10:10:49.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013833500000000001 2024-09-13T10:10:49.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559698+0000", 2024-09-13T10:10:49.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5982000000000002e-05 2024-09-13T10:10:49.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559764+0000", 2024-09-13T10:10:49.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6019000000000006e-05 2024-09-13T10:10:49.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559829+0000", 2024-09-13T10:10:49.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5302999999999995e-05 2024-09-13T10:10:49.763 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.763 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.764 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.764 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:99 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.764 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:10:49.764 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.147006992000001, 2024-09-13T10:10:49.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.037637084000000001, 2024-09-13T10:10:49.764 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.764 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.764 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.764 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.764 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.764 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.765 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 99 2024-09-13T10:10:49.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.765 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:10:49.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:10:49.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524028+0000", 2024-09-13T10:10:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7999999999999995e-07 2024-09-13T10:10:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524109+0000", 2024-09-13T10:10:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0834999999999994e-05 2024-09-13T10:10:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524119+0000", 2024-09-13T10:10:49.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0149999999999999e-05 2024-09-13T10:10:49.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524127+0000", 2024-09-13T10:10:49.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8660000000000006e-06 2024-09-13T10:10:49.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556868+0000", 2024-09-13T10:10:49.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.03274146 2024-09-13T10:10:49.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.557001+0000", 2024-09-13T10:10:49.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000133249 2024-09-13T10:10:49.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.557224+0000", 2024-09-13T10:10:49.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000222914 2024-09-13T10:10:49.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561387+0000", 2024-09-13T10:10:49.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041627019999999999 2024-09-13T10:10:49.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.769 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561419+0000", 2024-09-13T10:10:49.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1924999999999997e-05 2024-09-13T10:10:49.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.769 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561476+0000", 2024-09-13T10:10:49.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7346999999999998e-05 2024-09-13T10:10:49.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561552+0000", 2024-09-13T10:10:49.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5847999999999996e-05 2024-09-13T10:10:49.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561664+0000", 2024-09-13T10:10:49.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112108 2024-09-13T10:10:49.770 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.770 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.770 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.771 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:100 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.771 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:10:49.771 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.146830842, 2024-09-13T10:10:49.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.038825510000000001, 2024-09-13T10:10:49.771 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.771 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.771 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.771 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.771 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.771 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.771 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 100 2024-09-13T10:10:49.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.771 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:10:49.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:10:49.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524204+0000", 2024-09-13T10:10:49.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.54e-07 2024-09-13T10:10:49.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524211+0000", 2024-09-13T10:10:49.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6919999999999996e-06 2024-09-13T10:10:49.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524221+0000", 2024-09-13T10:10:49.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0223e-05 2024-09-13T10:10:49.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524229+0000", 2024-09-13T10:10:49.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9859999999999994e-06 2024-09-13T10:10:49.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559920+0000", 2024-09-13T10:10:49.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035690854000000001 2024-09-13T10:10:49.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.560052+0000", 2024-09-13T10:10:49.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013199899999999999 2024-09-13T10:10:49.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.560356+0000", 2024-09-13T10:10:49.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030450200000000001 2024-09-13T10:10:49.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562115+0000", 2024-09-13T10:10:49.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0017586430000000001 2024-09-13T10:10:49.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562512+0000", 2024-09-13T10:10:49.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00039728099999999998 2024-09-13T10:10:49.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562828+0000", 2024-09-13T10:10:49.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00031528799999999999 2024-09-13T10:10:49.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562913+0000", 2024-09-13T10:10:49.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5628000000000001e-05 2024-09-13T10:10:49.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.563029+0000", 2024-09-13T10:10:49.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011556 2024-09-13T10:10:49.777 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.777 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.777 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.777 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:121 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.777 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.640808+0000", 2024-09-13T10:10:49.777 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.030226363000001, 2024-09-13T10:10:49.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.027560819, 2024-09-13T10:10:49.777 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.778 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.778 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.778 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.778 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.778 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.778 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 121 2024-09-13T10:10:49.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.778 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.778 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.778 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640808+0000", 2024-09-13T10:10:49.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640808+0000", 2024-09-13T10:10:49.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640809+0000", 2024-09-13T10:10:49.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7e-06 2024-09-13T10:10:49.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640821+0000", 2024-09-13T10:10:49.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1666e-05 2024-09-13T10:10:49.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640839+0000", 2024-09-13T10:10:49.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8338999999999998e-05 2024-09-13T10:10:49.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640859+0000", 2024-09-13T10:10:49.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9972000000000001e-05 2024-09-13T10:10:49.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.660184+0000", 2024-09-13T10:10:49.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019324108999999999 2024-09-13T10:10:49.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.660413+0000", 2024-09-13T10:10:49.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000229163 2024-09-13T10:10:49.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.782 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.782 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.660777+0000", 2024-09-13T10:10:49.782 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00036470200000000001 2024-09-13T10:10:49.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.782 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.782 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.662947+0000", 2024-09-13T10:10:49.782 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021698059999999998 2024-09-13T10:10:49.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.782 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.782 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663078+0000", 2024-09-13T10:10:49.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013046800000000001 2024-09-13T10:10:49.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.783 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.783 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.783 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668230+0000", 2024-09-13T10:10:49.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0051526719999999996 2024-09-13T10:10:49.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.783 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.783 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.783 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668303+0000", 2024-09-13T10:10:49.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3102000000000006e-05 2024-09-13T10:10:49.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.784 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668369+0000", 2024-09-13T10:10:49.784 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.512e-05 2024-09-13T10:10:49.784 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.784 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.784 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.784 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:122 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.784 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:10:49.784 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.030146361, 2024-09-13T10:10:49.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.027900708, 2024-09-13T10:10:49.785 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.785 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.785 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.785 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.785 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.785 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.785 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 122 2024-09-13T10:10:49.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.785 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:10:49.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:10:49.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640889+0000", 2024-09-13T10:10:49.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4139999999999999e-06 2024-09-13T10:10:49.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.787 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.787 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640900+0000", 2024-09-13T10:10:49.787 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0559e-05 2024-09-13T10:10:49.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.787 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.787 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640916+0000", 2024-09-13T10:10:49.787 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6399e-05 2024-09-13T10:10:49.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.787 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.787 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640931+0000", 2024-09-13T10:10:49.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4659e-05 2024-09-13T10:10:49.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663602+0000", 2024-09-13T10:10:49.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.022671674999999999 2024-09-13T10:10:49.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663734+0000", 2024-09-13T10:10:49.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013189899999999999 2024-09-13T10:10:49.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663946+0000", 2024-09-13T10:10:49.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021198699999999999 2024-09-13T10:10:49.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668523+0000", 2024-09-13T10:10:49.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045765789999999999 2024-09-13T10:10:49.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668638+0000", 2024-09-13T10:10:49.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504 2024-09-13T10:10:49.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668657+0000", 2024-09-13T10:10:49.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9298999999999999e-05 2024-09-13T10:10:49.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668722+0000", 2024-09-13T10:10:49.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5126e-05 2024-09-13T10:10:49.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.791 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.791 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668788+0000", 2024-09-13T10:10:49.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6072e-05 2024-09-13T10:10:49.791 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.791 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.791 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.791 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:123 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.791 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:10:49.791 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.030080483999999, 2024-09-13T10:10:49.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028273795000000001, 2024-09-13T10:10:49.791 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.792 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.792 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.792 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.792 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.792 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.792 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 123 2024-09-13T10:10:49.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.792 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.792 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.792 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:10:49.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:10:49.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640955+0000", 2024-09-13T10:10:49.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.356e-06 2024-09-13T10:10:49.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641083+0000", 2024-09-13T10:10:49.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012763600000000001 2024-09-13T10:10:49.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641102+0000", 2024-09-13T10:10:49.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9055e-05 2024-09-13T10:10:49.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641117+0000", 2024-09-13T10:10:49.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5665999999999999e-05 2024-09-13T10:10:49.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.665747+0000", 2024-09-13T10:10:49.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.024629796999999998 2024-09-13T10:10:49.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.665876+0000", 2024-09-13T10:10:49.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012925600000000001 2024-09-13T10:10:49.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.666105+0000", 2024-09-13T10:10:49.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000228543 2024-09-13T10:10:49.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668924+0000", 2024-09-13T10:10:49.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028194309999999998 2024-09-13T10:10:49.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669070+0000", 2024-09-13T10:10:49.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014560199999999999 2024-09-13T10:10:49.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669104+0000", 2024-09-13T10:10:49.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3654e-05 2024-09-13T10:10:49.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669173+0000", 2024-09-13T10:10:49.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9819000000000004e-05 2024-09-13T10:10:49.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669227+0000", 2024-09-13T10:10:49.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3980000000000002e-05 2024-09-13T10:10:49.797 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.798 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.798 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.798 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:124 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:10:49.798 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:10:49.798 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.029887443, 2024-09-13T10:10:49.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030852801999999999, 2024-09-13T10:10:49.798 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.798 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.798 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.798 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.798 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.799 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.799 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 124 2024-09-13T10:10:49.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.799 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:10:49.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:10:49.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641148+0000", 2024-09-13T10:10:49.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4529999999999999e-06 2024-09-13T10:10:49.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641158+0000", 2024-09-13T10:10:49.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0003999999999999e-05 2024-09-13T10:10:49.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641199+0000", 2024-09-13T10:10:49.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0426999999999998e-05 2024-09-13T10:10:49.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641214+0000", 2024-09-13T10:10:49.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5716e-05 2024-09-13T10:10:49.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669267+0000", 2024-09-13T10:10:49.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028053112000000002 2024-09-13T10:10:49.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669394+0000", 2024-09-13T10:10:49.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012656000000000001 2024-09-13T10:10:49.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669614+0000", 2024-09-13T10:10:49.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021970400000000001 2024-09-13T10:10:49.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671267+0000", 2024-09-13T10:10:49.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001652976 2024-09-13T10:10:49.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671383+0000", 2024-09-13T10:10:49.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000116113 2024-09-13T10:10:49.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671828+0000", 2024-09-13T10:10:49.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000444871 2024-09-13T10:10:49.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671935+0000", 2024-09-13T10:10:49.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010703 2024-09-13T10:10:49.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671999+0000", 2024-09-13T10:10:49.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4836000000000001e-05 2024-09-13T10:10:49.804 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.804 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.804 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.805 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:142 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T10:10:49.805 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:31.498847+0000", 2024-09-13T10:10:49.805 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.172187331, 2024-09-13T10:10:49.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.026348094999999998, 2024-09-13T10:10:49.805 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:49.805 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:49.805 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:49.805 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:10:49.805 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:10:49.805 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:10:49.805 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 142 2024-09-13T10:10:49.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.806 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:49.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:49.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:31.498847+0000", 2024-09-13T10:10:49.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:49.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:31.498847+0000", 2024-09-13T10:10:49.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:49.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:49.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:31.498848+0000", 2024-09-13T10:10:49.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6300000000000001e-06 2024-09-13T10:10:49.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:49.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:31.498863+0000", 2024-09-13T10:10:49.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4926000000000001e-05 2024-09-13T10:10:49.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:49.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:31.498881+0000", 2024-09-13T10:10:49.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7189000000000001e-05 2024-09-13T10:10:49.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:31.498896+0000", 2024-09-13T10:10:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5038999999999999e-05 2024-09-13T10:10:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:31.521795+0000", 2024-09-13T10:10:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.022899631 2024-09-13T10:10:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:31.521912+0000", 2024-09-13T10:10:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011707899999999999 2024-09-13T10:10:49.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:10:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:31.522126+0000", 2024-09-13T10:10:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021385800000000001 2024-09-13T10:10:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:10:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:31.524562+0000", 2024-09-13T10:10:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002436236 2024-09-13T10:10:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:31.524851+0000", 2024-09-13T10:10:49.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028837999999999997 2024-09-13T10:10:49.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:49.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:31.524995+0000", 2024-09-13T10:10:49.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014390200000000001 2024-09-13T10:10:49.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:49.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:31.525079+0000", 2024-09-13T10:10:49.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4618999999999997e-05 2024-09-13T10:10:49.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:49.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:49.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:49.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:31.525195+0000", 2024-09-13T10:10:49.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115606 2024-09-13T10:10:49.811 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.811 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.811 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.811 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:49.811 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:49.811 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:10:49.812 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:10:49.812 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:10:49.813 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-13T10:10:49.814 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:10:49.814 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:10:49.814 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:10:49.815 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:10:49.815 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:10:49.815 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:10:49.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:10:49.738+0000 7fc8137fe640 0 -- 172.21.15.73:0/2251343285 >> v2:172.21.15.181:6804/1173379175 conn(0x7fc814221dd0 msgr2=0x7fc8142241e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:10:49.816 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:10:49.816 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:10:49.816 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:10:49.817 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:10:49.817 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:49.808+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:10:49.913 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:10:49.910+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:10:49.982 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:10:49.982 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:10:49.982 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:10:49.982 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:10:49.996 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-13T10:10:50.014 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:10:50.013+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:10:50.115 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:50.113+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:10:50.176 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-13T10:10:50.221 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:10:50.222+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:10:50.267 INFO:tasks.rados.rados.0.smithi195.stdout:1: write initial oid 1 2024-09-13T10:10:50.267 INFO:tasks.rados.rados.0.smithi195.stdout:1: seq_num 0 ranges {416807=475249,1542129=474678} 2024-09-13T10:10:50.272 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T10:10:50.282 INFO:tasks.rados.rados.0.smithi195.stdout:1: writing smithi19530714-1 from 416807 to 892056 tid 1 2024-09-13T10:10:50.301 INFO:tasks.rados.rados.0.smithi195.stdout:1: writing smithi19530714-1 from 1542129 to 2016807 tid 2 2024-09-13T10:10:50.301 INFO:tasks.rados.rados.0.smithi195.stdout:2: write initial oid 2 2024-09-13T10:10:50.301 INFO:tasks.rados.rados.0.smithi195.stdout:2: seq_num 1 ranges {416807=475249,1542129=474678} 2024-09-13T10:10:50.302 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:10:50.302 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:10:50.302 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:10:50.302 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:10:50.302 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:50.302 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4365.0:1 1.3 e25/24 1:c4c92e5a:::rbd_trash:head v 25'1, mlcod=0'0)", 2024-09-13T10:10:50.302 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.139280+0000", 2024-09-13T10:10:50.302 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 20.164021319, 2024-09-13T10:10:50.302 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000874579, 2024-09-13T10:10:50.303 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:10:50.303 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:10:50.303 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.303 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:10:50.303 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:50.303 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:10:50.303 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.139280+0000", 2024-09-13T10:10:50.303 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:50.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:50.303 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:50.303 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:10:50.303 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.139280+0000", 2024-09-13T10:10:50.303 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:10:50.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:50.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:50.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:10:50.304 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.139281+0000", 2024-09-13T10:10:50.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2e-07 2024-09-13T10:10:50.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:50.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:50.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:10:50.304 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.139289+0000", 2024-09-13T10:10:50.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7619999999999995e-06 2024-09-13T10:10:50.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:50.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:50.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:10:50.304 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.139303+0000", 2024-09-13T10:10:50.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4515000000000001e-05 2024-09-13T10:10:50.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:50.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:50.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.139312+0000", 2024-09-13T10:10:50.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9269999999999994e-06 2024-09-13T10:10:50.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:50.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:50.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:10:50.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.139371+0000", 2024-09-13T10:10:50.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8653999999999998e-05 2024-09-13T10:10:50.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:50.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:50.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:10:50.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.139391+0000", 2024-09-13T10:10:50.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0387e-05 2024-09-13T10:10:50.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:50.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:50.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:10:50.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.140053+0000", 2024-09-13T10:10:50.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00066136999999999999 2024-09-13T10:10:50.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:10:50.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:10:50.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:10:50.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.140155+0000", 2024-09-13T10:10:50.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102444 2024-09-13T10:10:50.307 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:50.307 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:50.307 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:50.307 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:10:50.307 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:10:50.307 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:10:50.315 INFO:tasks.rados.rados.0.smithi195.stdout:2: writing smithi19530714-2 from 416807 to 892056 tid 1 2024-09-13T10:10:50.316 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T10:10:50.318 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:10:50.313+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:10:50.330 INFO:tasks.rados.rados.0.smithi195.stdout:2: writing smithi19530714-2 from 1542129 to 2016807 tid 2 2024-09-13T10:10:50.330 INFO:tasks.rados.rados.0.smithi195.stdout:3: write initial oid 3 2024-09-13T10:10:50.330 INFO:tasks.rados.rados.0.smithi195.stdout:3: seq_num 2 ranges {433614=550498,1600611=433003} 2024-09-13T10:10:50.340 INFO:tasks.rados.rados.0.smithi195.stdout:3: writing smithi19530714-3 from 433614 to 984112 tid 1 2024-09-13T10:10:50.350 INFO:tasks.rados.rados.0.smithi195.stdout:3: writing smithi19530714-3 from 1600611 to 2033614 tid 2 2024-09-13T10:10:50.350 INFO:tasks.rados.rados.0.smithi195.stdout:4: write initial oid 4 2024-09-13T10:10:50.350 INFO:tasks.rados.rados.0.smithi195.stdout:4: seq_num 3 ranges {450421=625747,1659093=391328} 2024-09-13T10:10:50.358 INFO:tasks.rados.rados.0.smithi195.stdout:4: writing smithi19530714-4 from 450421 to 1076168 tid 1 2024-09-13T10:10:50.365 INFO:tasks.rados.rados.0.smithi195.stdout:4: writing smithi19530714-4 from 1659093 to 2050421 tid 2 2024-09-13T10:10:50.365 INFO:tasks.rados.rados.0.smithi195.stdout:5: write initial oid 5 2024-09-13T10:10:50.366 INFO:tasks.rados.rados.0.smithi195.stdout:1: finishing write tid 1 to smithi19530714-1 2024-09-13T10:10:50.366 INFO:tasks.rados.rados.0.smithi195.stdout:5: seq_num 4 ranges {467228=700996,1717575=349653} 2024-09-13T10:10:50.373 INFO:tasks.rados.rados.0.smithi195.stdout:5: writing smithi19530714-5 from 467228 to 1168224 tid 1 2024-09-13T10:10:50.380 INFO:tasks.rados.rados.0.smithi195.stdout:5: writing smithi19530714-5 from 1717575 to 2067228 tid 2 2024-09-13T10:10:50.380 INFO:tasks.rados.rados.0.smithi195.stdout:6: write initial oid 6 2024-09-13T10:10:50.380 INFO:tasks.rados.rados.0.smithi195.stdout:6: seq_num 5 ranges {484035=776245,2059704=24331} 2024-09-13T10:10:50.389 INFO:tasks.rados.rados.0.smithi195.stdout:6: writing smithi19530714-6 from 484035 to 1260280 tid 1 2024-09-13T10:10:50.395 INFO:tasks.rados.rados.0.smithi195.stdout:6: writing smithi19530714-6 from 2059704 to 2084035 tid 2 2024-09-13T10:10:50.395 INFO:tasks.rados.rados.0.smithi195.stdout:7: write initial oid 7 2024-09-13T10:10:50.395 INFO:tasks.rados.rados.0.smithi195.stdout:7: seq_num 6 ranges {500842=451494,1718186=382656} 2024-09-13T10:10:50.402 INFO:tasks.rados.rados.0.smithi195.stdout:7: writing smithi19530714-7 from 500842 to 952336 tid 1 2024-09-13T10:10:50.409 INFO:tasks.rados.rados.0.smithi195.stdout:7: writing smithi19530714-7 from 1718186 to 2100842 tid 2 2024-09-13T10:10:50.409 INFO:tasks.rados.rados.0.smithi195.stdout:8: write initial oid 8 2024-09-13T10:10:50.410 INFO:tasks.rados.rados.0.smithi195.stdout:8: seq_num 7 ranges {517649=526743,1776668=340981} 2024-09-13T10:10:50.416 INFO:tasks.rados.rados.0.smithi195.stdout:8: writing smithi19530714-8 from 517649 to 1044392 tid 1 2024-09-13T10:10:50.420 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:50.411+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:50.423 INFO:tasks.rados.rados.0.smithi195.stdout:8: writing smithi19530714-8 from 1776668 to 2117649 tid 2 2024-09-13T10:10:50.423 INFO:tasks.rados.rados.0.smithi195.stdout:9: write initial oid 9 2024-09-13T10:10:50.423 INFO:tasks.rados.rados.0.smithi195.stdout:9: seq_num 8 ranges {534456=718345,1951503=182953} 2024-09-13T10:10:50.430 INFO:tasks.rados.rados.0.smithi195.stdout:9: writing smithi19530714-9 from 534456 to 1252801 tid 1 2024-09-13T10:10:50.435 INFO:tasks.rados.rados.0.smithi195.stdout:9: writing smithi19530714-9 from 1951503 to 2134456 tid 2 2024-09-13T10:10:50.435 INFO:tasks.rados.rados.0.smithi195.stdout:10: write initial oid 10 2024-09-13T10:10:50.435 INFO:tasks.rados.rados.0.smithi195.stdout:10: seq_num 9 ranges {551263=793594,1893632=257631} 2024-09-13T10:10:50.443 INFO:tasks.rados.rados.0.smithi195.stdout:10: writing smithi19530714-10 from 551263 to 1344857 tid 1 2024-09-13T10:10:50.447 INFO:tasks.rados.rados.0.smithi195.stdout:10: writing smithi19530714-10 from 1893632 to 2151263 tid 2 2024-09-13T10:10:50.447 INFO:tasks.rados.rados.0.smithi195.stdout:11: write initial oid 11 2024-09-13T10:10:50.447 INFO:tasks.rados.rados.0.smithi195.stdout:11: seq_num 10 ranges {568070=468843,1552114=615956} 2024-09-13T10:10:50.453 INFO:tasks.rados.rados.0.smithi195.stdout:11: writing smithi19530714-11 from 568070 to 1036913 tid 1 2024-09-13T10:10:50.454 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:10:50.454 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:10:50.454 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:10:50.454 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:10:50.460 INFO:tasks.rados.rados.0.smithi195.stdout:11: writing smithi19530714-11 from 1552114 to 2168070 tid 2 2024-09-13T10:10:50.460 INFO:tasks.rados.rados.0.smithi195.stdout:12: write initial oid 12 2024-09-13T10:10:50.460 INFO:tasks.rados.rados.0.smithi195.stdout:1: finishing write tid 2 to smithi19530714-1 2024-09-13T10:10:50.460 INFO:tasks.rados.rados.0.smithi195.stdout:1: finishing write tid 3 to smithi19530714-1 2024-09-13T10:10:50.460 INFO:tasks.rados.rados.0.smithi195.stdout:1: finishing write tid 4 to smithi19530714-1 2024-09-13T10:10:50.460 INFO:tasks.rados.rados.0.smithi195.stdout:1: oid 1 updating version 0 to 1 2024-09-13T10:10:50.460 INFO:tasks.rados.rados.0.smithi195.stdout:1: oid 1 updating version 1 to 3 2024-09-13T10:10:50.461 INFO:tasks.rados.rados.0.smithi195.stdout:1: oid 1 version 3 is already newer than 2 2024-09-13T10:10:50.461 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-13T10:10:50.461 INFO:tasks.rados.rados.0.smithi195.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-09-13T10:10:50.461 INFO:tasks.rados.rados.0.smithi195.stdout:2: finishing write tid 1 to smithi19530714-2 2024-09-13T10:10:50.461 INFO:tasks.rados.rados.0.smithi195.stdout:12: seq_num 11 ranges {584877=544092,1610596=562003} 2024-09-13T10:10:50.466 INFO:tasks.rados.rados.0.smithi195.stdout:12: writing smithi19530714-12 from 584877 to 1128969 tid 1 2024-09-13T10:10:50.468 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T10:10:50.472 INFO:tasks.rados.rados.0.smithi195.stdout:12: writing smithi19530714-12 from 1610596 to 2172599 tid 2 2024-09-13T10:10:50.472 INFO:tasks.rados.rados.0.smithi195.stdout:1: done (11 left) 2024-09-13T10:10:50.472 INFO:tasks.rados.rados.0.smithi195.stdout:13: write initial oid 13 2024-09-13T10:10:50.472 INFO:tasks.rados.rados.0.smithi195.stdout:13: seq_num 12 ranges {601684=619341,1669078=532606} 2024-09-13T10:10:50.479 INFO:tasks.rados.rados.0.smithi195.stdout:13: writing smithi19530714-13 from 601684 to 1221025 tid 1 2024-09-13T10:10:50.485 INFO:tasks.rados.rados.0.smithi195.stdout:13: writing smithi19530714-13 from 1669078 to 2201684 tid 2 2024-09-13T10:10:50.485 INFO:tasks.rados.rados.0.smithi195.stdout:14: write initial oid 14 2024-09-13T10:10:50.485 INFO:tasks.rados.rados.0.smithi195.stdout:14: seq_num 13 ranges {618491=694590,2011207=207284} 2024-09-13T10:10:50.492 INFO:tasks.rados.rados.0.smithi195.stdout:14: writing smithi19530714-14 from 618491 to 1313081 tid 1 2024-09-13T10:10:50.498 INFO:tasks.rados.rados.0.smithi195.stdout:14: writing smithi19530714-14 from 2011207 to 2218491 tid 2 2024-09-13T10:10:50.498 INFO:tasks.rados.rados.0.smithi195.stdout:15: write initial oid 15 2024-09-13T10:10:50.498 INFO:tasks.rados.rados.0.smithi195.stdout:15: seq_num 14 ranges {635298=769839,2069689=165609} 2024-09-13T10:10:50.505 INFO:tasks.rados.rados.0.smithi195.stdout:15: writing smithi19530714-15 from 635298 to 1405137 tid 1 2024-09-13T10:10:50.510 INFO:tasks.rados.rados.0.smithi195.stdout:15: writing smithi19530714-15 from 2069689 to 2235298 tid 2 2024-09-13T10:10:50.511 INFO:tasks.rados.rados.0.smithi195.stdout:16: write initial oid 16 2024-09-13T10:10:50.511 INFO:tasks.rados.rados.0.smithi195.stdout:16: seq_num 15 ranges {652105=445088,1728171=452988} 2024-09-13T10:10:50.516 INFO:tasks.rados.rados.0.smithi195.stdout:16: writing smithi19530714-16 from 652105 to 1097193 tid 1 2024-09-13T10:10:50.520 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:50.518+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:10:50.523 INFO:tasks.rados.rados.0.smithi195.stdout:16: writing smithi19530714-16 from 1728171 to 2181159 tid 2 2024-09-13T10:10:50.523 INFO:tasks.rados.rados.0.smithi195.stdout:17: write initial oid 17 2024-09-13T10:10:50.523 INFO:tasks.rados.rados.0.smithi195.stdout:17: seq_num 16 ranges {668912=636690,1903006=365906} 2024-09-13T10:10:50.530 INFO:tasks.rados.rados.0.smithi195.stdout:17: writing smithi19530714-17 from 668912 to 1305602 tid 1 2024-09-13T10:10:50.536 INFO:tasks.rados.rados.0.smithi195.stdout:17: writing smithi19530714-17 from 1903006 to 2268912 tid 2 2024-09-13T10:10:50.536 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.536 INFO:tasks.rados.rados.0.smithi195.stdout:2: finishing write tid 2 to smithi19530714-2 2024-09-13T10:10:50.536 INFO:tasks.rados.rados.0.smithi195.stdout:2: finishing write tid 3 to smithi19530714-2 2024-09-13T10:10:50.536 INFO:tasks.rados.rados.0.smithi195.stdout:2: finishing write tid 4 to smithi19530714-2 2024-09-13T10:10:50.536 INFO:tasks.rados.rados.0.smithi195.stdout:2: oid 2 updating version 0 to 1 2024-09-13T10:10:50.536 INFO:tasks.rados.rados.0.smithi195.stdout:2: oid 2 updating version 1 to 2 2024-09-13T10:10:50.536 INFO:tasks.rados.rados.0.smithi195.stdout:2: oid 2 updating version 2 to 3 2024-09-13T10:10:50.536 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-13T10:10:50.536 INFO:tasks.rados.rados.0.smithi195.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-09-13T10:10:50.536 INFO:tasks.rados.rados.0.smithi195.stdout:3: finishing write tid 1 to smithi19530714-3 2024-09-13T10:10:50.536 INFO:tasks.rados.rados.0.smithi195.stdout:3: finishing write tid 2 to smithi19530714-3 2024-09-13T10:10:50.537 INFO:tasks.rados.rados.0.smithi195.stdout:3: finishing write tid 3 to smithi19530714-3 2024-09-13T10:10:50.537 INFO:tasks.rados.rados.0.smithi195.stdout:3: finishing write tid 4 to smithi19530714-3 2024-09-13T10:10:50.537 INFO:tasks.rados.rados.0.smithi195.stdout:3: oid 3 updating version 0 to 2 2024-09-13T10:10:50.537 INFO:tasks.rados.rados.0.smithi195.stdout:3: oid 3 updating version 2 to 3 2024-09-13T10:10:50.537 INFO:tasks.rados.rados.0.smithi195.stdout:3: oid 3 version 3 is already newer than 1 2024-09-13T10:10:50.537 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 3 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-13T10:10:50.537 INFO:tasks.rados.rados.0.smithi195.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-09-13T10:10:50.537 INFO:tasks.rados.rados.0.smithi195.stdout:2: done (15 left) 2024-09-13T10:10:50.537 INFO:tasks.rados.rados.0.smithi195.stdout:3: done (14 left) 2024-09-13T10:10:50.537 INFO:tasks.rados.rados.0.smithi195.stdout:18: write initial oid 18 2024-09-13T10:10:50.537 INFO:tasks.rados.rados.0.smithi195.stdout:18: seq_num 17 ranges {685719=711939,1845135=440584} 2024-09-13T10:10:50.543 INFO:tasks.rados.rados.0.smithi195.stdout:18: writing smithi19530714-18 from 685719 to 1397658 tid 1 2024-09-13T10:10:50.549 INFO:tasks.rados.rados.0.smithi195.stdout:18: writing smithi19530714-18 from 1845135 to 2285719 tid 2 2024-09-13T10:10:50.549 INFO:tasks.rados.rados.0.smithi195.stdout:19: write initial oid 19 2024-09-13T10:10:50.549 INFO:tasks.rados.rados.0.smithi195.stdout:19: seq_num 18 ranges {702526=787188,1903617=398909} 2024-09-13T10:10:50.557 INFO:tasks.rados.rados.0.smithi195.stdout:19: writing smithi19530714-19 from 702526 to 1489714 tid 1 2024-09-13T10:10:50.562 INFO:tasks.rados.rados.0.smithi195.stdout:19: writing smithi19530714-19 from 1903617 to 2302526 tid 2 2024-09-13T10:10:50.562 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.562 INFO:tasks.rados.rados.0.smithi195.stdout:4: finishing write tid 1 to smithi19530714-4 2024-09-13T10:10:50.563 INFO:tasks.rados.rados.0.smithi195.stdout:4: finishing write tid 2 to smithi19530714-4 2024-09-13T10:10:50.563 INFO:tasks.rados.rados.0.smithi195.stdout:4: finishing write tid 3 to smithi19530714-4 2024-09-13T10:10:50.563 INFO:tasks.rados.rados.0.smithi195.stdout:4: finishing write tid 4 to smithi19530714-4 2024-09-13T10:10:50.563 INFO:tasks.rados.rados.0.smithi195.stdout:4: oid 4 updating version 0 to 1 2024-09-13T10:10:50.563 INFO:tasks.rados.rados.0.smithi195.stdout:4: oid 4 updating version 1 to 2 2024-09-13T10:10:50.563 INFO:tasks.rados.rados.0.smithi195.stdout:4: oid 4 updating version 2 to 3 2024-09-13T10:10:50.563 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 3 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-13T10:10:50.563 INFO:tasks.rados.rados.0.smithi195.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-09-13T10:10:50.563 INFO:tasks.rados.rados.0.smithi195.stdout:5: finishing write tid 1 to smithi19530714-5 2024-09-13T10:10:50.563 INFO:tasks.rados.rados.0.smithi195.stdout:5: finishing write tid 2 to smithi19530714-5 2024-09-13T10:10:50.563 INFO:tasks.rados.rados.0.smithi195.stdout:5: finishing write tid 3 to smithi19530714-5 2024-09-13T10:10:50.563 INFO:tasks.rados.rados.0.smithi195.stdout:5: finishing write tid 4 to smithi19530714-5 2024-09-13T10:10:50.563 INFO:tasks.rados.rados.0.smithi195.stdout:5: oid 5 updating version 0 to 2 2024-09-13T10:10:50.564 INFO:tasks.rados.rados.0.smithi195.stdout:5: oid 5 version 2 is already newer than 1 2024-09-13T10:10:50.564 INFO:tasks.rados.rados.0.smithi195.stdout:5: oid 5 updating version 2 to 3 2024-09-13T10:10:50.564 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 3 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-13T10:10:50.564 INFO:tasks.rados.rados.0.smithi195.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-09-13T10:10:50.564 INFO:tasks.rados.rados.0.smithi195.stdout:6: finishing write tid 1 to smithi19530714-6 2024-09-13T10:10:50.564 INFO:tasks.rados.rados.0.smithi195.stdout:6: finishing write tid 2 to smithi19530714-6 2024-09-13T10:10:50.564 INFO:tasks.rados.rados.0.smithi195.stdout:6: finishing write tid 3 to smithi19530714-6 2024-09-13T10:10:50.564 INFO:tasks.rados.rados.0.smithi195.stdout:6: finishing write tid 4 to smithi19530714-6 2024-09-13T10:10:50.564 INFO:tasks.rados.rados.0.smithi195.stdout:6: oid 6 updating version 0 to 5 2024-09-13T10:10:50.564 INFO:tasks.rados.rados.0.smithi195.stdout:6: oid 6 version 5 is already newer than 4 2024-09-13T10:10:50.564 INFO:tasks.rados.rados.0.smithi195.stdout:6: oid 6 updating version 5 to 6 2024-09-13T10:10:50.564 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 6 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-13T10:10:50.564 INFO:tasks.rados.rados.0.smithi195.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-09-13T10:10:50.564 INFO:tasks.rados.rados.0.smithi195.stdout:4: done (15 left) 2024-09-13T10:10:50.565 INFO:tasks.rados.rados.0.smithi195.stdout:5: done (14 left) 2024-09-13T10:10:50.565 INFO:tasks.rados.rados.0.smithi195.stdout:6: done (13 left) 2024-09-13T10:10:50.565 INFO:tasks.rados.rados.0.smithi195.stdout:20: write initial oid 20 2024-09-13T10:10:50.565 INFO:tasks.rados.rados.0.smithi195.stdout:20: seq_num 19 ranges {719333=462437,1962099=357234} 2024-09-13T10:10:50.569 INFO:tasks.rados.rados.0.smithi195.stdout:20: writing smithi19530714-20 from 719333 to 1181770 tid 1 2024-09-13T10:10:50.575 INFO:tasks.rados.rados.0.smithi195.stdout:20: writing smithi19530714-20 from 1962099 to 2319333 tid 2 2024-09-13T10:10:50.575 INFO:tasks.rados.rados.0.smithi195.stdout:21: write initial oid 21 2024-09-13T10:10:50.575 INFO:tasks.rados.rados.0.smithi195.stdout:21: seq_num 20 ranges {736140=537686,2020581=315559} 2024-09-13T10:10:50.582 INFO:tasks.rados.rados.0.smithi195.stdout:21: writing smithi19530714-21 from 736140 to 1273826 tid 1 2024-09-13T10:10:50.588 INFO:tasks.rados.rados.0.smithi195.stdout:21: writing smithi19530714-21 from 2020581 to 2336140 tid 2 2024-09-13T10:10:50.588 INFO:tasks.rados.rados.0.smithi195.stdout:22: write initial oid 22 2024-09-13T10:10:50.588 INFO:tasks.rados.rados.0.smithi195.stdout:22: seq_num 21 ranges {752947=612935,1962710=390237} 2024-09-13T10:10:50.596 INFO:tasks.rados.rados.0.smithi195.stdout:22: writing smithi19530714-22 from 752947 to 1365882 tid 1 2024-09-13T10:10:50.597 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:10:50.597 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:10:50.598 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:10:50.598 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:10:50.598 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:10:50.602 INFO:tasks.rados.rados.0.smithi195.stdout:22: writing smithi19530714-22 from 1962710 to 2352947 tid 2 2024-09-13T10:10:50.602 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.602 INFO:tasks.rados.rados.0.smithi195.stdout:7: finishing write tid 1 to smithi19530714-7 2024-09-13T10:10:50.602 INFO:tasks.rados.rados.0.smithi195.stdout:7: finishing write tid 2 to smithi19530714-7 2024-09-13T10:10:50.602 INFO:tasks.rados.rados.0.smithi195.stdout:7: finishing write tid 3 to smithi19530714-7 2024-09-13T10:10:50.602 INFO:tasks.rados.rados.0.smithi195.stdout:7: finishing write tid 4 to smithi19530714-7 2024-09-13T10:10:50.602 INFO:tasks.rados.rados.0.smithi195.stdout:7: oid 7 updating version 0 to 4 2024-09-13T10:10:50.602 INFO:tasks.rados.rados.0.smithi195.stdout:7: oid 7 updating version 4 to 5 2024-09-13T10:10:50.602 INFO:tasks.rados.rados.0.smithi195.stdout:7: oid 7 updating version 5 to 6 2024-09-13T10:10:50.602 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 6 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-13T10:10:50.602 INFO:tasks.rados.rados.0.smithi195.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-09-13T10:10:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:8: finishing write tid 1 to smithi19530714-8 2024-09-13T10:10:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:8: finishing write tid 2 to smithi19530714-8 2024-09-13T10:10:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:8: finishing write tid 3 to smithi19530714-8 2024-09-13T10:10:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:8: finishing write tid 4 to smithi19530714-8 2024-09-13T10:10:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:8: oid 8 updating version 0 to 3 2024-09-13T10:10:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:8: oid 8 version 3 is already newer than 2 2024-09-13T10:10:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:8: oid 8 version 3 is already newer than 1 2024-09-13T10:10:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 3 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-13T10:10:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-09-13T10:10:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:7: done (15 left) 2024-09-13T10:10:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:8: done (14 left) 2024-09-13T10:10:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:23: write initial oid 23 2024-09-13T10:10:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:23: seq_num 22 ranges {769754=688184,2021192=348562} 2024-09-13T10:10:50.610 INFO:tasks.rados.rados.0.smithi195.stdout:23: writing smithi19530714-23 from 769754 to 1457938 tid 1 2024-09-13T10:10:50.613 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T10:10:50.615 INFO:tasks.rados.rados.0.smithi195.stdout:23: writing smithi19530714-23 from 2021192 to 2369754 tid 2 2024-09-13T10:10:50.615 INFO:tasks.rados.rados.0.smithi195.stdout:24: write initial oid 24 2024-09-13T10:10:50.615 INFO:tasks.rados.rados.0.smithi195.stdout:24: seq_num 23 ranges {786561=479786,1796027=467664} 2024-09-13T10:10:50.621 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:10:50.618+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:10:50.622 INFO:tasks.rados.rados.0.smithi195.stdout:24: writing smithi19530714-24 from 786561 to 1266347 tid 1 2024-09-13T10:10:50.628 INFO:tasks.rados.rados.0.smithi195.stdout:24: writing smithi19530714-24 from 1796027 to 2263691 tid 2 2024-09-13T10:10:50.628 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.628 INFO:tasks.rados.rados.0.smithi195.stdout:9: finishing write tid 1 to smithi19530714-9 2024-09-13T10:10:50.628 INFO:tasks.rados.rados.0.smithi195.stdout:9: finishing write tid 2 to smithi19530714-9 2024-09-13T10:10:50.628 INFO:tasks.rados.rados.0.smithi195.stdout:9: finishing write tid 3 to smithi19530714-9 2024-09-13T10:10:50.629 INFO:tasks.rados.rados.0.smithi195.stdout:9: finishing write tid 4 to smithi19530714-9 2024-09-13T10:10:50.629 INFO:tasks.rados.rados.0.smithi195.stdout:9: oid 9 updating version 0 to 6 2024-09-13T10:10:50.629 INFO:tasks.rados.rados.0.smithi195.stdout:9: oid 9 version 6 is already newer than 5 2024-09-13T10:10:50.629 INFO:tasks.rados.rados.0.smithi195.stdout:9: oid 9 version 6 is already newer than 4 2024-09-13T10:10:50.629 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 6 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-13T10:10:50.629 INFO:tasks.rados.rados.0.smithi195.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-09-13T10:10:50.629 INFO:tasks.rados.rados.0.smithi195.stdout:10: finishing write tid 1 to smithi19530714-10 2024-09-13T10:10:50.629 INFO:tasks.rados.rados.0.smithi195.stdout:9: done (15 left) 2024-09-13T10:10:50.629 INFO:tasks.rados.rados.0.smithi195.stdout:25: write initial oid 25 2024-09-13T10:10:50.629 INFO:tasks.rados.rados.0.smithi195.stdout:25: seq_num 24 ranges {403368=555035,1454509=727675} 2024-09-13T10:10:50.633 INFO:tasks.rados.rados.0.smithi195.stdout:25: writing smithi19530714-25 from 403368 to 958403 tid 1 2024-09-13T10:10:50.640 INFO:tasks.rados.rados.0.smithi195.stdout:25: writing smithi19530714-25 from 1454509 to 2182184 tid 2 2024-09-13T10:10:50.641 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.641 INFO:tasks.rados.rados.0.smithi195.stdout:10: finishing write tid 2 to smithi19530714-10 2024-09-13T10:10:50.641 INFO:tasks.rados.rados.0.smithi195.stdout:10: finishing write tid 3 to smithi19530714-10 2024-09-13T10:10:50.641 INFO:tasks.rados.rados.0.smithi195.stdout:11: finishing write tid 1 to smithi19530714-11 2024-09-13T10:10:50.641 INFO:tasks.rados.rados.0.smithi195.stdout:10: finishing write tid 4 to smithi19530714-10 2024-09-13T10:10:50.641 INFO:tasks.rados.rados.0.smithi195.stdout:10: oid 10 updating version 0 to 1 2024-09-13T10:10:50.641 INFO:tasks.rados.rados.0.smithi195.stdout:10: oid 10 updating version 1 to 2 2024-09-13T10:10:50.641 INFO:tasks.rados.rados.0.smithi195.stdout:10: oid 10 updating version 2 to 3 2024-09-13T10:10:50.641 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 3 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-13T10:10:50.642 INFO:tasks.rados.rados.0.smithi195.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-09-13T10:10:50.642 INFO:tasks.rados.rados.0.smithi195.stdout:11: finishing write tid 2 to smithi19530714-11 2024-09-13T10:10:50.642 INFO:tasks.rados.rados.0.smithi195.stdout:10: done (15 left) 2024-09-13T10:10:50.642 INFO:tasks.rados.rados.0.smithi195.stdout:26: write initial oid 26 2024-09-13T10:10:50.642 INFO:tasks.rados.rados.0.smithi195.stdout:26: seq_num 25 ranges {420175=630284,1796638=471333} 2024-09-13T10:10:50.646 INFO:tasks.rados.rados.0.smithi195.stdout:26: writing smithi19530714-26 from 420175 to 1050459 tid 1 2024-09-13T10:10:50.653 INFO:tasks.rados.rados.0.smithi195.stdout:26: writing smithi19530714-26 from 1796638 to 2267971 tid 2 2024-09-13T10:10:50.653 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.653 INFO:tasks.rados.rados.0.smithi195.stdout:11: finishing write tid 3 to smithi19530714-11 2024-09-13T10:10:50.653 INFO:tasks.rados.rados.0.smithi195.stdout:11: finishing write tid 4 to smithi19530714-11 2024-09-13T10:10:50.653 INFO:tasks.rados.rados.0.smithi195.stdout:11: oid 11 updating version 0 to 2 2024-09-13T10:10:50.653 INFO:tasks.rados.rados.0.smithi195.stdout:11: oid 11 version 2 is already newer than 1 2024-09-13T10:10:50.653 INFO:tasks.rados.rados.0.smithi195.stdout:11: oid 11 updating version 2 to 3 2024-09-13T10:10:50.653 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 3 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-13T10:10:50.653 INFO:tasks.rados.rados.0.smithi195.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-09-13T10:10:50.654 INFO:tasks.rados.rados.0.smithi195.stdout:12: finishing write tid 1 to smithi19530714-12 2024-09-13T10:10:50.654 INFO:tasks.rados.rados.0.smithi195.stdout:12: finishing write tid 2 to smithi19530714-12 2024-09-13T10:10:50.654 INFO:tasks.rados.rados.0.smithi195.stdout:11: done (15 left) 2024-09-13T10:10:50.654 INFO:tasks.rados.rados.0.smithi195.stdout:27: write initial oid 27 2024-09-13T10:10:50.654 INFO:tasks.rados.rados.0.smithi195.stdout:27: seq_num 26 ranges {436982=705533,1855120=581862} 2024-09-13T10:10:50.659 INFO:tasks.rados.rados.0.smithi195.stdout:27: writing smithi19530714-27 from 436982 to 1142515 tid 1 2024-09-13T10:10:50.667 INFO:tasks.rados.rados.0.smithi195.stdout:27: writing smithi19530714-27 from 1855120 to 2436982 tid 2 2024-09-13T10:10:50.667 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.667 INFO:tasks.rados.rados.0.smithi195.stdout:12: finishing write tid 3 to smithi19530714-12 2024-09-13T10:10:50.667 INFO:tasks.rados.rados.0.smithi195.stdout:12: finishing write tid 4 to smithi19530714-12 2024-09-13T10:10:50.667 INFO:tasks.rados.rados.0.smithi195.stdout:12: oid 12 updating version 0 to 1 2024-09-13T10:10:50.667 INFO:tasks.rados.rados.0.smithi195.stdout:12: oid 12 updating version 1 to 2 2024-09-13T10:10:50.667 INFO:tasks.rados.rados.0.smithi195.stdout:12: oid 12 updating version 2 to 3 2024-09-13T10:10:50.667 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 3 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-13T10:10:50.667 INFO:tasks.rados.rados.0.smithi195.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-09-13T10:10:50.667 INFO:tasks.rados.rados.0.smithi195.stdout:13: finishing write tid 1 to smithi19530714-13 2024-09-13T10:10:50.667 INFO:tasks.rados.rados.0.smithi195.stdout:12: done (15 left) 2024-09-13T10:10:50.667 INFO:tasks.rados.rados.0.smithi195.stdout:28: write initial oid 28 2024-09-13T10:10:50.668 INFO:tasks.rados.rados.0.smithi195.stdout:28: seq_num 27 ranges {453789=780782,1913602=475002} 2024-09-13T10:10:50.674 INFO:tasks.rados.rados.0.smithi195.stdout:28: writing smithi19530714-28 from 453789 to 1234571 tid 1 2024-09-13T10:10:50.681 INFO:tasks.rados.rados.0.smithi195.stdout:28: writing smithi19530714-28 from 1913602 to 2388604 tid 2 2024-09-13T10:10:50.681 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.681 INFO:tasks.rados.rados.0.smithi195.stdout:13: finishing write tid 2 to smithi19530714-13 2024-09-13T10:10:50.681 INFO:tasks.rados.rados.0.smithi195.stdout:13: finishing write tid 3 to smithi19530714-13 2024-09-13T10:10:50.681 INFO:tasks.rados.rados.0.smithi195.stdout:13: finishing write tid 4 to smithi19530714-13 2024-09-13T10:10:50.681 INFO:tasks.rados.rados.0.smithi195.stdout:13: oid 13 updating version 0 to 2 2024-09-13T10:10:50.681 INFO:tasks.rados.rados.0.smithi195.stdout:13: oid 13 updating version 2 to 3 2024-09-13T10:10:50.681 INFO:tasks.rados.rados.0.smithi195.stdout:13: oid 13 version 3 is already newer than 1 2024-09-13T10:10:50.681 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 3 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-13T10:10:50.681 INFO:tasks.rados.rados.0.smithi195.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-09-13T10:10:50.681 INFO:tasks.rados.rados.0.smithi195.stdout:14: finishing write tid 1 to smithi19530714-14 2024-09-13T10:10:50.682 INFO:tasks.rados.rados.0.smithi195.stdout:13: done (15 left) 2024-09-13T10:10:50.682 INFO:tasks.rados.rados.0.smithi195.stdout:29: write initial oid 29 2024-09-13T10:10:50.682 INFO:tasks.rados.rados.0.smithi195.stdout:29: seq_num 28 ranges {470596=456031,1572084=618660} 2024-09-13T10:10:50.685 INFO:tasks.rados.rados.0.smithi195.stdout:29: writing smithi19530714-29 from 470596 to 926627 tid 1 2024-09-13T10:10:50.693 INFO:tasks.rados.rados.0.smithi195.stdout:29: writing smithi19530714-29 from 1572084 to 2190744 tid 2 2024-09-13T10:10:50.693 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.693 INFO:tasks.rados.rados.0.smithi195.stdout:14: finishing write tid 2 to smithi19530714-14 2024-09-13T10:10:50.693 INFO:tasks.rados.rados.0.smithi195.stdout:14: finishing write tid 3 to smithi19530714-14 2024-09-13T10:10:50.693 INFO:tasks.rados.rados.0.smithi195.stdout:14: finishing write tid 4 to smithi19530714-14 2024-09-13T10:10:50.693 INFO:tasks.rados.rados.0.smithi195.stdout:14: oid 14 updating version 0 to 6 2024-09-13T10:10:50.693 INFO:tasks.rados.rados.0.smithi195.stdout:14: oid 14 version 6 is already newer than 4 2024-09-13T10:10:50.693 INFO:tasks.rados.rados.0.smithi195.stdout:14: oid 14 version 6 is already newer than 5 2024-09-13T10:10:50.693 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 6 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-13T10:10:50.694 INFO:tasks.rados.rados.0.smithi195.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-09-13T10:10:50.694 INFO:tasks.rados.rados.0.smithi195.stdout:15: finishing write tid 1 to smithi19530714-15 2024-09-13T10:10:50.694 INFO:tasks.rados.rados.0.smithi195.stdout:15: finishing write tid 2 to smithi19530714-15 2024-09-13T10:10:50.694 INFO:tasks.rados.rados.0.smithi195.stdout:15: finishing write tid 3 to smithi19530714-15 2024-09-13T10:10:50.694 INFO:tasks.rados.rados.0.smithi195.stdout:15: finishing write tid 4 to smithi19530714-15 2024-09-13T10:10:50.694 INFO:tasks.rados.rados.0.smithi195.stdout:15: oid 15 updating version 0 to 9 2024-09-13T10:10:50.694 INFO:tasks.rados.rados.0.smithi195.stdout:15: oid 15 version 9 is already newer than 8 2024-09-13T10:10:50.694 INFO:tasks.rados.rados.0.smithi195.stdout:15: oid 15 version 9 is already newer than 7 2024-09-13T10:10:50.694 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 9 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-13T10:10:50.694 INFO:tasks.rados.rados.0.smithi195.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-09-13T10:10:50.694 INFO:tasks.rados.rados.0.smithi195.stdout:14: done (15 left) 2024-09-13T10:10:50.694 INFO:tasks.rados.rados.0.smithi195.stdout:15: done (14 left) 2024-09-13T10:10:50.694 INFO:tasks.rados.rados.0.smithi195.stdout:30: write initial oid 30 2024-09-13T10:10:50.694 INFO:tasks.rados.rados.0.smithi195.stdout:30: seq_num 29 ranges {487403=531280,1514213=478671} 2024-09-13T10:10:50.698 INFO:tasks.rados.rados.0.smithi195.stdout:30: writing smithi19530714-30 from 487403 to 1018683 tid 1 2024-09-13T10:10:50.704 INFO:tasks.rados.rados.0.smithi195.stdout:30: writing smithi19530714-30 from 1514213 to 1992884 tid 2 2024-09-13T10:10:50.704 INFO:tasks.rados.rados.0.smithi195.stdout:31: write initial oid 31 2024-09-13T10:10:50.704 INFO:tasks.rados.rados.0.smithi195.stdout:31: seq_num 30 ranges {504210=606529,1572695=622329} 2024-09-13T10:10:50.710 INFO:tasks.rados.rados.0.smithi195.stdout:31: writing smithi19530714-31 from 504210 to 1110739 tid 1 2024-09-13T10:10:50.717 INFO:tasks.rados.rados.0.smithi195.stdout:31: writing smithi19530714-31 from 1572695 to 2195024 tid 2 2024-09-13T10:10:50.717 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.717 INFO:tasks.rados.rados.0.smithi195.stdout:17: finishing write tid 1 to smithi19530714-17 2024-09-13T10:10:50.717 INFO:tasks.rados.rados.0.smithi195.stdout:17: finishing write tid 2 to smithi19530714-17 2024-09-13T10:10:50.717 INFO:tasks.rados.rados.0.smithi195.stdout:17: finishing write tid 3 to smithi19530714-17 2024-09-13T10:10:50.717 INFO:tasks.rados.rados.0.smithi195.stdout:17: finishing write tid 4 to smithi19530714-17 2024-09-13T10:10:50.718 INFO:tasks.rados.rados.0.smithi195.stdout:17: oid 17 updating version 0 to 7 2024-09-13T10:10:50.718 INFO:tasks.rados.rados.0.smithi195.stdout:17: oid 17 updating version 7 to 8 2024-09-13T10:10:50.718 INFO:tasks.rados.rados.0.smithi195.stdout:17: oid 17 updating version 8 to 9 2024-09-13T10:10:50.718 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 9 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-13T10:10:50.718 INFO:tasks.rados.rados.0.smithi195.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-09-13T10:10:50.718 INFO:tasks.rados.rados.0.smithi195.stdout:18: finishing write tid 1 to smithi19530714-18 2024-09-13T10:10:50.718 INFO:tasks.rados.rados.0.smithi195.stdout:17: done (15 left) 2024-09-13T10:10:50.718 INFO:tasks.rados.rados.0.smithi195.stdout:32: write initial oid 32 2024-09-13T10:10:50.718 INFO:tasks.rados.rados.0.smithi195.stdout:32: seq_num 31 ranges {521017=798131,1747530=482340} 2024-09-13T10:10:50.722 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:10:50.716+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:10:50.724 INFO:tasks.rados.rados.0.smithi195.stdout:32: writing smithi19530714-32 from 521017 to 1319148 tid 1 2024-09-13T10:10:50.729 INFO:tasks.rados.rados.0.smithi195.stdout:32: writing smithi19530714-32 from 1747530 to 2229870 tid 2 2024-09-13T10:10:50.730 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.730 INFO:tasks.rados.rados.0.smithi195.stdout:18: finishing write tid 2 to smithi19530714-18 2024-09-13T10:10:50.730 INFO:tasks.rados.rados.0.smithi195.stdout:18: finishing write tid 3 to smithi19530714-18 2024-09-13T10:10:50.730 INFO:tasks.rados.rados.0.smithi195.stdout:18: finishing write tid 4 to smithi19530714-18 2024-09-13T10:10:50.730 INFO:tasks.rados.rados.0.smithi195.stdout:18: oid 18 updating version 0 to 12 2024-09-13T10:10:50.730 INFO:tasks.rados.rados.0.smithi195.stdout:18: oid 18 version 12 is already newer than 11 2024-09-13T10:10:50.730 INFO:tasks.rados.rados.0.smithi195.stdout:18: oid 18 version 12 is already newer than 10 2024-09-13T10:10:50.730 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 12 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-13T10:10:50.730 INFO:tasks.rados.rados.0.smithi195.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-09-13T10:10:50.730 INFO:tasks.rados.rados.0.smithi195.stdout:19: finishing write tid 1 to smithi19530714-19 2024-09-13T10:10:50.730 INFO:tasks.rados.rados.0.smithi195.stdout:19: finishing write tid 2 to smithi19530714-19 2024-09-13T10:10:50.730 INFO:tasks.rados.rados.0.smithi195.stdout:18: done (15 left) 2024-09-13T10:10:50.731 INFO:tasks.rados.rados.0.smithi195.stdout:33: write initial oid 33 2024-09-13T10:10:50.731 INFO:tasks.rados.rados.0.smithi195.stdout:33: seq_num 32 ranges {537824=473380,1806012=625998} 2024-09-13T10:10:50.735 INFO:tasks.rados.rados.0.smithi195.stdout:33: writing smithi19530714-33 from 537824 to 1011204 tid 1 2024-09-13T10:10:50.743 INFO:tasks.rados.rados.0.smithi195.stdout:33: writing smithi19530714-33 from 1806012 to 2432010 tid 2 2024-09-13T10:10:50.743 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.743 INFO:tasks.rados.rados.0.smithi195.stdout:19: finishing write tid 3 to smithi19530714-19 2024-09-13T10:10:50.743 INFO:tasks.rados.rados.0.smithi195.stdout:19: finishing write tid 4 to smithi19530714-19 2024-09-13T10:10:50.743 INFO:tasks.rados.rados.0.smithi195.stdout:19: oid 19 updating version 0 to 7 2024-09-13T10:10:50.743 INFO:tasks.rados.rados.0.smithi195.stdout:19: oid 19 updating version 7 to 8 2024-09-13T10:10:50.743 INFO:tasks.rados.rados.0.smithi195.stdout:19: oid 19 updating version 8 to 9 2024-09-13T10:10:50.744 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 9 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-13T10:10:50.744 INFO:tasks.rados.rados.0.smithi195.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-09-13T10:10:50.744 INFO:tasks.rados.rados.0.smithi195.stdout:19: done (15 left) 2024-09-13T10:10:50.744 INFO:tasks.rados.rados.0.smithi195.stdout:34: write initial oid 34 2024-09-13T10:10:50.744 INFO:tasks.rados.rados.0.smithi195.stdout:34: seq_num 33 ranges {554631=548629,1748141=486009} 2024-09-13T10:10:50.747 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:10:50.747 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:10:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:10:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:10:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:9 4.6 e38/31 4:6ddb2b23:::smithi19530714-3:head v 38'1, mlcod=0'0)", 2024-09-13T10:10:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.344772+0000", 2024-09-13T10:10:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.401567968, 2024-09-13T10:10:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003151049, 2024-09-13T10:10:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.344772+0000", 2024-09-13T10:10:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.344772+0000", 2024-09-13T10:10:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.344773+0000", 2024-09-13T10:10:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1999999999999999e-07 2024-09-13T10:10:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.345631+0000", 2024-09-13T10:10:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085816600000000005 2024-09-13T10:10:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.345652+0000", 2024-09-13T10:10:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0919000000000001e-05 2024-09-13T10:10:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.345669+0000", 2024-09-13T10:10:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6688e-05 2024-09-13T10:10:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.345774+0000", 2024-09-13T10:10:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104762 2024-09-13T10:10:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.345809+0000", 2024-09-13T10:10:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5717999999999999e-05 2024-09-13T10:10:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.347869+0000", 2024-09-13T10:10:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020599749999999999 2024-09-13T10:10:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.347924+0000", 2024-09-13T10:10:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4200999999999998e-05 2024-09-13T10:10:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:13 4.7 4:e81d9ad0:::smithi19530714-4:head [write 450421~625747,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.361038+0000", 2024-09-13T10:10:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.38530209199999998, 2024-09-13T10:10:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085135430000000002, 2024-09-13T10:10:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 13 2024-09-13T10:10:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.361038+0000", 2024-09-13T10:10:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.361038+0000", 2024-09-13T10:10:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.361040+0000", 2024-09-13T10:10:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.832e-06 2024-09-13T10:10:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.361973+0000", 2024-09-13T10:10:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093250899999999999 2024-09-13T10:10:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.361990+0000", 2024-09-13T10:10:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7326e-05 2024-09-13T10:10:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.361999+0000", 2024-09-13T10:10:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2110000000000004e-06 2024-09-13T10:10:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.362073+0000", 2024-09-13T10:10:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3852999999999999e-05 2024-09-13T10:10:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.362229+0000", 2024-09-13T10:10:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015545399999999999 2024-09-13T10:10:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:10:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.362365+0000", 2024-09-13T10:10:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136742 2024-09-13T10:10:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.364642+0000", 2024-09-13T10:10:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002276485 2024-09-13T10:10:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367079+0000", 2024-09-13T10:10:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024377169999999998 2024-09-13T10:10:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.369463+0000", 2024-09-13T10:10:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023832559999999998 2024-09-13T10:10:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.369509+0000", 2024-09-13T10:10:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5942999999999997e-05 2024-09-13T10:10:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.369552+0000", 2024-09-13T10:10:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3214999999999999e-05 2024-09-13T10:10:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:14 4.7 4:e81d9ad0:::smithi19530714-4:head [write 1659093~391328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.366920+0000", 2024-09-13T10:10:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.37942059900000003, 2024-09-13T10:10:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056107809999999996, 2024-09-13T10:10:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 14 2024-09-13T10:10:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.366920+0000", 2024-09-13T10:10:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.366920+0000", 2024-09-13T10:10:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.366922+0000", 2024-09-13T10:10:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4229999999999999e-06 2024-09-13T10:10:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367309+0000", 2024-09-13T10:10:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000386241 2024-09-13T10:10:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367327+0000", 2024-09-13T10:10:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8417999999999999e-05 2024-09-13T10:10:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367338+0000", 2024-09-13T10:10:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0824e-05 2024-09-13T10:10:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367386+0000", 2024-09-13T10:10:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8151e-05 2024-09-13T10:10:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367468+0000", 2024-09-13T10:10:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2341000000000003e-05 2024-09-13T10:10:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:10:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367588+0000", 2024-09-13T10:10:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119889 2024-09-13T10:10:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.369001+0000", 2024-09-13T10:10:50.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001413021 2024-09-13T10:10:50.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.370684+0000", 2024-09-13T10:10:50.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001683319 2024-09-13T10:10:50.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.372458+0000", 2024-09-13T10:10:50.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001773403 2024-09-13T10:10:50.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:50.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.372501+0000", 2024-09-13T10:10:50.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3120000000000001e-05 2024-09-13T10:10:50.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.372531+0000", 2024-09-13T10:10:50.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9631000000000001e-05 2024-09-13T10:10:50.774 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.775 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.775 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.775 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:15 4.7 4:e81d9ad0:::smithi19530714-4:head [setxattr _header (55) in=7b,truncate 2050421] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:50.775 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.367359+0000", 2024-09-13T10:10:50.775 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.37898177399999999, 2024-09-13T10:10:50.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053508569999999997, 2024-09-13T10:10:50.775 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.775 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.775 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.775 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:50.776 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:50.776 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:50.776 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 15 2024-09-13T10:10:50.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.776 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367359+0000", 2024-09-13T10:10:50.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367359+0000", 2024-09-13T10:10:50.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367360+0000", 2024-09-13T10:10:50.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.156e-06 2024-09-13T10:10:50.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367370+0000", 2024-09-13T10:10:50.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0508e-05 2024-09-13T10:10:50.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367380+0000", 2024-09-13T10:10:50.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.004e-05 2024-09-13T10:10:50.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367396+0000", 2024-09-13T10:10:50.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6078000000000001e-05 2024-09-13T10:10:50.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.368466+0000", 2024-09-13T10:10:50.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010696950000000001 2024-09-13T10:10:50.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.368543+0000", 2024-09-13T10:10:50.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7119000000000005e-05 2024-09-13T10:10:50.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:10:50.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.368671+0000", 2024-09-13T10:10:50.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012808900000000001 2024-09-13T10:10:50.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:50.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.371008+0000", 2024-09-13T10:10:50.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002337034 2024-09-13T10:10:50.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.371904+0000", 2024-09-13T10:10:50.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089606500000000003 2024-09-13T10:10:50.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.372587+0000", 2024-09-13T10:10:50.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068267899999999995 2024-09-13T10:10:50.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:50.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.372633+0000", 2024-09-13T10:10:50.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5751000000000003e-05 2024-09-13T10:10:50.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.372710+0000", 2024-09-13T10:10:50.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6643000000000005e-05 2024-09-13T10:10:50.781 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.782 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.782 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.782 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:16 4.7 4:e81d9ad0:::smithi19530714-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:50.782 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.367408+0000", 2024-09-13T10:10:50.782 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.37893202100000001, 2024-09-13T10:10:50.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054595499999999996, 2024-09-13T10:10:50.782 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.782 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.782 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:10:50.782 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:50.783 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:50.783 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:50.783 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 16 2024-09-13T10:10:50.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.783 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367408+0000", 2024-09-13T10:10:50.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367408+0000", 2024-09-13T10:10:50.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367410+0000", 2024-09-13T10:10:50.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.237e-06 2024-09-13T10:10:50.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367413+0000", 2024-09-13T10:10:50.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.788e-06 2024-09-13T10:10:50.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367422+0000", 2024-09-13T10:10:50.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5229999999999995e-06 2024-09-13T10:10:50.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.367430+0000", 2024-09-13T10:10:50.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2479999999999996e-06 2024-09-13T10:10:50.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.369033+0000", 2024-09-13T10:10:50.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00160232 2024-09-13T10:10:50.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:10:50.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.369105+0000", 2024-09-13T10:10:50.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2433000000000003e-05 2024-09-13T10:10:50.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.372740+0000", 2024-09-13T10:10:50.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036349569999999999 2024-09-13T10:10:50.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.372793+0000", 2024-09-13T10:10:50.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3156999999999999e-05 2024-09-13T10:10:50.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.372868+0000", 2024-09-13T10:10:50.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4887000000000003e-05 2024-09-13T10:10:50.788 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.788 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.788 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.789 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:21 4.6 e38/31 4:6150ba38:::smithi19530714-6:head v 38'4, mlcod=38'2)", 2024-09-13T10:10:50.789 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.393062+0000", 2024-09-13T10:10:50.789 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.35327867200000002, 2024-09-13T10:10:50.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031347520000000002, 2024-09-13T10:10:50.789 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.789 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.789 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.789 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.393062+0000", 2024-09-13T10:10:50.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.393062+0000", 2024-09-13T10:10:50.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.393079+0000", 2024-09-13T10:10:50.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7558999999999999e-05 2024-09-13T10:10:50.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.394148+0000", 2024-09-13T10:10:50.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010685860000000001 2024-09-13T10:10:50.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.394184+0000", 2024-09-13T10:10:50.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6396000000000002e-05 2024-09-13T10:10:50.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.394205+0000", 2024-09-13T10:10:50.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1132e-05 2024-09-13T10:10:50.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.394296+0000", 2024-09-13T10:10:50.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0203000000000004e-05 2024-09-13T10:10:50.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.394318+0000", 2024-09-13T10:10:50.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2785999999999998e-05 2024-09-13T10:10:50.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:50.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.396135+0000", 2024-09-13T10:10:50.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018166 2024-09-13T10:10:50.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.396197+0000", 2024-09-13T10:10:50.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1489999999999996e-05 2024-09-13T10:10:50.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.793 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.794 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.794 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:25 4.7 4:e8c2cbec:::smithi19530714-7:head [write 500842~451494,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:50.794 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.403240+0000", 2024-09-13T10:10:50.794 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.34310006199999998, 2024-09-13T10:10:50.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045220470000000004, 2024-09-13T10:10:50.794 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.794 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.794 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.795 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:50.795 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:50.795 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:50.795 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 25 2024-09-13T10:10:50.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.795 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.403240+0000", 2024-09-13T10:10:50.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.403240+0000", 2024-09-13T10:10:50.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.403243+0000", 2024-09-13T10:10:50.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3070000000000001e-06 2024-09-13T10:10:50.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.403652+0000", 2024-09-13T10:10:50.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00040887699999999998 2024-09-13T10:10:50.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.403670+0000", 2024-09-13T10:10:50.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8116000000000001e-05 2024-09-13T10:10:50.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.403678+0000", 2024-09-13T10:10:50.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6629999999999999e-06 2024-09-13T10:10:50.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.403750+0000", 2024-09-13T10:10:50.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2032000000000004e-05 2024-09-13T10:10:50.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.403879+0000", 2024-09-13T10:10:50.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012875500000000001 2024-09-13T10:10:50.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:10:50.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.404015+0000", 2024-09-13T10:10:50.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013554600000000001 2024-09-13T10:10:50.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:50.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.405297+0000", 2024-09-13T10:10:50.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012826980000000001 2024-09-13T10:10:50.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.407532+0000", 2024-09-13T10:10:50.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002235075 2024-09-13T10:10:50.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.407676+0000", 2024-09-13T10:10:50.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000143962 2024-09-13T10:10:50.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:50.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.407721+0000", 2024-09-13T10:10:50.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4709999999999997e-05 2024-09-13T10:10:50.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.407762+0000", 2024-09-13T10:10:50.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1306000000000001e-05 2024-09-13T10:10:50.804 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.804 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.804 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.805 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:26 4.7 4:e8c2cbec:::smithi19530714-7:head [write 1718186~382656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:50.805 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.410992+0000", 2024-09-13T10:10:50.805 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.33534819100000002, 2024-09-13T10:10:50.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050294169999999996, 2024-09-13T10:10:50.805 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.806 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.806 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.806 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:50.806 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:50.806 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:50.806 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 26 2024-09-13T10:10:50.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.807 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.410992+0000", 2024-09-13T10:10:50.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.410992+0000", 2024-09-13T10:10:50.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.410994+0000", 2024-09-13T10:10:50.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.697e-06 2024-09-13T10:10:50.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411345+0000", 2024-09-13T10:10:50.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035109600000000001 2024-09-13T10:10:50.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411359+0000", 2024-09-13T10:10:50.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4117000000000001e-05 2024-09-13T10:10:50.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411366+0000", 2024-09-13T10:10:50.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1949999999999997e-06 2024-09-13T10:10:50.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411438+0000", 2024-09-13T10:10:50.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1748999999999994e-05 2024-09-13T10:10:50.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411527+0000", 2024-09-13T10:10:50.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9073999999999995e-05 2024-09-13T10:10:50.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:10:50.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411660+0000", 2024-09-13T10:10:50.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013308800000000001 2024-09-13T10:10:50.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:50.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.413285+0000", 2024-09-13T10:10:50.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001625034 2024-09-13T10:10:50.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.415857+0000", 2024-09-13T10:10:50.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002571729 2024-09-13T10:10:50.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.415935+0000", 2024-09-13T10:10:50.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7917000000000006e-05 2024-09-13T10:10:50.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:50.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.415980+0000", 2024-09-13T10:10:50.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5114000000000002e-05 2024-09-13T10:10:50.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.416022+0000", 2024-09-13T10:10:50.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1607000000000003e-05 2024-09-13T10:10:50.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.820 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.820 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.820 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:27 4.7 4:e8c2cbec:::smithi19530714-7:head [setxattr _header (55) in=7b,truncate 2100842] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:50.820 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.411383+0000", 2024-09-13T10:10:50.821 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.33495779199999998, 2024-09-13T10:10:50.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052003489999999999, 2024-09-13T10:10:50.821 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.821 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.821 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.822 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:50.822 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:50.822 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:50.822 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 27 2024-09-13T10:10:50.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.823 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411383+0000", 2024-09-13T10:10:50.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411383+0000", 2024-09-13T10:10:50.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411384+0000", 2024-09-13T10:10:50.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.037e-06 2024-09-13T10:10:50.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411388+0000", 2024-09-13T10:10:50.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7600000000000002e-06 2024-09-13T10:10:50.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411397+0000", 2024-09-13T10:10:50.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2800000000000003e-06 2024-09-13T10:10:50.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411404+0000", 2024-09-13T10:10:50.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3170000000000004e-06 2024-09-13T10:10:50.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.412705+0000", 2024-09-13T10:10:50.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001301246 2024-09-13T10:10:50.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.412790+0000", 2024-09-13T10:10:50.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4522000000000002e-05 2024-09-13T10:10:50.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:10:50.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.412906+0000", 2024-09-13T10:10:50.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011596599999999999 2024-09-13T10:10:50.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:50.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.413831+0000", 2024-09-13T10:10:50.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092499399999999995 2024-09-13T10:10:50.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.416223+0000", 2024-09-13T10:10:50.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023922850000000001 2024-09-13T10:10:50.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.416504+0000", 2024-09-13T10:10:50.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028126200000000001 2024-09-13T10:10:50.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:50.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.416542+0000", 2024-09-13T10:10:50.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7265999999999999e-05 2024-09-13T10:10:50.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.416583+0000", 2024-09-13T10:10:50.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1414e-05 2024-09-13T10:10:50.830 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.830 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.830 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.830 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:28 4.7 4:e8c2cbec:::smithi19530714-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:50.830 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.411415+0000", 2024-09-13T10:10:50.830 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.33492527300000002, 2024-09-13T10:10:50.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005342098, 2024-09-13T10:10:50.831 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.831 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.831 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:10:50.831 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:50.831 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:50.831 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:50.831 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 28 2024-09-13T10:10:50.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.831 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411415+0000", 2024-09-13T10:10:50.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411415+0000", 2024-09-13T10:10:50.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411416+0000", 2024-09-13T10:10:50.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.06e-07 2024-09-13T10:10:50.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411419+0000", 2024-09-13T10:10:50.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4769999999999999e-06 2024-09-13T10:10:50.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411437+0000", 2024-09-13T10:10:50.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7592999999999999e-05 2024-09-13T10:10:50.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.411445+0000", 2024-09-13T10:10:50.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2490000000000001e-06 2024-09-13T10:10:50.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.413192+0000", 2024-09-13T10:10:50.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00174709 2024-09-13T10:10:50.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:10:50.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.413274+0000", 2024-09-13T10:10:50.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1403999999999995e-05 2024-09-13T10:10:50.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.416629+0000", 2024-09-13T10:10:50.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033550160000000002 2024-09-13T10:10:50.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.416682+0000", 2024-09-13T10:10:50.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3152999999999997e-05 2024-09-13T10:10:50.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.416757+0000", 2024-09-13T10:10:50.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5409999999999998e-05 2024-09-13T10:10:50.836 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.836 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.836 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.836 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:65 4.7 4:e55883d8:::smithi19530714-17:head [write 668912~636690,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:50.836 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.531573+0000", 2024-09-13T10:10:50.836 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.21476768700000001, 2024-09-13T10:10:50.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097462729999999997, 2024-09-13T10:10:50.836 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.836 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.836 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.836 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:50.837 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:50.837 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:50.837 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 65 2024-09-13T10:10:50.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.837 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.531573+0000", 2024-09-13T10:10:50.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.531573+0000", 2024-09-13T10:10:50.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.531576+0000", 2024-09-13T10:10:50.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9110000000000002e-06 2024-09-13T10:10:50.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.532124+0000", 2024-09-13T10:10:50.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054878200000000005 2024-09-13T10:10:50.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.532146+0000", 2024-09-13T10:10:50.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1263999999999998e-05 2024-09-13T10:10:50.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.532156+0000", 2024-09-13T10:10:50.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.005e-05 2024-09-13T10:10:50.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.532289+0000", 2024-09-13T10:10:50.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013335699999999999 2024-09-13T10:10:50.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.532527+0000", 2024-09-13T10:10:50.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000238206 2024-09-13T10:10:50.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:10:50.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.532701+0000", 2024-09-13T10:10:50.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017368799999999999 2024-09-13T10:10:50.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:50.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.534593+0000", 2024-09-13T10:10:50.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018920899999999999 2024-09-13T10:10:50.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.540142+0000", 2024-09-13T10:10:50.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005548847 2024-09-13T10:10:50.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.541249+0000", 2024-09-13T10:10:50.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001106587 2024-09-13T10:10:50.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:50.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.541283+0000", 2024-09-13T10:10:50.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4412999999999997e-05 2024-09-13T10:10:50.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.541319+0000", 2024-09-13T10:10:50.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6078000000000003e-05 2024-09-13T10:10:50.843 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.843 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.843 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.843 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:66 4.7 4:e55883d8:::smithi19530714-17:head [write 1903006~365906] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:50.843 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.536834+0000", 2024-09-13T10:10:50.843 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.209506252, 2024-09-13T10:10:50.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049250459999999998, 2024-09-13T10:10:50.844 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.844 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.844 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.844 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:50.844 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:50.844 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:50.844 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 66 2024-09-13T10:10:50.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.844 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.536834+0000", 2024-09-13T10:10:50.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.536834+0000", 2024-09-13T10:10:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.536836+0000", 2024-09-13T10:10:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.584e-06 2024-09-13T10:10:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537092+0000", 2024-09-13T10:10:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025603900000000001 2024-09-13T10:10:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537106+0000", 2024-09-13T10:10:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3733999999999999e-05 2024-09-13T10:10:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537112+0000", 2024-09-13T10:10:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6490000000000002e-06 2024-09-13T10:10:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537172+0000", 2024-09-13T10:10:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9787999999999997e-05 2024-09-13T10:10:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537245+0000", 2024-09-13T10:10:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2642e-05 2024-09-13T10:10:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:10:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537339+0000", 2024-09-13T10:10:50.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4592000000000005e-05 2024-09-13T10:10:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.538659+0000", 2024-09-13T10:10:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013199970000000001 2024-09-13T10:10:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.541115+0000", 2024-09-13T10:10:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024557870000000001 2024-09-13T10:10:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.541712+0000", 2024-09-13T10:10:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059686500000000003 2024-09-13T10:10:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.541739+0000", 2024-09-13T10:10:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7489999999999999e-05 2024-09-13T10:10:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.541759+0000", 2024-09-13T10:10:50.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9879000000000001e-05 2024-09-13T10:10:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:67 4.7 4:e55883d8:::smithi19530714-17:head [setxattr _header (56) in=7b,truncate 2268912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.537129+0000", 2024-09-13T10:10:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.20921192999999999, 2024-09-13T10:10:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050388259999999997, 2024-09-13T10:10:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.850 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 67 2024-09-13T10:10:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537129+0000", 2024-09-13T10:10:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537129+0000", 2024-09-13T10:10:50.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537129+0000", 2024-09-13T10:10:50.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0699999999999997e-07 2024-09-13T10:10:50.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537133+0000", 2024-09-13T10:10:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.472e-06 2024-09-13T10:10:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537142+0000", 2024-09-13T10:10:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5070000000000006e-06 2024-09-13T10:10:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537148+0000", 2024-09-13T10:10:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1460000000000001e-06 2024-09-13T10:10:50.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.538077+0000", 2024-09-13T10:10:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00092827600000000001 2024-09-13T10:10:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.538157+0000", 2024-09-13T10:10:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0146000000000001e-05 2024-09-13T10:10:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:10:50.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.538296+0000", 2024-09-13T10:10:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013951999999999999 2024-09-13T10:10:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.539359+0000", 2024-09-13T10:10:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010624600000000001 2024-09-13T10:10:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.541540+0000", 2024-09-13T10:10:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021816439999999999 2024-09-13T10:10:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.542116+0000", 2024-09-13T10:10:50.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00057531900000000003 2024-09-13T10:10:50.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.542140+0000", 2024-09-13T10:10:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4553999999999999e-05 2024-09-13T10:10:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.542167+0000", 2024-09-13T10:10:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7175e-05 2024-09-13T10:10:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.857 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:68 4.7 4:e55883d8:::smithi19530714-17:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.537168+0000", 2024-09-13T10:10:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.209172472, 2024-09-13T10:10:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051331520000000002, 2024-09-13T10:10:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:10:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:50.858 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 68 2024-09-13T10:10:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537168+0000", 2024-09-13T10:10:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537168+0000", 2024-09-13T10:10:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537168+0000", 2024-09-13T10:10:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1099999999999996e-07 2024-09-13T10:10:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537172+0000", 2024-09-13T10:10:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2059999999999999e-06 2024-09-13T10:10:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537177+0000", 2024-09-13T10:10:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3299999999999998e-06 2024-09-13T10:10:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.537183+0000", 2024-09-13T10:10:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.553e-06 2024-09-13T10:10:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.538594+0000", 2024-09-13T10:10:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014109909999999999 2024-09-13T10:10:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:10:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.538648+0000", 2024-09-13T10:10:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4506999999999998e-05 2024-09-13T10:10:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.542205+0000", 2024-09-13T10:10:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035573509999999998 2024-09-13T10:10:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.542246+0000", 2024-09-13T10:10:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1025000000000001e-05 2024-09-13T10:10:50.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.542301+0000", 2024-09-13T10:10:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4678000000000001e-05 2024-09-13T10:10:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:69 4.7 4:ea79a1ec:::smithi19530714-18:head [write 685719~711939,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.545181+0000", 2024-09-13T10:10:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.201159271, 2024-09-13T10:10:50.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0090297710000000007, 2024-09-13T10:10:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 69 2024-09-13T10:10:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.545181+0000", 2024-09-13T10:10:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.545181+0000", 2024-09-13T10:10:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.545183+0000", 2024-09-13T10:10:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5060000000000001e-06 2024-09-13T10:10:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.545741+0000", 2024-09-13T10:10:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00055815400000000005 2024-09-13T10:10:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.545756+0000", 2024-09-13T10:10:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4779e-05 2024-09-13T10:10:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.545763+0000", 2024-09-13T10:10:50.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8450000000000003e-06 2024-09-13T10:10:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.545819+0000", 2024-09-13T10:10:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5520999999999999e-05 2024-09-13T10:10:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.545899+0000", 2024-09-13T10:10:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0302000000000005e-05 2024-09-13T10:10:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:10:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.545995+0000", 2024-09-13T10:10:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6058999999999996e-05 2024-09-13T10:10:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.547464+0000", 2024-09-13T10:10:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014688100000000001 2024-09-13T10:10:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550303+0000", 2024-09-13T10:10:50.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028385429999999998 2024-09-13T10:10:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.554144+0000", 2024-09-13T10:10:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00384147 2024-09-13T10:10:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.554176+0000", 2024-09-13T10:10:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1918000000000002e-05 2024-09-13T10:10:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.554211+0000", 2024-09-13T10:10:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4863999999999997e-05 2024-09-13T10:10:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:70 4.7 4:ea79a1ec:::smithi19530714-18:head [write 1845135~440584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.549896+0000", 2024-09-13T10:10:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.19644428999999999, 2024-09-13T10:10:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047647820000000004, 2024-09-13T10:10:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.870 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 70 2024-09-13T10:10:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.549896+0000", 2024-09-13T10:10:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.549896+0000", 2024-09-13T10:10:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.549897+0000", 2024-09-13T10:10:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2669999999999999e-06 2024-09-13T10:10:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550321+0000", 2024-09-13T10:10:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042382800000000001 2024-09-13T10:10:50.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550333+0000", 2024-09-13T10:10:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1581e-05 2024-09-13T10:10:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550338+0000", 2024-09-13T10:10:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4720000000000003e-06 2024-09-13T10:10:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550381+0000", 2024-09-13T10:10:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2406e-05 2024-09-13T10:10:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550434+0000", 2024-09-13T10:10:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3712999999999999e-05 2024-09-13T10:10:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:10:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550519+0000", 2024-09-13T10:10:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4756e-05 2024-09-13T10:10:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.552436+0000", 2024-09-13T10:10:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019163419999999999 2024-09-13T10:10:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.554302+0000", 2024-09-13T10:10:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018667379999999999 2024-09-13T10:10:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.554615+0000", 2024-09-13T10:10:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000312868 2024-09-13T10:10:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.554642+0000", 2024-09-13T10:10:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6415999999999999e-05 2024-09-13T10:10:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.554661+0000", 2024-09-13T10:10:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9395e-05 2024-09-13T10:10:50.876 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:71 4.7 4:ea79a1ec:::smithi19530714-18:head [setxattr _header (56) in=7b,truncate 2285719] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.550352+0000", 2024-09-13T10:10:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.19598885599999999, 2024-09-13T10:10:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048952020000000004, 2024-09-13T10:10:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:50.877 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 71 2024-09-13T10:10:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550352+0000", 2024-09-13T10:10:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550352+0000", 2024-09-13T10:10:50.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550352+0000", 2024-09-13T10:10:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7300000000000001e-07 2024-09-13T10:10:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550355+0000", 2024-09-13T10:10:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1559999999999999e-06 2024-09-13T10:10:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550359+0000", 2024-09-13T10:10:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.194e-06 2024-09-13T10:10:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550364+0000", 2024-09-13T10:10:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8810000000000004e-06 2024-09-13T10:10:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.551385+0000", 2024-09-13T10:10:50.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010211899999999999 2024-09-13T10:10:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.551543+0000", 2024-09-13T10:10:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015705800000000001 2024-09-13T10:10:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:10:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.551822+0000", 2024-09-13T10:10:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027924299999999998 2024-09-13T10:10:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.552585+0000", 2024-09-13T10:10:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00076318899999999995 2024-09-13T10:10:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.554739+0000", 2024-09-13T10:10:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021543140000000001 2024-09-13T10:10:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.555180+0000", 2024-09-13T10:10:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044111699999999998 2024-09-13T10:10:50.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.555206+0000", 2024-09-13T10:10:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5145999999999999e-05 2024-09-13T10:10:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.555247+0000", 2024-09-13T10:10:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1241e-05 2024-09-13T10:10:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.883 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:72 4.7 4:ea79a1ec:::smithi19530714-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.550372+0000", 2024-09-13T10:10:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.19596835200000001, 2024-09-13T10:10:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049839979999999999, 2024-09-13T10:10:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:10:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:50.884 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 72 2024-09-13T10:10:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550372+0000", 2024-09-13T10:10:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550372+0000", 2024-09-13T10:10:50.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550372+0000", 2024-09-13T10:10:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6399999999999998e-07 2024-09-13T10:10:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550374+0000", 2024-09-13T10:10:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9999999999999999e-06 2024-09-13T10:10:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550379+0000", 2024-09-13T10:10:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0960000000000003e-06 2024-09-13T10:10:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.550382+0000", 2024-09-13T10:10:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7349999999999998e-06 2024-09-13T10:10:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.552346+0000", 2024-09-13T10:10:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019634689999999998 2024-09-13T10:10:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:10:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.552421+0000", 2024-09-13T10:10:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5316999999999997e-05 2024-09-13T10:10:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.555268+0000", 2024-09-13T10:10:50.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028465750000000001 2024-09-13T10:10:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.555304+0000", 2024-09-13T10:10:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6356999999999998e-05 2024-09-13T10:10:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.555356+0000", 2024-09-13T10:10:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2085e-05 2024-09-13T10:10:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.889 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:73 4.6 e38/31 4:6fb28ed4:::smithi19530714-19:head v 38'7, mlcod=38'5)", 2024-09-13T10:10:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.560184+0000", 2024-09-13T10:10:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.186156556, 2024-09-13T10:10:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046834750000000003, 2024-09-13T10:10:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.560184+0000", 2024-09-13T10:10:50.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.560184+0000", 2024-09-13T10:10:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.560185+0000", 2024-09-13T10:10:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5600000000000004e-07 2024-09-13T10:10:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.561455+0000", 2024-09-13T10:10:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012700369999999999 2024-09-13T10:10:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.561475+0000", 2024-09-13T10:10:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9681e-05 2024-09-13T10:10:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.561484+0000", 2024-09-13T10:10:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7950000000000002e-06 2024-09-13T10:10:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.561625+0000", 2024-09-13T10:10:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014052199999999999 2024-09-13T10:10:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.561685+0000", 2024-09-13T10:10:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9691000000000002e-05 2024-09-13T10:10:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:50.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.564813+0000", 2024-09-13T10:10:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031283869999999998 2024-09-13T10:10:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.564867+0000", 2024-09-13T10:10:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4406000000000001e-05 2024-09-13T10:10:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:74 4.6 e38/31 4:6fb28ed4:::smithi19530714-19:head v 38'8, mlcod=38'5)", 2024-09-13T10:10:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.564139+0000", 2024-09-13T10:10:50.894 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.182201593, 2024-09-13T10:10:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002944842, 2024-09-13T10:10:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.564139+0000", 2024-09-13T10:10:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.564139+0000", 2024-09-13T10:10:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.564140+0000", 2024-09-13T10:10:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4499999999999996e-07 2024-09-13T10:10:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.564864+0000", 2024-09-13T10:10:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00072395399999999996 2024-09-13T10:10:50.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.564892+0000", 2024-09-13T10:10:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8711999999999999e-05 2024-09-13T10:10:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.564912+0000", 2024-09-13T10:10:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9686000000000001e-05 2024-09-13T10:10:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:50.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.565004+0000", 2024-09-13T10:10:50.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2034999999999996e-05 2024-09-13T10:10:50.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:50.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.565029+0000", 2024-09-13T10:10:50.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4808e-05 2024-09-13T10:10:50.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:50.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.567030+0000", 2024-09-13T10:10:50.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020013190000000001 2024-09-13T10:10:50.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:50.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:50.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:50.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.567084+0000", 2024-09-13T10:10:50.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3483000000000003e-05 2024-09-13T10:10:50.899 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.899 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.899 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.899 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:50.899 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:50.899 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:10:50.902 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T10:10:50.906 INFO:tasks.rados.rados.0.smithi195.stdout:34: writing smithi19530714-34 from 554631 to 1103260 tid 1 2024-09-13T10:10:50.906 INFO:tasks.rados.rados.0.smithi195.stdout:34: writing smithi19530714-34 from 1748141 to 2234150 tid 2 2024-09-13T10:10:50.906 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.906 INFO:tasks.rados.rados.0.smithi195.stdout:20: finishing write tid 1 to smithi19530714-20 2024-09-13T10:10:50.906 INFO:tasks.rados.rados.0.smithi195.stdout:20: finishing write tid 2 to smithi19530714-20 2024-09-13T10:10:50.906 INFO:tasks.rados.rados.0.smithi195.stdout:20: finishing write tid 3 to smithi19530714-20 2024-09-13T10:10:50.906 INFO:tasks.rados.rados.0.smithi195.stdout:20: finishing write tid 4 to smithi19530714-20 2024-09-13T10:10:50.907 INFO:tasks.rados.rados.0.smithi195.stdout:20: oid 20 updating version 0 to 6 2024-09-13T10:10:50.907 INFO:tasks.rados.rados.0.smithi195.stdout:20: oid 20 version 6 is already newer than 5 2024-09-13T10:10:50.907 INFO:tasks.rados.rados.0.smithi195.stdout:20: oid 20 version 6 is already newer than 4 2024-09-13T10:10:50.907 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 6 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-13T10:10:50.907 INFO:tasks.rados.rados.0.smithi195.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-09-13T10:10:50.907 INFO:tasks.rados.rados.0.smithi195.stdout:21: finishing write tid 1 to smithi19530714-21 2024-09-13T10:10:50.907 INFO:tasks.rados.rados.0.smithi195.stdout:21: finishing write tid 2 to smithi19530714-21 2024-09-13T10:10:50.907 INFO:tasks.rados.rados.0.smithi195.stdout:20: done (15 left) 2024-09-13T10:10:50.907 INFO:tasks.rados.rados.0.smithi195.stdout:35: write initial oid 35 2024-09-13T10:10:50.907 INFO:tasks.rados.rados.0.smithi195.stdout:35: seq_num 34 ranges {571438=623878,1806623=629667} 2024-09-13T10:10:50.907 INFO:tasks.rados.rados.0.smithi195.stdout:35: writing smithi19530714-35 from 571438 to 1195316 tid 1 2024-09-13T10:10:50.907 INFO:tasks.rados.rados.0.smithi195.stdout:35: writing smithi19530714-35 from 1806623 to 2436290 tid 2 2024-09-13T10:10:50.907 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.908 INFO:tasks.rados.rados.0.smithi195.stdout:21: finishing write tid 3 to smithi19530714-21 2024-09-13T10:10:50.908 INFO:tasks.rados.rados.0.smithi195.stdout:21: finishing write tid 4 to smithi19530714-21 2024-09-13T10:10:50.908 INFO:tasks.rados.rados.0.smithi195.stdout:21: oid 21 updating version 0 to 7 2024-09-13T10:10:50.908 INFO:tasks.rados.rados.0.smithi195.stdout:21: oid 21 updating version 7 to 9 2024-09-13T10:10:50.908 INFO:tasks.rados.rados.0.smithi195.stdout:21: oid 21 version 9 is already newer than 8 2024-09-13T10:10:50.908 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 9 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-13T10:10:50.908 INFO:tasks.rados.rados.0.smithi195.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-09-13T10:10:50.908 INFO:tasks.rados.rados.0.smithi195.stdout:22: finishing write tid 1 to smithi19530714-22 2024-09-13T10:10:50.908 INFO:tasks.rados.rados.0.smithi195.stdout:22: finishing write tid 2 to smithi19530714-22 2024-09-13T10:10:50.908 INFO:tasks.rados.rados.0.smithi195.stdout:22: finishing write tid 3 to smithi19530714-22 2024-09-13T10:10:50.908 INFO:tasks.rados.rados.0.smithi195.stdout:22: finishing write tid 4 to smithi19530714-22 2024-09-13T10:10:50.908 INFO:tasks.rados.rados.0.smithi195.stdout:22: oid 22 updating version 0 to 12 2024-09-13T10:10:50.908 INFO:tasks.rados.rados.0.smithi195.stdout:22: oid 22 version 12 is already newer than 11 2024-09-13T10:10:50.909 INFO:tasks.rados.rados.0.smithi195.stdout:22: oid 22 version 12 is already newer than 10 2024-09-13T10:10:50.909 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 12 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-13T10:10:50.909 INFO:tasks.rados.rados.0.smithi195.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-09-13T10:10:50.909 INFO:tasks.rados.rados.0.smithi195.stdout:24: finishing write tid 1 to smithi19530714-24 2024-09-13T10:10:50.909 INFO:tasks.rados.rados.0.smithi195.stdout:24: finishing write tid 2 to smithi19530714-24 2024-09-13T10:10:50.909 INFO:tasks.rados.rados.0.smithi195.stdout:24: finishing write tid 3 to smithi19530714-24 2024-09-13T10:10:50.909 INFO:tasks.rados.rados.0.smithi195.stdout:24: finishing write tid 4 to smithi19530714-24 2024-09-13T10:10:50.909 INFO:tasks.rados.rados.0.smithi195.stdout:24: oid 24 updating version 0 to 2 2024-09-13T10:10:50.909 INFO:tasks.rados.rados.0.smithi195.stdout:24: oid 24 version 2 is already newer than 1 2024-09-13T10:10:50.909 INFO:tasks.rados.rados.0.smithi195.stdout:24: oid 24 updating version 2 to 3 2024-09-13T10:10:50.909 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 3 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-13T10:10:50.909 INFO:tasks.rados.rados.0.smithi195.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-09-13T10:10:50.909 INFO:tasks.rados.rados.0.smithi195.stdout:25: finishing write tid 1 to smithi19530714-25 2024-09-13T10:10:50.910 INFO:tasks.rados.rados.0.smithi195.stdout:25: finishing write tid 2 to smithi19530714-25 2024-09-13T10:10:50.910 INFO:tasks.rados.rados.0.smithi195.stdout:25: finishing write tid 3 to smithi19530714-25 2024-09-13T10:10:50.910 INFO:tasks.rados.rados.0.smithi195.stdout:25: finishing write tid 4 to smithi19530714-25 2024-09-13T10:10:50.910 INFO:tasks.rados.rados.0.smithi195.stdout:25: oid 25 updating version 0 to 5 2024-09-13T10:10:50.910 INFO:tasks.rados.rados.0.smithi195.stdout:25: oid 25 version 5 is already newer than 4 2024-09-13T10:10:50.910 INFO:tasks.rados.rados.0.smithi195.stdout:25: oid 25 updating version 5 to 6 2024-09-13T10:10:50.910 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 6 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-13T10:10:50.910 INFO:tasks.rados.rados.0.smithi195.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-09-13T10:10:50.910 INFO:tasks.rados.rados.0.smithi195.stdout:21: done (15 left) 2024-09-13T10:10:50.910 INFO:tasks.rados.rados.0.smithi195.stdout:22: done (14 left) 2024-09-13T10:10:50.910 INFO:tasks.rados.rados.0.smithi195.stdout:24: done (13 left) 2024-09-13T10:10:50.910 INFO:tasks.rados.rados.0.smithi195.stdout:25: done (12 left) 2024-09-13T10:10:50.910 INFO:tasks.rados.rados.0.smithi195.stdout:36: write initial oid 36 2024-09-13T10:10:50.910 INFO:tasks.rados.rados.0.smithi195.stdout:36: seq_num 35 ranges {588245=699127,1865105=489678} 2024-09-13T10:10:50.911 INFO:tasks.rados.rados.0.smithi195.stdout:36: writing smithi19530714-36 from 588245 to 1287372 tid 1 2024-09-13T10:10:50.911 INFO:tasks.rados.rados.0.smithi195.stdout:36: writing smithi19530714-36 from 1865105 to 2354783 tid 2 2024-09-13T10:10:50.911 INFO:tasks.rados.rados.0.smithi195.stdout:37: write initial oid 37 2024-09-13T10:10:50.911 INFO:tasks.rados.rados.0.smithi195.stdout:37: seq_num 36 ranges {605052=774376,1923587=633336} 2024-09-13T10:10:50.911 INFO:tasks.rados.rados.0.smithi195.stdout:37: writing smithi19530714-37 from 605052 to 1379428 tid 1 2024-09-13T10:10:50.911 INFO:tasks.rados.rados.0.smithi195.stdout:37: writing smithi19530714-37 from 1923587 to 2556923 tid 2 2024-09-13T10:10:50.911 INFO:tasks.rados.rados.0.smithi195.stdout:38: write initial oid 38 2024-09-13T10:10:50.911 INFO:tasks.rados.rados.0.smithi195.stdout:38: seq_num 37 ranges {621859=449625,1865716=756143} 2024-09-13T10:10:50.911 INFO:tasks.rados.rados.0.smithi195.stdout:38: writing smithi19530714-38 from 621859 to 1071484 tid 1 2024-09-13T10:10:50.911 INFO:tasks.rados.rados.0.smithi195.stdout:38: writing smithi19530714-38 from 1865716 to 2621859 tid 2 2024-09-13T10:10:50.911 INFO:tasks.rados.rados.0.smithi195.stdout:39: write initial oid 39 2024-09-13T10:10:50.911 INFO:tasks.rados.rados.0.smithi195.stdout:39: seq_num 38 ranges {638666=524874,1924198=637005} 2024-09-13T10:10:50.911 INFO:tasks.rados.rados.0.smithi195.stdout:39: writing smithi19530714-39 from 638666 to 1163540 tid 1 2024-09-13T10:10:50.912 INFO:tasks.rados.rados.0.smithi195.stdout:39: writing smithi19530714-39 from 1924198 to 2561203 tid 2 2024-09-13T10:10:50.912 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.912 INFO:tasks.rados.rados.0.smithi195.stdout:26: finishing write tid 1 to smithi19530714-26 2024-09-13T10:10:50.912 INFO:tasks.rados.rados.0.smithi195.stdout:26: finishing write tid 2 to smithi19530714-26 2024-09-13T10:10:50.912 INFO:tasks.rados.rados.0.smithi195.stdout:26: finishing write tid 3 to smithi19530714-26 2024-09-13T10:10:50.912 INFO:tasks.rados.rados.0.smithi195.stdout:26: finishing write tid 4 to smithi19530714-26 2024-09-13T10:10:50.912 INFO:tasks.rados.rados.0.smithi195.stdout:26: oid 26 updating version 0 to 3 2024-09-13T10:10:50.912 INFO:tasks.rados.rados.0.smithi195.stdout:26: oid 26 version 3 is already newer than 2 2024-09-13T10:10:50.912 INFO:tasks.rados.rados.0.smithi195.stdout:26: oid 26 version 3 is already newer than 1 2024-09-13T10:10:50.912 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 3 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-13T10:10:50.912 INFO:tasks.rados.rados.0.smithi195.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-09-13T10:10:50.912 INFO:tasks.rados.rados.0.smithi195.stdout:27: finishing write tid 1 to smithi19530714-27 2024-09-13T10:10:50.912 INFO:tasks.rados.rados.0.smithi195.stdout:26: done (15 left) 2024-09-13T10:10:50.913 INFO:tasks.rados.rados.0.smithi195.stdout:40: write initial oid 40 2024-09-13T10:10:50.913 INFO:tasks.rados.rados.0.smithi195.stdout:40: seq_num 39 ranges {655473=716476,2099033=556440} 2024-09-13T10:10:50.913 INFO:tasks.rados.rados.0.smithi195.stdout:40: writing smithi19530714-40 from 655473 to 1371949 tid 1 2024-09-13T10:10:50.913 INFO:tasks.rados.rados.0.smithi195.stdout:40: writing smithi19530714-40 from 2099033 to 2655473 tid 2 2024-09-13T10:10:50.913 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.913 INFO:tasks.rados.rados.0.smithi195.stdout:27: finishing write tid 2 to smithi19530714-27 2024-09-13T10:10:50.913 INFO:tasks.rados.rados.0.smithi195.stdout:27: finishing write tid 3 to smithi19530714-27 2024-09-13T10:10:50.913 INFO:tasks.rados.rados.0.smithi195.stdout:27: finishing write tid 4 to smithi19530714-27 2024-09-13T10:10:50.913 INFO:tasks.rados.rados.0.smithi195.stdout:27: oid 27 updating version 0 to 15 2024-09-13T10:10:50.913 INFO:tasks.rados.rados.0.smithi195.stdout:27: oid 27 version 15 is already newer than 13 2024-09-13T10:10:50.913 INFO:tasks.rados.rados.0.smithi195.stdout:27: oid 27 version 15 is already newer than 14 2024-09-13T10:10:50.913 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 15 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-13T10:10:50.913 INFO:tasks.rados.rados.0.smithi195.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-09-13T10:10:50.914 INFO:tasks.rados.rados.0.smithi195.stdout:28: finishing write tid 1 to smithi19530714-28 2024-09-13T10:10:50.914 INFO:tasks.rados.rados.0.smithi195.stdout:28: finishing write tid 2 to smithi19530714-28 2024-09-13T10:10:50.914 INFO:tasks.rados.rados.0.smithi195.stdout:28: finishing write tid 3 to smithi19530714-28 2024-09-13T10:10:50.914 INFO:tasks.rados.rados.0.smithi195.stdout:28: finishing write tid 4 to smithi19530714-28 2024-09-13T10:10:50.914 INFO:tasks.rados.rados.0.smithi195.stdout:28: oid 28 updating version 0 to 7 2024-09-13T10:10:50.914 INFO:tasks.rados.rados.0.smithi195.stdout:28: oid 28 updating version 7 to 8 2024-09-13T10:10:50.914 INFO:tasks.rados.rados.0.smithi195.stdout:28: oid 28 updating version 8 to 9 2024-09-13T10:10:50.914 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 9 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-13T10:10:50.914 INFO:tasks.rados.rados.0.smithi195.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-09-13T10:10:50.914 INFO:tasks.rados.rados.0.smithi195.stdout:29: finishing write tid 1 to smithi19530714-29 2024-09-13T10:10:50.914 INFO:tasks.rados.rados.0.smithi195.stdout:27: done (15 left) 2024-09-13T10:10:50.914 INFO:tasks.rados.rados.0.smithi195.stdout:28: done (14 left) 2024-09-13T10:10:50.915 INFO:tasks.rados.rados.0.smithi195.stdout:41: write initial oid 41 2024-09-13T10:10:50.915 INFO:tasks.rados.rados.0.smithi195.stdout:41: seq_num 40 ranges {672280=791725,2157515=514765} 2024-09-13T10:10:50.915 INFO:tasks.rados.rados.0.smithi195.stdout:41: writing smithi19530714-41 from 672280 to 1464005 tid 1 2024-09-13T10:10:50.915 INFO:tasks.rados.rados.0.smithi195.stdout:41: writing smithi19530714-41 from 2157515 to 2672280 tid 2 2024-09-13T10:10:50.915 INFO:tasks.rados.rados.0.smithi195.stdout:42: write initial oid 42 2024-09-13T10:10:50.915 INFO:tasks.rados.rados.0.smithi195.stdout:42: seq_num 41 ranges {689087=466974,1699644=784332} 2024-09-13T10:10:50.915 INFO:tasks.rados.rados.0.smithi195.stdout:42: writing smithi19530714-42 from 689087 to 1156061 tid 1 2024-09-13T10:10:50.915 INFO:tasks.rados.rados.0.smithi195.stdout:42: writing smithi19530714-42 from 1699644 to 2483976 tid 2 2024-09-13T10:10:50.915 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.915 INFO:tasks.rados.rados.0.smithi195.stdout:29: finishing write tid 2 to smithi19530714-29 2024-09-13T10:10:50.915 INFO:tasks.rados.rados.0.smithi195.stdout:29: finishing write tid 3 to smithi19530714-29 2024-09-13T10:10:50.915 INFO:tasks.rados.rados.0.smithi195.stdout:29: finishing write tid 4 to smithi19530714-29 2024-09-13T10:10:50.915 INFO:tasks.rados.rados.0.smithi195.stdout:29: oid 29 updating version 0 to 12 2024-09-13T10:10:50.915 INFO:tasks.rados.rados.0.smithi195.stdout:29: oid 29 version 12 is already newer than 10 2024-09-13T10:10:50.916 INFO:tasks.rados.rados.0.smithi195.stdout:29: oid 29 version 12 is already newer than 11 2024-09-13T10:10:50.916 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 12 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-13T10:10:50.916 INFO:tasks.rados.rados.0.smithi195.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-09-13T10:10:50.916 INFO:tasks.rados.rados.0.smithi195.stdout:30: finishing write tid 1 to smithi19530714-30 2024-09-13T10:10:50.916 INFO:tasks.rados.rados.0.smithi195.stdout:30: finishing write tid 2 to smithi19530714-30 2024-09-13T10:10:50.916 INFO:tasks.rados.rados.0.smithi195.stdout:30: finishing write tid 3 to smithi19530714-30 2024-09-13T10:10:50.916 INFO:tasks.rados.rados.0.smithi195.stdout:30: finishing write tid 4 to smithi19530714-30 2024-09-13T10:10:50.916 INFO:tasks.rados.rados.0.smithi195.stdout:30: oid 30 updating version 0 to 17 2024-09-13T10:10:50.916 INFO:tasks.rados.rados.0.smithi195.stdout:30: oid 30 updating version 17 to 18 2024-09-13T10:10:50.916 INFO:tasks.rados.rados.0.smithi195.stdout:30: oid 30 version 18 is already newer than 16 2024-09-13T10:10:50.916 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 18 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-13T10:10:50.916 INFO:tasks.rados.rados.0.smithi195.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-09-13T10:10:50.916 INFO:tasks.rados.rados.0.smithi195.stdout:29: done (15 left) 2024-09-13T10:10:50.917 INFO:tasks.rados.rados.0.smithi195.stdout:30: done (14 left) 2024-09-13T10:10:50.917 INFO:tasks.rados.rados.0.smithi195.stdout:43: write initial oid 43 2024-09-13T10:10:50.917 INFO:tasks.rados.rados.0.smithi195.stdout:43: seq_num 42 ranges {705894=542223,1758126=644343} 2024-09-13T10:10:50.917 INFO:tasks.rados.rados.0.smithi195.stdout:43: writing smithi19530714-43 from 705894 to 1248117 tid 1 2024-09-13T10:10:50.917 INFO:tasks.rados.rados.0.smithi195.stdout:43: writing smithi19530714-43 from 1758126 to 2402469 tid 2 2024-09-13T10:10:50.917 INFO:tasks.rados.rados.0.smithi195.stdout:44: write initial oid 44 2024-09-13T10:10:50.917 INFO:tasks.rados.rados.0.smithi195.stdout:44: seq_num 43 ranges {722701=617472,1816608=788001} 2024-09-13T10:10:50.917 INFO:tasks.rados.rados.0.smithi195.stdout:44: writing smithi19530714-44 from 722701 to 1340173 tid 1 2024-09-13T10:10:50.917 INFO:tasks.rados.rados.0.smithi195.stdout:44: writing smithi19530714-44 from 1816608 to 2604609 tid 2 2024-09-13T10:10:50.917 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:50.899+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:50.918 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.918 INFO:tasks.rados.rados.0.smithi195.stdout:31: finishing write tid 1 to smithi19530714-31 2024-09-13T10:10:50.919 INFO:tasks.rados.rados.0.smithi195.stdout:31: finishing write tid 2 to smithi19530714-31 2024-09-13T10:10:50.919 INFO:tasks.rados.rados.0.smithi195.stdout:31: finishing write tid 3 to smithi19530714-31 2024-09-13T10:10:50.919 INFO:tasks.rados.rados.0.smithi195.stdout:31: finishing write tid 4 to smithi19530714-31 2024-09-13T10:10:50.919 INFO:tasks.rados.rados.0.smithi195.stdout:31: oid 31 updating version 0 to 6 2024-09-13T10:10:50.919 INFO:tasks.rados.rados.0.smithi195.stdout:31: oid 31 version 6 is already newer than 5 2024-09-13T10:10:50.919 INFO:tasks.rados.rados.0.smithi195.stdout:31: oid 31 version 6 is already newer than 4 2024-09-13T10:10:50.919 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 6 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-13T10:10:50.919 INFO:tasks.rados.rados.0.smithi195.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-09-13T10:10:50.919 INFO:tasks.rados.rados.0.smithi195.stdout:16: finishing write tid 1 to smithi19530714-16 2024-09-13T10:10:50.919 INFO:tasks.rados.rados.0.smithi195.stdout:16: finishing write tid 2 to smithi19530714-16 2024-09-13T10:10:50.919 INFO:tasks.rados.rados.0.smithi195.stdout:16: finishing write tid 3 to smithi19530714-16 2024-09-13T10:10:50.919 INFO:tasks.rados.rados.0.smithi195.stdout:16: finishing write tid 4 to smithi19530714-16 2024-09-13T10:10:50.919 INFO:tasks.rados.rados.0.smithi195.stdout:16: oid 16 updating version 0 to 1 2024-09-13T10:10:50.920 INFO:tasks.rados.rados.0.smithi195.stdout:16: oid 16 updating version 1 to 2 2024-09-13T10:10:50.920 INFO:tasks.rados.rados.0.smithi195.stdout:16: oid 16 updating version 2 to 3 2024-09-13T10:10:50.920 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 3 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-13T10:10:50.920 INFO:tasks.rados.rados.0.smithi195.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-09-13T10:10:50.920 INFO:tasks.rados.rados.0.smithi195.stdout:32: finishing write tid 1 to smithi19530714-32 2024-09-13T10:10:50.920 INFO:tasks.rados.rados.0.smithi195.stdout:32: finishing write tid 2 to smithi19530714-32 2024-09-13T10:10:50.920 INFO:tasks.rados.rados.0.smithi195.stdout:32: finishing write tid 3 to smithi19530714-32 2024-09-13T10:10:50.920 INFO:tasks.rados.rados.0.smithi195.stdout:32: finishing write tid 4 to smithi19530714-32 2024-09-13T10:10:50.920 INFO:tasks.rados.rados.0.smithi195.stdout:32: oid 32 updating version 0 to 14 2024-09-13T10:10:50.920 INFO:tasks.rados.rados.0.smithi195.stdout:32: oid 32 version 14 is already newer than 13 2024-09-13T10:10:50.920 INFO:tasks.rados.rados.0.smithi195.stdout:32: oid 32 updating version 14 to 15 2024-09-13T10:10:50.920 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 15 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-13T10:10:50.920 INFO:tasks.rados.rados.0.smithi195.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-09-13T10:10:50.921 INFO:tasks.rados.rados.0.smithi195.stdout:34: finishing write tid 1 to smithi19530714-34 2024-09-13T10:10:50.921 INFO:tasks.rados.rados.0.smithi195.stdout:34: finishing write tid 2 to smithi19530714-34 2024-09-13T10:10:50.921 INFO:tasks.rados.rados.0.smithi195.stdout:16: done (15 left) 2024-09-13T10:10:50.921 INFO:tasks.rados.rados.0.smithi195.stdout:31: done (14 left) 2024-09-13T10:10:50.921 INFO:tasks.rados.rados.0.smithi195.stdout:32: done (13 left) 2024-09-13T10:10:50.921 INFO:tasks.rados.rados.0.smithi195.stdout:45: write initial oid 45 2024-09-13T10:10:50.921 INFO:tasks.rados.rados.0.smithi195.stdout:45: seq_num 44 ranges {739508=692721,1875090=648012} 2024-09-13T10:10:50.921 INFO:tasks.rados.rados.0.smithi195.stdout:45: writing smithi19530714-45 from 739508 to 1432229 tid 1 2024-09-13T10:10:50.921 INFO:tasks.rados.rados.0.smithi195.stdout:45: writing smithi19530714-45 from 1875090 to 2523102 tid 2 2024-09-13T10:10:50.921 INFO:tasks.rados.rados.0.smithi195.stdout:46: write initial oid 46 2024-09-13T10:10:50.921 INFO:tasks.rados.rados.0.smithi195.stdout:46: seq_num 45 ranges {756315=767970,1933572=791670} 2024-09-13T10:10:50.923 INFO:tasks.rados.rados.0.smithi195.stdout:46: writing smithi19530714-46 from 756315 to 1524285 tid 1 2024-09-13T10:10:50.930 INFO:tasks.rados.rados.0.smithi195.stdout:46: writing smithi19530714-46 from 1933572 to 2725242 tid 2 2024-09-13T10:10:50.931 INFO:tasks.rados.rados.0.smithi195.stdout:47: write initial oid 47 2024-09-13T10:10:50.931 INFO:tasks.rados.rados.0.smithi195.stdout:34: finishing write tid 3 to smithi19530714-34 2024-09-13T10:10:50.931 INFO:tasks.rados.rados.0.smithi195.stdout:47: seq_num 46 ranges {773122=559572,1992054=651681} 2024-09-13T10:10:50.938 INFO:tasks.rados.rados.0.smithi195.stdout:47: writing smithi19530714-47 from 773122 to 1332694 tid 1 2024-09-13T10:10:50.946 INFO:tasks.rados.rados.0.smithi195.stdout:47: writing smithi19530714-47 from 1992054 to 2643735 tid 2 2024-09-13T10:10:50.946 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.946 INFO:tasks.rados.rados.0.smithi195.stdout:34: finishing write tid 4 to smithi19530714-34 2024-09-13T10:10:50.946 INFO:tasks.rados.rados.0.smithi195.stdout:34: oid 34 updating version 0 to 4 2024-09-13T10:10:50.946 INFO:tasks.rados.rados.0.smithi195.stdout:34: oid 34 updating version 4 to 5 2024-09-13T10:10:50.947 INFO:tasks.rados.rados.0.smithi195.stdout:34: oid 34 updating version 5 to 6 2024-09-13T10:10:50.947 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 34 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-13T10:10:50.947 INFO:tasks.rados.rados.0.smithi195.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-09-13T10:10:50.947 INFO:tasks.rados.rados.0.smithi195.stdout:35: finishing write tid 1 to smithi19530714-35 2024-09-13T10:10:50.947 INFO:tasks.rados.rados.0.smithi195.stdout:35: finishing write tid 2 to smithi19530714-35 2024-09-13T10:10:50.947 INFO:tasks.rados.rados.0.smithi195.stdout:35: finishing write tid 3 to smithi19530714-35 2024-09-13T10:10:50.947 INFO:tasks.rados.rados.0.smithi195.stdout:35: finishing write tid 4 to smithi19530714-35 2024-09-13T10:10:50.947 INFO:tasks.rados.rados.0.smithi195.stdout:35: oid 35 updating version 0 to 12 2024-09-13T10:10:50.947 INFO:tasks.rados.rados.0.smithi195.stdout:35: oid 35 version 12 is already newer than 10 2024-09-13T10:10:50.947 INFO:tasks.rados.rados.0.smithi195.stdout:35: oid 35 version 12 is already newer than 11 2024-09-13T10:10:50.947 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 12 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-13T10:10:50.947 INFO:tasks.rados.rados.0.smithi195.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-09-13T10:10:50.947 INFO:tasks.rados.rados.0.smithi195.stdout:36: finishing write tid 1 to smithi19530714-36 2024-09-13T10:10:50.948 INFO:tasks.rados.rados.0.smithi195.stdout:36: finishing write tid 2 to smithi19530714-36 2024-09-13T10:10:50.948 INFO:tasks.rados.rados.0.smithi195.stdout:36: finishing write tid 3 to smithi19530714-36 2024-09-13T10:10:50.948 INFO:tasks.rados.rados.0.smithi195.stdout:36: finishing write tid 4 to smithi19530714-36 2024-09-13T10:10:50.948 INFO:tasks.rados.rados.0.smithi195.stdout:36: oid 36 updating version 0 to 6 2024-09-13T10:10:50.948 INFO:tasks.rados.rados.0.smithi195.stdout:36: oid 36 version 6 is already newer than 5 2024-09-13T10:10:50.948 INFO:tasks.rados.rados.0.smithi195.stdout:36: oid 36 version 6 is already newer than 4 2024-09-13T10:10:50.948 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 6 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-13T10:10:50.948 INFO:tasks.rados.rados.0.smithi195.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-09-13T10:10:50.948 INFO:tasks.rados.rados.0.smithi195.stdout:37: finishing write tid 1 to smithi19530714-37 2024-09-13T10:10:50.948 INFO:tasks.rados.rados.0.smithi195.stdout:37: finishing write tid 2 to smithi19530714-37 2024-09-13T10:10:50.948 INFO:tasks.rados.rados.0.smithi195.stdout:37: finishing write tid 3 to smithi19530714-37 2024-09-13T10:10:50.948 INFO:tasks.rados.rados.0.smithi195.stdout:37: finishing write tid 4 to smithi19530714-37 2024-09-13T10:10:50.948 INFO:tasks.rados.rados.0.smithi195.stdout:37: oid 37 updating version 0 to 2 2024-09-13T10:10:50.948 INFO:tasks.rados.rados.0.smithi195.stdout:37: oid 37 updating version 2 to 3 2024-09-13T10:10:50.949 INFO:tasks.rados.rados.0.smithi195.stdout:37: oid 37 version 3 is already newer than 1 2024-09-13T10:10:50.949 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 3 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-13T10:10:50.949 INFO:tasks.rados.rados.0.smithi195.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-09-13T10:10:50.949 INFO:tasks.rados.rados.0.smithi195.stdout:34: done (15 left) 2024-09-13T10:10:50.949 INFO:tasks.rados.rados.0.smithi195.stdout:35: done (14 left) 2024-09-13T10:10:50.949 INFO:tasks.rados.rados.0.smithi195.stdout:36: done (13 left) 2024-09-13T10:10:50.949 INFO:tasks.rados.rados.0.smithi195.stdout:37: done (12 left) 2024-09-13T10:10:50.949 INFO:tasks.rados.rados.0.smithi195.stdout:48: write initial oid 48 2024-09-13T10:10:50.949 INFO:tasks.rados.rados.0.smithi195.stdout:48: seq_num 47 ranges {789929=634821,2050536=739393} 2024-09-13T10:10:50.954 INFO:tasks.rados.rados.0.smithi195.stdout:48: writing smithi19530714-48 from 789929 to 1424750 tid 1 2024-09-13T10:10:50.962 INFO:tasks.rados.rados.0.smithi195.stdout:48: writing smithi19530714-48 from 2050536 to 2789929 tid 2 2024-09-13T10:10:50.962 INFO:tasks.rados.rados.0.smithi195.stdout:49: write initial oid 49 2024-09-13T10:10:50.962 INFO:tasks.rados.rados.0.smithi195.stdout:49: seq_num 48 ranges {406736=710070,1709018=655350} 2024-09-13T10:10:50.968 INFO:tasks.rados.rados.0.smithi195.stdout:49: writing smithi19530714-49 from 406736 to 1116806 tid 1 2024-09-13T10:10:50.975 INFO:tasks.rados.rados.0.smithi195.stdout:49: writing smithi19530714-49 from 1709018 to 2364368 tid 2 2024-09-13T10:10:50.975 INFO:tasks.rados.rados.0.smithi195.stdout:50: write initial oid 50 2024-09-13T10:10:50.975 INFO:tasks.rados.rados.0.smithi195.stdout:50: seq_num 49 ranges {423543=785319,1767500=799008} 2024-09-13T10:10:50.982 INFO:tasks.rados.rados.0.smithi195.stdout:50: writing smithi19530714-50 from 423543 to 1208862 tid 1 2024-09-13T10:10:50.992 INFO:tasks.rados.rados.0.smithi195.stdout:50: writing smithi19530714-50 from 1767500 to 2566508 tid 2 2024-09-13T10:10:50.993 INFO:tasks.rados.rados.0.smithi195.stdout:51: copy oid 2 target oid 2 low_tier 2024-09-13T10:10:50.993 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 2 to 2 low_tier tgt_pool_name low_tier 2024-09-13T10:10:50.993 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.993 INFO:tasks.rados.rados.0.smithi195.stdout:38: finishing write tid 1 to smithi19530714-38 2024-09-13T10:10:50.993 INFO:tasks.rados.rados.0.smithi195.stdout:23: finishing write tid 1 to smithi19530714-23 2024-09-13T10:10:50.993 INFO:tasks.rados.rados.0.smithi195.stdout:23: finishing write tid 2 to smithi19530714-23 2024-09-13T10:10:50.993 INFO:tasks.rados.rados.0.smithi195.stdout:23: finishing write tid 3 to smithi19530714-23 2024-09-13T10:10:50.993 INFO:tasks.rados.rados.0.smithi195.stdout:38: finishing write tid 2 to smithi19530714-38 2024-09-13T10:10:50.993 INFO:tasks.rados.rados.0.smithi195.stdout:38: finishing write tid 3 to smithi19530714-38 2024-09-13T10:10:50.993 INFO:tasks.rados.rados.0.smithi195.stdout:23: finishing write tid 4 to smithi19530714-23 2024-09-13T10:10:50.993 INFO:tasks.rados.rados.0.smithi195.stdout:23: oid 23 updating version 0 to 1 2024-09-13T10:10:50.993 INFO:tasks.rados.rados.0.smithi195.stdout:23: oid 23 updating version 1 to 2 2024-09-13T10:10:50.994 INFO:tasks.rados.rados.0.smithi195.stdout:23: oid 23 updating version 2 to 3 2024-09-13T10:10:50.994 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 3 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-13T10:10:50.994 INFO:tasks.rados.rados.0.smithi195.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-09-13T10:10:50.994 INFO:tasks.rados.rados.0.smithi195.stdout:38: finishing write tid 4 to smithi19530714-38 2024-09-13T10:10:50.994 INFO:tasks.rados.rados.0.smithi195.stdout:38: oid 38 updating version 0 to 10 2024-09-13T10:10:50.994 INFO:tasks.rados.rados.0.smithi195.stdout:38: oid 38 updating version 10 to 11 2024-09-13T10:10:50.994 INFO:tasks.rados.rados.0.smithi195.stdout:38: oid 38 updating version 11 to 12 2024-09-13T10:10:50.994 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 12 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-13T10:10:50.994 INFO:tasks.rados.rados.0.smithi195.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-09-13T10:10:50.994 INFO:tasks.rados.rados.0.smithi195.stdout:23: done (15 left) 2024-09-13T10:10:50.994 INFO:tasks.rados.rados.0.smithi195.stdout:38: done (14 left) 2024-09-13T10:10:50.994 INFO:tasks.rados.rados.0.smithi195.stdout:52: copy oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:50.994 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 3 to 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:50.995 INFO:tasks.rados.rados.0.smithi195.stdout:53: copy oid 4 target oid 4 low_tier 2024-09-13T10:10:50.995 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 4 to 4 low_tier tgt_pool_name low_tier 2024-09-13T10:10:50.995 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.995 INFO:tasks.rados.rados.0.smithi195.stdout:33: finishing write tid 1 to smithi19530714-33 2024-09-13T10:10:50.995 INFO:tasks.rados.rados.0.smithi195.stdout:33: finishing write tid 2 to smithi19530714-33 2024-09-13T10:10:50.995 INFO:tasks.rados.rados.0.smithi195.stdout:33: finishing write tid 3 to smithi19530714-33 2024-09-13T10:10:50.995 INFO:tasks.rados.rados.0.smithi195.stdout:33: finishing write tid 4 to smithi19530714-33 2024-09-13T10:10:50.995 INFO:tasks.rados.rados.0.smithi195.stdout:33: oid 33 updating version 0 to 6 2024-09-13T10:10:50.995 INFO:tasks.rados.rados.0.smithi195.stdout:33: oid 33 version 6 is already newer than 4 2024-09-13T10:10:50.995 INFO:tasks.rados.rados.0.smithi195.stdout:33: oid 33 version 6 is already newer than 5 2024-09-13T10:10:50.995 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 6 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-13T10:10:50.995 INFO:tasks.rados.rados.0.smithi195.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-09-13T10:10:50.995 INFO:tasks.rados.rados.0.smithi195.stdout:39: finishing write tid 1 to smithi19530714-39 2024-09-13T10:10:50.995 INFO:tasks.rados.rados.0.smithi195.stdout:39: finishing write tid 2 to smithi19530714-39 2024-09-13T10:10:50.996 INFO:tasks.rados.rados.0.smithi195.stdout:39: finishing write tid 3 to smithi19530714-39 2024-09-13T10:10:50.996 INFO:tasks.rados.rados.0.smithi195.stdout:33: done (15 left) 2024-09-13T10:10:50.996 INFO:tasks.rados.rados.0.smithi195.stdout:54: copy oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:50.996 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 5 to 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:50.996 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.996 INFO:tasks.rados.rados.0.smithi195.stdout:39: finishing write tid 4 to smithi19530714-39 2024-09-13T10:10:50.996 INFO:tasks.rados.rados.0.smithi195.stdout:39: oid 39 updating version 0 to 7 2024-09-13T10:10:50.996 INFO:tasks.rados.rados.0.smithi195.stdout:39: oid 39 updating version 7 to 9 2024-09-13T10:10:50.996 INFO:tasks.rados.rados.0.smithi195.stdout:39: oid 39 version 9 is already newer than 8 2024-09-13T10:10:50.996 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 9 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-13T10:10:50.996 INFO:tasks.rados.rados.0.smithi195.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-09-13T10:10:50.996 INFO:tasks.rados.rados.0.smithi195.stdout:39: done (15 left) 2024-09-13T10:10:50.996 INFO:tasks.rados.rados.0.smithi195.stdout:55: copy oid 6 target oid 6 low_tier 2024-09-13T10:10:50.997 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 6 to 6 low_tier tgt_pool_name low_tier 2024-09-13T10:10:50.997 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.997 INFO:tasks.rados.rados.0.smithi195.stdout:40: finishing write tid 1 to smithi19530714-40 2024-09-13T10:10:50.997 INFO:tasks.rados.rados.0.smithi195.stdout:40: finishing write tid 2 to smithi19530714-40 2024-09-13T10:10:50.997 INFO:tasks.rados.rados.0.smithi195.stdout:40: finishing write tid 3 to smithi19530714-40 2024-09-13T10:10:50.997 INFO:tasks.rados.rados.0.smithi195.stdout:40: finishing write tid 4 to smithi19530714-40 2024-09-13T10:10:50.997 INFO:tasks.rados.rados.0.smithi195.stdout:40: oid 40 updating version 0 to 4 2024-09-13T10:10:50.997 INFO:tasks.rados.rados.0.smithi195.stdout:40: oid 40 updating version 4 to 5 2024-09-13T10:10:50.997 INFO:tasks.rados.rados.0.smithi195.stdout:40: oid 40 updating version 5 to 6 2024-09-13T10:10:50.997 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-13T10:10:50.997 INFO:tasks.rados.rados.0.smithi195.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-09-13T10:10:50.997 INFO:tasks.rados.rados.0.smithi195.stdout:41: finishing write tid 1 to smithi19530714-41 2024-09-13T10:10:50.997 INFO:tasks.rados.rados.0.smithi195.stdout:41: finishing write tid 2 to smithi19530714-41 2024-09-13T10:10:50.998 INFO:tasks.rados.rados.0.smithi195.stdout:41: finishing write tid 3 to smithi19530714-41 2024-09-13T10:10:50.998 INFO:tasks.rados.rados.0.smithi195.stdout:41: finishing write tid 4 to smithi19530714-41 2024-09-13T10:10:50.998 INFO:tasks.rados.rados.0.smithi195.stdout:41: oid 41 updating version 0 to 1 2024-09-13T10:10:50.998 INFO:tasks.rados.rados.0.smithi195.stdout:41: oid 41 updating version 1 to 2 2024-09-13T10:10:50.998 INFO:tasks.rados.rados.0.smithi195.stdout:41: oid 41 updating version 2 to 3 2024-09-13T10:10:50.998 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 3 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-13T10:10:50.998 INFO:tasks.rados.rados.0.smithi195.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-09-13T10:10:50.998 INFO:tasks.rados.rados.0.smithi195.stdout:40: done (15 left) 2024-09-13T10:10:50.998 INFO:tasks.rados.rados.0.smithi195.stdout:41: done (14 left) 2024-09-13T10:10:50.998 INFO:tasks.rados.rados.0.smithi195.stdout:56: copy oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:50.998 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 7 to 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:50.998 INFO:tasks.rados.rados.0.smithi195.stdout:57: copy oid 8 target oid 8 low_tier 2024-09-13T10:10:50.998 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 8 to 8 low_tier tgt_pool_name low_tier 2024-09-13T10:10:50.999 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:50.999 INFO:tasks.rados.rados.0.smithi195.stdout:42: finishing write tid 1 to smithi19530714-42 2024-09-13T10:10:50.999 INFO:tasks.rados.rados.0.smithi195.stdout:42: finishing write tid 2 to smithi19530714-42 2024-09-13T10:10:50.999 INFO:tasks.rados.rados.0.smithi195.stdout:42: finishing write tid 3 to smithi19530714-42 2024-09-13T10:10:50.999 INFO:tasks.rados.rados.0.smithi195.stdout:42: finishing write tid 4 to smithi19530714-42 2024-09-13T10:10:50.999 INFO:tasks.rados.rados.0.smithi195.stdout:42: oid 42 updating version 0 to 5 2024-09-13T10:10:50.999 INFO:tasks.rados.rados.0.smithi195.stdout:42: oid 42 version 5 is already newer than 4 2024-09-13T10:10:50.999 INFO:tasks.rados.rados.0.smithi195.stdout:42: oid 42 updating version 5 to 6 2024-09-13T10:10:50.999 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 6 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-13T10:10:50.999 INFO:tasks.rados.rados.0.smithi195.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-09-13T10:10:50.999 INFO:tasks.rados.rados.0.smithi195.stdout:42: done (15 left) 2024-09-13T10:10:50.999 INFO:tasks.rados.rados.0.smithi195.stdout:58: copy oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:50.999 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 9 to 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:50.999 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.000 INFO:tasks.rados.rados.0.smithi195.stdout:43: finishing write tid 1 to smithi19530714-43 2024-09-13T10:10:51.000 INFO:tasks.rados.rados.0.smithi195.stdout:43: finishing write tid 2 to smithi19530714-43 2024-09-13T10:10:51.000 INFO:tasks.rados.rados.0.smithi195.stdout:43: finishing write tid 3 to smithi19530714-43 2024-09-13T10:10:51.000 INFO:tasks.rados.rados.0.smithi195.stdout:43: finishing write tid 4 to smithi19530714-43 2024-09-13T10:10:51.000 INFO:tasks.rados.rados.0.smithi195.stdout:43: oid 43 updating version 0 to 8 2024-09-13T10:10:51.000 INFO:tasks.rados.rados.0.smithi195.stdout:43: oid 43 updating version 8 to 9 2024-09-13T10:10:51.000 INFO:tasks.rados.rados.0.smithi195.stdout:43: oid 43 version 9 is already newer than 7 2024-09-13T10:10:51.000 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 9 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-13T10:10:51.000 INFO:tasks.rados.rados.0.smithi195.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-09-13T10:10:51.000 INFO:tasks.rados.rados.0.smithi195.stdout:44: finishing write tid 1 to smithi19530714-44 2024-09-13T10:10:51.000 INFO:tasks.rados.rados.0.smithi195.stdout:43: done (15 left) 2024-09-13T10:10:51.000 INFO:tasks.rados.rados.0.smithi195.stdout:59: copy oid 10 target oid 10 low_tier 2024-09-13T10:10:51.000 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 10 to 10 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.001 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.001 INFO:tasks.rados.rados.0.smithi195.stdout:44: finishing write tid 2 to smithi19530714-44 2024-09-13T10:10:51.001 INFO:tasks.rados.rados.0.smithi195.stdout:44: finishing write tid 3 to smithi19530714-44 2024-09-13T10:10:51.001 INFO:tasks.rados.rados.0.smithi195.stdout:44: finishing write tid 4 to smithi19530714-44 2024-09-13T10:10:51.001 INFO:tasks.rados.rados.0.smithi195.stdout:44: oid 44 updating version 0 to 9 2024-09-13T10:10:51.001 INFO:tasks.rados.rados.0.smithi195.stdout:44: oid 44 version 9 is already newer than 7 2024-09-13T10:10:51.001 INFO:tasks.rados.rados.0.smithi195.stdout:44: oid 44 version 9 is already newer than 8 2024-09-13T10:10:51.001 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 9 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-13T10:10:51.001 INFO:tasks.rados.rados.0.smithi195.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-09-13T10:10:51.001 INFO:tasks.rados.rados.0.smithi195.stdout:45: finishing write tid 1 to smithi19530714-45 2024-09-13T10:10:51.001 INFO:tasks.rados.rados.0.smithi195.stdout:45: finishing write tid 2 to smithi19530714-45 2024-09-13T10:10:51.001 INFO:tasks.rados.rados.0.smithi195.stdout:45: finishing write tid 3 to smithi19530714-45 2024-09-13T10:10:51.001 INFO:tasks.rados.rados.0.smithi195.stdout:45: finishing write tid 4 to smithi19530714-45 2024-09-13T10:10:51.001 INFO:tasks.rados.rados.0.smithi195.stdout:45: oid 45 updating version 0 to 14 2024-09-13T10:10:51.002 INFO:tasks.rados.rados.0.smithi195.stdout:45: oid 45 version 14 is already newer than 13 2024-09-13T10:10:51.002 INFO:tasks.rados.rados.0.smithi195.stdout:45: oid 45 updating version 14 to 15 2024-09-13T10:10:51.002 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 15 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-13T10:10:51.002 INFO:tasks.rados.rados.0.smithi195.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-09-13T10:10:51.002 INFO:tasks.rados.rados.0.smithi195.stdout:46: finishing write tid 1 to smithi19530714-46 2024-09-13T10:10:51.002 INFO:tasks.rados.rados.0.smithi195.stdout:46: finishing write tid 2 to smithi19530714-46 2024-09-13T10:10:51.002 INFO:tasks.rados.rados.0.smithi195.stdout:46: finishing write tid 3 to smithi19530714-46 2024-09-13T10:10:51.002 INFO:tasks.rados.rados.0.smithi195.stdout:46: finishing write tid 4 to smithi19530714-46 2024-09-13T10:10:51.002 INFO:tasks.rados.rados.0.smithi195.stdout:46: oid 46 updating version 0 to 5 2024-09-13T10:10:51.002 INFO:tasks.rados.rados.0.smithi195.stdout:46: oid 46 version 5 is already newer than 4 2024-09-13T10:10:51.002 INFO:tasks.rados.rados.0.smithi195.stdout:46: oid 46 updating version 5 to 6 2024-09-13T10:10:51.002 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 6 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-13T10:10:51.002 INFO:tasks.rados.rados.0.smithi195.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-09-13T10:10:51.003 INFO:tasks.rados.rados.0.smithi195.stdout:47: finishing write tid 1 to smithi19530714-47 2024-09-13T10:10:51.003 INFO:tasks.rados.rados.0.smithi195.stdout:47: finishing write tid 2 to smithi19530714-47 2024-09-13T10:10:51.003 INFO:tasks.rados.rados.0.smithi195.stdout:47: finishing write tid 3 to smithi19530714-47 2024-09-13T10:10:51.003 INFO:tasks.rados.rados.0.smithi195.stdout:47: finishing write tid 4 to smithi19530714-47 2024-09-13T10:10:51.003 INFO:tasks.rados.rados.0.smithi195.stdout:47: oid 47 updating version 0 to 9 2024-09-13T10:10:51.003 INFO:tasks.rados.rados.0.smithi195.stdout:47: oid 47 version 9 is already newer than 8 2024-09-13T10:10:51.003 INFO:tasks.rados.rados.0.smithi195.stdout:47: oid 47 version 9 is already newer than 7 2024-09-13T10:10:51.003 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 9 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-13T10:10:51.003 INFO:tasks.rados.rados.0.smithi195.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-09-13T10:10:51.003 INFO:tasks.rados.rados.0.smithi195.stdout:48: finishing write tid 1 to smithi19530714-48 2024-09-13T10:10:51.003 INFO:tasks.rados.rados.0.smithi195.stdout:48: finishing write tid 2 to smithi19530714-48 2024-09-13T10:10:51.003 INFO:tasks.rados.rados.0.smithi195.stdout:48: finishing write tid 3 to smithi19530714-48 2024-09-13T10:10:51.003 INFO:tasks.rados.rados.0.smithi195.stdout:48: finishing write tid 4 to smithi19530714-48 2024-09-13T10:10:51.003 INFO:tasks.rados.rados.0.smithi195.stdout:48: oid 48 updating version 0 to 15 2024-09-13T10:10:51.004 INFO:tasks.rados.rados.0.smithi195.stdout:48: oid 48 version 15 is already newer than 14 2024-09-13T10:10:51.004 INFO:tasks.rados.rados.0.smithi195.stdout:48: oid 48 version 15 is already newer than 13 2024-09-13T10:10:51.004 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 15 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-13T10:10:51.004 INFO:tasks.rados.rados.0.smithi195.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-09-13T10:10:51.004 INFO:tasks.rados.rados.0.smithi195.stdout:49: finishing write tid 1 to smithi19530714-49 2024-09-13T10:10:51.004 INFO:tasks.rados.rados.0.smithi195.stdout:49: finishing write tid 2 to smithi19530714-49 2024-09-13T10:10:51.004 INFO:tasks.rados.rados.0.smithi195.stdout:49: finishing write tid 3 to smithi19530714-49 2024-09-13T10:10:51.004 INFO:tasks.rados.rados.0.smithi195.stdout:49: finishing write tid 4 to smithi19530714-49 2024-09-13T10:10:51.004 INFO:tasks.rados.rados.0.smithi195.stdout:49: oid 49 updating version 0 to 16 2024-09-13T10:10:51.004 INFO:tasks.rados.rados.0.smithi195.stdout:49: oid 49 updating version 16 to 18 2024-09-13T10:10:51.004 INFO:tasks.rados.rados.0.smithi195.stdout:49: oid 49 version 18 is already newer than 17 2024-09-13T10:10:51.004 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 18 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-13T10:10:51.004 INFO:tasks.rados.rados.0.smithi195.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-09-13T10:10:51.005 INFO:tasks.rados.rados.0.smithi195.stdout:50: finishing write tid 1 to smithi19530714-50 2024-09-13T10:10:51.005 INFO:tasks.rados.rados.0.smithi195.stdout:44: done (15 left) 2024-09-13T10:10:51.005 INFO:tasks.rados.rados.0.smithi195.stdout:45: done (14 left) 2024-09-13T10:10:51.005 INFO:tasks.rados.rados.0.smithi195.stdout:46: done (13 left) 2024-09-13T10:10:51.005 INFO:tasks.rados.rados.0.smithi195.stdout:47: done (12 left) 2024-09-13T10:10:51.005 INFO:tasks.rados.rados.0.smithi195.stdout:48: done (11 left) 2024-09-13T10:10:51.005 INFO:tasks.rados.rados.0.smithi195.stdout:49: done (10 left) 2024-09-13T10:10:51.005 INFO:tasks.rados.rados.0.smithi195.stdout:60: copy oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.005 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 11 to 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.005 INFO:tasks.rados.rados.0.smithi195.stdout:61: copy oid 12 target oid 12 low_tier 2024-09-13T10:10:51.005 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 12 to 12 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.005 INFO:tasks.rados.rados.0.smithi195.stdout:62: copy oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.005 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 13 to 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.005 INFO:tasks.rados.rados.0.smithi195.stdout:63: copy oid 14 target oid 14 low_tier 2024-09-13T10:10:51.006 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 14 to 14 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.006 INFO:tasks.rados.rados.0.smithi195.stdout:64: copy oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.006 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 15 to 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.006 INFO:tasks.rados.rados.0.smithi195.stdout:65: copy oid 16 target oid 16 low_tier 2024-09-13T10:10:51.006 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 16 to 16 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.006 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.007 INFO:tasks.rados.rados.0.smithi195.stdout:50: finishing write tid 2 to smithi19530714-50 2024-09-13T10:10:51.007 INFO:tasks.rados.rados.0.smithi195.stdout:50: finishing write tid 3 to smithi19530714-50 2024-09-13T10:10:51.007 INFO:tasks.rados.rados.0.smithi195.stdout:50: finishing write tid 4 to smithi19530714-50 2024-09-13T10:10:51.007 INFO:tasks.rados.rados.0.smithi195.stdout:50: oid 50 updating version 0 to 8 2024-09-13T10:10:51.008 INFO:tasks.rados.rados.0.smithi195.stdout:50: oid 50 version 8 is already newer than 7 2024-09-13T10:10:51.008 INFO:tasks.rados.rados.0.smithi195.stdout:50: oid 50 updating version 8 to 9 2024-09-13T10:10:51.008 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 9 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-13T10:10:51.008 INFO:tasks.rados.rados.0.smithi195.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-09-13T10:10:51.008 INFO:tasks.rados.rados.0.smithi195.stdout:50: done (15 left) 2024-09-13T10:10:51.008 INFO:tasks.rados.rados.0.smithi195.stdout:66: copy oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.008 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 17 to 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.008 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.008 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:10:51.005+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:10:51.026 INFO:tasks.rados.rados.0.smithi195.stdout:53: finishing copy op to oid 4 low_tier 2024-09-13T10:10:51.026 INFO:tasks.rados.rados.0.smithi195.stdout:53: done (15 left) 2024-09-13T10:10:51.026 INFO:tasks.rados.rados.0.smithi195.stdout:67: copy oid 18 target oid 18 low_tier 2024-09-13T10:10:51.027 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 18 to 18 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.027 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.032 INFO:tasks.rados.rados.0.smithi195.stdout:52: finishing copy op to oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.032 INFO:tasks.rados.rados.0.smithi195.stdout:52: done (15 left) 2024-09-13T10:10:51.032 INFO:tasks.rados.rados.0.smithi195.stdout:68: copy oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.032 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 19 to 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.032 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.032 INFO:tasks.rados.rados.0.smithi195.stdout:56: finishing copy op to oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.032 INFO:tasks.rados.rados.0.smithi195.stdout:56: done (15 left) 2024-09-13T10:10:51.032 INFO:tasks.rados.rados.0.smithi195.stdout:69: copy oid 20 target oid 20 low_tier 2024-09-13T10:10:51.032 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 20 to 20 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.032 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.034 INFO:tasks.rados.rados.0.smithi195.stdout:61: finishing copy op to oid 12 low_tier 2024-09-13T10:10:51.034 INFO:tasks.rados.rados.0.smithi195.stdout:61: done (15 left) 2024-09-13T10:10:51.034 INFO:tasks.rados.rados.0.smithi195.stdout:70: copy oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.035 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 21 to 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.035 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.037 INFO:tasks.rados.rados.0.smithi195.stdout:64: finishing copy op to oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.038 INFO:tasks.rados.rados.0.smithi195.stdout:64: done (15 left) 2024-09-13T10:10:51.038 INFO:tasks.rados.rados.0.smithi195.stdout:71: copy oid 22 target oid 22 low_tier 2024-09-13T10:10:51.038 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 22 to 22 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.038 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.045 INFO:tasks.rados.rados.0.smithi195.stdout:51: finishing copy op to oid 2 low_tier 2024-09-13T10:10:51.045 INFO:tasks.rados.rados.0.smithi195.stdout:51: done (15 left) 2024-09-13T10:10:51.045 INFO:tasks.rados.rados.0.smithi195.stdout:72: copy oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.045 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 23 to 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.045 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.049 INFO:tasks.rados.rados.0.smithi195.stdout:65: finishing copy op to oid 16 low_tier 2024-09-13T10:10:51.049 INFO:tasks.rados.rados.0.smithi195.stdout:65: done (15 left) 2024-09-13T10:10:51.049 INFO:tasks.rados.rados.0.smithi195.stdout:73: copy oid 24 target oid 24 low_tier 2024-09-13T10:10:51.049 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 24 to 24 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.049 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.049 INFO:tasks.rados.rados.0.smithi195.stdout:60: finishing copy op to oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.050 INFO:tasks.rados.rados.0.smithi195.stdout:60: done (15 left) 2024-09-13T10:10:51.050 INFO:tasks.rados.rados.0.smithi195.stdout:74: copy oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.050 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 25 to 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.050 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.051 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:10:51.051 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:10:51.051 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:10:51.051 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:10:51.057 INFO:tasks.rados.rados.0.smithi195.stdout:68: finishing copy op to oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.057 INFO:tasks.rados.rados.0.smithi195.stdout:68: done (15 left) 2024-09-13T10:10:51.057 INFO:tasks.rados.rados.0.smithi195.stdout:75: copy oid 26 target oid 26 low_tier 2024-09-13T10:10:51.057 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 26 to 26 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.057 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.059 INFO:tasks.rados.rados.0.smithi195.stdout:55: finishing copy op to oid 6 low_tier 2024-09-13T10:10:51.059 INFO:tasks.rados.rados.0.smithi195.stdout:63: finishing copy op to oid 14 low_tier 2024-09-13T10:10:51.059 INFO:tasks.rados.rados.0.smithi195.stdout:55: done (15 left) 2024-09-13T10:10:51.059 INFO:tasks.rados.rados.0.smithi195.stdout:63: done (14 left) 2024-09-13T10:10:51.059 INFO:tasks.rados.rados.0.smithi195.stdout:76: copy oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.059 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 27 to 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.060 INFO:tasks.rados.rados.0.smithi195.stdout:77: copy oid 28 target oid 28 low_tier 2024-09-13T10:10:51.060 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 28 to 28 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.060 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.060 INFO:tasks.rados.rados.0.smithi195.stdout:67: finishing copy op to oid 18 low_tier 2024-09-13T10:10:51.060 INFO:tasks.rados.rados.0.smithi195.stdout:67: done (15 left) 2024-09-13T10:10:51.060 INFO:tasks.rados.rados.0.smithi195.stdout:78: copy oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.060 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 29 to 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.060 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.061 INFO:tasks.rados.rados.0.smithi195.stdout:59: finishing copy op to oid 10 low_tier 2024-09-13T10:10:51.061 INFO:tasks.rados.rados.0.smithi195.stdout:59: done (15 left) 2024-09-13T10:10:51.061 INFO:tasks.rados.rados.0.smithi195.stdout:79: copy oid 30 target oid 30 low_tier 2024-09-13T10:10:51.061 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 30 to 30 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.061 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.070 INFO:tasks.rados.rados.0.smithi195.stdout:58: finishing copy op to oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.070 INFO:tasks.rados.rados.0.smithi195.stdout:58: done (15 left) 2024-09-13T10:10:51.070 INFO:tasks.rados.rados.0.smithi195.stdout:80: copy oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.070 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 31 to 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.070 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.070 INFO:tasks.rados.rados.0.smithi195.stdout:62: finishing copy op to oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.070 INFO:tasks.rados.rados.0.smithi195.stdout:62: done (15 left) 2024-09-13T10:10:51.070 INFO:tasks.rados.rados.0.smithi195.stdout:81: copy oid 32 target oid 32 low_tier 2024-09-13T10:10:51.070 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 32 to 32 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.071 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.071 INFO:tasks.rados.rados.0.smithi195.stdout:57: finishing copy op to oid 8 low_tier 2024-09-13T10:10:51.071 INFO:tasks.rados.rados.0.smithi195.stdout:57: done (15 left) 2024-09-13T10:10:51.071 INFO:tasks.rados.rados.0.smithi195.stdout:82: copy oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.071 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 33 to 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.071 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.071 INFO:tasks.rados.rados.0.smithi195.stdout:70: finishing copy op to oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.071 INFO:tasks.rados.rados.0.smithi195.stdout:70: done (15 left) 2024-09-13T10:10:51.071 INFO:tasks.rados.rados.0.smithi195.stdout:83: copy oid 34 target oid 34 low_tier 2024-09-13T10:10:51.071 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 34 to 34 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.072 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.072 INFO:tasks.rados.rados.0.smithi195.stdout:66: finishing copy op to oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.072 INFO:tasks.rados.rados.0.smithi195.stdout:66: done (15 left) 2024-09-13T10:10:51.072 INFO:tasks.rados.rados.0.smithi195.stdout:84: copy oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.072 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 35 to 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.072 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.072 INFO:tasks.rados.rados.0.smithi195.stdout:69: finishing copy op to oid 20 low_tier 2024-09-13T10:10:51.072 INFO:tasks.rados.rados.0.smithi195.stdout:69: done (15 left) 2024-09-13T10:10:51.072 INFO:tasks.rados.rados.0.smithi195.stdout:85: copy oid 36 target oid 36 low_tier 2024-09-13T10:10:51.072 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 36 to 36 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.072 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.074 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T10:10:51.075 INFO:tasks.rados.rados.0.smithi195.stdout:74: finishing copy op to oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.075 INFO:tasks.rados.rados.0.smithi195.stdout:74: done (15 left) 2024-09-13T10:10:51.075 INFO:tasks.rados.rados.0.smithi195.stdout:86: copy oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.075 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 37 to 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.075 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.078 INFO:tasks.rados.rados.0.smithi195.stdout:71: finishing copy op to oid 22 low_tier 2024-09-13T10:10:51.078 INFO:tasks.rados.rados.0.smithi195.stdout:71: done (15 left) 2024-09-13T10:10:51.078 INFO:tasks.rados.rados.0.smithi195.stdout:87: copy oid 38 target oid 38 low_tier 2024-09-13T10:10:51.078 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 38 to 38 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.078 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.087 INFO:tasks.rados.rados.0.smithi195.stdout:72: finishing copy op to oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.087 INFO:tasks.rados.rados.0.smithi195.stdout:72: done (15 left) 2024-09-13T10:10:51.087 INFO:tasks.rados.rados.0.smithi195.stdout:88: copy oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.088 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 39 to 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.088 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.088 INFO:tasks.rados.rados.0.smithi195.stdout:75: finishing copy op to oid 26 low_tier 2024-09-13T10:10:51.088 INFO:tasks.rados.rados.0.smithi195.stdout:75: done (15 left) 2024-09-13T10:10:51.089 INFO:tasks.rados.rados.0.smithi195.stdout:89: copy oid 40 target oid 40 low_tier 2024-09-13T10:10:51.089 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 40 to 40 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.089 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.089 INFO:tasks.rados.rados.0.smithi195.stdout:78: finishing copy op to oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.090 INFO:tasks.rados.rados.0.smithi195.stdout:78: done (15 left) 2024-09-13T10:10:51.090 INFO:tasks.rados.rados.0.smithi195.stdout:90: copy oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.090 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 41 to 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.090 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.094 INFO:tasks.rados.rados.0.smithi195.stdout:77: finishing copy op to oid 28 low_tier 2024-09-13T10:10:51.095 INFO:tasks.rados.rados.0.smithi195.stdout:77: done (15 left) 2024-09-13T10:10:51.095 INFO:tasks.rados.rados.0.smithi195.stdout:91: copy oid 42 target oid 42 low_tier 2024-09-13T10:10:51.095 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 42 to 42 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.095 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.095 INFO:tasks.rados.rados.0.smithi195.stdout:79: finishing copy op to oid 30 low_tier 2024-09-13T10:10:51.095 INFO:tasks.rados.rados.0.smithi195.stdout:79: done (15 left) 2024-09-13T10:10:51.095 INFO:tasks.rados.rados.0.smithi195.stdout:92: copy oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.095 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 43 to 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.095 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.096 INFO:tasks.rados.rados.0.smithi195.stdout:81: finishing copy op to oid 32 low_tier 2024-09-13T10:10:51.096 INFO:tasks.rados.rados.0.smithi195.stdout:81: done (15 left) 2024-09-13T10:10:51.096 INFO:tasks.rados.rados.0.smithi195.stdout:93: copy oid 44 target oid 44 low_tier 2024-09-13T10:10:51.097 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 44 to 44 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.097 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.108 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:10:51.106+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:10:51.111 INFO:tasks.rados.rados.0.smithi195.stdout:76: finishing copy op to oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.111 INFO:tasks.rados.rados.0.smithi195.stdout:76: done (15 left) 2024-09-13T10:10:51.111 INFO:tasks.rados.rados.0.smithi195.stdout:94: copy oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.111 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 45 to 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.112 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.112 INFO:tasks.rados.rados.0.smithi195.stdout:84: finishing copy op to oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.112 INFO:tasks.rados.rados.0.smithi195.stdout:84: done (15 left) 2024-09-13T10:10:51.112 INFO:tasks.rados.rados.0.smithi195.stdout:95: copy oid 46 target oid 46 low_tier 2024-09-13T10:10:51.112 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 46 to 46 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.112 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.114 INFO:tasks.rados.rados.0.smithi195.stdout:73: finishing copy op to oid 24 low_tier 2024-09-13T10:10:51.115 INFO:tasks.rados.rados.0.smithi195.stdout:73: done (15 left) 2024-09-13T10:10:51.115 INFO:tasks.rados.rados.0.smithi195.stdout:96: copy oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.115 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 47 to 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.115 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.115 INFO:tasks.rados.rados.0.smithi195.stdout:83: finishing copy op to oid 34 low_tier 2024-09-13T10:10:51.115 INFO:tasks.rados.rados.0.smithi195.stdout:83: done (15 left) 2024-09-13T10:10:51.115 INFO:tasks.rados.rados.0.smithi195.stdout:97: copy oid 48 target oid 48 low_tier 2024-09-13T10:10:51.115 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 48 to 48 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.115 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.115 INFO:tasks.rados.rados.0.smithi195.stdout:80: finishing copy op to oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.116 INFO:tasks.rados.rados.0.smithi195.stdout:80: done (15 left) 2024-09-13T10:10:51.116 INFO:tasks.rados.rados.0.smithi195.stdout:98: copy oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.116 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 49 to 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.116 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.116 INFO:tasks.rados.rados.0.smithi195.stdout:82: finishing copy op to oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.116 INFO:tasks.rados.rados.0.smithi195.stdout:82: done (15 left) 2024-09-13T10:10:51.116 INFO:tasks.rados.rados.0.smithi195.stdout:99: copy oid 50 target oid 50 low_tier 2024-09-13T10:10:51.116 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 50 to 50 low_tier tgt_pool_name low_tier 2024-09-13T10:10:51.116 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.116 INFO:tasks.rados.rados.0.smithi195.stdout:89: finishing copy op to oid 40 low_tier 2024-09-13T10:10:51.116 INFO:tasks.rados.rados.0.smithi195.stdout:89: done (15 left) 2024-09-13T10:10:51.117 INFO:tasks.rados.rados.0.smithi195.stdout:100: copy oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.117 INFO:tasks.rados.rados.0.smithi195.stdout:copy op oid 1 to 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm tgt_pool_name low_tier 2024-09-13T10:10:51.117 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.118 INFO:tasks.rados.rados.0.smithi195.stdout:86: finishing copy op to oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.118 INFO:tasks.rados.rados.0.smithi195.stdout:86: done (15 left) 2024-09-13T10:10:51.118 INFO:tasks.rados.rados.0.smithi195.stdout:101: set_chunk oid 2 target oid 2 low_tier 2024-09-13T10:10:51.118 INFO:tasks.rados.rados.0.smithi195.stdout:101: set_chunk oid 2 offset: 737280 length: 736768 target oid smithi19530714-2 low_tier offset: 737280 snap -1 2024-09-13T10:10:51.118 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.122 INFO:tasks.rados.rados.0.smithi195.stdout:85: finishing copy op to oid 36 low_tier 2024-09-13T10:10:51.122 INFO:tasks.rados.rados.0.smithi195.stdout:85: done (15 left) 2024-09-13T10:10:51.122 INFO:tasks.rados.rados.0.smithi195.stdout:102: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.122 INFO:tasks.rados.rados.0.smithi195.stdout:102: set_chunk oid 3 offset: 1152000 length: 409600 target oid smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1152000 snap -1 2024-09-13T10:10:51.122 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.133 INFO:tasks.rados.rados.0.smithi195.stdout:93: finishing copy op to oid 44 low_tier 2024-09-13T10:10:51.133 INFO:tasks.rados.rados.0.smithi195.stdout:93: done (15 left) 2024-09-13T10:10:51.133 INFO:tasks.rados.rados.0.smithi195.stdout:103: set_chunk oid 4 target oid 4 low_tier 2024-09-13T10:10:51.133 INFO:tasks.rados.rados.0.smithi195.stdout:103: set_chunk oid 4 offset: 1100800 length: 320000 target oid smithi19530714-4 low_tier offset: 1100800 snap -1 2024-09-13T10:10:51.134 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.138 INFO:tasks.rados.rados.0.smithi195.stdout:90: finishing copy op to oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.139 INFO:tasks.rados.rados.0.smithi195.stdout:90: done (15 left) 2024-09-13T10:10:51.139 INFO:tasks.rados.rados.0.smithi195.stdout:103 retry set_chunk ! 2024-09-13T10:10:51.139 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 15 2024-09-13T10:10:51.139 INFO:tasks.rados.rados.0.smithi195.stdout:92: finishing copy op to oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.139 INFO:tasks.rados.rados.0.smithi195.stdout:92: done (14 left) 2024-09-13T10:10:51.139 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 14 2024-09-13T10:10:51.141 INFO:tasks.rados.rados.0.smithi195.stdout:94: finishing copy op to oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.141 INFO:tasks.rados.rados.0.smithi195.stdout:94: done (13 left) 2024-09-13T10:10:51.141 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 13 2024-09-13T10:10:51.142 INFO:tasks.rados.rados.0.smithi195.stdout:98: finishing copy op to oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.143 INFO:tasks.rados.rados.0.smithi195.stdout:98: done (12 left) 2024-09-13T10:10:51.143 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 12 2024-09-13T10:10:51.143 INFO:tasks.rados.rados.0.smithi195.stdout:100: finishing copy op to oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.143 INFO:tasks.rados.rados.0.smithi195.stdout:100: done (11 left) 2024-09-13T10:10:51.143 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 11 2024-09-13T10:10:51.144 INFO:tasks.rados.rados.0.smithi195.stdout:97: finishing copy op to oid 48 low_tier 2024-09-13T10:10:51.145 INFO:tasks.rados.rados.0.smithi195.stdout:97: done (10 left) 2024-09-13T10:10:51.145 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 10 2024-09-13T10:10:51.147 INFO:tasks.rados.rados.0.smithi195.stdout:103: finishing set_chunk to oid 4 2024-09-13T10:10:51.147 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 3 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-13T10:10:51.147 INFO:tasks.rados.rados.0.smithi195.stdout:103: done (9 left) 2024-09-13T10:10:51.147 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 9 2024-09-13T10:10:51.151 INFO:tasks.rados.rados.0.smithi195.stdout:95: finishing copy op to oid 46 low_tier 2024-09-13T10:10:51.151 INFO:tasks.rados.rados.0.smithi195.stdout:95: done (8 left) 2024-09-13T10:10:51.151 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 8 2024-09-13T10:10:51.151 INFO:tasks.rados.rados.0.smithi195.stdout:87: finishing copy op to oid 38 low_tier 2024-09-13T10:10:51.151 INFO:tasks.rados.rados.0.smithi195.stdout:87: done (7 left) 2024-09-13T10:10:51.152 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 7 2024-09-13T10:10:51.153 INFO:tasks.rados.rados.0.smithi195.stdout:91: finishing copy op to oid 42 low_tier 2024-09-13T10:10:51.153 INFO:tasks.rados.rados.0.smithi195.stdout:91: done (6 left) 2024-09-13T10:10:51.153 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 6 2024-09-13T10:10:51.153 INFO:tasks.rados.rados.0.smithi195.stdout:102: finishing set_chunk to oid 3 2024-09-13T10:10:51.153 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 3 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-13T10:10:51.153 INFO:tasks.rados.rados.0.smithi195.stdout:102: done (5 left) 2024-09-13T10:10:51.153 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 5 2024-09-13T10:10:51.157 INFO:tasks.rados.rados.0.smithi195.stdout:88: finishing copy op to oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.157 INFO:tasks.rados.rados.0.smithi195.stdout:88: done (4 left) 2024-09-13T10:10:51.157 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 4 2024-09-13T10:10:51.159 INFO:tasks.rados.rados.0.smithi195.stdout:101: finishing set_chunk to oid 2 2024-09-13T10:10:51.159 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-13T10:10:51.159 INFO:tasks.rados.rados.0.smithi195.stdout:101: done (3 left) 2024-09-13T10:10:51.159 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 3 2024-09-13T10:10:51.164 INFO:tasks.rados.rados.0.smithi195.stdout:96: finishing copy op to oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.164 INFO:tasks.rados.rados.0.smithi195.stdout:96: done (2 left) 2024-09-13T10:10:51.164 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 2 2024-09-13T10:10:51.165 INFO:tasks.rados.rados.0.smithi195.stdout:99: finishing copy op to oid 50 low_tier 2024-09-13T10:10:51.165 INFO:tasks.rados.rados.0.smithi195.stdout:99: done (1 left) 2024-09-13T10:10:51.165 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 1 2024-09-13T10:10:51.209 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:51.207+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:51.215 INFO:tasks.rados.rados.0.smithi195.stdout:54: finishing copy op to oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.215 INFO:tasks.rados.rados.0.smithi195.stdout:54: done (0 left) 2024-09-13T10:10:51.216 INFO:tasks.rados.rados.0.smithi195.stdout:104: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.216 INFO:tasks.rados.rados.0.smithi195.stdout:104: set_chunk oid 5 offset: 557568 length: 1014784 target oid smithi19530714-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 557568 snap -1 2024-09-13T10:10:51.216 INFO:tasks.rados.rados.0.smithi195.stdout:105: set_chunk oid 6 target oid 6 low_tier 2024-09-13T10:10:51.216 INFO:tasks.rados.rados.0.smithi195.stdout:105: set_chunk oid 6 offset: 349184 length: 38400 target oid smithi19530714-6 low_tier offset: 349184 snap -1 2024-09-13T10:10:51.216 INFO:tasks.rados.rados.0.smithi195.stdout:106: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.216 INFO:tasks.rados.rados.0.smithi195.stdout:106: set_chunk oid 7 offset: 165376 length: 1688064 target oid smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 165376 snap -1 2024-09-13T10:10:51.216 INFO:tasks.rados.rados.0.smithi195.stdout:107: set_chunk oid 8 target oid 8 low_tier 2024-09-13T10:10:51.216 INFO:tasks.rados.rados.0.smithi195.stdout:107: set_chunk oid 8 offset: 969728 length: 852992 target oid smithi19530714-8 low_tier offset: 969728 snap -1 2024-09-13T10:10:51.216 INFO:tasks.rados.rados.0.smithi195.stdout:108: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.217 INFO:tasks.rados.rados.0.smithi195.stdout:108: set_chunk oid 9 offset: 343552 length: 162816 target oid smithi19530714-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 343552 snap -1 2024-09-13T10:10:51.217 INFO:tasks.rados.rados.0.smithi195.stdout:109: set_chunk oid 10 target oid 10 low_tier 2024-09-13T10:10:51.217 INFO:tasks.rados.rados.0.smithi195.stdout:109: set_chunk oid 10 offset: 830464 length: 746496 target oid smithi19530714-10 low_tier offset: 830464 snap -1 2024-09-13T10:10:51.217 INFO:tasks.rados.rados.0.smithi195.stdout:110: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.217 INFO:tasks.rados.rados.0.smithi195.stdout:110: set_chunk oid 11 offset: 1900544 length: 115712 target oid smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1900544 snap -1 2024-09-13T10:10:51.217 INFO:tasks.rados.rados.0.smithi195.stdout:111: set_chunk oid 12 target oid 12 low_tier 2024-09-13T10:10:51.217 INFO:tasks.rados.rados.0.smithi195.stdout:111: set_chunk oid 12 offset: 939520 length: 310784 target oid smithi19530714-12 low_tier offset: 939520 snap -1 2024-09-13T10:10:51.217 INFO:tasks.rados.rados.0.smithi195.stdout:112: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.217 INFO:tasks.rados.rados.0.smithi195.stdout:112: set_chunk oid 13 offset: 141312 length: 838144 target oid smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 141312 snap -1 2024-09-13T10:10:51.217 INFO:tasks.rados.rados.0.smithi195.stdout:113: set_chunk oid 14 target oid 14 low_tier 2024-09-13T10:10:51.217 INFO:tasks.rados.rados.0.smithi195.stdout:113: set_chunk oid 14 offset: 2041344 length: 46080 target oid smithi19530714-14 low_tier offset: 2041344 snap -1 2024-09-13T10:10:51.218 INFO:tasks.rados.rados.0.smithi195.stdout:114: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.218 INFO:tasks.rados.rados.0.smithi195.stdout:114: set_chunk oid 15 offset: 1573888 length: 35328 target oid smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1573888 snap -1 2024-09-13T10:10:51.218 INFO:tasks.rados.rados.0.smithi195.stdout:115: set_chunk oid 16 target oid 16 low_tier 2024-09-13T10:10:51.218 INFO:tasks.rados.rados.0.smithi195.stdout:115: set_chunk oid 16 offset: 290816 length: 986112 target oid smithi19530714-16 low_tier offset: 290816 snap -1 2024-09-13T10:10:51.218 INFO:tasks.rados.rados.0.smithi195.stdout:116: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.218 INFO:tasks.rados.rados.0.smithi195.stdout:116: set_chunk oid 17 offset: 724992 length: 1302016 target oid smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 724992 snap -1 2024-09-13T10:10:51.218 INFO:tasks.rados.rados.0.smithi195.stdout:117: set_chunk oid 18 target oid 18 low_tier 2024-09-13T10:10:51.218 INFO:tasks.rados.rados.0.smithi195.stdout:117: set_chunk oid 18 offset: 408576 length: 1870336 target oid smithi19530714-18 low_tier offset: 408576 snap -1 2024-09-13T10:10:51.218 INFO:tasks.rados.rados.0.smithi195.stdout:118: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.218 INFO:tasks.rados.rados.0.smithi195.stdout:118: set_chunk oid 19 offset: 687616 length: 278016 target oid smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 687616 snap -1 2024-09-13T10:10:51.218 INFO:tasks.rados.rados.0.smithi195.stdout:119: set_chunk oid 20 target oid 20 low_tier 2024-09-13T10:10:51.219 INFO:tasks.rados.rados.0.smithi195.stdout:119: set_chunk oid 20 offset: 233472 length: 1342976 target oid smithi19530714-20 low_tier offset: 233472 snap -1 2024-09-13T10:10:51.219 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.225 INFO:tasks.rados.rados.0.smithi195.stdout:108: finishing set_chunk to oid 9 2024-09-13T10:10:51.225 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 6 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-13T10:10:51.225 INFO:tasks.rados.rados.0.smithi195.stdout:108: done (15 left) 2024-09-13T10:10:51.226 INFO:tasks.rados.rados.0.smithi195.stdout:120: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.226 INFO:tasks.rados.rados.0.smithi195.stdout:120: set_chunk oid 21 offset: 382976 length: 692736 target oid smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 382976 snap -1 2024-09-13T10:10:51.226 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.226 INFO:tasks.rados.rados.0.smithi195.stdout:111: finishing set_chunk to oid 12 2024-09-13T10:10:51.226 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 3 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-13T10:10:51.226 INFO:tasks.rados.rados.0.smithi195.stdout:111: done (15 left) 2024-09-13T10:10:51.226 INFO:tasks.rados.rados.0.smithi195.stdout:121: set_chunk oid 22 target oid 22 low_tier 2024-09-13T10:10:51.226 INFO:tasks.rados.rados.0.smithi195.stdout:121: set_chunk oid 22 offset: 346112 length: 979456 target oid smithi19530714-22 low_tier offset: 346112 snap -1 2024-09-13T10:10:51.226 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.227 INFO:tasks.rados.rados.0.smithi195.stdout:105: finishing set_chunk to oid 6 2024-09-13T10:10:51.227 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 6 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-13T10:10:51.227 INFO:tasks.rados.rados.0.smithi195.stdout:105: done (15 left) 2024-09-13T10:10:51.228 INFO:tasks.rados.rados.0.smithi195.stdout:122: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.228 INFO:tasks.rados.rados.0.smithi195.stdout:122: set_chunk oid 23 offset: 483840 length: 252928 target oid smithi19530714-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 483840 snap -1 2024-09-13T10:10:51.228 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.228 INFO:tasks.rados.rados.0.smithi195.stdout:115: finishing set_chunk to oid 16 2024-09-13T10:10:51.228 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 3 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-13T10:10:51.228 INFO:tasks.rados.rados.0.smithi195.stdout:115: done (15 left) 2024-09-13T10:10:51.228 INFO:tasks.rados.rados.0.smithi195.stdout:123: set_chunk oid 24 target oid 24 low_tier 2024-09-13T10:10:51.228 INFO:tasks.rados.rados.0.smithi195.stdout:123: set_chunk oid 24 offset: 243712 length: 457728 target oid smithi19530714-24 low_tier offset: 243712 snap -1 2024-09-13T10:10:51.228 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.229 INFO:tasks.rados.rados.0.smithi195.stdout:107: finishing set_chunk to oid 8 2024-09-13T10:10:51.229 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 3 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-13T10:10:51.229 INFO:tasks.rados.rados.0.smithi195.stdout:107: done (15 left) 2024-09-13T10:10:51.229 INFO:tasks.rados.rados.0.smithi195.stdout:124: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.229 INFO:tasks.rados.rados.0.smithi195.stdout:104: finishing set_chunk to oid 5 2024-09-13T10:10:51.229 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 3 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-13T10:10:51.229 INFO:tasks.rados.rados.0.smithi195.stdout:124: set_chunk oid 25 offset: 1639424 length: 254976 target oid smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1639424 snap -1 2024-09-13T10:10:51.230 INFO:tasks.rados.rados.0.smithi195.stdout:104: done (15 left) 2024-09-13T10:10:51.230 INFO:tasks.rados.rados.0.smithi195.stdout:125: set_chunk oid 26 target oid 26 low_tier 2024-09-13T10:10:51.230 INFO:tasks.rados.rados.0.smithi195.stdout:125: set_chunk oid 26 offset: 844288 length: 924672 target oid smithi19530714-26 low_tier offset: 844288 snap -1 2024-09-13T10:10:51.230 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.231 INFO:tasks.rados.rados.0.smithi195.stdout:109: finishing set_chunk to oid 10 2024-09-13T10:10:51.231 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 3 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-13T10:10:51.231 INFO:tasks.rados.rados.0.smithi195.stdout:113: finishing set_chunk to oid 14 2024-09-13T10:10:51.231 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 6 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-13T10:10:51.231 INFO:tasks.rados.rados.0.smithi195.stdout:119: finishing set_chunk to oid 20 2024-09-13T10:10:51.231 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 6 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-13T10:10:51.231 INFO:tasks.rados.rados.0.smithi195.stdout:109: done (15 left) 2024-09-13T10:10:51.231 INFO:tasks.rados.rados.0.smithi195.stdout:113: done (14 left) 2024-09-13T10:10:51.231 INFO:tasks.rados.rados.0.smithi195.stdout:119: done (13 left) 2024-09-13T10:10:51.231 INFO:tasks.rados.rados.0.smithi195.stdout:126: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.231 INFO:tasks.rados.rados.0.smithi195.stdout:126: set_chunk oid 27 offset: 487424 length: 1831424 target oid smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 487424 snap -1 2024-09-13T10:10:51.232 INFO:tasks.rados.rados.0.smithi195.stdout:127: set_chunk oid 28 target oid 28 low_tier 2024-09-13T10:10:51.232 INFO:tasks.rados.rados.0.smithi195.stdout:127: set_chunk oid 28 offset: 1956352 length: 137728 target oid smithi19530714-28 low_tier offset: 1956352 snap -1 2024-09-13T10:10:51.232 INFO:tasks.rados.rados.0.smithi195.stdout:128: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.232 INFO:tasks.rados.rados.0.smithi195.stdout:128: set_chunk oid 29 offset: 180736 length: 955904 target oid smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 180736 snap -1 2024-09-13T10:10:51.232 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.235 INFO:tasks.rados.rados.0.smithi195.stdout:122: finishing set_chunk to oid 23 2024-09-13T10:10:51.235 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 3 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-13T10:10:51.235 INFO:tasks.rados.rados.0.smithi195.stdout:122: done (15 left) 2024-09-13T10:10:51.235 INFO:tasks.rados.rados.0.smithi195.stdout:129: set_chunk oid 30 target oid 30 low_tier 2024-09-13T10:10:51.235 INFO:tasks.rados.rados.0.smithi195.stdout:129: set_chunk oid 30 offset: 617472 length: 726016 target oid smithi19530714-30 low_tier offset: 617472 snap -1 2024-09-13T10:10:51.236 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.236 INFO:tasks.rados.rados.0.smithi195.stdout:116: finishing set_chunk to oid 17 2024-09-13T10:10:51.236 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 9 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-13T10:10:51.236 INFO:tasks.rados.rados.0.smithi195.stdout:116: done (15 left) 2024-09-13T10:10:51.236 INFO:tasks.rados.rados.0.smithi195.stdout:130: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.236 INFO:tasks.rados.rados.0.smithi195.stdout:130: set_chunk oid 31 offset: 967168 length: 879616 target oid smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 967168 snap -1 2024-09-13T10:10:51.236 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.237 INFO:tasks.rados.rados.0.smithi195.stdout:124: finishing set_chunk to oid 25 2024-09-13T10:10:51.237 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 6 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-13T10:10:51.237 INFO:tasks.rados.rados.0.smithi195.stdout:124: done (15 left) 2024-09-13T10:10:51.237 INFO:tasks.rados.rados.0.smithi195.stdout:131: set_chunk oid 32 target oid 32 low_tier 2024-09-13T10:10:51.237 INFO:tasks.rados.rados.0.smithi195.stdout:131: set_chunk oid 32 offset: 221696 length: 1451008 target oid smithi19530714-32 low_tier offset: 221696 snap -1 2024-09-13T10:10:51.237 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.240 INFO:tasks.rados.rados.0.smithi195.stdout:121: finishing set_chunk to oid 22 2024-09-13T10:10:51.240 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 12 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-13T10:10:51.240 INFO:tasks.rados.rados.0.smithi195.stdout:121: done (15 left) 2024-09-13T10:10:51.240 INFO:tasks.rados.rados.0.smithi195.stdout:132: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.240 INFO:tasks.rados.rados.0.smithi195.stdout:132: set_chunk oid 33 offset: 690176 length: 396288 target oid smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 690176 snap -1 2024-09-13T10:10:51.240 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.242 INFO:tasks.rados.rados.0.smithi195.stdout:123: finishing set_chunk to oid 24 2024-09-13T10:10:51.242 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 3 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-13T10:10:51.242 INFO:tasks.rados.rados.0.smithi195.stdout:123: done (15 left) 2024-09-13T10:10:51.242 INFO:tasks.rados.rados.0.smithi195.stdout:133: set_chunk oid 34 target oid 34 low_tier 2024-09-13T10:10:51.242 INFO:tasks.rados.rados.0.smithi195.stdout:133: set_chunk oid 34 offset: 814080 length: 480256 target oid smithi19530714-34 low_tier offset: 814080 snap -1 2024-09-13T10:10:51.242 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.242 INFO:tasks.rados.rados.0.smithi195.stdout:128: finishing set_chunk to oid 29 2024-09-13T10:10:51.242 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 12 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-13T10:10:51.242 INFO:tasks.rados.rados.0.smithi195.stdout:128: done (15 left) 2024-09-13T10:10:51.242 INFO:tasks.rados.rados.0.smithi195.stdout:134: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.242 INFO:tasks.rados.rados.0.smithi195.stdout:134: set_chunk oid 35 offset: 1519104 length: 66048 target oid smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1519104 snap -1 2024-09-13T10:10:51.242 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.244 INFO:tasks.rados.rados.0.smithi195.stdout:126: finishing set_chunk to oid 27 2024-09-13T10:10:51.244 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 15 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-13T10:10:51.244 INFO:tasks.rados.rados.0.smithi195.stdout:126: done (15 left) 2024-09-13T10:10:51.244 INFO:tasks.rados.rados.0.smithi195.stdout:135: set_chunk oid 36 target oid 36 low_tier 2024-09-13T10:10:51.244 INFO:tasks.rados.rados.0.smithi195.stdout:135: set_chunk oid 36 offset: 558080 length: 61952 target oid smithi19530714-36 low_tier offset: 558080 snap -1 2024-09-13T10:10:51.244 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.247 INFO:tasks.rados.rados.0.smithi195.stdout:129: finishing set_chunk to oid 30 2024-09-13T10:10:51.247 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 18 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-13T10:10:51.247 INFO:tasks.rados.rados.0.smithi195.stdout:129: done (15 left) 2024-09-13T10:10:51.247 INFO:tasks.rados.rados.0.smithi195.stdout:136: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.247 INFO:tasks.rados.rados.0.smithi195.stdout:136: set_chunk oid 37 offset: 973824 length: 950784 target oid smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 973824 snap -1 2024-09-13T10:10:51.247 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.248 INFO:tasks.rados.rados.0.smithi195.stdout:131: finishing set_chunk to oid 32 2024-09-13T10:10:51.248 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 15 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-13T10:10:51.248 INFO:tasks.rados.rados.0.smithi195.stdout:131: done (15 left) 2024-09-13T10:10:51.248 INFO:tasks.rados.rados.0.smithi195.stdout:137: set_chunk oid 38 target oid 38 low_tier 2024-09-13T10:10:51.248 INFO:tasks.rados.rados.0.smithi195.stdout:137: set_chunk oid 38 offset: 372736 length: 763904 target oid smithi19530714-38 low_tier offset: 372736 snap -1 2024-09-13T10:10:51.248 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.249 INFO:tasks.rados.rados.0.smithi195.stdout:112: finishing set_chunk to oid 13 2024-09-13T10:10:51.249 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 3 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-13T10:10:51.249 INFO:tasks.rados.rados.0.smithi195.stdout:112: done (15 left) 2024-09-13T10:10:51.250 INFO:tasks.rados.rados.0.smithi195.stdout:138: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.250 INFO:tasks.rados.rados.0.smithi195.stdout:138: set_chunk oid 39 offset: 1338880 length: 846336 target oid smithi19530714-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1338880 snap -1 2024-09-13T10:10:51.250 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.250 INFO:tasks.rados.rados.0.smithi195.stdout:134: finishing set_chunk to oid 35 2024-09-13T10:10:51.250 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 12 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-13T10:10:51.250 INFO:tasks.rados.rados.0.smithi195.stdout:134: done (15 left) 2024-09-13T10:10:51.250 INFO:tasks.rados.rados.0.smithi195.stdout:139: set_chunk oid 40 target oid 40 low_tier 2024-09-13T10:10:51.250 INFO:tasks.rados.rados.0.smithi195.stdout:139: set_chunk oid 40 offset: 705024 length: 654848 target oid smithi19530714-40 low_tier offset: 705024 snap -1 2024-09-13T10:10:51.250 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.251 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:10:51.251 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:10:51.251 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:10:51.251 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:10:51.251 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:10:51.253 INFO:tasks.rados.rados.0.smithi195.stdout:125: finishing set_chunk to oid 26 2024-09-13T10:10:51.254 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 3 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-13T10:10:51.254 INFO:tasks.rados.rados.0.smithi195.stdout:125: done (15 left) 2024-09-13T10:10:51.254 INFO:tasks.rados.rados.0.smithi195.stdout:140: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.254 INFO:tasks.rados.rados.0.smithi195.stdout:140: set_chunk oid 41 offset: 1520640 length: 415232 target oid smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1520640 snap -1 2024-09-13T10:10:51.254 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.257 INFO:tasks.rados.rados.0.smithi195.stdout:118: finishing set_chunk to oid 19 2024-09-13T10:10:51.257 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 9 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-13T10:10:51.257 INFO:tasks.rados.rados.0.smithi195.stdout:110: finishing set_chunk to oid 11 2024-09-13T10:10:51.257 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 3 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-13T10:10:51.257 INFO:tasks.rados.rados.0.smithi195.stdout:110: done (15 left) 2024-09-13T10:10:51.258 INFO:tasks.rados.rados.0.smithi195.stdout:118: done (14 left) 2024-09-13T10:10:51.258 INFO:tasks.rados.rados.0.smithi195.stdout:141: set_chunk oid 42 target oid 42 low_tier 2024-09-13T10:10:51.258 INFO:tasks.rados.rados.0.smithi195.stdout:141: set_chunk oid 42 offset: 1868800 length: 445952 target oid smithi19530714-42 low_tier offset: 1868800 snap -1 2024-09-13T10:10:51.258 INFO:tasks.rados.rados.0.smithi195.stdout:142: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.258 INFO:tasks.rados.rados.0.smithi195.stdout:142: set_chunk oid 43 offset: 688128 length: 1368064 target oid smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 688128 snap -1 2024-09-13T10:10:51.258 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.258 INFO:tasks.rados.rados.0.smithi195.stdout:130: finishing set_chunk to oid 31 2024-09-13T10:10:51.258 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 6 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-13T10:10:51.258 INFO:tasks.rados.rados.0.smithi195.stdout:130: done (15 left) 2024-09-13T10:10:51.258 INFO:tasks.rados.rados.0.smithi195.stdout:143: set_chunk oid 44 target oid 44 low_tier 2024-09-13T10:10:51.258 INFO:tasks.rados.rados.0.smithi195.stdout:143: set_chunk oid 44 offset: 48640 length: 97792 target oid smithi19530714-44 low_tier offset: 48640 snap -1 2024-09-13T10:10:51.259 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.260 INFO:tasks.rados.rados.0.smithi195.stdout:137: finishing set_chunk to oid 38 2024-09-13T10:10:51.260 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 12 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-13T10:10:51.260 INFO:tasks.rados.rados.0.smithi195.stdout:137: done (15 left) 2024-09-13T10:10:51.260 INFO:tasks.rados.rados.0.smithi195.stdout:144: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.260 INFO:tasks.rados.rados.0.smithi195.stdout:144: set_chunk oid 45 offset: 699392 length: 1423360 target oid smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 699392 snap -1 2024-09-13T10:10:51.260 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.260 INFO:tasks.rados.rados.0.smithi195.stdout:135: finishing set_chunk to oid 36 2024-09-13T10:10:51.260 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 6 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-13T10:10:51.260 INFO:tasks.rados.rados.0.smithi195.stdout:135: done (15 left) 2024-09-13T10:10:51.260 INFO:tasks.rados.rados.0.smithi195.stdout:145: set_chunk oid 46 target oid 46 low_tier 2024-09-13T10:10:51.261 INFO:tasks.rados.rados.0.smithi195.stdout:145: set_chunk oid 46 offset: 2099200 length: 463872 target oid smithi19530714-46 low_tier offset: 2099200 snap -1 2024-09-13T10:10:51.261 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.262 INFO:tasks.rados.rados.0.smithi195.stdout:138: finishing set_chunk to oid 39 2024-09-13T10:10:51.262 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 9 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-13T10:10:51.262 INFO:tasks.rados.rados.0.smithi195.stdout:133: finishing set_chunk to oid 34 2024-09-13T10:10:51.262 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 34 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-13T10:10:51.262 INFO:tasks.rados.rados.0.smithi195.stdout:139: finishing set_chunk to oid 40 2024-09-13T10:10:51.262 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-13T10:10:51.262 INFO:tasks.rados.rados.0.smithi195.stdout:133: done (15 left) 2024-09-13T10:10:51.262 INFO:tasks.rados.rados.0.smithi195.stdout:138: done (14 left) 2024-09-13T10:10:51.262 INFO:tasks.rados.rados.0.smithi195.stdout:139: done (13 left) 2024-09-13T10:10:51.262 INFO:tasks.rados.rados.0.smithi195.stdout:146: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.262 INFO:tasks.rados.rados.0.smithi195.stdout:146: set_chunk oid 47 offset: 1557504 length: 418304 target oid smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1557504 snap -1 2024-09-13T10:10:51.262 INFO:tasks.rados.rados.0.smithi195.stdout:147: set_chunk oid 48 target oid 48 low_tier 2024-09-13T10:10:51.262 INFO:tasks.rados.rados.0.smithi195.stdout:147: set_chunk oid 48 offset: 1543680 length: 773632 target oid smithi19530714-48 low_tier offset: 1543680 snap -1 2024-09-13T10:10:51.263 INFO:tasks.rados.rados.0.smithi195.stdout:148: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.263 INFO:tasks.rados.rados.0.smithi195.stdout:148: set_chunk oid 49 offset: 837120 length: 1093120 target oid smithi19530714-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 837120 snap -1 2024-09-13T10:10:51.263 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.263 INFO:tasks.rados.rados.0.smithi195.stdout:132: finishing set_chunk to oid 33 2024-09-13T10:10:51.264 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 6 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-13T10:10:51.264 INFO:tasks.rados.rados.0.smithi195.stdout:132: done (15 left) 2024-09-13T10:10:51.264 INFO:tasks.rados.rados.0.smithi195.stdout:149: set_chunk oid 50 target oid 50 low_tier 2024-09-13T10:10:51.264 INFO:tasks.rados.rados.0.smithi195.stdout:149: set_chunk oid 50 offset: 149504 length: 2460672 target oid smithi19530714-50 low_tier offset: 149504 snap -1 2024-09-13T10:10:51.264 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.264 INFO:tasks.rados.rados.0.smithi195.stdout:106: finishing set_chunk to oid 7 2024-09-13T10:10:51.264 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 6 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-13T10:10:51.264 INFO:tasks.rados.rados.0.smithi195.stdout:106: done (15 left) 2024-09-13T10:10:51.264 INFO:tasks.rados.rados.0.smithi195.stdout:150: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.264 INFO:tasks.rados.rados.0.smithi195.stdout:150: set_chunk oid 1 offset: 933888 length: 49152 target oid smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 933888 snap -1 2024-09-13T10:10:51.264 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.266 INFO:tasks.rados.rados.0.smithi195.stdout:117: finishing set_chunk to oid 18 2024-09-13T10:10:51.267 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 12 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-13T10:10:51.267 INFO:tasks.rados.rados.0.smithi195.stdout:117: done (15 left) 2024-09-13T10:10:51.267 INFO:tasks.rados.rados.0.smithi195.stdout:151: set_chunk oid 2 target oid 2 low_tier 2024-09-13T10:10:51.267 INFO:tasks.rados.rados.0.smithi195.stdout:151: set_chunk oid 2 offset: 1001472 length: 397824 target oid smithi19530714-2 low_tier offset: 1001472 snap -1 2024-09-13T10:10:51.267 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.267 INFO:tasks.rados.rados.0.smithi195.stdout:120: finishing set_chunk to oid 21 2024-09-13T10:10:51.267 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 9 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-13T10:10:51.267 INFO:tasks.rados.rados.0.smithi195.stdout:120: done (15 left) 2024-09-13T10:10:51.267 INFO:tasks.rados.rados.0.smithi195.stdout:152: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.268 INFO:tasks.rados.rados.0.smithi195.stdout:152: set_chunk oid 3 offset: 1364480 length: 51712 target oid smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1364480 snap -1 2024-09-13T10:10:51.268 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.270 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T10:10:51.272 INFO:tasks.rados.rados.0.smithi195.stdout:151: finishing set_chunk to oid 2 2024-09-13T10:10:51.272 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 2 set_chunk 2 low_tier returned error code -95 offset: 1001472 length: 397824 2024-09-13T10:10:51.273 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-13T10:10:51.273 INFO:tasks.rados.rados.0.smithi195.stdout:141: finishing set_chunk to oid 42 2024-09-13T10:10:51.273 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 6 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-13T10:10:51.273 INFO:tasks.rados.rados.0.smithi195.stdout:141: done (15 left) 2024-09-13T10:10:51.273 INFO:tasks.rados.rados.0.smithi195.stdout:151: done (14 left) 2024-09-13T10:10:51.273 INFO:tasks.rados.rados.0.smithi195.stdout:153: set_chunk oid 4 target oid 4 low_tier 2024-09-13T10:10:51.273 INFO:tasks.rados.rados.0.smithi195.stdout:153: set_chunk oid 4 offset: 609280 length: 393216 target oid smithi19530714-4 low_tier offset: 609280 snap -1 2024-09-13T10:10:51.273 INFO:tasks.rados.rados.0.smithi195.stdout:154: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.273 INFO:tasks.rados.rados.0.smithi195.stdout:154: set_chunk oid 5 offset: 827904 length: 777728 target oid smithi19530714-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 827904 snap -1 2024-09-13T10:10:51.273 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.274 INFO:tasks.rados.rados.0.smithi195.stdout:152: finishing set_chunk to oid 3 2024-09-13T10:10:51.274 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 3 set_chunk 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1364480 length: 51712 2024-09-13T10:10:51.274 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 0 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-13T10:10:51.274 INFO:tasks.rados.rados.0.smithi195.stdout:152: done (15 left) 2024-09-13T10:10:51.274 INFO:tasks.rados.rados.0.smithi195.stdout:155: set_chunk oid 6 target oid 6 low_tier 2024-09-13T10:10:51.274 INFO:tasks.rados.rados.0.smithi195.stdout:155: set_chunk oid 6 offset: 436736 length: 796160 target oid smithi19530714-6 low_tier offset: 436736 snap -1 2024-09-13T10:10:51.274 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.274 INFO:tasks.rados.rados.0.smithi195.stdout:154: finishing set_chunk to oid 5 2024-09-13T10:10:51.275 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 5 set_chunk 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 827904 length: 777728 2024-09-13T10:10:51.275 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 0 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-13T10:10:51.275 INFO:tasks.rados.rados.0.smithi195.stdout:154: done (15 left) 2024-09-13T10:10:51.275 INFO:tasks.rados.rados.0.smithi195.stdout:156: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.275 INFO:tasks.rados.rados.0.smithi195.stdout:156: set_chunk oid 7 offset: 951808 length: 701952 target oid smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 951808 snap -1 2024-09-13T10:10:51.275 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.275 INFO:tasks.rados.rados.0.smithi195.stdout:114: finishing set_chunk to oid 15 2024-09-13T10:10:51.275 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 9 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-13T10:10:51.275 INFO:tasks.rados.rados.0.smithi195.stdout:114: done (15 left) 2024-09-13T10:10:51.275 INFO:tasks.rados.rados.0.smithi195.stdout:157: set_chunk oid 8 target oid 8 low_tier 2024-09-13T10:10:51.275 INFO:tasks.rados.rados.0.smithi195.stdout:157: set_chunk oid 8 offset: 283136 length: 410624 target oid smithi19530714-8 low_tier offset: 283136 snap -1 2024-09-13T10:10:51.276 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.276 INFO:tasks.rados.rados.0.smithi195.stdout:156: finishing set_chunk to oid 7 2024-09-13T10:10:51.276 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 951808 length: 701952 2024-09-13T10:10:51.276 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-13T10:10:51.276 INFO:tasks.rados.rados.0.smithi195.stdout:156: done (15 left) 2024-09-13T10:10:51.276 INFO:tasks.rados.rados.0.smithi195.stdout:158: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.276 INFO:tasks.rados.rados.0.smithi195.stdout:158: set_chunk oid 9 offset: 957440 length: 257024 target oid smithi19530714-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 957440 snap -1 2024-09-13T10:10:51.276 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.276 INFO:tasks.rados.rados.0.smithi195.stdout:146: finishing set_chunk to oid 47 2024-09-13T10:10:51.276 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 9 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-13T10:10:51.276 INFO:tasks.rados.rados.0.smithi195.stdout:146: done (15 left) 2024-09-13T10:10:51.276 INFO:tasks.rados.rados.0.smithi195.stdout:159: set_chunk oid 10 target oid 10 low_tier 2024-09-13T10:10:51.276 INFO:tasks.rados.rados.0.smithi195.stdout:159: set_chunk oid 10 offset: 1302016 length: 466432 target oid smithi19530714-10 low_tier offset: 1302016 snap -1 2024-09-13T10:10:51.277 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.277 INFO:tasks.rados.rados.0.smithi195.stdout:159: finishing set_chunk to oid 10 2024-09-13T10:10:51.277 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 10 set_chunk 10 low_tier returned error code -95 offset: 1302016 length: 466432 2024-09-13T10:10:51.277 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 0 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-13T10:10:51.277 INFO:tasks.rados.rados.0.smithi195.stdout:159: done (15 left) 2024-09-13T10:10:51.277 INFO:tasks.rados.rados.0.smithi195.stdout:160: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.277 INFO:tasks.rados.rados.0.smithi195.stdout:160: set_chunk oid 11 offset: 1119232 length: 450048 target oid smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1119232 snap -1 2024-09-13T10:10:51.277 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.277 INFO:tasks.rados.rados.0.smithi195.stdout:143: finishing set_chunk to oid 44 2024-09-13T10:10:51.277 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 9 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-13T10:10:51.277 INFO:tasks.rados.rados.0.smithi195.stdout:143: done (15 left) 2024-09-13T10:10:51.277 INFO:tasks.rados.rados.0.smithi195.stdout:161: set_chunk oid 12 target oid 12 low_tier 2024-09-13T10:10:51.278 INFO:tasks.rados.rados.0.smithi195.stdout:161: set_chunk oid 12 offset: 956928 length: 286208 target oid smithi19530714-12 low_tier offset: 956928 snap -1 2024-09-13T10:10:51.278 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.291 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-13T10:10:51.300 INFO:tasks.rados.rados.0.smithi195.stdout:161: finishing set_chunk to oid 12 2024-09-13T10:10:51.300 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 12 set_chunk 12 low_tier returned error code -95 offset: 956928 length: 286208 2024-09-13T10:10:51.300 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-13T10:10:51.300 INFO:tasks.rados.rados.0.smithi195.stdout:147: finishing set_chunk to oid 48 2024-09-13T10:10:51.300 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 15 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-13T10:10:51.300 INFO:tasks.rados.rados.0.smithi195.stdout:147: done (15 left) 2024-09-13T10:10:51.300 INFO:tasks.rados.rados.0.smithi195.stdout:161: done (14 left) 2024-09-13T10:10:51.300 INFO:tasks.rados.rados.0.smithi195.stdout:162: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.300 INFO:tasks.rados.rados.0.smithi195.stdout:162: set_chunk oid 13 offset: 475136 length: 1304576 target oid smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 475136 snap -1 2024-09-13T10:10:51.300 INFO:tasks.rados.rados.0.smithi195.stdout:163: set_chunk oid 14 target oid 14 low_tier 2024-09-13T10:10:51.300 INFO:tasks.rados.rados.0.smithi195.stdout:163: set_chunk oid 14 offset: 1759744 length: 235520 target oid smithi19530714-14 low_tier offset: 1759744 snap -1 2024-09-13T10:10:51.301 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.301 INFO:tasks.rados.rados.0.smithi195.stdout:162: finishing set_chunk to oid 13 2024-09-13T10:10:51.301 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 475136 length: 1304576 2024-09-13T10:10:51.301 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-13T10:10:51.301 INFO:tasks.rados.rados.0.smithi195.stdout:162: done (15 left) 2024-09-13T10:10:51.301 INFO:tasks.rados.rados.0.smithi195.stdout:164: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.301 INFO:tasks.rados.rados.0.smithi195.stdout:164: set_chunk oid 15 offset: 1186304 length: 130048 target oid smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1186304 snap -1 2024-09-13T10:10:51.301 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.301 INFO:tasks.rados.rados.0.smithi195.stdout:140: finishing set_chunk to oid 41 2024-09-13T10:10:51.301 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 3 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-13T10:10:51.301 INFO:tasks.rados.rados.0.smithi195.stdout:140: done (15 left) 2024-09-13T10:10:51.301 INFO:tasks.rados.rados.0.smithi195.stdout:165: set_chunk oid 16 target oid 16 low_tier 2024-09-13T10:10:51.302 INFO:tasks.rados.rados.0.smithi195.stdout:165: set_chunk oid 16 offset: 920064 length: 11264 target oid smithi19530714-16 low_tier offset: 920064 snap -1 2024-09-13T10:10:51.302 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.302 INFO:tasks.rados.rados.0.smithi195.stdout:127: finishing set_chunk to oid 28 2024-09-13T10:10:51.302 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 9 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-13T10:10:51.302 INFO:tasks.rados.rados.0.smithi195.stdout:127: done (15 left) 2024-09-13T10:10:51.302 INFO:tasks.rados.rados.0.smithi195.stdout:166: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.302 INFO:tasks.rados.rados.0.smithi195.stdout:166: set_chunk oid 17 offset: 1914880 length: 342528 target oid smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1914880 snap -1 2024-09-13T10:10:51.302 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.302 INFO:tasks.rados.rados.0.smithi195.stdout:166: finishing set_chunk to oid 17 2024-09-13T10:10:51.302 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1914880 length: 342528 2024-09-13T10:10:51.302 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-13T10:10:51.302 INFO:tasks.rados.rados.0.smithi195.stdout:166: done (15 left) 2024-09-13T10:10:51.302 INFO:tasks.rados.rados.0.smithi195.stdout:167: set_chunk oid 18 target oid 18 low_tier 2024-09-13T10:10:51.302 INFO:tasks.rados.rados.0.smithi195.stdout:167: set_chunk oid 18 offset: 20992 length: 1436160 target oid smithi19530714-18 low_tier offset: 20992 snap -1 2024-09-13T10:10:51.303 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.303 INFO:tasks.rados.rados.0.smithi195.stdout:165: finishing set_chunk to oid 16 2024-09-13T10:10:51.303 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 920064 length: 11264 2024-09-13T10:10:51.303 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-13T10:10:51.303 INFO:tasks.rados.rados.0.smithi195.stdout:165: done (15 left) 2024-09-13T10:10:51.303 INFO:tasks.rados.rados.0.smithi195.stdout:168: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.303 INFO:tasks.rados.rados.0.smithi195.stdout:168: set_chunk oid 19 offset: 489472 length: 223232 target oid smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 489472 snap -1 2024-09-13T10:10:51.303 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.303 INFO:tasks.rados.rados.0.smithi195.stdout:167: finishing set_chunk to oid 18 2024-09-13T10:10:51.303 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 20992 length: 1436160 2024-09-13T10:10:51.303 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-13T10:10:51.303 INFO:tasks.rados.rados.0.smithi195.stdout:167: done (15 left) 2024-09-13T10:10:51.303 INFO:tasks.rados.rados.0.smithi195.stdout:169: set_chunk oid 20 target oid 20 low_tier 2024-09-13T10:10:51.304 INFO:tasks.rados.rados.0.smithi195.stdout:169: set_chunk oid 20 offset: 635904 length: 1286144 target oid smithi19530714-20 low_tier offset: 635904 snap -1 2024-09-13T10:10:51.304 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.304 INFO:tasks.rados.rados.0.smithi195.stdout:168: finishing set_chunk to oid 19 2024-09-13T10:10:51.304 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 19 set_chunk 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 489472 length: 223232 2024-09-13T10:10:51.304 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 0 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-13T10:10:51.304 INFO:tasks.rados.rados.0.smithi195.stdout:168: done (15 left) 2024-09-13T10:10:51.304 INFO:tasks.rados.rados.0.smithi195.stdout:170: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.304 INFO:tasks.rados.rados.0.smithi195.stdout:170: set_chunk oid 21 offset: 225792 length: 378368 target oid smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 225792 snap -1 2024-09-13T10:10:51.304 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.304 INFO:tasks.rados.rados.0.smithi195.stdout:169: finishing set_chunk to oid 20 2024-09-13T10:10:51.304 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 635904 length: 1286144 2024-09-13T10:10:51.304 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-13T10:10:51.304 INFO:tasks.rados.rados.0.smithi195.stdout:155: finishing set_chunk to oid 6 2024-09-13T10:10:51.305 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 6 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-13T10:10:51.305 INFO:tasks.rados.rados.0.smithi195.stdout:155: done (15 left) 2024-09-13T10:10:51.305 INFO:tasks.rados.rados.0.smithi195.stdout:169: done (14 left) 2024-09-13T10:10:51.305 INFO:tasks.rados.rados.0.smithi195.stdout:171: set_chunk oid 22 target oid 22 low_tier 2024-09-13T10:10:51.305 INFO:tasks.rados.rados.0.smithi195.stdout:171: set_chunk oid 22 offset: 629760 length: 1496576 target oid smithi19530714-22 low_tier offset: 629760 snap -1 2024-09-13T10:10:51.305 INFO:tasks.rados.rados.0.smithi195.stdout:172: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.305 INFO:tasks.rados.rados.0.smithi195.stdout:172: set_chunk oid 23 offset: 1334272 length: 393216 target oid smithi19530714-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1334272 snap -1 2024-09-13T10:10:51.305 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.305 INFO:tasks.rados.rados.0.smithi195.stdout:170: finishing set_chunk to oid 21 2024-09-13T10:10:51.305 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 21 set_chunk 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 225792 length: 378368 2024-09-13T10:10:51.305 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 0 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-13T10:10:51.305 INFO:tasks.rados.rados.0.smithi195.stdout:170: done (15 left) 2024-09-13T10:10:51.306 INFO:tasks.rados.rados.0.smithi195.stdout:173: set_chunk oid 24 target oid 24 low_tier 2024-09-13T10:10:51.306 INFO:tasks.rados.rados.0.smithi195.stdout:173: set_chunk oid 24 offset: 577024 length: 1248768 target oid smithi19530714-24 low_tier offset: 577024 snap -1 2024-09-13T10:10:51.306 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.306 INFO:tasks.rados.rados.0.smithi195.stdout:171: finishing set_chunk to oid 22 2024-09-13T10:10:51.306 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 629760 length: 1496576 2024-09-13T10:10:51.306 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-13T10:10:51.306 INFO:tasks.rados.rados.0.smithi195.stdout:171: done (15 left) 2024-09-13T10:10:51.306 INFO:tasks.rados.rados.0.smithi195.stdout:174: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.306 INFO:tasks.rados.rados.0.smithi195.stdout:174: set_chunk oid 25 offset: 841728 length: 678400 target oid smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 841728 snap -1 2024-09-13T10:10:51.306 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.306 INFO:tasks.rados.rados.0.smithi195.stdout:142: finishing set_chunk to oid 43 2024-09-13T10:10:51.306 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 9 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-13T10:10:51.306 INFO:tasks.rados.rados.0.smithi195.stdout:142: done (15 left) 2024-09-13T10:10:51.306 INFO:tasks.rados.rados.0.smithi195.stdout:175: set_chunk oid 26 target oid 26 low_tier 2024-09-13T10:10:51.307 INFO:tasks.rados.rados.0.smithi195.stdout:175: set_chunk oid 26 offset: 184320 length: 1456640 target oid smithi19530714-26 low_tier offset: 184320 snap -1 2024-09-13T10:10:51.307 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.307 INFO:tasks.rados.rados.0.smithi195.stdout:136: finishing set_chunk to oid 37 2024-09-13T10:10:51.307 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 3 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-13T10:10:51.307 INFO:tasks.rados.rados.0.smithi195.stdout:173: finishing set_chunk to oid 24 2024-09-13T10:10:51.307 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 24 set_chunk 24 low_tier returned error code -95 offset: 577024 length: 1248768 2024-09-13T10:10:51.307 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 0 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-13T10:10:51.307 INFO:tasks.rados.rados.0.smithi195.stdout:136: done (15 left) 2024-09-13T10:10:51.307 INFO:tasks.rados.rados.0.smithi195.stdout:173: done (14 left) 2024-09-13T10:10:51.307 INFO:tasks.rados.rados.0.smithi195.stdout:176: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.307 INFO:tasks.rados.rados.0.smithi195.stdout:176: set_chunk oid 27 offset: 896000 length: 1035776 target oid smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 896000 snap -1 2024-09-13T10:10:51.307 INFO:tasks.rados.rados.0.smithi195.stdout:177: set_chunk oid 28 target oid 28 low_tier 2024-09-13T10:10:51.308 INFO:tasks.rados.rados.0.smithi195.stdout:177: set_chunk oid 28 offset: 393216 length: 1541120 target oid smithi19530714-28 low_tier offset: 393216 snap -1 2024-09-13T10:10:51.308 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.308 INFO:tasks.rados.rados.0.smithi195.stdout:144: finishing set_chunk to oid 45 2024-09-13T10:10:51.308 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 15 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-13T10:10:51.308 INFO:tasks.rados.rados.0.smithi195.stdout:144: done (15 left) 2024-09-13T10:10:51.308 INFO:tasks.rados.rados.0.smithi195.stdout:178: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.308 INFO:tasks.rados.rados.0.smithi195.stdout:178: set_chunk oid 29 offset: 708608 length: 1098752 target oid smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 708608 snap -1 2024-09-13T10:10:51.308 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.308 INFO:tasks.rados.rados.0.smithi195.stdout:145: finishing set_chunk to oid 46 2024-09-13T10:10:51.308 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 6 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-13T10:10:51.308 INFO:tasks.rados.rados.0.smithi195.stdout:145: done (15 left) 2024-09-13T10:10:51.308 INFO:tasks.rados.rados.0.smithi195.stdout:179: set_chunk oid 30 target oid 30 low_tier 2024-09-13T10:10:51.308 INFO:tasks.rados.rados.0.smithi195.stdout:179: set_chunk oid 30 offset: 928768 length: 1375744 target oid smithi19530714-30 low_tier offset: 928768 snap -1 2024-09-13T10:10:51.309 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.309 INFO:tasks.rados.rados.0.smithi195.stdout:175: finishing set_chunk to oid 26 2024-09-13T10:10:51.309 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 26 set_chunk 26 low_tier returned error code -95 offset: 184320 length: 1456640 2024-09-13T10:10:51.309 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 0 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-13T10:10:51.309 INFO:tasks.rados.rados.0.smithi195.stdout:175: done (15 left) 2024-09-13T10:10:51.309 INFO:tasks.rados.rados.0.smithi195.stdout:180: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.309 INFO:tasks.rados.rados.0.smithi195.stdout:180: set_chunk oid 31 offset: 485376 length: 1436672 target oid smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 485376 snap -1 2024-09-13T10:10:51.309 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.309 INFO:tasks.rados.rados.0.smithi195.stdout:176: finishing set_chunk to oid 27 2024-09-13T10:10:51.309 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 27 set_chunk 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 896000 length: 1035776 2024-09-13T10:10:51.309 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-13T10:10:51.309 INFO:tasks.rados.rados.0.smithi195.stdout:176: done (15 left) 2024-09-13T10:10:51.309 INFO:tasks.rados.rados.0.smithi195.stdout:181: set_chunk oid 32 target oid 32 low_tier 2024-09-13T10:10:51.309 INFO:tasks.rados.rados.0.smithi195.stdout:181: set_chunk oid 32 offset: 872960 length: 95232 target oid smithi19530714-32 low_tier offset: 872960 snap -1 2024-09-13T10:10:51.310 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.310 INFO:tasks.rados.rados.0.smithi195.stdout:180: finishing set_chunk to oid 31 2024-09-13T10:10:51.310 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 31 set_chunk 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 485376 length: 1436672 2024-09-13T10:10:51.310 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-13T10:10:51.310 INFO:tasks.rados.rados.0.smithi195.stdout:180: done (15 left) 2024-09-13T10:10:51.310 INFO:tasks.rados.rados.0.smithi195.stdout:182: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.310 INFO:tasks.rados.rados.0.smithi195.stdout:182: set_chunk oid 33 offset: 262656 length: 1290752 target oid smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 262656 snap -1 2024-09-13T10:10:51.310 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.310 INFO:tasks.rados.rados.0.smithi195.stdout:178: finishing set_chunk to oid 29 2024-09-13T10:10:51.310 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 29 set_chunk 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 708608 length: 1098752 2024-09-13T10:10:51.310 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 0 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-13T10:10:51.310 INFO:tasks.rados.rados.0.smithi195.stdout:179: finishing set_chunk to oid 30 2024-09-13T10:10:51.310 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 30 set_chunk 30 low_tier returned error code -95 offset: 928768 length: 1375744 2024-09-13T10:10:51.310 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 0 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-13T10:10:51.311 INFO:tasks.rados.rados.0.smithi195.stdout:178: done (15 left) 2024-09-13T10:10:51.311 INFO:tasks.rados.rados.0.smithi195.stdout:179: done (14 left) 2024-09-13T10:10:51.311 INFO:tasks.rados.rados.0.smithi195.stdout:183: set_chunk oid 34 target oid 34 low_tier 2024-09-13T10:10:51.311 INFO:tasks.rados.rados.0.smithi195.stdout:183: set_chunk oid 34 offset: 1428992 length: 1006080 target oid smithi19530714-34 low_tier offset: 1428992 snap -1 2024-09-13T10:10:51.311 INFO:tasks.rados.rados.0.smithi195.stdout:184: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.311 INFO:tasks.rados.rados.0.smithi195.stdout:184: set_chunk oid 35 offset: 61440 length: 2357248 target oid smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 61440 snap -1 2024-09-13T10:10:51.311 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.311 INFO:tasks.rados.rados.0.smithi195.stdout:181: finishing set_chunk to oid 32 2024-09-13T10:10:51.311 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 872960 length: 95232 2024-09-13T10:10:51.311 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-13T10:10:51.311 INFO:tasks.rados.rados.0.smithi195.stdout:181: done (15 left) 2024-09-13T10:10:51.311 INFO:tasks.rados.rados.0.smithi195.stdout:185: set_chunk oid 36 target oid 36 low_tier 2024-09-13T10:10:51.311 INFO:tasks.rados.rados.0.smithi195.stdout:185: set_chunk oid 36 offset: 1261056 length: 50688 target oid smithi19530714-36 low_tier offset: 1261056 snap -1 2024-09-13T10:10:51.312 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.312 INFO:tasks.rados.rados.0.smithi195.stdout:184: finishing set_chunk to oid 35 2024-09-13T10:10:51.312 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 35 set_chunk 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 61440 length: 2357248 2024-09-13T10:10:51.312 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 0 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-13T10:10:51.312 INFO:tasks.rados.rados.0.smithi195.stdout:182: finishing set_chunk to oid 33 2024-09-13T10:10:51.312 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 33 set_chunk 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 262656 length: 1290752 2024-09-13T10:10:51.312 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 0 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-13T10:10:51.312 INFO:tasks.rados.rados.0.smithi195.stdout:182: done (15 left) 2024-09-13T10:10:51.312 INFO:tasks.rados.rados.0.smithi195.stdout:184: done (14 left) 2024-09-13T10:10:51.312 INFO:tasks.rados.rados.0.smithi195.stdout:186: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.312 INFO:tasks.rados.rados.0.smithi195.stdout:186: set_chunk oid 37 offset: 326144 length: 787968 target oid smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 326144 snap -1 2024-09-13T10:10:51.312 INFO:tasks.rados.rados.0.smithi195.stdout:187: set_chunk oid 38 target oid 38 low_tier 2024-09-13T10:10:51.313 INFO:tasks.rados.rados.0.smithi195.stdout:187: set_chunk oid 38 offset: 1901056 length: 353280 target oid smithi19530714-38 low_tier offset: 1901056 snap -1 2024-09-13T10:10:51.313 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.313 INFO:tasks.rados.rados.0.smithi195.stdout:150: finishing set_chunk to oid 1 2024-09-13T10:10:51.313 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-13T10:10:51.313 INFO:tasks.rados.rados.0.smithi195.stdout:150: done (15 left) 2024-09-13T10:10:51.313 INFO:tasks.rados.rados.0.smithi195.stdout:188: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.313 INFO:tasks.rados.rados.0.smithi195.stdout:188: set_chunk oid 39 offset: 260608 length: 1477632 target oid smithi19530714-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 260608 snap -1 2024-09-13T10:10:51.313 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.313 INFO:tasks.rados.rados.0.smithi195.stdout:157: finishing set_chunk to oid 8 2024-09-13T10:10:51.313 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 3 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-13T10:10:51.313 INFO:tasks.rados.rados.0.smithi195.stdout:157: done (15 left) 2024-09-13T10:10:51.313 INFO:tasks.rados.rados.0.smithi195.stdout:189: set_chunk oid 40 target oid 40 low_tier 2024-09-13T10:10:51.313 INFO:tasks.rados.rados.0.smithi195.stdout:189: set_chunk oid 40 offset: 809984 length: 306176 target oid smithi19530714-40 low_tier offset: 809984 snap -1 2024-09-13T10:10:51.314 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.314 INFO:tasks.rados.rados.0.smithi195.stdout:186: finishing set_chunk to oid 37 2024-09-13T10:10:51.314 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 37 set_chunk 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 326144 length: 787968 2024-09-13T10:10:51.314 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 0 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-13T10:10:51.314 INFO:tasks.rados.rados.0.smithi195.stdout:186: done (15 left) 2024-09-13T10:10:51.314 INFO:tasks.rados.rados.0.smithi195.stdout:190: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.314 INFO:tasks.rados.rados.0.smithi195.stdout:190: set_chunk oid 41 offset: 660992 length: 395264 target oid smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 660992 snap -1 2024-09-13T10:10:51.314 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.314 INFO:tasks.rados.rados.0.smithi195.stdout:189: finishing set_chunk to oid 40 2024-09-13T10:10:51.314 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 40 set_chunk 40 low_tier returned error code -95 offset: 809984 length: 306176 2024-09-13T10:10:51.314 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 0 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-13T10:10:51.314 INFO:tasks.rados.rados.0.smithi195.stdout:189: done (15 left) 2024-09-13T10:10:51.314 INFO:tasks.rados.rados.0.smithi195.stdout:191: set_chunk oid 42 target oid 42 low_tier 2024-09-13T10:10:51.314 INFO:tasks.rados.rados.0.smithi195.stdout:191: set_chunk oid 42 offset: 568320 length: 1748992 target oid smithi19530714-42 low_tier offset: 568320 snap -1 2024-09-13T10:10:51.315 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.315 INFO:tasks.rados.rados.0.smithi195.stdout:188: finishing set_chunk to oid 39 2024-09-13T10:10:51.315 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 260608 length: 1477632 2024-09-13T10:10:51.315 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-13T10:10:51.315 INFO:tasks.rados.rados.0.smithi195.stdout:188: done (15 left) 2024-09-13T10:10:51.315 INFO:tasks.rados.rados.0.smithi195.stdout:192: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.315 INFO:tasks.rados.rados.0.smithi195.stdout:192: set_chunk oid 43 offset: 801792 length: 1325568 target oid smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 801792 snap -1 2024-09-13T10:10:51.315 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.315 INFO:tasks.rados.rados.0.smithi195.stdout:192: finishing set_chunk to oid 43 2024-09-13T10:10:51.315 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 801792 length: 1325568 2024-09-13T10:10:51.315 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-13T10:10:51.315 INFO:tasks.rados.rados.0.smithi195.stdout:192: done (15 left) 2024-09-13T10:10:51.315 INFO:tasks.rados.rados.0.smithi195.stdout:193: set_chunk oid 44 target oid 44 low_tier 2024-09-13T10:10:51.316 INFO:tasks.rados.rados.0.smithi195.stdout:193: set_chunk oid 44 offset: 1265152 length: 542208 target oid smithi19530714-44 low_tier offset: 1265152 snap -1 2024-09-13T10:10:51.316 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.316 INFO:tasks.rados.rados.0.smithi195.stdout:149: finishing set_chunk to oid 50 2024-09-13T10:10:51.316 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 9 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-13T10:10:51.316 INFO:tasks.rados.rados.0.smithi195.stdout:191: finishing set_chunk to oid 42 2024-09-13T10:10:51.316 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 42 set_chunk 42 low_tier returned error code -95 offset: 568320 length: 1748992 2024-09-13T10:10:51.316 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-13T10:10:51.316 INFO:tasks.rados.rados.0.smithi195.stdout:149: done (15 left) 2024-09-13T10:10:51.316 INFO:tasks.rados.rados.0.smithi195.stdout:191: done (14 left) 2024-09-13T10:10:51.316 INFO:tasks.rados.rados.0.smithi195.stdout:194: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.316 INFO:tasks.rados.rados.0.smithi195.stdout:194: set_chunk oid 45 offset: 915968 length: 1623552 target oid smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 915968 snap -1 2024-09-13T10:10:51.316 INFO:tasks.rados.rados.0.smithi195.stdout:195: set_chunk oid 46 target oid 46 low_tier 2024-09-13T10:10:51.317 INFO:tasks.rados.rados.0.smithi195.stdout:195: set_chunk oid 46 offset: 16384 length: 644096 target oid smithi19530714-46 low_tier offset: 16384 snap -1 2024-09-13T10:10:51.317 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.317 INFO:tasks.rados.rados.0.smithi195.stdout:158: finishing set_chunk to oid 9 2024-09-13T10:10:51.317 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 6 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-13T10:10:51.317 INFO:tasks.rados.rados.0.smithi195.stdout:158: done (15 left) 2024-09-13T10:10:51.317 INFO:tasks.rados.rados.0.smithi195.stdout:196: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.317 INFO:tasks.rados.rados.0.smithi195.stdout:196: set_chunk oid 47 offset: 228352 length: 962560 target oid smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 228352 snap -1 2024-09-13T10:10:51.317 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.317 INFO:tasks.rados.rados.0.smithi195.stdout:148: finishing set_chunk to oid 49 2024-09-13T10:10:51.317 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 18 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-13T10:10:51.317 INFO:tasks.rados.rados.0.smithi195.stdout:148: done (15 left) 2024-09-13T10:10:51.317 INFO:tasks.rados.rados.0.smithi195.stdout:197: set_chunk oid 48 target oid 48 low_tier 2024-09-13T10:10:51.318 INFO:tasks.rados.rados.0.smithi195.stdout:197: set_chunk oid 48 offset: 1217536 length: 1225216 target oid smithi19530714-48 low_tier offset: 1217536 snap -1 2024-09-13T10:10:51.318 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.318 INFO:tasks.rados.rados.0.smithi195.stdout:160: finishing set_chunk to oid 11 2024-09-13T10:10:51.318 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 3 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-13T10:10:51.318 INFO:tasks.rados.rados.0.smithi195.stdout:160: done (15 left) 2024-09-13T10:10:51.318 INFO:tasks.rados.rados.0.smithi195.stdout:198: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.318 INFO:tasks.rados.rados.0.smithi195.stdout:198: set_chunk oid 49 offset: 1582592 length: 653312 target oid smithi19530714-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1582592 snap -1 2024-09-13T10:10:51.318 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.318 INFO:tasks.rados.rados.0.smithi195.stdout:172: finishing set_chunk to oid 23 2024-09-13T10:10:51.318 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 3 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-13T10:10:51.318 INFO:tasks.rados.rados.0.smithi195.stdout:172: done (15 left) 2024-09-13T10:10:51.318 INFO:tasks.rados.rados.0.smithi195.stdout:199: set_chunk oid 50 target oid 50 low_tier 2024-09-13T10:10:51.318 INFO:tasks.rados.rados.0.smithi195.stdout:199: set_chunk oid 50 offset: 52736 length: 332800 target oid smithi19530714-50 low_tier offset: 52736 snap -1 2024-09-13T10:10:51.318 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.318 INFO:tasks.rados.rados.0.smithi195.stdout:153: finishing set_chunk to oid 4 2024-09-13T10:10:51.319 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 3 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-13T10:10:51.319 INFO:tasks.rados.rados.0.smithi195.stdout:153: done (15 left) 2024-09-13T10:10:51.319 INFO:tasks.rados.rados.0.smithi195.stdout:200: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.319 INFO:tasks.rados.rados.0.smithi195.stdout:200: set_chunk oid 1 offset: 869376 length: 1138688 target oid smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 869376 snap -1 2024-09-13T10:10:51.319 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.319 INFO:tasks.rados.rados.0.smithi195.stdout:197: finishing set_chunk to oid 48 2024-09-13T10:10:51.319 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 48 set_chunk 48 low_tier returned error code -95 offset: 1217536 length: 1225216 2024-09-13T10:10:51.319 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-13T10:10:51.319 INFO:tasks.rados.rados.0.smithi195.stdout:197: done (15 left) 2024-09-13T10:10:51.319 INFO:tasks.rados.rados.0.smithi195.stdout:201: set_chunk oid 2 target oid 2 low_tier 2024-09-13T10:10:51.319 INFO:tasks.rados.rados.0.smithi195.stdout:201: set_chunk oid 2 offset: 1195008 length: 273920 target oid smithi19530714-2 low_tier offset: 1195008 snap -1 2024-09-13T10:10:51.319 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.320 INFO:tasks.rados.rados.0.smithi195.stdout:199: finishing set_chunk to oid 50 2024-09-13T10:10:51.320 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 50 set_chunk 50 low_tier returned error code -95 offset: 52736 length: 332800 2024-09-13T10:10:51.320 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 0 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-13T10:10:51.320 INFO:tasks.rados.rados.0.smithi195.stdout:199: done (15 left) 2024-09-13T10:10:51.320 INFO:tasks.rados.rados.0.smithi195.stdout:202: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.320 INFO:tasks.rados.rados.0.smithi195.stdout:202: set_chunk oid 3 offset: 608768 length: 210944 target oid smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 608768 snap -1 2024-09-13T10:10:51.320 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.320 INFO:tasks.rados.rados.0.smithi195.stdout:200: finishing set_chunk to oid 1 2024-09-13T10:10:51.320 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 1 set_chunk 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 869376 length: 1138688 2024-09-13T10:10:51.320 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 0 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-13T10:10:51.320 INFO:tasks.rados.rados.0.smithi195.stdout:200: done (15 left) 2024-09-13T10:10:51.320 INFO:tasks.rados.rados.0.smithi195.stdout:203: set_chunk oid 4 target oid 4 low_tier 2024-09-13T10:10:51.320 INFO:tasks.rados.rados.0.smithi195.stdout:203: set_chunk oid 4 offset: 547328 length: 1396736 target oid smithi19530714-4 low_tier offset: 547328 snap -1 2024-09-13T10:10:51.321 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.321 INFO:tasks.rados.rados.0.smithi195.stdout:203: finishing set_chunk to oid 4 2024-09-13T10:10:51.321 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 4 set_chunk 4 low_tier returned error code -95 offset: 547328 length: 1396736 2024-09-13T10:10:51.321 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 0 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-13T10:10:51.321 INFO:tasks.rados.rados.0.smithi195.stdout:203: done (15 left) 2024-09-13T10:10:51.321 INFO:tasks.rados.rados.0.smithi195.stdout:204: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.321 INFO:tasks.rados.rados.0.smithi195.stdout:204: set_chunk oid 5 offset: 201216 length: 1024512 target oid smithi19530714-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 201216 snap -1 2024-09-13T10:10:51.321 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.321 INFO:tasks.rados.rados.0.smithi195.stdout:204: finishing set_chunk to oid 5 2024-09-13T10:10:51.321 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 5 set_chunk 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 201216 length: 1024512 2024-09-13T10:10:51.321 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 0 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-13T10:10:51.321 INFO:tasks.rados.rados.0.smithi195.stdout:204: done (15 left) 2024-09-13T10:10:51.321 INFO:tasks.rados.rados.0.smithi195.stdout:205: set_chunk oid 6 target oid 6 low_tier 2024-09-13T10:10:51.322 INFO:tasks.rados.rados.0.smithi195.stdout:205: set_chunk oid 6 offset: 426496 length: 1378304 target oid smithi19530714-6 low_tier offset: 426496 snap -1 2024-09-13T10:10:51.322 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.322 INFO:tasks.rados.rados.0.smithi195.stdout:205: finishing set_chunk to oid 6 2024-09-13T10:10:51.322 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 6 set_chunk 6 low_tier returned error code -95 offset: 426496 length: 1378304 2024-09-13T10:10:51.322 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 0 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-13T10:10:51.322 INFO:tasks.rados.rados.0.smithi195.stdout:205: done (15 left) 2024-09-13T10:10:51.322 INFO:tasks.rados.rados.0.smithi195.stdout:206: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.322 INFO:tasks.rados.rados.0.smithi195.stdout:206: set_chunk oid 7 offset: 903680 length: 608256 target oid smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 903680 snap -1 2024-09-13T10:10:51.322 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.322 INFO:tasks.rados.rados.0.smithi195.stdout:206: finishing set_chunk to oid 7 2024-09-13T10:10:51.322 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 903680 length: 608256 2024-09-13T10:10:51.322 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-13T10:10:51.322 INFO:tasks.rados.rados.0.smithi195.stdout:206: done (15 left) 2024-09-13T10:10:51.322 INFO:tasks.rados.rados.0.smithi195.stdout:207: set_chunk oid 8 target oid 8 low_tier 2024-09-13T10:10:51.323 INFO:tasks.rados.rados.0.smithi195.stdout:207: set_chunk oid 8 offset: 1033728 length: 404480 target oid smithi19530714-8 low_tier offset: 1033728 snap -1 2024-09-13T10:10:51.323 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.323 INFO:tasks.rados.rados.0.smithi195.stdout:207: finishing set_chunk to oid 8 2024-09-13T10:10:51.323 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 8 set_chunk 8 low_tier returned error code -95 offset: 1033728 length: 404480 2024-09-13T10:10:51.323 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-13T10:10:51.323 INFO:tasks.rados.rados.0.smithi195.stdout:207: done (15 left) 2024-09-13T10:10:51.323 INFO:tasks.rados.rados.0.smithi195.stdout:208: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.323 INFO:tasks.rados.rados.0.smithi195.stdout:208: set_chunk oid 9 offset: 297984 length: 829440 target oid smithi19530714-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 297984 snap -1 2024-09-13T10:10:51.323 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.323 INFO:tasks.rados.rados.0.smithi195.stdout:183: finishing set_chunk to oid 34 2024-09-13T10:10:51.323 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 34 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-13T10:10:51.323 INFO:tasks.rados.rados.0.smithi195.stdout:183: done (15 left) 2024-09-13T10:10:51.323 INFO:tasks.rados.rados.0.smithi195.stdout:209: set_chunk oid 10 target oid 10 low_tier 2024-09-13T10:10:51.323 INFO:tasks.rados.rados.0.smithi195.stdout:209: set_chunk oid 10 offset: 700416 length: 1409024 target oid smithi19530714-10 low_tier offset: 700416 snap -1 2024-09-13T10:10:51.324 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.324 INFO:tasks.rados.rados.0.smithi195.stdout:190: finishing set_chunk to oid 41 2024-09-13T10:10:51.324 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 3 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-13T10:10:51.324 INFO:tasks.rados.rados.0.smithi195.stdout:190: done (15 left) 2024-09-13T10:10:51.324 INFO:tasks.rados.rados.0.smithi195.stdout:210: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.324 INFO:tasks.rados.rados.0.smithi195.stdout:210: set_chunk oid 11 offset: 159744 length: 520704 target oid smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 159744 snap -1 2024-09-13T10:10:51.324 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.324 INFO:tasks.rados.rados.0.smithi195.stdout:164: finishing set_chunk to oid 15 2024-09-13T10:10:51.324 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 9 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-13T10:10:51.324 INFO:tasks.rados.rados.0.smithi195.stdout:164: done (15 left) 2024-09-13T10:10:51.324 INFO:tasks.rados.rados.0.smithi195.stdout:211: set_chunk oid 12 target oid 12 low_tier 2024-09-13T10:10:51.324 INFO:tasks.rados.rados.0.smithi195.stdout:211: set_chunk oid 12 offset: 695296 length: 1366528 target oid smithi19530714-12 low_tier offset: 695296 snap -1 2024-09-13T10:10:51.325 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.325 INFO:tasks.rados.rados.0.smithi195.stdout:196: finishing set_chunk to oid 47 2024-09-13T10:10:51.325 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 9 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-13T10:10:51.325 INFO:tasks.rados.rados.0.smithi195.stdout:196: done (15 left) 2024-09-13T10:10:51.325 INFO:tasks.rados.rados.0.smithi195.stdout:212: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.325 INFO:tasks.rados.rados.0.smithi195.stdout:212: set_chunk oid 13 offset: 390656 length: 701952 target oid smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 390656 snap -1 2024-09-13T10:10:51.325 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.325 INFO:tasks.rados.rados.0.smithi195.stdout:163: finishing set_chunk to oid 14 2024-09-13T10:10:51.325 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 6 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-13T10:10:51.325 INFO:tasks.rados.rados.0.smithi195.stdout:163: done (15 left) 2024-09-13T10:10:51.325 INFO:tasks.rados.rados.0.smithi195.stdout:213: set_chunk oid 14 target oid 14 low_tier 2024-09-13T10:10:51.325 INFO:tasks.rados.rados.0.smithi195.stdout:213: set_chunk oid 14 offset: 239104 length: 920064 target oid smithi19530714-14 low_tier offset: 239104 snap -1 2024-09-13T10:10:51.326 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.326 INFO:tasks.rados.rados.0.smithi195.stdout:195: finishing set_chunk to oid 46 2024-09-13T10:10:51.326 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 6 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-13T10:10:51.326 INFO:tasks.rados.rados.0.smithi195.stdout:195: done (15 left) 2024-09-13T10:10:51.326 INFO:tasks.rados.rados.0.smithi195.stdout:214: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.326 INFO:tasks.rados.rados.0.smithi195.stdout:214: set_chunk oid 15 offset: 237568 length: 828928 target oid smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 237568 snap -1 2024-09-13T10:10:51.326 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.326 INFO:tasks.rados.rados.0.smithi195.stdout:211: finishing set_chunk to oid 12 2024-09-13T10:10:51.326 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 12 set_chunk 12 low_tier returned error code -95 offset: 695296 length: 1366528 2024-09-13T10:10:51.326 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-13T10:10:51.326 INFO:tasks.rados.rados.0.smithi195.stdout:211: done (15 left) 2024-09-13T10:10:51.326 INFO:tasks.rados.rados.0.smithi195.stdout:215: set_chunk oid 16 target oid 16 low_tier 2024-09-13T10:10:51.327 INFO:tasks.rados.rados.0.smithi195.stdout:215: set_chunk oid 16 offset: 139264 length: 953344 target oid smithi19530714-16 low_tier offset: 139264 snap -1 2024-09-13T10:10:51.327 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.327 INFO:tasks.rados.rados.0.smithi195.stdout:202: finishing set_chunk to oid 3 2024-09-13T10:10:51.327 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 3 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-13T10:10:51.327 INFO:tasks.rados.rados.0.smithi195.stdout:202: done (15 left) 2024-09-13T10:10:51.327 INFO:tasks.rados.rados.0.smithi195.stdout:216: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.327 INFO:tasks.rados.rados.0.smithi195.stdout:216: set_chunk oid 17 offset: 329216 length: 567808 target oid smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 329216 snap -1 2024-09-13T10:10:51.327 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.327 INFO:tasks.rados.rados.0.smithi195.stdout:215: finishing set_chunk to oid 16 2024-09-13T10:10:51.327 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 139264 length: 953344 2024-09-13T10:10:51.327 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-13T10:10:51.327 INFO:tasks.rados.rados.0.smithi195.stdout:215: done (15 left) 2024-09-13T10:10:51.327 INFO:tasks.rados.rados.0.smithi195.stdout:217: set_chunk oid 18 target oid 18 low_tier 2024-09-13T10:10:51.328 INFO:tasks.rados.rados.0.smithi195.stdout:217: set_chunk oid 18 offset: 494592 length: 906752 target oid smithi19530714-18 low_tier offset: 494592 snap -1 2024-09-13T10:10:51.328 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.328 INFO:tasks.rados.rados.0.smithi195.stdout:209: finishing set_chunk to oid 10 2024-09-13T10:10:51.328 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 10 set_chunk 10 low_tier returned error code -95 offset: 700416 length: 1409024 2024-09-13T10:10:51.328 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 0 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-13T10:10:51.328 INFO:tasks.rados.rados.0.smithi195.stdout:216: finishing set_chunk to oid 17 2024-09-13T10:10:51.328 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 329216 length: 567808 2024-09-13T10:10:51.328 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-13T10:10:51.328 INFO:tasks.rados.rados.0.smithi195.stdout:209: done (15 left) 2024-09-13T10:10:51.328 INFO:tasks.rados.rados.0.smithi195.stdout:216: done (14 left) 2024-09-13T10:10:51.328 INFO:tasks.rados.rados.0.smithi195.stdout:218: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.328 INFO:tasks.rados.rados.0.smithi195.stdout:218: set_chunk oid 19 offset: 496128 length: 1413120 target oid smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 496128 snap -1 2024-09-13T10:10:51.328 INFO:tasks.rados.rados.0.smithi195.stdout:219: set_chunk oid 20 target oid 20 low_tier 2024-09-13T10:10:51.329 INFO:tasks.rados.rados.0.smithi195.stdout:219: set_chunk oid 20 offset: 478208 length: 1342976 target oid smithi19530714-20 low_tier offset: 478208 snap -1 2024-09-13T10:10:51.329 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.329 INFO:tasks.rados.rados.0.smithi195.stdout:217: finishing set_chunk to oid 18 2024-09-13T10:10:51.329 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 494592 length: 906752 2024-09-13T10:10:51.329 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-13T10:10:51.329 INFO:tasks.rados.rados.0.smithi195.stdout:217: done (15 left) 2024-09-13T10:10:51.329 INFO:tasks.rados.rados.0.smithi195.stdout:220: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.329 INFO:tasks.rados.rados.0.smithi195.stdout:220: set_chunk oid 21 offset: 1243648 length: 1088000 target oid smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1243648 snap -1 2024-09-13T10:10:51.329 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.329 INFO:tasks.rados.rados.0.smithi195.stdout:219: finishing set_chunk to oid 20 2024-09-13T10:10:51.329 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 478208 length: 1342976 2024-09-13T10:10:51.329 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-13T10:10:51.330 INFO:tasks.rados.rados.0.smithi195.stdout:219: done (15 left) 2024-09-13T10:10:51.330 INFO:tasks.rados.rados.0.smithi195.stdout:221: set_chunk oid 22 target oid 22 low_tier 2024-09-13T10:10:51.330 INFO:tasks.rados.rados.0.smithi195.stdout:221: set_chunk oid 22 offset: 1090560 length: 353792 target oid smithi19530714-22 low_tier offset: 1090560 snap -1 2024-09-13T10:10:51.330 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.330 INFO:tasks.rados.rados.0.smithi195.stdout:218: finishing set_chunk to oid 19 2024-09-13T10:10:51.330 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 19 set_chunk 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 496128 length: 1413120 2024-09-13T10:10:51.330 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 0 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-13T10:10:51.330 INFO:tasks.rados.rados.0.smithi195.stdout:218: done (15 left) 2024-09-13T10:10:51.330 INFO:tasks.rados.rados.0.smithi195.stdout:222: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.330 INFO:tasks.rados.rados.0.smithi195.stdout:222: set_chunk oid 23 offset: 2032640 length: 309248 target oid smithi19530714-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 2032640 snap -1 2024-09-13T10:10:51.330 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.330 INFO:tasks.rados.rados.0.smithi195.stdout:212: finishing set_chunk to oid 13 2024-09-13T10:10:51.330 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 390656 length: 701952 2024-09-13T10:10:51.330 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-13T10:10:51.331 INFO:tasks.rados.rados.0.smithi195.stdout:212: done (15 left) 2024-09-13T10:10:51.331 INFO:tasks.rados.rados.0.smithi195.stdout:223: set_chunk oid 24 target oid 24 low_tier 2024-09-13T10:10:51.331 INFO:tasks.rados.rados.0.smithi195.stdout:223: set_chunk oid 24 offset: 317952 length: 1897472 target oid smithi19530714-24 low_tier offset: 317952 snap -1 2024-09-13T10:10:51.331 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.331 INFO:tasks.rados.rados.0.smithi195.stdout:213: finishing set_chunk to oid 14 2024-09-13T10:10:51.331 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 6 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-13T10:10:51.331 INFO:tasks.rados.rados.0.smithi195.stdout:213: done (15 left) 2024-09-13T10:10:51.331 INFO:tasks.rados.rados.0.smithi195.stdout:223 retry set_chunk ! 2024-09-13T10:10:51.331 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 15 2024-09-13T10:10:51.331 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:10:51.332 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:10:51.332 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:10:51.332 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2024-09-13T10:10:51.332 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:10:51.338 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:10:51.329+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:10:51.418 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:10:51.419 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:10:51.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:10:51.419 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:10:51.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.419 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:61 4.0 4:0b4c105e:::smithi19530714-16:head [write 652105~445088,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:51.419 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.720236+0000", 2024-09-13T10:10:51.419 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.69543267799999997, 2024-09-13T10:10:51.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086548600000000003, 2024-09-13T10:10:51.419 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.419 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.419 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 61 2024-09-13T10:10:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720236+0000", 2024-09-13T10:10:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720236+0000", 2024-09-13T10:10:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720240+0000", 2024-09-13T10:10:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3629999999999998e-06 2024-09-13T10:10:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720942+0000", 2024-09-13T10:10:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070250300000000005 2024-09-13T10:10:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720968+0000", 2024-09-13T10:10:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5245999999999999e-05 2024-09-13T10:10:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720984+0000", 2024-09-13T10:10:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6821999999999999e-05 2024-09-13T10:10:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721070+0000", 2024-09-13T10:10:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5660000000000003e-05 2024-09-13T10:10:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721221+0000", 2024-09-13T10:10:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015077499999999999 2024-09-13T10:10:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:10:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721364+0000", 2024-09-13T10:10:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014364299999999999 2024-09-13T10:10:51.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728585+0000", 2024-09-13T10:10:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072205680000000001 2024-09-13T10:10:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728595+0000", 2024-09-13T10:10:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0108e-05 2024-09-13T10:10:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728688+0000", 2024-09-13T10:10:51.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2405999999999996e-05 2024-09-13T10:10:51.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:51.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728769+0000", 2024-09-13T10:10:51.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1174000000000006e-05 2024-09-13T10:10:51.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728891+0000", 2024-09-13T10:10:51.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012259200000000001 2024-09-13T10:10:51.426 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.426 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.426 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.426 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:62 4.0 4:0b4c105e:::smithi19530714-16:head [write 1728171~452988] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:51.426 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.721021+0000", 2024-09-13T10:10:51.426 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.69464764800000001, 2024-09-13T10:10:51.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096364669999999993, 2024-09-13T10:10:51.426 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.426 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 62 2024-09-13T10:10:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721021+0000", 2024-09-13T10:10:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721021+0000", 2024-09-13T10:10:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721023+0000", 2024-09-13T10:10:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-13T10:10:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721744+0000", 2024-09-13T10:10:51.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000721092 2024-09-13T10:10:51.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721766+0000", 2024-09-13T10:10:51.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2391999999999998e-05 2024-09-13T10:10:51.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721779+0000", 2024-09-13T10:10:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2864000000000001e-05 2024-09-13T10:10:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.724492+0000", 2024-09-13T10:10:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027128759999999999 2024-09-13T10:10:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.724746+0000", 2024-09-13T10:10:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000253914 2024-09-13T10:10:51.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:10:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.725111+0000", 2024-09-13T10:10:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036536400000000002 2024-09-13T10:10:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728978+0000", 2024-09-13T10:10:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038665969999999998 2024-09-13T10:10:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728991+0000", 2024-09-13T10:10:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3711e-05 2024-09-13T10:10:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.730564+0000", 2024-09-13T10:10:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001572564 2024-09-13T10:10:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.730617+0000", 2024-09-13T10:10:51.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2744999999999997e-05 2024-09-13T10:10:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.730658+0000", 2024-09-13T10:10:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1291999999999997e-05 2024-09-13T10:10:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:63 4.0 4:0b4c105e:::smithi19530714-16:head [setxattr _header (56) in=7b,truncate 2252105] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:51.433 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.721807+0000", 2024-09-13T10:10:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.69386210500000001, 2024-09-13T10:10:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093328749999999992, 2024-09-13T10:10:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 63 2024-09-13T10:10:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721807+0000", 2024-09-13T10:10:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721807+0000", 2024-09-13T10:10:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721809+0000", 2024-09-13T10:10:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6559999999999999e-06 2024-09-13T10:10:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721818+0000", 2024-09-13T10:10:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1530000000000003e-06 2024-09-13T10:10:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721836+0000", 2024-09-13T10:10:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8240000000000002e-05 2024-09-13T10:10:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721854+0000", 2024-09-13T10:10:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7532000000000001e-05 2024-09-13T10:10:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.727557+0000", 2024-09-13T10:10:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057033259999999999 2024-09-13T10:10:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.727772+0000", 2024-09-13T10:10:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021469600000000001 2024-09-13T10:10:51.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:10:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728048+0000", 2024-09-13T10:10:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027671299999999999 2024-09-13T10:10:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.729819+0000", 2024-09-13T10:10:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017708489999999999 2024-09-13T10:10:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:51.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.730310+0000", 2024-09-13T10:10:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049081999999999999 2024-09-13T10:10:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731047+0000", 2024-09-13T10:10:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073742899999999995 2024-09-13T10:10:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731096+0000", 2024-09-13T10:10:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8573999999999999e-05 2024-09-13T10:10:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731140+0000", 2024-09-13T10:10:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3887000000000002e-05 2024-09-13T10:10:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:64 4.0 4:0b4c105e:::smithi19530714-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.721878+0000", 2024-09-13T10:10:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.69379096500000004, 2024-09-13T10:10:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094623740000000008, 2024-09-13T10:10:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.440 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:10:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 64 2024-09-13T10:10:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721878+0000", 2024-09-13T10:10:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721878+0000", 2024-09-13T10:10:51.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721880+0000", 2024-09-13T10:10:51.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3939999999999999e-06 2024-09-13T10:10:51.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721886+0000", 2024-09-13T10:10:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0390000000000001e-06 2024-09-13T10:10:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721902+0000", 2024-09-13T10:10:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6520000000000001e-05 2024-09-13T10:10:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721917+0000", 2024-09-13T10:10:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4959e-05 2024-09-13T10:10:51.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.729034+0000", 2024-09-13T10:10:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071171100000000003 2024-09-13T10:10:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:10:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.729161+0000", 2024-09-13T10:10:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012695600000000001 2024-09-13T10:10:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731172+0000", 2024-09-13T10:10:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002010651 2024-09-13T10:10:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731241+0000", 2024-09-13T10:10:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9646000000000004e-05 2024-09-13T10:10:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731341+0000", 2024-09-13T10:10:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9098999999999994e-05 2024-09-13T10:10:51.445 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:89 4.9 e38/31 4:96fcf081:::smithi19530714-23:head v 38'1, mlcod=0'0)", 2024-09-13T10:10:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.811350+0000", 2024-09-13T10:10:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.604318825, 2024-09-13T10:10:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034011459999999999, 2024-09-13T10:10:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:51.446 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.811350+0000", 2024-09-13T10:10:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.811350+0000", 2024-09-13T10:10:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.811351+0000", 2024-09-13T10:10:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3599999999999999e-07 2024-09-13T10:10:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.812700+0000", 2024-09-13T10:10:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001349704 2024-09-13T10:10:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.812735+0000", 2024-09-13T10:10:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4598999999999999e-05 2024-09-13T10:10:51.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.812754+0000", 2024-09-13T10:10:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8632999999999999e-05 2024-09-13T10:10:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.812851+0000", 2024-09-13T10:10:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7897000000000001e-05 2024-09-13T10:10:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.449 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.812883+0000", 2024-09-13T10:10:51.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1600999999999998e-05 2024-09-13T10:10:51.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:51.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.814667+0000", 2024-09-13T10:10:51.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017839589999999999 2024-09-13T10:10:51.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.814751+0000", 2024-09-13T10:10:51.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4417000000000002e-05 2024-09-13T10:10:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:150 4.4 e38/31 4:24275569:::smithi19530714-38:head v 38'11, mlcod=38'9)", 2024-09-13T10:10:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.815678+0000", 2024-09-13T10:10:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.59999151900000003, 2024-09-13T10:10:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066202199999999996, 2024-09-13T10:10:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:51.451 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815678+0000", 2024-09-13T10:10:51.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815678+0000", 2024-09-13T10:10:51.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815689+0000", 2024-09-13T10:10:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1334e-05 2024-09-13T10:10:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816641+0000", 2024-09-13T10:10:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000952307 2024-09-13T10:10:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816667+0000", 2024-09-13T10:10:51.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5636e-05 2024-09-13T10:10:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816679+0000", 2024-09-13T10:10:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2373e-05 2024-09-13T10:10:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816864+0000", 2024-09-13T10:10:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018461399999999999 2024-09-13T10:10:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816918+0000", 2024-09-13T10:10:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4413999999999998e-05 2024-09-13T10:10:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822241+0000", 2024-09-13T10:10:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053230430000000004 2024-09-13T10:10:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822298+0000", 2024-09-13T10:10:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6499000000000002e-05 2024-09-13T10:10:51.455 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:129 4.9 e38/31 4:998d9bf1:::smithi19530714-33:head v 38'4, mlcod=0'0)", 2024-09-13T10:10:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.817704+0000", 2024-09-13T10:10:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.59796538099999996, 2024-09-13T10:10:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052000190000000002, 2024-09-13T10:10:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:51.456 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817704+0000", 2024-09-13T10:10:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817704+0000", 2024-09-13T10:10:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817705+0000", 2024-09-13T10:10:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.263e-06 2024-09-13T10:10:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818390+0000", 2024-09-13T10:10:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068477299999999996 2024-09-13T10:10:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818406+0000", 2024-09-13T10:10:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5896000000000002e-05 2024-09-13T10:10:51.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818421+0000", 2024-09-13T10:10:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.489e-05 2024-09-13T10:10:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818514+0000", 2024-09-13T10:10:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3318000000000003e-05 2024-09-13T10:10:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818541+0000", 2024-09-13T10:10:51.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7271999999999999e-05 2024-09-13T10:10:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822802+0000", 2024-09-13T10:10:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042610440000000003 2024-09-13T10:10:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822904+0000", 2024-09-13T10:10:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101563 2024-09-13T10:10:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.460 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.461 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:130 4.9 e38/31 4:998d9bf1:::smithi19530714-33:head v 38'5, mlcod=0'0)", 2024-09-13T10:10:51.461 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.819506+0000", 2024-09-13T10:10:51.461 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.59616303100000001, 2024-09-13T10:10:51.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040062850000000001, 2024-09-13T10:10:51.461 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.461 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.461 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:51.461 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819506+0000", 2024-09-13T10:10:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819506+0000", 2024-09-13T10:10:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819508+0000", 2024-09-13T10:10:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0370000000000001e-06 2024-09-13T10:10:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820559+0000", 2024-09-13T10:10:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00105064 2024-09-13T10:10:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820586+0000", 2024-09-13T10:10:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6962999999999999e-05 2024-09-13T10:10:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820606+0000", 2024-09-13T10:10:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0475999999999998e-05 2024-09-13T10:10:51.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820770+0000", 2024-09-13T10:10:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016347700000000001 2024-09-13T10:10:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820858+0000", 2024-09-13T10:10:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8016999999999994e-05 2024-09-13T10:10:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.823428+0000", 2024-09-13T10:10:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025704640000000002 2024-09-13T10:10:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.823512+0000", 2024-09-13T10:10:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4210999999999997e-05 2024-09-13T10:10:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.465 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:157 4.0 4:03fab808:::smithi19530714-40:head [write 655473~716476,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.836742+0000", 2024-09-13T10:10:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.57892662299999997, 2024-09-13T10:10:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0076813209999999996, 2024-09-13T10:10:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 157 2024-09-13T10:10:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.836742+0000", 2024-09-13T10:10:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.836742+0000", 2024-09-13T10:10:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.836746+0000", 2024-09-13T10:10:51.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1930000000000002e-06 2024-09-13T10:10:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837565+0000", 2024-09-13T10:10:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081902899999999998 2024-09-13T10:10:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837613+0000", 2024-09-13T10:10:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8640999999999997e-05 2024-09-13T10:10:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837628+0000", 2024-09-13T10:10:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4867e-05 2024-09-13T10:10:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837701+0000", 2024-09-13T10:10:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2862999999999997e-05 2024-09-13T10:10:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837831+0000", 2024-09-13T10:10:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012971599999999999 2024-09-13T10:10:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:10:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837974+0000", 2024-09-13T10:10:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014289899999999999 2024-09-13T10:10:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.839710+0000", 2024-09-13T10:10:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00173613 2024-09-13T10:10:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.842419+0000", 2024-09-13T10:10:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027087539999999998 2024-09-13T10:10:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844336+0000", 2024-09-13T10:10:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001916947 2024-09-13T10:10:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844381+0000", 2024-09-13T10:10:51.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5377000000000003e-05 2024-09-13T10:10:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844424+0000", 2024-09-13T10:10:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2904999999999997e-05 2024-09-13T10:10:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:158 4.0 4:03fab808:::smithi19530714-40:head [write 2099033~556440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.843944+0000", 2024-09-13T10:10:51.472 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.57172472900000004, 2024-09-13T10:10:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006981821, 2024-09-13T10:10:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 158 2024-09-13T10:10:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.843944+0000", 2024-09-13T10:10:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.843944+0000", 2024-09-13T10:10:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.843946+0000", 2024-09-13T10:10:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.75e-06 2024-09-13T10:10:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844474+0000", 2024-09-13T10:10:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052765500000000003 2024-09-13T10:10:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844496+0000", 2024-09-13T10:10:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2552e-05 2024-09-13T10:10:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844509+0000", 2024-09-13T10:10:51.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2387e-05 2024-09-13T10:10:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844578+0000", 2024-09-13T10:10:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9664000000000003e-05 2024-09-13T10:10:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844657+0000", 2024-09-13T10:10:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8417000000000005e-05 2024-09-13T10:10:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:10:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844784+0000", 2024-09-13T10:10:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012675800000000001 2024-09-13T10:10:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.846495+0000", 2024-09-13T10:10:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017114350000000001 2024-09-13T10:10:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.848918+0000", 2024-09-13T10:10:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002422783 2024-09-13T10:10:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.850771+0000", 2024-09-13T10:10:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001853616 2024-09-13T10:10:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.850859+0000", 2024-09-13T10:10:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7786000000000003e-05 2024-09-13T10:10:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.850926+0000", 2024-09-13T10:10:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7018000000000001e-05 2024-09-13T10:10:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:159 4.0 4:03fab808:::smithi19530714-40:head [setxattr _header (56) in=7b,truncate 2655473] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.844538+0000", 2024-09-13T10:10:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.57113126299999994, 2024-09-13T10:10:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067064480000000003, 2024-09-13T10:10:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.479 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 159 2024-09-13T10:10:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844538+0000", 2024-09-13T10:10:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844538+0000", 2024-09-13T10:10:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844539+0000", 2024-09-13T10:10:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.223e-06 2024-09-13T10:10:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844545+0000", 2024-09-13T10:10:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8100000000000003e-06 2024-09-13T10:10:51.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844555+0000", 2024-09-13T10:10:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6490000000000003e-06 2024-09-13T10:10:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844563+0000", 2024-09-13T10:10:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4370000000000007e-06 2024-09-13T10:10:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.845602+0000", 2024-09-13T10:10:51.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00103863 2024-09-13T10:10:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.845695+0000", 2024-09-13T10:10:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2913999999999998e-05 2024-09-13T10:10:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:10:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.845812+0000", 2024-09-13T10:10:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011715899999999999 2024-09-13T10:10:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:51.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.846904+0000", 2024-09-13T10:10:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001092733 2024-09-13T10:10:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.849487+0000", 2024-09-13T10:10:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025827459999999999 2024-09-13T10:10:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851147+0000", 2024-09-13T10:10:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001659878 2024-09-13T10:10:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851184+0000", 2024-09-13T10:10:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7435999999999999e-05 2024-09-13T10:10:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851244+0000", 2024-09-13T10:10:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9833000000000001e-05 2024-09-13T10:10:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:160 4.0 4:03fab808:::smithi19530714-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:51.485 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.844579+0000", 2024-09-13T10:10:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.57108967499999996, 2024-09-13T10:10:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006862506, 2024-09-13T10:10:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:10:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 160 2024-09-13T10:10:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844579+0000", 2024-09-13T10:10:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844579+0000", 2024-09-13T10:10:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844580+0000", 2024-09-13T10:10:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1699999999999997e-07 2024-09-13T10:10:51.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844585+0000", 2024-09-13T10:10:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6360000000000002e-06 2024-09-13T10:10:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844595+0000", 2024-09-13T10:10:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9739999999999997e-06 2024-09-13T10:10:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844604+0000", 2024-09-13T10:10:51.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7560000000000005e-06 2024-09-13T10:10:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.846062+0000", 2024-09-13T10:10:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001458446 2024-09-13T10:10:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:10:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.846121+0000", 2024-09-13T10:10:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8697999999999999e-05 2024-09-13T10:10:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851297+0000", 2024-09-13T10:10:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051764810000000001 2024-09-13T10:10:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851355+0000", 2024-09-13T10:10:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7794999999999998e-05 2024-09-13T10:10:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851442+0000", 2024-09-13T10:10:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6903000000000005e-05 2024-09-13T10:10:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:177 4.4 e38/31 4:25c904b0:::smithi19530714-45:head v 38'13, mlcod=38'11)", 2024-09-13T10:10:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.912497+0000", 2024-09-13T10:10:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.50317193500000001, 2024-09-13T10:10:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033419600000000002, 2024-09-13T10:10:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.912497+0000", 2024-09-13T10:10:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.912497+0000", 2024-09-13T10:10:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.912499+0000", 2024-09-13T10:10:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2330000000000001e-06 2024-09-13T10:10:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.913044+0000", 2024-09-13T10:10:51.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054459400000000002 2024-09-13T10:10:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.913076+0000", 2024-09-13T10:10:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1971999999999998e-05 2024-09-13T10:10:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.913093+0000", 2024-09-13T10:10:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7030000000000001e-05 2024-09-13T10:10:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.913244+0000", 2024-09-13T10:10:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015073800000000001 2024-09-13T10:10:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.913284+0000", 2024-09-13T10:10:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0392999999999998e-05 2024-09-13T10:10:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.915754+0000", 2024-09-13T10:10:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024693829999999999 2024-09-13T10:10:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.915839+0000", 2024-09-13T10:10:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5617000000000004e-05 2024-09-13T10:10:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:178 4.4 e38/31 4:25c904b0:::smithi19530714-45:head v 38'14, mlcod=38'12)", 2024-09-13T10:10:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.917984+0000", 2024-09-13T10:10:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.49768484200000002, 2024-09-13T10:10:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0034735780000000002, 2024-09-13T10:10:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:51.495 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.917984+0000", 2024-09-13T10:10:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.917984+0000", 2024-09-13T10:10:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.917986+0000", 2024-09-13T10:10:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4279999999999999e-06 2024-09-13T10:10:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.918569+0000", 2024-09-13T10:10:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00058317899999999997 2024-09-13T10:10:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.918589+0000", 2024-09-13T10:10:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0051000000000002e-05 2024-09-13T10:10:51.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.918600+0000", 2024-09-13T10:10:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1568e-05 2024-09-13T10:10:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.918749+0000", 2024-09-13T10:10:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000148243 2024-09-13T10:10:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.918790+0000", 2024-09-13T10:10:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1675000000000003e-05 2024-09-13T10:10:51.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.921415+0000", 2024-09-13T10:10:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026245690000000002 2024-09-13T10:10:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.921458+0000", 2024-09-13T10:10:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2864999999999997e-05 2024-09-13T10:10:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:193 4.4 e38/31 4:24da8222:::smithi19530714-49:head v 38'16, mlcod=38'14)", 2024-09-13T10:10:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.971441+0000", 2024-09-13T10:10:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.44422799899999998, 2024-09-13T10:10:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052917290000000002, 2024-09-13T10:10:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.971441+0000", 2024-09-13T10:10:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.971441+0000", 2024-09-13T10:10:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.971443+0000", 2024-09-13T10:10:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2529999999999999e-06 2024-09-13T10:10:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.972416+0000", 2024-09-13T10:10:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00097274999999999998 2024-09-13T10:10:51.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.972451+0000", 2024-09-13T10:10:51.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5342000000000001e-05 2024-09-13T10:10:51.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.972478+0000", 2024-09-13T10:10:51.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6149e-05 2024-09-13T10:10:51.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.972603+0000", 2024-09-13T10:10:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012573 2024-09-13T10:10:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.972626+0000", 2024-09-13T10:10:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2575e-05 2024-09-13T10:10:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.976613+0000", 2024-09-13T10:10:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039869950000000001 2024-09-13T10:10:51.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.976733+0000", 2024-09-13T10:10:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119935 2024-09-13T10:10:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:197 4.0 4:0bd0428c:::smithi19530714-50:head [write 423543~785319,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.983566+0000", 2024-09-13T10:10:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.43210306399999998, 2024-09-13T10:10:51.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062390049999999997, 2024-09-13T10:10:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 197 2024-09-13T10:10:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.983566+0000", 2024-09-13T10:10:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.983566+0000", 2024-09-13T10:10:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.983569+0000", 2024-09-13T10:10:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5900000000000002e-06 2024-09-13T10:10:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.984399+0000", 2024-09-13T10:10:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083044399999999995 2024-09-13T10:10:51.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.984418+0000", 2024-09-13T10:10:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8672999999999999e-05 2024-09-13T10:10:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.984427+0000", 2024-09-13T10:10:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9930000000000001e-06 2024-09-13T10:10:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.984495+0000", 2024-09-13T10:10:51.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8405000000000007e-05 2024-09-13T10:10:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.984628+0000", 2024-09-13T10:10:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013301400000000001 2024-09-13T10:10:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:10:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.984764+0000", 2024-09-13T10:10:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013590299999999999 2024-09-13T10:10:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.987014+0000", 2024-09-13T10:10:51.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002250008 2024-09-13T10:10:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.989035+0000", 2024-09-13T10:10:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020208299999999999 2024-09-13T10:10:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.989722+0000", 2024-09-13T10:10:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068683199999999996 2024-09-13T10:10:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:51.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.989765+0000", 2024-09-13T10:10:51.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3656999999999999e-05 2024-09-13T10:10:51.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.989805+0000", 2024-09-13T10:10:51.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9656000000000001e-05 2024-09-13T10:10:51.510 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.510 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.510 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:198 4.0 4:0bd0428c:::smithi19530714-50:head [write 1767500~799008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991140+0000", 2024-09-13T10:10:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.42452868599999999, 2024-09-13T10:10:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079451699999999997, 2024-09-13T10:10:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 198 2024-09-13T10:10:51.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991140+0000", 2024-09-13T10:10:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991140+0000", 2024-09-13T10:10:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991144+0000", 2024-09-13T10:10:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2499999999999998e-06 2024-09-13T10:10:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992036+0000", 2024-09-13T10:10:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000891931 2024-09-13T10:10:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992052+0000", 2024-09-13T10:10:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6070999999999999e-05 2024-09-13T10:10:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992060+0000", 2024-09-13T10:10:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9719999999999992e-06 2024-09-13T10:10:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992128+0000", 2024-09-13T10:10:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7858e-05 2024-09-13T10:10:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992208+0000", 2024-09-13T10:10:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0383000000000006e-05 2024-09-13T10:10:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:10:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992324+0000", 2024-09-13T10:10:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011587 2024-09-13T10:10:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997567+0000", 2024-09-13T10:10:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052430419999999998 2024-09-13T10:10:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.998107+0000", 2024-09-13T10:10:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054063099999999997 2024-09-13T10:10:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.998940+0000", 2024-09-13T10:10:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083243599999999998 2024-09-13T10:10:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.999021+0000", 2024-09-13T10:10:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1335999999999995e-05 2024-09-13T10:10:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.999086+0000", 2024-09-13T10:10:51.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4389999999999998e-05 2024-09-13T10:10:51.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.517 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.517 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:199 4.0 4:0bd0428c:::smithi19530714-50:head [setxattr _header (56) in=7b,truncate 2823543] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:51.517 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.992081+0000", 2024-09-13T10:10:51.517 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.42358818500000001, 2024-09-13T10:10:51.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011866829000000001, 2024-09-13T10:10:51.517 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.517 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 199 2024-09-13T10:10:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992081+0000", 2024-09-13T10:10:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992081+0000", 2024-09-13T10:10:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992082+0000", 2024-09-13T10:10:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.004e-06 2024-09-13T10:10:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992090+0000", 2024-09-13T10:10:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7440000000000007e-06 2024-09-13T10:10:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:10:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3650000000000004e-06 2024-09-13T10:10:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992107+0000", 2024-09-13T10:10:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9230000000000008e-06 2024-09-13T10:10:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.995938+0000", 2024-09-13T10:10:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003830773 2024-09-13T10:10:51.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.996229+0000", 2024-09-13T10:10:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029133300000000001 2024-09-13T10:10:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:10:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.996692+0000", 2024-09-13T10:10:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046326700000000001 2024-09-13T10:10:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:10:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997974+0000", 2024-09-13T10:10:51.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001282016 2024-09-13T10:10:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.999243+0000", 2024-09-13T10:10:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012682360000000001 2024-09-13T10:10:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.003767+0000", 2024-09-13T10:10:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045243970000000003 2024-09-13T10:10:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.003851+0000", 2024-09-13T10:10:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4195000000000003e-05 2024-09-13T10:10:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.003948+0000", 2024-09-13T10:10:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6576000000000005e-05 2024-09-13T10:10:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:200 4.0 4:0bd0428c:::smithi19530714-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:51.523 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.992119+0000", 2024-09-13T10:10:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.423550025, 2024-09-13T10:10:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012149748, 2024-09-13T10:10:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:10:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:10:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:10:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 200 2024-09-13T10:10:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992119+0000", 2024-09-13T10:10:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992119+0000", 2024-09-13T10:10:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992120+0000", 2024-09-13T10:10:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9999999999999997e-07 2024-09-13T10:10:51.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992123+0000", 2024-09-13T10:10:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9569999999999998e-06 2024-09-13T10:10:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992131+0000", 2024-09-13T10:10:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2024-09-13T10:10:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992140+0000", 2024-09-13T10:10:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7900000000000005e-06 2024-09-13T10:10:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997418+0000", 2024-09-13T10:10:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052784349999999997 2024-09-13T10:10:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:10:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997546+0000", 2024-09-13T10:10:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012788900000000001 2024-09-13T10:10:51.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.004004+0000", 2024-09-13T10:10:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064580549999999999 2024-09-13T10:10:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.004128+0000", 2024-09-13T10:10:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123628 2024-09-13T10:10:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.004269+0000", 2024-09-13T10:10:51.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000141292 2024-09-13T10:10:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.10.22:10 4.0 4:0bd0428c:::smithi19530714-50:head [assert-version v9,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.115513+0000", 2024-09-13T10:10:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.30015575100000003, 2024-09-13T10:10:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038513810000000001, 2024-09-13T10:10:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:10:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:51.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.115513+0000", 2024-09-13T10:10:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.115513+0000", 2024-09-13T10:10:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.115516+0000", 2024-09-13T10:10:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9450000000000002e-06 2024-09-13T10:10:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.115527+0000", 2024-09-13T10:10:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0526000000000001e-05 2024-09-13T10:10:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.115544+0000", 2024-09-13T10:10:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7499e-05 2024-09-13T10:10:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.115552+0000", 2024-09-13T10:10:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5859999999999999e-06 2024-09-13T10:10:51.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.115647+0000", 2024-09-13T10:10:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5328000000000006e-05 2024-09-13T10:10:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.115788+0000", 2024-09-13T10:10:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000140941 2024-09-13T10:10:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.119365+0000", 2024-09-13T10:10:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035765559999999998 2024-09-13T10:10:51.532 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:51.533 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:51.533 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:10:51.537 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-13T10:10:51.540 INFO:tasks.rados.rados.0.smithi195.stdout:193: finishing set_chunk to oid 44 2024-09-13T10:10:51.540 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 9 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-13T10:10:51.540 INFO:tasks.rados.rados.0.smithi195.stdout:193: done (14 left) 2024-09-13T10:10:51.540 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 14 2024-09-13T10:10:51.541 INFO:tasks.rados.rados.0.smithi195.stdout:222: finishing set_chunk to oid 23 2024-09-13T10:10:51.541 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 3 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-13T10:10:51.541 INFO:tasks.rados.rados.0.smithi195.stdout:222: done (13 left) 2024-09-13T10:10:51.541 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 13 2024-09-13T10:10:51.541 INFO:tasks.rados.rados.0.smithi195.stdout:210: finishing set_chunk to oid 11 2024-09-13T10:10:51.541 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 3 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-13T10:10:51.541 INFO:tasks.rados.rados.0.smithi195.stdout:210: done (12 left) 2024-09-13T10:10:51.541 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 12 2024-09-13T10:10:51.541 INFO:tasks.rados.rados.0.smithi195.stdout:214: finishing set_chunk to oid 15 2024-09-13T10:10:51.541 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 9 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-13T10:10:51.541 INFO:tasks.rados.rados.0.smithi195.stdout:214: done (11 left) 2024-09-13T10:10:51.541 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 11 2024-09-13T10:10:51.542 INFO:tasks.rados.rados.0.smithi195.stdout:194: finishing set_chunk to oid 45 2024-09-13T10:10:51.542 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 45 set_chunk 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 915968 length: 1623552 2024-09-13T10:10:51.542 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 0 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-13T10:10:51.542 INFO:tasks.rados.rados.0.smithi195.stdout:198: finishing set_chunk to oid 49 2024-09-13T10:10:51.542 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 49 set_chunk 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1582592 length: 653312 2024-09-13T10:10:51.542 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 0 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-13T10:10:51.542 INFO:tasks.rados.rados.0.smithi195.stdout:194: done (10 left) 2024-09-13T10:10:51.542 INFO:tasks.rados.rados.0.smithi195.stdout:198: done (9 left) 2024-09-13T10:10:51.542 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 9 2024-09-13T10:10:51.542 INFO:tasks.rados.rados.0.smithi195.stdout:201: finishing set_chunk to oid 2 2024-09-13T10:10:51.542 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 2 set_chunk 2 low_tier returned error code -95 offset: 1195008 length: 273920 2024-09-13T10:10:51.542 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-13T10:10:51.542 INFO:tasks.rados.rados.0.smithi195.stdout:201: done (8 left) 2024-09-13T10:10:51.543 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 8 2024-09-13T10:10:51.543 INFO:tasks.rados.rados.0.smithi195.stdout:223: finishing set_chunk to oid 24 2024-09-13T10:10:51.543 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 24 set_chunk 24 low_tier returned error code -95 offset: 317952 length: 1897472 2024-09-13T10:10:51.543 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 0 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-13T10:10:51.543 INFO:tasks.rados.rados.0.smithi195.stdout:223: done (7 left) 2024-09-13T10:10:51.543 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 7 2024-09-13T10:10:51.543 INFO:tasks.rados.rados.0.smithi195.stdout:208: finishing set_chunk to oid 9 2024-09-13T10:10:51.543 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 9 set_chunk 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 297984 length: 829440 2024-09-13T10:10:51.543 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 0 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-13T10:10:51.543 INFO:tasks.rados.rados.0.smithi195.stdout:208: done (6 left) 2024-09-13T10:10:51.543 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 6 2024-09-13T10:10:51.543 INFO:tasks.rados.rados.0.smithi195.stdout:221: finishing set_chunk to oid 22 2024-09-13T10:10:51.544 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 1090560 length: 353792 2024-09-13T10:10:51.544 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-13T10:10:51.544 INFO:tasks.rados.rados.0.smithi195.stdout:174: finishing set_chunk to oid 25 2024-09-13T10:10:51.544 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 6 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-13T10:10:51.544 INFO:tasks.rados.rados.0.smithi195.stdout:174: done (5 left) 2024-09-13T10:10:51.544 INFO:tasks.rados.rados.0.smithi195.stdout:221: done (4 left) 2024-09-13T10:10:51.544 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 4 2024-09-13T10:10:51.544 INFO:tasks.rados.rados.0.smithi195.stdout:185: finishing set_chunk to oid 36 2024-09-13T10:10:51.544 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 6 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-13T10:10:51.544 INFO:tasks.rados.rados.0.smithi195.stdout:187: finishing set_chunk to oid 38 2024-09-13T10:10:51.544 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 12 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-13T10:10:51.544 INFO:tasks.rados.rados.0.smithi195.stdout:185: done (3 left) 2024-09-13T10:10:51.545 INFO:tasks.rados.rados.0.smithi195.stdout:187: done (2 left) 2024-09-13T10:10:51.545 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 2 2024-09-13T10:10:51.545 INFO:tasks.rados.rados.0.smithi195.stdout:177: finishing set_chunk to oid 28 2024-09-13T10:10:51.545 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 9 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-13T10:10:51.545 INFO:tasks.rados.rados.0.smithi195.stdout:177: done (1 left) 2024-09-13T10:10:51.545 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 1 2024-09-13T10:10:51.545 INFO:tasks.rados.rados.0.smithi195.stdout:220: finishing set_chunk to oid 21 2024-09-13T10:10:51.545 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 9 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-13T10:10:51.545 INFO:tasks.rados.rados.0.smithi195.stdout:220: done (0 left) 2024-09-13T10:10:51.545 INFO:tasks.rados.rados.0.smithi195.stdout:224: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.545 INFO:tasks.rados.rados.0.smithi195.stdout:224: set_chunk oid 25 offset: 1858048 length: 318464 target oid smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1858048 snap -1 2024-09-13T10:10:51.545 INFO:tasks.rados.rados.0.smithi195.stdout:225: set_chunk oid 26 target oid 26 low_tier 2024-09-13T10:10:51.545 INFO:tasks.rados.rados.0.smithi195.stdout:225: set_chunk oid 26 offset: 525824 length: 1743360 target oid smithi19530714-26 low_tier offset: 525824 snap -1 2024-09-13T10:10:51.546 INFO:tasks.rados.rados.0.smithi195.stdout:226: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.546 INFO:tasks.rados.rados.0.smithi195.stdout:226: set_chunk oid 27 offset: 62464 length: 2215936 target oid smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 62464 snap -1 2024-09-13T10:10:51.546 INFO:tasks.rados.rados.0.smithi195.stdout:227: set_chunk oid 28 target oid 28 low_tier 2024-09-13T10:10:51.546 INFO:tasks.rados.rados.0.smithi195.stdout:227: set_chunk oid 28 offset: 1263616 length: 1092096 target oid smithi19530714-28 low_tier offset: 1263616 snap -1 2024-09-13T10:10:51.546 INFO:tasks.rados.rados.0.smithi195.stdout:228: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.546 INFO:tasks.rados.rados.0.smithi195.stdout:228: set_chunk oid 29 offset: 1467904 length: 745472 target oid smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1467904 snap -1 2024-09-13T10:10:51.546 INFO:tasks.rados.rados.0.smithi195.stdout:229: set_chunk oid 30 target oid 30 low_tier 2024-09-13T10:10:51.546 INFO:tasks.rados.rados.0.smithi195.stdout:229: set_chunk oid 30 offset: 1337856 length: 1075200 target oid smithi19530714-30 low_tier offset: 1337856 snap -1 2024-09-13T10:10:51.546 INFO:tasks.rados.rados.0.smithi195.stdout:230: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.546 INFO:tasks.rados.rados.0.smithi195.stdout:230: set_chunk oid 31 offset: 1693696 length: 58368 target oid smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1693696 snap -1 2024-09-13T10:10:51.546 INFO:tasks.rados.rados.0.smithi195.stdout:231: set_chunk oid 32 target oid 32 low_tier 2024-09-13T10:10:51.546 INFO:tasks.rados.rados.0.smithi195.stdout:231: set_chunk oid 32 offset: 333312 length: 1395200 target oid smithi19530714-32 low_tier offset: 333312 snap -1 2024-09-13T10:10:51.547 INFO:tasks.rados.rados.0.smithi195.stdout:232: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.547 INFO:tasks.rados.rados.0.smithi195.stdout:232: set_chunk oid 33 offset: 1498624 length: 459264 target oid smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1498624 snap -1 2024-09-13T10:10:51.547 INFO:tasks.rados.rados.0.smithi195.stdout:233: set_chunk oid 34 target oid 34 low_tier 2024-09-13T10:10:51.547 INFO:tasks.rados.rados.0.smithi195.stdout:233: set_chunk oid 34 offset: 2133504 length: 394752 target oid smithi19530714-34 low_tier offset: 2133504 snap -1 2024-09-13T10:10:51.547 INFO:tasks.rados.rados.0.smithi195.stdout:234: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.547 INFO:tasks.rados.rados.0.smithi195.stdout:234: set_chunk oid 35 offset: 892416 length: 452608 target oid smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 892416 snap -1 2024-09-13T10:10:51.547 INFO:tasks.rados.rados.0.smithi195.stdout:235: set_chunk oid 36 target oid 36 low_tier 2024-09-13T10:10:51.547 INFO:tasks.rados.rados.0.smithi195.stdout:235: set_chunk oid 36 offset: 583680 length: 605696 target oid smithi19530714-36 low_tier offset: 583680 snap -1 2024-09-13T10:10:51.547 INFO:tasks.rados.rados.0.smithi195.stdout:236: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.547 INFO:tasks.rados.rados.0.smithi195.stdout:236: set_chunk oid 37 offset: 1634816 length: 693248 target oid smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1634816 snap -1 2024-09-13T10:10:51.547 INFO:tasks.rados.rados.0.smithi195.stdout:237: set_chunk oid 38 target oid 38 low_tier 2024-09-13T10:10:51.548 INFO:tasks.rados.rados.0.smithi195.stdout:237: set_chunk oid 38 offset: 1160192 length: 640512 target oid smithi19530714-38 low_tier offset: 1160192 snap -1 2024-09-13T10:10:51.548 INFO:tasks.rados.rados.0.smithi195.stdout:238: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.548 INFO:tasks.rados.rados.0.smithi195.stdout:238: set_chunk oid 39 offset: 2112000 length: 141312 target oid smithi19530714-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 2112000 snap -1 2024-09-13T10:10:51.548 INFO:tasks.rados.rados.0.smithi195.stdout:239: set_chunk oid 40 target oid 40 low_tier 2024-09-13T10:10:51.548 INFO:tasks.rados.rados.0.smithi195.stdout:239: set_chunk oid 40 offset: 734720 length: 1690112 target oid smithi19530714-40 low_tier offset: 734720 snap -1 2024-09-13T10:10:51.548 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.548 INFO:tasks.rados.rados.0.smithi195.stdout:226: finishing set_chunk to oid 27 2024-09-13T10:10:51.548 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 27 set_chunk 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 62464 length: 2215936 2024-09-13T10:10:51.548 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-13T10:10:51.548 INFO:tasks.rados.rados.0.smithi195.stdout:225: finishing set_chunk to oid 26 2024-09-13T10:10:51.548 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 26 set_chunk 26 low_tier returned error code -95 offset: 525824 length: 1743360 2024-09-13T10:10:51.548 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 0 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-13T10:10:51.549 INFO:tasks.rados.rados.0.smithi195.stdout:224: finishing set_chunk to oid 25 2024-09-13T10:10:51.549 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 25 set_chunk 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1858048 length: 318464 2024-09-13T10:10:51.549 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-13T10:10:51.549 INFO:tasks.rados.rados.0.smithi195.stdout:224: done (15 left) 2024-09-13T10:10:51.549 INFO:tasks.rados.rados.0.smithi195.stdout:225: done (14 left) 2024-09-13T10:10:51.549 INFO:tasks.rados.rados.0.smithi195.stdout:226: done (13 left) 2024-09-13T10:10:51.549 INFO:tasks.rados.rados.0.smithi195.stdout:240: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.549 INFO:tasks.rados.rados.0.smithi195.stdout:240: set_chunk oid 41 offset: 1618432 length: 844288 target oid smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1618432 snap -1 2024-09-13T10:10:51.549 INFO:tasks.rados.rados.0.smithi195.stdout:241: set_chunk oid 42 target oid 42 low_tier 2024-09-13T10:10:51.549 INFO:tasks.rados.rados.0.smithi195.stdout:241: set_chunk oid 42 offset: 141824 length: 638976 target oid smithi19530714-42 low_tier offset: 141824 snap -1 2024-09-13T10:10:51.549 INFO:tasks.rados.rados.0.smithi195.stdout:242: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.549 INFO:tasks.rados.rados.0.smithi195.stdout:242: set_chunk oid 43 offset: 642560 length: 1248256 target oid smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 642560 snap -1 2024-09-13T10:10:51.549 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.550 INFO:tasks.rados.rados.0.smithi195.stdout:230: finishing set_chunk to oid 31 2024-09-13T10:10:51.550 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 31 set_chunk 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1693696 length: 58368 2024-09-13T10:10:51.550 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-13T10:10:51.550 INFO:tasks.rados.rados.0.smithi195.stdout:230: done (15 left) 2024-09-13T10:10:51.550 INFO:tasks.rados.rados.0.smithi195.stdout:243: set_chunk oid 44 target oid 44 low_tier 2024-09-13T10:10:51.550 INFO:tasks.rados.rados.0.smithi195.stdout:243: set_chunk oid 44 offset: 939520 length: 202752 target oid smithi19530714-44 low_tier offset: 939520 snap -1 2024-09-13T10:10:51.550 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.550 INFO:tasks.rados.rados.0.smithi195.stdout:233: finishing set_chunk to oid 34 2024-09-13T10:10:51.550 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 34 set_chunk 34 low_tier returned error code -95 offset: 2133504 length: 394752 2024-09-13T10:10:51.550 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 34 v 0 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-13T10:10:51.550 INFO:tasks.rados.rados.0.smithi195.stdout:231: finishing set_chunk to oid 32 2024-09-13T10:10:51.550 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 333312 length: 1395200 2024-09-13T10:10:51.551 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-13T10:10:51.551 INFO:tasks.rados.rados.0.smithi195.stdout:236: finishing set_chunk to oid 37 2024-09-13T10:10:51.551 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 37 set_chunk 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1634816 length: 693248 2024-09-13T10:10:51.551 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 0 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-13T10:10:51.551 INFO:tasks.rados.rados.0.smithi195.stdout:229: finishing set_chunk to oid 30 2024-09-13T10:10:51.551 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 30 set_chunk 30 low_tier returned error code -95 offset: 1337856 length: 1075200 2024-09-13T10:10:51.551 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 0 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-13T10:10:51.551 INFO:tasks.rados.rados.0.smithi195.stdout:235: finishing set_chunk to oid 36 2024-09-13T10:10:51.551 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 36 set_chunk 36 low_tier returned error code -95 offset: 583680 length: 605696 2024-09-13T10:10:51.551 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 0 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-13T10:10:51.551 INFO:tasks.rados.rados.0.smithi195.stdout:239: finishing set_chunk to oid 40 2024-09-13T10:10:51.551 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 40 set_chunk 40 low_tier returned error code -95 offset: 734720 length: 1690112 2024-09-13T10:10:51.551 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 0 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-13T10:10:51.552 INFO:tasks.rados.rados.0.smithi195.stdout:238: finishing set_chunk to oid 39 2024-09-13T10:10:51.552 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 2112000 length: 141312 2024-09-13T10:10:51.552 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-13T10:10:51.552 INFO:tasks.rados.rados.0.smithi195.stdout:227: finishing set_chunk to oid 28 2024-09-13T10:10:51.552 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 28 set_chunk 28 low_tier returned error code -95 offset: 1263616 length: 1092096 2024-09-13T10:10:51.552 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 0 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-13T10:10:51.552 INFO:tasks.rados.rados.0.smithi195.stdout:227: done (15 left) 2024-09-13T10:10:51.552 INFO:tasks.rados.rados.0.smithi195.stdout:229: done (14 left) 2024-09-13T10:10:51.552 INFO:tasks.rados.rados.0.smithi195.stdout:231: done (13 left) 2024-09-13T10:10:51.552 INFO:tasks.rados.rados.0.smithi195.stdout:233: done (12 left) 2024-09-13T10:10:51.552 INFO:tasks.rados.rados.0.smithi195.stdout:235: done (11 left) 2024-09-13T10:10:51.552 INFO:tasks.rados.rados.0.smithi195.stdout:236: done (10 left) 2024-09-13T10:10:51.553 INFO:tasks.rados.rados.0.smithi195.stdout:238: done (9 left) 2024-09-13T10:10:51.553 INFO:tasks.rados.rados.0.smithi195.stdout:239: done (8 left) 2024-09-13T10:10:51.553 INFO:tasks.rados.rados.0.smithi195.stdout:244: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.553 INFO:tasks.rados.rados.0.smithi195.stdout:244: set_chunk oid 45 offset: 1379328 length: 838656 target oid smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1379328 snap -1 2024-09-13T10:10:51.553 INFO:tasks.rados.rados.0.smithi195.stdout:245: set_chunk oid 46 target oid 46 low_tier 2024-09-13T10:10:51.553 INFO:tasks.rados.rados.0.smithi195.stdout:245: set_chunk oid 46 offset: 1426432 length: 138752 target oid smithi19530714-46 low_tier offset: 1426432 snap -1 2024-09-13T10:10:51.553 INFO:tasks.rados.rados.0.smithi195.stdout:246: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.553 INFO:tasks.rados.rados.0.smithi195.stdout:246: set_chunk oid 47 offset: 2177024 length: 399872 target oid smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 2177024 snap -1 2024-09-13T10:10:51.553 INFO:tasks.rados.rados.0.smithi195.stdout:247: set_chunk oid 48 target oid 48 low_tier 2024-09-13T10:10:51.553 INFO:tasks.rados.rados.0.smithi195.stdout:247: set_chunk oid 48 offset: 1006080 length: 540160 target oid smithi19530714-48 low_tier offset: 1006080 snap -1 2024-09-13T10:10:51.553 INFO:tasks.rados.rados.0.smithi195.stdout:248: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.553 INFO:tasks.rados.rados.0.smithi195.stdout:248: set_chunk oid 49 offset: 1474048 length: 1185280 target oid smithi19530714-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1474048 snap -1 2024-09-13T10:10:51.554 INFO:tasks.rados.rados.0.smithi195.stdout:249: set_chunk oid 50 target oid 50 low_tier 2024-09-13T10:10:51.554 INFO:tasks.rados.rados.0.smithi195.stdout:249: set_chunk oid 50 offset: 961536 length: 926208 target oid smithi19530714-50 low_tier offset: 961536 snap -1 2024-09-13T10:10:51.554 INFO:tasks.rados.rados.0.smithi195.stdout:250: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.554 INFO:tasks.rados.rados.0.smithi195.stdout:250: set_chunk oid 1 offset: 361472 length: 1528832 target oid smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 361472 snap -1 2024-09-13T10:10:51.554 INFO:tasks.rados.rados.0.smithi195.stdout:251: set_chunk oid 2 target oid 2 low_tier 2024-09-13T10:10:51.554 INFO:tasks.rados.rados.0.smithi195.stdout:251: set_chunk oid 2 offset: 328192 length: 1367552 target oid smithi19530714-2 low_tier offset: 328192 snap -1 2024-09-13T10:10:51.554 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.554 INFO:tasks.rados.rados.0.smithi195.stdout:242: finishing set_chunk to oid 43 2024-09-13T10:10:51.554 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 642560 length: 1248256 2024-09-13T10:10:51.554 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-13T10:10:51.554 INFO:tasks.rados.rados.0.smithi195.stdout:240: finishing set_chunk to oid 41 2024-09-13T10:10:51.554 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 41 set_chunk 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1618432 length: 844288 2024-09-13T10:10:51.554 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-13T10:10:51.555 INFO:tasks.rados.rados.0.smithi195.stdout:247: finishing set_chunk to oid 48 2024-09-13T10:10:51.555 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 48 set_chunk 48 low_tier returned error code -95 offset: 1006080 length: 540160 2024-09-13T10:10:51.555 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-13T10:10:51.555 INFO:tasks.rados.rados.0.smithi195.stdout:250: finishing set_chunk to oid 1 2024-09-13T10:10:51.555 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 1 set_chunk 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 361472 length: 1528832 2024-09-13T10:10:51.555 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 0 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-13T10:10:51.555 INFO:tasks.rados.rados.0.smithi195.stdout:249: finishing set_chunk to oid 50 2024-09-13T10:10:51.555 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 50 set_chunk 50 low_tier returned error code -95 offset: 961536 length: 926208 2024-09-13T10:10:51.555 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 0 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-13T10:10:51.555 INFO:tasks.rados.rados.0.smithi195.stdout:240: done (15 left) 2024-09-13T10:10:51.555 INFO:tasks.rados.rados.0.smithi195.stdout:242: done (14 left) 2024-09-13T10:10:51.555 INFO:tasks.rados.rados.0.smithi195.stdout:247: done (13 left) 2024-09-13T10:10:51.555 INFO:tasks.rados.rados.0.smithi195.stdout:249: done (12 left) 2024-09-13T10:10:51.556 INFO:tasks.rados.rados.0.smithi195.stdout:250: done (11 left) 2024-09-13T10:10:51.556 INFO:tasks.rados.rados.0.smithi195.stdout:252: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.556 INFO:tasks.rados.rados.0.smithi195.stdout:252: set_chunk oid 3 offset: 572928 length: 1291776 target oid smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 572928 snap -1 2024-09-13T10:10:51.556 INFO:tasks.rados.rados.0.smithi195.stdout:253: set_chunk oid 4 target oid 4 low_tier 2024-09-13T10:10:51.556 INFO:tasks.rados.rados.0.smithi195.stdout:253: set_chunk oid 4 offset: 689664 length: 18432 target oid smithi19530714-4 low_tier offset: 689664 snap -1 2024-09-13T10:10:51.556 INFO:tasks.rados.rados.0.smithi195.stdout:254: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.556 INFO:tasks.rados.rados.0.smithi195.stdout:254: set_chunk oid 5 offset: 1908224 length: 19456 target oid smithi19530714-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1908224 snap -1 2024-09-13T10:10:51.556 INFO:tasks.rados.rados.0.smithi195.stdout:255: set_chunk oid 6 target oid 6 low_tier 2024-09-13T10:10:51.556 INFO:tasks.rados.rados.0.smithi195.stdout:255: set_chunk oid 6 offset: 524800 length: 1141760 target oid smithi19530714-6 low_tier offset: 524800 snap -1 2024-09-13T10:10:51.556 INFO:tasks.rados.rados.0.smithi195.stdout:256: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.556 INFO:tasks.rados.rados.0.smithi195.stdout:256: set_chunk oid 7 offset: 456192 length: 379904 target oid smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 456192 snap -1 2024-09-13T10:10:51.556 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.557 INFO:tasks.rados.rados.0.smithi195.stdout:244: finishing set_chunk to oid 45 2024-09-13T10:10:51.557 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 45 set_chunk 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1379328 length: 838656 2024-09-13T10:10:51.557 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 0 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-13T10:10:51.557 INFO:tasks.rados.rados.0.smithi195.stdout:244: done (15 left) 2024-09-13T10:10:51.557 INFO:tasks.rados.rados.0.smithi195.stdout:257: set_chunk oid 8 target oid 8 low_tier 2024-09-13T10:10:51.557 INFO:tasks.rados.rados.0.smithi195.stdout:257: set_chunk oid 8 offset: 539136 length: 655872 target oid smithi19530714-8 low_tier offset: 539136 snap -1 2024-09-13T10:10:51.557 INFO:tasks.rados.rados.0.smithi195.stdout:253: finishing set_chunk to oid 4 2024-09-13T10:10:51.557 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 4 set_chunk 4 low_tier returned error code -95 offset: 689664 length: 18432 2024-09-13T10:10:51.557 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 0 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-13T10:10:51.557 INFO:tasks.rados.rados.0.smithi195.stdout:248: finishing set_chunk to oid 49 2024-09-13T10:10:51.557 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 49 set_chunk 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1474048 length: 1185280 2024-09-13T10:10:51.557 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 0 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-13T10:10:51.557 INFO:tasks.rados.rados.0.smithi195.stdout:251: finishing set_chunk to oid 2 2024-09-13T10:10:51.558 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 2 set_chunk 2 low_tier returned error code -95 offset: 328192 length: 1367552 2024-09-13T10:10:51.558 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-13T10:10:51.558 INFO:tasks.rados.rados.0.smithi195.stdout:248: done (15 left) 2024-09-13T10:10:51.558 INFO:tasks.rados.rados.0.smithi195.stdout:251: done (14 left) 2024-09-13T10:10:51.558 INFO:tasks.rados.rados.0.smithi195.stdout:253: done (13 left) 2024-09-13T10:10:51.558 INFO:tasks.rados.rados.0.smithi195.stdout:258: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.558 INFO:tasks.rados.rados.0.smithi195.stdout:258: set_chunk oid 9 offset: 47616 length: 567296 target oid smithi19530714-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 47616 snap -1 2024-09-13T10:10:51.558 INFO:tasks.rados.rados.0.smithi195.stdout:259: set_chunk oid 10 target oid 10 low_tier 2024-09-13T10:10:51.558 INFO:tasks.rados.rados.0.smithi195.stdout:256: finishing set_chunk to oid 7 2024-09-13T10:10:51.558 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 456192 length: 379904 2024-09-13T10:10:51.558 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-13T10:10:51.558 INFO:tasks.rados.rados.0.smithi195.stdout:259: set_chunk oid 10 offset: 697856 length: 538112 target oid smithi19530714-10 low_tier offset: 697856 snap -1 2024-09-13T10:10:51.558 INFO:tasks.rados.rados.0.smithi195.stdout:256: done (14 left) 2024-09-13T10:10:51.558 INFO:tasks.rados.rados.0.smithi195.stdout:260: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.559 INFO:tasks.rados.rados.0.smithi195.stdout:260: set_chunk oid 11 offset: 521216 length: 520192 target oid smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 521216 snap -1 2024-09-13T10:10:51.559 INFO:tasks.rados.rados.0.smithi195.stdout:252: finishing set_chunk to oid 3 2024-09-13T10:10:51.559 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 3 set_chunk 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 572928 length: 1291776 2024-09-13T10:10:51.559 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 0 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-13T10:10:51.559 INFO:tasks.rados.rados.0.smithi195.stdout:257: finishing set_chunk to oid 8 2024-09-13T10:10:51.559 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 8 set_chunk 8 low_tier returned error code -95 offset: 539136 length: 655872 2024-09-13T10:10:51.559 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-13T10:10:51.559 INFO:tasks.rados.rados.0.smithi195.stdout:252: done (14 left) 2024-09-13T10:10:51.559 INFO:tasks.rados.rados.0.smithi195.stdout:257: done (13 left) 2024-09-13T10:10:51.559 INFO:tasks.rados.rados.0.smithi195.stdout:261: set_chunk oid 12 target oid 12 low_tier 2024-09-13T10:10:51.559 INFO:tasks.rados.rados.0.smithi195.stdout:261: set_chunk oid 12 offset: 957952 length: 240128 target oid smithi19530714-12 low_tier offset: 957952 snap -1 2024-09-13T10:10:51.559 INFO:tasks.rados.rados.0.smithi195.stdout:262: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.559 INFO:tasks.rados.rados.0.smithi195.stdout:262: set_chunk oid 13 offset: 591872 length: 1571840 target oid smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 591872 snap -1 2024-09-13T10:10:51.559 INFO:tasks.rados.rados.0.smithi195.stdout:263: set_chunk oid 14 target oid 14 low_tier 2024-09-13T10:10:51.560 INFO:tasks.rados.rados.0.smithi195.stdout:263: set_chunk oid 14 offset: 49664 length: 392192 target oid smithi19530714-14 low_tier offset: 49664 snap -1 2024-09-13T10:10:51.560 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.560 INFO:tasks.rados.rados.0.smithi195.stdout:259: finishing set_chunk to oid 10 2024-09-13T10:10:51.560 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 10 set_chunk 10 low_tier returned error code -95 offset: 697856 length: 538112 2024-09-13T10:10:51.560 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 0 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-13T10:10:51.560 INFO:tasks.rados.rados.0.smithi195.stdout:259: done (15 left) 2024-09-13T10:10:51.560 INFO:tasks.rados.rados.0.smithi195.stdout:264: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.560 INFO:tasks.rados.rados.0.smithi195.stdout:264: set_chunk oid 15 offset: 1828864 length: 107008 target oid smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1828864 snap -1 2024-09-13T10:10:51.560 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.560 INFO:tasks.rados.rados.0.smithi195.stdout:258: finishing set_chunk to oid 9 2024-09-13T10:10:51.560 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 9 set_chunk 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 47616 length: 567296 2024-09-13T10:10:51.560 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 0 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-13T10:10:51.560 INFO:tasks.rados.rados.0.smithi195.stdout:258: done (15 left) 2024-09-13T10:10:51.561 INFO:tasks.rados.rados.0.smithi195.stdout:265: set_chunk oid 16 target oid 16 low_tier 2024-09-13T10:10:51.561 INFO:tasks.rados.rados.0.smithi195.stdout:265: set_chunk oid 16 offset: 1268224 length: 44544 target oid smithi19530714-16 low_tier offset: 1268224 snap -1 2024-09-13T10:10:51.561 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.561 INFO:tasks.rados.rados.0.smithi195.stdout:260: finishing set_chunk to oid 11 2024-09-13T10:10:51.561 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 11 set_chunk 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 521216 length: 520192 2024-09-13T10:10:51.561 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 0 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-13T10:10:51.561 INFO:tasks.rados.rados.0.smithi195.stdout:261: finishing set_chunk to oid 12 2024-09-13T10:10:51.561 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 12 set_chunk 12 low_tier returned error code -95 offset: 957952 length: 240128 2024-09-13T10:10:51.561 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-13T10:10:51.561 INFO:tasks.rados.rados.0.smithi195.stdout:255: finishing set_chunk to oid 6 2024-09-13T10:10:51.561 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 6 set_chunk 6 low_tier returned error code -95 offset: 524800 length: 1141760 2024-09-13T10:10:51.561 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 0 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-13T10:10:51.561 INFO:tasks.rados.rados.0.smithi195.stdout:255: done (15 left) 2024-09-13T10:10:51.561 INFO:tasks.rados.rados.0.smithi195.stdout:260: done (14 left) 2024-09-13T10:10:51.562 INFO:tasks.rados.rados.0.smithi195.stdout:261: done (13 left) 2024-09-13T10:10:51.562 INFO:tasks.rados.rados.0.smithi195.stdout:266: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.562 INFO:tasks.rados.rados.0.smithi195.stdout:266: set_chunk oid 17 offset: 834560 length: 1210368 target oid smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 834560 snap -1 2024-09-13T10:10:51.562 INFO:tasks.rados.rados.0.smithi195.stdout:267: set_chunk oid 18 target oid 18 low_tier 2024-09-13T10:10:51.562 INFO:tasks.rados.rados.0.smithi195.stdout:267: set_chunk oid 18 offset: 379392 length: 625664 target oid smithi19530714-18 low_tier offset: 379392 snap -1 2024-09-13T10:10:51.562 INFO:tasks.rados.rados.0.smithi195.stdout:268: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.562 INFO:tasks.rados.rados.0.smithi195.stdout:268: set_chunk oid 19 offset: 1141248 length: 239616 target oid smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1141248 snap -1 2024-09-13T10:10:51.562 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.562 INFO:tasks.rados.rados.0.smithi195.stdout:262: finishing set_chunk to oid 13 2024-09-13T10:10:51.562 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 591872 length: 1571840 2024-09-13T10:10:51.562 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-13T10:10:51.562 INFO:tasks.rados.rados.0.smithi195.stdout:263: finishing set_chunk to oid 14 2024-09-13T10:10:51.562 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 14 set_chunk 14 low_tier returned error code -95 offset: 49664 length: 392192 2024-09-13T10:10:51.563 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 0 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-13T10:10:51.563 INFO:tasks.rados.rados.0.smithi195.stdout:265: finishing set_chunk to oid 16 2024-09-13T10:10:51.563 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 1268224 length: 44544 2024-09-13T10:10:51.563 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-13T10:10:51.563 INFO:tasks.rados.rados.0.smithi195.stdout:266: finishing set_chunk to oid 17 2024-09-13T10:10:51.563 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 834560 length: 1210368 2024-09-13T10:10:51.563 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-13T10:10:51.563 INFO:tasks.rados.rados.0.smithi195.stdout:262: done (15 left) 2024-09-13T10:10:51.563 INFO:tasks.rados.rados.0.smithi195.stdout:263: done (14 left) 2024-09-13T10:10:51.563 INFO:tasks.rados.rados.0.smithi195.stdout:265: done (13 left) 2024-09-13T10:10:51.563 INFO:tasks.rados.rados.0.smithi195.stdout:266: done (12 left) 2024-09-13T10:10:51.563 INFO:tasks.rados.rados.0.smithi195.stdout:269: set_chunk oid 20 target oid 20 low_tier 2024-09-13T10:10:51.563 INFO:tasks.rados.rados.0.smithi195.stdout:269: set_chunk oid 20 offset: 819200 length: 739840 target oid smithi19530714-20 low_tier offset: 819200 snap -1 2024-09-13T10:10:51.564 INFO:tasks.rados.rados.0.smithi195.stdout:270: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.564 INFO:tasks.rados.rados.0.smithi195.stdout:270: set_chunk oid 21 offset: 394240 length: 972288 target oid smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 394240 snap -1 2024-09-13T10:10:51.564 INFO:tasks.rados.rados.0.smithi195.stdout:271: set_chunk oid 22 target oid 22 low_tier 2024-09-13T10:10:51.564 INFO:tasks.rados.rados.0.smithi195.stdout:271: set_chunk oid 22 offset: 461824 length: 1789952 target oid smithi19530714-22 low_tier offset: 461824 snap -1 2024-09-13T10:10:51.564 INFO:tasks.rados.rados.0.smithi195.stdout:267: finishing set_chunk to oid 18 2024-09-13T10:10:51.564 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 379392 length: 625664 2024-09-13T10:10:51.564 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-13T10:10:51.564 INFO:tasks.rados.rados.0.smithi195.stdout:267: done (14 left) 2024-09-13T10:10:51.564 INFO:tasks.rados.rados.0.smithi195.stdout:272: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.564 INFO:tasks.rados.rados.0.smithi195.stdout:272: set_chunk oid 23 offset: 401408 length: 676352 target oid smithi19530714-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 401408 snap -1 2024-09-13T10:10:51.564 INFO:tasks.rados.rados.0.smithi195.stdout:273: set_chunk oid 24 target oid 24 low_tier 2024-09-13T10:10:51.564 INFO:tasks.rados.rados.0.smithi195.stdout:273: set_chunk oid 24 offset: 1562624 length: 424960 target oid smithi19530714-24 low_tier offset: 1562624 snap -1 2024-09-13T10:10:51.564 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.564 INFO:tasks.rados.rados.0.smithi195.stdout:270: finishing set_chunk to oid 21 2024-09-13T10:10:51.565 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 21 set_chunk 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 394240 length: 972288 2024-09-13T10:10:51.565 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 0 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-13T10:10:51.565 INFO:tasks.rados.rados.0.smithi195.stdout:270: done (15 left) 2024-09-13T10:10:51.565 INFO:tasks.rados.rados.0.smithi195.stdout:274: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.565 INFO:tasks.rados.rados.0.smithi195.stdout:274: set_chunk oid 25 offset: 573952 length: 77824 target oid smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 573952 snap -1 2024-09-13T10:10:51.565 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.565 INFO:tasks.rados.rados.0.smithi195.stdout:269: finishing set_chunk to oid 20 2024-09-13T10:10:51.565 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 819200 length: 739840 2024-09-13T10:10:51.565 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-13T10:10:51.565 INFO:tasks.rados.rados.0.smithi195.stdout:269: done (15 left) 2024-09-13T10:10:51.565 INFO:tasks.rados.rados.0.smithi195.stdout:275: set_chunk oid 26 target oid 26 low_tier 2024-09-13T10:10:51.565 INFO:tasks.rados.rados.0.smithi195.stdout:271: finishing set_chunk to oid 22 2024-09-13T10:10:51.565 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 461824 length: 1789952 2024-09-13T10:10:51.566 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-13T10:10:51.566 INFO:tasks.rados.rados.0.smithi195.stdout:275: set_chunk oid 26 offset: 257024 length: 2080256 target oid smithi19530714-26 low_tier offset: 257024 snap -1 2024-09-13T10:10:51.566 INFO:tasks.rados.rados.0.smithi195.stdout:271: done (15 left) 2024-09-13T10:10:51.566 INFO:tasks.rados.rados.0.smithi195.stdout:276: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.566 INFO:tasks.rados.rados.0.smithi195.stdout:276: set_chunk oid 27 offset: 1079296 length: 336384 target oid smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1079296 snap -1 2024-09-13T10:10:51.566 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.566 INFO:tasks.rados.rados.0.smithi195.stdout:237: finishing set_chunk to oid 38 2024-09-13T10:10:51.566 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 12 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-13T10:10:51.566 INFO:tasks.rados.rados.0.smithi195.stdout:237: done (15 left) 2024-09-13T10:10:51.566 INFO:tasks.rados.rados.0.smithi195.stdout:277: set_chunk oid 28 target oid 28 low_tier 2024-09-13T10:10:51.566 INFO:tasks.rados.rados.0.smithi195.stdout:277: set_chunk oid 28 offset: 2390016 length: 56320 target oid smithi19530714-28 low_tier offset: 2390016 snap -1 2024-09-13T10:10:51.566 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.566 INFO:tasks.rados.rados.0.smithi195.stdout:275: finishing set_chunk to oid 26 2024-09-13T10:10:51.567 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 26 set_chunk 26 low_tier returned error code -95 offset: 257024 length: 2080256 2024-09-13T10:10:51.567 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 0 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-13T10:10:51.567 INFO:tasks.rados.rados.0.smithi195.stdout:276: finishing set_chunk to oid 27 2024-09-13T10:10:51.567 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 27 set_chunk 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1079296 length: 336384 2024-09-13T10:10:51.567 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-13T10:10:51.567 INFO:tasks.rados.rados.0.smithi195.stdout:272: finishing set_chunk to oid 23 2024-09-13T10:10:51.567 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 23 set_chunk 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 401408 length: 676352 2024-09-13T10:10:51.567 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-13T10:10:51.567 INFO:tasks.rados.rados.0.smithi195.stdout:272: done (15 left) 2024-09-13T10:10:51.567 INFO:tasks.rados.rados.0.smithi195.stdout:275: done (14 left) 2024-09-13T10:10:51.567 INFO:tasks.rados.rados.0.smithi195.stdout:276: done (13 left) 2024-09-13T10:10:51.567 INFO:tasks.rados.rados.0.smithi195.stdout:277 retry set_chunk ! 2024-09-13T10:10:51.567 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 13 2024-09-13T10:10:51.568 INFO:tasks.rados.rados.0.smithi195.stdout:232: finishing set_chunk to oid 33 2024-09-13T10:10:51.568 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 6 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-13T10:10:51.568 INFO:tasks.rados.rados.0.smithi195.stdout:234: finishing set_chunk to oid 35 2024-09-13T10:10:51.568 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 12 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-13T10:10:51.568 INFO:tasks.rados.rados.0.smithi195.stdout:246: finishing set_chunk to oid 47 2024-09-13T10:10:51.568 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 9 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-13T10:10:51.568 INFO:tasks.rados.rados.0.smithi195.stdout:228: finishing set_chunk to oid 29 2024-09-13T10:10:51.568 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 12 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-13T10:10:51.568 INFO:tasks.rados.rados.0.smithi195.stdout:228: done (12 left) 2024-09-13T10:10:51.568 INFO:tasks.rados.rados.0.smithi195.stdout:232: done (11 left) 2024-09-13T10:10:51.568 INFO:tasks.rados.rados.0.smithi195.stdout:234: done (10 left) 2024-09-13T10:10:51.568 INFO:tasks.rados.rados.0.smithi195.stdout:246: done (9 left) 2024-09-13T10:10:51.568 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 9 2024-09-13T10:10:51.568 INFO:tasks.rados.rados.0.smithi195.stdout:243: finishing set_chunk to oid 44 2024-09-13T10:10:51.569 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 9 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-13T10:10:51.569 INFO:tasks.rados.rados.0.smithi195.stdout:243: done (8 left) 2024-09-13T10:10:51.569 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 8 2024-09-13T10:10:51.569 INFO:tasks.rados.rados.0.smithi195.stdout:245: finishing set_chunk to oid 46 2024-09-13T10:10:51.569 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 6 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-13T10:10:51.569 INFO:tasks.rados.rados.0.smithi195.stdout:245: done (7 left) 2024-09-13T10:10:51.572 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 7 2024-09-13T10:10:51.572 INFO:tasks.rados.rados.0.smithi195.stdout:273: finishing set_chunk to oid 24 2024-09-13T10:10:51.572 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 3 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-13T10:10:51.572 INFO:tasks.rados.rados.0.smithi195.stdout:273: done (6 left) 2024-09-13T10:10:51.572 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 6 2024-09-13T10:10:51.572 INFO:tasks.rados.rados.0.smithi195.stdout:264: finishing set_chunk to oid 15 2024-09-13T10:10:51.572 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 9 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-13T10:10:51.573 INFO:tasks.rados.rados.0.smithi195.stdout:264: done (5 left) 2024-09-13T10:10:51.573 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 5 2024-09-13T10:10:51.573 INFO:tasks.rados.rados.0.smithi195.stdout:254: finishing set_chunk to oid 5 2024-09-13T10:10:51.573 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 3 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-13T10:10:51.573 INFO:tasks.rados.rados.0.smithi195.stdout:254: done (4 left) 2024-09-13T10:10:51.573 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 4 2024-09-13T10:10:51.573 INFO:tasks.rados.rados.0.smithi195.stdout:268: finishing set_chunk to oid 19 2024-09-13T10:10:51.573 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 9 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-13T10:10:51.573 INFO:tasks.rados.rados.0.smithi195.stdout:268: done (3 left) 2024-09-13T10:10:51.573 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 3 2024-09-13T10:10:51.573 INFO:tasks.rados.rados.0.smithi195.stdout:241: finishing set_chunk to oid 42 2024-09-13T10:10:51.573 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 6 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-13T10:10:51.573 INFO:tasks.rados.rados.0.smithi195.stdout:241: done (2 left) 2024-09-13T10:10:51.573 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 2 2024-09-13T10:10:51.574 INFO:tasks.rados.rados.0.smithi195.stdout:274: finishing set_chunk to oid 25 2024-09-13T10:10:51.574 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 6 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-13T10:10:51.574 INFO:tasks.rados.rados.0.smithi195.stdout:274: done (1 left) 2024-09-13T10:10:51.574 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 1 2024-09-13T10:10:51.574 INFO:tasks.rados.rados.0.smithi195.stdout:277: finishing set_chunk to oid 28 2024-09-13T10:10:51.574 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 9 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-13T10:10:51.574 INFO:tasks.rados.rados.0.smithi195.stdout:277: done (0 left) 2024-09-13T10:10:51.574 INFO:tasks.rados.rados.0.smithi195.stdout:278: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.574 INFO:tasks.rados.rados.0.smithi195.stdout:278: set_chunk oid 29 offset: 790016 length: 1564672 target oid smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 790016 snap -1 2024-09-13T10:10:51.574 INFO:tasks.rados.rados.0.smithi195.stdout:279: set_chunk oid 30 target oid 30 low_tier 2024-09-13T10:10:51.574 INFO:tasks.rados.rados.0.smithi195.stdout:279: set_chunk oid 30 offset: 883200 length: 625664 target oid smithi19530714-30 low_tier offset: 883200 snap -1 2024-09-13T10:10:51.574 INFO:tasks.rados.rados.0.smithi195.stdout:280: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.574 INFO:tasks.rados.rados.0.smithi195.stdout:280: set_chunk oid 31 offset: 1738752 length: 88576 target oid smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1738752 snap -1 2024-09-13T10:10:51.575 INFO:tasks.rados.rados.0.smithi195.stdout:281: set_chunk oid 32 target oid 32 low_tier 2024-09-13T10:10:51.575 INFO:tasks.rados.rados.0.smithi195.stdout:281: set_chunk oid 32 offset: 467456 length: 1510912 target oid smithi19530714-32 low_tier offset: 467456 snap -1 2024-09-13T10:10:51.575 INFO:tasks.rados.rados.0.smithi195.stdout:282: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.575 INFO:tasks.rados.rados.0.smithi195.stdout:282: set_chunk oid 33 offset: 1769984 length: 594944 target oid smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1769984 snap -1 2024-09-13T10:10:51.575 INFO:tasks.rados.rados.0.smithi195.stdout:283: set_chunk oid 34 target oid 34 low_tier 2024-09-13T10:10:51.575 INFO:tasks.rados.rados.0.smithi195.stdout:283: set_chunk oid 34 offset: 428032 length: 165888 target oid smithi19530714-34 low_tier offset: 428032 snap -1 2024-09-13T10:10:51.575 INFO:tasks.rados.rados.0.smithi195.stdout:284: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.575 INFO:tasks.rados.rados.0.smithi195.stdout:284: set_chunk oid 35 offset: 1022464 length: 217088 target oid smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1022464 snap -1 2024-09-13T10:10:51.575 INFO:tasks.rados.rados.0.smithi195.stdout:285: set_chunk oid 36 target oid 36 low_tier 2024-09-13T10:10:51.575 INFO:tasks.rados.rados.0.smithi195.stdout:285: set_chunk oid 36 offset: 138752 length: 551424 target oid smithi19530714-36 low_tier offset: 138752 snap -1 2024-09-13T10:10:51.576 INFO:tasks.rados.rados.0.smithi195.stdout:286: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.576 INFO:tasks.rados.rados.0.smithi195.stdout:286: set_chunk oid 37 offset: 489472 length: 815616 target oid smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 489472 snap -1 2024-09-13T10:10:51.576 INFO:tasks.rados.rados.0.smithi195.stdout:287: set_chunk oid 38 target oid 38 low_tier 2024-09-13T10:10:51.576 INFO:tasks.rados.rados.0.smithi195.stdout:287: set_chunk oid 38 offset: 1055744 length: 1519616 target oid smithi19530714-38 low_tier offset: 1055744 snap -1 2024-09-13T10:10:51.576 INFO:tasks.rados.rados.0.smithi195.stdout:288: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.576 INFO:tasks.rados.rados.0.smithi195.stdout:288: set_chunk oid 39 offset: 821248 length: 455680 target oid smithi19530714-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 821248 snap -1 2024-09-13T10:10:51.576 INFO:tasks.rados.rados.0.smithi195.stdout:289: set_chunk oid 40 target oid 40 low_tier 2024-09-13T10:10:51.576 INFO:tasks.rados.rados.0.smithi195.stdout:289: set_chunk oid 40 offset: 153600 length: 1208832 target oid smithi19530714-40 low_tier offset: 153600 snap -1 2024-09-13T10:10:51.576 INFO:tasks.rados.rados.0.smithi195.stdout:290: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.576 INFO:tasks.rados.rados.0.smithi195.stdout:290: set_chunk oid 41 offset: 2462720 length: 171520 target oid smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 2462720 snap -1 2024-09-13T10:10:51.576 INFO:tasks.rados.rados.0.smithi195.stdout:291: set_chunk oid 42 target oid 42 low_tier 2024-09-13T10:10:51.576 INFO:tasks.rados.rados.0.smithi195.stdout:291: set_chunk oid 42 offset: 156672 length: 1352704 target oid smithi19530714-42 low_tier offset: 156672 snap -1 2024-09-13T10:10:51.576 INFO:tasks.rados.rados.0.smithi195.stdout:292: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.577 INFO:tasks.rados.rados.0.smithi195.stdout:292: set_chunk oid 43 offset: 552448 length: 376832 target oid smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 552448 snap -1 2024-09-13T10:10:51.577 INFO:tasks.rados.rados.0.smithi195.stdout:293: set_chunk oid 44 target oid 44 low_tier 2024-09-13T10:10:51.577 INFO:tasks.rados.rados.0.smithi195.stdout:293: set_chunk oid 44 offset: 1905152 length: 795648 target oid smithi19530714-44 low_tier offset: 1905152 snap -1 2024-09-13T10:10:51.577 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.577 INFO:tasks.rados.rados.0.smithi195.stdout:279: finishing set_chunk to oid 30 2024-09-13T10:10:51.577 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 30 set_chunk 30 low_tier returned error code -95 offset: 883200 length: 625664 2024-09-13T10:10:51.577 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 0 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-13T10:10:51.577 INFO:tasks.rados.rados.0.smithi195.stdout:280: finishing set_chunk to oid 31 2024-09-13T10:10:51.577 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 31 set_chunk 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1738752 length: 88576 2024-09-13T10:10:51.577 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-13T10:10:51.577 INFO:tasks.rados.rados.0.smithi195.stdout:285: finishing set_chunk to oid 36 2024-09-13T10:10:51.577 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 36 set_chunk 36 low_tier returned error code -95 offset: 138752 length: 551424 2024-09-13T10:10:51.578 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 0 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-13T10:10:51.578 INFO:tasks.rados.rados.0.smithi195.stdout:279: done (15 left) 2024-09-13T10:10:51.578 INFO:tasks.rados.rados.0.smithi195.stdout:280: done (14 left) 2024-09-13T10:10:51.578 INFO:tasks.rados.rados.0.smithi195.stdout:285: done (13 left) 2024-09-13T10:10:51.578 INFO:tasks.rados.rados.0.smithi195.stdout:294: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.578 INFO:tasks.rados.rados.0.smithi195.stdout:287: finishing set_chunk to oid 38 2024-09-13T10:10:51.578 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 38 set_chunk 38 low_tier returned error code -95 offset: 1055744 length: 1519616 2024-09-13T10:10:51.578 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 0 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-13T10:10:51.578 INFO:tasks.rados.rados.0.smithi195.stdout:286: finishing set_chunk to oid 37 2024-09-13T10:10:51.578 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 37 set_chunk 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 489472 length: 815616 2024-09-13T10:10:51.578 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 0 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-13T10:10:51.578 INFO:tasks.rados.rados.0.smithi195.stdout:278: finishing set_chunk to oid 29 2024-09-13T10:10:51.579 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 29 set_chunk 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 790016 length: 1564672 2024-09-13T10:10:51.579 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 0 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-13T10:10:51.579 INFO:tasks.rados.rados.0.smithi195.stdout:281: finishing set_chunk to oid 32 2024-09-13T10:10:51.579 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 467456 length: 1510912 2024-09-13T10:10:51.579 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-13T10:10:51.579 INFO:tasks.rados.rados.0.smithi195.stdout:292: finishing set_chunk to oid 43 2024-09-13T10:10:51.579 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 552448 length: 376832 2024-09-13T10:10:51.579 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-13T10:10:51.579 INFO:tasks.rados.rados.0.smithi195.stdout:284: finishing set_chunk to oid 35 2024-09-13T10:10:51.579 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 35 set_chunk 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1022464 length: 217088 2024-09-13T10:10:51.579 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 0 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-13T10:10:51.579 INFO:tasks.rados.rados.0.smithi195.stdout:289: finishing set_chunk to oid 40 2024-09-13T10:10:51.579 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 40 set_chunk 40 low_tier returned error code -95 offset: 153600 length: 1208832 2024-09-13T10:10:51.579 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 0 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-13T10:10:51.580 INFO:tasks.rados.rados.0.smithi195.stdout:294: set_chunk oid 45 offset: 717312 length: 128000 target oid smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 717312 snap -1 2024-09-13T10:10:51.580 INFO:tasks.rados.rados.0.smithi195.stdout:278: done (13 left) 2024-09-13T10:10:51.580 INFO:tasks.rados.rados.0.smithi195.stdout:281: done (12 left) 2024-09-13T10:10:51.580 INFO:tasks.rados.rados.0.smithi195.stdout:284: done (11 left) 2024-09-13T10:10:51.580 INFO:tasks.rados.rados.0.smithi195.stdout:286: done (10 left) 2024-09-13T10:10:51.580 INFO:tasks.rados.rados.0.smithi195.stdout:287: done (9 left) 2024-09-13T10:10:51.580 INFO:tasks.rados.rados.0.smithi195.stdout:289: done (8 left) 2024-09-13T10:10:51.580 INFO:tasks.rados.rados.0.smithi195.stdout:292: done (7 left) 2024-09-13T10:10:51.580 INFO:tasks.rados.rados.0.smithi195.stdout:295: set_chunk oid 46 target oid 46 low_tier 2024-09-13T10:10:51.580 INFO:tasks.rados.rados.0.smithi195.stdout:295: set_chunk oid 46 offset: 547328 length: 357888 target oid smithi19530714-46 low_tier offset: 547328 snap -1 2024-09-13T10:10:51.581 INFO:tasks.rados.rados.0.smithi195.stdout:296: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.581 INFO:tasks.rados.rados.0.smithi195.stdout:296: set_chunk oid 47 offset: 667136 length: 749568 target oid smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 667136 snap -1 2024-09-13T10:10:51.581 INFO:tasks.rados.rados.0.smithi195.stdout:297: set_chunk oid 48 target oid 48 low_tier 2024-09-13T10:10:51.581 INFO:tasks.rados.rados.0.smithi195.stdout:282: finishing set_chunk to oid 33 2024-09-13T10:10:51.581 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 33 set_chunk 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1769984 length: 594944 2024-09-13T10:10:51.581 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 0 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-13T10:10:51.581 INFO:tasks.rados.rados.0.smithi195.stdout:297: set_chunk oid 48 offset: 303104 length: 2416640 target oid smithi19530714-48 low_tier offset: 303104 snap -1 2024-09-13T10:10:51.581 INFO:tasks.rados.rados.0.smithi195.stdout:282: done (9 left) 2024-09-13T10:10:51.581 INFO:tasks.rados.rados.0.smithi195.stdout:298: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.581 INFO:tasks.rados.rados.0.smithi195.stdout:298: set_chunk oid 49 offset: 774656 length: 1175040 target oid smithi19530714-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 774656 snap -1 2024-09-13T10:10:51.581 INFO:tasks.rados.rados.0.smithi195.stdout:299: set_chunk oid 50 target oid 50 low_tier 2024-09-13T10:10:51.581 INFO:tasks.rados.rados.0.smithi195.stdout:299: set_chunk oid 50 offset: 644608 length: 1198080 target oid smithi19530714-50 low_tier offset: 644608 snap -1 2024-09-13T10:10:51.581 INFO:tasks.rados.rados.0.smithi195.stdout:300: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.582 INFO:tasks.rados.rados.0.smithi195.stdout:300: set_chunk oid 1 offset: 1076224 length: 862208 target oid smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1076224 snap -1 2024-09-13T10:10:51.582 INFO:tasks.rados.rados.0.smithi195.stdout:301: set_chunk oid 2 target oid 2 low_tier 2024-09-13T10:10:51.582 INFO:tasks.rados.rados.0.smithi195.stdout:301: set_chunk oid 2 offset: 956928 length: 1031168 target oid smithi19530714-2 low_tier offset: 956928 snap -1 2024-09-13T10:10:51.582 INFO:tasks.rados.rados.0.smithi195.stdout:291: finishing set_chunk to oid 42 2024-09-13T10:10:51.582 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 42 set_chunk 42 low_tier returned error code -95 offset: 156672 length: 1352704 2024-09-13T10:10:51.582 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-13T10:10:51.582 INFO:tasks.rados.rados.0.smithi195.stdout:295: finishing set_chunk to oid 46 2024-09-13T10:10:51.582 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 46 set_chunk 46 low_tier returned error code -95 offset: 547328 length: 357888 2024-09-13T10:10:51.582 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 0 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-13T10:10:51.582 INFO:tasks.rados.rados.0.smithi195.stdout:291: done (12 left) 2024-09-13T10:10:51.582 INFO:tasks.rados.rados.0.smithi195.stdout:295: done (11 left) 2024-09-13T10:10:51.582 INFO:tasks.rados.rados.0.smithi195.stdout:302: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.582 INFO:tasks.rados.rados.0.smithi195.stdout:302: set_chunk oid 3 offset: 449536 length: 1348608 target oid smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 449536 snap -1 2024-09-13T10:10:51.582 INFO:tasks.rados.rados.0.smithi195.stdout:303: set_chunk oid 4 target oid 4 low_tier 2024-09-13T10:10:51.582 INFO:tasks.rados.rados.0.smithi195.stdout:303: set_chunk oid 4 offset: 345600 length: 1569280 target oid smithi19530714-4 low_tier offset: 345600 snap -1 2024-09-13T10:10:51.583 INFO:tasks.rados.rados.0.smithi195.stdout:304: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.583 INFO:tasks.rados.rados.0.smithi195.stdout:297: finishing set_chunk to oid 48 2024-09-13T10:10:51.583 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 48 set_chunk 48 low_tier returned error code -95 offset: 303104 length: 2416640 2024-09-13T10:10:51.583 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-13T10:10:51.583 INFO:tasks.rados.rados.0.smithi195.stdout:304: set_chunk oid 5 offset: 759296 length: 586752 target oid smithi19530714-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 759296 snap -1 2024-09-13T10:10:51.583 INFO:tasks.rados.rados.0.smithi195.stdout:297: done (13 left) 2024-09-13T10:10:51.583 INFO:tasks.rados.rados.0.smithi195.stdout:305: set_chunk oid 6 target oid 6 low_tier 2024-09-13T10:10:51.583 INFO:tasks.rados.rados.0.smithi195.stdout:305: set_chunk oid 6 offset: 1506304 length: 151040 target oid smithi19530714-6 low_tier offset: 1506304 snap -1 2024-09-13T10:10:51.583 INFO:tasks.rados.rados.0.smithi195.stdout:306: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.583 INFO:tasks.rados.rados.0.smithi195.stdout:306: set_chunk oid 7 offset: 861696 length: 46080 target oid smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 861696 snap -1 2024-09-13T10:10:51.583 INFO:tasks.rados.rados.0.smithi195.stdout:299: finishing set_chunk to oid 50 2024-09-13T10:10:51.583 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 50 set_chunk 50 low_tier returned error code -95 offset: 644608 length: 1198080 2024-09-13T10:10:51.583 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 0 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-13T10:10:51.584 INFO:tasks.rados.rados.0.smithi195.stdout:299: done (14 left) 2024-09-13T10:10:51.584 INFO:tasks.rados.rados.0.smithi195.stdout:307: set_chunk oid 8 target oid 8 low_tier 2024-09-13T10:10:51.584 INFO:tasks.rados.rados.0.smithi195.stdout:307: set_chunk oid 8 offset: 57344 length: 558080 target oid smithi19530714-8 low_tier offset: 57344 snap -1 2024-09-13T10:10:51.584 INFO:tasks.rados.rados.0.smithi195.stdout:308: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.584 INFO:tasks.rados.rados.0.smithi195.stdout:308: set_chunk oid 9 offset: 90112 length: 1654272 target oid smithi19530714-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 90112 snap -1 2024-09-13T10:10:51.584 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.584 INFO:tasks.rados.rados.0.smithi195.stdout:294: finishing set_chunk to oid 45 2024-09-13T10:10:51.584 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 45 set_chunk 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 717312 length: 128000 2024-09-13T10:10:51.584 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 0 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-13T10:10:51.584 INFO:tasks.rados.rados.0.smithi195.stdout:298: finishing set_chunk to oid 49 2024-09-13T10:10:51.584 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 49 set_chunk 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 774656 length: 1175040 2024-09-13T10:10:51.584 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 0 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-13T10:10:51.585 INFO:tasks.rados.rados.0.smithi195.stdout:303: finishing set_chunk to oid 4 2024-09-13T10:10:51.585 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 4 set_chunk 4 low_tier returned error code -95 offset: 345600 length: 1569280 2024-09-13T10:10:51.585 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 0 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-13T10:10:51.585 INFO:tasks.rados.rados.0.smithi195.stdout:301: finishing set_chunk to oid 2 2024-09-13T10:10:51.585 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 2 set_chunk 2 low_tier returned error code -95 offset: 956928 length: 1031168 2024-09-13T10:10:51.585 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-13T10:10:51.585 INFO:tasks.rados.rados.0.smithi195.stdout:296: finishing set_chunk to oid 47 2024-09-13T10:10:51.585 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 47 set_chunk 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 667136 length: 749568 2024-09-13T10:10:51.585 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-13T10:10:51.585 INFO:tasks.rados.rados.0.smithi195.stdout:294: done (15 left) 2024-09-13T10:10:51.585 INFO:tasks.rados.rados.0.smithi195.stdout:296: done (14 left) 2024-09-13T10:10:51.585 INFO:tasks.rados.rados.0.smithi195.stdout:298: done (13 left) 2024-09-13T10:10:51.585 INFO:tasks.rados.rados.0.smithi195.stdout:301: done (12 left) 2024-09-13T10:10:51.585 INFO:tasks.rados.rados.0.smithi195.stdout:303: done (11 left) 2024-09-13T10:10:51.586 INFO:tasks.rados.rados.0.smithi195.stdout:309: set_chunk oid 10 target oid 10 low_tier 2024-09-13T10:10:51.586 INFO:tasks.rados.rados.0.smithi195.stdout:309: set_chunk oid 10 offset: 314880 length: 200704 target oid smithi19530714-10 low_tier offset: 314880 snap -1 2024-09-13T10:10:51.586 INFO:tasks.rados.rados.0.smithi195.stdout:310: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.586 INFO:tasks.rados.rados.0.smithi195.stdout:306: finishing set_chunk to oid 7 2024-09-13T10:10:51.586 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 861696 length: 46080 2024-09-13T10:10:51.586 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-13T10:10:51.586 INFO:tasks.rados.rados.0.smithi195.stdout:307: finishing set_chunk to oid 8 2024-09-13T10:10:51.586 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 8 set_chunk 8 low_tier returned error code -95 offset: 57344 length: 558080 2024-09-13T10:10:51.586 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-13T10:10:51.586 INFO:tasks.rados.rados.0.smithi195.stdout:310: set_chunk oid 11 offset: 1051136 length: 719360 target oid smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1051136 snap -1 2024-09-13T10:10:51.586 INFO:tasks.rados.rados.0.smithi195.stdout:306: done (12 left) 2024-09-13T10:10:51.586 INFO:tasks.rados.rados.0.smithi195.stdout:307: done (11 left) 2024-09-13T10:10:51.586 INFO:tasks.rados.rados.0.smithi195.stdout:311: set_chunk oid 12 target oid 12 low_tier 2024-09-13T10:10:51.586 INFO:tasks.rados.rados.0.smithi195.stdout:311: set_chunk oid 12 offset: 793088 length: 582144 target oid smithi19530714-12 low_tier offset: 793088 snap -1 2024-09-13T10:10:51.587 INFO:tasks.rados.rados.0.smithi195.stdout:312: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.587 INFO:tasks.rados.rados.0.smithi195.stdout:312: set_chunk oid 13 offset: 415744 length: 946176 target oid smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 415744 snap -1 2024-09-13T10:10:51.587 INFO:tasks.rados.rados.0.smithi195.stdout:313: set_chunk oid 14 target oid 14 low_tier 2024-09-13T10:10:51.587 INFO:tasks.rados.rados.0.smithi195.stdout:313: set_chunk oid 14 offset: 262656 length: 1950720 target oid smithi19530714-14 low_tier offset: 262656 snap -1 2024-09-13T10:10:51.587 INFO:tasks.rados.rados.0.smithi195.stdout:314: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.587 INFO:tasks.rados.rados.0.smithi195.stdout:314: set_chunk oid 15 offset: 1161216 length: 215552 target oid smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1161216 snap -1 2024-09-13T10:10:51.587 INFO:tasks.rados.rados.0.smithi195.stdout:315: set_chunk oid 16 target oid 16 low_tier 2024-09-13T10:10:51.587 INFO:tasks.rados.rados.0.smithi195.stdout:315: set_chunk oid 16 offset: 1256448 length: 564736 target oid smithi19530714-16 low_tier offset: 1256448 snap -1 2024-09-13T10:10:51.587 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.587 INFO:tasks.rados.rados.0.smithi195.stdout:308: finishing set_chunk to oid 9 2024-09-13T10:10:51.588 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 9 set_chunk 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 90112 length: 1654272 2024-09-13T10:10:51.588 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 0 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-13T10:10:51.588 INFO:tasks.rados.rados.0.smithi195.stdout:308: done (15 left) 2024-09-13T10:10:51.588 INFO:tasks.rados.rados.0.smithi195.stdout:316: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.588 INFO:tasks.rados.rados.0.smithi195.stdout:316: set_chunk oid 17 offset: 537088 length: 466432 target oid smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 537088 snap -1 2024-09-13T10:10:51.588 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.588 INFO:tasks.rados.rados.0.smithi195.stdout:304: finishing set_chunk to oid 5 2024-09-13T10:10:51.588 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 5 set_chunk 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 759296 length: 586752 2024-09-13T10:10:51.588 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 0 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-13T10:10:51.588 INFO:tasks.rados.rados.0.smithi195.stdout:310: finishing set_chunk to oid 11 2024-09-13T10:10:51.588 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 11 set_chunk 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1051136 length: 719360 2024-09-13T10:10:51.588 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 0 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-13T10:10:51.589 INFO:tasks.rados.rados.0.smithi195.stdout:315: finishing set_chunk to oid 16 2024-09-13T10:10:51.589 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 1256448 length: 564736 2024-09-13T10:10:51.589 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-13T10:10:51.589 INFO:tasks.rados.rados.0.smithi195.stdout:312: finishing set_chunk to oid 13 2024-09-13T10:10:51.589 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 415744 length: 946176 2024-09-13T10:10:51.589 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-13T10:10:51.589 INFO:tasks.rados.rados.0.smithi195.stdout:302: finishing set_chunk to oid 3 2024-09-13T10:10:51.589 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 3 set_chunk 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 449536 length: 1348608 2024-09-13T10:10:51.589 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 0 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-13T10:10:51.589 INFO:tasks.rados.rados.0.smithi195.stdout:311: finishing set_chunk to oid 12 2024-09-13T10:10:51.589 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 12 set_chunk 12 low_tier returned error code -95 offset: 793088 length: 582144 2024-09-13T10:10:51.589 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-13T10:10:51.590 INFO:tasks.rados.rados.0.smithi195.stdout:316: finishing set_chunk to oid 17 2024-09-13T10:10:51.590 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 537088 length: 466432 2024-09-13T10:10:51.590 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-13T10:10:51.590 INFO:tasks.rados.rados.0.smithi195.stdout:302: done (15 left) 2024-09-13T10:10:51.590 INFO:tasks.rados.rados.0.smithi195.stdout:304: done (14 left) 2024-09-13T10:10:51.590 INFO:tasks.rados.rados.0.smithi195.stdout:310: done (13 left) 2024-09-13T10:10:51.590 INFO:tasks.rados.rados.0.smithi195.stdout:311: done (12 left) 2024-09-13T10:10:51.590 INFO:tasks.rados.rados.0.smithi195.stdout:312: done (11 left) 2024-09-13T10:10:51.590 INFO:tasks.rados.rados.0.smithi195.stdout:315: done (10 left) 2024-09-13T10:10:51.590 INFO:tasks.rados.rados.0.smithi195.stdout:316: done (9 left) 2024-09-13T10:10:51.590 INFO:tasks.rados.rados.0.smithi195.stdout:317: set_chunk oid 18 target oid 18 low_tier 2024-09-13T10:10:51.590 INFO:tasks.rados.rados.0.smithi195.stdout:317: set_chunk oid 18 offset: 395264 length: 856064 target oid smithi19530714-18 low_tier offset: 395264 snap -1 2024-09-13T10:10:51.590 INFO:tasks.rados.rados.0.smithi195.stdout:318: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.590 INFO:tasks.rados.rados.0.smithi195.stdout:318: set_chunk oid 19 offset: 1415680 length: 272384 target oid smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1415680 snap -1 2024-09-13T10:10:51.591 INFO:tasks.rados.rados.0.smithi195.stdout:319: set_chunk oid 20 target oid 20 low_tier 2024-09-13T10:10:51.591 INFO:tasks.rados.rados.0.smithi195.stdout:319: set_chunk oid 20 offset: 326656 length: 1084416 target oid smithi19530714-20 low_tier offset: 326656 snap -1 2024-09-13T10:10:51.591 INFO:tasks.rados.rados.0.smithi195.stdout:320: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.591 INFO:tasks.rados.rados.0.smithi195.stdout:320: set_chunk oid 21 offset: 328704 length: 933376 target oid smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 328704 snap -1 2024-09-13T10:10:51.591 INFO:tasks.rados.rados.0.smithi195.stdout:321: set_chunk oid 22 target oid 22 low_tier 2024-09-13T10:10:51.591 INFO:tasks.rados.rados.0.smithi195.stdout:321: set_chunk oid 22 offset: 116736 length: 1747968 target oid smithi19530714-22 low_tier offset: 116736 snap -1 2024-09-13T10:10:51.591 INFO:tasks.rados.rados.0.smithi195.stdout:322: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.591 INFO:tasks.rados.rados.0.smithi195.stdout:322: set_chunk oid 23 offset: 981504 length: 1097216 target oid smithi19530714-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 981504 snap -1 2024-09-13T10:10:51.591 INFO:tasks.rados.rados.0.smithi195.stdout:323: set_chunk oid 24 target oid 24 low_tier 2024-09-13T10:10:51.591 INFO:tasks.rados.rados.0.smithi195.stdout:323: set_chunk oid 24 offset: 1673216 length: 238592 target oid smithi19530714-24 low_tier offset: 1673216 snap -1 2024-09-13T10:10:51.591 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.591 INFO:tasks.rados.rados.0.smithi195.stdout:313: finishing set_chunk to oid 14 2024-09-13T10:10:51.591 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 14 set_chunk 14 low_tier returned error code -95 offset: 262656 length: 1950720 2024-09-13T10:10:51.591 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 0 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-13T10:10:51.592 INFO:tasks.rados.rados.0.smithi195.stdout:314: finishing set_chunk to oid 15 2024-09-13T10:10:51.592 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 15 set_chunk 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1161216 length: 215552 2024-09-13T10:10:51.592 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-13T10:10:51.592 INFO:tasks.rados.rados.0.smithi195.stdout:313: done (15 left) 2024-09-13T10:10:51.592 INFO:tasks.rados.rados.0.smithi195.stdout:314: done (14 left) 2024-09-13T10:10:51.592 INFO:tasks.rados.rados.0.smithi195.stdout:324: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.592 INFO:tasks.rados.rados.0.smithi195.stdout:324: set_chunk oid 25 offset: 166912 length: 824832 target oid smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 166912 snap -1 2024-09-13T10:10:51.592 INFO:tasks.rados.rados.0.smithi195.stdout:325: set_chunk oid 26 target oid 26 low_tier 2024-09-13T10:10:51.592 INFO:tasks.rados.rados.0.smithi195.stdout:325: set_chunk oid 26 offset: 436224 length: 199680 target oid smithi19530714-26 low_tier offset: 436224 snap -1 2024-09-13T10:10:51.592 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.592 INFO:tasks.rados.rados.0.smithi195.stdout:317: finishing set_chunk to oid 18 2024-09-13T10:10:51.592 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 395264 length: 856064 2024-09-13T10:10:51.592 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-13T10:10:51.593 INFO:tasks.rados.rados.0.smithi195.stdout:317: done (15 left) 2024-09-13T10:10:51.593 INFO:tasks.rados.rados.0.smithi195.stdout:326: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.593 INFO:tasks.rados.rados.0.smithi195.stdout:326: set_chunk oid 27 offset: 2130944 length: 72704 target oid smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 2130944 snap -1 2024-09-13T10:10:51.593 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.593 INFO:tasks.rados.rados.0.smithi195.stdout:319: finishing set_chunk to oid 20 2024-09-13T10:10:51.593 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 326656 length: 1084416 2024-09-13T10:10:51.593 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-13T10:10:51.593 INFO:tasks.rados.rados.0.smithi195.stdout:319: done (15 left) 2024-09-13T10:10:51.593 INFO:tasks.rados.rados.0.smithi195.stdout:327: set_chunk oid 28 target oid 28 low_tier 2024-09-13T10:10:51.593 INFO:tasks.rados.rados.0.smithi195.stdout:327: set_chunk oid 28 offset: 1059840 length: 6144 target oid smithi19530714-28 low_tier offset: 1059840 snap -1 2024-09-13T10:10:51.593 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.593 INFO:tasks.rados.rados.0.smithi195.stdout:322: finishing set_chunk to oid 23 2024-09-13T10:10:51.593 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 23 set_chunk 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 981504 length: 1097216 2024-09-13T10:10:51.593 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-13T10:10:51.594 INFO:tasks.rados.rados.0.smithi195.stdout:323: finishing set_chunk to oid 24 2024-09-13T10:10:51.594 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 24 set_chunk 24 low_tier returned error code -95 offset: 1673216 length: 238592 2024-09-13T10:10:51.594 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 0 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-13T10:10:51.594 INFO:tasks.rados.rados.0.smithi195.stdout:320: finishing set_chunk to oid 21 2024-09-13T10:10:51.594 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 21 set_chunk 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 328704 length: 933376 2024-09-13T10:10:51.594 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 0 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-13T10:10:51.594 INFO:tasks.rados.rados.0.smithi195.stdout:288: finishing set_chunk to oid 39 2024-09-13T10:10:51.594 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 9 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-13T10:10:51.594 INFO:tasks.rados.rados.0.smithi195.stdout:288: done (15 left) 2024-09-13T10:10:51.594 INFO:tasks.rados.rados.0.smithi195.stdout:320: done (14 left) 2024-09-13T10:10:51.594 INFO:tasks.rados.rados.0.smithi195.stdout:322: done (13 left) 2024-09-13T10:10:51.594 INFO:tasks.rados.rados.0.smithi195.stdout:323: done (12 left) 2024-09-13T10:10:51.594 INFO:tasks.rados.rados.0.smithi195.stdout:328: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.594 INFO:tasks.rados.rados.0.smithi195.stdout:328: set_chunk oid 29 offset: 929792 length: 46080 target oid smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 929792 snap -1 2024-09-13T10:10:51.595 INFO:tasks.rados.rados.0.smithi195.stdout:329: set_chunk oid 30 target oid 30 low_tier 2024-09-13T10:10:51.595 INFO:tasks.rados.rados.0.smithi195.stdout:329: set_chunk oid 30 offset: 1737216 length: 171008 target oid smithi19530714-30 low_tier offset: 1737216 snap -1 2024-09-13T10:10:51.595 INFO:tasks.rados.rados.0.smithi195.stdout:330: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.595 INFO:tasks.rados.rados.0.smithi195.stdout:330: set_chunk oid 31 offset: 1016320 length: 1422848 target oid smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1016320 snap -1 2024-09-13T10:10:51.595 INFO:tasks.rados.rados.0.smithi195.stdout:331: set_chunk oid 32 target oid 32 low_tier 2024-09-13T10:10:51.595 INFO:tasks.rados.rados.0.smithi195.stdout:331: set_chunk oid 32 offset: 1113088 length: 754176 target oid smithi19530714-32 low_tier offset: 1113088 snap -1 2024-09-13T10:10:51.595 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.595 INFO:tasks.rados.rados.0.smithi195.stdout:321: finishing set_chunk to oid 22 2024-09-13T10:10:51.595 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 116736 length: 1747968 2024-09-13T10:10:51.595 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-13T10:10:51.595 INFO:tasks.rados.rados.0.smithi195.stdout:321: done (15 left) 2024-09-13T10:10:51.595 INFO:tasks.rados.rados.0.smithi195.stdout:332: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.595 INFO:tasks.rados.rados.0.smithi195.stdout:332: set_chunk oid 33 offset: 1170944 length: 110592 target oid smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1170944 snap -1 2024-09-13T10:10:51.596 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.596 INFO:tasks.rados.rados.0.smithi195.stdout:324: finishing set_chunk to oid 25 2024-09-13T10:10:51.596 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 25 set_chunk 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 166912 length: 824832 2024-09-13T10:10:51.596 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-13T10:10:51.596 INFO:tasks.rados.rados.0.smithi195.stdout:326: finishing set_chunk to oid 27 2024-09-13T10:10:51.596 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 27 set_chunk 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 2130944 length: 72704 2024-09-13T10:10:51.596 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-13T10:10:51.596 INFO:tasks.rados.rados.0.smithi195.stdout:324: done (15 left) 2024-09-13T10:10:51.596 INFO:tasks.rados.rados.0.smithi195.stdout:326: done (14 left) 2024-09-13T10:10:51.596 INFO:tasks.rados.rados.0.smithi195.stdout:332 retry set_chunk ! 2024-09-13T10:10:51.596 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 14 2024-09-13T10:10:51.596 INFO:tasks.rados.rados.0.smithi195.stdout:328: finishing set_chunk to oid 29 2024-09-13T10:10:51.597 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 29 set_chunk 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 929792 length: 46080 2024-09-13T10:10:51.597 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 0 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-13T10:10:51.597 INFO:tasks.rados.rados.0.smithi195.stdout:328: done (13 left) 2024-09-13T10:10:51.597 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 13 2024-09-13T10:10:51.597 INFO:tasks.rados.rados.0.smithi195.stdout:330: finishing set_chunk to oid 31 2024-09-13T10:10:51.597 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 31 set_chunk 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1016320 length: 1422848 2024-09-13T10:10:51.597 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-13T10:10:51.597 INFO:tasks.rados.rados.0.smithi195.stdout:327: finishing set_chunk to oid 28 2024-09-13T10:10:51.597 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 28 set_chunk 28 low_tier returned error code -95 offset: 1059840 length: 6144 2024-09-13T10:10:51.597 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 0 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-13T10:10:51.597 INFO:tasks.rados.rados.0.smithi195.stdout:327: done (12 left) 2024-09-13T10:10:51.597 INFO:tasks.rados.rados.0.smithi195.stdout:330: done (11 left) 2024-09-13T10:10:51.598 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 11 2024-09-13T10:10:51.598 INFO:tasks.rados.rados.0.smithi195.stdout:331: finishing set_chunk to oid 32 2024-09-13T10:10:51.598 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 1113088 length: 754176 2024-09-13T10:10:51.598 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-13T10:10:51.598 INFO:tasks.rados.rados.0.smithi195.stdout:331: done (10 left) 2024-09-13T10:10:51.598 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 10 2024-09-13T10:10:51.598 INFO:tasks.rados.rados.0.smithi195.stdout:283: finishing set_chunk to oid 34 2024-09-13T10:10:51.598 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 34 v 6 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-13T10:10:51.598 INFO:tasks.rados.rados.0.smithi195.stdout:283: done (9 left) 2024-09-13T10:10:51.598 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 9 2024-09-13T10:10:51.598 INFO:tasks.rados.rados.0.smithi195.stdout:309: finishing set_chunk to oid 10 2024-09-13T10:10:51.598 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 3 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-13T10:10:51.598 INFO:tasks.rados.rados.0.smithi195.stdout:290: finishing set_chunk to oid 41 2024-09-13T10:10:51.599 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 3 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-13T10:10:51.599 INFO:tasks.rados.rados.0.smithi195.stdout:290: done (8 left) 2024-09-13T10:10:51.599 INFO:tasks.rados.rados.0.smithi195.stdout:309: done (7 left) 2024-09-13T10:10:51.599 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 7 2024-09-13T10:10:51.599 INFO:tasks.rados.rados.0.smithi195.stdout:293: finishing set_chunk to oid 44 2024-09-13T10:10:51.599 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 9 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-13T10:10:51.599 INFO:tasks.rados.rados.0.smithi195.stdout:293: done (6 left) 2024-09-13T10:10:51.599 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 6 2024-09-13T10:10:51.599 INFO:tasks.rados.rados.0.smithi195.stdout:300: finishing set_chunk to oid 1 2024-09-13T10:10:51.599 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-13T10:10:51.599 INFO:tasks.rados.rados.0.smithi195.stdout:300: done (5 left) 2024-09-13T10:10:51.599 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 5 2024-09-13T10:10:51.600 INFO:tasks.rados.rados.0.smithi195.stdout:305: finishing set_chunk to oid 6 2024-09-13T10:10:51.600 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 6 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-13T10:10:51.600 INFO:tasks.rados.rados.0.smithi195.stdout:325: finishing set_chunk to oid 26 2024-09-13T10:10:51.600 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 3 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-13T10:10:51.600 INFO:tasks.rados.rados.0.smithi195.stdout:305: done (4 left) 2024-09-13T10:10:51.600 INFO:tasks.rados.rados.0.smithi195.stdout:325: done (3 left) 2024-09-13T10:10:51.600 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 3 2024-09-13T10:10:51.600 INFO:tasks.rados.rados.0.smithi195.stdout:329: finishing set_chunk to oid 30 2024-09-13T10:10:51.600 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 18 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-13T10:10:51.600 INFO:tasks.rados.rados.0.smithi195.stdout:329: done (2 left) 2024-09-13T10:10:51.600 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 2 2024-09-13T10:10:51.600 INFO:tasks.rados.rados.0.smithi195.stdout:332: finishing set_chunk to oid 33 2024-09-13T10:10:51.601 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 6 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-13T10:10:51.601 INFO:tasks.rados.rados.0.smithi195.stdout:318: finishing set_chunk to oid 19 2024-09-13T10:10:51.601 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 9 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-13T10:10:51.601 INFO:tasks.rados.rados.0.smithi195.stdout:318: done (1 left) 2024-09-13T10:10:51.601 INFO:tasks.rados.rados.0.smithi195.stdout:332: done (0 left) 2024-09-13T10:10:51.601 INFO:tasks.rados.rados.0.smithi195.stdout:333: set_chunk oid 34 target oid 34 low_tier 2024-09-13T10:10:51.601 INFO:tasks.rados.rados.0.smithi195.stdout:333: set_chunk oid 34 offset: 1948672 length: 156160 target oid smithi19530714-34 low_tier offset: 1948672 snap -1 2024-09-13T10:10:51.601 INFO:tasks.rados.rados.0.smithi195.stdout:334: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.601 INFO:tasks.rados.rados.0.smithi195.stdout:334: set_chunk oid 35 offset: 320000 length: 1256448 target oid smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 320000 snap -1 2024-09-13T10:10:51.601 INFO:tasks.rados.rados.0.smithi195.stdout:335: set_chunk oid 36 target oid 36 low_tier 2024-09-13T10:10:51.601 INFO:tasks.rados.rados.0.smithi195.stdout:335: set_chunk oid 36 offset: 752640 length: 1161728 target oid smithi19530714-36 low_tier offset: 752640 snap -1 2024-09-13T10:10:51.601 INFO:tasks.rados.rados.0.smithi195.stdout:336: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.601 INFO:tasks.rados.rados.0.smithi195.stdout:336: set_chunk oid 37 offset: 81408 length: 2231808 target oid smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 81408 snap -1 2024-09-13T10:10:51.602 INFO:tasks.rados.rados.0.smithi195.stdout:337: set_chunk oid 38 target oid 38 low_tier 2024-09-13T10:10:51.602 INFO:tasks.rados.rados.0.smithi195.stdout:337: set_chunk oid 38 offset: 1363968 length: 991232 target oid smithi19530714-38 low_tier offset: 1363968 snap -1 2024-09-13T10:10:51.602 INFO:tasks.rados.rados.0.smithi195.stdout:338: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.602 INFO:tasks.rados.rados.0.smithi195.stdout:338: set_chunk oid 39 offset: 1147392 length: 418816 target oid smithi19530714-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1147392 snap -1 2024-09-13T10:10:51.602 INFO:tasks.rados.rados.0.smithi195.stdout:339: set_chunk oid 40 target oid 40 low_tier 2024-09-13T10:10:51.602 INFO:tasks.rados.rados.0.smithi195.stdout:339: set_chunk oid 40 offset: 539648 length: 65536 target oid smithi19530714-40 low_tier offset: 539648 snap -1 2024-09-13T10:10:51.602 INFO:tasks.rados.rados.0.smithi195.stdout:340: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.602 INFO:tasks.rados.rados.0.smithi195.stdout:340: set_chunk oid 41 offset: 160256 length: 1954304 target oid smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 160256 snap -1 2024-09-13T10:10:51.602 INFO:tasks.rados.rados.0.smithi195.stdout:341: set_chunk oid 42 target oid 42 low_tier 2024-09-13T10:10:51.602 INFO:tasks.rados.rados.0.smithi195.stdout:341: set_chunk oid 42 offset: 574976 length: 866304 target oid smithi19530714-42 low_tier offset: 574976 snap -1 2024-09-13T10:10:51.602 INFO:tasks.rados.rados.0.smithi195.stdout:342: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.602 INFO:tasks.rados.rados.0.smithi195.stdout:342: set_chunk oid 43 offset: 730112 length: 229888 target oid smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 730112 snap -1 2024-09-13T10:10:51.602 INFO:tasks.rados.rados.0.smithi195.stdout:343: set_chunk oid 44 target oid 44 low_tier 2024-09-13T10:10:51.603 INFO:tasks.rados.rados.0.smithi195.stdout:343: set_chunk oid 44 offset: 1224704 length: 57344 target oid smithi19530714-44 low_tier offset: 1224704 snap -1 2024-09-13T10:10:51.603 INFO:tasks.rados.rados.0.smithi195.stdout:344: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.603 INFO:tasks.rados.rados.0.smithi195.stdout:344: set_chunk oid 45 offset: 1145856 length: 1406976 target oid smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1145856 snap -1 2024-09-13T10:10:51.603 INFO:tasks.rados.rados.0.smithi195.stdout:345: set_chunk oid 46 target oid 46 low_tier 2024-09-13T10:10:51.603 INFO:tasks.rados.rados.0.smithi195.stdout:345: set_chunk oid 46 offset: 1072128 length: 1648128 target oid smithi19530714-46 low_tier offset: 1072128 snap -1 2024-09-13T10:10:51.603 INFO:tasks.rados.rados.0.smithi195.stdout:346: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.603 INFO:tasks.rados.rados.0.smithi195.stdout:346: set_chunk oid 47 offset: 1131520 length: 663552 target oid smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1131520 snap -1 2024-09-13T10:10:51.603 INFO:tasks.rados.rados.0.smithi195.stdout:347: set_chunk oid 48 target oid 48 low_tier 2024-09-13T10:10:51.603 INFO:tasks.rados.rados.0.smithi195.stdout:347: set_chunk oid 48 offset: 369152 length: 236032 target oid smithi19530714-48 low_tier offset: 369152 snap -1 2024-09-13T10:10:51.603 INFO:tasks.rados.rados.0.smithi195.stdout:348: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.603 INFO:tasks.rados.rados.0.smithi195.stdout:348: set_chunk oid 49 offset: 292864 length: 1489408 target oid smithi19530714-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 292864 snap -1 2024-09-13T10:10:51.603 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.604 INFO:tasks.rados.rados.0.smithi195.stdout:336: finishing set_chunk to oid 37 2024-09-13T10:10:51.604 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 37 set_chunk 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 81408 length: 2231808 2024-09-13T10:10:51.604 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 0 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-13T10:10:51.604 INFO:tasks.rados.rados.0.smithi195.stdout:340: finishing set_chunk to oid 41 2024-09-13T10:10:51.604 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 41 set_chunk 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 160256 length: 1954304 2024-09-13T10:10:51.604 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-13T10:10:51.604 INFO:tasks.rados.rados.0.smithi195.stdout:342: finishing set_chunk to oid 43 2024-09-13T10:10:51.604 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 730112 length: 229888 2024-09-13T10:10:51.604 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-13T10:10:51.604 INFO:tasks.rados.rados.0.smithi195.stdout:336: done (15 left) 2024-09-13T10:10:51.604 INFO:tasks.rados.rados.0.smithi195.stdout:340: done (14 left) 2024-09-13T10:10:51.605 INFO:tasks.rados.rados.0.smithi195.stdout:342: done (13 left) 2024-09-13T10:10:51.605 INFO:tasks.rados.rados.0.smithi195.stdout:349: set_chunk oid 50 target oid 50 low_tier 2024-09-13T10:10:51.605 INFO:tasks.rados.rados.0.smithi195.stdout:349: set_chunk oid 50 offset: 1096192 length: 381952 target oid smithi19530714-50 low_tier offset: 1096192 snap -1 2024-09-13T10:10:51.605 INFO:tasks.rados.rados.0.smithi195.stdout:350: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.605 INFO:tasks.rados.rados.0.smithi195.stdout:350: set_chunk oid 1 offset: 783872 length: 15872 target oid smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 783872 snap -1 2024-09-13T10:10:51.605 INFO:tasks.rados.rados.0.smithi195.stdout:351: set_chunk oid 2 target oid 2 low_tier 2024-09-13T10:10:51.605 INFO:tasks.rados.rados.0.smithi195.stdout:351: set_chunk oid 2 offset: 1147904 length: 200704 target oid smithi19530714-2 low_tier offset: 1147904 snap -1 2024-09-13T10:10:51.605 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.605 INFO:tasks.rados.rados.0.smithi195.stdout:333: finishing set_chunk to oid 34 2024-09-13T10:10:51.605 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 34 set_chunk 34 low_tier returned error code -95 offset: 1948672 length: 156160 2024-09-13T10:10:51.605 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 34 v 0 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-13T10:10:51.605 INFO:tasks.rados.rados.0.smithi195.stdout:334: finishing set_chunk to oid 35 2024-09-13T10:10:51.605 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 35 set_chunk 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 320000 length: 1256448 2024-09-13T10:10:51.606 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 0 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-13T10:10:51.606 INFO:tasks.rados.rados.0.smithi195.stdout:338: finishing set_chunk to oid 39 2024-09-13T10:10:51.606 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1147392 length: 418816 2024-09-13T10:10:51.606 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-13T10:10:51.606 INFO:tasks.rados.rados.0.smithi195.stdout:341: finishing set_chunk to oid 42 2024-09-13T10:10:51.606 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 42 set_chunk 42 low_tier returned error code -95 offset: 574976 length: 866304 2024-09-13T10:10:51.606 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-13T10:10:51.606 INFO:tasks.rados.rados.0.smithi195.stdout:345: finishing set_chunk to oid 46 2024-09-13T10:10:51.606 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 46 set_chunk 46 low_tier returned error code -95 offset: 1072128 length: 1648128 2024-09-13T10:10:51.606 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 0 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-13T10:10:51.606 INFO:tasks.rados.rados.0.smithi195.stdout:335: finishing set_chunk to oid 36 2024-09-13T10:10:51.606 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 36 set_chunk 36 low_tier returned error code -95 offset: 752640 length: 1161728 2024-09-13T10:10:51.607 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 0 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-13T10:10:51.607 INFO:tasks.rados.rados.0.smithi195.stdout:337: finishing set_chunk to oid 38 2024-09-13T10:10:51.607 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 38 set_chunk 38 low_tier returned error code -95 offset: 1363968 length: 991232 2024-09-13T10:10:51.607 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 0 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-13T10:10:51.607 INFO:tasks.rados.rados.0.smithi195.stdout:343: finishing set_chunk to oid 44 2024-09-13T10:10:51.607 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 44 set_chunk 44 low_tier returned error code -95 offset: 1224704 length: 57344 2024-09-13T10:10:51.607 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 0 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-13T10:10:51.607 INFO:tasks.rados.rados.0.smithi195.stdout:344: finishing set_chunk to oid 45 2024-09-13T10:10:51.607 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 45 set_chunk 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1145856 length: 1406976 2024-09-13T10:10:51.607 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 0 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-13T10:10:51.607 INFO:tasks.rados.rados.0.smithi195.stdout:346: finishing set_chunk to oid 47 2024-09-13T10:10:51.607 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 47 set_chunk 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1131520 length: 663552 2024-09-13T10:10:51.608 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-13T10:10:51.608 INFO:tasks.rados.rados.0.smithi195.stdout:333: done (15 left) 2024-09-13T10:10:51.608 INFO:tasks.rados.rados.0.smithi195.stdout:334: done (14 left) 2024-09-13T10:10:51.608 INFO:tasks.rados.rados.0.smithi195.stdout:335: done (13 left) 2024-09-13T10:10:51.608 INFO:tasks.rados.rados.0.smithi195.stdout:337: done (12 left) 2024-09-13T10:10:51.608 INFO:tasks.rados.rados.0.smithi195.stdout:338: done (11 left) 2024-09-13T10:10:51.608 INFO:tasks.rados.rados.0.smithi195.stdout:341: done (10 left) 2024-09-13T10:10:51.608 INFO:tasks.rados.rados.0.smithi195.stdout:343: done (9 left) 2024-09-13T10:10:51.608 INFO:tasks.rados.rados.0.smithi195.stdout:344: done (8 left) 2024-09-13T10:10:51.608 INFO:tasks.rados.rados.0.smithi195.stdout:345: done (7 left) 2024-09-13T10:10:51.608 INFO:tasks.rados.rados.0.smithi195.stdout:346: done (6 left) 2024-09-13T10:10:51.608 INFO:tasks.rados.rados.0.smithi195.stdout:352: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.609 INFO:tasks.rados.rados.0.smithi195.stdout:352: set_chunk oid 3 offset: 78848 length: 1620992 target oid smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 78848 snap -1 2024-09-13T10:10:51.609 INFO:tasks.rados.rados.0.smithi195.stdout:353: set_chunk oid 4 target oid 4 low_tier 2024-09-13T10:10:51.609 INFO:tasks.rados.rados.0.smithi195.stdout:353: set_chunk oid 4 offset: 752640 length: 379392 target oid smithi19530714-4 low_tier offset: 752640 snap -1 2024-09-13T10:10:51.609 INFO:tasks.rados.rados.0.smithi195.stdout:354: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.609 INFO:tasks.rados.rados.0.smithi195.stdout:348: finishing set_chunk to oid 49 2024-09-13T10:10:51.609 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 49 set_chunk 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 292864 length: 1489408 2024-09-13T10:10:51.609 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 0 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-13T10:10:51.609 INFO:tasks.rados.rados.0.smithi195.stdout:351: finishing set_chunk to oid 2 2024-09-13T10:10:51.609 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 2 set_chunk 2 low_tier returned error code -95 offset: 1147904 length: 200704 2024-09-13T10:10:51.609 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-13T10:10:51.609 INFO:tasks.rados.rados.0.smithi195.stdout:349: finishing set_chunk to oid 50 2024-09-13T10:10:51.609 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 50 set_chunk 50 low_tier returned error code -95 offset: 1096192 length: 381952 2024-09-13T10:10:51.609 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 0 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-13T10:10:51.609 INFO:tasks.rados.rados.0.smithi195.stdout:353: finishing set_chunk to oid 4 2024-09-13T10:10:51.610 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 4 set_chunk 4 low_tier returned error code -95 offset: 752640 length: 379392 2024-09-13T10:10:51.610 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 0 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-13T10:10:51.610 INFO:tasks.rados.rados.0.smithi195.stdout:352: finishing set_chunk to oid 3 2024-09-13T10:10:51.610 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 3 set_chunk 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 78848 length: 1620992 2024-09-13T10:10:51.610 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 0 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-13T10:10:51.610 INFO:tasks.rados.rados.0.smithi195.stdout:354: set_chunk oid 5 offset: 476672 length: 883712 target oid smithi19530714-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 476672 snap -1 2024-09-13T10:10:51.610 INFO:tasks.rados.rados.0.smithi195.stdout:348: done (8 left) 2024-09-13T10:10:51.610 INFO:tasks.rados.rados.0.smithi195.stdout:349: done (7 left) 2024-09-13T10:10:51.610 INFO:tasks.rados.rados.0.smithi195.stdout:351: done (6 left) 2024-09-13T10:10:51.610 INFO:tasks.rados.rados.0.smithi195.stdout:352: done (5 left) 2024-09-13T10:10:51.610 INFO:tasks.rados.rados.0.smithi195.stdout:353: done (4 left) 2024-09-13T10:10:51.610 INFO:tasks.rados.rados.0.smithi195.stdout:355: set_chunk oid 6 target oid 6 low_tier 2024-09-13T10:10:51.611 INFO:tasks.rados.rados.0.smithi195.stdout:355: set_chunk oid 6 offset: 1366528 length: 351744 target oid smithi19530714-6 low_tier offset: 1366528 snap -1 2024-09-13T10:10:51.611 INFO:tasks.rados.rados.0.smithi195.stdout:356: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.611 INFO:tasks.rados.rados.0.smithi195.stdout:356: set_chunk oid 7 offset: 196608 length: 1504256 target oid smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 196608 snap -1 2024-09-13T10:10:51.611 INFO:tasks.rados.rados.0.smithi195.stdout:357: set_chunk oid 8 target oid 8 low_tier 2024-09-13T10:10:51.611 INFO:tasks.rados.rados.0.smithi195.stdout:357: set_chunk oid 8 offset: 771072 length: 1116672 target oid smithi19530714-8 low_tier offset: 771072 snap -1 2024-09-13T10:10:51.611 INFO:tasks.rados.rados.0.smithi195.stdout:358: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.611 INFO:tasks.rados.rados.0.smithi195.stdout:358: set_chunk oid 9 offset: 631808 length: 521216 target oid smithi19530714-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 631808 snap -1 2024-09-13T10:10:51.611 INFO:tasks.rados.rados.0.smithi195.stdout:359: set_chunk oid 10 target oid 10 low_tier 2024-09-13T10:10:51.611 INFO:tasks.rados.rados.0.smithi195.stdout:359: set_chunk oid 10 offset: 310784 length: 244224 target oid smithi19530714-10 low_tier offset: 310784 snap -1 2024-09-13T10:10:51.611 INFO:tasks.rados.rados.0.smithi195.stdout:360: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.611 INFO:tasks.rados.rados.0.smithi195.stdout:360: set_chunk oid 11 offset: 1373696 length: 196608 target oid smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1373696 snap -1 2024-09-13T10:10:51.611 INFO:tasks.rados.rados.0.smithi195.stdout:361: set_chunk oid 12 target oid 12 low_tier 2024-09-13T10:10:51.611 INFO:tasks.rados.rados.0.smithi195.stdout:361: set_chunk oid 12 offset: 625664 length: 1376256 target oid smithi19530714-12 low_tier offset: 625664 snap -1 2024-09-13T10:10:51.611 INFO:tasks.rados.rados.0.smithi195.stdout:362: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.612 INFO:tasks.rados.rados.0.smithi195.stdout:362: set_chunk oid 13 offset: 148480 length: 1853440 target oid smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 148480 snap -1 2024-09-13T10:10:51.612 INFO:tasks.rados.rados.0.smithi195.stdout:363: set_chunk oid 14 target oid 14 low_tier 2024-09-13T10:10:51.612 INFO:tasks.rados.rados.0.smithi195.stdout:363: set_chunk oid 14 offset: 439296 length: 620544 target oid smithi19530714-14 low_tier offset: 439296 snap -1 2024-09-13T10:10:51.612 INFO:tasks.rados.rados.0.smithi195.stdout:364: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.612 INFO:tasks.rados.rados.0.smithi195.stdout:364: set_chunk oid 15 offset: 142848 length: 932864 target oid smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 142848 snap -1 2024-09-13T10:10:51.612 INFO:tasks.rados.rados.0.smithi195.stdout:356: finishing set_chunk to oid 7 2024-09-13T10:10:51.612 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 196608 length: 1504256 2024-09-13T10:10:51.612 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-13T10:10:51.612 INFO:tasks.rados.rados.0.smithi195.stdout:356: done (13 left) 2024-09-13T10:10:51.612 INFO:tasks.rados.rados.0.smithi195.stdout:365: set_chunk oid 16 target oid 16 low_tier 2024-09-13T10:10:51.612 INFO:tasks.rados.rados.0.smithi195.stdout:365: set_chunk oid 16 offset: 1026560 length: 1053184 target oid smithi19530714-16 low_tier offset: 1026560 snap -1 2024-09-13T10:10:51.612 INFO:tasks.rados.rados.0.smithi195.stdout:366: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.612 INFO:tasks.rados.rados.0.smithi195.stdout:366: set_chunk oid 17 offset: 359936 length: 1446912 target oid smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 359936 snap -1 2024-09-13T10:10:51.612 INFO:tasks.rados.rados.0.smithi195.stdout:367: set_chunk oid 18 target oid 18 low_tier 2024-09-13T10:10:51.613 INFO:tasks.rados.rados.0.smithi195.stdout:367: set_chunk oid 18 offset: 174592 length: 829440 target oid smithi19530714-18 low_tier offset: 174592 snap -1 2024-09-13T10:10:51.613 INFO:tasks.rados.rados.0.smithi195.stdout:357: finishing set_chunk to oid 8 2024-09-13T10:10:51.613 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 8 set_chunk 8 low_tier returned error code -95 offset: 771072 length: 1116672 2024-09-13T10:10:51.613 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-13T10:10:51.613 INFO:tasks.rados.rados.0.smithi195.stdout:357: done (15 left) 2024-09-13T10:10:51.613 INFO:tasks.rados.rados.0.smithi195.stdout:368: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.613 INFO:tasks.rados.rados.0.smithi195.stdout:368: set_chunk oid 19 offset: 346112 length: 109568 target oid smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 346112 snap -1 2024-09-13T10:10:51.613 INFO:tasks.rados.rados.0.smithi195.stdout:354: finishing set_chunk to oid 5 2024-09-13T10:10:51.613 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 5 set_chunk 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 476672 length: 883712 2024-09-13T10:10:51.613 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 0 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-13T10:10:51.613 INFO:tasks.rados.rados.0.smithi195.stdout:361: finishing set_chunk to oid 12 2024-09-13T10:10:51.613 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 12 set_chunk 12 low_tier returned error code -95 offset: 625664 length: 1376256 2024-09-13T10:10:51.613 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-13T10:10:51.614 INFO:tasks.rados.rados.0.smithi195.stdout:358: finishing set_chunk to oid 9 2024-09-13T10:10:51.614 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 9 set_chunk 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 631808 length: 521216 2024-09-13T10:10:51.614 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 0 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-13T10:10:51.614 INFO:tasks.rados.rados.0.smithi195.stdout:359: finishing set_chunk to oid 10 2024-09-13T10:10:51.614 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 10 set_chunk 10 low_tier returned error code -95 offset: 310784 length: 244224 2024-09-13T10:10:51.614 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 0 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-13T10:10:51.614 INFO:tasks.rados.rados.0.smithi195.stdout:360: finishing set_chunk to oid 11 2024-09-13T10:10:51.614 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 11 set_chunk 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1373696 length: 196608 2024-09-13T10:10:51.614 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 0 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-13T10:10:51.614 INFO:tasks.rados.rados.0.smithi195.stdout:354: done (15 left) 2024-09-13T10:10:51.614 INFO:tasks.rados.rados.0.smithi195.stdout:358: done (14 left) 2024-09-13T10:10:51.614 INFO:tasks.rados.rados.0.smithi195.stdout:359: done (13 left) 2024-09-13T10:10:51.614 INFO:tasks.rados.rados.0.smithi195.stdout:360: done (12 left) 2024-09-13T10:10:51.614 INFO:tasks.rados.rados.0.smithi195.stdout:361: done (11 left) 2024-09-13T10:10:51.614 INFO:tasks.rados.rados.0.smithi195.stdout:369: set_chunk oid 20 target oid 20 low_tier 2024-09-13T10:10:51.615 INFO:tasks.rados.rados.0.smithi195.stdout:369: set_chunk oid 20 offset: 922112 length: 323584 target oid smithi19530714-20 low_tier offset: 922112 snap -1 2024-09-13T10:10:51.615 INFO:tasks.rados.rados.0.smithi195.stdout:370: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.615 INFO:tasks.rados.rados.0.smithi195.stdout:370: set_chunk oid 21 offset: 418816 length: 1631232 target oid smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 418816 snap -1 2024-09-13T10:10:51.615 INFO:tasks.rados.rados.0.smithi195.stdout:371: set_chunk oid 22 target oid 22 low_tier 2024-09-13T10:10:51.615 INFO:tasks.rados.rados.0.smithi195.stdout:371: set_chunk oid 22 offset: 1287680 length: 156672 target oid smithi19530714-22 low_tier offset: 1287680 snap -1 2024-09-13T10:10:51.615 INFO:tasks.rados.rados.0.smithi195.stdout:372: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.615 INFO:tasks.rados.rados.0.smithi195.stdout:372: set_chunk oid 23 offset: 1387520 length: 551936 target oid smithi19530714-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1387520 snap -1 2024-09-13T10:10:51.615 INFO:tasks.rados.rados.0.smithi195.stdout:373: set_chunk oid 24 target oid 24 low_tier 2024-09-13T10:10:51.615 INFO:tasks.rados.rados.0.smithi195.stdout:373: set_chunk oid 24 offset: 1897472 length: 217088 target oid smithi19530714-24 low_tier offset: 1897472 snap -1 2024-09-13T10:10:51.615 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.615 INFO:tasks.rados.rados.0.smithi195.stdout:362: finishing set_chunk to oid 13 2024-09-13T10:10:51.615 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 148480 length: 1853440 2024-09-13T10:10:51.615 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-13T10:10:51.615 INFO:tasks.rados.rados.0.smithi195.stdout:363: finishing set_chunk to oid 14 2024-09-13T10:10:51.616 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 14 set_chunk 14 low_tier returned error code -95 offset: 439296 length: 620544 2024-09-13T10:10:51.616 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 0 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-13T10:10:51.616 INFO:tasks.rados.rados.0.smithi195.stdout:365: finishing set_chunk to oid 16 2024-09-13T10:10:51.616 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 1026560 length: 1053184 2024-09-13T10:10:51.616 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-13T10:10:51.616 INFO:tasks.rados.rados.0.smithi195.stdout:366: finishing set_chunk to oid 17 2024-09-13T10:10:51.616 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 359936 length: 1446912 2024-09-13T10:10:51.616 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-13T10:10:51.616 INFO:tasks.rados.rados.0.smithi195.stdout:364: finishing set_chunk to oid 15 2024-09-13T10:10:51.616 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 15 set_chunk 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 142848 length: 932864 2024-09-13T10:10:51.616 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-13T10:10:51.616 INFO:tasks.rados.rados.0.smithi195.stdout:367: finishing set_chunk to oid 18 2024-09-13T10:10:51.617 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 174592 length: 829440 2024-09-13T10:10:51.617 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-13T10:10:51.617 INFO:tasks.rados.rados.0.smithi195.stdout:362: done (15 left) 2024-09-13T10:10:51.617 INFO:tasks.rados.rados.0.smithi195.stdout:363: done (14 left) 2024-09-13T10:10:51.617 INFO:tasks.rados.rados.0.smithi195.stdout:364: done (13 left) 2024-09-13T10:10:51.617 INFO:tasks.rados.rados.0.smithi195.stdout:365: done (12 left) 2024-09-13T10:10:51.617 INFO:tasks.rados.rados.0.smithi195.stdout:366: done (11 left) 2024-09-13T10:10:51.617 INFO:tasks.rados.rados.0.smithi195.stdout:367: done (10 left) 2024-09-13T10:10:51.617 INFO:tasks.rados.rados.0.smithi195.stdout:374: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.617 INFO:tasks.rados.rados.0.smithi195.stdout:374: set_chunk oid 25 offset: 996352 length: 1362944 target oid smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 996352 snap -1 2024-09-13T10:10:51.617 INFO:tasks.rados.rados.0.smithi195.stdout:375: set_chunk oid 26 target oid 26 low_tier 2024-09-13T10:10:51.617 INFO:tasks.rados.rados.0.smithi195.stdout:375: set_chunk oid 26 offset: 656896 length: 1323520 target oid smithi19530714-26 low_tier offset: 656896 snap -1 2024-09-13T10:10:51.617 INFO:tasks.rados.rados.0.smithi195.stdout:376: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.618 INFO:tasks.rados.rados.0.smithi195.stdout:376: set_chunk oid 27 offset: 916480 length: 689664 target oid smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 916480 snap -1 2024-09-13T10:10:51.618 INFO:tasks.rados.rados.0.smithi195.stdout:377: set_chunk oid 28 target oid 28 low_tier 2024-09-13T10:10:51.618 INFO:tasks.rados.rados.0.smithi195.stdout:377: set_chunk oid 28 offset: 423424 length: 1176576 target oid smithi19530714-28 low_tier offset: 423424 snap -1 2024-09-13T10:10:51.618 INFO:tasks.rados.rados.0.smithi195.stdout:378: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.618 INFO:tasks.rados.rados.0.smithi195.stdout:378: set_chunk oid 29 offset: 945152 length: 1524224 target oid smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 945152 snap -1 2024-09-13T10:10:51.618 INFO:tasks.rados.rados.0.smithi195.stdout:379: set_chunk oid 30 target oid 30 low_tier 2024-09-13T10:10:51.618 INFO:tasks.rados.rados.0.smithi195.stdout:379: set_chunk oid 30 offset: 1100800 length: 253952 target oid smithi19530714-30 low_tier offset: 1100800 snap -1 2024-09-13T10:10:51.618 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.618 INFO:tasks.rados.rados.0.smithi195.stdout:369: finishing set_chunk to oid 20 2024-09-13T10:10:51.618 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 922112 length: 323584 2024-09-13T10:10:51.618 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-13T10:10:51.618 INFO:tasks.rados.rados.0.smithi195.stdout:370: finishing set_chunk to oid 21 2024-09-13T10:10:51.619 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 21 set_chunk 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 418816 length: 1631232 2024-09-13T10:10:51.619 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 0 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-13T10:10:51.619 INFO:tasks.rados.rados.0.smithi195.stdout:373: finishing set_chunk to oid 24 2024-09-13T10:10:51.619 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 24 set_chunk 24 low_tier returned error code -95 offset: 1897472 length: 217088 2024-09-13T10:10:51.619 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 0 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-13T10:10:51.619 INFO:tasks.rados.rados.0.smithi195.stdout:371: finishing set_chunk to oid 22 2024-09-13T10:10:51.619 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 1287680 length: 156672 2024-09-13T10:10:51.619 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-13T10:10:51.619 INFO:tasks.rados.rados.0.smithi195.stdout:369: done (15 left) 2024-09-13T10:10:51.619 INFO:tasks.rados.rados.0.smithi195.stdout:370: done (14 left) 2024-09-13T10:10:51.619 INFO:tasks.rados.rados.0.smithi195.stdout:371: done (13 left) 2024-09-13T10:10:51.619 INFO:tasks.rados.rados.0.smithi195.stdout:373: done (12 left) 2024-09-13T10:10:51.620 INFO:tasks.rados.rados.0.smithi195.stdout:380: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.620 INFO:tasks.rados.rados.0.smithi195.stdout:380: set_chunk oid 31 offset: 332288 length: 101376 target oid smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 332288 snap -1 2024-09-13T10:10:51.620 INFO:tasks.rados.rados.0.smithi195.stdout:381: set_chunk oid 32 target oid 32 low_tier 2024-09-13T10:10:51.620 INFO:tasks.rados.rados.0.smithi195.stdout:381: set_chunk oid 32 offset: 710144 length: 1349120 target oid smithi19530714-32 low_tier offset: 710144 snap -1 2024-09-13T10:10:51.620 INFO:tasks.rados.rados.0.smithi195.stdout:382: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.620 INFO:tasks.rados.rados.0.smithi195.stdout:382: set_chunk oid 33 offset: 450560 length: 364032 target oid smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 450560 snap -1 2024-09-13T10:10:51.620 INFO:tasks.rados.rados.0.smithi195.stdout:383: set_chunk oid 34 target oid 34 low_tier 2024-09-13T10:10:51.620 INFO:tasks.rados.rados.0.smithi195.stdout:383: set_chunk oid 34 offset: 184320 length: 2251776 target oid smithi19530714-34 low_tier offset: 184320 snap -1 2024-09-13T10:10:51.620 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.620 INFO:tasks.rados.rados.0.smithi195.stdout:372: finishing set_chunk to oid 23 2024-09-13T10:10:51.620 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 23 set_chunk 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1387520 length: 551936 2024-09-13T10:10:51.620 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-13T10:10:51.621 INFO:tasks.rados.rados.0.smithi195.stdout:375: finishing set_chunk to oid 26 2024-09-13T10:10:51.621 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 26 set_chunk 26 low_tier returned error code -95 offset: 656896 length: 1323520 2024-09-13T10:10:51.621 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 0 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-13T10:10:51.621 INFO:tasks.rados.rados.0.smithi195.stdout:374: finishing set_chunk to oid 25 2024-09-13T10:10:51.621 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 25 set_chunk 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 996352 length: 1362944 2024-09-13T10:10:51.621 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-13T10:10:51.621 INFO:tasks.rados.rados.0.smithi195.stdout:378: finishing set_chunk to oid 29 2024-09-13T10:10:51.621 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 29 set_chunk 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 945152 length: 1524224 2024-09-13T10:10:51.621 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 0 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-13T10:10:51.621 INFO:tasks.rados.rados.0.smithi195.stdout:376: finishing set_chunk to oid 27 2024-09-13T10:10:51.621 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 27 set_chunk 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 916480 length: 689664 2024-09-13T10:10:51.621 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-13T10:10:51.621 INFO:tasks.rados.rados.0.smithi195.stdout:372: done (15 left) 2024-09-13T10:10:51.622 INFO:tasks.rados.rados.0.smithi195.stdout:374: done (14 left) 2024-09-13T10:10:51.622 INFO:tasks.rados.rados.0.smithi195.stdout:375: done (13 left) 2024-09-13T10:10:51.622 INFO:tasks.rados.rados.0.smithi195.stdout:376: done (12 left) 2024-09-13T10:10:51.622 INFO:tasks.rados.rados.0.smithi195.stdout:378: done (11 left) 2024-09-13T10:10:51.622 INFO:tasks.rados.rados.0.smithi195.stdout:384: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.622 INFO:tasks.rados.rados.0.smithi195.stdout:384: set_chunk oid 35 offset: 91648 length: 1482752 target oid smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 91648 snap -1 2024-09-13T10:10:51.622 INFO:tasks.rados.rados.0.smithi195.stdout:385: set_chunk oid 36 target oid 36 low_tier 2024-09-13T10:10:51.622 INFO:tasks.rados.rados.0.smithi195.stdout:385: set_chunk oid 36 offset: 634368 length: 298496 target oid smithi19530714-36 low_tier offset: 634368 snap -1 2024-09-13T10:10:51.622 INFO:tasks.rados.rados.0.smithi195.stdout:386: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.622 INFO:tasks.rados.rados.0.smithi195.stdout:386: set_chunk oid 37 offset: 1205248 length: 570880 target oid smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1205248 snap -1 2024-09-13T10:10:51.622 INFO:tasks.rados.rados.0.smithi195.stdout:387: set_chunk oid 38 target oid 38 low_tier 2024-09-13T10:10:51.622 INFO:tasks.rados.rados.0.smithi195.stdout:387: set_chunk oid 38 offset: 1186304 length: 718848 target oid smithi19530714-38 low_tier offset: 1186304 snap -1 2024-09-13T10:10:51.623 INFO:tasks.rados.rados.0.smithi195.stdout:388: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.623 INFO:tasks.rados.rados.0.smithi195.stdout:388: set_chunk oid 39 offset: 222720 length: 1769472 target oid smithi19530714-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 222720 snap -1 2024-09-13T10:10:51.623 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.623 INFO:tasks.rados.rados.0.smithi195.stdout:381: finishing set_chunk to oid 32 2024-09-13T10:10:51.623 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 710144 length: 1349120 2024-09-13T10:10:51.623 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-13T10:10:51.623 INFO:tasks.rados.rados.0.smithi195.stdout:379: finishing set_chunk to oid 30 2024-09-13T10:10:51.623 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 30 set_chunk 30 low_tier returned error code -95 offset: 1100800 length: 253952 2024-09-13T10:10:51.623 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 0 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-13T10:10:51.623 INFO:tasks.rados.rados.0.smithi195.stdout:377: finishing set_chunk to oid 28 2024-09-13T10:10:51.623 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 28 set_chunk 28 low_tier returned error code -95 offset: 423424 length: 1176576 2024-09-13T10:10:51.623 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 0 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-13T10:10:51.624 INFO:tasks.rados.rados.0.smithi195.stdout:377: done (15 left) 2024-09-13T10:10:51.624 INFO:tasks.rados.rados.0.smithi195.stdout:379: done (14 left) 2024-09-13T10:10:51.624 INFO:tasks.rados.rados.0.smithi195.stdout:381: done (13 left) 2024-09-13T10:10:51.624 INFO:tasks.rados.rados.0.smithi195.stdout:388 retry set_chunk ! 2024-09-13T10:10:51.624 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 13 2024-09-13T10:10:51.624 INFO:tasks.rados.rados.0.smithi195.stdout:383: finishing set_chunk to oid 34 2024-09-13T10:10:51.624 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 34 set_chunk 34 low_tier returned error code -95 offset: 184320 length: 2251776 2024-09-13T10:10:51.624 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 34 v 0 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-13T10:10:51.624 INFO:tasks.rados.rados.0.smithi195.stdout:383: done (12 left) 2024-09-13T10:10:51.624 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 12 2024-09-13T10:10:51.624 INFO:tasks.rados.rados.0.smithi195.stdout:382: finishing set_chunk to oid 33 2024-09-13T10:10:51.624 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 33 set_chunk 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 450560 length: 364032 2024-09-13T10:10:51.624 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 0 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-13T10:10:51.625 INFO:tasks.rados.rados.0.smithi195.stdout:382: done (11 left) 2024-09-13T10:10:51.625 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 11 2024-09-13T10:10:51.625 INFO:tasks.rados.rados.0.smithi195.stdout:386: finishing set_chunk to oid 37 2024-09-13T10:10:51.625 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 37 set_chunk 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1205248 length: 570880 2024-09-13T10:10:51.625 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 0 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-13T10:10:51.625 INFO:tasks.rados.rados.0.smithi195.stdout:386: done (10 left) 2024-09-13T10:10:51.625 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 10 2024-09-13T10:10:51.625 INFO:tasks.rados.rados.0.smithi195.stdout:355: finishing set_chunk to oid 6 2024-09-13T10:10:51.625 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 6 set_chunk 6 low_tier returned error code -95 offset: 1366528 length: 351744 2024-09-13T10:10:51.625 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 0 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-13T10:10:51.625 INFO:tasks.rados.rados.0.smithi195.stdout:387: finishing set_chunk to oid 38 2024-09-13T10:10:51.625 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 38 set_chunk 38 low_tier returned error code -95 offset: 1186304 length: 718848 2024-09-13T10:10:51.626 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 0 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-13T10:10:51.626 INFO:tasks.rados.rados.0.smithi195.stdout:388: finishing set_chunk to oid 39 2024-09-13T10:10:51.626 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 222720 length: 1769472 2024-09-13T10:10:51.626 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-13T10:10:51.626 INFO:tasks.rados.rados.0.smithi195.stdout:355: done (9 left) 2024-09-13T10:10:51.626 INFO:tasks.rados.rados.0.smithi195.stdout:387: done (8 left) 2024-09-13T10:10:51.626 INFO:tasks.rados.rados.0.smithi195.stdout:388: done (7 left) 2024-09-13T10:10:51.626 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 7 2024-09-13T10:10:51.626 INFO:tasks.rados.rados.0.smithi195.stdout:384: finishing set_chunk to oid 35 2024-09-13T10:10:51.626 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 35 set_chunk 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 91648 length: 1482752 2024-09-13T10:10:51.626 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 0 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-13T10:10:51.626 INFO:tasks.rados.rados.0.smithi195.stdout:384: done (6 left) 2024-09-13T10:10:51.627 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 6 2024-09-13T10:10:51.627 INFO:tasks.rados.rados.0.smithi195.stdout:350: finishing set_chunk to oid 1 2024-09-13T10:10:51.627 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-13T10:10:51.627 INFO:tasks.rados.rados.0.smithi195.stdout:350: done (5 left) 2024-09-13T10:10:51.627 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 5 2024-09-13T10:10:51.627 INFO:tasks.rados.rados.0.smithi195.stdout:347: finishing set_chunk to oid 48 2024-09-13T10:10:51.627 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 15 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-13T10:10:51.627 INFO:tasks.rados.rados.0.smithi195.stdout:347: done (4 left) 2024-09-13T10:10:51.627 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 4 2024-09-13T10:10:51.627 INFO:tasks.rados.rados.0.smithi195.stdout:380: finishing set_chunk to oid 31 2024-09-13T10:10:51.627 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 6 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-13T10:10:51.627 INFO:tasks.rados.rados.0.smithi195.stdout:385: finishing set_chunk to oid 36 2024-09-13T10:10:51.627 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 6 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-13T10:10:51.628 INFO:tasks.rados.rados.0.smithi195.stdout:380: done (3 left) 2024-09-13T10:10:51.628 INFO:tasks.rados.rados.0.smithi195.stdout:385: done (2 left) 2024-09-13T10:10:51.628 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 2 2024-09-13T10:10:51.628 INFO:tasks.rados.rados.0.smithi195.stdout:368: finishing set_chunk to oid 19 2024-09-13T10:10:51.628 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 9 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-13T10:10:51.628 INFO:tasks.rados.rados.0.smithi195.stdout:368: done (1 left) 2024-09-13T10:10:51.628 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 1 2024-09-13T10:10:51.628 INFO:tasks.rados.rados.0.smithi195.stdout:339: finishing set_chunk to oid 40 2024-09-13T10:10:51.628 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-13T10:10:51.628 INFO:tasks.rados.rados.0.smithi195.stdout:339: done (0 left) 2024-09-13T10:10:51.628 INFO:tasks.rados.rados.0.smithi195.stdout:389: set_chunk oid 40 target oid 40 low_tier 2024-09-13T10:10:51.628 INFO:tasks.rados.rados.0.smithi195.stdout:389: set_chunk oid 40 offset: 155648 length: 1387520 target oid smithi19530714-40 low_tier offset: 155648 snap -1 2024-09-13T10:10:51.628 INFO:tasks.rados.rados.0.smithi195.stdout:390: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.628 INFO:tasks.rados.rados.0.smithi195.stdout:390: set_chunk oid 41 offset: 27136 length: 1469952 target oid smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 27136 snap -1 2024-09-13T10:10:51.629 INFO:tasks.rados.rados.0.smithi195.stdout:391: set_chunk oid 42 target oid 42 low_tier 2024-09-13T10:10:51.629 INFO:tasks.rados.rados.0.smithi195.stdout:391: set_chunk oid 42 offset: 1422848 length: 467968 target oid smithi19530714-42 low_tier offset: 1422848 snap -1 2024-09-13T10:10:51.629 INFO:tasks.rados.rados.0.smithi195.stdout:392: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.629 INFO:tasks.rados.rados.0.smithi195.stdout:392: set_chunk oid 43 offset: 156160 length: 1119232 target oid smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 156160 snap -1 2024-09-13T10:10:51.629 INFO:tasks.rados.rados.0.smithi195.stdout:393: set_chunk oid 44 target oid 44 low_tier 2024-09-13T10:10:51.629 INFO:tasks.rados.rados.0.smithi195.stdout:393: set_chunk oid 44 offset: 438272 length: 996352 target oid smithi19530714-44 low_tier offset: 438272 snap -1 2024-09-13T10:10:51.629 INFO:tasks.rados.rados.0.smithi195.stdout:394: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.629 INFO:tasks.rados.rados.0.smithi195.stdout:394: set_chunk oid 45 offset: 1198592 length: 774144 target oid smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1198592 snap -1 2024-09-13T10:10:51.629 INFO:tasks.rados.rados.0.smithi195.stdout:395: set_chunk oid 46 target oid 46 low_tier 2024-09-13T10:10:51.629 INFO:tasks.rados.rados.0.smithi195.stdout:395: set_chunk oid 46 offset: 108544 length: 2104832 target oid smithi19530714-46 low_tier offset: 108544 snap -1 2024-09-13T10:10:51.629 INFO:tasks.rados.rados.0.smithi195.stdout:396: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.629 INFO:tasks.rados.rados.0.smithi195.stdout:396: set_chunk oid 47 offset: 743936 length: 1839616 target oid smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 743936 snap -1 2024-09-13T10:10:51.629 INFO:tasks.rados.rados.0.smithi195.stdout:397: set_chunk oid 48 target oid 48 low_tier 2024-09-13T10:10:51.630 INFO:tasks.rados.rados.0.smithi195.stdout:397: set_chunk oid 48 offset: 1347584 length: 323072 target oid smithi19530714-48 low_tier offset: 1347584 snap -1 2024-09-13T10:10:51.630 INFO:tasks.rados.rados.0.smithi195.stdout:398: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.630 INFO:tasks.rados.rados.0.smithi195.stdout:398: set_chunk oid 49 offset: 889344 length: 266752 target oid smithi19530714-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 889344 snap -1 2024-09-13T10:10:51.630 INFO:tasks.rados.rados.0.smithi195.stdout:399: set_chunk oid 50 target oid 50 low_tier 2024-09-13T10:10:51.630 INFO:tasks.rados.rados.0.smithi195.stdout:399: set_chunk oid 50 offset: 310272 length: 315904 target oid smithi19530714-50 low_tier offset: 310272 snap -1 2024-09-13T10:10:51.630 INFO:tasks.rados.rados.0.smithi195.stdout:400: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.630 INFO:tasks.rados.rados.0.smithi195.stdout:400: set_chunk oid 1 offset: 555520 length: 123392 target oid smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 555520 snap -1 2024-09-13T10:10:51.630 INFO:tasks.rados.rados.0.smithi195.stdout:401: set_chunk oid 2 target oid 2 low_tier 2024-09-13T10:10:51.630 INFO:tasks.rados.rados.0.smithi195.stdout:401: set_chunk oid 2 offset: 826880 length: 572416 target oid smithi19530714-2 low_tier offset: 826880 snap -1 2024-09-13T10:10:51.630 INFO:tasks.rados.rados.0.smithi195.stdout:402: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.630 INFO:tasks.rados.rados.0.smithi195.stdout:402: set_chunk oid 3 offset: 608768 length: 457728 target oid smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 608768 snap -1 2024-09-13T10:10:51.630 INFO:tasks.rados.rados.0.smithi195.stdout:403: set_chunk oid 4 target oid 4 low_tier 2024-09-13T10:10:51.630 INFO:tasks.rados.rados.0.smithi195.stdout:403: set_chunk oid 4 offset: 1715712 length: 140288 target oid smithi19530714-4 low_tier offset: 1715712 snap -1 2024-09-13T10:10:51.631 INFO:tasks.rados.rados.0.smithi195.stdout:404: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.631 INFO:tasks.rados.rados.0.smithi195.stdout:404: set_chunk oid 5 offset: 443904 length: 249344 target oid smithi19530714-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 443904 snap -1 2024-09-13T10:10:51.631 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.631 INFO:tasks.rados.rados.0.smithi195.stdout:392: finishing set_chunk to oid 43 2024-09-13T10:10:51.631 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 156160 length: 1119232 2024-09-13T10:10:51.631 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-13T10:10:51.631 INFO:tasks.rados.rados.0.smithi195.stdout:392: done (15 left) 2024-09-13T10:10:51.631 INFO:tasks.rados.rados.0.smithi195.stdout:405: set_chunk oid 6 target oid 6 low_tier 2024-09-13T10:10:51.631 INFO:tasks.rados.rados.0.smithi195.stdout:405: set_chunk oid 6 offset: 39936 length: 1348096 target oid smithi19530714-6 low_tier offset: 39936 snap -1 2024-09-13T10:10:51.631 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.631 INFO:tasks.rados.rados.0.smithi195.stdout:389: finishing set_chunk to oid 40 2024-09-13T10:10:51.631 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 40 set_chunk 40 low_tier returned error code -95 offset: 155648 length: 1387520 2024-09-13T10:10:51.631 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 0 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-13T10:10:51.631 INFO:tasks.rados.rados.0.smithi195.stdout:393: finishing set_chunk to oid 44 2024-09-13T10:10:51.632 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 44 set_chunk 44 low_tier returned error code -95 offset: 438272 length: 996352 2024-09-13T10:10:51.632 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 0 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-13T10:10:51.632 INFO:tasks.rados.rados.0.smithi195.stdout:389: done (15 left) 2024-09-13T10:10:51.632 INFO:tasks.rados.rados.0.smithi195.stdout:393: done (14 left) 2024-09-13T10:10:51.632 INFO:tasks.rados.rados.0.smithi195.stdout:406: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.632 INFO:tasks.rados.rados.0.smithi195.stdout:406: set_chunk oid 7 offset: 603648 length: 181248 target oid smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 603648 snap -1 2024-09-13T10:10:51.632 INFO:tasks.rados.rados.0.smithi195.stdout:407: set_chunk oid 8 target oid 8 low_tier 2024-09-13T10:10:51.632 INFO:tasks.rados.rados.0.smithi195.stdout:407: set_chunk oid 8 offset: 972288 length: 991744 target oid smithi19530714-8 low_tier offset: 972288 snap -1 2024-09-13T10:10:51.632 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.632 INFO:tasks.rados.rados.0.smithi195.stdout:391: finishing set_chunk to oid 42 2024-09-13T10:10:51.632 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 42 set_chunk 42 low_tier returned error code -95 offset: 1422848 length: 467968 2024-09-13T10:10:51.632 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-13T10:10:51.632 INFO:tasks.rados.rados.0.smithi195.stdout:399: finishing set_chunk to oid 50 2024-09-13T10:10:51.633 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 50 set_chunk 50 low_tier returned error code -95 offset: 310272 length: 315904 2024-09-13T10:10:51.633 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 0 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-13T10:10:51.633 INFO:tasks.rados.rados.0.smithi195.stdout:391: done (15 left) 2024-09-13T10:10:51.633 INFO:tasks.rados.rados.0.smithi195.stdout:399: done (14 left) 2024-09-13T10:10:51.633 INFO:tasks.rados.rados.0.smithi195.stdout:408: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.633 INFO:tasks.rados.rados.0.smithi195.stdout:408: set_chunk oid 9 offset: 1533440 length: 208384 target oid smithi19530714-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1533440 snap -1 2024-09-13T10:10:51.633 INFO:tasks.rados.rados.0.smithi195.stdout:409: set_chunk oid 10 target oid 10 low_tier 2024-09-13T10:10:51.633 INFO:tasks.rados.rados.0.smithi195.stdout:409: set_chunk oid 10 offset: 218624 length: 886784 target oid smithi19530714-10 low_tier offset: 218624 snap -1 2024-09-13T10:10:51.633 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.633 INFO:tasks.rados.rados.0.smithi195.stdout:394: finishing set_chunk to oid 45 2024-09-13T10:10:51.633 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 45 set_chunk 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1198592 length: 774144 2024-09-13T10:10:51.633 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 0 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-13T10:10:51.633 INFO:tasks.rados.rados.0.smithi195.stdout:398: finishing set_chunk to oid 49 2024-09-13T10:10:51.634 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 49 set_chunk 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 889344 length: 266752 2024-09-13T10:10:51.634 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 0 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-13T10:10:51.634 INFO:tasks.rados.rados.0.smithi195.stdout:401: finishing set_chunk to oid 2 2024-09-13T10:10:51.634 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 2 set_chunk 2 low_tier returned error code -95 offset: 826880 length: 572416 2024-09-13T10:10:51.634 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-13T10:10:51.634 INFO:tasks.rados.rados.0.smithi195.stdout:390: finishing set_chunk to oid 41 2024-09-13T10:10:51.634 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 41 set_chunk 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 27136 length: 1469952 2024-09-13T10:10:51.634 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-13T10:10:51.634 INFO:tasks.rados.rados.0.smithi195.stdout:397: finishing set_chunk to oid 48 2024-09-13T10:10:51.634 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 48 set_chunk 48 low_tier returned error code -95 offset: 1347584 length: 323072 2024-09-13T10:10:51.634 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-13T10:10:51.634 INFO:tasks.rados.rados.0.smithi195.stdout:395: finishing set_chunk to oid 46 2024-09-13T10:10:51.634 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 46 set_chunk 46 low_tier returned error code -95 offset: 108544 length: 2104832 2024-09-13T10:10:51.634 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 0 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-13T10:10:51.635 INFO:tasks.rados.rados.0.smithi195.stdout:396: finishing set_chunk to oid 47 2024-09-13T10:10:51.635 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 47 set_chunk 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 743936 length: 1839616 2024-09-13T10:10:51.635 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-13T10:10:51.635 INFO:tasks.rados.rados.0.smithi195.stdout:404: finishing set_chunk to oid 5 2024-09-13T10:10:51.635 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 5 set_chunk 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 443904 length: 249344 2024-09-13T10:10:51.635 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 0 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-13T10:10:51.635 INFO:tasks.rados.rados.0.smithi195.stdout:390: done (15 left) 2024-09-13T10:10:51.635 INFO:tasks.rados.rados.0.smithi195.stdout:394: done (14 left) 2024-09-13T10:10:51.635 INFO:tasks.rados.rados.0.smithi195.stdout:395: done (13 left) 2024-09-13T10:10:51.635 INFO:tasks.rados.rados.0.smithi195.stdout:396: done (12 left) 2024-09-13T10:10:51.635 INFO:tasks.rados.rados.0.smithi195.stdout:397: done (11 left) 2024-09-13T10:10:51.635 INFO:tasks.rados.rados.0.smithi195.stdout:398: done (10 left) 2024-09-13T10:10:51.635 INFO:tasks.rados.rados.0.smithi195.stdout:401: done (9 left) 2024-09-13T10:10:51.636 INFO:tasks.rados.rados.0.smithi195.stdout:404: done (8 left) 2024-09-13T10:10:51.636 INFO:tasks.rados.rados.0.smithi195.stdout:410: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.636 INFO:tasks.rados.rados.0.smithi195.stdout:410: set_chunk oid 11 offset: 25600 length: 885248 target oid smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 25600 snap -1 2024-09-13T10:10:51.636 INFO:tasks.rados.rados.0.smithi195.stdout:411: set_chunk oid 12 target oid 12 low_tier 2024-09-13T10:10:51.636 INFO:tasks.rados.rados.0.smithi195.stdout:411: set_chunk oid 12 offset: 1361920 length: 311808 target oid smithi19530714-12 low_tier offset: 1361920 snap -1 2024-09-13T10:10:51.636 INFO:tasks.rados.rados.0.smithi195.stdout:412: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.636 INFO:tasks.rados.rados.0.smithi195.stdout:402: finishing set_chunk to oid 3 2024-09-13T10:10:51.636 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 3 set_chunk 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 608768 length: 457728 2024-09-13T10:10:51.636 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 0 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-13T10:10:51.637 INFO:tasks.rados.rados.0.smithi195.stdout:412: set_chunk oid 13 offset: 165888 length: 1608192 target oid smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 165888 snap -1 2024-09-13T10:10:51.637 INFO:tasks.rados.rados.0.smithi195.stdout:402: done (10 left) 2024-09-13T10:10:51.637 INFO:tasks.rados.rados.0.smithi195.stdout:413: set_chunk oid 14 target oid 14 low_tier 2024-09-13T10:10:51.637 INFO:tasks.rados.rados.0.smithi195.stdout:413: set_chunk oid 14 offset: 242688 length: 1581568 target oid smithi19530714-14 low_tier offset: 242688 snap -1 2024-09-13T10:10:51.637 INFO:tasks.rados.rados.0.smithi195.stdout:414: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.637 INFO:tasks.rados.rados.0.smithi195.stdout:414: set_chunk oid 15 offset: 753664 length: 836608 target oid smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 753664 snap -1 2024-09-13T10:10:51.637 INFO:tasks.rados.rados.0.smithi195.stdout:415: set_chunk oid 16 target oid 16 low_tier 2024-09-13T10:10:51.637 INFO:tasks.rados.rados.0.smithi195.stdout:415: set_chunk oid 16 offset: 333824 length: 909312 target oid smithi19530714-16 low_tier offset: 333824 snap -1 2024-09-13T10:10:51.637 INFO:tasks.rados.rados.0.smithi195.stdout:416: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.637 INFO:tasks.rados.rados.0.smithi195.stdout:416: set_chunk oid 17 offset: 552960 length: 299008 target oid smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 552960 snap -1 2024-09-13T10:10:51.637 INFO:tasks.rados.rados.0.smithi195.stdout:417: set_chunk oid 18 target oid 18 low_tier 2024-09-13T10:10:51.637 INFO:tasks.rados.rados.0.smithi195.stdout:417: set_chunk oid 18 offset: 118784 length: 177152 target oid smithi19530714-18 low_tier offset: 118784 snap -1 2024-09-13T10:10:51.637 INFO:tasks.rados.rados.0.smithi195.stdout:418: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.637 INFO:tasks.rados.rados.0.smithi195.stdout:418: set_chunk oid 19 offset: 400896 length: 266752 target oid smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 400896 snap -1 2024-09-13T10:10:51.638 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.638 INFO:tasks.rados.rados.0.smithi195.stdout:405: finishing set_chunk to oid 6 2024-09-13T10:10:51.638 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 6 set_chunk 6 low_tier returned error code -95 offset: 39936 length: 1348096 2024-09-13T10:10:51.638 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 0 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-13T10:10:51.638 INFO:tasks.rados.rados.0.smithi195.stdout:405: done (15 left) 2024-09-13T10:10:51.638 INFO:tasks.rados.rados.0.smithi195.stdout:419: set_chunk oid 20 target oid 20 low_tier 2024-09-13T10:10:51.638 INFO:tasks.rados.rados.0.smithi195.stdout:419: set_chunk oid 20 offset: 1441792 length: 125440 target oid smithi19530714-20 low_tier offset: 1441792 snap -1 2024-09-13T10:10:51.638 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.638 INFO:tasks.rados.rados.0.smithi195.stdout:407: finishing set_chunk to oid 8 2024-09-13T10:10:51.638 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 8 set_chunk 8 low_tier returned error code -95 offset: 972288 length: 991744 2024-09-13T10:10:51.638 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-13T10:10:51.638 INFO:tasks.rados.rados.0.smithi195.stdout:407: done (15 left) 2024-09-13T10:10:51.638 INFO:tasks.rados.rados.0.smithi195.stdout:420: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.638 INFO:tasks.rados.rados.0.smithi195.stdout:420: set_chunk oid 21 offset: 88064 length: 2073600 target oid smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 88064 snap -1 2024-09-13T10:10:51.639 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.639 INFO:tasks.rados.rados.0.smithi195.stdout:406: finishing set_chunk to oid 7 2024-09-13T10:10:51.639 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 603648 length: 181248 2024-09-13T10:10:51.639 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-13T10:10:51.639 INFO:tasks.rados.rados.0.smithi195.stdout:406: done (15 left) 2024-09-13T10:10:51.639 INFO:tasks.rados.rados.0.smithi195.stdout:421: set_chunk oid 22 target oid 22 low_tier 2024-09-13T10:10:51.639 INFO:tasks.rados.rados.0.smithi195.stdout:421: set_chunk oid 22 offset: 343040 length: 793600 target oid smithi19530714-22 low_tier offset: 343040 snap -1 2024-09-13T10:10:51.639 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.639 INFO:tasks.rados.rados.0.smithi195.stdout:409: finishing set_chunk to oid 10 2024-09-13T10:10:51.639 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 10 set_chunk 10 low_tier returned error code -95 offset: 218624 length: 886784 2024-09-13T10:10:51.639 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 0 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-13T10:10:51.639 INFO:tasks.rados.rados.0.smithi195.stdout:410: finishing set_chunk to oid 11 2024-09-13T10:10:51.639 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 11 set_chunk 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 25600 length: 885248 2024-09-13T10:10:51.640 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 0 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-13T10:10:51.640 INFO:tasks.rados.rados.0.smithi195.stdout:409: done (15 left) 2024-09-13T10:10:51.640 INFO:tasks.rados.rados.0.smithi195.stdout:410: done (14 left) 2024-09-13T10:10:51.640 INFO:tasks.rados.rados.0.smithi195.stdout:422: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.640 INFO:tasks.rados.rados.0.smithi195.stdout:422: set_chunk oid 23 offset: 63488 length: 1249792 target oid smithi19530714-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 63488 snap -1 2024-09-13T10:10:51.640 INFO:tasks.rados.rados.0.smithi195.stdout:423: set_chunk oid 24 target oid 24 low_tier 2024-09-13T10:10:51.640 INFO:tasks.rados.rados.0.smithi195.stdout:423: set_chunk oid 24 offset: 1148928 length: 53760 target oid smithi19530714-24 low_tier offset: 1148928 snap -1 2024-09-13T10:10:51.640 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.640 INFO:tasks.rados.rados.0.smithi195.stdout:413: finishing set_chunk to oid 14 2024-09-13T10:10:51.640 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 14 set_chunk 14 low_tier returned error code -95 offset: 242688 length: 1581568 2024-09-13T10:10:51.640 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 0 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-13T10:10:51.640 INFO:tasks.rados.rados.0.smithi195.stdout:418: finishing set_chunk to oid 19 2024-09-13T10:10:51.640 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 19 set_chunk 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 400896 length: 266752 2024-09-13T10:10:51.641 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 0 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-13T10:10:51.641 INFO:tasks.rados.rados.0.smithi195.stdout:415: finishing set_chunk to oid 16 2024-09-13T10:10:51.641 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 333824 length: 909312 2024-09-13T10:10:51.641 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-13T10:10:51.641 INFO:tasks.rados.rados.0.smithi195.stdout:413: done (15 left) 2024-09-13T10:10:51.641 INFO:tasks.rados.rados.0.smithi195.stdout:415: done (14 left) 2024-09-13T10:10:51.641 INFO:tasks.rados.rados.0.smithi195.stdout:418: done (13 left) 2024-09-13T10:10:51.641 INFO:tasks.rados.rados.0.smithi195.stdout:424: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.641 INFO:tasks.rados.rados.0.smithi195.stdout:424: set_chunk oid 25 offset: 127488 length: 319488 target oid smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 127488 snap -1 2024-09-13T10:10:51.641 INFO:tasks.rados.rados.0.smithi195.stdout:425: set_chunk oid 26 target oid 26 low_tier 2024-09-13T10:10:51.641 INFO:tasks.rados.rados.0.smithi195.stdout:425: set_chunk oid 26 offset: 63488 length: 2240512 target oid smithi19530714-26 low_tier offset: 63488 snap -1 2024-09-13T10:10:51.641 INFO:tasks.rados.rados.0.smithi195.stdout:426: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.641 INFO:tasks.rados.rados.0.smithi195.stdout:426: set_chunk oid 27 offset: 950272 length: 65536 target oid smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 950272 snap -1 2024-09-13T10:10:51.641 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.642 INFO:tasks.rados.rados.0.smithi195.stdout:414: finishing set_chunk to oid 15 2024-09-13T10:10:51.642 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 15 set_chunk 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 753664 length: 836608 2024-09-13T10:10:51.642 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-13T10:10:51.642 INFO:tasks.rados.rados.0.smithi195.stdout:416: finishing set_chunk to oid 17 2024-09-13T10:10:51.642 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 552960 length: 299008 2024-09-13T10:10:51.642 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-13T10:10:51.642 INFO:tasks.rados.rados.0.smithi195.stdout:419: finishing set_chunk to oid 20 2024-09-13T10:10:51.642 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 1441792 length: 125440 2024-09-13T10:10:51.642 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-13T10:10:51.642 INFO:tasks.rados.rados.0.smithi195.stdout:420: finishing set_chunk to oid 21 2024-09-13T10:10:51.642 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 21 set_chunk 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 88064 length: 2073600 2024-09-13T10:10:51.642 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 0 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-13T10:10:51.642 INFO:tasks.rados.rados.0.smithi195.stdout:414: done (15 left) 2024-09-13T10:10:51.643 INFO:tasks.rados.rados.0.smithi195.stdout:416: done (14 left) 2024-09-13T10:10:51.643 INFO:tasks.rados.rados.0.smithi195.stdout:419: done (13 left) 2024-09-13T10:10:51.643 INFO:tasks.rados.rados.0.smithi195.stdout:420: done (12 left) 2024-09-13T10:10:51.643 INFO:tasks.rados.rados.0.smithi195.stdout:427: set_chunk oid 28 target oid 28 low_tier 2024-09-13T10:10:51.643 INFO:tasks.rados.rados.0.smithi195.stdout:427: set_chunk oid 28 offset: 850432 length: 263168 target oid smithi19530714-28 low_tier offset: 850432 snap -1 2024-09-13T10:10:51.643 INFO:tasks.rados.rados.0.smithi195.stdout:428: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.643 INFO:tasks.rados.rados.0.smithi195.stdout:428: set_chunk oid 29 offset: 988160 length: 527360 target oid smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 988160 snap -1 2024-09-13T10:10:51.643 INFO:tasks.rados.rados.0.smithi195.stdout:429: set_chunk oid 30 target oid 30 low_tier 2024-09-13T10:10:51.643 INFO:tasks.rados.rados.0.smithi195.stdout:429: set_chunk oid 30 offset: 1354752 length: 1050624 target oid smithi19530714-30 low_tier offset: 1354752 snap -1 2024-09-13T10:10:51.643 INFO:tasks.rados.rados.0.smithi195.stdout:430: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.643 INFO:tasks.rados.rados.0.smithi195.stdout:430: set_chunk oid 31 offset: 1771520 length: 84992 target oid smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1771520 snap -1 2024-09-13T10:10:51.643 INFO:tasks.rados.rados.0.smithi195.stdout:412: finishing set_chunk to oid 13 2024-09-13T10:10:51.643 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 165888 length: 1608192 2024-09-13T10:10:51.644 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-13T10:10:51.644 INFO:tasks.rados.rados.0.smithi195.stdout:422: finishing set_chunk to oid 23 2024-09-13T10:10:51.644 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 23 set_chunk 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 63488 length: 1249792 2024-09-13T10:10:51.644 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-13T10:10:51.644 INFO:tasks.rados.rados.0.smithi195.stdout:421: finishing set_chunk to oid 22 2024-09-13T10:10:51.644 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 343040 length: 793600 2024-09-13T10:10:51.644 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-13T10:10:51.644 INFO:tasks.rados.rados.0.smithi195.stdout:412: done (15 left) 2024-09-13T10:10:51.644 INFO:tasks.rados.rados.0.smithi195.stdout:421: done (14 left) 2024-09-13T10:10:51.644 INFO:tasks.rados.rados.0.smithi195.stdout:422: done (13 left) 2024-09-13T10:10:51.644 INFO:tasks.rados.rados.0.smithi195.stdout:431: set_chunk oid 32 target oid 32 low_tier 2024-09-13T10:10:51.644 INFO:tasks.rados.rados.0.smithi195.stdout:431: set_chunk oid 32 offset: 1034752 length: 261120 target oid smithi19530714-32 low_tier offset: 1034752 snap -1 2024-09-13T10:10:51.645 INFO:tasks.rados.rados.0.smithi195.stdout:432: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.645 INFO:tasks.rados.rados.0.smithi195.stdout:432: set_chunk oid 33 offset: 229376 length: 2245632 target oid smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 229376 snap -1 2024-09-13T10:10:51.645 INFO:tasks.rados.rados.0.smithi195.stdout:433: set_chunk oid 34 target oid 34 low_tier 2024-09-13T10:10:51.645 INFO:tasks.rados.rados.0.smithi195.stdout:433: set_chunk oid 34 offset: 1412096 length: 584704 target oid smithi19530714-34 low_tier offset: 1412096 snap -1 2024-09-13T10:10:51.645 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.645 INFO:tasks.rados.rados.0.smithi195.stdout:425: finishing set_chunk to oid 26 2024-09-13T10:10:51.645 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 26 set_chunk 26 low_tier returned error code -95 offset: 63488 length: 2240512 2024-09-13T10:10:51.645 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 0 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-13T10:10:51.645 INFO:tasks.rados.rados.0.smithi195.stdout:425: done (15 left) 2024-09-13T10:10:51.645 INFO:tasks.rados.rados.0.smithi195.stdout:434: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.645 INFO:tasks.rados.rados.0.smithi195.stdout:434: set_chunk oid 35 offset: 702464 length: 1186816 target oid smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 702464 snap -1 2024-09-13T10:10:51.646 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.646 INFO:tasks.rados.rados.0.smithi195.stdout:427: finishing set_chunk to oid 28 2024-09-13T10:10:51.646 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 28 set_chunk 28 low_tier returned error code -95 offset: 850432 length: 263168 2024-09-13T10:10:51.646 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 0 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-13T10:10:51.646 INFO:tasks.rados.rados.0.smithi195.stdout:427: done (15 left) 2024-09-13T10:10:51.646 INFO:tasks.rados.rados.0.smithi195.stdout:435: set_chunk oid 36 target oid 36 low_tier 2024-09-13T10:10:51.646 INFO:tasks.rados.rados.0.smithi195.stdout:435: set_chunk oid 36 offset: 128512 length: 1168384 target oid smithi19530714-36 low_tier offset: 128512 snap -1 2024-09-13T10:10:51.646 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.646 INFO:tasks.rados.rados.0.smithi195.stdout:426: finishing set_chunk to oid 27 2024-09-13T10:10:51.646 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 27 set_chunk 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 950272 length: 65536 2024-09-13T10:10:51.646 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-13T10:10:51.646 INFO:tasks.rados.rados.0.smithi195.stdout:428: finishing set_chunk to oid 29 2024-09-13T10:10:51.647 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 29 set_chunk 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 988160 length: 527360 2024-09-13T10:10:51.647 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 0 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-13T10:10:51.647 INFO:tasks.rados.rados.0.smithi195.stdout:430: finishing set_chunk to oid 31 2024-09-13T10:10:51.647 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 31 set_chunk 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1771520 length: 84992 2024-09-13T10:10:51.647 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-13T10:10:51.647 INFO:tasks.rados.rados.0.smithi195.stdout:433: finishing set_chunk to oid 34 2024-09-13T10:10:51.647 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 34 set_chunk 34 low_tier returned error code -95 offset: 1412096 length: 584704 2024-09-13T10:10:51.647 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 34 v 0 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-13T10:10:51.647 INFO:tasks.rados.rados.0.smithi195.stdout:431: finishing set_chunk to oid 32 2024-09-13T10:10:51.647 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 1034752 length: 261120 2024-09-13T10:10:51.647 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-13T10:10:51.648 INFO:tasks.rados.rados.0.smithi195.stdout:432: finishing set_chunk to oid 33 2024-09-13T10:10:51.648 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 33 set_chunk 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 229376 length: 2245632 2024-09-13T10:10:51.648 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 0 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-13T10:10:51.648 INFO:tasks.rados.rados.0.smithi195.stdout:429: finishing set_chunk to oid 30 2024-09-13T10:10:51.648 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 30 set_chunk 30 low_tier returned error code -95 offset: 1354752 length: 1050624 2024-09-13T10:10:51.648 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 0 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-13T10:10:51.648 INFO:tasks.rados.rados.0.smithi195.stdout:426: done (15 left) 2024-09-13T10:10:51.648 INFO:tasks.rados.rados.0.smithi195.stdout:428: done (14 left) 2024-09-13T10:10:51.648 INFO:tasks.rados.rados.0.smithi195.stdout:429: done (13 left) 2024-09-13T10:10:51.648 INFO:tasks.rados.rados.0.smithi195.stdout:430: done (12 left) 2024-09-13T10:10:51.648 INFO:tasks.rados.rados.0.smithi195.stdout:431: done (11 left) 2024-09-13T10:10:51.648 INFO:tasks.rados.rados.0.smithi195.stdout:432: done (10 left) 2024-09-13T10:10:51.649 INFO:tasks.rados.rados.0.smithi195.stdout:433: done (9 left) 2024-09-13T10:10:51.649 INFO:tasks.rados.rados.0.smithi195.stdout:436: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.649 INFO:tasks.rados.rados.0.smithi195.stdout:436: set_chunk oid 37 offset: 1199104 length: 1042432 target oid smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1199104 snap -1 2024-09-13T10:10:51.649 INFO:tasks.rados.rados.0.smithi195.stdout:437: set_chunk oid 38 target oid 38 low_tier 2024-09-13T10:10:51.649 INFO:tasks.rados.rados.0.smithi195.stdout:437: set_chunk oid 38 offset: 619520 length: 854016 target oid smithi19530714-38 low_tier offset: 619520 snap -1 2024-09-13T10:10:51.649 INFO:tasks.rados.rados.0.smithi195.stdout:438: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.649 INFO:tasks.rados.rados.0.smithi195.stdout:438: set_chunk oid 39 offset: 431616 length: 2176000 target oid smithi19530714-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 431616 snap -1 2024-09-13T10:10:51.649 INFO:tasks.rados.rados.0.smithi195.stdout:439: set_chunk oid 40 target oid 40 low_tier 2024-09-13T10:10:51.649 INFO:tasks.rados.rados.0.smithi195.stdout:439: set_chunk oid 40 offset: 525824 length: 1060864 target oid smithi19530714-40 low_tier offset: 525824 snap -1 2024-09-13T10:10:51.649 INFO:tasks.rados.rados.0.smithi195.stdout:434: finishing set_chunk to oid 35 2024-09-13T10:10:51.649 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 35 set_chunk 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 702464 length: 1186816 2024-09-13T10:10:51.650 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 0 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-13T10:10:51.650 INFO:tasks.rados.rados.0.smithi195.stdout:435: finishing set_chunk to oid 36 2024-09-13T10:10:51.650 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 36 set_chunk 36 low_tier returned error code -95 offset: 128512 length: 1168384 2024-09-13T10:10:51.650 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 0 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-13T10:10:51.650 INFO:tasks.rados.rados.0.smithi195.stdout:434: done (12 left) 2024-09-13T10:10:51.650 INFO:tasks.rados.rados.0.smithi195.stdout:435: done (11 left) 2024-09-13T10:10:51.650 INFO:tasks.rados.rados.0.smithi195.stdout:440: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.650 INFO:tasks.rados.rados.0.smithi195.stdout:440: set_chunk oid 41 offset: 195072 length: 2406400 target oid smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 195072 snap -1 2024-09-13T10:10:51.650 INFO:tasks.rados.rados.0.smithi195.stdout:441: set_chunk oid 42 target oid 42 low_tier 2024-09-13T10:10:51.650 INFO:tasks.rados.rados.0.smithi195.stdout:441: set_chunk oid 42 offset: 1164288 length: 1070592 target oid smithi19530714-42 low_tier offset: 1164288 snap -1 2024-09-13T10:10:51.650 INFO:tasks.rados.rados.0.smithi195.stdout:442: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.650 INFO:tasks.rados.rados.0.smithi195.stdout:442: set_chunk oid 43 offset: 586752 length: 679936 target oid smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 586752 snap -1 2024-09-13T10:10:51.650 INFO:tasks.rados.rados.0.smithi195.stdout:443: set_chunk oid 44 target oid 44 low_tier 2024-09-13T10:10:51.651 INFO:tasks.rados.rados.0.smithi195.stdout:443: set_chunk oid 44 offset: 298496 length: 1584128 target oid smithi19530714-44 low_tier offset: 298496 snap -1 2024-09-13T10:10:51.651 INFO:tasks.rados.rados.0.smithi195.stdout:444: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.651 INFO:tasks.rados.rados.0.smithi195.stdout:444: set_chunk oid 45 offset: 1724928 length: 742912 target oid smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1724928 snap -1 2024-09-13T10:10:51.651 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.651 INFO:tasks.rados.rados.0.smithi195.stdout:436: finishing set_chunk to oid 37 2024-09-13T10:10:51.651 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 37 set_chunk 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1199104 length: 1042432 2024-09-13T10:10:51.651 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 0 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-13T10:10:51.651 INFO:tasks.rados.rados.0.smithi195.stdout:436: done (15 left) 2024-09-13T10:10:51.651 INFO:tasks.rados.rados.0.smithi195.stdout:445: set_chunk oid 46 target oid 46 low_tier 2024-09-13T10:10:51.651 INFO:tasks.rados.rados.0.smithi195.stdout:445: set_chunk oid 46 offset: 667136 length: 126464 target oid smithi19530714-46 low_tier offset: 667136 snap -1 2024-09-13T10:10:51.651 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.651 INFO:tasks.rados.rados.0.smithi195.stdout:439: finishing set_chunk to oid 40 2024-09-13T10:10:51.651 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 40 set_chunk 40 low_tier returned error code -95 offset: 525824 length: 1060864 2024-09-13T10:10:51.652 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 0 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-13T10:10:51.652 INFO:tasks.rados.rados.0.smithi195.stdout:439: done (15 left) 2024-09-13T10:10:51.652 INFO:tasks.rados.rados.0.smithi195.stdout:446: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.652 INFO:tasks.rados.rados.0.smithi195.stdout:446: set_chunk oid 47 offset: 418304 length: 1520128 target oid smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 418304 snap -1 2024-09-13T10:10:51.652 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.652 INFO:tasks.rados.rados.0.smithi195.stdout:442: finishing set_chunk to oid 43 2024-09-13T10:10:51.652 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 586752 length: 679936 2024-09-13T10:10:51.652 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-13T10:10:51.652 INFO:tasks.rados.rados.0.smithi195.stdout:438: finishing set_chunk to oid 39 2024-09-13T10:10:51.652 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 431616 length: 2176000 2024-09-13T10:10:51.652 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-13T10:10:51.652 INFO:tasks.rados.rados.0.smithi195.stdout:441: finishing set_chunk to oid 42 2024-09-13T10:10:51.653 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 42 set_chunk 42 low_tier returned error code -95 offset: 1164288 length: 1070592 2024-09-13T10:10:51.653 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-13T10:10:51.653 INFO:tasks.rados.rados.0.smithi195.stdout:440: finishing set_chunk to oid 41 2024-09-13T10:10:51.653 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 41 set_chunk 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 195072 length: 2406400 2024-09-13T10:10:51.653 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-13T10:10:51.653 INFO:tasks.rados.rados.0.smithi195.stdout:438: done (15 left) 2024-09-13T10:10:51.653 INFO:tasks.rados.rados.0.smithi195.stdout:440: done (14 left) 2024-09-13T10:10:51.653 INFO:tasks.rados.rados.0.smithi195.stdout:441: done (13 left) 2024-09-13T10:10:51.653 INFO:tasks.rados.rados.0.smithi195.stdout:442: done (12 left) 2024-09-13T10:10:51.653 INFO:tasks.rados.rados.0.smithi195.stdout:447: set_chunk oid 48 target oid 48 low_tier 2024-09-13T10:10:51.653 INFO:tasks.rados.rados.0.smithi195.stdout:447: set_chunk oid 48 offset: 615936 length: 1325568 target oid smithi19530714-48 low_tier offset: 615936 snap -1 2024-09-13T10:10:51.653 INFO:tasks.rados.rados.0.smithi195.stdout:448: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.654 INFO:tasks.rados.rados.0.smithi195.stdout:448: set_chunk oid 49 offset: 586240 length: 1568256 target oid smithi19530714-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 586240 snap -1 2024-09-13T10:10:51.654 INFO:tasks.rados.rados.0.smithi195.stdout:449: set_chunk oid 50 target oid 50 low_tier 2024-09-13T10:10:51.654 INFO:tasks.rados.rados.0.smithi195.stdout:449: set_chunk oid 50 offset: 2304000 length: 348160 target oid smithi19530714-50 low_tier offset: 2304000 snap -1 2024-09-13T10:10:51.654 INFO:tasks.rados.rados.0.smithi195.stdout:449 retry set_chunk ! 2024-09-13T10:10:51.654 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 15 2024-09-13T10:10:51.654 INFO:tasks.rados.rados.0.smithi195.stdout:437: finishing set_chunk to oid 38 2024-09-13T10:10:51.654 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 38 set_chunk 38 low_tier returned error code -95 offset: 619520 length: 854016 2024-09-13T10:10:51.654 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 0 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-13T10:10:51.654 INFO:tasks.rados.rados.0.smithi195.stdout:437: done (14 left) 2024-09-13T10:10:51.654 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 14 2024-09-13T10:10:51.654 INFO:tasks.rados.rados.0.smithi195.stdout:446: finishing set_chunk to oid 47 2024-09-13T10:10:51.654 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 47 set_chunk 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 418304 length: 1520128 2024-09-13T10:10:51.654 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-13T10:10:51.655 INFO:tasks.rados.rados.0.smithi195.stdout:444: finishing set_chunk to oid 45 2024-09-13T10:10:51.655 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 45 set_chunk 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1724928 length: 742912 2024-09-13T10:10:51.655 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 0 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-13T10:10:51.655 INFO:tasks.rados.rados.0.smithi195.stdout:444: done (13 left) 2024-09-13T10:10:51.655 INFO:tasks.rados.rados.0.smithi195.stdout:446: done (12 left) 2024-09-13T10:10:51.655 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 12 2024-09-13T10:10:51.655 INFO:tasks.rados.rados.0.smithi195.stdout:403: finishing set_chunk to oid 4 2024-09-13T10:10:51.655 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 3 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-13T10:10:51.655 INFO:tasks.rados.rados.0.smithi195.stdout:449: finishing set_chunk to oid 50 2024-09-13T10:10:51.655 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 50 set_chunk 50 low_tier returned error code -95 offset: 2304000 length: 348160 2024-09-13T10:10:51.655 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 0 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-13T10:10:51.655 INFO:tasks.rados.rados.0.smithi195.stdout:447: finishing set_chunk to oid 48 2024-09-13T10:10:51.655 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 48 set_chunk 48 low_tier returned error code -95 offset: 615936 length: 1325568 2024-09-13T10:10:51.656 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-13T10:10:51.656 INFO:tasks.rados.rados.0.smithi195.stdout:403: done (11 left) 2024-09-13T10:10:51.656 INFO:tasks.rados.rados.0.smithi195.stdout:447: done (10 left) 2024-09-13T10:10:51.656 INFO:tasks.rados.rados.0.smithi195.stdout:449: done (9 left) 2024-09-13T10:10:51.656 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 9 2024-09-13T10:10:51.656 INFO:tasks.rados.rados.0.smithi195.stdout:448: finishing set_chunk to oid 49 2024-09-13T10:10:51.656 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 49 set_chunk 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 586240 length: 1568256 2024-09-13T10:10:51.656 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 0 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-13T10:10:51.656 INFO:tasks.rados.rados.0.smithi195.stdout:448: done (8 left) 2024-09-13T10:10:51.656 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 8 2024-09-13T10:10:51.656 INFO:tasks.rados.rados.0.smithi195.stdout:400: finishing set_chunk to oid 1 2024-09-13T10:10:51.656 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-13T10:10:51.656 INFO:tasks.rados.rados.0.smithi195.stdout:400: done (7 left) 2024-09-13T10:10:51.656 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 7 2024-09-13T10:10:51.657 INFO:tasks.rados.rados.0.smithi195.stdout:443: finishing set_chunk to oid 44 2024-09-13T10:10:51.657 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 44 set_chunk 44 low_tier returned error code -95 offset: 298496 length: 1584128 2024-09-13T10:10:51.657 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 0 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-13T10:10:51.657 INFO:tasks.rados.rados.0.smithi195.stdout:443: done (6 left) 2024-09-13T10:10:51.657 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 6 2024-09-13T10:10:51.657 INFO:tasks.rados.rados.0.smithi195.stdout:408: finishing set_chunk to oid 9 2024-09-13T10:10:51.657 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 6 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-13T10:10:51.657 INFO:tasks.rados.rados.0.smithi195.stdout:408: done (5 left) 2024-09-13T10:10:51.657 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 5 2024-09-13T10:10:51.657 INFO:tasks.rados.rados.0.smithi195.stdout:411: finishing set_chunk to oid 12 2024-09-13T10:10:51.657 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 3 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-13T10:10:51.657 INFO:tasks.rados.rados.0.smithi195.stdout:411: done (4 left) 2024-09-13T10:10:51.657 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 4 2024-09-13T10:10:51.658 INFO:tasks.rados.rados.0.smithi195.stdout:424: finishing set_chunk to oid 25 2024-09-13T10:10:51.658 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 6 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-13T10:10:51.658 INFO:tasks.rados.rados.0.smithi195.stdout:424: done (3 left) 2024-09-13T10:10:51.658 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 3 2024-09-13T10:10:51.658 INFO:tasks.rados.rados.0.smithi195.stdout:423: finishing set_chunk to oid 24 2024-09-13T10:10:51.658 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 3 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-13T10:10:51.658 INFO:tasks.rados.rados.0.smithi195.stdout:423: done (2 left) 2024-09-13T10:10:51.658 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 2 2024-09-13T10:10:51.658 INFO:tasks.rados.rados.0.smithi195.stdout:445: finishing set_chunk to oid 46 2024-09-13T10:10:51.658 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 6 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-13T10:10:51.658 INFO:tasks.rados.rados.0.smithi195.stdout:445: done (1 left) 2024-09-13T10:10:51.658 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 1 2024-09-13T10:10:51.658 INFO:tasks.rados.rados.0.smithi195.stdout:417: finishing set_chunk to oid 18 2024-09-13T10:10:51.659 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 12 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-13T10:10:51.659 INFO:tasks.rados.rados.0.smithi195.stdout:417: done (0 left) 2024-09-13T10:10:51.659 INFO:tasks.rados.rados.0.smithi195.stdout:450: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.659 INFO:tasks.rados.rados.0.smithi195.stdout:450: set_chunk oid 1 offset: 764928 length: 1019904 target oid smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 764928 snap -1 2024-09-13T10:10:51.659 INFO:tasks.rados.rados.0.smithi195.stdout:451: set_chunk oid 2 target oid 2 low_tier 2024-09-13T10:10:51.659 INFO:tasks.rados.rados.0.smithi195.stdout:451: set_chunk oid 2 offset: 409600 length: 1185792 target oid smithi19530714-2 low_tier offset: 409600 snap -1 2024-09-13T10:10:51.659 INFO:tasks.rados.rados.0.smithi195.stdout:452: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.659 INFO:tasks.rados.rados.0.smithi195.stdout:452: set_chunk oid 3 offset: 632832 length: 345088 target oid smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 632832 snap -1 2024-09-13T10:10:51.659 INFO:tasks.rados.rados.0.smithi195.stdout:453: set_chunk oid 4 target oid 4 low_tier 2024-09-13T10:10:51.659 INFO:tasks.rados.rados.0.smithi195.stdout:453: set_chunk oid 4 offset: 621056 length: 909824 target oid smithi19530714-4 low_tier offset: 621056 snap -1 2024-09-13T10:10:51.659 INFO:tasks.rados.rados.0.smithi195.stdout:454: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.659 INFO:tasks.rados.rados.0.smithi195.stdout:454: set_chunk oid 5 offset: 240128 length: 1794048 target oid smithi19530714-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 240128 snap -1 2024-09-13T10:10:51.659 INFO:tasks.rados.rados.0.smithi195.stdout:455: set_chunk oid 6 target oid 6 low_tier 2024-09-13T10:10:51.660 INFO:tasks.rados.rados.0.smithi195.stdout:455: set_chunk oid 6 offset: 455168 length: 422912 target oid smithi19530714-6 low_tier offset: 455168 snap -1 2024-09-13T10:10:51.660 INFO:tasks.rados.rados.0.smithi195.stdout:456: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.660 INFO:tasks.rados.rados.0.smithi195.stdout:456: set_chunk oid 7 offset: 763904 length: 665088 target oid smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 763904 snap -1 2024-09-13T10:10:51.660 INFO:tasks.rados.rados.0.smithi195.stdout:457: set_chunk oid 8 target oid 8 low_tier 2024-09-13T10:10:51.660 INFO:tasks.rados.rados.0.smithi195.stdout:457: set_chunk oid 8 offset: 1057280 length: 358400 target oid smithi19530714-8 low_tier offset: 1057280 snap -1 2024-09-13T10:10:51.660 INFO:tasks.rados.rados.0.smithi195.stdout:458: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.660 INFO:tasks.rados.rados.0.smithi195.stdout:458: set_chunk oid 9 offset: 1398272 length: 695808 target oid smithi19530714-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1398272 snap -1 2024-09-13T10:10:51.660 INFO:tasks.rados.rados.0.smithi195.stdout:459: set_chunk oid 10 target oid 10 low_tier 2024-09-13T10:10:51.660 INFO:tasks.rados.rados.0.smithi195.stdout:459: set_chunk oid 10 offset: 40960 length: 1754112 target oid smithi19530714-10 low_tier offset: 40960 snap -1 2024-09-13T10:10:51.660 INFO:tasks.rados.rados.0.smithi195.stdout:460: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.660 INFO:tasks.rados.rados.0.smithi195.stdout:460: set_chunk oid 11 offset: 926208 length: 35840 target oid smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 926208 snap -1 2024-09-13T10:10:51.660 INFO:tasks.rados.rados.0.smithi195.stdout:461: set_chunk oid 12 target oid 12 low_tier 2024-09-13T10:10:51.661 INFO:tasks.rados.rados.0.smithi195.stdout:461: set_chunk oid 12 offset: 1296896 length: 486912 target oid smithi19530714-12 low_tier offset: 1296896 snap -1 2024-09-13T10:10:51.661 INFO:tasks.rados.rados.0.smithi195.stdout:462: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.661 INFO:tasks.rados.rados.0.smithi195.stdout:462: set_chunk oid 13 offset: 327168 length: 231936 target oid smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 327168 snap -1 2024-09-13T10:10:51.661 INFO:tasks.rados.rados.0.smithi195.stdout:463: set_chunk oid 14 target oid 14 low_tier 2024-09-13T10:10:51.661 INFO:tasks.rados.rados.0.smithi195.stdout:463: set_chunk oid 14 offset: 866816 length: 115200 target oid smithi19530714-14 low_tier offset: 866816 snap -1 2024-09-13T10:10:51.661 INFO:tasks.rados.rados.0.smithi195.stdout:464: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.661 INFO:tasks.rados.rados.0.smithi195.stdout:464: set_chunk oid 15 offset: 1213440 length: 1003008 target oid smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1213440 snap -1 2024-09-13T10:10:51.661 INFO:tasks.rados.rados.0.smithi195.stdout:465: set_chunk oid 16 target oid 16 low_tier 2024-09-13T10:10:51.661 INFO:tasks.rados.rados.0.smithi195.stdout:465: set_chunk oid 16 offset: 164864 length: 1204736 target oid smithi19530714-16 low_tier offset: 164864 snap -1 2024-09-13T10:10:51.661 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.661 INFO:tasks.rados.rados.0.smithi195.stdout:451: finishing set_chunk to oid 2 2024-09-13T10:10:51.661 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 2 set_chunk 2 low_tier returned error code -95 offset: 409600 length: 1185792 2024-09-13T10:10:51.662 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-13T10:10:51.662 INFO:tasks.rados.rados.0.smithi195.stdout:450: finishing set_chunk to oid 1 2024-09-13T10:10:51.662 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 1 set_chunk 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 764928 length: 1019904 2024-09-13T10:10:51.662 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 0 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-13T10:10:51.662 INFO:tasks.rados.rados.0.smithi195.stdout:452: finishing set_chunk to oid 3 2024-09-13T10:10:51.662 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 3 set_chunk 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 632832 length: 345088 2024-09-13T10:10:51.662 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 0 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-13T10:10:51.662 INFO:tasks.rados.rados.0.smithi195.stdout:454: finishing set_chunk to oid 5 2024-09-13T10:10:51.662 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 5 set_chunk 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 240128 length: 1794048 2024-09-13T10:10:51.662 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 0 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-13T10:10:51.662 INFO:tasks.rados.rados.0.smithi195.stdout:453: finishing set_chunk to oid 4 2024-09-13T10:10:51.662 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 4 set_chunk 4 low_tier returned error code -95 offset: 621056 length: 909824 2024-09-13T10:10:51.663 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 0 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-13T10:10:51.663 INFO:tasks.rados.rados.0.smithi195.stdout:457: finishing set_chunk to oid 8 2024-09-13T10:10:51.663 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 8 set_chunk 8 low_tier returned error code -95 offset: 1057280 length: 358400 2024-09-13T10:10:51.663 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-13T10:10:51.663 INFO:tasks.rados.rados.0.smithi195.stdout:456: finishing set_chunk to oid 7 2024-09-13T10:10:51.663 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 763904 length: 665088 2024-09-13T10:10:51.663 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-13T10:10:51.663 INFO:tasks.rados.rados.0.smithi195.stdout:455: finishing set_chunk to oid 6 2024-09-13T10:10:51.663 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 6 set_chunk 6 low_tier returned error code -95 offset: 455168 length: 422912 2024-09-13T10:10:51.663 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 0 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-13T10:10:51.663 INFO:tasks.rados.rados.0.smithi195.stdout:459: finishing set_chunk to oid 10 2024-09-13T10:10:51.663 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 10 set_chunk 10 low_tier returned error code -95 offset: 40960 length: 1754112 2024-09-13T10:10:51.663 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 0 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-13T10:10:51.663 INFO:tasks.rados.rados.0.smithi195.stdout:458: finishing set_chunk to oid 9 2024-09-13T10:10:51.664 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 9 set_chunk 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1398272 length: 695808 2024-09-13T10:10:51.664 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 0 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-13T10:10:51.664 INFO:tasks.rados.rados.0.smithi195.stdout:450: done (15 left) 2024-09-13T10:10:51.664 INFO:tasks.rados.rados.0.smithi195.stdout:451: done (14 left) 2024-09-13T10:10:51.664 INFO:tasks.rados.rados.0.smithi195.stdout:452: done (13 left) 2024-09-13T10:10:51.664 INFO:tasks.rados.rados.0.smithi195.stdout:453: done (12 left) 2024-09-13T10:10:51.664 INFO:tasks.rados.rados.0.smithi195.stdout:454: done (11 left) 2024-09-13T10:10:51.664 INFO:tasks.rados.rados.0.smithi195.stdout:455: done (10 left) 2024-09-13T10:10:51.664 INFO:tasks.rados.rados.0.smithi195.stdout:456: done (9 left) 2024-09-13T10:10:51.664 INFO:tasks.rados.rados.0.smithi195.stdout:457: done (8 left) 2024-09-13T10:10:51.664 INFO:tasks.rados.rados.0.smithi195.stdout:458: done (7 left) 2024-09-13T10:10:51.664 INFO:tasks.rados.rados.0.smithi195.stdout:459: done (6 left) 2024-09-13T10:10:51.664 INFO:tasks.rados.rados.0.smithi195.stdout:466: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.664 INFO:tasks.rados.rados.0.smithi195.stdout:466: set_chunk oid 17 offset: 1579008 length: 605696 target oid smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1579008 snap -1 2024-09-13T10:10:51.665 INFO:tasks.rados.rados.0.smithi195.stdout:467: set_chunk oid 18 target oid 18 low_tier 2024-09-13T10:10:51.665 INFO:tasks.rados.rados.0.smithi195.stdout:467: set_chunk oid 18 offset: 402944 length: 973312 target oid smithi19530714-18 low_tier offset: 402944 snap -1 2024-09-13T10:10:51.665 INFO:tasks.rados.rados.0.smithi195.stdout:468: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.665 INFO:tasks.rados.rados.0.smithi195.stdout:468: set_chunk oid 19 offset: 84992 length: 1627136 target oid smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 84992 snap -1 2024-09-13T10:10:51.665 INFO:tasks.rados.rados.0.smithi195.stdout:469: set_chunk oid 20 target oid 20 low_tier 2024-09-13T10:10:51.665 INFO:tasks.rados.rados.0.smithi195.stdout:469: set_chunk oid 20 offset: 211968 length: 1422848 target oid smithi19530714-20 low_tier offset: 211968 snap -1 2024-09-13T10:10:51.665 INFO:tasks.rados.rados.0.smithi195.stdout:470: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.665 INFO:tasks.rados.rados.0.smithi195.stdout:470: set_chunk oid 21 offset: 1129472 length: 845312 target oid smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1129472 snap -1 2024-09-13T10:10:51.665 INFO:tasks.rados.rados.0.smithi195.stdout:471: set_chunk oid 22 target oid 22 low_tier 2024-09-13T10:10:51.665 INFO:tasks.rados.rados.0.smithi195.stdout:471: set_chunk oid 22 offset: 112128 length: 2237952 target oid smithi19530714-22 low_tier offset: 112128 snap -1 2024-09-13T10:10:51.665 INFO:tasks.rados.rados.0.smithi195.stdout:472: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.665 INFO:tasks.rados.rados.0.smithi195.stdout:472: set_chunk oid 23 offset: 1486848 length: 735232 target oid smithi19530714-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1486848 snap -1 2024-09-13T10:10:51.665 INFO:tasks.rados.rados.0.smithi195.stdout:473: set_chunk oid 24 target oid 24 low_tier 2024-09-13T10:10:51.665 INFO:tasks.rados.rados.0.smithi195.stdout:473: set_chunk oid 24 offset: 596480 length: 942592 target oid smithi19530714-24 low_tier offset: 596480 snap -1 2024-09-13T10:10:51.666 INFO:tasks.rados.rados.0.smithi195.stdout:474: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.666 INFO:tasks.rados.rados.0.smithi195.stdout:474: set_chunk oid 25 offset: 137728 length: 164864 target oid smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 137728 snap -1 2024-09-13T10:10:51.666 INFO:tasks.rados.rados.0.smithi195.stdout:475: set_chunk oid 26 target oid 26 low_tier 2024-09-13T10:10:51.666 INFO:tasks.rados.rados.0.smithi195.stdout:475: set_chunk oid 26 offset: 109568 length: 330752 target oid smithi19530714-26 low_tier offset: 109568 snap -1 2024-09-13T10:10:51.666 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.666 INFO:tasks.rados.rados.0.smithi195.stdout:461: finishing set_chunk to oid 12 2024-09-13T10:10:51.666 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 12 set_chunk 12 low_tier returned error code -95 offset: 1296896 length: 486912 2024-09-13T10:10:51.666 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-13T10:10:51.666 INFO:tasks.rados.rados.0.smithi195.stdout:462: finishing set_chunk to oid 13 2024-09-13T10:10:51.666 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 327168 length: 231936 2024-09-13T10:10:51.666 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-13T10:10:51.666 INFO:tasks.rados.rados.0.smithi195.stdout:465: finishing set_chunk to oid 16 2024-09-13T10:10:51.666 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 164864 length: 1204736 2024-09-13T10:10:51.667 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-13T10:10:51.667 INFO:tasks.rados.rados.0.smithi195.stdout:463: finishing set_chunk to oid 14 2024-09-13T10:10:51.667 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 14 set_chunk 14 low_tier returned error code -95 offset: 866816 length: 115200 2024-09-13T10:10:51.667 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 0 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-13T10:10:51.667 INFO:tasks.rados.rados.0.smithi195.stdout:464: finishing set_chunk to oid 15 2024-09-13T10:10:51.667 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 15 set_chunk 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1213440 length: 1003008 2024-09-13T10:10:51.667 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-13T10:10:51.667 INFO:tasks.rados.rados.0.smithi195.stdout:461: done (15 left) 2024-09-13T10:10:51.667 INFO:tasks.rados.rados.0.smithi195.stdout:462: done (14 left) 2024-09-13T10:10:51.667 INFO:tasks.rados.rados.0.smithi195.stdout:463: done (13 left) 2024-09-13T10:10:51.667 INFO:tasks.rados.rados.0.smithi195.stdout:464: done (12 left) 2024-09-13T10:10:51.667 INFO:tasks.rados.rados.0.smithi195.stdout:465: done (11 left) 2024-09-13T10:10:51.667 INFO:tasks.rados.rados.0.smithi195.stdout:476: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.668 INFO:tasks.rados.rados.0.smithi195.stdout:476: set_chunk oid 27 offset: 230400 length: 859648 target oid smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 230400 snap -1 2024-09-13T10:10:51.668 INFO:tasks.rados.rados.0.smithi195.stdout:477: set_chunk oid 28 target oid 28 low_tier 2024-09-13T10:10:51.668 INFO:tasks.rados.rados.0.smithi195.stdout:477: set_chunk oid 28 offset: 1262592 length: 70144 target oid smithi19530714-28 low_tier offset: 1262592 snap -1 2024-09-13T10:10:51.668 INFO:tasks.rados.rados.0.smithi195.stdout:478: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.668 INFO:tasks.rados.rados.0.smithi195.stdout:478: set_chunk oid 29 offset: 307200 length: 62464 target oid smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 307200 snap -1 2024-09-13T10:10:51.668 INFO:tasks.rados.rados.0.smithi195.stdout:479: set_chunk oid 30 target oid 30 low_tier 2024-09-13T10:10:51.668 INFO:tasks.rados.rados.0.smithi195.stdout:479: set_chunk oid 30 offset: 835584 length: 1400320 target oid smithi19530714-30 low_tier offset: 835584 snap -1 2024-09-13T10:10:51.668 INFO:tasks.rados.rados.0.smithi195.stdout:480: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.668 INFO:tasks.rados.rados.0.smithi195.stdout:480: set_chunk oid 31 offset: 757760 length: 111104 target oid smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 757760 snap -1 2024-09-13T10:10:51.668 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.668 INFO:tasks.rados.rados.0.smithi195.stdout:468: finishing set_chunk to oid 19 2024-09-13T10:10:51.669 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 19 set_chunk 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 84992 length: 1627136 2024-09-13T10:10:51.669 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 0 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-13T10:10:51.669 INFO:tasks.rados.rados.0.smithi195.stdout:466: finishing set_chunk to oid 17 2024-09-13T10:10:51.669 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1579008 length: 605696 2024-09-13T10:10:51.669 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-13T10:10:51.669 INFO:tasks.rados.rados.0.smithi195.stdout:466: done (15 left) 2024-09-13T10:10:51.669 INFO:tasks.rados.rados.0.smithi195.stdout:468: done (14 left) 2024-09-13T10:10:51.669 INFO:tasks.rados.rados.0.smithi195.stdout:481: set_chunk oid 32 target oid 32 low_tier 2024-09-13T10:10:51.669 INFO:tasks.rados.rados.0.smithi195.stdout:481: set_chunk oid 32 offset: 319488 length: 1460736 target oid smithi19530714-32 low_tier offset: 319488 snap -1 2024-09-13T10:10:51.669 INFO:tasks.rados.rados.0.smithi195.stdout:482: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.669 INFO:tasks.rados.rados.0.smithi195.stdout:482: set_chunk oid 33 offset: 1677824 length: 117248 target oid smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1677824 snap -1 2024-09-13T10:10:51.669 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.670 INFO:tasks.rados.rados.0.smithi195.stdout:472: finishing set_chunk to oid 23 2024-09-13T10:10:51.670 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 23 set_chunk 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1486848 length: 735232 2024-09-13T10:10:51.670 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-13T10:10:51.670 INFO:tasks.rados.rados.0.smithi195.stdout:472: done (15 left) 2024-09-13T10:10:51.670 INFO:tasks.rados.rados.0.smithi195.stdout:483: set_chunk oid 34 target oid 34 low_tier 2024-09-13T10:10:51.670 INFO:tasks.rados.rados.0.smithi195.stdout:483: set_chunk oid 34 offset: 763904 length: 1560576 target oid smithi19530714-34 low_tier offset: 763904 snap -1 2024-09-13T10:10:51.670 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.670 INFO:tasks.rados.rados.0.smithi195.stdout:470: finishing set_chunk to oid 21 2024-09-13T10:10:51.670 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 21 set_chunk 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1129472 length: 845312 2024-09-13T10:10:51.670 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 0 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-13T10:10:51.670 INFO:tasks.rados.rados.0.smithi195.stdout:473: finishing set_chunk to oid 24 2024-09-13T10:10:51.670 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 24 set_chunk 24 low_tier returned error code -95 offset: 596480 length: 942592 2024-09-13T10:10:51.670 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 0 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-13T10:10:51.671 INFO:tasks.rados.rados.0.smithi195.stdout:475: finishing set_chunk to oid 26 2024-09-13T10:10:51.671 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 26 set_chunk 26 low_tier returned error code -95 offset: 109568 length: 330752 2024-09-13T10:10:51.671 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 0 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-13T10:10:51.671 INFO:tasks.rados.rados.0.smithi195.stdout:470: done (15 left) 2024-09-13T10:10:51.671 INFO:tasks.rados.rados.0.smithi195.stdout:473: done (14 left) 2024-09-13T10:10:51.671 INFO:tasks.rados.rados.0.smithi195.stdout:475: done (13 left) 2024-09-13T10:10:51.671 INFO:tasks.rados.rados.0.smithi195.stdout:484: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.671 INFO:tasks.rados.rados.0.smithi195.stdout:484: set_chunk oid 35 offset: 5120 length: 401920 target oid smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 5120 snap -1 2024-09-13T10:10:51.671 INFO:tasks.rados.rados.0.smithi195.stdout:471: finishing set_chunk to oid 22 2024-09-13T10:10:51.671 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 112128 length: 2237952 2024-09-13T10:10:51.671 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-13T10:10:51.671 INFO:tasks.rados.rados.0.smithi195.stdout:478: finishing set_chunk to oid 29 2024-09-13T10:10:51.671 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 29 set_chunk 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 307200 length: 62464 2024-09-13T10:10:51.671 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 0 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-13T10:10:51.671 INFO:tasks.rados.rados.0.smithi195.stdout:471: done (13 left) 2024-09-13T10:10:51.672 INFO:tasks.rados.rados.0.smithi195.stdout:478: done (12 left) 2024-09-13T10:10:51.672 INFO:tasks.rados.rados.0.smithi195.stdout:485: set_chunk oid 36 target oid 36 low_tier 2024-09-13T10:10:51.672 INFO:tasks.rados.rados.0.smithi195.stdout:485: set_chunk oid 36 offset: 84480 length: 1993728 target oid smithi19530714-36 low_tier offset: 84480 snap -1 2024-09-13T10:10:51.672 INFO:tasks.rados.rados.0.smithi195.stdout:486: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.672 INFO:tasks.rados.rados.0.smithi195.stdout:486: set_chunk oid 37 offset: 471552 length: 347648 target oid smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 471552 snap -1 2024-09-13T10:10:51.672 INFO:tasks.rados.rados.0.smithi195.stdout:487: set_chunk oid 38 target oid 38 low_tier 2024-09-13T10:10:51.672 INFO:tasks.rados.rados.0.smithi195.stdout:474: finishing set_chunk to oid 25 2024-09-13T10:10:51.672 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 25 set_chunk 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 137728 length: 164864 2024-09-13T10:10:51.672 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-13T10:10:51.672 INFO:tasks.rados.rados.0.smithi195.stdout:487: set_chunk oid 38 offset: 56832 length: 2095616 target oid smithi19530714-38 low_tier offset: 56832 snap -1 2024-09-13T10:10:51.672 INFO:tasks.rados.rados.0.smithi195.stdout:474: done (14 left) 2024-09-13T10:10:51.672 INFO:tasks.rados.rados.0.smithi195.stdout:488: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.672 INFO:tasks.rados.rados.0.smithi195.stdout:488: set_chunk oid 39 offset: 1660416 length: 726016 target oid smithi19530714-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1660416 snap -1 2024-09-13T10:10:51.673 INFO:tasks.rados.rados.0.smithi195.stdout:489: set_chunk oid 40 target oid 40 low_tier 2024-09-13T10:10:51.673 INFO:tasks.rados.rados.0.smithi195.stdout:489: set_chunk oid 40 offset: 635392 length: 986112 target oid smithi19530714-40 low_tier offset: 635392 snap -1 2024-09-13T10:10:51.673 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.673 INFO:tasks.rados.rados.0.smithi195.stdout:469: finishing set_chunk to oid 20 2024-09-13T10:10:51.673 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 211968 length: 1422848 2024-09-13T10:10:51.673 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-13T10:10:51.673 INFO:tasks.rados.rados.0.smithi195.stdout:469: done (15 left) 2024-09-13T10:10:51.673 INFO:tasks.rados.rados.0.smithi195.stdout:490: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.673 INFO:tasks.rados.rados.0.smithi195.stdout:490: set_chunk oid 41 offset: 507392 length: 1451008 target oid smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 507392 snap -1 2024-09-13T10:10:51.673 INFO:tasks.rados.rados.0.smithi195.stdout:482: finishing set_chunk to oid 33 2024-09-13T10:10:51.673 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 33 set_chunk 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1677824 length: 117248 2024-09-13T10:10:51.673 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 0 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-13T10:10:51.673 INFO:tasks.rados.rados.0.smithi195.stdout:483: finishing set_chunk to oid 34 2024-09-13T10:10:51.674 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 34 set_chunk 34 low_tier returned error code -95 offset: 763904 length: 1560576 2024-09-13T10:10:51.674 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 34 v 0 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-13T10:10:51.674 INFO:tasks.rados.rados.0.smithi195.stdout:467: finishing set_chunk to oid 18 2024-09-13T10:10:51.674 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 402944 length: 973312 2024-09-13T10:10:51.674 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-13T10:10:51.674 INFO:tasks.rados.rados.0.smithi195.stdout:467: done (15 left) 2024-09-13T10:10:51.674 INFO:tasks.rados.rados.0.smithi195.stdout:482: done (14 left) 2024-09-13T10:10:51.674 INFO:tasks.rados.rados.0.smithi195.stdout:483: done (13 left) 2024-09-13T10:10:51.674 INFO:tasks.rados.rados.0.smithi195.stdout:491: set_chunk oid 42 target oid 42 low_tier 2024-09-13T10:10:51.674 INFO:tasks.rados.rados.0.smithi195.stdout:491: set_chunk oid 42 offset: 667136 length: 1382912 target oid smithi19530714-42 low_tier offset: 667136 snap -1 2024-09-13T10:10:51.674 INFO:tasks.rados.rados.0.smithi195.stdout:492: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.674 INFO:tasks.rados.rados.0.smithi195.stdout:492: set_chunk oid 43 offset: 1358336 length: 232960 target oid smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1358336 snap -1 2024-09-13T10:10:51.674 INFO:tasks.rados.rados.0.smithi195.stdout:493: set_chunk oid 44 target oid 44 low_tier 2024-09-13T10:10:51.674 INFO:tasks.rados.rados.0.smithi195.stdout:493: set_chunk oid 44 offset: 412672 length: 305664 target oid smithi19530714-44 low_tier offset: 412672 snap -1 2024-09-13T10:10:51.675 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.675 INFO:tasks.rados.rados.0.smithi195.stdout:476: finishing set_chunk to oid 27 2024-09-13T10:10:51.675 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 27 set_chunk 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 230400 length: 859648 2024-09-13T10:10:51.675 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-13T10:10:51.675 INFO:tasks.rados.rados.0.smithi195.stdout:477: finishing set_chunk to oid 28 2024-09-13T10:10:51.675 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 28 set_chunk 28 low_tier returned error code -95 offset: 1262592 length: 70144 2024-09-13T10:10:51.675 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 0 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-13T10:10:51.675 INFO:tasks.rados.rados.0.smithi195.stdout:485: finishing set_chunk to oid 36 2024-09-13T10:10:51.675 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 36 set_chunk 36 low_tier returned error code -95 offset: 84480 length: 1993728 2024-09-13T10:10:51.675 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 0 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-13T10:10:51.675 INFO:tasks.rados.rados.0.smithi195.stdout:479: finishing set_chunk to oid 30 2024-09-13T10:10:51.675 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 30 set_chunk 30 low_tier returned error code -95 offset: 835584 length: 1400320 2024-09-13T10:10:51.675 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 0 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-13T10:10:51.675 INFO:tasks.rados.rados.0.smithi195.stdout:476: done (15 left) 2024-09-13T10:10:51.676 INFO:tasks.rados.rados.0.smithi195.stdout:477: done (14 left) 2024-09-13T10:10:51.676 INFO:tasks.rados.rados.0.smithi195.stdout:479: done (13 left) 2024-09-13T10:10:51.676 INFO:tasks.rados.rados.0.smithi195.stdout:485: done (12 left) 2024-09-13T10:10:51.676 INFO:tasks.rados.rados.0.smithi195.stdout:494: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.676 INFO:tasks.rados.rados.0.smithi195.stdout:494: set_chunk oid 45 offset: 1272832 length: 519680 target oid smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1272832 snap -1 2024-09-13T10:10:51.676 INFO:tasks.rados.rados.0.smithi195.stdout:495: set_chunk oid 46 target oid 46 low_tier 2024-09-13T10:10:51.676 INFO:tasks.rados.rados.0.smithi195.stdout:495: set_chunk oid 46 offset: 975360 length: 300032 target oid smithi19530714-46 low_tier offset: 975360 snap -1 2024-09-13T10:10:51.676 INFO:tasks.rados.rados.0.smithi195.stdout:496: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.676 INFO:tasks.rados.rados.0.smithi195.stdout:496: set_chunk oid 47 offset: 1048064 length: 669696 target oid smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1048064 snap -1 2024-09-13T10:10:51.676 INFO:tasks.rados.rados.0.smithi195.stdout:497: set_chunk oid 48 target oid 48 low_tier 2024-09-13T10:10:51.676 INFO:tasks.rados.rados.0.smithi195.stdout:497: set_chunk oid 48 offset: 275968 length: 1064448 target oid smithi19530714-48 low_tier offset: 275968 snap -1 2024-09-13T10:10:51.676 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.676 INFO:tasks.rados.rados.0.smithi195.stdout:490: finishing set_chunk to oid 41 2024-09-13T10:10:51.677 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 41 set_chunk 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 507392 length: 1451008 2024-09-13T10:10:51.677 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-13T10:10:51.677 INFO:tasks.rados.rados.0.smithi195.stdout:488: finishing set_chunk to oid 39 2024-09-13T10:10:51.677 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1660416 length: 726016 2024-09-13T10:10:51.677 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-13T10:10:51.677 INFO:tasks.rados.rados.0.smithi195.stdout:481: finishing set_chunk to oid 32 2024-09-13T10:10:51.677 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 319488 length: 1460736 2024-09-13T10:10:51.677 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-13T10:10:51.677 INFO:tasks.rados.rados.0.smithi195.stdout:489: finishing set_chunk to oid 40 2024-09-13T10:10:51.677 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 40 set_chunk 40 low_tier returned error code -95 offset: 635392 length: 986112 2024-09-13T10:10:51.677 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 0 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-13T10:10:51.677 INFO:tasks.rados.rados.0.smithi195.stdout:481: done (15 left) 2024-09-13T10:10:51.678 INFO:tasks.rados.rados.0.smithi195.stdout:488: done (14 left) 2024-09-13T10:10:51.678 INFO:tasks.rados.rados.0.smithi195.stdout:489: done (13 left) 2024-09-13T10:10:51.678 INFO:tasks.rados.rados.0.smithi195.stdout:490: done (12 left) 2024-09-13T10:10:51.678 INFO:tasks.rados.rados.0.smithi195.stdout:498: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.678 INFO:tasks.rados.rados.0.smithi195.stdout:498: set_chunk oid 49 offset: 1739776 length: 559616 target oid smithi19530714-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1739776 snap -1 2024-09-13T10:10:51.678 INFO:tasks.rados.rados.0.smithi195.stdout:499: set_chunk oid 50 target oid 50 low_tier 2024-09-13T10:10:51.678 INFO:tasks.rados.rados.0.smithi195.stdout:499: set_chunk oid 50 offset: 54784 length: 678912 target oid smithi19530714-50 low_tier offset: 54784 snap -1 2024-09-13T10:10:51.678 INFO:tasks.rados.rados.0.smithi195.stdout:500: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.678 INFO:tasks.rados.rados.0.smithi195.stdout:500: set_chunk oid 1 offset: 543744 length: 746496 target oid smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 543744 snap -1 2024-09-13T10:10:51.678 INFO:tasks.rados.rados.0.smithi195.stdout:501: set_chunk oid 2 target oid 2 low_tier 2024-09-13T10:10:51.678 INFO:tasks.rados.rados.0.smithi195.stdout:501: set_chunk oid 2 offset: 378368 length: 116736 target oid smithi19530714-2 low_tier offset: 378368 snap -1 2024-09-13T10:10:51.678 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.678 INFO:tasks.rados.rados.0.smithi195.stdout:491: finishing set_chunk to oid 42 2024-09-13T10:10:51.679 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 42 set_chunk 42 low_tier returned error code -95 offset: 667136 length: 1382912 2024-09-13T10:10:51.679 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-13T10:10:51.679 INFO:tasks.rados.rados.0.smithi195.stdout:492: finishing set_chunk to oid 43 2024-09-13T10:10:51.679 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1358336 length: 232960 2024-09-13T10:10:51.679 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-13T10:10:51.679 INFO:tasks.rados.rados.0.smithi195.stdout:491: done (15 left) 2024-09-13T10:10:51.679 INFO:tasks.rados.rados.0.smithi195.stdout:492: done (14 left) 2024-09-13T10:10:51.679 INFO:tasks.rados.rados.0.smithi195.stdout:502: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.679 INFO:tasks.rados.rados.0.smithi195.stdout:502: set_chunk oid 3 offset: 978944 length: 914432 target oid smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 978944 snap -1 2024-09-13T10:10:51.679 INFO:tasks.rados.rados.0.smithi195.stdout:503: set_chunk oid 4 target oid 4 low_tier 2024-09-13T10:10:51.679 INFO:tasks.rados.rados.0.smithi195.stdout:503: set_chunk oid 4 offset: 328192 length: 1309184 target oid smithi19530714-4 low_tier offset: 328192 snap -1 2024-09-13T10:10:51.679 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.679 INFO:tasks.rados.rados.0.smithi195.stdout:497: finishing set_chunk to oid 48 2024-09-13T10:10:51.680 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 48 set_chunk 48 low_tier returned error code -95 offset: 275968 length: 1064448 2024-09-13T10:10:51.680 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-13T10:10:51.680 INFO:tasks.rados.rados.0.smithi195.stdout:496: finishing set_chunk to oid 47 2024-09-13T10:10:51.680 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 47 set_chunk 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1048064 length: 669696 2024-09-13T10:10:51.680 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-13T10:10:51.680 INFO:tasks.rados.rados.0.smithi195.stdout:487: finishing set_chunk to oid 38 2024-09-13T10:10:51.680 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 38 set_chunk 38 low_tier returned error code -95 offset: 56832 length: 2095616 2024-09-13T10:10:51.680 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 0 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-13T10:10:51.680 INFO:tasks.rados.rados.0.smithi195.stdout:499: finishing set_chunk to oid 50 2024-09-13T10:10:51.680 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 50 set_chunk 50 low_tier returned error code -95 offset: 54784 length: 678912 2024-09-13T10:10:51.680 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 0 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-13T10:10:51.680 INFO:tasks.rados.rados.0.smithi195.stdout:500: finishing set_chunk to oid 1 2024-09-13T10:10:51.680 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 1 set_chunk 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 543744 length: 746496 2024-09-13T10:10:51.680 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 0 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-13T10:10:51.681 INFO:tasks.rados.rados.0.smithi195.stdout:494: finishing set_chunk to oid 45 2024-09-13T10:10:51.681 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 45 set_chunk 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1272832 length: 519680 2024-09-13T10:10:51.681 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 0 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-13T10:10:51.681 INFO:tasks.rados.rados.0.smithi195.stdout:503: finishing set_chunk to oid 4 2024-09-13T10:10:51.681 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 4 set_chunk 4 low_tier returned error code -95 offset: 328192 length: 1309184 2024-09-13T10:10:51.681 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 0 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-13T10:10:51.681 INFO:tasks.rados.rados.0.smithi195.stdout:502: finishing set_chunk to oid 3 2024-09-13T10:10:51.681 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 3 set_chunk 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 978944 length: 914432 2024-09-13T10:10:51.681 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 0 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-13T10:10:51.681 INFO:tasks.rados.rados.0.smithi195.stdout:487: done (15 left) 2024-09-13T10:10:51.681 INFO:tasks.rados.rados.0.smithi195.stdout:494: done (14 left) 2024-09-13T10:10:51.681 INFO:tasks.rados.rados.0.smithi195.stdout:496: done (13 left) 2024-09-13T10:10:51.681 INFO:tasks.rados.rados.0.smithi195.stdout:497: done (12 left) 2024-09-13T10:10:51.681 INFO:tasks.rados.rados.0.smithi195.stdout:499: done (11 left) 2024-09-13T10:10:51.681 INFO:tasks.rados.rados.0.smithi195.stdout:500: done (10 left) 2024-09-13T10:10:51.682 INFO:tasks.rados.rados.0.smithi195.stdout:502: done (9 left) 2024-09-13T10:10:51.682 INFO:tasks.rados.rados.0.smithi195.stdout:503: done (8 left) 2024-09-13T10:10:51.682 INFO:tasks.rados.rados.0.smithi195.stdout:504: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.682 INFO:tasks.rados.rados.0.smithi195.stdout:504: set_chunk oid 5 offset: 1108480 length: 7168 target oid smithi19530714-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1108480 snap -1 2024-09-13T10:10:51.682 INFO:tasks.rados.rados.0.smithi195.stdout:505: set_chunk oid 6 target oid 6 low_tier 2024-09-13T10:10:51.682 INFO:tasks.rados.rados.0.smithi195.stdout:505: set_chunk oid 6 offset: 83968 length: 1974272 target oid smithi19530714-6 low_tier offset: 83968 snap -1 2024-09-13T10:10:51.682 INFO:tasks.rados.rados.0.smithi195.stdout:506: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.682 INFO:tasks.rados.rados.0.smithi195.stdout:506: set_chunk oid 7 offset: 673792 length: 1273856 target oid smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 673792 snap -1 2024-09-13T10:10:51.682 INFO:tasks.rados.rados.0.smithi195.stdout:498: finishing set_chunk to oid 49 2024-09-13T10:10:51.682 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 49 set_chunk 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1739776 length: 559616 2024-09-13T10:10:51.682 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 0 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-13T10:10:51.682 INFO:tasks.rados.rados.0.smithi195.stdout:498: done (10 left) 2024-09-13T10:10:51.683 INFO:tasks.rados.rados.0.smithi195.stdout:507: set_chunk oid 8 target oid 8 low_tier 2024-09-13T10:10:51.683 INFO:tasks.rados.rados.0.smithi195.stdout:507: set_chunk oid 8 offset: 1075200 length: 70656 target oid smithi19530714-8 low_tier offset: 1075200 snap -1 2024-09-13T10:10:51.683 INFO:tasks.rados.rados.0.smithi195.stdout:508: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.683 INFO:tasks.rados.rados.0.smithi195.stdout:508: set_chunk oid 9 offset: 896512 length: 994304 target oid smithi19530714-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 896512 snap -1 2024-09-13T10:10:51.683 INFO:tasks.rados.rados.0.smithi195.stdout:509: set_chunk oid 10 target oid 10 low_tier 2024-09-13T10:10:51.683 INFO:tasks.rados.rados.0.smithi195.stdout:509: set_chunk oid 10 offset: 741888 length: 321536 target oid smithi19530714-10 low_tier offset: 741888 snap -1 2024-09-13T10:10:51.683 INFO:tasks.rados.rados.0.smithi195.stdout:509 retry set_chunk ! 2024-09-13T10:10:51.683 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 13 2024-09-13T10:10:51.683 INFO:tasks.rados.rados.0.smithi195.stdout:506: finishing set_chunk to oid 7 2024-09-13T10:10:51.683 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 673792 length: 1273856 2024-09-13T10:10:51.683 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-13T10:10:51.683 INFO:tasks.rados.rados.0.smithi195.stdout:505: finishing set_chunk to oid 6 2024-09-13T10:10:51.684 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 6 set_chunk 6 low_tier returned error code -95 offset: 83968 length: 1974272 2024-09-13T10:10:51.684 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 0 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-13T10:10:51.684 INFO:tasks.rados.rados.0.smithi195.stdout:505: done (12 left) 2024-09-13T10:10:51.684 INFO:tasks.rados.rados.0.smithi195.stdout:506: done (11 left) 2024-09-13T10:10:51.684 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 11 2024-09-13T10:10:51.684 INFO:tasks.rados.rados.0.smithi195.stdout:507: finishing set_chunk to oid 8 2024-09-13T10:10:51.684 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 8 set_chunk 8 low_tier returned error code -95 offset: 1075200 length: 70656 2024-09-13T10:10:51.684 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-13T10:10:51.684 INFO:tasks.rados.rados.0.smithi195.stdout:507: done (10 left) 2024-09-13T10:10:51.684 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 10 2024-09-13T10:10:51.684 INFO:tasks.rados.rados.0.smithi195.stdout:504: finishing set_chunk to oid 5 2024-09-13T10:10:51.684 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 5 set_chunk 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1108480 length: 7168 2024-09-13T10:10:51.684 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 0 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-13T10:10:51.685 INFO:tasks.rados.rados.0.smithi195.stdout:504: done (9 left) 2024-09-13T10:10:51.685 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 9 2024-09-13T10:10:51.685 INFO:tasks.rados.rados.0.smithi195.stdout:508: finishing set_chunk to oid 9 2024-09-13T10:10:51.685 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 9 set_chunk 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 896512 length: 994304 2024-09-13T10:10:51.685 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 0 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-13T10:10:51.685 INFO:tasks.rados.rados.0.smithi195.stdout:508: done (8 left) 2024-09-13T10:10:51.685 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 8 2024-09-13T10:10:51.685 INFO:tasks.rados.rados.0.smithi195.stdout:509: finishing set_chunk to oid 10 2024-09-13T10:10:51.685 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 10 set_chunk 10 low_tier returned error code -95 offset: 741888 length: 321536 2024-09-13T10:10:51.685 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 0 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-13T10:10:51.685 INFO:tasks.rados.rados.0.smithi195.stdout:509: done (7 left) 2024-09-13T10:10:51.685 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 7 2024-09-13T10:10:51.685 INFO:tasks.rados.rados.0.smithi195.stdout:460: finishing set_chunk to oid 11 2024-09-13T10:10:51.686 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 3 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-13T10:10:51.686 INFO:tasks.rados.rados.0.smithi195.stdout:460: done (6 left) 2024-09-13T10:10:51.686 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 6 2024-09-13T10:10:51.686 INFO:tasks.rados.rados.0.smithi195.stdout:480: finishing set_chunk to oid 31 2024-09-13T10:10:51.686 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 6 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-13T10:10:51.686 INFO:tasks.rados.rados.0.smithi195.stdout:480: done (5 left) 2024-09-13T10:10:51.686 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 5 2024-09-13T10:10:51.686 INFO:tasks.rados.rados.0.smithi195.stdout:501: finishing set_chunk to oid 2 2024-09-13T10:10:51.686 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-13T10:10:51.686 INFO:tasks.rados.rados.0.smithi195.stdout:493: finishing set_chunk to oid 44 2024-09-13T10:10:51.686 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 9 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-13T10:10:51.686 INFO:tasks.rados.rados.0.smithi195.stdout:493: done (4 left) 2024-09-13T10:10:51.686 INFO:tasks.rados.rados.0.smithi195.stdout:501: done (3 left) 2024-09-13T10:10:51.687 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 3 2024-09-13T10:10:51.687 INFO:tasks.rados.rados.0.smithi195.stdout:495: finishing set_chunk to oid 46 2024-09-13T10:10:51.687 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 6 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-13T10:10:51.687 INFO:tasks.rados.rados.0.smithi195.stdout:495: done (2 left) 2024-09-13T10:10:51.687 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 2 2024-09-13T10:10:51.687 INFO:tasks.rados.rados.0.smithi195.stdout:484: finishing set_chunk to oid 35 2024-09-13T10:10:51.687 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 12 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-13T10:10:51.687 INFO:tasks.rados.rados.0.smithi195.stdout:486: finishing set_chunk to oid 37 2024-09-13T10:10:51.687 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 3 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-13T10:10:51.687 INFO:tasks.rados.rados.0.smithi195.stdout:484: done (1 left) 2024-09-13T10:10:51.687 INFO:tasks.rados.rados.0.smithi195.stdout:486: done (0 left) 2024-09-13T10:10:51.687 INFO:tasks.rados.rados.0.smithi195.stdout:510: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.687 INFO:tasks.rados.rados.0.smithi195.stdout:510: set_chunk oid 11 offset: 1615360 length: 365568 target oid smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1615360 snap -1 2024-09-13T10:10:51.687 INFO:tasks.rados.rados.0.smithi195.stdout:511: set_chunk oid 12 target oid 12 low_tier 2024-09-13T10:10:51.688 INFO:tasks.rados.rados.0.smithi195.stdout:511: set_chunk oid 12 offset: 613376 length: 902144 target oid smithi19530714-12 low_tier offset: 613376 snap -1 2024-09-13T10:10:51.688 INFO:tasks.rados.rados.0.smithi195.stdout:512: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.688 INFO:tasks.rados.rados.0.smithi195.stdout:512: set_chunk oid 13 offset: 462336 length: 843264 target oid smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 462336 snap -1 2024-09-13T10:10:51.688 INFO:tasks.rados.rados.0.smithi195.stdout:513: set_chunk oid 14 target oid 14 low_tier 2024-09-13T10:10:51.688 INFO:tasks.rados.rados.0.smithi195.stdout:513: set_chunk oid 14 offset: 221696 length: 1081344 target oid smithi19530714-14 low_tier offset: 221696 snap -1 2024-09-13T10:10:51.688 INFO:tasks.rados.rados.0.smithi195.stdout:514: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.688 INFO:tasks.rados.rados.0.smithi195.stdout:514: set_chunk oid 15 offset: 438272 length: 1362944 target oid smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 438272 snap -1 2024-09-13T10:10:51.688 INFO:tasks.rados.rados.0.smithi195.stdout:515: set_chunk oid 16 target oid 16 low_tier 2024-09-13T10:10:51.688 INFO:tasks.rados.rados.0.smithi195.stdout:515: set_chunk oid 16 offset: 69120 length: 1032192 target oid smithi19530714-16 low_tier offset: 69120 snap -1 2024-09-13T10:10:51.688 INFO:tasks.rados.rados.0.smithi195.stdout:516: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.688 INFO:tasks.rados.rados.0.smithi195.stdout:516: set_chunk oid 17 offset: 459776 length: 1601024 target oid smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 459776 snap -1 2024-09-13T10:10:51.688 INFO:tasks.rados.rados.0.smithi195.stdout:517: set_chunk oid 18 target oid 18 low_tier 2024-09-13T10:10:51.689 INFO:tasks.rados.rados.0.smithi195.stdout:517: set_chunk oid 18 offset: 1148928 length: 871936 target oid smithi19530714-18 low_tier offset: 1148928 snap -1 2024-09-13T10:10:51.689 INFO:tasks.rados.rados.0.smithi195.stdout:518: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.689 INFO:tasks.rados.rados.0.smithi195.stdout:518: set_chunk oid 19 offset: 13824 length: 2179584 target oid smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 13824 snap -1 2024-09-13T10:10:51.689 INFO:tasks.rados.rados.0.smithi195.stdout:519: set_chunk oid 20 target oid 20 low_tier 2024-09-13T10:10:51.689 INFO:tasks.rados.rados.0.smithi195.stdout:519: set_chunk oid 20 offset: 1228800 length: 67072 target oid smithi19530714-20 low_tier offset: 1228800 snap -1 2024-09-13T10:10:51.689 INFO:tasks.rados.rados.0.smithi195.stdout:520: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.689 INFO:tasks.rados.rados.0.smithi195.stdout:520: set_chunk oid 21 offset: 199680 length: 433152 target oid smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 199680 snap -1 2024-09-13T10:10:51.689 INFO:tasks.rados.rados.0.smithi195.stdout:521: set_chunk oid 22 target oid 22 low_tier 2024-09-13T10:10:51.689 INFO:tasks.rados.rados.0.smithi195.stdout:521: set_chunk oid 22 offset: 1113088 length: 608256 target oid smithi19530714-22 low_tier offset: 1113088 snap -1 2024-09-13T10:10:51.689 INFO:tasks.rados.rados.0.smithi195.stdout:522: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.689 INFO:tasks.rados.rados.0.smithi195.stdout:522: set_chunk oid 23 offset: 1366528 length: 618496 target oid smithi19530714-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1366528 snap -1 2024-09-13T10:10:51.689 INFO:tasks.rados.rados.0.smithi195.stdout:523: set_chunk oid 24 target oid 24 low_tier 2024-09-13T10:10:51.689 INFO:tasks.rados.rados.0.smithi195.stdout:523: set_chunk oid 24 offset: 868352 length: 712704 target oid smithi19530714-24 low_tier offset: 868352 snap -1 2024-09-13T10:10:51.689 INFO:tasks.rados.rados.0.smithi195.stdout:524: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.690 INFO:tasks.rados.rados.0.smithi195.stdout:524: set_chunk oid 25 offset: 598528 length: 1768960 target oid smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 598528 snap -1 2024-09-13T10:10:51.690 INFO:tasks.rados.rados.0.smithi195.stdout:525: set_chunk oid 26 target oid 26 low_tier 2024-09-13T10:10:51.690 INFO:tasks.rados.rados.0.smithi195.stdout:525: set_chunk oid 26 offset: 1046528 length: 286208 target oid smithi19530714-26 low_tier offset: 1046528 snap -1 2024-09-13T10:10:51.690 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.690 INFO:tasks.rados.rados.0.smithi195.stdout:510: finishing set_chunk to oid 11 2024-09-13T10:10:51.690 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 11 set_chunk 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1615360 length: 365568 2024-09-13T10:10:51.690 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 0 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-13T10:10:51.690 INFO:tasks.rados.rados.0.smithi195.stdout:512: finishing set_chunk to oid 13 2024-09-13T10:10:51.690 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 462336 length: 843264 2024-09-13T10:10:51.690 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-13T10:10:51.690 INFO:tasks.rados.rados.0.smithi195.stdout:515: finishing set_chunk to oid 16 2024-09-13T10:10:51.690 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 69120 length: 1032192 2024-09-13T10:10:51.690 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-13T10:10:51.691 INFO:tasks.rados.rados.0.smithi195.stdout:516: finishing set_chunk to oid 17 2024-09-13T10:10:51.691 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 459776 length: 1601024 2024-09-13T10:10:51.691 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-13T10:10:51.691 INFO:tasks.rados.rados.0.smithi195.stdout:518: finishing set_chunk to oid 19 2024-09-13T10:10:51.691 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 19 set_chunk 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 13824 length: 2179584 2024-09-13T10:10:51.691 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 0 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-13T10:10:51.691 INFO:tasks.rados.rados.0.smithi195.stdout:519: finishing set_chunk to oid 20 2024-09-13T10:10:51.691 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 1228800 length: 67072 2024-09-13T10:10:51.691 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-13T10:10:51.691 INFO:tasks.rados.rados.0.smithi195.stdout:511: finishing set_chunk to oid 12 2024-09-13T10:10:51.691 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 12 set_chunk 12 low_tier returned error code -95 offset: 613376 length: 902144 2024-09-13T10:10:51.691 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-13T10:10:51.691 INFO:tasks.rados.rados.0.smithi195.stdout:522: finishing set_chunk to oid 23 2024-09-13T10:10:51.692 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 23 set_chunk 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1366528 length: 618496 2024-09-13T10:10:51.692 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-13T10:10:51.692 INFO:tasks.rados.rados.0.smithi195.stdout:517: finishing set_chunk to oid 18 2024-09-13T10:10:51.692 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 1148928 length: 871936 2024-09-13T10:10:51.692 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-13T10:10:51.692 INFO:tasks.rados.rados.0.smithi195.stdout:520: finishing set_chunk to oid 21 2024-09-13T10:10:51.692 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 21 set_chunk 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 199680 length: 433152 2024-09-13T10:10:51.692 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 0 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-13T10:10:51.692 INFO:tasks.rados.rados.0.smithi195.stdout:523: finishing set_chunk to oid 24 2024-09-13T10:10:51.692 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 24 set_chunk 24 low_tier returned error code -95 offset: 868352 length: 712704 2024-09-13T10:10:51.692 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 0 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-13T10:10:51.692 INFO:tasks.rados.rados.0.smithi195.stdout:510: done (15 left) 2024-09-13T10:10:51.692 INFO:tasks.rados.rados.0.smithi195.stdout:511: done (14 left) 2024-09-13T10:10:51.693 INFO:tasks.rados.rados.0.smithi195.stdout:512: done (13 left) 2024-09-13T10:10:51.693 INFO:tasks.rados.rados.0.smithi195.stdout:515: done (12 left) 2024-09-13T10:10:51.693 INFO:tasks.rados.rados.0.smithi195.stdout:516: done (11 left) 2024-09-13T10:10:51.693 INFO:tasks.rados.rados.0.smithi195.stdout:517: done (10 left) 2024-09-13T10:10:51.693 INFO:tasks.rados.rados.0.smithi195.stdout:518: done (9 left) 2024-09-13T10:10:51.693 INFO:tasks.rados.rados.0.smithi195.stdout:519: done (8 left) 2024-09-13T10:10:51.693 INFO:tasks.rados.rados.0.smithi195.stdout:520: done (7 left) 2024-09-13T10:10:51.693 INFO:tasks.rados.rados.0.smithi195.stdout:522: done (6 left) 2024-09-13T10:10:51.693 INFO:tasks.rados.rados.0.smithi195.stdout:523: done (5 left) 2024-09-13T10:10:51.693 INFO:tasks.rados.rados.0.smithi195.stdout:526: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.693 INFO:tasks.rados.rados.0.smithi195.stdout:526: set_chunk oid 27 offset: 374272 length: 1311744 target oid smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 374272 snap -1 2024-09-13T10:10:51.693 INFO:tasks.rados.rados.0.smithi195.stdout:527: set_chunk oid 28 target oid 28 low_tier 2024-09-13T10:10:51.693 INFO:tasks.rados.rados.0.smithi195.stdout:527: set_chunk oid 28 offset: 336384 length: 1190912 target oid smithi19530714-28 low_tier offset: 336384 snap -1 2024-09-13T10:10:51.694 INFO:tasks.rados.rados.0.smithi195.stdout:528: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.694 INFO:tasks.rados.rados.0.smithi195.stdout:528: set_chunk oid 29 offset: 180736 length: 348672 target oid smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 180736 snap -1 2024-09-13T10:10:51.694 INFO:tasks.rados.rados.0.smithi195.stdout:529: set_chunk oid 30 target oid 30 low_tier 2024-09-13T10:10:51.694 INFO:tasks.rados.rados.0.smithi195.stdout:529: set_chunk oid 30 offset: 807936 length: 563712 target oid smithi19530714-30 low_tier offset: 807936 snap -1 2024-09-13T10:10:51.694 INFO:tasks.rados.rados.0.smithi195.stdout:530: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.694 INFO:tasks.rados.rados.0.smithi195.stdout:530: set_chunk oid 31 offset: 833536 length: 4608 target oid smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 833536 snap -1 2024-09-13T10:10:51.694 INFO:tasks.rados.rados.0.smithi195.stdout:531: set_chunk oid 32 target oid 32 low_tier 2024-09-13T10:10:51.694 INFO:tasks.rados.rados.0.smithi195.stdout:531: set_chunk oid 32 offset: 651776 length: 1061888 target oid smithi19530714-32 low_tier offset: 651776 snap -1 2024-09-13T10:10:51.694 INFO:tasks.rados.rados.0.smithi195.stdout:532: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.694 INFO:tasks.rados.rados.0.smithi195.stdout:532: set_chunk oid 33 offset: 352256 length: 2146304 target oid smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 352256 snap -1 2024-09-13T10:10:51.694 INFO:tasks.rados.rados.0.smithi195.stdout:533: set_chunk oid 34 target oid 34 low_tier 2024-09-13T10:10:51.694 INFO:tasks.rados.rados.0.smithi195.stdout:533: set_chunk oid 34 offset: 347136 length: 1634816 target oid smithi19530714-34 low_tier offset: 347136 snap -1 2024-09-13T10:10:51.694 INFO:tasks.rados.rados.0.smithi195.stdout:534: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.694 INFO:tasks.rados.rados.0.smithi195.stdout:534: set_chunk oid 35 offset: 36864 length: 560128 target oid smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 36864 snap -1 2024-09-13T10:10:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:535: set_chunk oid 36 target oid 36 low_tier 2024-09-13T10:10:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:535: set_chunk oid 36 offset: 1856000 length: 125440 target oid smithi19530714-36 low_tier offset: 1856000 snap -1 2024-09-13T10:10:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:536: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:536: set_chunk oid 37 offset: 1460736 length: 369664 target oid smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1460736 snap -1 2024-09-13T10:10:51.695 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:525: finishing set_chunk to oid 26 2024-09-13T10:10:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 26 set_chunk 26 low_tier returned error code -95 offset: 1046528 length: 286208 2024-09-13T10:10:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 0 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-13T10:10:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:513: finishing set_chunk to oid 14 2024-09-13T10:10:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 14 set_chunk 14 low_tier returned error code -95 offset: 221696 length: 1081344 2024-09-13T10:10:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 0 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-13T10:10:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:521: finishing set_chunk to oid 22 2024-09-13T10:10:51.696 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 1113088 length: 608256 2024-09-13T10:10:51.696 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-13T10:10:51.696 INFO:tasks.rados.rados.0.smithi195.stdout:524: finishing set_chunk to oid 25 2024-09-13T10:10:51.696 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 25 set_chunk 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 598528 length: 1768960 2024-09-13T10:10:51.696 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-13T10:10:51.696 INFO:tasks.rados.rados.0.smithi195.stdout:514: finishing set_chunk to oid 15 2024-09-13T10:10:51.696 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 15 set_chunk 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 438272 length: 1362944 2024-09-13T10:10:51.696 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-13T10:10:51.696 INFO:tasks.rados.rados.0.smithi195.stdout:528: finishing set_chunk to oid 29 2024-09-13T10:10:51.696 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 29 set_chunk 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 180736 length: 348672 2024-09-13T10:10:51.696 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 0 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-13T10:10:51.697 INFO:tasks.rados.rados.0.smithi195.stdout:527: finishing set_chunk to oid 28 2024-09-13T10:10:51.697 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 28 set_chunk 28 low_tier returned error code -95 offset: 336384 length: 1190912 2024-09-13T10:10:51.697 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 0 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-13T10:10:51.697 INFO:tasks.rados.rados.0.smithi195.stdout:526: finishing set_chunk to oid 27 2024-09-13T10:10:51.697 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 27 set_chunk 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 374272 length: 1311744 2024-09-13T10:10:51.697 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-13T10:10:51.697 INFO:tasks.rados.rados.0.smithi195.stdout:513: done (15 left) 2024-09-13T10:10:51.697 INFO:tasks.rados.rados.0.smithi195.stdout:514: done (14 left) 2024-09-13T10:10:51.697 INFO:tasks.rados.rados.0.smithi195.stdout:521: done (13 left) 2024-09-13T10:10:51.697 INFO:tasks.rados.rados.0.smithi195.stdout:524: done (12 left) 2024-09-13T10:10:51.697 INFO:tasks.rados.rados.0.smithi195.stdout:525: done (11 left) 2024-09-13T10:10:51.697 INFO:tasks.rados.rados.0.smithi195.stdout:526: done (10 left) 2024-09-13T10:10:51.698 INFO:tasks.rados.rados.0.smithi195.stdout:527: done (9 left) 2024-09-13T10:10:51.698 INFO:tasks.rados.rados.0.smithi195.stdout:528: done (8 left) 2024-09-13T10:10:51.698 INFO:tasks.rados.rados.0.smithi195.stdout:537: set_chunk oid 38 target oid 38 low_tier 2024-09-13T10:10:51.698 INFO:tasks.rados.rados.0.smithi195.stdout:537: set_chunk oid 38 offset: 445440 length: 1791488 target oid smithi19530714-38 low_tier offset: 445440 snap -1 2024-09-13T10:10:51.698 INFO:tasks.rados.rados.0.smithi195.stdout:538: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.698 INFO:tasks.rados.rados.0.smithi195.stdout:538: set_chunk oid 39 offset: 492032 length: 599552 target oid smithi19530714-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 492032 snap -1 2024-09-13T10:10:51.698 INFO:tasks.rados.rados.0.smithi195.stdout:539: set_chunk oid 40 target oid 40 low_tier 2024-09-13T10:10:51.698 INFO:tasks.rados.rados.0.smithi195.stdout:539: set_chunk oid 40 offset: 1047552 length: 1081856 target oid smithi19530714-40 low_tier offset: 1047552 snap -1 2024-09-13T10:10:51.698 INFO:tasks.rados.rados.0.smithi195.stdout:540: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.698 INFO:tasks.rados.rados.0.smithi195.stdout:540: set_chunk oid 41 offset: 1323008 length: 526336 target oid smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1323008 snap -1 2024-09-13T10:10:51.698 INFO:tasks.rados.rados.0.smithi195.stdout:541: set_chunk oid 42 target oid 42 low_tier 2024-09-13T10:10:51.698 INFO:tasks.rados.rados.0.smithi195.stdout:541: set_chunk oid 42 offset: 914944 length: 1126400 target oid smithi19530714-42 low_tier offset: 914944 snap -1 2024-09-13T10:10:51.699 INFO:tasks.rados.rados.0.smithi195.stdout:542: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.699 INFO:tasks.rados.rados.0.smithi195.stdout:542: set_chunk oid 43 offset: 343552 length: 2048000 target oid smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 343552 snap -1 2024-09-13T10:10:51.699 INFO:tasks.rados.rados.0.smithi195.stdout:543: set_chunk oid 44 target oid 44 low_tier 2024-09-13T10:10:51.699 INFO:tasks.rados.rados.0.smithi195.stdout:543: set_chunk oid 44 offset: 40960 length: 2434048 target oid smithi19530714-44 low_tier offset: 40960 snap -1 2024-09-13T10:10:51.699 INFO:tasks.rados.rados.0.smithi195.stdout:544: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.699 INFO:tasks.rados.rados.0.smithi195.stdout:544: set_chunk oid 45 offset: 2488832 length: 132608 target oid smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 2488832 snap -1 2024-09-13T10:10:51.699 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.699 INFO:tasks.rados.rados.0.smithi195.stdout:529: finishing set_chunk to oid 30 2024-09-13T10:10:51.699 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 30 set_chunk 30 low_tier returned error code -95 offset: 807936 length: 563712 2024-09-13T10:10:51.699 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 0 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-13T10:10:51.699 INFO:tasks.rados.rados.0.smithi195.stdout:529: done (15 left) 2024-09-13T10:10:51.699 INFO:tasks.rados.rados.0.smithi195.stdout:545: set_chunk oid 46 target oid 46 low_tier 2024-09-13T10:10:51.700 INFO:tasks.rados.rados.0.smithi195.stdout:545: set_chunk oid 46 offset: 719872 length: 486400 target oid smithi19530714-46 low_tier offset: 719872 snap -1 2024-09-13T10:10:51.700 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.700 INFO:tasks.rados.rados.0.smithi195.stdout:530: finishing set_chunk to oid 31 2024-09-13T10:10:51.700 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 31 set_chunk 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 833536 length: 4608 2024-09-13T10:10:51.700 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-13T10:10:51.700 INFO:tasks.rados.rados.0.smithi195.stdout:531: finishing set_chunk to oid 32 2024-09-13T10:10:51.700 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 651776 length: 1061888 2024-09-13T10:10:51.700 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-13T10:10:51.700 INFO:tasks.rados.rados.0.smithi195.stdout:532: finishing set_chunk to oid 33 2024-09-13T10:10:51.700 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 33 set_chunk 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 352256 length: 2146304 2024-09-13T10:10:51.700 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 0 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-13T10:10:51.700 INFO:tasks.rados.rados.0.smithi195.stdout:534: finishing set_chunk to oid 35 2024-09-13T10:10:51.701 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 35 set_chunk 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 36864 length: 560128 2024-09-13T10:10:51.701 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 0 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-13T10:10:51.701 INFO:tasks.rados.rados.0.smithi195.stdout:533: finishing set_chunk to oid 34 2024-09-13T10:10:51.701 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 34 set_chunk 34 low_tier returned error code -95 offset: 347136 length: 1634816 2024-09-13T10:10:51.701 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 34 v 0 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-13T10:10:51.701 INFO:tasks.rados.rados.0.smithi195.stdout:536: finishing set_chunk to oid 37 2024-09-13T10:10:51.701 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 37 set_chunk 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1460736 length: 369664 2024-09-13T10:10:51.701 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 0 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-13T10:10:51.701 INFO:tasks.rados.rados.0.smithi195.stdout:530: done (15 left) 2024-09-13T10:10:51.701 INFO:tasks.rados.rados.0.smithi195.stdout:531: done (14 left) 2024-09-13T10:10:51.701 INFO:tasks.rados.rados.0.smithi195.stdout:532: done (13 left) 2024-09-13T10:10:51.701 INFO:tasks.rados.rados.0.smithi195.stdout:533: done (12 left) 2024-09-13T10:10:51.701 INFO:tasks.rados.rados.0.smithi195.stdout:534: done (11 left) 2024-09-13T10:10:51.702 INFO:tasks.rados.rados.0.smithi195.stdout:536: done (10 left) 2024-09-13T10:10:51.702 INFO:tasks.rados.rados.0.smithi195.stdout:546: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.702 INFO:tasks.rados.rados.0.smithi195.stdout:546: set_chunk oid 47 offset: 913408 length: 120320 target oid smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 913408 snap -1 2024-09-13T10:10:51.702 INFO:tasks.rados.rados.0.smithi195.stdout:547: set_chunk oid 48 target oid 48 low_tier 2024-09-13T10:10:51.702 INFO:tasks.rados.rados.0.smithi195.stdout:547: set_chunk oid 48 offset: 2393088 length: 147968 target oid smithi19530714-48 low_tier offset: 2393088 snap -1 2024-09-13T10:10:51.702 INFO:tasks.rados.rados.0.smithi195.stdout:548: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.702 INFO:tasks.rados.rados.0.smithi195.stdout:548: set_chunk oid 49 offset: 544768 length: 2006016 target oid smithi19530714-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 544768 snap -1 2024-09-13T10:10:51.702 INFO:tasks.rados.rados.0.smithi195.stdout:549: set_chunk oid 50 target oid 50 low_tier 2024-09-13T10:10:51.702 INFO:tasks.rados.rados.0.smithi195.stdout:549: set_chunk oid 50 offset: 652288 length: 796160 target oid smithi19530714-50 low_tier offset: 652288 snap -1 2024-09-13T10:10:51.702 INFO:tasks.rados.rados.0.smithi195.stdout:550: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.702 INFO:tasks.rados.rados.0.smithi195.stdout:550: set_chunk oid 1 offset: 190464 length: 409600 target oid smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 190464 snap -1 2024-09-13T10:10:51.702 INFO:tasks.rados.rados.0.smithi195.stdout:551: set_chunk oid 2 target oid 2 low_tier 2024-09-13T10:10:51.702 INFO:tasks.rados.rados.0.smithi195.stdout:551: set_chunk oid 2 offset: 22528 length: 1902592 target oid smithi19530714-2 low_tier offset: 22528 snap -1 2024-09-13T10:10:51.703 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.703 INFO:tasks.rados.rados.0.smithi195.stdout:537: finishing set_chunk to oid 38 2024-09-13T10:10:51.703 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 38 set_chunk 38 low_tier returned error code -95 offset: 445440 length: 1791488 2024-09-13T10:10:51.703 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 0 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-13T10:10:51.703 INFO:tasks.rados.rados.0.smithi195.stdout:539: finishing set_chunk to oid 40 2024-09-13T10:10:51.703 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 40 set_chunk 40 low_tier returned error code -95 offset: 1047552 length: 1081856 2024-09-13T10:10:51.703 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 0 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-13T10:10:51.703 INFO:tasks.rados.rados.0.smithi195.stdout:542: finishing set_chunk to oid 43 2024-09-13T10:10:51.703 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 343552 length: 2048000 2024-09-13T10:10:51.703 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-13T10:10:51.703 INFO:tasks.rados.rados.0.smithi195.stdout:537: done (15 left) 2024-09-13T10:10:51.703 INFO:tasks.rados.rados.0.smithi195.stdout:539: done (14 left) 2024-09-13T10:10:51.703 INFO:tasks.rados.rados.0.smithi195.stdout:542: done (13 left) 2024-09-13T10:10:51.703 INFO:tasks.rados.rados.0.smithi195.stdout:552: set_chunk oid 3 target oid 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.704 INFO:tasks.rados.rados.0.smithi195.stdout:552: set_chunk oid 3 offset: 1288192 length: 734720 target oid smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1288192 snap -1 2024-09-13T10:10:51.704 INFO:tasks.rados.rados.0.smithi195.stdout:553: set_chunk oid 4 target oid 4 low_tier 2024-09-13T10:10:51.704 INFO:tasks.rados.rados.0.smithi195.stdout:553: set_chunk oid 4 offset: 1331712 length: 160256 target oid smithi19530714-4 low_tier offset: 1331712 snap -1 2024-09-13T10:10:51.704 INFO:tasks.rados.rados.0.smithi195.stdout:554: set_chunk oid 5 target oid 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.704 INFO:tasks.rados.rados.0.smithi195.stdout:554: set_chunk oid 5 offset: 803840 length: 974848 target oid smithi19530714-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 803840 snap -1 2024-09-13T10:10:51.704 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.704 INFO:tasks.rados.rados.0.smithi195.stdout:538: finishing set_chunk to oid 39 2024-09-13T10:10:51.704 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 492032 length: 599552 2024-09-13T10:10:51.704 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-13T10:10:51.704 INFO:tasks.rados.rados.0.smithi195.stdout:541: finishing set_chunk to oid 42 2024-09-13T10:10:51.704 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 42 set_chunk 42 low_tier returned error code -95 offset: 914944 length: 1126400 2024-09-13T10:10:51.704 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-13T10:10:51.704 INFO:tasks.rados.rados.0.smithi195.stdout:540: finishing set_chunk to oid 41 2024-09-13T10:10:51.705 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 41 set_chunk 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1323008 length: 526336 2024-09-13T10:10:51.705 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-13T10:10:51.705 INFO:tasks.rados.rados.0.smithi195.stdout:543: finishing set_chunk to oid 44 2024-09-13T10:10:51.705 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 44 set_chunk 44 low_tier returned error code -95 offset: 40960 length: 2434048 2024-09-13T10:10:51.705 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 0 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-13T10:10:51.705 INFO:tasks.rados.rados.0.smithi195.stdout:538: done (15 left) 2024-09-13T10:10:51.705 INFO:tasks.rados.rados.0.smithi195.stdout:540: done (14 left) 2024-09-13T10:10:51.705 INFO:tasks.rados.rados.0.smithi195.stdout:541: done (13 left) 2024-09-13T10:10:51.705 INFO:tasks.rados.rados.0.smithi195.stdout:543: done (12 left) 2024-09-13T10:10:51.705 INFO:tasks.rados.rados.0.smithi195.stdout:555: set_chunk oid 6 target oid 6 low_tier 2024-09-13T10:10:51.705 INFO:tasks.rados.rados.0.smithi195.stdout:555: set_chunk oid 6 offset: 168960 length: 551424 target oid smithi19530714-6 low_tier offset: 168960 snap -1 2024-09-13T10:10:51.705 INFO:tasks.rados.rados.0.smithi195.stdout:556: set_chunk oid 7 target oid 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.705 INFO:tasks.rados.rados.0.smithi195.stdout:556: set_chunk oid 7 offset: 272896 length: 763392 target oid smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 272896 snap -1 2024-09-13T10:10:51.705 INFO:tasks.rados.rados.0.smithi195.stdout:557: set_chunk oid 8 target oid 8 low_tier 2024-09-13T10:10:51.706 INFO:tasks.rados.rados.0.smithi195.stdout:557: set_chunk oid 8 offset: 462848 length: 118272 target oid smithi19530714-8 low_tier offset: 462848 snap -1 2024-09-13T10:10:51.706 INFO:tasks.rados.rados.0.smithi195.stdout:550: finishing set_chunk to oid 1 2024-09-13T10:10:51.706 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 1 set_chunk 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 190464 length: 409600 2024-09-13T10:10:51.706 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 0 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-13T10:10:51.706 INFO:tasks.rados.rados.0.smithi195.stdout:550: done (14 left) 2024-09-13T10:10:51.706 INFO:tasks.rados.rados.0.smithi195.stdout:558: set_chunk oid 9 target oid 9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.706 INFO:tasks.rados.rados.0.smithi195.stdout:558: set_chunk oid 9 offset: 583680 length: 322560 target oid smithi19530714-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 583680 snap -1 2024-09-13T10:10:51.706 INFO:tasks.rados.rados.0.smithi195.stdout:559: set_chunk oid 10 target oid 10 low_tier 2024-09-13T10:10:51.706 INFO:tasks.rados.rados.0.smithi195.stdout:559: set_chunk oid 10 offset: 156672 length: 1692672 target oid smithi19530714-10 low_tier offset: 156672 snap -1 2024-09-13T10:10:51.707 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.707 INFO:tasks.rados.rados.0.smithi195.stdout:545: finishing set_chunk to oid 46 2024-09-13T10:10:51.707 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 46 set_chunk 46 low_tier returned error code -95 offset: 719872 length: 486400 2024-09-13T10:10:51.707 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 0 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-13T10:10:51.707 INFO:tasks.rados.rados.0.smithi195.stdout:546: finishing set_chunk to oid 47 2024-09-13T10:10:51.707 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 47 set_chunk 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 913408 length: 120320 2024-09-13T10:10:51.707 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-13T10:10:51.707 INFO:tasks.rados.rados.0.smithi195.stdout:549: finishing set_chunk to oid 50 2024-09-13T10:10:51.707 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 50 set_chunk 50 low_tier returned error code -95 offset: 652288 length: 796160 2024-09-13T10:10:51.707 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 0 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-13T10:10:51.707 INFO:tasks.rados.rados.0.smithi195.stdout:545: done (15 left) 2024-09-13T10:10:51.707 INFO:tasks.rados.rados.0.smithi195.stdout:546: done (14 left) 2024-09-13T10:10:51.708 INFO:tasks.rados.rados.0.smithi195.stdout:549: done (13 left) 2024-09-13T10:10:51.708 INFO:tasks.rados.rados.0.smithi195.stdout:560: set_chunk oid 11 target oid 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.708 INFO:tasks.rados.rados.0.smithi195.stdout:560: set_chunk oid 11 offset: 1542144 length: 553472 target oid smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1542144 snap -1 2024-09-13T10:10:51.708 INFO:tasks.rados.rados.0.smithi195.stdout:561: set_chunk oid 12 target oid 12 low_tier 2024-09-13T10:10:51.708 INFO:tasks.rados.rados.0.smithi195.stdout:561: set_chunk oid 12 offset: 991744 length: 729600 target oid smithi19530714-12 low_tier offset: 991744 snap -1 2024-09-13T10:10:51.708 INFO:tasks.rados.rados.0.smithi195.stdout:562: set_chunk oid 13 target oid 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.708 INFO:tasks.rados.rados.0.smithi195.stdout:562: set_chunk oid 13 offset: 964096 length: 1120768 target oid smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 964096 snap -1 2024-09-13T10:10:51.708 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.708 INFO:tasks.rados.rados.0.smithi195.stdout:554: finishing set_chunk to oid 5 2024-09-13T10:10:51.708 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 5 set_chunk 5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 803840 length: 974848 2024-09-13T10:10:51.708 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 0 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-09-13T10:10:51.708 INFO:tasks.rados.rados.0.smithi195.stdout:554: done (15 left) 2024-09-13T10:10:51.709 INFO:tasks.rados.rados.0.smithi195.stdout:563: set_chunk oid 14 target oid 14 low_tier 2024-09-13T10:10:51.709 INFO:tasks.rados.rados.0.smithi195.stdout:563: set_chunk oid 14 offset: 1360896 length: 673280 target oid smithi19530714-14 low_tier offset: 1360896 snap -1 2024-09-13T10:10:51.709 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.709 INFO:tasks.rados.rados.0.smithi195.stdout:553: finishing set_chunk to oid 4 2024-09-13T10:10:51.709 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 4 set_chunk 4 low_tier returned error code -95 offset: 1331712 length: 160256 2024-09-13T10:10:51.709 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 0 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-09-13T10:10:51.709 INFO:tasks.rados.rados.0.smithi195.stdout:548: finishing set_chunk to oid 49 2024-09-13T10:10:51.709 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 49 set_chunk 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 544768 length: 2006016 2024-09-13T10:10:51.709 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 0 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-13T10:10:51.709 INFO:tasks.rados.rados.0.smithi195.stdout:551: finishing set_chunk to oid 2 2024-09-13T10:10:51.709 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 2 set_chunk 2 low_tier returned error code -95 offset: 22528 length: 1902592 2024-09-13T10:10:51.709 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 0 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-09-13T10:10:51.709 INFO:tasks.rados.rados.0.smithi195.stdout:552: finishing set_chunk to oid 3 2024-09-13T10:10:51.710 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 3 set_chunk 3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1288192 length: 734720 2024-09-13T10:10:51.710 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 0 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-09-13T10:10:51.710 INFO:tasks.rados.rados.0.smithi195.stdout:548: done (15 left) 2024-09-13T10:10:51.710 INFO:tasks.rados.rados.0.smithi195.stdout:551: done (14 left) 2024-09-13T10:10:51.710 INFO:tasks.rados.rados.0.smithi195.stdout:552: done (13 left) 2024-09-13T10:10:51.710 INFO:tasks.rados.rados.0.smithi195.stdout:553: done (12 left) 2024-09-13T10:10:51.710 INFO:tasks.rados.rados.0.smithi195.stdout:564: set_chunk oid 15 target oid 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.710 INFO:tasks.rados.rados.0.smithi195.stdout:564: set_chunk oid 15 offset: 1822720 length: 125952 target oid smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1822720 snap -1 2024-09-13T10:10:51.710 INFO:tasks.rados.rados.0.smithi195.stdout:565: set_chunk oid 16 target oid 16 low_tier 2024-09-13T10:10:51.710 INFO:tasks.rados.rados.0.smithi195.stdout:565: set_chunk oid 16 offset: 599040 length: 1227264 target oid smithi19530714-16 low_tier offset: 599040 snap -1 2024-09-13T10:10:51.710 INFO:tasks.rados.rados.0.smithi195.stdout:566: set_chunk oid 17 target oid 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.710 INFO:tasks.rados.rados.0.smithi195.stdout:566: set_chunk oid 17 offset: 1729536 length: 176128 target oid smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1729536 snap -1 2024-09-13T10:10:51.710 INFO:tasks.rados.rados.0.smithi195.stdout:567: set_chunk oid 18 target oid 18 low_tier 2024-09-13T10:10:51.710 INFO:tasks.rados.rados.0.smithi195.stdout:567: set_chunk oid 18 offset: 1812992 length: 402432 target oid smithi19530714-18 low_tier offset: 1812992 snap -1 2024-09-13T10:10:51.711 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.711 INFO:tasks.rados.rados.0.smithi195.stdout:556: finishing set_chunk to oid 7 2024-09-13T10:10:51.711 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 7 set_chunk 7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 272896 length: 763392 2024-09-13T10:10:51.711 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 0 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-09-13T10:10:51.711 INFO:tasks.rados.rados.0.smithi195.stdout:557: finishing set_chunk to oid 8 2024-09-13T10:10:51.711 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 8 set_chunk 8 low_tier returned error code -95 offset: 462848 length: 118272 2024-09-13T10:10:51.711 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-13T10:10:51.711 INFO:tasks.rados.rados.0.smithi195.stdout:556: done (15 left) 2024-09-13T10:10:51.711 INFO:tasks.rados.rados.0.smithi195.stdout:557: done (14 left) 2024-09-13T10:10:51.711 INFO:tasks.rados.rados.0.smithi195.stdout:568: set_chunk oid 19 target oid 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.711 INFO:tasks.rados.rados.0.smithi195.stdout:568: set_chunk oid 19 offset: 947200 length: 241664 target oid smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 947200 snap -1 2024-09-13T10:10:51.711 INFO:tasks.rados.rados.0.smithi195.stdout:569: set_chunk oid 20 target oid 20 low_tier 2024-09-13T10:10:51.711 INFO:tasks.rados.rados.0.smithi195.stdout:569: set_chunk oid 20 offset: 281088 length: 369664 target oid smithi19530714-20 low_tier offset: 281088 snap -1 2024-09-13T10:10:51.712 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.712 INFO:tasks.rados.rados.0.smithi195.stdout:555: finishing set_chunk to oid 6 2024-09-13T10:10:51.712 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 6 set_chunk 6 low_tier returned error code -95 offset: 168960 length: 551424 2024-09-13T10:10:51.712 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 0 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-09-13T10:10:51.712 INFO:tasks.rados.rados.0.smithi195.stdout:555: done (15 left) 2024-09-13T10:10:51.712 INFO:tasks.rados.rados.0.smithi195.stdout:570: set_chunk oid 21 target oid 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.712 INFO:tasks.rados.rados.0.smithi195.stdout:570: set_chunk oid 21 offset: 387584 length: 665088 target oid smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 387584 snap -1 2024-09-13T10:10:51.712 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.712 INFO:tasks.rados.rados.0.smithi195.stdout:561: finishing set_chunk to oid 12 2024-09-13T10:10:51.712 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 12 set_chunk 12 low_tier returned error code -95 offset: 991744 length: 729600 2024-09-13T10:10:51.712 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-13T10:10:51.712 INFO:tasks.rados.rados.0.smithi195.stdout:561: done (15 left) 2024-09-13T10:10:51.712 INFO:tasks.rados.rados.0.smithi195.stdout:571: set_chunk oid 22 target oid 22 low_tier 2024-09-13T10:10:51.713 INFO:tasks.rados.rados.0.smithi195.stdout:571: set_chunk oid 22 offset: 444928 length: 1777152 target oid smithi19530714-22 low_tier offset: 444928 snap -1 2024-09-13T10:10:51.713 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.713 INFO:tasks.rados.rados.0.smithi195.stdout:562: finishing set_chunk to oid 13 2024-09-13T10:10:51.713 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 13 set_chunk 13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 964096 length: 1120768 2024-09-13T10:10:51.713 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 0 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-09-13T10:10:51.713 INFO:tasks.rados.rados.0.smithi195.stdout:559: finishing set_chunk to oid 10 2024-09-13T10:10:51.713 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 10 set_chunk 10 low_tier returned error code -95 offset: 156672 length: 1692672 2024-09-13T10:10:51.713 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 0 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-09-13T10:10:51.713 INFO:tasks.rados.rados.0.smithi195.stdout:560: finishing set_chunk to oid 11 2024-09-13T10:10:51.713 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 11 set_chunk 11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1542144 length: 553472 2024-09-13T10:10:51.713 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 0 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-09-13T10:10:51.713 INFO:tasks.rados.rados.0.smithi195.stdout:563: finishing set_chunk to oid 14 2024-09-13T10:10:51.714 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 14 set_chunk 14 low_tier returned error code -95 offset: 1360896 length: 673280 2024-09-13T10:10:51.714 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 0 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-09-13T10:10:51.714 INFO:tasks.rados.rados.0.smithi195.stdout:559: done (15 left) 2024-09-13T10:10:51.714 INFO:tasks.rados.rados.0.smithi195.stdout:560: done (14 left) 2024-09-13T10:10:51.714 INFO:tasks.rados.rados.0.smithi195.stdout:562: done (13 left) 2024-09-13T10:10:51.714 INFO:tasks.rados.rados.0.smithi195.stdout:563: done (12 left) 2024-09-13T10:10:51.714 INFO:tasks.rados.rados.0.smithi195.stdout:572: set_chunk oid 23 target oid 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.714 INFO:tasks.rados.rados.0.smithi195.stdout:572: set_chunk oid 23 offset: 1108992 length: 374784 target oid smithi19530714-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1108992 snap -1 2024-09-13T10:10:51.714 INFO:tasks.rados.rados.0.smithi195.stdout:573: set_chunk oid 24 target oid 24 low_tier 2024-09-13T10:10:51.714 INFO:tasks.rados.rados.0.smithi195.stdout:573: set_chunk oid 24 offset: 448512 length: 1740800 target oid smithi19530714-24 low_tier offset: 448512 snap -1 2024-09-13T10:10:51.714 INFO:tasks.rados.rados.0.smithi195.stdout:574: set_chunk oid 25 target oid 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.715 INFO:tasks.rados.rados.0.smithi195.stdout:574: set_chunk oid 25 offset: 733184 length: 1639936 target oid smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 733184 snap -1 2024-09-13T10:10:51.715 INFO:tasks.rados.rados.0.smithi195.stdout:568: finishing set_chunk to oid 19 2024-09-13T10:10:51.715 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 19 set_chunk 19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 947200 length: 241664 2024-09-13T10:10:51.715 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 0 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-09-13T10:10:51.715 INFO:tasks.rados.rados.0.smithi195.stdout:570: finishing set_chunk to oid 21 2024-09-13T10:10:51.715 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 21 set_chunk 21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 387584 length: 665088 2024-09-13T10:10:51.715 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 0 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-09-13T10:10:51.715 INFO:tasks.rados.rados.0.smithi195.stdout:565: finishing set_chunk to oid 16 2024-09-13T10:10:51.715 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 16 set_chunk 16 low_tier returned error code -95 offset: 599040 length: 1227264 2024-09-13T10:10:51.715 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-13T10:10:51.715 INFO:tasks.rados.rados.0.smithi195.stdout:565: done (14 left) 2024-09-13T10:10:51.715 INFO:tasks.rados.rados.0.smithi195.stdout:568: done (13 left) 2024-09-13T10:10:51.716 INFO:tasks.rados.rados.0.smithi195.stdout:570: done (12 left) 2024-09-13T10:10:51.716 INFO:tasks.rados.rados.0.smithi195.stdout:575: set_chunk oid 26 target oid 26 low_tier 2024-09-13T10:10:51.716 INFO:tasks.rados.rados.0.smithi195.stdout:575: set_chunk oid 26 offset: 997888 length: 422912 target oid smithi19530714-26 low_tier offset: 997888 snap -1 2024-09-13T10:10:51.716 INFO:tasks.rados.rados.0.smithi195.stdout:576: set_chunk oid 27 target oid 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.716 INFO:tasks.rados.rados.0.smithi195.stdout:576: set_chunk oid 27 offset: 1040384 length: 918528 target oid smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1040384 snap -1 2024-09-13T10:10:51.716 INFO:tasks.rados.rados.0.smithi195.stdout:577: set_chunk oid 28 target oid 28 low_tier 2024-09-13T10:10:51.716 INFO:tasks.rados.rados.0.smithi195.stdout:577: set_chunk oid 28 offset: 1565184 length: 475648 target oid smithi19530714-28 low_tier offset: 1565184 snap -1 2024-09-13T10:10:51.716 INFO:tasks.rados.rados.0.smithi195.stdout:578: set_chunk oid 29 target oid 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.716 INFO:tasks.rados.rados.0.smithi195.stdout:578: set_chunk oid 29 offset: 25600 length: 737280 target oid smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 25600 snap -1 2024-09-13T10:10:51.716 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.716 INFO:tasks.rados.rados.0.smithi195.stdout:571: finishing set_chunk to oid 22 2024-09-13T10:10:51.716 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 22 set_chunk 22 low_tier returned error code -95 offset: 444928 length: 1777152 2024-09-13T10:10:51.717 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 0 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-13T10:10:51.717 INFO:tasks.rados.rados.0.smithi195.stdout:564: finishing set_chunk to oid 15 2024-09-13T10:10:51.717 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 15 set_chunk 15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1822720 length: 125952 2024-09-13T10:10:51.717 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-13T10:10:51.717 INFO:tasks.rados.rados.0.smithi195.stdout:569: finishing set_chunk to oid 20 2024-09-13T10:10:51.717 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 20 set_chunk 20 low_tier returned error code -95 offset: 281088 length: 369664 2024-09-13T10:10:51.717 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-13T10:10:51.717 INFO:tasks.rados.rados.0.smithi195.stdout:566: finishing set_chunk to oid 17 2024-09-13T10:10:51.717 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 17 set_chunk 17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1729536 length: 176128 2024-09-13T10:10:51.717 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 0 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-09-13T10:10:51.717 INFO:tasks.rados.rados.0.smithi195.stdout:572: finishing set_chunk to oid 23 2024-09-13T10:10:51.718 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 23 set_chunk 23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1108992 length: 374784 2024-09-13T10:10:51.718 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-13T10:10:51.718 INFO:tasks.rados.rados.0.smithi195.stdout:573: finishing set_chunk to oid 24 2024-09-13T10:10:51.718 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 24 set_chunk 24 low_tier returned error code -95 offset: 448512 length: 1740800 2024-09-13T10:10:51.718 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 0 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-13T10:10:51.718 INFO:tasks.rados.rados.0.smithi195.stdout:564: done (15 left) 2024-09-13T10:10:51.718 INFO:tasks.rados.rados.0.smithi195.stdout:566: done (14 left) 2024-09-13T10:10:51.718 INFO:tasks.rados.rados.0.smithi195.stdout:569: done (13 left) 2024-09-13T10:10:51.718 INFO:tasks.rados.rados.0.smithi195.stdout:571: done (12 left) 2024-09-13T10:10:51.718 INFO:tasks.rados.rados.0.smithi195.stdout:572: done (11 left) 2024-09-13T10:10:51.718 INFO:tasks.rados.rados.0.smithi195.stdout:573: done (10 left) 2024-09-13T10:10:51.718 INFO:tasks.rados.rados.0.smithi195.stdout:579: set_chunk oid 30 target oid 30 low_tier 2024-09-13T10:10:51.718 INFO:tasks.rados.rados.0.smithi195.stdout:579: set_chunk oid 30 offset: 2354688 length: 24576 target oid smithi19530714-30 low_tier offset: 2354688 snap -1 2024-09-13T10:10:51.718 INFO:tasks.rados.rados.0.smithi195.stdout:580: set_chunk oid 31 target oid 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.719 INFO:tasks.rados.rados.0.smithi195.stdout:580: set_chunk oid 31 offset: 667648 length: 826880 target oid smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 667648 snap -1 2024-09-13T10:10:51.719 INFO:tasks.rados.rados.0.smithi195.stdout:581: set_chunk oid 32 target oid 32 low_tier 2024-09-13T10:10:51.719 INFO:tasks.rados.rados.0.smithi195.stdout:581: set_chunk oid 32 offset: 1632256 length: 377856 target oid smithi19530714-32 low_tier offset: 1632256 snap -1 2024-09-13T10:10:51.719 INFO:tasks.rados.rados.0.smithi195.stdout:582: set_chunk oid 33 target oid 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.719 INFO:tasks.rados.rados.0.smithi195.stdout:582: set_chunk oid 33 offset: 261120 length: 429568 target oid smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 261120 snap -1 2024-09-13T10:10:51.719 INFO:tasks.rados.rados.0.smithi195.stdout:583: set_chunk oid 34 target oid 34 low_tier 2024-09-13T10:10:51.719 INFO:tasks.rados.rados.0.smithi195.stdout:583: set_chunk oid 34 offset: 1360384 length: 1090560 target oid smithi19530714-34 low_tier offset: 1360384 snap -1 2024-09-13T10:10:51.719 INFO:tasks.rados.rados.0.smithi195.stdout:584: set_chunk oid 35 target oid 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.719 INFO:tasks.rados.rados.0.smithi195.stdout:584: set_chunk oid 35 offset: 710144 length: 584192 target oid smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 710144 snap -1 2024-09-13T10:10:51.719 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.719 INFO:tasks.rados.rados.0.smithi195.stdout:567: finishing set_chunk to oid 18 2024-09-13T10:10:51.719 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 18 set_chunk 18 low_tier returned error code -95 offset: 1812992 length: 402432 2024-09-13T10:10:51.719 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 0 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-09-13T10:10:51.720 INFO:tasks.rados.rados.0.smithi195.stdout:574: finishing set_chunk to oid 25 2024-09-13T10:10:51.720 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 25 set_chunk 25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 733184 length: 1639936 2024-09-13T10:10:51.720 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-13T10:10:51.720 INFO:tasks.rados.rados.0.smithi195.stdout:578: finishing set_chunk to oid 29 2024-09-13T10:10:51.720 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 29 set_chunk 29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 25600 length: 737280 2024-09-13T10:10:51.720 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 0 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-09-13T10:10:51.720 INFO:tasks.rados.rados.0.smithi195.stdout:567: done (15 left) 2024-09-13T10:10:51.720 INFO:tasks.rados.rados.0.smithi195.stdout:574: done (14 left) 2024-09-13T10:10:51.720 INFO:tasks.rados.rados.0.smithi195.stdout:578: done (13 left) 2024-09-13T10:10:51.720 INFO:tasks.rados.rados.0.smithi195.stdout:584 retry set_chunk ! 2024-09-13T10:10:51.720 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 13 2024-09-13T10:10:51.720 INFO:tasks.rados.rados.0.smithi195.stdout:576: finishing set_chunk to oid 27 2024-09-13T10:10:51.720 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 27 set_chunk 27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1040384 length: 918528 2024-09-13T10:10:51.721 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-13T10:10:51.721 INFO:tasks.rados.rados.0.smithi195.stdout:577: finishing set_chunk to oid 28 2024-09-13T10:10:51.721 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 28 set_chunk 28 low_tier returned error code -95 offset: 1565184 length: 475648 2024-09-13T10:10:51.721 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 0 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-13T10:10:51.721 INFO:tasks.rados.rados.0.smithi195.stdout:580: finishing set_chunk to oid 31 2024-09-13T10:10:51.721 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 31 set_chunk 31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 667648 length: 826880 2024-09-13T10:10:51.721 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 0 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-09-13T10:10:51.721 INFO:tasks.rados.rados.0.smithi195.stdout:576: done (12 left) 2024-09-13T10:10:51.721 INFO:tasks.rados.rados.0.smithi195.stdout:577: done (11 left) 2024-09-13T10:10:51.721 INFO:tasks.rados.rados.0.smithi195.stdout:580: done (10 left) 2024-09-13T10:10:51.721 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 10 2024-09-13T10:10:51.721 INFO:tasks.rados.rados.0.smithi195.stdout:582: finishing set_chunk to oid 33 2024-09-13T10:10:51.721 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 33 set_chunk 33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 261120 length: 429568 2024-09-13T10:10:51.722 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 0 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-09-13T10:10:51.722 INFO:tasks.rados.rados.0.smithi195.stdout:583: finishing set_chunk to oid 34 2024-09-13T10:10:51.722 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 34 set_chunk 34 low_tier returned error code -95 offset: 1360384 length: 1090560 2024-09-13T10:10:51.722 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 34 v 0 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-09-13T10:10:51.722 INFO:tasks.rados.rados.0.smithi195.stdout:575: finishing set_chunk to oid 26 2024-09-13T10:10:51.722 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 26 set_chunk 26 low_tier returned error code -95 offset: 997888 length: 422912 2024-09-13T10:10:51.722 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 0 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-09-13T10:10:51.722 INFO:tasks.rados.rados.0.smithi195.stdout:575: done (9 left) 2024-09-13T10:10:51.722 INFO:tasks.rados.rados.0.smithi195.stdout:582: done (8 left) 2024-09-13T10:10:51.722 INFO:tasks.rados.rados.0.smithi195.stdout:583: done (7 left) 2024-09-13T10:10:51.722 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 7 2024-09-13T10:10:51.722 INFO:tasks.rados.rados.0.smithi195.stdout:581: finishing set_chunk to oid 32 2024-09-13T10:10:51.722 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 32 set_chunk 32 low_tier returned error code -95 offset: 1632256 length: 377856 2024-09-13T10:10:51.723 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 0 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-09-13T10:10:51.723 INFO:tasks.rados.rados.0.smithi195.stdout:581: done (6 left) 2024-09-13T10:10:51.723 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 6 2024-09-13T10:10:51.723 INFO:tasks.rados.rados.0.smithi195.stdout:584: finishing set_chunk to oid 35 2024-09-13T10:10:51.723 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 35 set_chunk 35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 710144 length: 584192 2024-09-13T10:10:51.723 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 0 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-09-13T10:10:51.723 INFO:tasks.rados.rados.0.smithi195.stdout:584: done (5 left) 2024-09-13T10:10:51.723 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 5 2024-09-13T10:10:51.723 INFO:tasks.rados.rados.0.smithi195.stdout:535: finishing set_chunk to oid 36 2024-09-13T10:10:51.723 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 6 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-13T10:10:51.723 INFO:tasks.rados.rados.0.smithi195.stdout:535: done (4 left) 2024-09-13T10:10:51.723 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 4 2024-09-13T10:10:51.723 INFO:tasks.rados.rados.0.smithi195.stdout:544: finishing set_chunk to oid 45 2024-09-13T10:10:51.724 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 15 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-13T10:10:51.724 INFO:tasks.rados.rados.0.smithi195.stdout:544: done (3 left) 2024-09-13T10:10:51.724 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 3 2024-09-13T10:10:51.724 INFO:tasks.rados.rados.0.smithi195.stdout:558: finishing set_chunk to oid 9 2024-09-13T10:10:51.724 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 6 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-09-13T10:10:51.724 INFO:tasks.rados.rados.0.smithi195.stdout:558: done (2 left) 2024-09-13T10:10:51.724 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 2 2024-09-13T10:10:51.724 INFO:tasks.rados.rados.0.smithi195.stdout:547: finishing set_chunk to oid 48 2024-09-13T10:10:51.724 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 15 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-13T10:10:51.724 INFO:tasks.rados.rados.0.smithi195.stdout:547: done (1 left) 2024-09-13T10:10:51.724 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 1 2024-09-13T10:10:51.724 INFO:tasks.rados.rados.0.smithi195.stdout:579: finishing set_chunk to oid 30 2024-09-13T10:10:51.724 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 18 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-09-13T10:10:51.725 INFO:tasks.rados.rados.0.smithi195.stdout:579: done (0 left) 2024-09-13T10:10:51.725 INFO:tasks.rados.rados.0.smithi195.stdout:585: set_chunk oid 36 target oid 36 low_tier 2024-09-13T10:10:51.725 INFO:tasks.rados.rados.0.smithi195.stdout:585: set_chunk oid 36 offset: 1544704 length: 891904 target oid smithi19530714-36 low_tier offset: 1544704 snap -1 2024-09-13T10:10:51.725 INFO:tasks.rados.rados.0.smithi195.stdout:586: set_chunk oid 37 target oid 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.725 INFO:tasks.rados.rados.0.smithi195.stdout:586: set_chunk oid 37 offset: 1065984 length: 913408 target oid smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1065984 snap -1 2024-09-13T10:10:51.725 INFO:tasks.rados.rados.0.smithi195.stdout:587: set_chunk oid 38 target oid 38 low_tier 2024-09-13T10:10:51.725 INFO:tasks.rados.rados.0.smithi195.stdout:587: set_chunk oid 38 offset: 1349632 length: 358400 target oid smithi19530714-38 low_tier offset: 1349632 snap -1 2024-09-13T10:10:51.725 INFO:tasks.rados.rados.0.smithi195.stdout:588: set_chunk oid 39 target oid 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.725 INFO:tasks.rados.rados.0.smithi195.stdout:588: set_chunk oid 39 offset: 1419264 length: 218624 target oid smithi19530714-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1419264 snap -1 2024-09-13T10:10:51.725 INFO:tasks.rados.rados.0.smithi195.stdout:589: set_chunk oid 40 target oid 40 low_tier 2024-09-13T10:10:51.725 INFO:tasks.rados.rados.0.smithi195.stdout:589: set_chunk oid 40 offset: 2522624 length: 68096 target oid smithi19530714-40 low_tier offset: 2522624 snap -1 2024-09-13T10:10:51.725 INFO:tasks.rados.rados.0.smithi195.stdout:590: set_chunk oid 41 target oid 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.726 INFO:tasks.rados.rados.0.smithi195.stdout:590: set_chunk oid 41 offset: 789504 length: 372736 target oid smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 789504 snap -1 2024-09-13T10:10:51.726 INFO:tasks.rados.rados.0.smithi195.stdout:591: set_chunk oid 42 target oid 42 low_tier 2024-09-13T10:10:51.726 INFO:tasks.rados.rados.0.smithi195.stdout:591: set_chunk oid 42 offset: 417280 length: 676864 target oid smithi19530714-42 low_tier offset: 417280 snap -1 2024-09-13T10:10:51.726 INFO:tasks.rados.rados.0.smithi195.stdout:592: set_chunk oid 43 target oid 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.726 INFO:tasks.rados.rados.0.smithi195.stdout:592: set_chunk oid 43 offset: 1442816 length: 189440 target oid smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1442816 snap -1 2024-09-13T10:10:51.726 INFO:tasks.rados.rados.0.smithi195.stdout:593: set_chunk oid 44 target oid 44 low_tier 2024-09-13T10:10:51.726 INFO:tasks.rados.rados.0.smithi195.stdout:593: set_chunk oid 44 offset: 944640 length: 1681920 target oid smithi19530714-44 low_tier offset: 944640 snap -1 2024-09-13T10:10:51.726 INFO:tasks.rados.rados.0.smithi195.stdout:594: set_chunk oid 45 target oid 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.726 INFO:tasks.rados.rados.0.smithi195.stdout:594: set_chunk oid 45 offset: 1250304 length: 1052160 target oid smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1250304 snap -1 2024-09-13T10:10:51.726 INFO:tasks.rados.rados.0.smithi195.stdout:595: set_chunk oid 46 target oid 46 low_tier 2024-09-13T10:10:51.726 INFO:tasks.rados.rados.0.smithi195.stdout:595: set_chunk oid 46 offset: 567808 length: 969216 target oid smithi19530714-46 low_tier offset: 567808 snap -1 2024-09-13T10:10:51.727 INFO:tasks.rados.rados.0.smithi195.stdout:596: set_chunk oid 47 target oid 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.727 INFO:tasks.rados.rados.0.smithi195.stdout:596: set_chunk oid 47 offset: 1031680 length: 574464 target oid smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 1031680 snap -1 2024-09-13T10:10:51.727 INFO:tasks.rados.rados.0.smithi195.stdout:597: set_chunk oid 48 target oid 48 low_tier 2024-09-13T10:10:51.727 INFO:tasks.rados.rados.0.smithi195.stdout:597: set_chunk oid 48 offset: 135168 length: 1762304 target oid smithi19530714-48 low_tier offset: 135168 snap -1 2024-09-13T10:10:51.727 INFO:tasks.rados.rados.0.smithi195.stdout:598: set_chunk oid 49 target oid 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.727 INFO:tasks.rados.rados.0.smithi195.stdout:598: set_chunk oid 49 offset: 72704 length: 1824256 target oid smithi19530714-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 72704 snap -1 2024-09-13T10:10:51.727 INFO:tasks.rados.rados.0.smithi195.stdout:599: set_chunk oid 50 target oid 50 low_tier 2024-09-13T10:10:51.727 INFO:tasks.rados.rados.0.smithi195.stdout:599: set_chunk oid 50 offset: 876544 length: 1682944 target oid smithi19530714-50 low_tier offset: 876544 snap -1 2024-09-13T10:10:51.727 INFO:tasks.rados.rados.0.smithi195.stdout:600: set_chunk oid 1 target oid 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.727 INFO:tasks.rados.rados.0.smithi195.stdout:600: set_chunk oid 1 offset: 313856 length: 1449472 target oid smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm offset: 313856 snap -1 2024-09-13T10:10:51.727 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:51.727 INFO:tasks.rados.rados.0.smithi195.stdout:587: finishing set_chunk to oid 38 2024-09-13T10:10:51.728 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 38 set_chunk 38 low_tier returned error code -95 offset: 1349632 length: 358400 2024-09-13T10:10:51.728 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 0 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-09-13T10:10:51.728 INFO:tasks.rados.rados.0.smithi195.stdout:585: finishing set_chunk to oid 36 2024-09-13T10:10:51.728 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 36 set_chunk 36 low_tier returned error code -95 offset: 1544704 length: 891904 2024-09-13T10:10:51.728 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 0 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-09-13T10:10:51.728 INFO:tasks.rados.rados.0.smithi195.stdout:590: finishing set_chunk to oid 41 2024-09-13T10:10:51.728 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 41 set_chunk 41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 789504 length: 372736 2024-09-13T10:10:51.728 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 0 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-09-13T10:10:51.728 INFO:tasks.rados.rados.0.smithi195.stdout:588: finishing set_chunk to oid 39 2024-09-13T10:10:51.728 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 39 set_chunk 39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1419264 length: 218624 2024-09-13T10:10:51.728 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-13T10:10:51.728 INFO:tasks.rados.rados.0.smithi195.stdout:591: finishing set_chunk to oid 42 2024-09-13T10:10:51.729 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 42 set_chunk 42 low_tier returned error code -95 offset: 417280 length: 676864 2024-09-13T10:10:51.729 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-13T10:10:51.729 INFO:tasks.rados.rados.0.smithi195.stdout:585: done (15 left) 2024-09-13T10:10:51.729 INFO:tasks.rados.rados.0.smithi195.stdout:587: done (14 left) 2024-09-13T10:10:51.729 INFO:tasks.rados.rados.0.smithi195.stdout:588: done (13 left) 2024-09-13T10:10:51.729 INFO:tasks.rados.rados.0.smithi195.stdout:590: done (12 left) 2024-09-13T10:10:51.729 INFO:tasks.rados.rados.0.smithi195.stdout:591: done (11 left) 2024-09-13T10:10:51.729 INFO:tasks.rados.rados.0.smithi195.stdout:601 oid_not_in_use 39 oid_redirect_not_in_use 0 2024-09-13T10:10:51.729 INFO:tasks.rados.rados.0.smithi195.stdout:600 wait for manifest initialization 2024-09-13T10:10:51.729 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 11 2024-09-13T10:10:51.729 INFO:tasks.rados.rados.0.smithi195.stdout:597: finishing set_chunk to oid 48 2024-09-13T10:10:51.729 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 48 set_chunk 48 low_tier returned error code -95 offset: 135168 length: 1762304 2024-09-13T10:10:51.729 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-13T10:10:51.730 INFO:tasks.rados.rados.0.smithi195.stdout:599: finishing set_chunk to oid 50 2024-09-13T10:10:51.730 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 50 set_chunk 50 low_tier returned error code -95 offset: 876544 length: 1682944 2024-09-13T10:10:51.730 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 0 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-13T10:10:51.730 INFO:tasks.rados.rados.0.smithi195.stdout:597: done (10 left) 2024-09-13T10:10:51.730 INFO:tasks.rados.rados.0.smithi195.stdout:599: done (9 left) 2024-09-13T10:10:51.730 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 9 2024-09-13T10:10:51.730 INFO:tasks.rados.rados.0.smithi195.stdout:594: finishing set_chunk to oid 45 2024-09-13T10:10:51.730 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 45 set_chunk 45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1250304 length: 1052160 2024-09-13T10:10:51.730 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 0 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-09-13T10:10:51.730 INFO:tasks.rados.rados.0.smithi195.stdout:596: finishing set_chunk to oid 47 2024-09-13T10:10:51.730 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 47 set_chunk 47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1031680 length: 574464 2024-09-13T10:10:51.730 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 0 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-09-13T10:10:51.730 INFO:tasks.rados.rados.0.smithi195.stdout:592: finishing set_chunk to oid 43 2024-09-13T10:10:51.730 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 43 set_chunk 43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1442816 length: 189440 2024-09-13T10:10:51.731 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-13T10:10:51.731 INFO:tasks.rados.rados.0.smithi195.stdout:586: finishing set_chunk to oid 37 2024-09-13T10:10:51.731 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 37 set_chunk 37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 1065984 length: 913408 2024-09-13T10:10:51.731 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 0 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-13T10:10:51.731 INFO:tasks.rados.rados.0.smithi195.stdout:593: finishing set_chunk to oid 44 2024-09-13T10:10:51.731 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 44 set_chunk 44 low_tier returned error code -95 offset: 944640 length: 1681920 2024-09-13T10:10:51.731 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 0 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-09-13T10:10:51.731 INFO:tasks.rados.rados.0.smithi195.stdout:586: done (8 left) 2024-09-13T10:10:51.731 INFO:tasks.rados.rados.0.smithi195.stdout:592: done (7 left) 2024-09-13T10:10:51.731 INFO:tasks.rados.rados.0.smithi195.stdout:593: done (6 left) 2024-09-13T10:10:51.731 INFO:tasks.rados.rados.0.smithi195.stdout:594: done (5 left) 2024-09-13T10:10:51.731 INFO:tasks.rados.rados.0.smithi195.stdout:596: done (4 left) 2024-09-13T10:10:51.732 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 4 2024-09-13T10:10:51.732 INFO:tasks.rados.rados.0.smithi195.stdout:598: finishing set_chunk to oid 49 2024-09-13T10:10:51.732 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 49 set_chunk 49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 72704 length: 1824256 2024-09-13T10:10:51.732 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 0 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-09-13T10:10:51.732 INFO:tasks.rados.rados.0.smithi195.stdout:598: done (3 left) 2024-09-13T10:10:51.732 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 3 2024-09-13T10:10:51.732 INFO:tasks.rados.rados.0.smithi195.stdout:595: finishing set_chunk to oid 46 2024-09-13T10:10:51.732 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 46 set_chunk 46 low_tier returned error code -95 offset: 567808 length: 969216 2024-09-13T10:10:51.732 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 0 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-09-13T10:10:51.732 INFO:tasks.rados.rados.0.smithi195.stdout:595: done (2 left) 2024-09-13T10:10:51.732 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 2 2024-09-13T10:10:51.732 INFO:tasks.rados.rados.0.smithi195.stdout:600: finishing set_chunk to oid 1 2024-09-13T10:10:51.733 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 1 set_chunk 1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm returned error code -95 offset: 313856 length: 1449472 2024-09-13T10:10:51.733 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 0 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-09-13T10:10:51.733 INFO:tasks.rados.rados.0.smithi195.stdout:600: done (1 left) 2024-09-13T10:10:51.733 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 1 2024-09-13T10:10:51.733 INFO:tasks.rados.rados.0.smithi195.stdout:589: finishing set_chunk to oid 40 2024-09-13T10:10:51.733 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 6 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-09-13T10:10:51.733 INFO:tasks.rados.rados.0.smithi195.stdout:589: done (0 left) 2024-09-13T10:10:51.733 INFO:tasks.rados.rados.0.smithi195.stdout:601 oid_not_in_use 50 oid_redirect_not_in_use 0 2024-09-13T10:10:51.733 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 51 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.733 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 52 low_tier 2024-09-13T10:10:51.733 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 53 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.733 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 54 low_tier 2024-09-13T10:10:51.734 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 55 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.734 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 56 low_tier 2024-09-13T10:10:51.734 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 57 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.734 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 58 low_tier 2024-09-13T10:10:51.734 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 59 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.734 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 60 low_tier 2024-09-13T10:10:51.734 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 61 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.734 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 62 low_tier 2024-09-13T10:10:51.734 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 63 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.734 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 64 low_tier 2024-09-13T10:10:51.734 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 65 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.734 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 66 low_tier 2024-09-13T10:10:51.735 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 67 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.735 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 68 low_tier 2024-09-13T10:10:51.735 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 69 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.735 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 70 low_tier 2024-09-13T10:10:51.735 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 71 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.735 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 72 low_tier 2024-09-13T10:10:51.735 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 73 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.735 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 74 low_tier 2024-09-13T10:10:51.735 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 75 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.735 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 76 low_tier 2024-09-13T10:10:51.735 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 77 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.735 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 78 low_tier 2024-09-13T10:10:51.735 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 79 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.736 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 80 low_tier 2024-09-13T10:10:51.736 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 81 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.736 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 82 low_tier 2024-09-13T10:10:51.736 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 83 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.736 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 84 low_tier 2024-09-13T10:10:51.736 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 85 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.736 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 86 low_tier 2024-09-13T10:10:51.736 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 87 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.736 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 88 low_tier 2024-09-13T10:10:51.736 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 89 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.736 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 90 low_tier 2024-09-13T10:10:51.736 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 91 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.737 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 92 low_tier 2024-09-13T10:10:51.737 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 93 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.737 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 94 low_tier 2024-09-13T10:10:51.737 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 95 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.737 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 96 low_tier 2024-09-13T10:10:51.737 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 97 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.737 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 98 low_tier 2024-09-13T10:10:51.737 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 99 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm 2024-09-13T10:10:51.737 INFO:tasks.rados.rados.0.smithi195.stdout: redirect_not_in_use: 100 low_tier 2024-09-13T10:10:51.737 INFO:tasks.rados.rados.0.smithi195.stdout:601: set_chunk oid 15 target oid 2024-09-13T10:10:51.737 INFO:tasks.rados.rados.0.smithi195.stdout:601: set_chunk oid 15 offset: 399872 length: 706048 target oid 4cf93571197ea284f770e8b4c2a4e4059edb1173e16df9330f6395e4ded4e670 offset: 0 snap -1 2024-09-13T10:10:51.738 INFO:tasks.rados.rados.0.smithi195.stdout:602: read oid 9 snap -1 2024-09-13T10:10:51.738 INFO:tasks.rados.rados.0.smithi195.stdout:602: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-13T10:10:51.738 INFO:tasks.rados.rados.0.smithi195.stdout:603: write (excl) oid 34 current snap is 0 2024-09-13T10:10:51.738 INFO:tasks.rados.rados.0.smithi195.stdout:603: seq_num 50 ranges {440350=460568,1309629=542666,2323973=516377} 2024-09-13T10:10:51.738 INFO:tasks.rados.rados.0.smithi195.stdout:603: writing smithi19530714-34 from 440350 to 900918 tid 1 2024-09-13T10:10:51.738 INFO:tasks.rados.rados.0.smithi195.stdout:603: writing smithi19530714-34 from 1309629 to 1852295 tid 2 2024-09-13T10:10:51.738 INFO:tasks.rados.rados.0.smithi195.stdout:603: writing smithi19530714-34 from 2323973 to 2840350 tid 3 2024-09-13T10:10:51.738 INFO:tasks.rados.rados.0.smithi195.stdout:604: write (excl) oid 18 current snap is 0 2024-09-13T10:10:51.738 INFO:tasks.rados.rados.0.smithi195.stdout:604: seq_num 51 ranges {457157=535817,1768111=402677} 2024-09-13T10:10:51.738 INFO:tasks.rados.rados.0.smithi195.stdout:604: writing smithi19530714-18 from 457157 to 992974 tid 1 2024-09-13T10:10:51.738 INFO:tasks.rados.rados.0.smithi195.stdout:604: writing smithi19530714-18 from 1768111 to 2170788 tid 2 2024-09-13T10:10:51.738 INFO:tasks.rados.rados.0.smithi195.stdout:605: delete oid 11 current snap is 0 2024-09-13T10:10:51.738 INFO:tasks.rados.rados.0.smithi195.stdout:601: finishing set_chunk to oid 15 2024-09-13T10:10:51.738 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 15 set_chunk returned error code -95 offset: 399872 length: 706048 2024-09-13T10:10:51.739 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-13T10:10:51.739 INFO:tasks.rados.rados.0.smithi195.stdout:602: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-13T10:10:51.739 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1321712 2024-09-13T10:10:51.739 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 595672 2024-09-13T10:10:51.739 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:51.739 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:10:51.535+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:10:51.742 INFO:tasks.rados.rados.0.smithi195.stdout:603: finishing write tid 1 to smithi19530714-34 2024-09-13T10:10:51.742 INFO:tasks.rados.rados.0.smithi195.stdout:603: finishing write tid 2 to smithi19530714-34 2024-09-13T10:10:51.742 INFO:tasks.rados.rados.0.smithi195.stdout:603: finishing write tid 3 to smithi19530714-34 2024-09-13T10:10:51.742 INFO:tasks.rados.rados.0.smithi195.stdout:603: finishing write tid 4 to smithi19530714-34 2024-09-13T10:10:51.742 INFO:tasks.rados.rados.0.smithi195.stdout:603: finishing write tid 5 to smithi19530714-34 2024-09-13T10:10:51.742 INFO:tasks.rados.rados.0.smithi195.stdout:603: oid 34 updating version 0 to 22 2024-09-13T10:10:51.743 INFO:tasks.rados.rados.0.smithi195.stdout:603: oid 34 version 22 is already newer than 19 2024-09-13T10:10:51.743 INFO:tasks.rados.rados.0.smithi195.stdout:603: oid 34 version 22 is already newer than 20 2024-09-13T10:10:51.743 INFO:tasks.rados.rados.0.smithi195.stdout:603: oid 34 version 22 is already newer than 21 2024-09-13T10:10:51.743 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 34 v 22 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-13T10:10:51.743 INFO:tasks.rados.rados.0.smithi195.stdout:603: left oid 34 (ObjNum 50 snap 0 seq_num 50) 2024-09-13T10:10:51.743 INFO:tasks.rados.rados.0.smithi195.stdout:604: finishing write tid 1 to smithi19530714-18 2024-09-13T10:10:51.743 INFO:tasks.rados.rados.0.smithi195.stdout:604: finishing write tid 2 to smithi19530714-18 2024-09-13T10:10:51.743 INFO:tasks.rados.rados.0.smithi195.stdout:604: finishing write tid 3 to smithi19530714-18 2024-09-13T10:10:51.743 INFO:tasks.rados.rados.0.smithi195.stdout:604: finishing write tid 4 to smithi19530714-18 2024-09-13T10:10:51.743 INFO:tasks.rados.rados.0.smithi195.stdout:604: oid 18 updating version 0 to 24 2024-09-13T10:10:51.743 INFO:tasks.rados.rados.0.smithi195.stdout:604: oid 18 updating version 24 to 26 2024-09-13T10:10:51.743 INFO:tasks.rados.rados.0.smithi195.stdout:604: oid 18 version 26 is already newer than 25 2024-09-13T10:10:51.743 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 26 (ObjNum 51 snap 0 seq_num 51) dirty exists 2024-09-13T10:10:51.744 INFO:tasks.rados.rados.0.smithi195.stdout:604: left oid 18 (ObjNum 51 snap 0 seq_num 51) 2024-09-13T10:10:51.744 INFO:tasks.rados.rados.0.smithi195.stdout:601: done (4 left) 2024-09-13T10:10:51.744 INFO:tasks.rados.rados.0.smithi195.stdout:602: done (3 left) 2024-09-13T10:10:51.744 INFO:tasks.rados.rados.0.smithi195.stdout:603: done (2 left) 2024-09-13T10:10:51.744 INFO:tasks.rados.rados.0.smithi195.stdout:604: done (1 left) 2024-09-13T10:10:51.744 INFO:tasks.rados.rados.0.smithi195.stdout:605: done (0 left) 2024-09-13T10:10:51.744 INFO:tasks.rados.rados.0.smithi195.stdout:606: write oid 17 current snap is 0 2024-09-13T10:10:51.744 INFO:tasks.rados.rados.0.smithi195.stdout:606: seq_num 53 ranges {490771=686315,1885075=406346} 2024-09-13T10:10:51.744 INFO:tasks.rados.rados.0.smithi195.stdout:606: writing smithi19530714-17 from 490771 to 1177086 tid 1 2024-09-13T10:10:51.744 INFO:tasks.rados.rados.0.smithi195.stdout:606: writing smithi19530714-17 from 1885075 to 2291421 tid 2 2024-09-13T10:10:51.744 INFO:tasks.rados.rados.0.smithi195.stdout:607: read oid 36 snap -1 2024-09-13T10:10:51.744 INFO:tasks.rados.rados.0.smithi195.stdout:607: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-13T10:10:51.744 INFO:tasks.rados.rados.0.smithi195.stdout:608: tier_promote oid 24 2024-09-13T10:10:51.745 INFO:tasks.rados.rados.0.smithi195.stdout:606: finishing write tid 1 to smithi19530714-17 2024-09-13T10:10:51.745 INFO:tasks.rados.rados.0.smithi195.stdout:609: read oid 19 snap -1 2024-09-13T10:10:51.745 INFO:tasks.rados.rados.0.smithi195.stdout:609: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-13T10:10:51.745 INFO:tasks.rados.rados.0.smithi195.stdout:610: set_chunk oid 39 target oid 2024-09-13T10:10:51.745 INFO:tasks.rados.rados.0.smithi195.stdout:610: set_chunk oid 39 offset: 315392 length: 1184256 target oid 4d6751ba15e95c360cd3577cdb9af7c034dc546188a4da3e77ae41469aed654d offset: 0 snap -1 2024-09-13T10:10:51.745 INFO:tasks.rados.rados.0.smithi195.stdout:611: write oid 29 current snap is 0 2024-09-13T10:10:51.745 INFO:tasks.rados.rados.0.smithi195.stdout:611: seq_num 54 ranges {507578=477917,1543557=550004,2833665=73913} 2024-09-13T10:10:51.745 INFO:tasks.rados.rados.0.smithi195.stdout:611: writing smithi19530714-29 from 507578 to 985495 tid 1 2024-09-13T10:10:51.745 INFO:tasks.rados.rados.0.smithi195.stdout:611: writing smithi19530714-29 from 1543557 to 2093561 tid 2 2024-09-13T10:10:51.745 INFO:tasks.rados.rados.0.smithi195.stdout:611: writing smithi19530714-29 from 2833665 to 2907578 tid 3 2024-09-13T10:10:51.745 INFO:tasks.rados.rados.0.smithi195.stdout:612: write (excl) oid 41 current snap is 0 2024-09-13T10:10:51.745 INFO:tasks.rados.rados.0.smithi195.stdout:612: seq_num 55 ranges {524385=553166,1602039=410015,2577441=346944} 2024-09-13T10:10:51.745 INFO:tasks.rados.rados.0.smithi195.stdout:612: writing smithi19530714-41 from 524385 to 1077551 tid 1 2024-09-13T10:10:51.746 INFO:tasks.rados.rados.0.smithi195.stdout:612: writing smithi19530714-41 from 1602039 to 2012054 tid 2 2024-09-13T10:10:51.746 INFO:tasks.rados.rados.0.smithi195.stdout:612: writing smithi19530714-41 from 2577441 to 2924385 tid 3 2024-09-13T10:10:51.746 INFO:tasks.rados.rados.0.smithi195.stdout:613: tier_evict oid 40 2024-09-13T10:10:51.746 INFO:tasks.rados.rados.0.smithi195.stdout:613: tier_evict oid 40 snap -1 2024-09-13T10:10:51.746 INFO:tasks.rados.rados.0.smithi195.stdout:607: expect (ObjNum 35 snap 0 seq_num 35) 2024-09-13T10:10:51.746 INFO:tasks.rados.rados.0.smithi195.stdout:606: finishing write tid 2 to smithi19530714-17 2024-09-13T10:10:51.746 INFO:tasks.rados.rados.0.smithi195.stdout:606: finishing write tid 3 to smithi19530714-17 2024-09-13T10:10:51.746 INFO:tasks.rados.rados.0.smithi195.stdout:606: finishing write tid 4 to smithi19530714-17 2024-09-13T10:10:51.746 INFO:tasks.rados.rados.0.smithi195.stdout:606: oid 17 updating version 0 to 28 2024-09-13T10:10:51.746 INFO:tasks.rados.rados.0.smithi195.stdout:606: oid 17 version 28 is already newer than 27 2024-09-13T10:10:51.746 INFO:tasks.rados.rados.0.smithi195.stdout:606: oid 17 updating version 28 to 29 2024-09-13T10:10:51.746 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 29 (ObjNum 53 snap 0 seq_num 53) dirty exists 2024-09-13T10:10:51.746 INFO:tasks.rados.rados.0.smithi195.stdout:606: left oid 17 (ObjNum 53 snap 0 seq_num 53) 2024-09-13T10:10:51.746 INFO:tasks.rados.rados.0.smithi195.stdout:609: expect (ObjNum 18 snap 0 seq_num 18) 2024-09-13T10:10:51.748 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:10:51.749 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:10:51.750 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-09-13T10:10:51.288502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7817298,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":92,"num_read_kb":76,"num_write":159,"num_write_kb":5964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":87,"acting":87,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":443772,"kb_used_data":7196,"kb_used_omap":24,"kb_used_meta":436263,"kb_avail":1509505668,"statfs":{"total":1546188226560,"available":1545733804032,"internally_reserved":0,"allocated":7368704,"data_stored":4285707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":446733391},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7226911,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":12,"num_write_kb":3290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.047011"},"pg_stats":[{"pgid":"4.f","version":"38'9","reported_seq":41,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T10:10:50.893235+0000","last_change":"2024-09-13T10:10:44.224347+0000","last_active":"2024-09-13T10:10:50.893235+0000","last_peered":"2024-09-13T10:10:50.893235+0000","last_clean":"2024-09-13T10:10:50.893235+0000","last_became_active":"2024-09-13T10:10:44.223700+0000","last_became_peered":"2024-09-13T10:10:44.223700+0000","last_unstale":"2024-09-13T10:10:50.893235+0000","last_undegraded":"2024-09-13T10:10:50.893235+0000","last_fullsized":"2024-09-13T10:10:50.893235+0000","mapping_epoch":31,"log_start":"38'6","ondisk_log_start":"38'6","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":6,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:03.714464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7226911,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":12,"num_write_kb":3290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,7],"acting":[13,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.385552+0000","last_change":"2024-09-13T10:10:44.218427+0000","last_active":"2024-09-13T10:10:47.385552+0000","last_peered":"2024-09-13T10:10:47.385552+0000","last_clean":"2024-09-13T10:10:47.385552+0000","last_became_active":"2024-09-13T10:10:44.217784+0000","last_became_peered":"2024-09-13T10:10:44.217784+0000","last_unstale":"2024-09-13T10:10:47.385552+0000","last_undegraded":"2024-09-13T10:10:47.385552+0000","last_fullsized":"2024-09-13T10:10:47.385552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:47.654438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,8],"acting":[4,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T10:10:48.247774+0000","last_change":"2024-09-13T10:10:44.613920+0000","last_active":"2024-09-13T10:10:48.247774+0000","last_peered":"2024-09-13T10:10:48.247774+0000","last_clean":"2024-09-13T10:10:48.247774+0000","last_became_active":"2024-09-13T10:10:44.613358+0000","last_became_peered":"2024-09-13T10:10:44.613358+0000","last_unstale":"2024-09-13T10:10:48.247774+0000","last_undegraded":"2024-09-13T10:10:48.247774+0000","last_fullsized":"2024-09-13T10:10:48.247774+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:09.710844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,15,7],"acting":[12,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T10:10:48.247405+0000","last_change":"2024-09-13T10:10:44.230321+0000","last_active":"2024-09-13T10:10:48.247405+0000","last_peered":"2024-09-13T10:10:48.247405+0000","last_clean":"2024-09-13T10:10:48.247405+0000","last_became_active":"2024-09-13T10:10:44.230013+0000","last_became_peered":"2024-09-13T10:10:44.230013+0000","last_unstale":"2024-09-13T10:10:48.247405+0000","last_undegraded":"2024-09-13T10:10:48.247405+0000","last_fullsized":"2024-09-13T10:10:48.247405+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:06.213121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[12,6,3],"acting":[12,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":12,"acting_primary":12,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.248141+0000","last_change":"2024-09-13T10:10:44.227804+0000","last_active":"2024-09-13T10:10:47.248141+0000","last_peered":"2024-09-13T10:10:47.248141+0000","last_clean":"2024-09-13T10:10:47.248141+0000","last_became_active":"2024-09-13T10:10:44.226956+0000","last_became_peered":"2024-09-13T10:10:44.226956+0000","last_unstale":"2024-09-13T10:10:47.248141+0000","last_undegraded":"2024-09-13T10:10:47.248141+0000","last_fullsized":"2024-09-13T10:10:47.248141+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:55.177011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,4],"acting":[14,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T10:10:48.765758+0000","last_change":"2024-09-13T10:10:44.230207+0000","last_active":"2024-09-13T10:10:48.765758+0000","last_peered":"2024-09-13T10:10:48.765758+0000","last_clean":"2024-09-13T10:10:48.765758+0000","last_became_active":"2024-09-13T10:10:44.229908+0000","last_became_peered":"2024-09-13T10:10:44.229908+0000","last_unstale":"2024-09-13T10:10:48.765758+0000","last_undegraded":"2024-09-13T10:10:48.765758+0000","last_fullsized":"2024-09-13T10:10:48.765758+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:09.108003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6,13],"acting":[8,6,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.311405+0000","last_change":"2024-09-13T10:10:44.787532+0000","last_active":"2024-09-13T10:10:47.311405+0000","last_peered":"2024-09-13T10:10:47.311405+0000","last_clean":"2024-09-13T10:10:47.311405+0000","last_became_active":"2024-09-13T10:10:44.786854+0000","last_became_peered":"2024-09-13T10:10:44.786854+0000","last_unstale":"2024-09-13T10:10:47.311405+0000","last_undegraded":"2024-09-13T10:10:47.311405+0000","last_fullsized":"2024-09-13T10:10:47.311405+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:11.801947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,9],"acting":[11,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.267306+0000","last_change":"2024-09-13T10:10:44.235409+0000","last_active":"2024-09-13T10:10:47.267306+0000","last_peered":"2024-09-13T10:10:47.267306+0000","last_clean":"2024-09-13T10:10:47.267306+0000","last_became_active":"2024-09-13T10:10:44.234463+0000","last_became_peered":"2024-09-13T10:10:44.234463+0000","last_unstale":"2024-09-13T10:10:47.267306+0000","last_undegraded":"2024-09-13T10:10:47.267306+0000","last_fullsized":"2024-09-13T10:10:47.267306+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:55.216634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,0],"acting":[3,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.768522+0000","last_change":"2024-09-13T10:10:44.269831+0000","last_active":"2024-09-13T10:10:47.768522+0000","last_peered":"2024-09-13T10:10:47.768522+0000","last_clean":"2024-09-13T10:10:47.768522+0000","last_became_active":"2024-09-13T10:10:44.268808+0000","last_became_peered":"2024-09-13T10:10:44.268808+0000","last_unstale":"2024-09-13T10:10:47.768522+0000","last_undegraded":"2024-09-13T10:10:47.768522+0000","last_fullsized":"2024-09-13T10:10:47.768522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:03.553508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,0],"acting":[6,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":37,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.768131+0000","last_change":"2024-09-13T10:10:32.149042+0000","last_active":"2024-09-13T10:10:47.768131+0000","last_peered":"2024-09-13T10:10:47.768131+0000","last_clean":"2024-09-13T10:10:47.768131+0000","last_became_active":"2024-09-13T10:10:30.272007+0000","last_became_peered":"2024-09-13T10:10:30.272007+0000","last_unstale":"2024-09-13T10:10:47.768131+0000","last_undegraded":"2024-09-13T10:10:47.768131+0000","last_fullsized":"2024-09-13T10:10:47.768131+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:49.552306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000987604,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.248017+0000","last_change":"2024-09-13T10:10:43.230767+0000","last_active":"2024-09-13T10:10:47.248017+0000","last_peered":"2024-09-13T10:10:47.248017+0000","last_clean":"2024-09-13T10:10:47.248017+0000","last_became_active":"2024-09-13T10:10:43.229961+0000","last_became_peered":"2024-09-13T10:10:43.229961+0000","last_unstale":"2024-09-13T10:10:47.248017+0000","last_undegraded":"2024-09-13T10:10:47.248017+0000","last_fullsized":"2024-09-13T10:10:47.248017+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:53.856674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.768569+0000","last_change":"2024-09-13T10:10:44.269629+0000","last_active":"2024-09-13T10:10:47.768569+0000","last_peered":"2024-09-13T10:10:47.768569+0000","last_clean":"2024-09-13T10:10:47.768569+0000","last_became_active":"2024-09-13T10:10:44.268699+0000","last_became_peered":"2024-09-13T10:10:44.268699+0000","last_unstale":"2024-09-13T10:10:47.768569+0000","last_undegraded":"2024-09-13T10:10:47.768569+0000","last_fullsized":"2024-09-13T10:10:47.768569+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:07.198621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10],"acting":[6,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":37,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.767080+0000","last_change":"2024-09-13T10:10:33.223611+0000","last_active":"2024-09-13T10:10:47.767080+0000","last_peered":"2024-09-13T10:10:47.767080+0000","last_clean":"2024-09-13T10:10:47.767080+0000","last_became_active":"2024-09-13T10:10:30.128310+0000","last_became_peered":"2024-09-13T10:10:30.128310+0000","last_unstale":"2024-09-13T10:10:47.767080+0000","last_undegraded":"2024-09-13T10:10:47.767080+0000","last_fullsized":"2024-09-13T10:10:47.767080+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:46.008600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081844899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.313272+0000","last_change":"2024-09-13T10:10:43.220955+0000","last_active":"2024-09-13T10:10:47.313272+0000","last_peered":"2024-09-13T10:10:47.313272+0000","last_clean":"2024-09-13T10:10:47.313272+0000","last_became_active":"2024-09-13T10:10:43.220074+0000","last_became_peered":"2024-09-13T10:10:43.220074+0000","last_unstale":"2024-09-13T10:10:47.313272+0000","last_undegraded":"2024-09-13T10:10:47.313272+0000","last_fullsized":"2024-09-13T10:10:47.313272+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:54.562129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.313949+0000","last_change":"2024-09-13T10:10:44.267465+0000","last_active":"2024-09-13T10:10:47.313949+0000","last_peered":"2024-09-13T10:10:47.313949+0000","last_clean":"2024-09-13T10:10:47.313949+0000","last_became_active":"2024-09-13T10:10:44.266976+0000","last_became_peered":"2024-09-13T10:10:44.266976+0000","last_unstale":"2024-09-13T10:10:47.313949+0000","last_undegraded":"2024-09-13T10:10:47.313949+0000","last_fullsized":"2024-09-13T10:10:47.313949+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:07.958549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,10],"acting":[1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":41,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T10:10:48.249528+0000","last_change":"2024-09-13T10:10:32.146042+0000","last_active":"2024-09-13T10:10:48.249528+0000","last_peered":"2024-09-13T10:10:48.249528+0000","last_clean":"2024-09-13T10:10:48.249528+0000","last_became_active":"2024-09-13T10:10:30.127232+0000","last_became_peered":"2024-09-13T10:10:30.127232+0000","last_unstale":"2024-09-13T10:10:48.249528+0000","last_undegraded":"2024-09-13T10:10:48.249528+0000","last_fullsized":"2024-09-13T10:10:48.249528+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:29.955802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034622780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.267226+0000","last_change":"2024-09-13T10:10:43.220235+0000","last_active":"2024-09-13T10:10:47.267226+0000","last_peered":"2024-09-13T10:10:47.267226+0000","last_clean":"2024-09-13T10:10:47.267226+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:10:47.267226+0000","last_undegraded":"2024-09-13T10:10:47.267226+0000","last_fullsized":"2024-09-13T10:10:47.267226+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:55.166722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.767266+0000","last_change":"2024-09-13T10:10:44.222687+0000","last_active":"2024-09-13T10:10:47.767266+0000","last_peered":"2024-09-13T10:10:47.767266+0000","last_clean":"2024-09-13T10:10:47.767266+0000","last_became_active":"2024-09-13T10:10:44.221749+0000","last_became_peered":"2024-09-13T10:10:44.221749+0000","last_unstale":"2024-09-13T10:10:47.767266+0000","last_undegraded":"2024-09-13T10:10:47.767266+0000","last_fullsized":"2024-09-13T10:10:47.767266+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:45.255506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,13,1],"acting":[7,13,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":44,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T10:10:50.265469+0000","last_change":"2024-09-13T10:10:32.702214+0000","last_active":"2024-09-13T10:10:50.265469+0000","last_peered":"2024-09-13T10:10:50.265469+0000","last_clean":"2024-09-13T10:10:50.265469+0000","last_became_active":"2024-09-13T10:10:30.124759+0000","last_became_peered":"2024-09-13T10:10:30.124759+0000","last_unstale":"2024-09-13T10:10:50.265469+0000","last_undegraded":"2024-09-13T10:10:50.265469+0000","last_fullsized":"2024-09-13T10:10:50.265469+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:45.386731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090474700000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":170,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T10:10:48.765396+0000","last_change":"2024-09-13T10:10:30.127365+0000","last_active":"2024-09-13T10:10:48.765396+0000","last_peered":"2024-09-13T10:10:48.765396+0000","last_clean":"2024-09-13T10:10:48.765396+0000","last_became_active":"2024-09-13T10:10:30.126496+0000","last_became_peered":"2024-09-13T10:10:30.126496+0000","last_unstale":"2024-09-13T10:10:48.765396+0000","last_undegraded":"2024-09-13T10:10:48.765396+0000","last_fullsized":"2024-09-13T10:10:48.765396+0000","mapping_epoch":24,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:53.401784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":26,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-13T10:10:50.265451+0000","last_change":"2024-09-13T10:10:43.231291+0000","last_active":"2024-09-13T10:10:50.265451+0000","last_peered":"2024-09-13T10:10:50.265451+0000","last_clean":"2024-09-13T10:10:50.265451+0000","last_became_active":"2024-09-13T10:10:43.230866+0000","last_became_peered":"2024-09-13T10:10:43.230866+0000","last_unstale":"2024-09-13T10:10:50.265451+0000","last_undegraded":"2024-09-13T10:10:50.265451+0000","last_fullsized":"2024-09-13T10:10:50.265451+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:54.564637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,4,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T10:10:48.249825+0000","last_change":"2024-09-13T10:10:44.218025+0000","last_active":"2024-09-13T10:10:48.249825+0000","last_peered":"2024-09-13T10:10:48.249825+0000","last_clean":"2024-09-13T10:10:48.249825+0000","last_became_active":"2024-09-13T10:10:44.216755+0000","last_became_peered":"2024-09-13T10:10:44.216755+0000","last_unstale":"2024-09-13T10:10:48.249825+0000","last_undegraded":"2024-09-13T10:10:48.249825+0000","last_fullsized":"2024-09-13T10:10:48.249825+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:53.773691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,12],"acting":[2,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.248059+0000","last_change":"2024-09-13T10:10:32.701581+0000","last_active":"2024-09-13T10:10:47.248059+0000","last_peered":"2024-09-13T10:10:47.248059+0000","last_clean":"2024-09-13T10:10:47.248059+0000","last_became_active":"2024-09-13T10:10:30.126588+0000","last_became_peered":"2024-09-13T10:10:30.126588+0000","last_unstale":"2024-09-13T10:10:47.248059+0000","last_undegraded":"2024-09-13T10:10:47.248059+0000","last_fullsized":"2024-09-13T10:10:47.248059+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:40.095103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089379399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.313074+0000","last_change":"2024-09-13T10:10:44.230767+0000","last_active":"2024-09-13T10:10:47.313074+0000","last_peered":"2024-09-13T10:10:47.313074+0000","last_clean":"2024-09-13T10:10:47.313074+0000","last_became_active":"2024-09-13T10:10:44.229839+0000","last_became_peered":"2024-09-13T10:10:44.229839+0000","last_unstale":"2024-09-13T10:10:47.313074+0000","last_undegraded":"2024-09-13T10:10:47.313074+0000","last_fullsized":"2024-09-13T10:10:47.313074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:07.564502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.303475+0000","last_change":"2024-09-13T10:10:32.177488+0000","last_active":"2024-09-13T10:10:47.303475+0000","last_peered":"2024-09-13T10:10:47.303475+0000","last_clean":"2024-09-13T10:10:47.303475+0000","last_became_active":"2024-09-13T10:10:30.128658+0000","last_became_peered":"2024-09-13T10:10:30.128658+0000","last_unstale":"2024-09-13T10:10:47.303475+0000","last_undegraded":"2024-09-13T10:10:47.303475+0000","last_fullsized":"2024-09-13T10:10:47.303475+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:50.278429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":23,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T10:10:48.249951+0000","last_change":"2024-09-13T10:10:44.223625+0000","last_active":"2024-09-13T10:10:48.249951+0000","last_peered":"2024-09-13T10:10:48.249951+0000","last_clean":"2024-09-13T10:10:48.249951+0000","last_became_active":"2024-09-13T10:10:44.222711+0000","last_became_peered":"2024-09-13T10:10:44.222711+0000","last_unstale":"2024-09-13T10:10:48.249951+0000","last_undegraded":"2024-09-13T10:10:48.249951+0000","last_fullsized":"2024-09-13T10:10:48.249951+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:49.533494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,8],"acting":[2,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":36,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.267360+0000","last_change":"2024-09-13T10:10:32.932079+0000","last_active":"2024-09-13T10:10:47.267360+0000","last_peered":"2024-09-13T10:10:47.267360+0000","last_clean":"2024-09-13T10:10:47.267360+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:10:47.267360+0000","last_undegraded":"2024-09-13T10:10:47.267360+0000","last_fullsized":"2024-09-13T10:10:47.267360+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:57.533673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-13T10:10:47.768334+0000","last_change":"2024-09-13T10:10:44.230456+0000","last_active":"2024-09-13T10:10:47.768334+0000","last_peered":"2024-09-13T10:10:47.768334+0000","last_clean":"2024-09-13T10:10:47.768334+0000","last_became_active":"2024-09-13T10:10:44.229659+0000","last_became_peered":"2024-09-13T10:10:44.229659+0000","last_unstale":"2024-09-13T10:10:47.768334+0000","last_undegraded":"2024-09-13T10:10:47.768334+0000","last_fullsized":"2024-09-13T10:10:47.768334+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:09.785651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,14],"acting":[6,8,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T10:10:48.765530+0000","last_change":"2024-09-13T10:10:32.135708+0000","last_active":"2024-09-13T10:10:48.765530+0000","last_peered":"2024-09-13T10:10:48.765530+0000","last_clean":"2024-09-13T10:10:48.765530+0000","last_became_active":"2024-09-13T10:10:30.125785+0000","last_became_peered":"2024-09-13T10:10:30.125785+0000","last_unstale":"2024-09-13T10:10:48.765530+0000","last_undegraded":"2024-09-13T10:10:48.765530+0000","last_fullsized":"2024-09-13T10:10:48.765530+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:35.380273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010905629999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":16,"stat_sum":{"num_bytes":7226911,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":12,"num_write_kb":3290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9060352,"data_stored":9032570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":48,"acting":48,"num_store_stats":16},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":11}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247815,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":159735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247817,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":364544,"data_stored":168071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247817,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":958464,"data_stored":758439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,6,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313224,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":331776,"data_stored":143182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313224,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":376832,"data_stored":176445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,3,4,6,7,8,9,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247815,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27380,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344460,"statfs":{"total":96636764160,"available":96608727040,"internally_reserved":0,"allocated":352256,"data_stored":159735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280519,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":151431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,6,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280519,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":946176,"data_stored":750103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,6,7,8,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280520,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":143163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,3,5,7,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280518,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":339968,"data_stored":151431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[3,4,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280519,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":933888,"data_stored":741799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280519,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":151431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280519,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":352256,"data_stored":159735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,5,7,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280519,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27344,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344496,"statfs":{"total":96636764160,"available":96608763904,"internally_reserved":0,"allocated":327680,"data_stored":143163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,8,9,10,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247815,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":168090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247814,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":356352,"data_stored":159754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,3,5,7,8,9,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":9060352,"data_stored":9032570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:10:51.752 INFO:tasks.ceph.ceph_manager.ceph:increase pool size by 10 2024-09-13T10:10:51.752 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pg_num 26 2024-09-13T10:10:51.752 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:10:51.742+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:10:51.761 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:51.785 INFO:tasks.rados.rados.0.smithi195.stdout:608: got (0) Success 2024-09-13T10:10:51.785 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 3 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-09-13T10:10:51.785 INFO:tasks.rados.rados.0.smithi195.stdout:610: finishing set_chunk to oid 39 2024-09-13T10:10:51.785 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 39 set_chunk returned error code -95 offset: 315392 length: 1184256 2024-09-13T10:10:51.785 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 0 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-09-13T10:10:51.786 INFO:tasks.rados.rados.0.smithi195.stdout:611: finishing write tid 1 to smithi19530714-29 2024-09-13T10:10:51.786 INFO:tasks.rados.rados.0.smithi195.stdout:611: finishing write tid 2 to smithi19530714-29 2024-09-13T10:10:51.786 INFO:tasks.rados.rados.0.smithi195.stdout:611: finishing write tid 3 to smithi19530714-29 2024-09-13T10:10:51.786 INFO:tasks.rados.rados.0.smithi195.stdout:611: finishing write tid 4 to smithi19530714-29 2024-09-13T10:10:51.786 INFO:tasks.rados.rados.0.smithi195.stdout:611: finishing write tid 5 to smithi19530714-29 2024-09-13T10:10:51.786 INFO:tasks.rados.rados.0.smithi195.stdout:611: oid 29 updating version 0 to 31 2024-09-13T10:10:51.786 INFO:tasks.rados.rados.0.smithi195.stdout:611: oid 29 updating version 31 to 33 2024-09-13T10:10:51.786 INFO:tasks.rados.rados.0.smithi195.stdout:611: oid 29 version 33 is already newer than 30 2024-09-13T10:10:51.786 INFO:tasks.rados.rados.0.smithi195.stdout:611: oid 29 version 33 is already newer than 32 2024-09-13T10:10:51.786 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 33 (ObjNum 54 snap 0 seq_num 54) dirty exists 2024-09-13T10:10:51.786 INFO:tasks.rados.rados.0.smithi195.stdout:611: left oid 29 (ObjNum 54 snap 0 seq_num 54) 2024-09-13T10:10:51.786 INFO:tasks.rados.rados.0.smithi195.stdout:612: finishing write tid 1 to smithi19530714-41 2024-09-13T10:10:51.787 INFO:tasks.rados.rados.0.smithi195.stdout:612: finishing write tid 2 to smithi19530714-41 2024-09-13T10:10:51.787 INFO:tasks.rados.rados.0.smithi195.stdout:613: got (22) Invalid argument 2024-09-13T10:10:51.787 INFO:tasks.rados.rados.0.smithi195.stdout:612: finishing write tid 3 to smithi19530714-41 2024-09-13T10:10:51.787 INFO:tasks.rados.rados.0.smithi195.stdout:612: finishing write tid 4 to smithi19530714-41 2024-09-13T10:10:51.787 INFO:tasks.rados.rados.0.smithi195.stdout:612: finishing write tid 5 to smithi19530714-41 2024-09-13T10:10:51.787 INFO:tasks.rados.rados.0.smithi195.stdout:612: oid 41 updating version 0 to 16 2024-09-13T10:10:51.787 INFO:tasks.rados.rados.0.smithi195.stdout:612: oid 41 updating version 16 to 17 2024-09-13T10:10:51.787 INFO:tasks.rados.rados.0.smithi195.stdout:612: oid 41 version 17 is already newer than 15 2024-09-13T10:10:51.787 INFO:tasks.rados.rados.0.smithi195.stdout:612: oid 41 updating version 17 to 18 2024-09-13T10:10:51.787 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 18 (ObjNum 55 snap 0 seq_num 55) dirty exists 2024-09-13T10:10:51.787 INFO:tasks.rados.rados.0.smithi195.stdout:612: left oid 41 (ObjNum 55 snap 0 seq_num 55) 2024-09-13T10:10:51.788 INFO:tasks.rados.rados.0.smithi195.stdout:606: done (7 left) 2024-09-13T10:10:51.788 INFO:tasks.rados.rados.0.smithi195.stdout:607: done (6 left) 2024-09-13T10:10:51.788 INFO:tasks.rados.rados.0.smithi195.stdout:608: done (5 left) 2024-09-13T10:10:51.788 INFO:tasks.rados.rados.0.smithi195.stdout:609: done (4 left) 2024-09-13T10:10:51.788 INFO:tasks.rados.rados.0.smithi195.stdout:610: done (3 left) 2024-09-13T10:10:51.788 INFO:tasks.rados.rados.0.smithi195.stdout:611: done (2 left) 2024-09-13T10:10:51.788 INFO:tasks.rados.rados.0.smithi195.stdout:612: done (1 left) 2024-09-13T10:10:51.788 INFO:tasks.rados.rados.0.smithi195.stdout:613: done (0 left) 2024-09-13T10:10:51.788 INFO:tasks.rados.rados.0.smithi195.stdout:614: set_chunk oid 37 target oid 2024-09-13T10:10:51.790 INFO:tasks.rados.rados.0.smithi195.stdout:614: set_chunk oid 37 offset: 2176512 length: 65536 target oid 92b194d1757ba97827f1278fdb4086faafa73237933cce7174cf64ca8160c042 offset: 0 snap -1 2024-09-13T10:10:51.790 INFO:tasks.rados.rados.0.smithi195.stdout:615: read oid 46 snap -1 2024-09-13T10:10:51.790 INFO:tasks.rados.rados.0.smithi195.stdout:615: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-13T10:10:51.790 INFO:tasks.rados.rados.0.smithi195.stdout:616: set_chunk oid 28 target oid 2024-09-13T10:10:51.798 INFO:tasks.rados.rados.0.smithi195.stdout:616: set_chunk oid 28 offset: 1760768 length: 221696 target oid dfc80f7c34ab2beac36fb23e7ea1af1f3b8dc9850c9546f20d77ccf2314fba00 offset: 0 snap -1 2024-09-13T10:10:51.798 INFO:tasks.rados.rados.0.smithi195.stdout:617: tier_flush oid 23 2024-09-13T10:10:51.798 INFO:tasks.rados.rados.0.smithi195.stdout:617: tier_flush oid 23 snap -1 2024-09-13T10:10:51.798 INFO:tasks.rados.rados.0.smithi195.stdout:618: read oid 27 snap -1 2024-09-13T10:10:51.798 INFO:tasks.rados.rados.0.smithi195.stdout:618: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-13T10:10:51.798 INFO:tasks.rados.rados.0.smithi195.stdout:619: read oid 30 snap -1 2024-09-13T10:10:51.798 INFO:tasks.rados.rados.0.smithi195.stdout:619: expect (ObjNum 29 snap 0 seq_num 29) 2024-09-13T10:10:51.798 INFO:tasks.rados.rados.0.smithi195.stdout:620: write (excl) oid 13 current snap is 0 2024-09-13T10:10:51.798 INFO:tasks.rados.rados.0.smithi195.stdout:614: finishing set_chunk to oid 37 2024-09-13T10:10:51.798 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 3 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-09-13T10:10:51.798 INFO:tasks.rados.rados.0.smithi195.stdout:620: seq_num 56 ranges {541192=628415,1660521=553673,2888511=52681} 2024-09-13T10:10:51.817 INFO:tasks.rados.rados.0.smithi195.stdout:620: writing smithi19530714-13 from 541192 to 1169607 tid 1 2024-09-13T10:10:51.818 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:10:51.834 INFO:tasks.rados.rados.0.smithi195.stdout:620: writing smithi19530714-13 from 1660521 to 2214194 tid 2 2024-09-13T10:10:51.842 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:10:51.836+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:10:51.845 INFO:tasks.rados.rados.0.smithi195.stdout:620: writing smithi19530714-13 from 2888511 to 2941192 tid 3 2024-09-13T10:10:51.845 INFO:tasks.rados.rados.0.smithi195.stdout:614: done (6 left) 2024-09-13T10:10:51.845 INFO:tasks.rados.rados.0.smithi195.stdout:621: write (excl) oid 31 current snap is 0 2024-09-13T10:10:51.845 INFO:tasks.rados.rados.0.smithi195.stdout:621: seq_num 57 ranges {557999=703664,1719003=413684,2632287=325712} 2024-09-13T10:10:51.862 INFO:tasks.rados.rados.0.smithi195.stdout:621: writing smithi19530714-31 from 557999 to 1261663 tid 1 2024-09-13T10:10:51.873 INFO:tasks.rados.rados.0.smithi195.stdout:621: writing smithi19530714-31 from 1719003 to 2132687 tid 2 2024-09-13T10:10:51.874 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:10:51.874 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:10:51.875 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:10:51.875 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:10:51.883 INFO:tasks.rados.rados.0.smithi195.stdout:621: writing smithi19530714-31 from 2632287 to 2957999 tid 3 2024-09-13T10:10:51.883 INFO:tasks.rados.rados.0.smithi195.stdout:622: write (excl) oid 4 current snap is 0 2024-09-13T10:10:51.883 INFO:tasks.rados.rados.0.smithi195.stdout:622: seq_num 58 ranges {574806=778913,2061132=557342} 2024-09-13T10:10:51.890 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-13T10:10:51.896 INFO:tasks.rados.rados.0.smithi195.stdout:622: writing smithi19530714-4 from 574806 to 1353719 tid 1 2024-09-13T10:10:51.909 INFO:tasks.rados.rados.0.smithi195.stdout:622: writing smithi19530714-4 from 2061132 to 2618474 tid 2 2024-09-13T10:10:51.909 INFO:tasks.rados.rados.0.smithi195.stdout:623: write oid 45 current snap is 0 2024-09-13T10:10:51.909 INFO:tasks.rados.rados.0.smithi195.stdout:623: seq_num 59 ranges {591613=454162,1719614=417353,2570780=420833} 2024-09-13T10:10:51.918 INFO:tasks.rados.rados.0.smithi195.stdout:623: writing smithi19530714-45 from 591613 to 1045775 tid 1 2024-09-13T10:10:51.927 INFO:tasks.rados.rados.0.smithi195.stdout:623: writing smithi19530714-45 from 1719614 to 2136967 tid 2 2024-09-13T10:10:51.933 INFO:tasks.rados.rados.0.smithi195.stdout:623: writing smithi19530714-45 from 2570780 to 2991613 tid 3 2024-09-13T10:10:51.933 INFO:tasks.rados.rados.0.smithi195.stdout:624: read oid 26 snap -1 2024-09-13T10:10:51.933 INFO:tasks.rados.rados.0.smithi195.stdout:624: expect (ObjNum 25 snap 0 seq_num 25) 2024-09-13T10:10:51.933 INFO:tasks.rados.rados.0.smithi195.stdout:625: write (excl) oid 3 current snap is 0 2024-09-13T10:10:51.933 INFO:tasks.rados.rados.0.smithi195.stdout:625: seq_num 60 ranges {608420=529411,1778096=561011,2881850=126570} 2024-09-13T10:10:51.940 INFO:tasks.rados.rados.0.smithi195.stdout:625: writing smithi19530714-3 from 608420 to 1137831 tid 1 2024-09-13T10:10:51.943 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:10:51.936+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:10:51.948 INFO:tasks.rados.rados.0.smithi195.stdout:625: writing smithi19530714-3 from 1778096 to 2339107 tid 2 2024-09-13T10:10:51.952 INFO:tasks.rados.rados.0.smithi195.stdout:625: writing smithi19530714-3 from 2881850 to 3008420 tid 3 2024-09-13T10:10:51.952 INFO:tasks.rados.rados.0.smithi195.stdout:626: delete oid 7 current snap is 0 2024-09-13T10:10:51.952 INFO:tasks.rados.rados.0.smithi195.stdout:616: finishing set_chunk to oid 28 2024-09-13T10:10:51.952 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 28 set_chunk returned error code -95 offset: 1760768 length: 221696 2024-09-13T10:10:51.952 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 0 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-09-13T10:10:51.952 INFO:tasks.rados.rados.0.smithi195.stdout:615: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-13T10:10:51.952 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:51.984 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:10:51.985 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:10:52.006 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:10:52.006 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:10:52.019 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:10:52.020 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:10:52.020 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:10:52.020 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:10:52.020 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:10:52.034 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-13T10:10:52.040 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:10:52.041 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:10:52.052 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:52.075 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:10:52.076 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:10:52.080 INFO:tasks.rados.rados.0.smithi195.stdout:618: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-13T10:10:52.083 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:52.081+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:10:52.092 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:10:52.092 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:10:52.105 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:52.110 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:10:52.110 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:10:52.130 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 20264 2024-09-13T10:10:52.131 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:10:52.131 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:10:52.144 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:52.142+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:10:52.151 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:10:52.151 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:10:52.155 INFO:tasks.rados.rados.0.smithi195.stdout:619: expect (ObjNum 29 snap 0 seq_num 29) 2024-09-13T10:10:52.161 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:10:52.161 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:10:52.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:10:52.161 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:10:52.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.161 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4365.0:5 1.2 e26/24 1:5cea7035:::rbd_info:head v 26'1, mlcod=0'0)", 2024-09-13T10:10:52.161 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:31.142893+0000", 2024-09-13T10:10:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.015863038999999, 2024-09-13T10:10:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024212330000000001, 2024-09-13T10:10:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142893+0000", 2024-09-13T10:10:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142893+0000", 2024-09-13T10:10:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142893+0000", 2024-09-13T10:10:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1699999999999998e-07 2024-09-13T10:10:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142914+0000", 2024-09-13T10:10:52.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0378999999999999e-05 2024-09-13T10:10:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142951+0000", 2024-09-13T10:10:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7471000000000001e-05 2024-09-13T10:10:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142983+0000", 2024-09-13T10:10:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2422000000000002e-05 2024-09-13T10:10:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.143202+0000", 2024-09-13T10:10:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021875799999999999 2024-09-13T10:10:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.143308+0000", 2024-09-13T10:10:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105487 2024-09-13T10:10:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.145207+0000", 2024-09-13T10:10:52.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001899848 2024-09-13T10:10:52.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.145314+0000", 2024-09-13T10:10:52.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106351 2024-09-13T10:10:52.166 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.166 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.166 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.166 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4365.0:7 1.2 e27/24 1:5cea7035:::rbd_info:head v 27'2, mlcod=0'0)", 2024-09-13T10:10:52.166 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:32.145197+0000", 2024-09-13T10:10:52.167 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.013559051000001, 2024-09-13T10:10:52.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020358360000000001, 2024-09-13T10:10:52.167 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.167 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.167 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.167 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145197+0000", 2024-09-13T10:10:52.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145197+0000", 2024-09-13T10:10:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145197+0000", 2024-09-13T10:10:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5000000000000002e-07 2024-09-13T10:10:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145212+0000", 2024-09-13T10:10:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4619e-05 2024-09-13T10:10:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145241+0000", 2024-09-13T10:10:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8676000000000001e-05 2024-09-13T10:10:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145257+0000", 2024-09-13T10:10:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6155e-05 2024-09-13T10:10:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145397+0000", 2024-09-13T10:10:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014000199999999999 2024-09-13T10:10:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145437+0000", 2024-09-13T10:10:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0615000000000003e-05 2024-09-13T10:10:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.147104+0000", 2024-09-13T10:10:52.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016668259999999999 2024-09-13T10:10:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.147232+0000", 2024-09-13T10:10:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000128293 2024-09-13T10:10:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:89 4.9 e38/31 4:96fcf081:::smithi19530714-23:head v 38'1, mlcod=0'0)", 2024-09-13T10:10:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.811705+0000", 2024-09-13T10:10:52.171 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.3470507940000001, 2024-09-13T10:10:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053533510000000001, 2024-09-13T10:10:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.811705+0000", 2024-09-13T10:10:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.811705+0000", 2024-09-13T10:10:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.811706+0000", 2024-09-13T10:10:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7700000000000003e-07 2024-09-13T10:10:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813717+0000", 2024-09-13T10:10:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002011494 2024-09-13T10:10:52.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813743+0000", 2024-09-13T10:10:52.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5528999999999999e-05 2024-09-13T10:10:52.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813759+0000", 2024-09-13T10:10:52.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6178999999999999e-05 2024-09-13T10:10:52.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813873+0000", 2024-09-13T10:10:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113863 2024-09-13T10:10:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813922+0000", 2024-09-13T10:10:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9558999999999998e-05 2024-09-13T10:10:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816994+0000", 2024-09-13T10:10:52.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003071299 2024-09-13T10:10:52.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817058+0000", 2024-09-13T10:10:52.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4550999999999999e-05 2024-09-13T10:10:52.176 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.176 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.176 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.177 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:90 4.9 e38/31 4:96fcf081:::smithi19530714-23:head v 38'2, mlcod=0'0)", 2024-09-13T10:10:52.177 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.815985+0000", 2024-09-13T10:10:52.177 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.342770459, 2024-09-13T10:10:52.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00489822, 2024-09-13T10:10:52.177 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.177 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.177 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.177 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815985+0000", 2024-09-13T10:10:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815985+0000", 2024-09-13T10:10:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815987+0000", 2024-09-13T10:10:52.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.277e-06 2024-09-13T10:10:52.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817028+0000", 2024-09-13T10:10:52.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001040664 2024-09-13T10:10:52.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817055+0000", 2024-09-13T10:10:52.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6835999999999999e-05 2024-09-13T10:10:52.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817074+0000", 2024-09-13T10:10:52.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8629000000000001e-05 2024-09-13T10:10:52.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817153+0000", 2024-09-13T10:10:52.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9794000000000002e-05 2024-09-13T10:10:52.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817183+0000", 2024-09-13T10:10:52.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9479e-05 2024-09-13T10:10:52.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820783+0000", 2024-09-13T10:10:52.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035997720000000002 2024-09-13T10:10:52.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820883+0000", 2024-09-13T10:10:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100769 2024-09-13T10:10:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:91 4.9 e38/31 4:96fcf081:::smithi19530714-23:head v 38'3, mlcod=0'0)", 2024-09-13T10:10:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.817117+0000", 2024-09-13T10:10:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.341638254, 2024-09-13T10:10:52.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038490590000000002, 2024-09-13T10:10:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817117+0000", 2024-09-13T10:10:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817117+0000", 2024-09-13T10:10:52.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817119+0000", 2024-09-13T10:10:52.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1999999999999999e-06 2024-09-13T10:10:52.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817130+0000", 2024-09-13T10:10:52.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1313e-05 2024-09-13T10:10:52.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817145+0000", 2024-09-13T10:10:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4786e-05 2024-09-13T10:10:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817157+0000", 2024-09-13T10:10:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2775999999999999e-05 2024-09-13T10:10:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818394+0000", 2024-09-13T10:10:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012367999999999999 2024-09-13T10:10:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818496+0000", 2024-09-13T10:10:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102098 2024-09-13T10:10:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820895+0000", 2024-09-13T10:10:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023989189999999998 2024-09-13T10:10:52.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820966+0000", 2024-09-13T10:10:52.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1167000000000005e-05 2024-09-13T10:10:52.188 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.188 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.188 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.188 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:129 4.9 e38/31 4:998d9bf1:::smithi19530714-33:head v 38'4, mlcod=0'0)", 2024-09-13T10:10:52.188 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.817768+0000", 2024-09-13T10:10:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.3409874500000001, 2024-09-13T10:10:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047213589999999996, 2024-09-13T10:10:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817768+0000", 2024-09-13T10:10:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817768+0000", 2024-09-13T10:10:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817769+0000", 2024-09-13T10:10:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.184e-06 2024-09-13T10:10:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818638+0000", 2024-09-13T10:10:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086842199999999999 2024-09-13T10:10:52.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818665+0000", 2024-09-13T10:10:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6979e-05 2024-09-13T10:10:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818685+0000", 2024-09-13T10:10:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0063000000000001e-05 2024-09-13T10:10:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818994+0000", 2024-09-13T10:10:52.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030959500000000002 2024-09-13T10:10:52.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819019+0000", 2024-09-13T10:10:52.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4508999999999999e-05 2024-09-13T10:10:52.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822389+0000", 2024-09-13T10:10:52.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033701970000000001 2024-09-13T10:10:52.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822490+0000", 2024-09-13T10:10:52.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010041 2024-09-13T10:10:52.194 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.194 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.195 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.195 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:130 4.9 e38/31 4:998d9bf1:::smithi19530714-33:head v 38'5, mlcod=0'0)", 2024-09-13T10:10:52.195 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.819657+0000", 2024-09-13T10:10:52.195 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.3390984859999999, 2024-09-13T10:10:52.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032383609999999999, 2024-09-13T10:10:52.196 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.196 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.196 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.196 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819657+0000", 2024-09-13T10:10:52.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819657+0000", 2024-09-13T10:10:52.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819659+0000", 2024-09-13T10:10:52.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.624e-06 2024-09-13T10:10:52.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820879+0000", 2024-09-13T10:10:52.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001220184 2024-09-13T10:10:52.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820906+0000", 2024-09-13T10:10:52.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6536000000000001e-05 2024-09-13T10:10:52.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820930+0000", 2024-09-13T10:10:52.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4552000000000001e-05 2024-09-13T10:10:52.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821017+0000", 2024-09-13T10:10:52.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6539999999999995e-05 2024-09-13T10:10:52.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821043+0000", 2024-09-13T10:10:52.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6018999999999999e-05 2024-09-13T10:10:52.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822848+0000", 2024-09-13T10:10:52.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001805673 2024-09-13T10:10:52.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822896+0000", 2024-09-13T10:10:52.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7233e-05 2024-09-13T10:10:52.206 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.206 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.206 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.207 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:131 4.9 e38/31 4:998d9bf1:::smithi19530714-33:head v 38'6, mlcod=0'0)", 2024-09-13T10:10:52.207 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.821886+0000", 2024-09-13T10:10:52.207 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.336870016, 2024-09-13T10:10:52.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783485, 2024-09-13T10:10:52.207 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.208 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.208 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.208 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821886+0000", 2024-09-13T10:10:52.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821886+0000", 2024-09-13T10:10:52.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821887+0000", 2024-09-13T10:10:52.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4559999999999999e-06 2024-09-13T10:10:52.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821897+0000", 2024-09-13T10:10:52.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5659999999999996e-06 2024-09-13T10:10:52.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821918+0000", 2024-09-13T10:10:52.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1223e-05 2024-09-13T10:10:52.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821935+0000", 2024-09-13T10:10:52.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7255999999999999e-05 2024-09-13T10:10:52.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822113+0000", 2024-09-13T10:10:52.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177808 2024-09-13T10:10:52.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822139+0000", 2024-09-13T10:10:52.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5728000000000002e-05 2024-09-13T10:10:52.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.823606+0000", 2024-09-13T10:10:52.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014676159999999999 2024-09-13T10:10:52.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.823669+0000", 2024-09-13T10:10:52.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2831999999999997e-05 2024-09-13T10:10:52.217 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.217 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.218 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.218 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:185 4.9 e38/31 4:9bec18ef:::smithi19530714-47:head v 38'7, mlcod=38'5)", 2024-09-13T10:10:52.218 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.941317+0000", 2024-09-13T10:10:52.219 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.217438856, 2024-09-13T10:10:52.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002782858, 2024-09-13T10:10:52.219 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.219 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.219 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.220 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.941317+0000", 2024-09-13T10:10:52.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.941317+0000", 2024-09-13T10:10:52.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.941319+0000", 2024-09-13T10:10:52.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9860000000000001e-06 2024-09-13T10:10:52.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942196+0000", 2024-09-13T10:10:52.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087739999999999997 2024-09-13T10:10:52.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942230+0000", 2024-09-13T10:10:52.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4118000000000001e-05 2024-09-13T10:10:52.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942252+0000", 2024-09-13T10:10:52.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2030000000000001e-05 2024-09-13T10:10:52.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942365+0000", 2024-09-13T10:10:52.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112689 2024-09-13T10:10:52.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942398+0000", 2024-09-13T10:10:52.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2897e-05 2024-09-13T10:10:52.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.944044+0000", 2024-09-13T10:10:52.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016460800000000001 2024-09-13T10:10:52.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.944100+0000", 2024-09-13T10:10:52.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5658000000000001e-05 2024-09-13T10:10:52.228 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.228 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.228 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.229 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:186 4.9 e38/31 4:9bec18ef:::smithi19530714-47:head v 38'8, mlcod=38'6)", 2024-09-13T10:10:52.229 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.948439+0000", 2024-09-13T10:10:52.229 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.2103167830000001, 2024-09-13T10:10:52.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002770018, 2024-09-13T10:10:52.229 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.229 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.229 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.948439+0000", 2024-09-13T10:10:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.948439+0000", 2024-09-13T10:10:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.948440+0000", 2024-09-13T10:10:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4440000000000001e-06 2024-09-13T10:10:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949519+0000", 2024-09-13T10:10:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010790039999999999 2024-09-13T10:10:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949537+0000", 2024-09-13T10:10:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7312e-05 2024-09-13T10:10:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949545+0000", 2024-09-13T10:10:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2570000000000007e-06 2024-09-13T10:10:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949615+0000", 2024-09-13T10:10:52.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9614000000000001e-05 2024-09-13T10:10:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949633+0000", 2024-09-13T10:10:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8640000000000001e-05 2024-09-13T10:10:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.951146+0000", 2024-09-13T10:10:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015129570000000001 2024-09-13T10:10:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.951209+0000", 2024-09-13T10:10:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.279e-05 2024-09-13T10:10:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:187 4.9 e38/31 4:9bec18ef:::smithi19530714-47:head v 38'9, mlcod=38'6)", 2024-09-13T10:10:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.950298+0000", 2024-09-13T10:10:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.2084572330000001, 2024-09-13T10:10:52.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013826859999999999, 2024-09-13T10:10:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.950298+0000", 2024-09-13T10:10:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.950298+0000", 2024-09-13T10:10:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.950300+0000", 2024-09-13T10:10:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5179999999999999e-06 2024-09-13T10:10:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.950309+0000", 2024-09-13T10:10:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9209999999999998e-06 2024-09-13T10:10:52.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.950327+0000", 2024-09-13T10:10:52.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8247e-05 2024-09-13T10:10:52.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.950336+0000", 2024-09-13T10:10:52.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7309999999999999e-06 2024-09-13T10:10:52.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.950457+0000", 2024-09-13T10:10:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000121116 2024-09-13T10:10:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.950511+0000", 2024-09-13T10:10:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4489e-05 2024-09-13T10:10:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.951629+0000", 2024-09-13T10:10:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011179880000000001 2024-09-13T10:10:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.951681+0000", 2024-09-13T10:10:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1675999999999998e-05 2024-09-13T10:10:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:272 4.9 e38/31 4:96fcf081:::smithi19530714-23:head v 38'10, mlcod=38'8)", 2024-09-13T10:10:52.239 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.234227+0000", 2024-09-13T10:10:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.92452881399999998, 2024-09-13T10:10:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081511400000000003, 2024-09-13T10:10:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.234227+0000", 2024-09-13T10:10:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.234227+0000", 2024-09-13T10:10:52.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.234229+0000", 2024-09-13T10:10:52.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2970000000000002e-06 2024-09-13T10:10:52.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.234243+0000", 2024-09-13T10:10:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3539e-05 2024-09-13T10:10:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.234281+0000", 2024-09-13T10:10:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8133000000000002e-05 2024-09-13T10:10:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.234304+0000", 2024-09-13T10:10:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2949e-05 2024-09-13T10:10:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.234380+0000", 2024-09-13T10:10:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6013999999999994e-05 2024-09-13T10:10:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.234400+0000", 2024-09-13T10:10:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0488000000000001e-05 2024-09-13T10:10:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.234942+0000", 2024-09-13T10:10:52.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054193999999999998 2024-09-13T10:10:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.235042+0000", 2024-09-13T10:10:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9753999999999994e-05 2024-09-13T10:10:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:282 4.9 e38/31 4:998d9bf1:::smithi19530714-33:head v 38'11, mlcod=38'9)", 2024-09-13T10:10:52.244 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.261272+0000", 2024-09-13T10:10:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.89748412200000005, 2024-09-13T10:10:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016402599999999999, 2024-09-13T10:10:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261272+0000", 2024-09-13T10:10:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261272+0000", 2024-09-13T10:10:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261274+0000", 2024-09-13T10:10:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2800000000000002e-06 2024-09-13T10:10:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261290+0000", 2024-09-13T10:10:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5923e-05 2024-09-13T10:10:52.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261321+0000", 2024-09-13T10:10:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1211999999999999e-05 2024-09-13T10:10:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261341+0000", 2024-09-13T10:10:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9853000000000001e-05 2024-09-13T10:10:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261506+0000", 2024-09-13T10:10:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016557799999999999 2024-09-13T10:10:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261554+0000", 2024-09-13T10:10:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7292000000000001e-05 2024-09-13T10:10:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.262796+0000", 2024-09-13T10:10:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012426220000000001 2024-09-13T10:10:52.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.262912+0000", 2024-09-13T10:10:52.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0001155 2024-09-13T10:10:52.249 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.249 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.249 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.249 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:296 4.9 e38/31 4:9bec18ef:::smithi19530714-47:head v 38'12, mlcod=38'10)", 2024-09-13T10:10:52.249 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.270077+0000", 2024-09-13T10:10:52.249 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.88867890199999999, 2024-09-13T10:10:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012488670000000001, 2024-09-13T10:10:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270077+0000", 2024-09-13T10:10:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270077+0000", 2024-09-13T10:10:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270078+0000", 2024-09-13T10:10:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.37e-06 2024-09-13T10:10:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270086+0000", 2024-09-13T10:10:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2719999999999997e-06 2024-09-13T10:10:52.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270101+0000", 2024-09-13T10:10:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4868999999999999e-05 2024-09-13T10:10:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270108+0000", 2024-09-13T10:10:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2899999999999999e-06 2024-09-13T10:10:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270167+0000", 2024-09-13T10:10:52.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9219999999999999e-05 2024-09-13T10:10:52.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270189+0000", 2024-09-13T10:10:52.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2317999999999999e-05 2024-09-13T10:10:52.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.271245+0000", 2024-09-13T10:10:52.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001056218 2024-09-13T10:10:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.271326+0000", 2024-09-13T10:10:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0309999999999995e-05 2024-09-13T10:10:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:322 4.9 e38/31 4:96fcf081:::smithi19530714-23:head v 38'13, mlcod=38'11)", 2024-09-13T10:10:52.254 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.285293+0000", 2024-09-13T10:10:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.87346234700000003, 2024-09-13T10:10:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00075994099999999998, 2024-09-13T10:10:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285293+0000", 2024-09-13T10:10:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285293+0000", 2024-09-13T10:10:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285294+0000", 2024-09-13T10:10:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.093e-06 2024-09-13T10:10:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285304+0000", 2024-09-13T10:10:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1099999999999992e-06 2024-09-13T10:10:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285319+0000", 2024-09-13T10:10:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5051e-05 2024-09-13T10:10:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285328+0000", 2024-09-13T10:10:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2040000000000003e-06 2024-09-13T10:10:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285404+0000", 2024-09-13T10:10:52.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6067000000000002e-05 2024-09-13T10:10:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285424+0000", 2024-09-13T10:10:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0063000000000001e-05 2024-09-13T10:10:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.286007+0000", 2024-09-13T10:10:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00058337899999999997 2024-09-13T10:10:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.286053+0000", 2024-09-13T10:10:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5973999999999997e-05 2024-09-13T10:10:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:346 4.9 e38/31 4:9bec18ef:::smithi19530714-47:head v 38'14, mlcod=38'12)", 2024-09-13T10:10:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.292966+0000", 2024-09-13T10:10:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.86578971999999998, 2024-09-13T10:10:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085901, 2024-09-13T10:10:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.259 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.292966+0000", 2024-09-13T10:10:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.292966+0000", 2024-09-13T10:10:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.292968+0000", 2024-09-13T10:10:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7969999999999999e-06 2024-09-13T10:10:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.292976+0000", 2024-09-13T10:10:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2379999999999997e-06 2024-09-13T10:10:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.292993+0000", 2024-09-13T10:10:52.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6784999999999999e-05 2024-09-13T10:10:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.293003+0000", 2024-09-13T10:10:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0658e-05 2024-09-13T10:10:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.293081+0000", 2024-09-13T10:10:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8028000000000006e-05 2024-09-13T10:10:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.293108+0000", 2024-09-13T10:10:52.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6299e-05 2024-09-13T10:10:52.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.293784+0000", 2024-09-13T10:10:52.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067593600000000003 2024-09-13T10:10:52.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.293825+0000", 2024-09-13T10:10:52.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1269000000000001e-05 2024-09-13T10:10:52.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:382 4.9 e39/31 4:998d9bf1:::smithi19530714-33:head v 39'16, mlcod=38'14)", 2024-09-13T10:10:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.325108+0000", 2024-09-13T10:10:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.83364809299999998, 2024-09-13T10:10:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017053599999999999, 2024-09-13T10:10:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.264 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.325108+0000", 2024-09-13T10:10:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.325108+0000", 2024-09-13T10:10:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.325110+0000", 2024-09-13T10:10:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.227e-06 2024-09-13T10:10:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.325121+0000", 2024-09-13T10:10:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0917999999999999e-05 2024-09-13T10:10:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.325144+0000", 2024-09-13T10:10:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3184e-05 2024-09-13T10:10:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.325156+0000", 2024-09-13T10:10:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2241e-05 2024-09-13T10:10:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.325248+0000", 2024-09-13T10:10:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2248000000000002e-05 2024-09-13T10:10:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.325271+0000", 2024-09-13T10:10:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2682999999999999e-05 2024-09-13T10:10:52.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326761+0000", 2024-09-13T10:10:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001489588 2024-09-13T10:10:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326813+0000", 2024-09-13T10:10:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2271000000000001e-05 2024-09-13T10:10:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:396 4.9 e39/31 4:9bec18ef:::smithi19530714-47:head v 39'17, mlcod=38'14)", 2024-09-13T10:10:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.326336+0000", 2024-09-13T10:10:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.83241979300000002, 2024-09-13T10:10:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00090133900000000002, 2024-09-13T10:10:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326336+0000", 2024-09-13T10:10:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326336+0000", 2024-09-13T10:10:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326337+0000", 2024-09-13T10:10:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6499999999999998e-07 2024-09-13T10:10:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326343+0000", 2024-09-13T10:10:52.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2024-09-13T10:10:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326361+0000", 2024-09-13T10:10:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7750000000000001e-05 2024-09-13T10:10:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326371+0000", 2024-09-13T10:10:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0023999999999999e-05 2024-09-13T10:10:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326439+0000", 2024-09-13T10:10:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7812999999999996e-05 2024-09-13T10:10:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326480+0000", 2024-09-13T10:10:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0778000000000002e-05 2024-09-13T10:10:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.327179+0000", 2024-09-13T10:10:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00069924099999999997 2024-09-13T10:10:52.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.327237+0000", 2024-09-13T10:10:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8103000000000003e-05 2024-09-13T10:10:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:482 4.9 e39/31 4:998d9bf1:::smithi19530714-33:head v 39'18, mlcod=39'16)", 2024-09-13T10:10:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.347448+0000", 2024-09-13T10:10:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.81130772200000001, 2024-09-13T10:10:52.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019968960000000002, 2024-09-13T10:10:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.347448+0000", 2024-09-13T10:10:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.347448+0000", 2024-09-13T10:10:52.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.347450+0000", 2024-09-13T10:10:52.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6670000000000001e-06 2024-09-13T10:10:52.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.347459+0000", 2024-09-13T10:10:52.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9560000000000003e-06 2024-09-13T10:10:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.347477+0000", 2024-09-13T10:10:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8101000000000002e-05 2024-09-13T10:10:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.347486+0000", 2024-09-13T10:10:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7729999999999994e-06 2024-09-13T10:10:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.347559+0000", 2024-09-13T10:10:52.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2569000000000003e-05 2024-09-13T10:10:52.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.347581+0000", 2024-09-13T10:10:52.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1965000000000001e-05 2024-09-13T10:10:52.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.349412+0000", 2024-09-13T10:10:52.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018305420000000001 2024-09-13T10:10:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.349445+0000", 2024-09-13T10:10:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3322999999999999e-05 2024-09-13T10:10:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.11.0:18 4.9 e39/31 4:96fcf081:::smithi19530714-23:head v 39'19, mlcod=39'17)", 2024-09-13T10:10:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.854654+0000", 2024-09-13T10:10:52.278 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.30410214099999999, 2024-09-13T10:10:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016285239999999999, 2024-09-13T10:10:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.854654+0000", 2024-09-13T10:10:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.854654+0000", 2024-09-13T10:10:52.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.854656+0000", 2024-09-13T10:10:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.137e-06 2024-09-13T10:10:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.854668+0000", 2024-09-13T10:10:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2216e-05 2024-09-13T10:10:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:52.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.854694+0000", 2024-09-13T10:10:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5772999999999999e-05 2024-09-13T10:10:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.854745+0000", 2024-09-13T10:10:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1103999999999997e-05 2024-09-13T10:10:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.854906+0000", 2024-09-13T10:10:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000161609 2024-09-13T10:10:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:52.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.854961+0000", 2024-09-13T10:10:52.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5043999999999997e-05 2024-09-13T10:10:52.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:52.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.856213+0000", 2024-09-13T10:10:52.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001251589 2024-09-13T10:10:52.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:52.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:52.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:52.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.856282+0000", 2024-09-13T10:10:52.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9052000000000002e-05 2024-09-13T10:10:52.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.283 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:52.283 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:52.283 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:10:52.284 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:10:52.285 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:10:52.285 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:52.286 INFO:tasks.rados.rados.0.smithi195.stdout:620: finishing write tid 1 to smithi19530714-13 2024-09-13T10:10:52.286 INFO:tasks.rados.rados.0.smithi195.stdout:620: finishing write tid 2 to smithi19530714-13 2024-09-13T10:10:52.286 INFO:tasks.rados.rados.0.smithi195.stdout:620: finishing write tid 3 to smithi19530714-13 2024-09-13T10:10:52.286 INFO:tasks.rados.rados.0.smithi195.stdout:620: finishing write tid 4 to smithi19530714-13 2024-09-13T10:10:52.286 INFO:tasks.rados.rados.0.smithi195.stdout:620: finishing write tid 5 to smithi19530714-13 2024-09-13T10:10:52.286 INFO:tasks.rados.rados.0.smithi195.stdout:620: oid 13 updating version 0 to 24 2024-09-13T10:10:52.286 INFO:tasks.rados.rados.0.smithi195.stdout:620: oid 13 updating version 24 to 25 2024-09-13T10:10:52.286 INFO:tasks.rados.rados.0.smithi195.stdout:620: oid 13 updating version 25 to 26 2024-09-13T10:10:52.286 INFO:tasks.rados.rados.0.smithi195.stdout:620: oid 13 version 26 is already newer than 23 2024-09-13T10:10:52.286 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 26 (ObjNum 56 snap 0 seq_num 56) dirty exists 2024-09-13T10:10:52.286 INFO:tasks.rados.rados.0.smithi195.stdout:620: left oid 13 (ObjNum 56 snap 0 seq_num 56) 2024-09-13T10:10:52.286 INFO:tasks.rados.rados.0.smithi195.stdout:617: got (0) Success 2024-09-13T10:10:52.287 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 0 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-09-13T10:10:52.287 INFO:tasks.rados.rados.0.smithi195.stdout:621: finishing write tid 1 to smithi19530714-31 2024-09-13T10:10:52.287 INFO:tasks.rados.rados.0.smithi195.stdout:621: finishing write tid 2 to smithi19530714-31 2024-09-13T10:10:52.287 INFO:tasks.rados.rados.0.smithi195.stdout:621: finishing write tid 3 to smithi19530714-31 2024-09-13T10:10:52.287 INFO:tasks.rados.rados.0.smithi195.stdout:621: finishing write tid 4 to smithi19530714-31 2024-09-13T10:10:52.287 INFO:tasks.rados.rados.0.smithi195.stdout:621: finishing write tid 5 to smithi19530714-31 2024-09-13T10:10:52.287 INFO:tasks.rados.rados.0.smithi195.stdout:621: oid 31 updating version 0 to 19 2024-09-13T10:10:52.287 INFO:tasks.rados.rados.0.smithi195.stdout:621: oid 31 updating version 19 to 20 2024-09-13T10:10:52.287 INFO:tasks.rados.rados.0.smithi195.stdout:621: oid 31 version 20 is already newer than 18 2024-09-13T10:10:52.287 INFO:tasks.rados.rados.0.smithi195.stdout:621: oid 31 updating version 20 to 21 2024-09-13T10:10:52.287 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 21 (ObjNum 57 snap 0 seq_num 57) dirty exists 2024-09-13T10:10:52.287 INFO:tasks.rados.rados.0.smithi195.stdout:621: left oid 31 (ObjNum 57 snap 0 seq_num 57) 2024-09-13T10:10:52.287 INFO:tasks.rados.rados.0.smithi195.stdout:622: finishing write tid 1 to smithi19530714-4 2024-09-13T10:10:52.288 INFO:tasks.rados.rados.0.smithi195.stdout:622: finishing write tid 2 to smithi19530714-4 2024-09-13T10:10:52.288 INFO:tasks.rados.rados.0.smithi195.stdout:622: finishing write tid 3 to smithi19530714-4 2024-09-13T10:10:52.288 INFO:tasks.rados.rados.0.smithi195.stdout:622: finishing write tid 4 to smithi19530714-4 2024-09-13T10:10:52.288 INFO:tasks.rados.rados.0.smithi195.stdout:622: oid 4 updating version 0 to 32 2024-09-13T10:10:52.288 INFO:tasks.rados.rados.0.smithi195.stdout:622: oid 4 version 32 is already newer than 31 2024-09-13T10:10:52.288 INFO:tasks.rados.rados.0.smithi195.stdout:622: oid 4 version 32 is already newer than 30 2024-09-13T10:10:52.288 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 32 (ObjNum 58 snap 0 seq_num 58) dirty exists 2024-09-13T10:10:52.288 INFO:tasks.rados.rados.0.smithi195.stdout:622: left oid 4 (ObjNum 58 snap 0 seq_num 58) 2024-09-13T10:10:52.288 INFO:tasks.rados.rados.0.smithi195.stdout:623: finishing write tid 1 to smithi19530714-45 2024-09-13T10:10:52.288 INFO:tasks.rados.rados.0.smithi195.stdout:623: finishing write tid 2 to smithi19530714-45 2024-09-13T10:10:52.288 INFO:tasks.rados.rados.0.smithi195.stdout:623: finishing write tid 3 to smithi19530714-45 2024-09-13T10:10:52.288 INFO:tasks.rados.rados.0.smithi195.stdout:623: finishing write tid 4 to smithi19530714-45 2024-09-13T10:10:52.288 INFO:tasks.rados.rados.0.smithi195.stdout:623: finishing write tid 5 to smithi19530714-45 2024-09-13T10:10:52.289 INFO:tasks.rados.rados.0.smithi195.stdout:623: oid 45 updating version 0 to 40 2024-09-13T10:10:52.289 INFO:tasks.rados.rados.0.smithi195.stdout:623: oid 45 version 40 is already newer than 39 2024-09-13T10:10:52.289 INFO:tasks.rados.rados.0.smithi195.stdout:623: oid 45 version 40 is already newer than 37 2024-09-13T10:10:52.289 INFO:tasks.rados.rados.0.smithi195.stdout:623: oid 45 version 40 is already newer than 38 2024-09-13T10:10:52.289 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 40 (ObjNum 59 snap 0 seq_num 59) dirty exists 2024-09-13T10:10:52.289 INFO:tasks.rados.rados.0.smithi195.stdout:623: left oid 45 (ObjNum 59 snap 0 seq_num 59) 2024-09-13T10:10:52.289 INFO:tasks.rados.rados.0.smithi195.stdout:625: finishing write tid 1 to smithi19530714-3 2024-09-13T10:10:52.289 INFO:tasks.rados.rados.0.smithi195.stdout:624: expect (ObjNum 25 snap 0 seq_num 25) 2024-09-13T10:10:52.289 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:52.289 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:52.290 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:10:52.290 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:10:52.290 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-13T10:10:52.291 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:52.281+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:10:52.291 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:10:52.291 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:10:52.292 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:10:52.292 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:10:52.292 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:52.293 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:10:52.293 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:10:52.293 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:10:52.293 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:10:52.294 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:10:52.294 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:10:52.294 INFO:teuthology.orchestra.run.smithi073.stderr:set pool 4 pg_num to 26 2024-09-13T10:10:52.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:10:52.306+0000 7f680ffff640 0 -- 172.21.15.73:0/3811041760 >> v2:172.21.15.195:6805/848686366 conn(0x7f68102255c0 msgr2=0x7f68102279b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:10:52.314 INFO:tasks.rados.rados.0.smithi195.stdout:625: finishing write tid 2 to smithi19530714-3 2024-09-13T10:10:52.314 INFO:tasks.rados.rados.0.smithi195.stdout:625: finishing write tid 3 to smithi19530714-3 2024-09-13T10:10:52.314 INFO:tasks.rados.rados.0.smithi195.stdout:625: finishing write tid 4 to smithi19530714-3 2024-09-13T10:10:52.314 INFO:tasks.rados.rados.0.smithi195.stdout:625: finishing write tid 5 to smithi19530714-3 2024-09-13T10:10:52.314 INFO:tasks.rados.rados.0.smithi195.stdout:625: oid 3 updating version 0 to 26 2024-09-13T10:10:52.314 INFO:tasks.rados.rados.0.smithi195.stdout:625: oid 3 version 26 is already newer than 25 2024-09-13T10:10:52.314 INFO:tasks.rados.rados.0.smithi195.stdout:625: oid 3 updating version 26 to 28 2024-09-13T10:10:52.314 INFO:tasks.rados.rados.0.smithi195.stdout:625: oid 3 version 28 is already newer than 27 2024-09-13T10:10:52.314 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 28 (ObjNum 60 snap 0 seq_num 60) dirty exists 2024-09-13T10:10:52.314 INFO:tasks.rados.rados.0.smithi195.stdout:625: left oid 3 (ObjNum 60 snap 0 seq_num 60) 2024-09-13T10:10:52.314 INFO:tasks.rados.rados.0.smithi195.stdout:615: done (11 left) 2024-09-13T10:10:52.315 INFO:tasks.rados.rados.0.smithi195.stdout:616: done (10 left) 2024-09-13T10:10:52.315 INFO:tasks.rados.rados.0.smithi195.stdout:617: done (9 left) 2024-09-13T10:10:52.315 INFO:tasks.rados.rados.0.smithi195.stdout:618: done (8 left) 2024-09-13T10:10:52.315 INFO:tasks.rados.rados.0.smithi195.stdout:619: done (7 left) 2024-09-13T10:10:52.315 INFO:tasks.rados.rados.0.smithi195.stdout:620: done (6 left) 2024-09-13T10:10:52.315 INFO:tasks.rados.rados.0.smithi195.stdout:621: done (5 left) 2024-09-13T10:10:52.315 INFO:tasks.rados.rados.0.smithi195.stdout:622: done (4 left) 2024-09-13T10:10:52.315 INFO:tasks.rados.rados.0.smithi195.stdout:623: done (3 left) 2024-09-13T10:10:52.315 INFO:tasks.rados.rados.0.smithi195.stdout:624: done (2 left) 2024-09-13T10:10:52.315 INFO:tasks.rados.rados.0.smithi195.stdout:625: done (1 left) 2024-09-13T10:10:52.315 INFO:tasks.rados.rados.0.smithi195.stdout:626: done (0 left) 2024-09-13T10:10:52.315 INFO:tasks.rados.rados.0.smithi195.stdout:627: read oid 29 snap -1 2024-09-13T10:10:52.315 INFO:tasks.rados.rados.0.smithi195.stdout:627: expect (ObjNum 54 snap 0 seq_num 54) 2024-09-13T10:10:52.315 INFO:tasks.rados.rados.0.smithi195.stdout:628: set_chunk oid 48 target oid 2024-09-13T10:10:52.316 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:10:52.351 INFO:tasks.rados.rados.0.smithi195.stdout:628: set_chunk oid 48 offset: 480256 length: 1873920 target oid acd4ba443c87831e17974fb7fed5ad5885cc38aabcc983205165ece558f043f6 offset: 0 snap -1 2024-09-13T10:10:52.351 INFO:tasks.rados.rados.0.smithi195.stdout:629: read oid 6 snap -1 2024-09-13T10:10:52.352 INFO:tasks.rados.rados.0.smithi195.stdout:629: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-13T10:10:52.352 INFO:tasks.rados.rados.0.smithi195.stdout:630: set_chunk oid 43 target oid 2024-09-13T10:10:52.383 INFO:tasks.rados.rados.0.smithi195.stdout:630: set_chunk oid 43 offset: 783872 length: 1580032 target oid 323d248cf06b7165631bd47a6a521b8b509bfb492323c6206060f8bf595b95ec offset: 0 snap -1 2024-09-13T10:10:52.383 INFO:tasks.rados.rados.0.smithi195.stdout:631: read oid 15 snap -1 2024-09-13T10:10:52.383 INFO:tasks.rados.rados.0.smithi195.stdout:631: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-13T10:10:52.383 INFO:tasks.rados.rados.0.smithi195.stdout:632: read oid 24 snap -1 2024-09-13T10:10:52.383 INFO:tasks.rados.rados.0.smithi195.stdout:632: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-13T10:10:52.383 INFO:tasks.rados.rados.0.smithi195.stdout:633: write oid 10 current snap is 0 2024-09-13T10:10:52.383 INFO:tasks.rados.rados.0.smithi195.stdout:633: seq_num 62 ranges {642034=796262,1895060=564680} 2024-09-13T10:10:52.386 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:10:52.383+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:10:52.406 INFO:tasks.rados.rados.0.smithi195.stdout:633: writing smithi19530714-10 from 642034 to 1438296 tid 1 2024-09-13T10:10:52.422 INFO:tasks.rados.rados.0.smithi195.stdout:633: writing smithi19530714-10 from 1895060 to 2459740 tid 2 2024-09-13T10:10:52.422 INFO:tasks.rados.rados.0.smithi195.stdout:634: tier_evict oid 30 2024-09-13T10:10:52.422 INFO:tasks.rados.rados.0.smithi195.stdout:634: tier_evict oid 30 snap -1 2024-09-13T10:10:52.422 INFO:tasks.rados.rados.0.smithi195.stdout:627: expect (ObjNum 54 snap 0 seq_num 54) 2024-09-13T10:10:52.465 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:52.465 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:52.479 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-13T10:10:52.487 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:10:52.484+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:10:52.568 INFO:tasks.rados.rados.0.smithi195.stdout:628: finishing set_chunk to oid 48 2024-09-13T10:10:52.568 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 48 set_chunk returned error code -95 offset: 480256 length: 1873920 2024-09-13T10:10:52.568 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 0 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-09-13T10:10:52.568 INFO:tasks.rados.rados.0.smithi195.stdout:629: expect (ObjNum 5 snap 0 seq_num 5) 2024-09-13T10:10:52.584 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:52.587 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:52.584+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:10:52.600 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:52.600 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:52.607 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:52.614 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-13T10:10:52.630 INFO:tasks.rados.rados.0.smithi195.stdout:630: finishing set_chunk to oid 43 2024-09-13T10:10:52.630 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 43 set_chunk returned error code -95 offset: 783872 length: 1580032 2024-09-13T10:10:52.630 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-13T10:10:52.630 INFO:tasks.rados.rados.0.smithi195.stdout:631: expect (ObjNum 14 snap 0 seq_num 14) 2024-09-13T10:10:52.645 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:52.683 INFO:tasks.rados.rados.0.smithi195.stdout:632: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-13T10:10:52.688 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:52.687+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:52.692 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2365064 2024-09-13T10:10:52.696 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:52.728 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1142408 2024-09-13T10:10:52.730 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:52.731 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:52.742 INFO:tasks.rados.rados.0.smithi195.stdout:633: finishing write tid 1 to smithi19530714-10 2024-09-13T10:10:52.742 INFO:tasks.rados.rados.0.smithi195.stdout:634: got (22) Invalid argument 2024-09-13T10:10:52.742 INFO:tasks.rados.rados.0.smithi195.stdout:633: finishing write tid 2 to smithi19530714-10 2024-09-13T10:10:52.742 INFO:tasks.rados.rados.0.smithi195.stdout:633: finishing write tid 3 to smithi19530714-10 2024-09-13T10:10:52.743 INFO:tasks.rados.rados.0.smithi195.stdout:633: finishing write tid 4 to smithi19530714-10 2024-09-13T10:10:52.743 INFO:tasks.rados.rados.0.smithi195.stdout:633: oid 10 updating version 0 to 12 2024-09-13T10:10:52.743 INFO:tasks.rados.rados.0.smithi195.stdout:633: oid 10 version 12 is already newer than 11 2024-09-13T10:10:52.743 INFO:tasks.rados.rados.0.smithi195.stdout:633: oid 10 updating version 12 to 13 2024-09-13T10:10:52.743 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 13 (ObjNum 62 snap 0 seq_num 62) dirty exists 2024-09-13T10:10:52.743 INFO:tasks.rados.rados.0.smithi195.stdout:633: left oid 10 (ObjNum 62 snap 0 seq_num 62) 2024-09-13T10:10:52.743 INFO:tasks.rados.rados.0.smithi195.stdout:627: done (7 left) 2024-09-13T10:10:52.744 INFO:tasks.rados.rados.0.smithi195.stdout:628: done (6 left) 2024-09-13T10:10:52.744 INFO:tasks.rados.rados.0.smithi195.stdout:629: done (5 left) 2024-09-13T10:10:52.744 INFO:tasks.rados.rados.0.smithi195.stdout:630: done (4 left) 2024-09-13T10:10:52.744 INFO:tasks.rados.rados.0.smithi195.stdout:631: done (3 left) 2024-09-13T10:10:52.744 INFO:tasks.rados.rados.0.smithi195.stdout:632: done (2 left) 2024-09-13T10:10:52.744 INFO:tasks.rados.rados.0.smithi195.stdout:633: done (1 left) 2024-09-13T10:10:52.744 INFO:tasks.rados.rados.0.smithi195.stdout:634: done (0 left) 2024-09-13T10:10:52.745 INFO:tasks.rados.rados.0.smithi195.stdout:635: set_chunk oid 43 target oid 2024-09-13T10:10:52.746 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T10:10:52.758 INFO:tasks.rados.rados.0.smithi195.stdout:635: set_chunk oid 43 offset: 830464 length: 899072 target oid 4dbc0174cc18146a3bd0362f41634cb709ff0db50285747e3e158ec93eb8cc5b offset: 0 snap -1 2024-09-13T10:10:52.758 INFO:tasks.rados.rados.0.smithi195.stdout:636: read oid 46 snap -1 2024-09-13T10:10:52.758 INFO:tasks.rados.rados.0.smithi195.stdout:636: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-13T10:10:52.758 INFO:tasks.rados.rados.0.smithi195.stdout:637: read oid 48 snap -1 2024-09-13T10:10:52.758 INFO:tasks.rados.rados.0.smithi195.stdout:637: expect (ObjNum 47 snap 0 seq_num 47) 2024-09-13T10:10:52.759 INFO:tasks.rados.rados.0.smithi195.stdout:638: read oid 4 snap -1 2024-09-13T10:10:52.759 INFO:tasks.rados.rados.0.smithi195.stdout:638: expect (ObjNum 58 snap 0 seq_num 58) 2024-09-13T10:10:52.759 INFO:tasks.rados.rados.0.smithi195.stdout:639: set_chunk oid 25 target oid 2024-09-13T10:10:52.773 INFO:tasks.rados.rados.0.smithi195.stdout:639: set_chunk oid 25 offset: 1602048 length: 677376 target oid 01e9ad0d71fdb5a704bcebeef6f2302fec34ffa63e00571dfcdf84a292b91816 offset: 0 snap -1 2024-09-13T10:10:52.774 INFO:tasks.rados.rados.0.smithi195.stdout:640: tier_evict oid 15 2024-09-13T10:10:52.774 INFO:tasks.rados.rados.0.smithi195.stdout:640: tier_evict oid 15 snap -1 2024-09-13T10:10:52.774 INFO:tasks.rados.rados.0.smithi195.stdout:635: finishing set_chunk to oid 43 2024-09-13T10:10:52.774 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 43 set_chunk returned error code -95 offset: 830464 length: 899072 2024-09-13T10:10:52.774 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-13T10:10:52.774 INFO:tasks.rados.rados.0.smithi195.stdout:638: expect (ObjNum 58 snap 0 seq_num 58) 2024-09-13T10:10:52.789 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:10:52.789+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:10:52.805 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2833616 2024-09-13T10:10:52.813 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 166926 2024-09-13T10:10:52.880 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:52.880 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:52.890 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:10:52.884+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:10:52.894 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T10:10:52.902 INFO:tasks.rados.rados.0.smithi195.stdout:636: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-13T10:10:52.928 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:52.963 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1262680 2024-09-13T10:10:52.977 INFO:tasks.rados.rados.0.smithi195.stdout:637: expect (ObjNum 47 snap 0 seq_num 47) 2024-09-13T10:10:52.977 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:52.991 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:52.989+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:10:53.048 INFO:tasks.rados.rados.0.smithi195.stdout:640: got (22) Invalid argument 2024-09-13T10:10:53.048 INFO:tasks.rados.rados.0.smithi195.stdout:639: finishing set_chunk to oid 25 2024-09-13T10:10:53.048 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 25 set_chunk returned error code -95 offset: 1602048 length: 677376 2024-09-13T10:10:53.048 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 0 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-09-13T10:10:53.048 INFO:tasks.rados.rados.0.smithi195.stdout:635: done (5 left) 2024-09-13T10:10:53.049 INFO:tasks.rados.rados.0.smithi195.stdout:636: done (4 left) 2024-09-13T10:10:53.049 INFO:tasks.rados.rados.0.smithi195.stdout:637: done (3 left) 2024-09-13T10:10:53.049 INFO:tasks.rados.rados.0.smithi195.stdout:638: done (2 left) 2024-09-13T10:10:53.049 INFO:tasks.rados.rados.0.smithi195.stdout:639: done (1 left) 2024-09-13T10:10:53.049 INFO:tasks.rados.rados.0.smithi195.stdout:640: done (0 left) 2024-09-13T10:10:53.049 INFO:tasks.rados.rados.0.smithi195.stdout:641: write (excl) oid 35 current snap is 0 2024-09-13T10:10:53.049 INFO:tasks.rados.rados.0.smithi195.stdout:641: seq_num 63 ranges {658841=471511,1553542=708338,2964119=94722} 2024-09-13T10:10:53.062 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:53.062 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:53.066 INFO:tasks.rados.rados.0.smithi195.stdout:641: writing smithi19530714-35 from 658841 to 1130352 tid 1 2024-09-13T10:10:53.076 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T10:10:53.085 INFO:tasks.rados.rados.0.smithi195.stdout:641: writing smithi19530714-35 from 1553542 to 2261880 tid 2 2024-09-13T10:10:53.091 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:53.091+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:53.097 INFO:tasks.rados.rados.0.smithi195.stdout:641: writing smithi19530714-35 from 2964119 to 3058841 tid 3 2024-09-13T10:10:53.097 INFO:tasks.rados.rados.0.smithi195.stdout:642: read oid 50 snap -1 2024-09-13T10:10:53.098 INFO:tasks.rados.rados.0.smithi195.stdout:642: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-13T10:10:53.098 INFO:tasks.rados.rados.0.smithi195.stdout:643: write oid 33 current snap is 0 2024-09-13T10:10:53.098 INFO:tasks.rados.rados.0.smithi195.stdout:643: seq_num 64 ranges {675648=546760,2012024=568349} 2024-09-13T10:10:53.113 INFO:tasks.rados.rados.0.smithi195.stdout:643: writing smithi19530714-33 from 675648 to 1222408 tid 1 2024-09-13T10:10:53.126 INFO:tasks.rados.rados.0.smithi195.stdout:643: writing smithi19530714-33 from 2012024 to 2580373 tid 2 2024-09-13T10:10:53.127 INFO:tasks.rados.rados.0.smithi195.stdout:644: read oid 11 snap -1 2024-09-13T10:10:53.127 INFO:tasks.rados.rados.0.smithi195.stdout:644: expect deleted 2024-09-13T10:10:53.127 INFO:tasks.rados.rados.0.smithi195.stdout:645: read oid 44 snap -1 2024-09-13T10:10:53.127 INFO:tasks.rados.rados.0.smithi195.stdout:645: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-13T10:10:53.127 INFO:tasks.rados.rados.0.smithi195.stdout:646: read oid 8 snap -1 2024-09-13T10:10:53.127 INFO:tasks.rados.rados.0.smithi195.stdout:646: expect (ObjNum 7 snap 0 seq_num 7) 2024-09-13T10:10:53.127 INFO:tasks.rados.rados.0.smithi195.stdout:647: read oid 29 snap -1 2024-09-13T10:10:53.127 INFO:tasks.rados.rados.0.smithi195.stdout:647: expect (ObjNum 54 snap 0 seq_num 54) 2024-09-13T10:10:53.127 INFO:tasks.rados.rados.0.smithi195.stdout:648: read oid 14 snap -1 2024-09-13T10:10:53.127 INFO:tasks.rados.rados.0.smithi195.stdout:648: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-13T10:10:53.127 INFO:tasks.rados.rados.0.smithi195.stdout:649: read oid 38 snap -1 2024-09-13T10:10:53.127 INFO:tasks.rados.rados.0.smithi195.stdout:649: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-13T10:10:53.127 INFO:tasks.rados.rados.0.smithi195.stdout:650: read oid 34 snap -1 2024-09-13T10:10:53.128 INFO:tasks.rados.rados.0.smithi195.stdout:650: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-13T10:10:53.128 INFO:tasks.rados.rados.0.smithi195.stdout:651: tier_evict oid 49 2024-09-13T10:10:53.128 INFO:tasks.rados.rados.0.smithi195.stdout:651: tier_evict oid 49 snap -1 2024-09-13T10:10:53.128 INFO:tasks.rados.rados.0.smithi195.stdout:641: finishing write tid 1 to smithi19530714-35 2024-09-13T10:10:53.128 INFO:tasks.rados.rados.0.smithi195.stdout:641: finishing write tid 2 to smithi19530714-35 2024-09-13T10:10:53.128 INFO:tasks.rados.rados.0.smithi195.stdout:641: finishing write tid 3 to smithi19530714-35 2024-09-13T10:10:53.128 INFO:tasks.rados.rados.0.smithi195.stdout:641: finishing write tid 4 to smithi19530714-35 2024-09-13T10:10:53.128 INFO:tasks.rados.rados.0.smithi195.stdout:641: finishing write tid 5 to smithi19530714-35 2024-09-13T10:10:53.128 INFO:tasks.rados.rados.0.smithi195.stdout:641: oid 35 updating version 0 to 29 2024-09-13T10:10:53.128 INFO:tasks.rados.rados.0.smithi195.stdout:641: oid 35 updating version 29 to 30 2024-09-13T10:10:53.128 INFO:tasks.rados.rados.0.smithi195.stdout:641: oid 35 updating version 30 to 31 2024-09-13T10:10:53.128 INFO:tasks.rados.rados.0.smithi195.stdout:641: oid 35 updating version 31 to 32 2024-09-13T10:10:53.128 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 32 (ObjNum 63 snap 0 seq_num 63) dirty exists 2024-09-13T10:10:53.129 INFO:tasks.rados.rados.0.smithi195.stdout:641: left oid 35 (ObjNum 63 snap 0 seq_num 63) 2024-09-13T10:10:53.129 INFO:tasks.rados.rados.0.smithi195.stdout:642: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-13T10:10:53.188 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:53.193 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:10:53.189+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:10:53.198 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:53.198 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:53.214 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T10:10:53.218 INFO:tasks.rados.rados.0.smithi195.stdout:643: finishing write tid 1 to smithi19530714-33 2024-09-13T10:10:53.219 INFO:tasks.rados.rados.0.smithi195.stdout:643: finishing write tid 2 to smithi19530714-33 2024-09-13T10:10:53.219 INFO:tasks.rados.rados.0.smithi195.stdout:643: finishing write tid 3 to smithi19530714-33 2024-09-13T10:10:53.219 INFO:tasks.rados.rados.0.smithi195.stdout:643: finishing write tid 4 to smithi19530714-33 2024-09-13T10:10:53.219 INFO:tasks.rados.rados.0.smithi195.stdout:643: oid 33 updating version 0 to 21 2024-09-13T10:10:53.219 INFO:tasks.rados.rados.0.smithi195.stdout:643: oid 33 updating version 21 to 22 2024-09-13T10:10:53.219 INFO:tasks.rados.rados.0.smithi195.stdout:643: oid 33 version 22 is already newer than 20 2024-09-13T10:10:53.219 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 22 (ObjNum 64 snap 0 seq_num 64) dirty exists 2024-09-13T10:10:53.219 INFO:tasks.rados.rados.0.smithi195.stdout:643: left oid 33 (ObjNum 64 snap 0 seq_num 64) 2024-09-13T10:10:53.219 INFO:tasks.rados.rados.0.smithi195.stdout:649: expect (ObjNum 37 snap 0 seq_num 37) 2024-09-13T10:10:53.257 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1942479 2024-09-13T10:10:53.263 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:53.291 INFO:tasks.rados.rados.0.smithi195.stdout:651: got (22) Invalid argument 2024-09-13T10:10:53.291 INFO:tasks.rados.rados.0.smithi195.stdout:647: expect (ObjNum 54 snap 0 seq_num 54) 2024-09-13T10:10:53.293 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:10:53.289+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:10:53.335 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T10:10:53.343 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:53.343 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:53.353 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2017895 2024-09-13T10:10:53.357 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T10:10:53.375 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:53.404 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:53.400+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:10:53.420 INFO:tasks.rados.rados.0.smithi195.stdout:645: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-13T10:10:53.424 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1580482 2024-09-13T10:10:53.440 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 412251 2024-09-13T10:10:53.465 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:53.482 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:53.482 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:53.494 INFO:tasks.rados.rados.0.smithi195.stdout:646: expect (ObjNum 7 snap 0 seq_num 7) 2024-09-13T10:10:53.495 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:10:53.493+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:10:53.496 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T10:10:53.497 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1677230 2024-09-13T10:10:53.530 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:53.553 INFO:tasks.rados.rados.0.smithi195.stdout:648: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-13T10:10:53.553 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:53.582 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 821452 2024-09-13T10:10:53.595 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:10:53.595+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:10:53.609 INFO:tasks.rados.rados.0.smithi195.stdout:650: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-13T10:10:53.652 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1090704 2024-09-13T10:10:53.663 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:53.663 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:53.677 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-13T10:10:53.678 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:53.718 INFO:tasks.rados.rados.0.smithi195.stdout:641: done (10 left) 2024-09-13T10:10:53.718 INFO:tasks.rados.rados.0.smithi195.stdout:642: done (9 left) 2024-09-13T10:10:53.719 INFO:tasks.rados.rados.0.smithi195.stdout:643: done (8 left) 2024-09-13T10:10:53.719 INFO:tasks.rados.rados.0.smithi195.stdout:644: done (7 left) 2024-09-13T10:10:53.719 INFO:tasks.rados.rados.0.smithi195.stdout:645: done (6 left) 2024-09-13T10:10:53.719 INFO:tasks.rados.rados.0.smithi195.stdout:646: done (5 left) 2024-09-13T10:10:53.719 INFO:tasks.rados.rados.0.smithi195.stdout:647: done (4 left) 2024-09-13T10:10:53.719 INFO:tasks.rados.rados.0.smithi195.stdout:648: done (3 left) 2024-09-13T10:10:53.719 INFO:tasks.rados.rados.0.smithi195.stdout:649: done (2 left) 2024-09-13T10:10:53.719 INFO:tasks.rados.rados.0.smithi195.stdout:650: done (1 left) 2024-09-13T10:10:53.719 INFO:tasks.rados.rados.0.smithi195.stdout:651: done (0 left) 2024-09-13T10:10:53.719 INFO:tasks.rados.rados.0.smithi195.stdout:652: write oid 41 current snap is 0 2024-09-13T10:10:53.719 INFO:tasks.rados.rados.0.smithi195.stdout:652: seq_num 65 ranges {692455=622009,2070506=712007} 2024-09-13T10:10:53.723 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:10:53.720+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:10:53.739 INFO:tasks.rados.rados.0.smithi195.stdout:652: writing smithi19530714-41 from 692455 to 1314464 tid 1 2024-09-13T10:10:53.764 INFO:tasks.rados.rados.0.smithi195.stdout:652: writing smithi19530714-41 from 2070506 to 2782513 tid 2 2024-09-13T10:10:53.765 INFO:tasks.rados.rados.0.smithi195.stdout:653: read oid 44 snap -1 2024-09-13T10:10:53.765 INFO:tasks.rados.rados.0.smithi195.stdout:653: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-13T10:10:53.765 INFO:tasks.rados.rados.0.smithi195.stdout:654: read oid 16 snap -1 2024-09-13T10:10:53.765 INFO:tasks.rados.rados.0.smithi195.stdout:654: expect (ObjNum 15 snap 0 seq_num 15) 2024-09-13T10:10:53.766 INFO:tasks.rados.rados.0.smithi195.stdout:655: read oid 21 snap -1 2024-09-13T10:10:53.766 INFO:tasks.rados.rados.0.smithi195.stdout:655: expect (ObjNum 20 snap 0 seq_num 20) 2024-09-13T10:10:53.766 INFO:tasks.rados.rados.0.smithi195.stdout:656: read oid 24 snap -1 2024-09-13T10:10:53.766 INFO:tasks.rados.rados.0.smithi195.stdout:656: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-13T10:10:53.766 INFO:tasks.rados.rados.0.smithi195.stdout:657: tier_flush oid 15 2024-09-13T10:10:53.766 INFO:tasks.rados.rados.0.smithi195.stdout:657: tier_flush oid 15 snap -1 2024-09-13T10:10:53.767 INFO:tasks.rados.rados.0.smithi195.stdout:652: finishing write tid 1 to smithi19530714-41 2024-09-13T10:10:53.767 INFO:tasks.rados.rados.0.smithi195.stdout:658: write (excl) oid 19 current snap is 0 2024-09-13T10:10:53.767 INFO:tasks.rados.rados.0.smithi195.stdout:658: seq_num 66 ranges {709262=697258,2012635=572018,3046388=62874} 2024-09-13T10:10:53.788 INFO:tasks.rados.rados.0.smithi195.stdout:658: writing smithi19530714-19 from 709262 to 1406520 tid 1 2024-09-13T10:10:53.798 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:10:53.797+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:10:53.799 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:53.799 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:53.803 INFO:tasks.rados.rados.0.smithi195.stdout:658: writing smithi19530714-19 from 2012635 to 2584653 tid 2 2024-09-13T10:10:53.809 INFO:tasks.rados.rados.0.smithi195.stdout:658: writing smithi19530714-19 from 3046388 to 3109262 tid 3 2024-09-13T10:10:53.809 INFO:tasks.rados.rados.0.smithi195.stdout:659: delete oid 36 current snap is 0 2024-09-13T10:10:53.809 INFO:tasks.rados.rados.0.smithi195.stdout:653: expect (ObjNum 43 snap 0 seq_num 43) 2024-09-13T10:10:53.809 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 220012 2024-09-13T10:10:53.815 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-13T10:10:53.877 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2594859 2024-09-13T10:10:53.878 INFO:tasks.rados.rados.0.smithi195.stdout:652: finishing write tid 2 to smithi19530714-41 2024-09-13T10:10:53.878 INFO:tasks.rados.rados.0.smithi195.stdout:652: finishing write tid 3 to smithi19530714-41 2024-09-13T10:10:53.878 INFO:tasks.rados.rados.0.smithi195.stdout:652: finishing write tid 4 to smithi19530714-41 2024-09-13T10:10:53.878 INFO:tasks.rados.rados.0.smithi195.stdout:652: oid 41 updating version 0 to 20 2024-09-13T10:10:53.878 INFO:tasks.rados.rados.0.smithi195.stdout:652: oid 41 version 20 is already newer than 19 2024-09-13T10:10:53.878 INFO:tasks.rados.rados.0.smithi195.stdout:652: oid 41 updating version 20 to 21 2024-09-13T10:10:53.878 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 21 (ObjNum 65 snap 0 seq_num 65) dirty exists 2024-09-13T10:10:53.879 INFO:tasks.rados.rados.0.smithi195.stdout:652: left oid 41 (ObjNum 65 snap 0 seq_num 65) 2024-09-13T10:10:53.879 INFO:tasks.rados.rados.0.smithi195.stdout:657: got (0) Success 2024-09-13T10:10:53.879 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-13T10:10:53.899 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:10:53.899+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:10:53.983 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:53.983 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:53.997 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-13T10:10:54.000 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:53.997+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:10:54.101 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:10:54.101+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:10:54.119 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:54.119 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:54.133 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-13T10:10:54.202 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:54.201+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:10:54.257 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:54.258 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:54.272 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-13T10:10:54.301 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-13T10:10:54.316 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:10:54.312+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:10:54.322 INFO:tasks.rados.rados.0.smithi195.stdout:655: expect (ObjNum 20 snap 0 seq_num 20) 2024-09-13T10:10:54.333 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:10:54.404 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:10:54.401+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:10:54.407 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:54.407 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:54.419 INFO:tasks.rados.rados.0.smithi195.stdout:656: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-13T10:10:54.421 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-13T10:10:54.423 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1614899 2024-09-13T10:10:54.435 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:54.466 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1717022 2024-09-13T10:10:54.477 INFO:tasks.rados.rados.0.smithi195.stdout:658: finishing write tid 1 to smithi19530714-19 2024-09-13T10:10:54.477 INFO:tasks.rados.rados.0.smithi195.stdout:658: finishing write tid 2 to smithi19530714-19 2024-09-13T10:10:54.477 INFO:tasks.rados.rados.0.smithi195.stdout:658: finishing write tid 3 to smithi19530714-19 2024-09-13T10:10:54.477 INFO:tasks.rados.rados.0.smithi195.stdout:658: finishing write tid 4 to smithi19530714-19 2024-09-13T10:10:54.478 INFO:tasks.rados.rados.0.smithi195.stdout:658: finishing write tid 5 to smithi19530714-19 2024-09-13T10:10:54.478 INFO:tasks.rados.rados.0.smithi195.stdout:658: oid 19 updating version 0 to 35 2024-09-13T10:10:54.478 INFO:tasks.rados.rados.0.smithi195.stdout:658: oid 19 version 35 is already newer than 34 2024-09-13T10:10:54.478 INFO:tasks.rados.rados.0.smithi195.stdout:658: oid 19 version 35 is already newer than 33 2024-09-13T10:10:54.478 INFO:tasks.rados.rados.0.smithi195.stdout:658: oid 19 updating version 35 to 36 2024-09-13T10:10:54.478 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 36 (ObjNum 66 snap 0 seq_num 66) dirty exists 2024-09-13T10:10:54.478 INFO:tasks.rados.rados.0.smithi195.stdout:658: left oid 19 (ObjNum 66 snap 0 seq_num 66) 2024-09-13T10:10:54.478 INFO:tasks.rados.rados.0.smithi195.stdout:654: expect (ObjNum 15 snap 0 seq_num 15) 2024-09-13T10:10:54.505 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:54.503+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:54.521 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:10:54.521 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:10:54.527 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 443711 2024-09-13T10:10:54.542 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:10:54.542 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:10:54.548 INFO:tasks.rados.rados.0.smithi195.stdout:652: done (7 left) 2024-09-13T10:10:54.548 INFO:tasks.rados.rados.0.smithi195.stdout:653: done (6 left) 2024-09-13T10:10:54.548 INFO:tasks.rados.rados.0.smithi195.stdout:654: done (5 left) 2024-09-13T10:10:54.549 INFO:tasks.rados.rados.0.smithi195.stdout:655: done (4 left) 2024-09-13T10:10:54.549 INFO:tasks.rados.rados.0.smithi195.stdout:656: done (3 left) 2024-09-13T10:10:54.549 INFO:tasks.rados.rados.0.smithi195.stdout:657: done (2 left) 2024-09-13T10:10:54.549 INFO:tasks.rados.rados.0.smithi195.stdout:658: done (1 left) 2024-09-13T10:10:54.549 INFO:tasks.rados.rados.0.smithi195.stdout:659: done (0 left) 2024-09-13T10:10:54.549 INFO:tasks.rados.rados.0.smithi195.stdout:660: write oid 2 current snap is 0 2024-09-13T10:10:54.549 INFO:tasks.rados.rados.0.smithi195.stdout:660: seq_num 68 ranges {742876=447756,1729599=575687,3101234=41642} 2024-09-13T10:10:54.561 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:10:54.561 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:10:54.568 INFO:tasks.rados.rados.0.smithi195.stdout:660: writing smithi19530714-2 from 742876 to 1190632 tid 1 2024-09-13T10:10:54.582 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:10:54.583 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:10:54.586 INFO:tasks.rados.rados.0.smithi195.stdout:660: writing smithi19530714-2 from 1729599 to 2305286 tid 2 2024-09-13T10:10:54.593 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:54.593 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:54.599 INFO:tasks.rados.rados.0.smithi195.stdout:660: writing smithi19530714-2 from 3101234 to 3142876 tid 3 2024-09-13T10:10:54.599 INFO:tasks.rados.rados.0.smithi195.stdout:661: read oid 39 snap -1 2024-09-13T10:10:54.599 INFO:tasks.rados.rados.0.smithi195.stdout:661: expect (ObjNum 38 snap 0 seq_num 38) 2024-09-13T10:10:54.599 INFO:tasks.rados.rados.0.smithi195.stdout:662: set_chunk oid 8 target oid 2024-09-13T10:10:54.600 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:10:54.601 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:10:54.607 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:10:54.604+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:10:54.609 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T10:10:54.615 INFO:tasks.rados.rados.0.smithi195.stdout:662: set_chunk oid 8 offset: 629760 length: 969216 target oid dd736a7d131bf8351fd64af6d206d9ce87f9dde27f883893a580c5470ea60671 offset: 0 snap -1 2024-09-13T10:10:54.616 INFO:tasks.rados.rados.0.smithi195.stdout:663: write oid 21 current snap is 0 2024-09-13T10:10:54.616 INFO:tasks.rados.rados.0.smithi195.stdout:660: finishing write tid 1 to smithi19530714-2 2024-09-13T10:10:54.616 INFO:tasks.rados.rados.0.smithi195.stdout:663: seq_num 69 ranges {759683=639358,1904434=719345,3128657=31026} 2024-09-13T10:10:54.624 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:10:54.624 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:10:54.636 INFO:tasks.rados.rados.0.smithi195.stdout:663: writing smithi19530714-21 from 759683 to 1399041 tid 1 2024-09-13T10:10:54.646 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:10:54.646 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:10:54.653 INFO:tasks.rados.rados.0.smithi195.stdout:663: writing smithi19530714-21 from 1904434 to 2623779 tid 2 2024-09-13T10:10:54.659 INFO:tasks.rados.rados.0.smithi195.stdout:663: writing smithi19530714-21 from 3128657 to 3159683 tid 3 2024-09-13T10:10:54.659 INFO:tasks.rados.rados.0.smithi195.stdout:660: finishing write tid 2 to smithi19530714-2 2024-09-13T10:10:54.659 INFO:tasks.rados.rados.0.smithi195.stdout:660: finishing write tid 3 to smithi19530714-2 2024-09-13T10:10:54.659 INFO:tasks.rados.rados.0.smithi195.stdout:660: finishing write tid 4 to smithi19530714-2 2024-09-13T10:10:54.660 INFO:tasks.rados.rados.0.smithi195.stdout:664: read oid 24 snap -1 2024-09-13T10:10:54.660 INFO:tasks.rados.rados.0.smithi195.stdout:664: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-13T10:10:54.660 INFO:tasks.rados.rados.0.smithi195.stdout:665: set_chunk oid 15 target oid 2024-09-13T10:10:54.666 INFO:tasks.rados.rados.0.smithi195.stdout:665: set_chunk oid 15 offset: 560640 length: 136704 target oid bf7eac47958afeac31da805f9dbb951d45fdd1416c699902783d2cb2976e248c offset: 0 snap -1 2024-09-13T10:10:54.666 INFO:tasks.rados.rados.0.smithi195.stdout:666: write (excl) oid 28 current snap is 0 2024-09-13T10:10:54.666 INFO:tasks.rados.rados.0.smithi195.stdout:660: finishing write tid 5 to smithi19530714-2 2024-09-13T10:10:54.667 INFO:tasks.rados.rados.0.smithi195.stdout:660: oid 2 updating version 0 to 35 2024-09-13T10:10:54.667 INFO:tasks.rados.rados.0.smithi195.stdout:660: oid 2 version 35 is already newer than 32 2024-09-13T10:10:54.667 INFO:tasks.rados.rados.0.smithi195.stdout:660: oid 2 version 35 is already newer than 34 2024-09-13T10:10:54.667 INFO:tasks.rados.rados.0.smithi195.stdout:660: oid 2 version 35 is already newer than 33 2024-09-13T10:10:54.667 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 35 (ObjNum 68 snap 0 seq_num 68) dirty exists 2024-09-13T10:10:54.667 INFO:tasks.rados.rados.0.smithi195.stdout:660: left oid 2 (ObjNum 68 snap 0 seq_num 68) 2024-09-13T10:10:54.667 INFO:tasks.rados.rados.0.smithi195.stdout:661: expect (ObjNum 38 snap 0 seq_num 38) 2024-09-13T10:10:54.667 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:10:54.668 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:10:54.685 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:10:54.686 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:10:54.708 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:10:54.708 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:10:54.709 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:10:54.707+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:10:54.729 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:10:54.730 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:10:54.736 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:10:54.736 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:10:54.736 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:10:54.736 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:10:54.739 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1425841 2024-09-13T10:10:54.750 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T10:10:54.752 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:10:54.752 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:10:54.757 INFO:tasks.rados.rados.0.smithi195.stdout:662: finishing set_chunk to oid 8 2024-09-13T10:10:54.757 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 8 set_chunk returned error code -95 offset: 629760 length: 969216 2024-09-13T10:10:54.757 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-13T10:10:54.757 INFO:tasks.rados.rados.0.smithi195.stdout:663: finishing write tid 1 to smithi19530714-21 2024-09-13T10:10:54.757 INFO:tasks.rados.rados.0.smithi195.stdout:663: finishing write tid 2 to smithi19530714-21 2024-09-13T10:10:54.757 INFO:tasks.rados.rados.0.smithi195.stdout:664: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-13T10:10:54.761 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1579869 2024-09-13T10:10:54.772 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:10:54.772 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:10:54.794 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:10:54.795 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:10:54.809 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:10:54.808+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:10:54.811 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2156528 2024-09-13T10:10:54.814 INFO:tasks.rados.rados.0.smithi195.stdout:663: finishing write tid 3 to smithi19530714-21 2024-09-13T10:10:54.814 INFO:tasks.rados.rados.0.smithi195.stdout:663: finishing write tid 4 to smithi19530714-21 2024-09-13T10:10:54.815 INFO:tasks.rados.rados.0.smithi195.stdout:663: finishing write tid 5 to smithi19530714-21 2024-09-13T10:10:54.815 INFO:tasks.rados.rados.0.smithi195.stdout:663: oid 21 updating version 0 to 37 2024-09-13T10:10:54.815 INFO:tasks.rados.rados.0.smithi195.stdout:663: oid 21 updating version 37 to 39 2024-09-13T10:10:54.815 INFO:tasks.rados.rados.0.smithi195.stdout:663: oid 21 version 39 is already newer than 38 2024-09-13T10:10:54.815 INFO:tasks.rados.rados.0.smithi195.stdout:663: oid 21 version 39 is already newer than 36 2024-09-13T10:10:54.815 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 39 (ObjNum 69 snap 0 seq_num 69) dirty exists 2024-09-13T10:10:54.815 INFO:tasks.rados.rados.0.smithi195.stdout:663: left oid 21 (ObjNum 69 snap 0 seq_num 69) 2024-09-13T10:10:54.816 INFO:tasks.rados.rados.0.smithi195.stdout:665: finishing set_chunk to oid 15 2024-09-13T10:10:54.816 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 15 set_chunk returned error code -95 offset: 560640 length: 136704 2024-09-13T10:10:54.816 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 0 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-09-13T10:10:54.816 INFO:tasks.rados.rados.0.smithi195.stdout:666: seq_num 70 ranges {776490=714607,2246563=579356} 2024-09-13T10:10:54.818 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:10:54.818 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:10:54.826 INFO:tasks.rados.rados.0.smithi195.stdout:666: writing smithi19530714-28 from 776490 to 1491097 tid 1 2024-09-13T10:10:54.837 INFO:tasks.rados.rados.0.smithi195.stdout:666: writing smithi19530714-28 from 2246563 to 2825919 tid 2 2024-09-13T10:10:54.837 INFO:tasks.rados.rados.0.smithi195.stdout:660: done (6 left) 2024-09-13T10:10:54.837 INFO:tasks.rados.rados.0.smithi195.stdout:661: done (5 left) 2024-09-13T10:10:54.837 INFO:tasks.rados.rados.0.smithi195.stdout:662: done (4 left) 2024-09-13T10:10:54.837 INFO:tasks.rados.rados.0.smithi195.stdout:663: done (3 left) 2024-09-13T10:10:54.837 INFO:tasks.rados.rados.0.smithi195.stdout:664: done (2 left) 2024-09-13T10:10:54.837 INFO:tasks.rados.rados.0.smithi195.stdout:665: done (1 left) 2024-09-13T10:10:54.837 INFO:tasks.rados.rados.0.smithi195.stdout:667: read oid 46 snap -1 2024-09-13T10:10:54.837 INFO:tasks.rados.rados.0.smithi195.stdout:667: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-13T10:10:54.837 INFO:tasks.rados.rados.0.smithi195.stdout:668: tier_evict oid 2 2024-09-13T10:10:54.838 INFO:tasks.rados.rados.0.smithi195.stdout:668: tier_evict oid 2 snap -1 2024-09-13T10:10:54.838 INFO:tasks.rados.rados.0.smithi195.stdout:669: read oid 4 snap -1 2024-09-13T10:10:54.838 INFO:tasks.rados.rados.0.smithi195.stdout:669: expect (ObjNum 58 snap 0 seq_num 58) 2024-09-13T10:10:54.838 INFO:tasks.rados.rados.0.smithi195.stdout:670: write oid 38 current snap is 0 2024-09-13T10:10:54.838 INFO:tasks.rados.rados.0.smithi195.stdout:670: seq_num 71 ranges {793297=789856,2305045=723014} 2024-09-13T10:10:54.840 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:10:54.840 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:10:54.854 INFO:tasks.rados.rados.0.smithi195.stdout:670: writing smithi19530714-38 from 793297 to 1583153 tid 1 2024-09-13T10:10:54.868 INFO:tasks.rados.rados.0.smithi195.stdout:670: writing smithi19530714-38 from 2305045 to 3028059 tid 2 2024-09-13T10:10:54.868 INFO:tasks.rados.rados.0.smithi195.stdout:671: read oid 7 snap -1 2024-09-13T10:10:54.868 INFO:tasks.rados.rados.0.smithi195.stdout:671: expect deleted 2024-09-13T10:10:54.868 INFO:tasks.rados.rados.0.smithi195.stdout:672: write (excl) oid 44 current snap is 0 2024-09-13T10:10:54.869 INFO:tasks.rados.rados.0.smithi195.stdout:666: finishing write tid 1 to smithi19530714-28 2024-09-13T10:10:54.869 INFO:tasks.rados.rados.0.smithi195.stdout:668: got (22) Invalid argument 2024-09-13T10:10:54.869 INFO:tasks.rados.rados.0.smithi195.stdout:666: finishing write tid 2 to smithi19530714-28 2024-09-13T10:10:54.869 INFO:tasks.rados.rados.0.smithi195.stdout:666: finishing write tid 3 to smithi19530714-28 2024-09-13T10:10:54.869 INFO:tasks.rados.rados.0.smithi195.stdout:666: finishing write tid 4 to smithi19530714-28 2024-09-13T10:10:54.869 INFO:tasks.rados.rados.0.smithi195.stdout:666: oid 28 updating version 0 to 43 2024-09-13T10:10:54.869 INFO:tasks.rados.rados.0.smithi195.stdout:666: oid 28 version 43 is already newer than 41 2024-09-13T10:10:54.869 INFO:tasks.rados.rados.0.smithi195.stdout:666: oid 28 version 43 is already newer than 42 2024-09-13T10:10:54.869 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 43 (ObjNum 70 snap 0 seq_num 70) dirty exists 2024-09-13T10:10:54.869 INFO:tasks.rados.rados.0.smithi195.stdout:666: left oid 28 (ObjNum 70 snap 0 seq_num 70) 2024-09-13T10:10:54.869 INFO:tasks.rados.rados.0.smithi195.stdout:667: expect (ObjNum 45 snap 0 seq_num 45) 2024-09-13T10:10:54.869 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 139216 2024-09-13T10:10:54.881 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:10:54.881 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:10:54.881 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:10:54.881 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:10:54.881 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:10:54.900 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T10:10:54.904 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 369609 2024-09-13T10:10:54.947 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:10:54.947+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:10:54.948 INFO:tasks.rados.rados.0.smithi195.stdout:669: expect (ObjNum 58 snap 0 seq_num 58) 2024-09-13T10:10:54.993 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:55.011 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:10:55.009+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:10:55.022 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:10:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:10:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:10:55.022 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:10:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:17 4.8 4:1de9da87:::smithi19530714-5:head [write 467228~700996,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.375430+0000", 2024-09-13T10:10:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.6429783239999995, 2024-09-13T10:10:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011556949, 2024-09-13T10:10:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.023 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 17 2024-09-13T10:10:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.375430+0000", 2024-09-13T10:10:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.375430+0000", 2024-09-13T10:10:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.375432+0000", 2024-09-13T10:10:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2460000000000002e-06 2024-09-13T10:10:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.376384+0000", 2024-09-13T10:10:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00095172799999999999 2024-09-13T10:10:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.376407+0000", 2024-09-13T10:10:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3200000000000001e-05 2024-09-13T10:10:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.376425+0000", 2024-09-13T10:10:55.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7235999999999999e-05 2024-09-13T10:10:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.376543+0000", 2024-09-13T10:10:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011828999999999999 2024-09-13T10:10:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op must wait for map", 2024-09-13T10:10:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.376553+0000", 2024-09-13T10:10:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9490000000000008e-06 2024-09-13T10:10:55.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.379840+0000", 2024-09-13T10:10:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032868620000000002 2024-09-13T10:10:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.380050+0000", 2024-09-13T10:10:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021082100000000001 2024-09-13T10:10:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0,11", 2024-09-13T10:10:55.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.380284+0000", 2024-09-13T10:10:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023372000000000001 2024-09-13T10:10:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.386795+0000", 2024-09-13T10:10:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0065110929999999999 2024-09-13T10:10:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.386811+0000", 2024-09-13T10:10:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5401e-05 2024-09-13T10:10:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.386861+0000", 2024-09-13T10:10:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9954000000000003e-05 2024-09-13T10:10:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.386923+0000", 2024-09-13T10:10:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2365000000000003e-05 2024-09-13T10:10:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.386987+0000", 2024-09-13T10:10:55.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4084000000000005e-05 2024-09-13T10:10:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:207 3.0 3:332cd44b:::smithi19530714-8 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:10:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.0271399529999998, 2024-09-13T10:10:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074034500000000003, 2024-09-13T10:10:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.033 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 207 2024-09-13T10:10:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:10:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:10:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991271+0000", 2024-09-13T10:10:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9289999999999998e-06 2024-09-13T10:10:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991284+0000", 2024-09-13T10:10:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2597e-05 2024-09-13T10:10:55.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991313+0000", 2024-09-13T10:10:55.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8982000000000001e-05 2024-09-13T10:10:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991334+0000", 2024-09-13T10:10:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1046000000000001e-05 2024-09-13T10:10:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991416+0000", 2024-09-13T10:10:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.153e-05 2024-09-13T10:10:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991504+0000", 2024-09-13T10:10:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8220999999999994e-05 2024-09-13T10:10:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:10:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017232+0000", 2024-09-13T10:10:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025728328000000002 2024-09-13T10:10:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.031588+0000", 2024-09-13T10:10:55.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014355557 2024-09-13T10:10:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034211+0000", 2024-09-13T10:10:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026232790000000001 2024-09-13T10:10:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065168+0000", 2024-09-13T10:10:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030956833999999999 2024-09-13T10:10:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065260+0000", 2024-09-13T10:10:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2044000000000002e-05 2024-09-13T10:10:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065303+0000", 2024-09-13T10:10:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3152999999999998e-05 2024-09-13T10:10:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:211 3.0 3:0596651d:::smithi19530714-12 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:10:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.0264165749999998, 2024-09-13T10:10:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.042923122000000001, 2024-09-13T10:10:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 211 2024-09-13T10:10:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.041 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:10:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:10:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991995+0000", 2024-09-13T10:10:55.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7640000000000001e-06 2024-09-13T10:10:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992007+0000", 2024-09-13T10:10:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2058999999999999e-05 2024-09-13T10:10:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992040+0000", 2024-09-13T10:10:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3466000000000001e-05 2024-09-13T10:10:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992059+0000", 2024-09-13T10:10:55.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8485999999999999e-05 2024-09-13T10:10:55.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992136+0000", 2024-09-13T10:10:55.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6994000000000002e-05 2024-09-13T10:10:55.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992207+0000", 2024-09-13T10:10:55.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1357000000000001e-05 2024-09-13T10:10:55.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:10:55.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.006377+0000", 2024-09-13T10:10:55.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014170455 2024-09-13T10:10:55.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.022844+0000", 2024-09-13T10:10:55.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016466275999999998 2024-09-13T10:10:55.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023031+0000", 2024-09-13T10:10:55.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018686000000000001 2024-09-13T10:10:55.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034788+0000", 2024-09-13T10:10:55.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011756967 2024-09-13T10:10:55.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034857+0000", 2024-09-13T10:10:55.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9169000000000001e-05 2024-09-13T10:10:55.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034915+0000", 2024-09-13T10:10:55.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8269000000000001e-05 2024-09-13T10:10:55.047 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.047 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.047 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.048 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:212 3.0 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.048 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:10:55.048 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.0263102420000001, 2024-09-13T10:10:55.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.073011809999999996, 2024-09-13T10:10:55.048 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.049 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.049 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.049 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.049 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.049 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.049 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 212 2024-09-13T10:10:55.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.049 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:10:55.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:10:55.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992099+0000", 2024-09-13T10:10:55.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4899999999999995e-07 2024-09-13T10:10:55.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992103+0000", 2024-09-13T10:10:55.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.33e-06 2024-09-13T10:10:55.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992110+0000", 2024-09-13T10:10:55.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8399999999999997e-06 2024-09-13T10:10:55.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992116+0000", 2024-09-13T10:10:55.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6790000000000002e-06 2024-09-13T10:10:55.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992292+0000", 2024-09-13T10:10:55.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017659199999999999 2024-09-13T10:10:55.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992385+0000", 2024-09-13T10:10:55.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2608000000000005e-05 2024-09-13T10:10:55.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:10:55.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011428+0000", 2024-09-13T10:10:55.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.019042631000000001 2024-09-13T10:10:55.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.022852+0000", 2024-09-13T10:10:55.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011424528999999999 2024-09-13T10:10:55.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.031660+0000", 2024-09-13T10:10:55.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088079899999999999 2024-09-13T10:10:55.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065032+0000", 2024-09-13T10:10:55.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033372247000000001 2024-09-13T10:10:55.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065073+0000", 2024-09-13T10:10:55.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0598e-05 2024-09-13T10:10:55.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065110+0000", 2024-09-13T10:10:55.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7116999999999997e-05 2024-09-13T10:10:55.058 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.058 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.058 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.058 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:202 3.2 e38/30 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'1, mlcod=0'0)", 2024-09-13T10:10:55.058 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:10:55.058 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.0168196360000001, 2024-09-13T10:10:55.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017280298999999999, 2024-09-13T10:10:55.058 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.058 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.059 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.059 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:10:55.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:10:55.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001594+0000", 2024-09-13T10:10:55.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5550000000000002e-06 2024-09-13T10:10:55.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009486+0000", 2024-09-13T10:10:55.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078916799999999999 2024-09-13T10:10:55.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009519+0000", 2024-09-13T10:10:55.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2898999999999998e-05 2024-09-13T10:10:55.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009540+0000", 2024-09-13T10:10:55.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1559000000000001e-05 2024-09-13T10:10:55.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009655+0000", 2024-09-13T10:10:55.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114271 2024-09-13T10:10:55.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009683+0000", 2024-09-13T10:10:55.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8379000000000001e-05 2024-09-13T10:10:55.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.018766+0000", 2024-09-13T10:10:55.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0090829589999999998 2024-09-13T10:10:55.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.018869+0000", 2024-09-13T10:10:55.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102997 2024-09-13T10:10:55.063 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.063 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.063 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.063 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:216 3.0 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.063 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:10:55.063 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.0138071000000002, 2024-09-13T10:10:55.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.063936142000000001, 2024-09-13T10:10:55.063 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.063 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.063 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.063 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.063 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.063 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.064 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 216 2024-09-13T10:10:55.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.064 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:10:55.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:10:55.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004603+0000", 2024-09-13T10:10:55.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5540000000000001e-06 2024-09-13T10:10:55.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004615+0000", 2024-09-13T10:10:55.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2597e-05 2024-09-13T10:10:55.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004637+0000", 2024-09-13T10:10:55.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1713e-05 2024-09-13T10:10:55.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004651+0000", 2024-09-13T10:10:55.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3878e-05 2024-09-13T10:10:55.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004985+0000", 2024-09-13T10:10:55.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033347299999999998 2024-09-13T10:10:55.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.005706+0000", 2024-09-13T10:10:55.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00072122199999999999 2024-09-13T10:10:55.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:10:55.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023801+0000", 2024-09-13T10:10:55.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018095466000000001 2024-09-13T10:10:55.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.032153+0000", 2024-09-13T10:10:55.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083516010000000002 2024-09-13T10:10:55.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.035250+0000", 2024-09-13T10:10:55.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030974409999999998 2024-09-13T10:10:55.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068439+0000", 2024-09-13T10:10:55.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033189042000000002 2024-09-13T10:10:55.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068499+0000", 2024-09-13T10:10:55.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9969000000000001e-05 2024-09-13T10:10:55.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068537+0000", 2024-09-13T10:10:55.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8186000000000003e-05 2024-09-13T10:10:55.069 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.069 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.069 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.070 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:221 3.0 3:18ae852b:::smithi19530714-22 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.070 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:10:55.070 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9798825930000001, 2024-09-13T10:10:55.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039809298, 2024-09-13T10:10:55.070 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.070 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.070 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.070 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.070 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.070 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.070 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 221 2024-09-13T10:10:55.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.070 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:10:55.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:10:55.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038528+0000", 2024-09-13T10:10:55.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8819999999999999e-06 2024-09-13T10:10:55.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038534+0000", 2024-09-13T10:10:55.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0730000000000001e-06 2024-09-13T10:10:55.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038546+0000", 2024-09-13T10:10:55.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.219e-05 2024-09-13T10:10:55.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038566+0000", 2024-09-13T10:10:55.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9867000000000001e-05 2024-09-13T10:10:55.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038608+0000", 2024-09-13T10:10:55.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1977999999999997e-05 2024-09-13T10:10:55.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038707+0000", 2024-09-13T10:10:55.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8708000000000004e-05 2024-09-13T10:10:55.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:10:55.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.057374+0000", 2024-09-13T10:10:55.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018667652999999999 2024-09-13T10:10:55.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061983+0000", 2024-09-13T10:10:55.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0046087089999999999 2024-09-13T10:10:55.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.063027+0000", 2024-09-13T10:10:55.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001044239 2024-09-13T10:10:55.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078233+0000", 2024-09-13T10:10:55.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015205639999999999 2024-09-13T10:10:55.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078298+0000", 2024-09-13T10:10:55.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5180999999999998e-05 2024-09-13T10:10:55.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078335+0000", 2024-09-13T10:10:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7178000000000002e-05 2024-09-13T10:10:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:225 3.0 3:26960251:::smithi19530714-26 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:10:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9599563189999998, 2024-09-13T10:10:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030704875999999999, 2024-09-13T10:10:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.076 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.077 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.077 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.077 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.077 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.077 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 225 2024-09-13T10:10:55.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.077 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:10:55.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:10:55.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058455+0000", 2024-09-13T10:10:55.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3220000000000001e-06 2024-09-13T10:10:55.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058467+0000", 2024-09-13T10:10:55.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1987999999999999e-05 2024-09-13T10:10:55.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058502+0000", 2024-09-13T10:10:55.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4873000000000003e-05 2024-09-13T10:10:55.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058520+0000", 2024-09-13T10:10:55.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7977000000000001e-05 2024-09-13T10:10:55.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060066+0000", 2024-09-13T10:10:55.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001545261 2024-09-13T10:10:55.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060680+0000", 2024-09-13T10:10:55.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00061431799999999998 2024-09-13T10:10:55.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:10:55.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070650+0000", 2024-09-13T10:10:55.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0099703099999999996 2024-09-13T10:10:55.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078192+0000", 2024-09-13T10:10:55.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0075419709999999997 2024-09-13T10:10:55.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084486+0000", 2024-09-13T10:10:55.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0062943779999999998 2024-09-13T10:10:55.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089029+0000", 2024-09-13T10:10:55.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045422379999999997 2024-09-13T10:10:55.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089092+0000", 2024-09-13T10:10:55.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3435999999999994e-05 2024-09-13T10:10:55.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089157+0000", 2024-09-13T10:10:55.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4803999999999998e-05 2024-09-13T10:10:55.083 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.083 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.083 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.084 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:228 3.0 3:0168a995:::smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.084 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:10:55.084 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9581553610000002, 2024-09-13T10:10:55.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030139446, 2024-09-13T10:10:55.084 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.084 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.084 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.084 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.084 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.084 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.084 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 228 2024-09-13T10:10:55.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.085 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:10:55.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:10:55.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060255+0000", 2024-09-13T10:10:55.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.787e-06 2024-09-13T10:10:55.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060262+0000", 2024-09-13T10:10:55.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8630000000000002e-06 2024-09-13T10:10:55.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060285+0000", 2024-09-13T10:10:55.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2935e-05 2024-09-13T10:10:55.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060294+0000", 2024-09-13T10:10:55.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3940000000000002e-06 2024-09-13T10:10:55.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061041+0000", 2024-09-13T10:10:55.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00074706700000000002 2024-09-13T10:10:55.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061148+0000", 2024-09-13T10:10:55.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107364 2024-09-13T10:10:55.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:10:55.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074199+0000", 2024-09-13T10:10:55.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013050447999999999 2024-09-13T10:10:55.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084506+0000", 2024-09-13T10:10:55.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010306789 2024-09-13T10:10:55.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089222+0000", 2024-09-13T10:10:55.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047163480000000004 2024-09-13T10:10:55.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090310+0000", 2024-09-13T10:10:55.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010880919999999999 2024-09-13T10:10:55.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090356+0000", 2024-09-13T10:10:55.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5522000000000002e-05 2024-09-13T10:10:55.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090393+0000", 2024-09-13T10:10:55.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6837e-05 2024-09-13T10:10:55.090 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.090 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.090 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.091 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:229 3.0 3:1e6cc58c:::smithi19530714-30 low_tier:head [copy-from ver 18] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.091 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:10:55.091 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9568489859999998, 2024-09-13T10:10:55.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033410479, 2024-09-13T10:10:55.091 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.091 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.091 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.091 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.091 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.091 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.091 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 229 2024-09-13T10:10:55.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.092 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:10:55.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:10:55.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061562+0000", 2024-09-13T10:10:55.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9730000000000002e-06 2024-09-13T10:10:55.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061577+0000", 2024-09-13T10:10:55.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5044e-05 2024-09-13T10:10:55.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061602+0000", 2024-09-13T10:10:55.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4494999999999998e-05 2024-09-13T10:10:55.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061615+0000", 2024-09-13T10:10:55.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3207000000000001e-05 2024-09-13T10:10:55.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061701+0000", 2024-09-13T10:10:55.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5694999999999999e-05 2024-09-13T10:10:55.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061871+0000", 2024-09-13T10:10:55.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017004100000000001 2024-09-13T10:10:55.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:10:55.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.079511+0000", 2024-09-13T10:10:55.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017639708 2024-09-13T10:10:55.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089295+0000", 2024-09-13T10:10:55.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0097846359999999993 2024-09-13T10:10:55.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089316+0000", 2024-09-13T10:10:55.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0288999999999998e-05 2024-09-13T10:10:55.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094875+0000", 2024-09-13T10:10:55.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0055598389999999996 2024-09-13T10:10:55.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094931+0000", 2024-09-13T10:10:55.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5556999999999997e-05 2024-09-13T10:10:55.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094970+0000", 2024-09-13T10:10:55.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8995000000000002e-05 2024-09-13T10:10:55.097 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.098 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.098 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.098 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:233 3.0 3:24ead787:::smithi19530714-34 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.098 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:10:55.098 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9526724820000001, 2024-09-13T10:10:55.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047200234000000001, 2024-09-13T10:10:55.098 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.098 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.098 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.098 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.098 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.098 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.099 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 233 2024-09-13T10:10:55.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.099 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:10:55.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:10:55.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065737+0000", 2024-09-13T10:10:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.283e-06 2024-09-13T10:10:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065745+0000", 2024-09-13T10:10:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3130000000000001e-06 2024-09-13T10:10:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065757+0000", 2024-09-13T10:10:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2627e-05 2024-09-13T10:10:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065765+0000", 2024-09-13T10:10:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4669999999999998e-06 2024-09-13T10:10:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065816+0000", 2024-09-13T10:10:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1545000000000002e-05 2024-09-13T10:10:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065940+0000", 2024-09-13T10:10:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012366699999999999 2024-09-13T10:10:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:10:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.086064+0000", 2024-09-13T10:10:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.020124487 2024-09-13T10:10:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090747+0000", 2024-09-13T10:10:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004682795 2024-09-13T10:10:55.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.091235+0000", 2024-09-13T10:10:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048779900000000002 2024-09-13T10:10:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112809+0000", 2024-09-13T10:10:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.021573967999999999 2024-09-13T10:10:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112878+0000", 2024-09-13T10:10:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9413999999999996e-05 2024-09-13T10:10:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112936+0000", 2024-09-13T10:10:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7868999999999998e-05 2024-09-13T10:10:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:227 3.2 e38/30 3:6ac7800c:::smithi19530714-28 low_tier:head v 38'8, mlcod=38'6)", 2024-09-13T10:10:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:10:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9443094190000001, 2024-09-13T10:10:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014433425999999999, 2024-09-13T10:10:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:10:55.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:10:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074101+0000", 2024-09-13T10:10:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.511e-06 2024-09-13T10:10:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083754+0000", 2024-09-13T10:10:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096533799999999996 2024-09-13T10:10:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083772+0000", 2024-09-13T10:10:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8241e-05 2024-09-13T10:10:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083782+0000", 2024-09-13T10:10:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.004e-05 2024-09-13T10:10:55.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083875+0000", 2024-09-13T10:10:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.268e-05 2024-09-13T10:10:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083918+0000", 2024-09-13T10:10:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-13T10:10:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.088456+0000", 2024-09-13T10:10:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045385540000000002 2024-09-13T10:10:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.088532+0000", 2024-09-13T10:10:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6069000000000006e-05 2024-09-13T10:10:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.109 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:239 3.0 3:078df4f4:::smithi19530714-40 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:10:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9291059069999998, 2024-09-13T10:10:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025192791999999999, 2024-09-13T10:10:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 239 2024-09-13T10:10:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:10:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:10:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089305+0000", 2024-09-13T10:10:55.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1710000000000001e-06 2024-09-13T10:10:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089312+0000", 2024-09-13T10:10:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6860000000000006e-06 2024-09-13T10:10:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089323+0000", 2024-09-13T10:10:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0467e-05 2024-09-13T10:10:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089330+0000", 2024-09-13T10:10:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3939999999999998e-06 2024-09-13T10:10:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089379+0000", 2024-09-13T10:10:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8990999999999999e-05 2024-09-13T10:10:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089543+0000", 2024-09-13T10:10:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016351700000000001 2024-09-13T10:10:55.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:10:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.098886+0000", 2024-09-13T10:10:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0093434980000000004 2024-09-13T10:10:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.105855+0000", 2024-09-13T10:10:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.006968279 2024-09-13T10:10:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.105868+0000", 2024-09-13T10:10:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3560999999999999e-05 2024-09-13T10:10:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114382+0000", 2024-09-13T10:10:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0085134730000000006 2024-09-13T10:10:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114458+0000", 2024-09-13T10:10:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6143999999999995e-05 2024-09-13T10:10:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114495+0000", 2024-09-13T10:10:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7611000000000003e-05 2024-09-13T10:10:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:232 3.2 e38/30 3:6538713c:::smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'10, mlcod=38'6)", 2024-09-13T10:10:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.093933+0000", 2024-09-13T10:10:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9244758719999999, 2024-09-13T10:10:55.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012304116, 2024-09-13T10:10:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.093933+0000", 2024-09-13T10:10:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.093933+0000", 2024-09-13T10:10:55.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.093933+0000", 2024-09-13T10:10:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.08e-07 2024-09-13T10:10:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.100900+0000", 2024-09-13T10:10:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0069670310000000003 2024-09-13T10:10:55.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.100928+0000", 2024-09-13T10:10:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8416999999999999e-05 2024-09-13T10:10:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.100946+0000", 2024-09-13T10:10:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7643e-05 2024-09-13T10:10:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.101024+0000", 2024-09-13T10:10:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8103999999999996e-05 2024-09-13T10:10:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.101050+0000", 2024-09-13T10:10:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5451000000000001e-05 2024-09-13T10:10:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.106184+0000", 2024-09-13T10:10:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0051339890000000003 2024-09-13T10:10:55.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.106237+0000", 2024-09-13T10:10:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3072999999999998e-05 2024-09-13T10:10:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:241 3.0 3:388ca573:::smithi19530714-42 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:10:55.121 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9232321849999998, 2024-09-13T10:10:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057595649999999998, 2024-09-13T10:10:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 241 2024-09-13T10:10:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:10:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:10:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095178+0000", 2024-09-13T10:10:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.345e-06 2024-09-13T10:10:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095184+0000", 2024-09-13T10:10:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4239999999999998e-06 2024-09-13T10:10:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095198+0000", 2024-09-13T10:10:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4037e-05 2024-09-13T10:10:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095206+0000", 2024-09-13T10:10:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1629999999999996e-06 2024-09-13T10:10:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095274+0000", 2024-09-13T10:10:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7948999999999996e-05 2024-09-13T10:10:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095485+0000", 2024-09-13T10:10:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021050100000000001 2024-09-13T10:10:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:10:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.134651+0000", 2024-09-13T10:10:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039166200999999998 2024-09-13T10:10:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.140536+0000", 2024-09-13T10:10:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058849719999999996 2024-09-13T10:10:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.140980+0000", 2024-09-13T10:10:55.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044394200000000001 2024-09-13T10:10:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152700+0000", 2024-09-13T10:10:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011720431999999999 2024-09-13T10:10:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152739+0000", 2024-09-13T10:10:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8448999999999997e-05 2024-09-13T10:10:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152772+0000", 2024-09-13T10:10:55.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3235000000000003e-05 2024-09-13T10:10:55.128 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.128 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.128 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.128 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:247 3.0 3:0b30fd85:::smithi19530714-48 low_tier:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.128 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:10:55.128 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.9053325340000002, 2024-09-13T10:10:55.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032448926000000003, 2024-09-13T10:10:55.128 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.129 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.129 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.129 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.129 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.129 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.129 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 247 2024-09-13T10:10:55.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.129 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:10:55.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:10:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113078+0000", 2024-09-13T10:10:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8670000000000001e-06 2024-09-13T10:10:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113086+0000", 2024-09-13T10:10:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2260000000000005e-06 2024-09-13T10:10:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113104+0000", 2024-09-13T10:10:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7887e-05 2024-09-13T10:10:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113115+0000", 2024-09-13T10:10:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0763e-05 2024-09-13T10:10:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113163+0000", 2024-09-13T10:10:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8541999999999997e-05 2024-09-13T10:10:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113322+0000", 2024-09-13T10:10:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000159121 2024-09-13T10:10:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:10:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.125165+0000", 2024-09-13T10:10:55.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011842320999999999 2024-09-13T10:10:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.131189+0000", 2024-09-13T10:10:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.006024728 2024-09-13T10:10:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.131203+0000", 2024-09-13T10:10:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3551e-05 2024-09-13T10:10:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145435+0000", 2024-09-13T10:10:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014232273 2024-09-13T10:10:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145481+0000", 2024-09-13T10:10:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5918000000000003e-05 2024-09-13T10:10:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145525+0000", 2024-09-13T10:10:55.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3729000000000001e-05 2024-09-13T10:10:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:244 3.2 e38/30 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'14, mlcod=38'10)", 2024-09-13T10:10:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:10:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.8918667309999999, 2024-09-13T10:10:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014967827, 2024-09-13T10:10:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.135 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:10:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:10:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126544+0000", 2024-09-13T10:10:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.03e-06 2024-09-13T10:10:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133374+0000", 2024-09-13T10:10:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068299119999999996 2024-09-13T10:10:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133398+0000", 2024-09-13T10:10:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4209000000000001e-05 2024-09-13T10:10:55.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133404+0000", 2024-09-13T10:10:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3570000000000003e-06 2024-09-13T10:10:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133457+0000", 2024-09-13T10:10:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2320000000000001e-05 2024-09-13T10:10:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133477+0000", 2024-09-13T10:10:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0721999999999999e-05 2024-09-13T10:10:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.141475+0000", 2024-09-13T10:10:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0079973649999999993 2024-09-13T10:10:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.141510+0000", 2024-09-13T10:10:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4912e-05 2024-09-13T10:10:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.139 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:254 4.8 4:1de9da87:::smithi19530714-5:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.214446+0000", 2024-09-13T10:10:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.8039623929999999, 2024-09-13T10:10:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014408274, 2024-09-13T10:10:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.140 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 254 2024-09-13T10:10:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214446+0000", 2024-09-13T10:10:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214446+0000", 2024-09-13T10:10:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214449+0000", 2024-09-13T10:10:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7860000000000001e-06 2024-09-13T10:10:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214464+0000", 2024-09-13T10:10:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5290000000000001e-05 2024-09-13T10:10:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214499+0000", 2024-09-13T10:10:55.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5119e-05 2024-09-13T10:10:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214517+0000", 2024-09-13T10:10:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7563000000000001e-05 2024-09-13T10:10:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214680+0000", 2024-09-13T10:10:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016348799999999999 2024-09-13T10:10:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214937+0000", 2024-09-13T10:10:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025682599999999998 2024-09-13T10:10:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0,11", 2024-09-13T10:10:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225660+0000", 2024-09-13T10:10:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010722768000000001 2024-09-13T10:10:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.227197+0000", 2024-09-13T10:10:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015373749999999999 2024-09-13T10:10:55.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228428+0000", 2024-09-13T10:10:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001230863 2024-09-13T10:10:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228646+0000", 2024-09-13T10:10:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021769700000000001 2024-09-13T10:10:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228755+0000", 2024-09-13T10:10:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109551 2024-09-13T10:10:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228854+0000", 2024-09-13T10:10:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8948000000000002e-05 2024-09-13T10:10:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(osd.11.22:16 3.0 3:1e59f26f:::f36fb4795b66b358920b15346e41df644c7e2d9386f5f260380eec916b18bcae:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:10:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.841613+0000", 2024-09-13T10:10:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.1767951179999998, 2024-09-13T10:10:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011923899, 2024-09-13T10:10:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841613+0000", 2024-09-13T10:10:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841613+0000", 2024-09-13T10:10:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841615+0000", 2024-09-13T10:10:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.593e-06 2024-09-13T10:10:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841739+0000", 2024-09-13T10:10:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012399000000000001 2024-09-13T10:10:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841762+0000", 2024-09-13T10:10:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3129000000000002e-05 2024-09-13T10:10:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841778+0000", 2024-09-13T10:10:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5982999999999999e-05 2024-09-13T10:10:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.848805+0000", 2024-09-13T10:10:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0070273339999999997 2024-09-13T10:10:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.848984+0000", 2024-09-13T10:10:55.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017825499999999999 2024-09-13T10:10:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:10:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.849268+0000", 2024-09-13T10:10:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028478200000000003 2024-09-13T10:10:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853103+0000", 2024-09-13T10:10:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0038350429999999998 2024-09-13T10:10:55.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853226+0000", 2024-09-13T10:10:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012296299999999999 2024-09-13T10:10:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853339+0000", 2024-09-13T10:10:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011251000000000001 2024-09-13T10:10:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853468+0000", 2024-09-13T10:10:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012881599999999999 2024-09-13T10:10:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853537+0000", 2024-09-13T10:10:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9500999999999997e-05 2024-09-13T10:10:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(osd.11.22:17 3.0 3:08307d25:::2b3cb797ae2d3abc24b2cae4a012f26325debe2fc38ccb08bd4503e949d305ac:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:10:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.841816+0000", 2024-09-13T10:10:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 3.1765920470000002, 2024-09-13T10:10:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011960942, 2024-09-13T10:10:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841816+0000", 2024-09-13T10:10:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841816+0000", 2024-09-13T10:10:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841818+0000", 2024-09-13T10:10:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.353e-06 2024-09-13T10:10:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841888+0000", 2024-09-13T10:10:55.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9808000000000007e-05 2024-09-13T10:10:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841905+0000", 2024-09-13T10:10:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7686e-05 2024-09-13T10:10:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841924+0000", 2024-09-13T10:10:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9176000000000001e-05 2024-09-13T10:10:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.850275+0000", 2024-09-13T10:10:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083507110000000002 2024-09-13T10:10:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.850536+0000", 2024-09-13T10:10:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026065000000000002 2024-09-13T10:10:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:10:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.851014+0000", 2024-09-13T10:10:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00047829799999999999 2024-09-13T10:10:55.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853304+0000", 2024-09-13T10:10:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002290233 2024-09-13T10:10:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853562+0000", 2024-09-13T10:10:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025763199999999999 2024-09-13T10:10:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853636+0000", 2024-09-13T10:10:55.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4196999999999995e-05 2024-09-13T10:10:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853710+0000", 2024-09-13T10:10:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4271999999999996e-05 2024-09-13T10:10:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853777+0000", 2024-09-13T10:10:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6926000000000003e-05 2024-09-13T10:10:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.159 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.160 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.160 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.160 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:10:55.162 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T10:10:55.163 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:55.163 INFO:tasks.rados.rados.0.smithi195.stdout:670: finishing write tid 1 to smithi19530714-38 2024-09-13T10:10:55.163 INFO:tasks.rados.rados.0.smithi195.stdout:670: finishing write tid 2 to smithi19530714-38 2024-09-13T10:10:55.163 INFO:tasks.rados.rados.0.smithi195.stdout:670: finishing write tid 3 to smithi19530714-38 2024-09-13T10:10:55.163 INFO:tasks.rados.rados.0.smithi195.stdout:670: finishing write tid 4 to smithi19530714-38 2024-09-13T10:10:55.164 INFO:tasks.rados.rados.0.smithi195.stdout:670: oid 38 updating version 0 to 46 2024-09-13T10:10:55.164 INFO:tasks.rados.rados.0.smithi195.stdout:670: oid 38 version 46 is already newer than 45 2024-09-13T10:10:55.164 INFO:tasks.rados.rados.0.smithi195.stdout:670: oid 38 version 46 is already newer than 44 2024-09-13T10:10:55.164 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 46 (ObjNum 71 snap 0 seq_num 71) dirty exists 2024-09-13T10:10:55.164 INFO:tasks.rados.rados.0.smithi195.stdout:670: left oid 38 (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:10:55.164 INFO:tasks.rados.rados.0.smithi195.stdout:672: seq_num 72 ranges {410104=465105,1563527=583025,2810926=399178} 2024-09-13T10:10:55.164 INFO:tasks.rados.rados.0.smithi195.stdout:672: writing smithi19530714-44 from 410104 to 875209 tid 1 2024-09-13T10:10:55.164 INFO:tasks.rados.rados.0.smithi195.stdout:672: writing smithi19530714-44 from 1563527 to 2146552 tid 2 2024-09-13T10:10:55.164 INFO:tasks.rados.rados.0.smithi195.stdout:672: writing smithi19530714-44 from 2810926 to 3210104 tid 3 2024-09-13T10:10:55.164 INFO:tasks.rados.rados.0.smithi195.stdout:666: done (6 left) 2024-09-13T10:10:55.164 INFO:tasks.rados.rados.0.smithi195.stdout:667: done (5 left) 2024-09-13T10:10:55.164 INFO:tasks.rados.rados.0.smithi195.stdout:668: done (4 left) 2024-09-13T10:10:55.164 INFO:tasks.rados.rados.0.smithi195.stdout:669: done (3 left) 2024-09-13T10:10:55.165 INFO:tasks.rados.rados.0.smithi195.stdout:670: done (2 left) 2024-09-13T10:10:55.165 INFO:tasks.rados.rados.0.smithi195.stdout:671: done (1 left) 2024-09-13T10:10:55.165 INFO:tasks.rados.rados.0.smithi195.stdout:673: set_chunk oid 45 target oid 2024-09-13T10:10:55.165 INFO:tasks.rados.rados.0.smithi195.stdout:673: set_chunk oid 45 offset: 756736 length: 1038336 target oid 0ce0556d49e3d107a4b09599016178b0eacb474223b61814958b982e2164bc90 offset: 0 snap -1 2024-09-13T10:10:55.165 INFO:tasks.rados.rados.0.smithi195.stdout:674: read oid 4 snap -1 2024-09-13T10:10:55.165 INFO:tasks.rados.rados.0.smithi195.stdout:674: expect (ObjNum 58 snap 0 seq_num 58) 2024-09-13T10:10:55.165 INFO:tasks.rados.rados.0.smithi195.stdout:675: write oid 37 current snap is 0 2024-09-13T10:10:55.165 INFO:tasks.rados.rados.0.smithi195.stdout:675: seq_num 73 ranges {426911=540354,1622009=726683,3121996=104915} 2024-09-13T10:10:55.165 INFO:tasks.rados.rados.0.smithi195.stdout:675: writing smithi19530714-37 from 426911 to 967265 tid 1 2024-09-13T10:10:55.165 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:10:55.159+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:10:55.170 INFO:tasks.rados.rados.0.smithi195.stdout:675: writing smithi19530714-37 from 1622009 to 2348692 tid 2 2024-09-13T10:10:55.180 INFO:tasks.rados.rados.0.smithi195.stdout:675: writing smithi19530714-37 from 3121996 to 3226911 tid 3 2024-09-13T10:10:55.181 INFO:tasks.rados.rados.0.smithi195.stdout:676: read oid 2 snap -1 2024-09-13T10:10:55.181 INFO:tasks.rados.rados.0.smithi195.stdout:676: expect (ObjNum 68 snap 0 seq_num 68) 2024-09-13T10:10:55.181 INFO:tasks.rados.rados.0.smithi195.stdout:677: read oid 20 snap -1 2024-09-13T10:10:55.181 INFO:tasks.rados.rados.0.smithi195.stdout:677: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-13T10:10:55.181 INFO:tasks.rados.rados.0.smithi195.stdout:678: read oid 34 snap -1 2024-09-13T10:10:55.181 INFO:tasks.rados.rados.0.smithi195.stdout:678: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-13T10:10:55.181 INFO:tasks.rados.rados.0.smithi195.stdout:679: delete oid 30 current snap is 0 2024-09-13T10:10:55.181 INFO:tasks.rados.rados.0.smithi195.stdout:672: finishing write tid 1 to smithi19530714-44 2024-09-13T10:10:55.181 INFO:tasks.rados.rados.0.smithi195.stdout:672: finishing write tid 2 to smithi19530714-44 2024-09-13T10:10:55.181 INFO:tasks.rados.rados.0.smithi195.stdout:672: finishing write tid 3 to smithi19530714-44 2024-09-13T10:10:55.181 INFO:tasks.rados.rados.0.smithi195.stdout:672: finishing write tid 4 to smithi19530714-44 2024-09-13T10:10:55.181 INFO:tasks.rados.rados.0.smithi195.stdout:672: finishing write tid 5 to smithi19530714-44 2024-09-13T10:10:55.182 INFO:tasks.rados.rados.0.smithi195.stdout:672: oid 44 updating version 0 to 29 2024-09-13T10:10:55.182 INFO:tasks.rados.rados.0.smithi195.stdout:672: oid 44 updating version 29 to 30 2024-09-13T10:10:55.182 INFO:tasks.rados.rados.0.smithi195.stdout:672: oid 44 version 30 is already newer than 27 2024-09-13T10:10:55.182 INFO:tasks.rados.rados.0.smithi195.stdout:672: oid 44 version 30 is already newer than 28 2024-09-13T10:10:55.182 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 30 (ObjNum 72 snap 0 seq_num 72) dirty exists 2024-09-13T10:10:55.182 INFO:tasks.rados.rados.0.smithi195.stdout:672: left oid 44 (ObjNum 72 snap 0 seq_num 72) 2024-09-13T10:10:55.182 INFO:tasks.rados.rados.0.smithi195.stdout:673: finishing set_chunk to oid 45 2024-09-13T10:10:55.182 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 40 (ObjNum 59 snap 0 seq_num 59) dirty exists 2024-09-13T10:10:55.182 INFO:tasks.rados.rados.0.smithi195.stdout:674: expect (ObjNum 58 snap 0 seq_num 58) 2024-09-13T10:10:55.262 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:10:55.261+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:10:55.300 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:10:55.300 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:10:55.300 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:10:55.301 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:10:55.303 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:55.314 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T10:10:55.349 INFO:tasks.rados.rados.0.smithi195.stdout:675: finishing write tid 1 to smithi19530714-37 2024-09-13T10:10:55.350 INFO:tasks.rados.rados.0.smithi195.stdout:675: finishing write tid 2 to smithi19530714-37 2024-09-13T10:10:55.350 INFO:tasks.rados.rados.0.smithi195.stdout:672: done (7 left) 2024-09-13T10:10:55.350 INFO:tasks.rados.rados.0.smithi195.stdout:673: done (6 left) 2024-09-13T10:10:55.350 INFO:tasks.rados.rados.0.smithi195.stdout:674: done (5 left) 2024-09-13T10:10:55.350 INFO:tasks.rados.rados.0.smithi195.stdout:679: done (4 left) 2024-09-13T10:10:55.351 INFO:tasks.rados.rados.0.smithi195.stdout:680: delete oid 48 current snap is 0 2024-09-13T10:10:55.351 INFO:tasks.rados.rados.0.smithi195.stdout:675: finishing write tid 3 to smithi19530714-37 2024-09-13T10:10:55.351 INFO:tasks.rados.rados.0.smithi195.stdout:675: finishing write tid 4 to smithi19530714-37 2024-09-13T10:10:55.351 INFO:tasks.rados.rados.0.smithi195.stdout:675: finishing write tid 5 to smithi19530714-37 2024-09-13T10:10:55.351 INFO:tasks.rados.rados.0.smithi195.stdout:675: oid 37 updating version 0 to 8 2024-09-13T10:10:55.351 INFO:tasks.rados.rados.0.smithi195.stdout:675: oid 37 updating version 8 to 10 2024-09-13T10:10:55.352 INFO:tasks.rados.rados.0.smithi195.stdout:675: oid 37 version 10 is already newer than 9 2024-09-13T10:10:55.352 INFO:tasks.rados.rados.0.smithi195.stdout:675: oid 37 version 10 is already newer than 7 2024-09-13T10:10:55.352 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 10 (ObjNum 73 snap 0 seq_num 73) dirty exists 2024-09-13T10:10:55.352 INFO:tasks.rados.rados.0.smithi195.stdout:675: left oid 37 (ObjNum 73 snap 0 seq_num 73) 2024-09-13T10:10:55.352 INFO:tasks.rados.rados.0.smithi195.stdout:676: expect (ObjNum 68 snap 0 seq_num 68) 2024-09-13T10:10:55.399 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:10:55.399+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:10:55.460 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1845563 2024-09-13T10:10:55.465 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:10:55.461+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:10:55.471 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:10:55.471 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:10:55.471 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:10:55.471 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:10:55.471 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:10:55.478 INFO:tasks.rados.rados.0.smithi195.stdout:677: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-13T10:10:55.478 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 362909 2024-09-13T10:10:55.486 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T10:10:55.506 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1415075 2024-09-13T10:10:55.519 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:55.545 INFO:tasks.rados.rados.0.smithi195.stdout:678: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-13T10:10:55.545 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:55.566 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:10:55.564+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:10:55.633 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1563710 2024-09-13T10:10:55.636 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:10:55.636 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:10:55.636 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:10:55.636 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:10:55.636 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.637 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:42 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'24, mlcod=25'19)", 2024-09-13T10:10:55.637 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.285251+0000", 2024-09-13T10:10:55.637 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.348039930999999, 2024-09-13T10:10:55.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.008704425, 2024-09-13T10:10:55.637 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.637 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.637 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.637 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.637 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.637 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.637 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285251+0000", 2024-09-13T10:10:55.637 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.637 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285251+0000", 2024-09-13T10:10:55.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285252+0000", 2024-09-13T10:10:55.638 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.33e-06 2024-09-13T10:10:55.638 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.638 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.638 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.638 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285294+0000", 2024-09-13T10:10:55.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1148e-05 2024-09-13T10:10:55.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.639 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.639 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285331+0000", 2024-09-13T10:10:55.639 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7478000000000003e-05 2024-09-13T10:10:55.639 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.639 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285355+0000", 2024-09-13T10:10:55.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3524999999999999e-05 2024-09-13T10:10:55.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286264+0000", 2024-09-13T10:10:55.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090990400000000001 2024-09-13T10:10:55.640 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.640 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.640 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.640 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286300+0000", 2024-09-13T10:10:55.640 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5395000000000002e-05 2024-09-13T10:10:55.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.641 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.641 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.293876+0000", 2024-09-13T10:10:55.641 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0075758589999999999 2024-09-13T10:10:55.641 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.641 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.642 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.642 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.293955+0000", 2024-09-13T10:10:55.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9785999999999998e-05 2024-09-13T10:10:55.642 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.642 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.642 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.642 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.642 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.642 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:43 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'25, mlcod=25'19)", 2024-09-13T10:10:55.642 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.285625+0000", 2024-09-13T10:10:55.642 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.347666057000001, 2024-09-13T10:10:55.642 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0084062870000000001, 2024-09-13T10:10:55.642 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.643 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.643 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.643 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.643 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.643 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.643 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285625+0000", 2024-09-13T10:10:55.643 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.643 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285625+0000", 2024-09-13T10:10:55.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285626+0000", 2024-09-13T10:10:55.644 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.297e-06 2024-09-13T10:10:55.644 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.644 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.644 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.644 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285638+0000", 2024-09-13T10:10:55.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1365999999999999e-05 2024-09-13T10:10:55.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285657+0000", 2024-09-13T10:10:55.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9612000000000001e-05 2024-09-13T10:10:55.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.645 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.645 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285672+0000", 2024-09-13T10:10:55.645 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5082999999999999e-05 2024-09-13T10:10:55.645 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.645 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287280+0000", 2024-09-13T10:10:55.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001607491 2024-09-13T10:10:55.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287334+0000", 2024-09-13T10:10:55.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3820000000000003e-05 2024-09-13T10:10:55.646 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.646 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.646 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.293963+0000", 2024-09-13T10:10:55.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066292290000000004 2024-09-13T10:10:55.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.294031+0000", 2024-09-13T10:10:55.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8388999999999999e-05 2024-09-13T10:10:55.647 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.647 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.647 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.647 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:45 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'27, mlcod=25'19)", 2024-09-13T10:10:55.647 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.286596+0000", 2024-09-13T10:10:55.647 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.346695245999999, 2024-09-13T10:10:55.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0093831250000000008, 2024-09-13T10:10:55.648 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.648 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.648 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.648 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286596+0000", 2024-09-13T10:10:55.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286596+0000", 2024-09-13T10:10:55.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286598+0000", 2024-09-13T10:10:55.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8500000000000001e-06 2024-09-13T10:10:55.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286611+0000", 2024-09-13T10:10:55.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3456e-05 2024-09-13T10:10:55.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286635+0000", 2024-09-13T10:10:55.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4165e-05 2024-09-13T10:10:55.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286655+0000", 2024-09-13T10:10:55.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9383000000000001e-05 2024-09-13T10:10:55.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289627+0000", 2024-09-13T10:10:55.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029728900000000002 2024-09-13T10:10:55.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289685+0000", 2024-09-13T10:10:55.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7040000000000003e-05 2024-09-13T10:10:55.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.295871+0000", 2024-09-13T10:10:55.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061868629999999999 2024-09-13T10:10:55.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.295979+0000", 2024-09-13T10:10:55.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107478 2024-09-13T10:10:55.653 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.653 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.653 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.653 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:46 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'28, mlcod=25'19)", 2024-09-13T10:10:55.653 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.286979+0000", 2024-09-13T10:10:55.653 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.346312335, 2024-09-13T10:10:55.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0090737700000000001, 2024-09-13T10:10:55.654 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.654 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.654 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.654 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286979+0000", 2024-09-13T10:10:55.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286979+0000", 2024-09-13T10:10:55.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286981+0000", 2024-09-13T10:10:55.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0880000000000002e-06 2024-09-13T10:10:55.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286994+0000", 2024-09-13T10:10:55.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2884999999999999e-05 2024-09-13T10:10:55.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287017+0000", 2024-09-13T10:10:55.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3408e-05 2024-09-13T10:10:55.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287033+0000", 2024-09-13T10:10:55.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.641e-05 2024-09-13T10:10:55.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.290659+0000", 2024-09-13T10:10:55.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036255749999999998 2024-09-13T10:10:55.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.290692+0000", 2024-09-13T10:10:55.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3154999999999997e-05 2024-09-13T10:10:55.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.295986+0000", 2024-09-13T10:10:55.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0052934180000000003 2024-09-13T10:10:55.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296052+0000", 2024-09-13T10:10:55.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6830999999999998e-05 2024-09-13T10:10:55.659 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.659 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.659 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.659 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:47 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'29, mlcod=25'20)", 2024-09-13T10:10:55.659 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.287544+0000", 2024-09-13T10:10:55.659 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.345747382999999, 2024-09-13T10:10:55.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0085821309999999998, 2024-09-13T10:10:55.659 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.660 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.660 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.660 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287544+0000", 2024-09-13T10:10:55.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287544+0000", 2024-09-13T10:10:55.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287545+0000", 2024-09-13T10:10:55.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.646e-06 2024-09-13T10:10:55.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287727+0000", 2024-09-13T10:10:55.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000181528 2024-09-13T10:10:55.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287749+0000", 2024-09-13T10:10:55.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1744999999999999e-05 2024-09-13T10:10:55.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287765+0000", 2024-09-13T10:10:55.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.649e-05 2024-09-13T10:10:55.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.291452+0000", 2024-09-13T10:10:55.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036870370000000002 2024-09-13T10:10:55.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.291506+0000", 2024-09-13T10:10:55.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4323000000000001e-05 2024-09-13T10:10:55.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296064+0000", 2024-09-13T10:10:55.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045575340000000002 2024-09-13T10:10:55.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296126+0000", 2024-09-13T10:10:55.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1828000000000005e-05 2024-09-13T10:10:55.663 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.663 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.664 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.664 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:48 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'30, mlcod=25'20)", 2024-09-13T10:10:55.664 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.288004+0000", 2024-09-13T10:10:55.664 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.345286952999999, 2024-09-13T10:10:55.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0084544640000000001, 2024-09-13T10:10:55.664 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.664 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.664 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.664 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288004+0000", 2024-09-13T10:10:55.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288004+0000", 2024-09-13T10:10:55.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288006+0000", 2024-09-13T10:10:55.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.657e-06 2024-09-13T10:10:55.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288019+0000", 2024-09-13T10:10:55.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3193999999999999e-05 2024-09-13T10:10:55.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288039+0000", 2024-09-13T10:10:55.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0466999999999999e-05 2024-09-13T10:10:55.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288055+0000", 2024-09-13T10:10:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5716999999999999e-05 2024-09-13T10:10:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.292441+0000", 2024-09-13T10:10:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0043859190000000003 2024-09-13T10:10:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.292470+0000", 2024-09-13T10:10:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9258999999999999e-05 2024-09-13T10:10:55.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296132+0000", 2024-09-13T10:10:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036618309999999999 2024-09-13T10:10:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296459+0000", 2024-09-13T10:10:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032642000000000001 2024-09-13T10:10:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:49 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'31, mlcod=25'21)", 2024-09-13T10:10:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.288570+0000", 2024-09-13T10:10:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.344720724999998, 2024-09-13T10:10:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0081216099999999996, 2024-09-13T10:10:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288570+0000", 2024-09-13T10:10:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288570+0000", 2024-09-13T10:10:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288572+0000", 2024-09-13T10:10:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.64e-06 2024-09-13T10:10:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288583+0000", 2024-09-13T10:10:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1208e-05 2024-09-13T10:10:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288604+0000", 2024-09-13T10:10:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1111000000000002e-05 2024-09-13T10:10:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.288621+0000", 2024-09-13T10:10:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7025e-05 2024-09-13T10:10:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.293038+0000", 2024-09-13T10:10:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0044171419999999998 2024-09-13T10:10:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.293097+0000", 2024-09-13T10:10:55.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8264000000000003e-05 2024-09-13T10:10:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296476+0000", 2024-09-13T10:10:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033792969999999999 2024-09-13T10:10:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296692+0000", 2024-09-13T10:10:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000215923 2024-09-13T10:10:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:21 4.6 4:6150ba38:::smithi19530714-6:head [write 484035~776245,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.390657+0000", 2024-09-13T10:10:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.2426337749999998, 2024-09-13T10:10:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088615770000000007, 2024-09-13T10:10:55.676 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 21 2024-09-13T10:10:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.390657+0000", 2024-09-13T10:10:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.390657+0000", 2024-09-13T10:10:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.390660+0000", 2024-09-13T10:10:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1970000000000001e-06 2024-09-13T10:10:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.678 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.391452+0000", 2024-09-13T10:10:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079113899999999995 2024-09-13T10:10:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.391487+0000", 2024-09-13T10:10:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5116000000000001e-05 2024-09-13T10:10:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.391504+0000", 2024-09-13T10:10:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7010000000000001e-05 2024-09-13T10:10:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.391641+0000", 2024-09-13T10:10:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013700499999999999 2024-09-13T10:10:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.391867+0000", 2024-09-13T10:10:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000226677 2024-09-13T10:10:55.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:10:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.392145+0000", 2024-09-13T10:10:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000277368 2024-09-13T10:10:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.395418+0000", 2024-09-13T10:10:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032730530000000002 2024-09-13T10:10:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.396126+0000", 2024-09-13T10:10:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00070789900000000001 2024-09-13T10:10:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.399358+0000", 2024-09-13T10:10:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032325209999999999 2024-09-13T10:10:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.399436+0000", 2024-09-13T10:10:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7834000000000001e-05 2024-09-13T10:10:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.399519+0000", 2024-09-13T10:10:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2757999999999996e-05 2024-09-13T10:10:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:211 3.0 e38/30 3:0596651d:::smithi19530714-12 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:10:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:10:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.6249720820000002, 2024-09-13T10:10:55.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013316957000000001, 2024-09-13T10:10:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:10:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:10:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008321+0000", 2024-09-13T10:10:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-13T10:10:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011069+0000", 2024-09-13T10:10:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002747893 2024-09-13T10:10:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011113+0000", 2024-09-13T10:10:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3921000000000002e-05 2024-09-13T10:10:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011140+0000", 2024-09-13T10:10:55.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7662999999999999e-05 2024-09-13T10:10:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011313+0000", 2024-09-13T10:10:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017232200000000001 2024-09-13T10:10:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011366+0000", 2024-09-13T10:10:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3888000000000003e-05 2024-09-13T10:10:55.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021577+0000", 2024-09-13T10:10:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010210472999999999 2024-09-13T10:10:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021636+0000", 2024-09-13T10:10:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8953999999999998e-05 2024-09-13T10:10:55.688 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:212 3.0 e38/30 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:10:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:10:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.6216549760000003, 2024-09-13T10:10:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014217904, 2024-09-13T10:10:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.689 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:10:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:10:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011638+0000", 2024-09-13T10:10:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.672e-06 2024-09-13T10:10:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013289+0000", 2024-09-13T10:10:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001651511 2024-09-13T10:10:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013310+0000", 2024-09-13T10:10:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0917999999999999e-05 2024-09-13T10:10:55.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013322+0000", 2024-09-13T10:10:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2067e-05 2024-09-13T10:10:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.016882+0000", 2024-09-13T10:10:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035599130000000001 2024-09-13T10:10:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.016934+0000", 2024-09-13T10:10:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1715999999999997e-05 2024-09-13T10:10:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025787+0000", 2024-09-13T10:10:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088531809999999999 2024-09-13T10:10:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025854+0000", 2024-09-13T10:10:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6926000000000003e-05 2024-09-13T10:10:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.693 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:207 3.0 e38/30 3:332cd44b:::smithi19530714-8 low_tier:head v 38'3, mlcod=0'0)", 2024-09-13T10:10:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:10:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.6158615349999996, 2024-09-13T10:10:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016660892, 2024-09-13T10:10:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:10:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:10:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017432+0000", 2024-09-13T10:10:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1490000000000001e-06 2024-09-13T10:10:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021310+0000", 2024-09-13T10:10:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003877985 2024-09-13T10:10:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021351+0000", 2024-09-13T10:10:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1105e-05 2024-09-13T10:10:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021376+0000", 2024-09-13T10:10:55.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5185999999999999e-05 2024-09-13T10:10:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021727+0000", 2024-09-13T10:10:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035131300000000002 2024-09-13T10:10:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021792+0000", 2024-09-13T10:10:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4368000000000004e-05 2024-09-13T10:10:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034027+0000", 2024-09-13T10:10:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012235582 2024-09-13T10:10:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034090+0000", 2024-09-13T10:10:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3204e-05 2024-09-13T10:10:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:216 3.0 e38/30 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:10:55.698 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:10:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.6092959650000003, 2024-09-13T10:10:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011044523000000001, 2024-09-13T10:10:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:10:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:10:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023997+0000", 2024-09-13T10:10:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.533e-06 2024-09-13T10:10:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024872+0000", 2024-09-13T10:10:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087517699999999999 2024-09-13T10:10:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024894+0000", 2024-09-13T10:10:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2571000000000001e-05 2024-09-13T10:10:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024909+0000", 2024-09-13T10:10:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5158e-05 2024-09-13T10:10:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025979+0000", 2024-09-13T10:10:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010692329999999999 2024-09-13T10:10:55.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.026027+0000", 2024-09-13T10:10:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8158000000000002e-05 2024-09-13T10:10:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034992+0000", 2024-09-13T10:10:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0089650960000000005 2024-09-13T10:10:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.035040+0000", 2024-09-13T10:10:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7596999999999998e-05 2024-09-13T10:10:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:225 3.0 e38/30 3:26960251:::smithi19530714-26 low_tier:head v 38'6, mlcod=38'3)", 2024-09-13T10:10:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:10:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5623988879999997, 2024-09-13T10:10:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013205889, 2024-09-13T10:10:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.703 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:10:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:10:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070894+0000", 2024-09-13T10:10:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5e-06 2024-09-13T10:10:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072562+0000", 2024-09-13T10:10:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016683469999999999 2024-09-13T10:10:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072632+0000", 2024-09-13T10:10:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9665000000000005e-05 2024-09-13T10:10:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072661+0000", 2024-09-13T10:10:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9714999999999999e-05 2024-09-13T10:10:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072800+0000", 2024-09-13T10:10:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013886000000000001 2024-09-13T10:10:55.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072828+0000", 2024-09-13T10:10:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7886e-05 2024-09-13T10:10:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084041+0000", 2024-09-13T10:10:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011213064999999999 2024-09-13T10:10:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084098+0000", 2024-09-13T10:10:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6851000000000002e-05 2024-09-13T10:10:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:228 3.0 e38/30 3:0168a995:::smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'7, mlcod=38'3)", 2024-09-13T10:10:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:10:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.5586153080000003, 2024-09-13T10:10:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094647819999999997, 2024-09-13T10:10:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.708 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:10:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:10:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074677+0000", 2024-09-13T10:10:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7099999999999999e-06 2024-09-13T10:10:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076599+0000", 2024-09-13T10:10:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019212459999999999 2024-09-13T10:10:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076616+0000", 2024-09-13T10:10:55.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7008999999999999e-05 2024-09-13T10:10:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076641+0000", 2024-09-13T10:10:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5380000000000001e-05 2024-09-13T10:10:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076772+0000", 2024-09-13T10:10:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013101599999999999 2024-09-13T10:10:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076867+0000", 2024-09-13T10:10:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5262999999999999e-05 2024-09-13T10:10:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084106+0000", 2024-09-13T10:10:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0072390859999999996 2024-09-13T10:10:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084140+0000", 2024-09-13T10:10:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4072000000000002e-05 2024-09-13T10:10:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.712 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:252 4.6 4:6ddb2b23:::smithi19530714-3:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:10:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.51122741, 2024-09-13T10:10:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.03110711, 2024-09-13T10:10:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.713 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.714 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 252 2024-09-13T10:10:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:10:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:10:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122066+0000", 2024-09-13T10:10:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4619999999999999e-06 2024-09-13T10:10:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122097+0000", 2024-09-13T10:10:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0488999999999999e-05 2024-09-13T10:10:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122120+0000", 2024-09-13T10:10:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3649999999999999e-05 2024-09-13T10:10:55.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122130+0000", 2024-09-13T10:10:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7950000000000002e-06 2024-09-13T10:10:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122212+0000", 2024-09-13T10:10:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2090999999999997e-05 2024-09-13T10:10:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122317+0000", 2024-09-13T10:10:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104954 2024-09-13T10:10:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:10:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.151952+0000", 2024-09-13T10:10:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.029634908000000001 2024-09-13T10:10:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152580+0000", 2024-09-13T10:10:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062796499999999997 2024-09-13T10:10:55.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152891+0000", 2024-09-13T10:10:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000310747 2024-09-13T10:10:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153077+0000", 2024-09-13T10:10:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018588200000000001 2024-09-13T10:10:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153130+0000", 2024-09-13T10:10:55.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3016000000000002e-05 2024-09-13T10:10:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153171+0000", 2024-09-13T10:10:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1151e-05 2024-09-13T10:10:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:255 4.6 4:6150ba38:::smithi19530714-6:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.720 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:10:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.4188132060000003, 2024-09-13T10:10:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011708387000000001, 2024-09-13T10:10:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 255 2024-09-13T10:10:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:10:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:10:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214482+0000", 2024-09-13T10:10:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8139999999999998e-06 2024-09-13T10:10:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214498+0000", 2024-09-13T10:10:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6636000000000001e-05 2024-09-13T10:10:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214537+0000", 2024-09-13T10:10:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8668999999999999e-05 2024-09-13T10:10:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214562+0000", 2024-09-13T10:10:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5242999999999999e-05 2024-09-13T10:10:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214770+0000", 2024-09-13T10:10:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020771299999999999 2024-09-13T10:10:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214996+0000", 2024-09-13T10:10:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022615999999999999 2024-09-13T10:10:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:10:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.223455+0000", 2024-09-13T10:10:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0084591960000000004 2024-09-13T10:10:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225042+0000", 2024-09-13T10:10:55.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001586896 2024-09-13T10:10:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225719+0000", 2024-09-13T10:10:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00067701700000000003 2024-09-13T10:10:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225845+0000", 2024-09-13T10:10:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012626299999999999 2024-09-13T10:10:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226070+0000", 2024-09-13T10:10:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022411600000000001 2024-09-13T10:10:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226186+0000", 2024-09-13T10:10:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011666400000000001 2024-09-13T10:10:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.727 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:268 4.6 4:6fb28ed4:::smithi19530714-19:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:10:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.4178946459999997, 2024-09-13T10:10:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041608707000000002, 2024-09-13T10:10:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 268 2024-09-13T10:10:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:10:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:10:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215399+0000", 2024-09-13T10:10:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8669999999999998e-06 2024-09-13T10:10:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215416+0000", 2024-09-13T10:10:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6792000000000001e-05 2024-09-13T10:10:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215480+0000", 2024-09-13T10:10:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3805000000000003e-05 2024-09-13T10:10:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215499+0000", 2024-09-13T10:10:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9349999999999999e-05 2024-09-13T10:10:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215600+0000", 2024-09-13T10:10:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010119 2024-09-13T10:10:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215715+0000", 2024-09-13T10:10:55.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011491 2024-09-13T10:10:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:10:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.253967+0000", 2024-09-13T10:10:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038251470000000003 2024-09-13T10:10:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.255748+0000", 2024-09-13T10:10:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017815089999999999 2024-09-13T10:10:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256188+0000", 2024-09-13T10:10:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00043959799999999997 2024-09-13T10:10:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256840+0000", 2024-09-13T10:10:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000652591 2024-09-13T10:10:55.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256932+0000", 2024-09-13T10:10:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1691e-05 2024-09-13T10:10:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.257005+0000", 2024-09-13T10:10:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2934000000000003e-05 2024-09-13T10:10:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:305 4.6 4:6150ba38:::smithi19530714-6:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:10:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.364663953, 2024-09-13T10:10:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010323297, 2024-09-13T10:10:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.735 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 305 2024-09-13T10:10:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:10:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:10:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268630+0000", 2024-09-13T10:10:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.548e-06 2024-09-13T10:10:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268644+0000", 2024-09-13T10:10:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.394e-05 2024-09-13T10:10:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268672+0000", 2024-09-13T10:10:55.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8242000000000002e-05 2024-09-13T10:10:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268689+0000", 2024-09-13T10:10:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7062e-05 2024-09-13T10:10:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268789+0000", 2024-09-13T10:10:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010026600000000001 2024-09-13T10:10:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268905+0000", 2024-09-13T10:10:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115908 2024-09-13T10:10:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:10:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.276776+0000", 2024-09-13T10:10:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078712000000000001 2024-09-13T10:10:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278412+0000", 2024-09-13T10:10:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016358670000000001 2024-09-13T10:10:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278664+0000", 2024-09-13T10:10:55.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025162600000000002 2024-09-13T10:10:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278785+0000", 2024-09-13T10:10:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012162100000000001 2024-09-13T10:10:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278895+0000", 2024-09-13T10:10:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109774 2024-09-13T10:10:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278950+0000", 2024-09-13T10:10:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5242999999999999e-05 2024-09-13T10:10:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:352 4.6 4:6ddb2b23:::smithi19530714-3:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.287738+0000", 2024-09-13T10:10:55.742 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.34555326, 2024-09-13T10:10:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0085772350000000008, 2024-09-13T10:10:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.743 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 352 2024-09-13T10:10:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.287738+0000", 2024-09-13T10:10:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.287738+0000", 2024-09-13T10:10:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.287740+0000", 2024-09-13T10:10:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.885e-06 2024-09-13T10:10:55.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.287750+0000", 2024-09-13T10:10:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0155e-05 2024-09-13T10:10:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.287775+0000", 2024-09-13T10:10:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5006000000000001e-05 2024-09-13T10:10:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.287790+0000", 2024-09-13T10:10:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4844999999999999e-05 2024-09-13T10:10:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.287946+0000", 2024-09-13T10:10:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000156601 2024-09-13T10:10:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.288081+0000", 2024-09-13T10:10:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000134719 2024-09-13T10:10:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:10:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294695+0000", 2024-09-13T10:10:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066140729999999998 2024-09-13T10:10:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.295777+0000", 2024-09-13T10:10:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001082153 2024-09-13T10:10:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.296096+0000", 2024-09-13T10:10:55.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031912200000000003 2024-09-13T10:10:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.296168+0000", 2024-09-13T10:10:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1231999999999998e-05 2024-09-13T10:10:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.296240+0000", 2024-09-13T10:10:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2112000000000003e-05 2024-09-13T10:10:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.296315+0000", 2024-09-13T10:10:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5332000000000003e-05 2024-09-13T10:10:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:634 4.6 4:61ec6a5a:::smithi19530714-35:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:10:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.381970+0000", 2024-09-13T10:10:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.2513210030000002, 2024-09-13T10:10:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0085002270000000008, 2024-09-13T10:10:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:55.751 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 634 2024-09-13T10:10:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.381970+0000", 2024-09-13T10:10:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.381970+0000", 2024-09-13T10:10:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.381972+0000", 2024-09-13T10:10:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5549999999999999e-06 2024-09-13T10:10:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.381980+0000", 2024-09-13T10:10:55.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8200000000000003e-06 2024-09-13T10:10:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.382007+0000", 2024-09-13T10:10:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6211999999999999e-05 2024-09-13T10:10:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.382021+0000", 2024-09-13T10:10:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4516e-05 2024-09-13T10:10:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:55.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.382111+0000", 2024-09-13T10:10:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9748999999999997e-05 2024-09-13T10:10:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.382238+0000", 2024-09-13T10:10:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000127234 2024-09-13T10:10:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:10:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.386366+0000", 2024-09-13T10:10:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004127487 2024-09-13T10:10:55.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:55.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.387170+0000", 2024-09-13T10:10:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000804508 2024-09-13T10:10:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.387569+0000", 2024-09-13T10:10:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00039894300000000002 2024-09-13T10:10:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.390352+0000", 2024-09-13T10:10:55.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027828420000000002 2024-09-13T10:10:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.390403+0000", 2024-09-13T10:10:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0877000000000001e-05 2024-09-13T10:10:55.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.390470+0000", 2024-09-13T10:10:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7484000000000006e-05 2024-09-13T10:10:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:55.759 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:55.759 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:10:55.761 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-13T10:10:55.763 INFO:tasks.rados.rados.0.smithi195.stdout:675: done (4 left) 2024-09-13T10:10:55.763 INFO:tasks.rados.rados.0.smithi195.stdout:676: done (3 left) 2024-09-13T10:10:55.763 INFO:tasks.rados.rados.0.smithi195.stdout:677: done (2 left) 2024-09-13T10:10:55.763 INFO:tasks.rados.rados.0.smithi195.stdout:678: done (1 left) 2024-09-13T10:10:55.764 INFO:tasks.rados.rados.0.smithi195.stdout:680: done (0 left) 2024-09-13T10:10:55.764 INFO:tasks.rados.rados.0.smithi195.stdout:681: read oid 50 snap -1 2024-09-13T10:10:55.764 INFO:tasks.rados.rados.0.smithi195.stdout:681: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-13T10:10:55.764 INFO:tasks.rados.rados.0.smithi195.stdout:682: read oid 10 snap -1 2024-09-13T10:10:55.764 INFO:tasks.rados.rados.0.smithi195.stdout:682: expect (ObjNum 62 snap 0 seq_num 62) 2024-09-13T10:10:55.764 INFO:tasks.rados.rados.0.smithi195.stdout:683: tier_evict oid 19 2024-09-13T10:10:55.764 INFO:tasks.rados.rados.0.smithi195.stdout:683: tier_evict oid 19 snap -1 2024-09-13T10:10:55.764 INFO:tasks.rados.rados.0.smithi195.stdout:684: tier_flush oid 20 2024-09-13T10:10:55.764 INFO:tasks.rados.rados.0.smithi195.stdout:684: tier_flush oid 20 snap -1 2024-09-13T10:10:55.764 INFO:tasks.rados.rados.0.smithi195.stdout:685: read oid 22 snap -1 2024-09-13T10:10:55.764 INFO:tasks.rados.rados.0.smithi195.stdout:685: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-13T10:10:55.764 INFO:tasks.rados.rados.0.smithi195.stdout:686: read oid 17 snap -1 2024-09-13T10:10:55.765 INFO:tasks.rados.rados.0.smithi195.stdout:686: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-13T10:10:55.765 INFO:tasks.rados.rados.0.smithi195.stdout:687: write oid 11 current snap is 0 2024-09-13T10:10:55.765 INFO:tasks.rados.rados.0.smithi195.stdout:687: seq_num 76 ranges {477332=766101,1681102=474010,2687912=589420} 2024-09-13T10:10:55.765 INFO:tasks.rados.rados.0.smithi195.stdout:687: writing smithi19530714-11 from 477332 to 1243433 tid 1 2024-09-13T10:10:55.765 INFO:tasks.rados.rados.0.smithi195.stdout:687: writing smithi19530714-11 from 1681102 to 2155112 tid 2 2024-09-13T10:10:55.765 INFO:tasks.rados.rados.0.smithi195.stdout:687: writing smithi19530714-11 from 2687912 to 3277332 tid 3 2024-09-13T10:10:55.765 INFO:tasks.rados.rados.0.smithi195.stdout:688: delete oid 46 current snap is 0 2024-09-13T10:10:55.766 INFO:tasks.rados.rados.0.smithi195.stdout:683: got (22) Invalid argument 2024-09-13T10:10:55.766 INFO:tasks.rados.rados.0.smithi195.stdout:686: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-13T10:10:55.766 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2791139 2024-09-13T10:10:55.766 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:55.760+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:10:55.790 INFO:tasks.rados.rados.0.smithi195.stdout:681: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-13T10:10:55.854 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2021403 2024-09-13T10:10:55.861 INFO:tasks.rados.rados.0.smithi195.stdout:685: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-13T10:10:55.862 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:10:55.861+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:10:55.880 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:55.912 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:10:55.912 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:10:55.912 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:10:55.912 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:10:55.926 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-13T10:10:55.931 INFO:tasks.rados.rados.0.smithi195.stdout:682: expect (ObjNum 62 snap 0 seq_num 62) 2024-09-13T10:10:55.963 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:55.960+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:10:56.043 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:10:56.043 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:10:56.043 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:10:56.043 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:10:56.043 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:10:56.058 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-13T10:10:56.064 INFO:tasks.rados.rados.0.smithi195.stdout:687: finishing write tid 1 to smithi19530714-11 2024-09-13T10:10:56.064 INFO:tasks.rados.rados.0.smithi195.stdout:687: finishing write tid 2 to smithi19530714-11 2024-09-13T10:10:56.065 INFO:tasks.rados.rados.0.smithi195.stdout:687: finishing write tid 3 to smithi19530714-11 2024-09-13T10:10:56.065 INFO:tasks.rados.rados.0.smithi195.stdout:687: finishing write tid 4 to smithi19530714-11 2024-09-13T10:10:56.065 INFO:tasks.rados.rados.0.smithi195.stdout:687: finishing write tid 5 to smithi19530714-11 2024-09-13T10:10:56.065 INFO:tasks.rados.rados.0.smithi195.stdout:687: oid 11 updating version 0 to 21 2024-09-13T10:10:56.065 INFO:tasks.rados.rados.0.smithi195.stdout:687: oid 11 version 21 is already newer than 18 2024-09-13T10:10:56.065 INFO:tasks.rados.rados.0.smithi195.stdout:687: oid 11 version 21 is already newer than 20 2024-09-13T10:10:56.065 INFO:tasks.rados.rados.0.smithi195.stdout:687: oid 11 version 21 is already newer than 19 2024-09-13T10:10:56.065 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 21 (ObjNum 76 snap 0 seq_num 76) dirty exists 2024-09-13T10:10:56.065 INFO:tasks.rados.rados.0.smithi195.stdout:687: left oid 11 (ObjNum 76 snap 0 seq_num 76) 2024-09-13T10:10:56.065 INFO:tasks.rados.rados.0.smithi195.stdout:684: got (0) Success 2024-09-13T10:10:56.065 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-13T10:10:56.065 INFO:tasks.rados.rados.0.smithi195.stdout:681: done (7 left) 2024-09-13T10:10:56.066 INFO:tasks.rados.rados.0.smithi195.stdout:682: done (6 left) 2024-09-13T10:10:56.066 INFO:tasks.rados.rados.0.smithi195.stdout:683: done (5 left) 2024-09-13T10:10:56.066 INFO:tasks.rados.rados.0.smithi195.stdout:684: done (4 left) 2024-09-13T10:10:56.066 INFO:tasks.rados.rados.0.smithi195.stdout:685: done (3 left) 2024-09-13T10:10:56.066 INFO:tasks.rados.rados.0.smithi195.stdout:686: done (2 left) 2024-09-13T10:10:56.066 INFO:tasks.rados.rados.0.smithi195.stdout:687: done (1 left) 2024-09-13T10:10:56.066 INFO:tasks.rados.rados.0.smithi195.stdout:688: done (0 left) 2024-09-13T10:10:56.066 INFO:tasks.rados.rados.0.smithi195.stdout:689: read oid 32 snap -1 2024-09-13T10:10:56.066 INFO:tasks.rados.rados.0.smithi195.stdout:689: expect (ObjNum 31 snap 0 seq_num 31) 2024-09-13T10:10:56.066 INFO:tasks.rados.rados.0.smithi195.stdout:690: write oid 19 current snap is 0 2024-09-13T10:10:56.066 INFO:tasks.rados.rados.0.smithi195.stdout:690: seq_num 78 ranges {510946=632952,1798066=477679,2742758=457217} 2024-09-13T10:10:56.067 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:10:56.061+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:10:56.071 INFO:tasks.rados.rados.0.smithi195.stdout:690: writing smithi19530714-19 from 510946 to 1143898 tid 1 2024-09-13T10:10:56.077 INFO:tasks.rados.rados.0.smithi195.stdout:690: writing smithi19530714-19 from 1798066 to 2275745 tid 2 2024-09-13T10:10:56.083 INFO:tasks.rados.rados.0.smithi195.stdout:690: writing smithi19530714-19 from 2742758 to 3199975 tid 3 2024-09-13T10:10:56.083 INFO:tasks.rados.rados.0.smithi195.stdout:691: read oid 46 snap -1 2024-09-13T10:10:56.083 INFO:tasks.rados.rados.0.smithi195.stdout:691: expect deleted 2024-09-13T10:10:56.083 INFO:tasks.rados.rados.0.smithi195.stdout:692: read oid 22 snap -1 2024-09-13T10:10:56.083 INFO:tasks.rados.rados.0.smithi195.stdout:692: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-13T10:10:56.083 INFO:tasks.rados.rados.0.smithi195.stdout:693: set_chunk oid 27 target oid 2024-09-13T10:10:56.095 INFO:tasks.rados.rados.0.smithi195.stdout:693: set_chunk oid 27 offset: 483328 length: 295424 target oid d1b0a2644aff45db605915195d297644a15bbe971360b5b7bb999701f6d9b0f4 offset: 0 snap -1 2024-09-13T10:10:56.095 INFO:tasks.rados.rados.0.smithi195.stdout:694: read oid 20 snap -1 2024-09-13T10:10:56.095 INFO:tasks.rados.rados.0.smithi195.stdout:694: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-13T10:10:56.095 INFO:tasks.rados.rados.0.smithi195.stdout:695: tier_flush oid 18 2024-09-13T10:10:56.095 INFO:tasks.rados.rados.0.smithi195.stdout:695: tier_flush oid 18 snap -1 2024-09-13T10:10:56.096 INFO:tasks.rados.rados.0.smithi195.stdout:696: tier_flush oid 35 2024-09-13T10:10:56.096 INFO:tasks.rados.rados.0.smithi195.stdout:696: tier_flush oid 35 snap -1 2024-09-13T10:10:56.096 INFO:tasks.rados.rados.0.smithi195.stdout:689: expect (ObjNum 31 snap 0 seq_num 31) 2024-09-13T10:10:56.134 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2495915 2024-09-13T10:10:56.138 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 439869 2024-09-13T10:10:56.162 INFO:tasks.rados.rados.0.smithi195.stdout:690: finishing write tid 1 to smithi19530714-19 2024-09-13T10:10:56.162 INFO:tasks.rados.rados.0.smithi195.stdout:689: done (7 left) 2024-09-13T10:10:56.162 INFO:tasks.rados.rados.0.smithi195.stdout:697: tier_promote oid 41 2024-09-13T10:10:56.162 INFO:tasks.rados.rados.0.smithi195.stdout:690: finishing write tid 2 to smithi19530714-19 2024-09-13T10:10:56.162 INFO:tasks.rados.rados.0.smithi195.stdout:690: finishing write tid 3 to smithi19530714-19 2024-09-13T10:10:56.162 INFO:tasks.rados.rados.0.smithi195.stdout:690: finishing write tid 4 to smithi19530714-19 2024-09-13T10:10:56.162 INFO:tasks.rados.rados.0.smithi195.stdout:692: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-13T10:10:56.165 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:10:56.161+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:10:56.192 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 916421 2024-09-13T10:10:56.203 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:10:56.203 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:10:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:10:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:10:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:89 4.9 4:96fcf081:::smithi19530714-23:head [write 769754~688184,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.809155+0000", 2024-09-13T10:10:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.390322276, 2024-09-13T10:10:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011562566, 2024-09-13T10:10:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.204 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.205 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.205 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 89 2024-09-13T10:10:56.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.205 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.809155+0000", 2024-09-13T10:10:56.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.809155+0000", 2024-09-13T10:10:56.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.809158+0000", 2024-09-13T10:10:56.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5770000000000001e-06 2024-09-13T10:10:56.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810210+0000", 2024-09-13T10:10:56.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010519889999999999 2024-09-13T10:10:56.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810238+0000", 2024-09-13T10:10:56.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8449000000000002e-05 2024-09-13T10:10:56.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810265+0000", 2024-09-13T10:10:56.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6832e-05 2024-09-13T10:10:56.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810374+0000", 2024-09-13T10:10:56.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109274 2024-09-13T10:10:56.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810519+0000", 2024-09-13T10:10:56.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014449299999999999 2024-09-13T10:10:56.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:10:56.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810684+0000", 2024-09-13T10:10:56.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000165744 2024-09-13T10:10:56.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816442+0000", 2024-09-13T10:10:56.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057578990000000004 2024-09-13T10:10:56.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:56.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816685+0000", 2024-09-13T10:10:56.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024254999999999999 2024-09-13T10:10:56.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820614+0000", 2024-09-13T10:10:56.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0039290970000000003 2024-09-13T10:10:56.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820668+0000", 2024-09-13T10:10:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3567999999999999e-05 2024-09-13T10:10:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820718+0000", 2024-09-13T10:10:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0093999999999998e-05 2024-09-13T10:10:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.210 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:90 4.9 4:96fcf081:::smithi19530714-23:head [write 2021192~348562] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:56.211 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810322+0000", 2024-09-13T10:10:56.211 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.3891550410000004, 2024-09-13T10:10:56.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01166355, 2024-09-13T10:10:56.211 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.211 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.211 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.211 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.211 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.211 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.211 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 90 2024-09-13T10:10:56.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.211 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810322+0000", 2024-09-13T10:10:56.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810322+0000", 2024-09-13T10:10:56.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810324+0000", 2024-09-13T10:10:56.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.097e-06 2024-09-13T10:10:56.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810494+0000", 2024-09-13T10:10:56.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016961499999999999 2024-09-13T10:10:56.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810515+0000", 2024-09-13T10:10:56.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0939999999999999e-05 2024-09-13T10:10:56.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810533+0000", 2024-09-13T10:10:56.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7909000000000001e-05 2024-09-13T10:10:56.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.814429+0000", 2024-09-13T10:10:56.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0038957409999999999 2024-09-13T10:10:56.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.814603+0000", 2024-09-13T10:10:56.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174294 2024-09-13T10:10:56.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:10:56.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.814804+0000", 2024-09-13T10:10:56.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020149799999999999 2024-09-13T10:10:56.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:56.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820569+0000", 2024-09-13T10:10:56.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057648389999999999 2024-09-13T10:10:56.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820761+0000", 2024-09-13T10:10:56.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019123 2024-09-13T10:10:56.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821893+0000", 2024-09-13T10:10:56.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001132051 2024-09-13T10:10:56.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821949+0000", 2024-09-13T10:10:56.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6832000000000001e-05 2024-09-13T10:10:56.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821986+0000", 2024-09-13T10:10:56.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6504000000000002e-05 2024-09-13T10:10:56.217 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.217 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.217 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.217 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:91 4.9 4:96fcf081:::smithi19530714-23:head [setxattr _header (56) in=7b,truncate 2369754] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:56.217 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810564+0000", 2024-09-13T10:10:56.217 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.388913831, 2024-09-13T10:10:56.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01178625, 2024-09-13T10:10:56.217 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.217 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.217 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.218 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.218 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.218 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.218 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 91 2024-09-13T10:10:56.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.218 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810564+0000", 2024-09-13T10:10:56.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810564+0000", 2024-09-13T10:10:56.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810565+0000", 2024-09-13T10:10:56.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3960000000000001e-06 2024-09-13T10:10:56.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810574+0000", 2024-09-13T10:10:56.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9339999999999995e-06 2024-09-13T10:10:56.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810590+0000", 2024-09-13T10:10:56.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5858999999999998e-05 2024-09-13T10:10:56.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810604+0000", 2024-09-13T10:10:56.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4533e-05 2024-09-13T10:10:56.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816017+0000", 2024-09-13T10:10:56.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0054122390000000001 2024-09-13T10:10:56.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816092+0000", 2024-09-13T10:10:56.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5537999999999994e-05 2024-09-13T10:10:56.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:10:56.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816206+0000", 2024-09-13T10:10:56.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114298 2024-09-13T10:10:56.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820805+0000", 2024-09-13T10:10:56.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004598347 2024-09-13T10:10:56.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:56.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821735+0000", 2024-09-13T10:10:56.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000930348 2024-09-13T10:10:56.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822118+0000", 2024-09-13T10:10:56.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038271099999999999 2024-09-13T10:10:56.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.223 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822208+0000", 2024-09-13T10:10:56.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0271000000000004e-05 2024-09-13T10:10:56.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.223 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822350+0000", 2024-09-13T10:10:56.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014177600000000001 2024-09-13T10:10:56.223 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.223 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.223 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.223 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:92 4.9 4:96fcf081:::smithi19530714-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:56.224 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810624+0000", 2024-09-13T10:10:56.224 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.388853406, 2024-09-13T10:10:56.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012449517, 2024-09-13T10:10:56.224 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.224 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.224 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:10:56.224 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.224 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.224 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.224 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 92 2024-09-13T10:10:56.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.224 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810624+0000", 2024-09-13T10:10:56.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810624+0000", 2024-09-13T10:10:56.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810625+0000", 2024-09-13T10:10:56.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3459999999999999e-06 2024-09-13T10:10:56.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810638+0000", 2024-09-13T10:10:56.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2853e-05 2024-09-13T10:10:56.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810655+0000", 2024-09-13T10:10:56.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6370000000000001e-05 2024-09-13T10:10:56.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810668+0000", 2024-09-13T10:10:56.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3906e-05 2024-09-13T10:10:56.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816513+0000", 2024-09-13T10:10:56.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058443369999999998 2024-09-13T10:10:56.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:10:56.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816657+0000", 2024-09-13T10:10:56.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014419199999999999 2024-09-13T10:10:56.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.228 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.228 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822492+0000", 2024-09-13T10:10:56.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.005835107 2024-09-13T10:10:56.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.228 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.228 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822740+0000", 2024-09-13T10:10:56.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024784600000000003 2024-09-13T10:10:56.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.228 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.228 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823074+0000", 2024-09-13T10:10:56.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033356 2024-09-13T10:10:56.229 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.229 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.229 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.229 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:129 4.9 4:998d9bf1:::smithi19530714-33:head [write 537824~473380,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:56.229 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810689+0000", 2024-09-13T10:10:56.229 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.3887886309999997, 2024-09-13T10:10:56.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01301894, 2024-09-13T10:10:56.229 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.229 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.229 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.230 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.230 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.230 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.230 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 129 2024-09-13T10:10:56.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.230 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810689+0000", 2024-09-13T10:10:56.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810689+0000", 2024-09-13T10:10:56.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810690+0000", 2024-09-13T10:10:56.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2330000000000001e-06 2024-09-13T10:10:56.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811247+0000", 2024-09-13T10:10:56.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055654999999999995 2024-09-13T10:10:56.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.232 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.232 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811264+0000", 2024-09-13T10:10:56.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7813e-05 2024-09-13T10:10:56.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.232 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.232 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811280+0000", 2024-09-13T10:10:56.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5676e-05 2024-09-13T10:10:56.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.232 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.232 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816755+0000", 2024-09-13T10:10:56.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0054754139999999996 2024-09-13T10:10:56.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816940+0000", 2024-09-13T10:10:56.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184091 2024-09-13T10:10:56.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:10:56.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.817141+0000", 2024-09-13T10:10:56.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020179099999999999 2024-09-13T10:10:56.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:56.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821746+0000", 2024-09-13T10:10:56.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004604543 2024-09-13T10:10:56.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823211+0000", 2024-09-13T10:10:56.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001464824 2024-09-13T10:10:56.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823536+0000", 2024-09-13T10:10:56.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032508599999999998 2024-09-13T10:10:56.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823620+0000", 2024-09-13T10:10:56.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4524999999999995e-05 2024-09-13T10:10:56.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823708+0000", 2024-09-13T10:10:56.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7393999999999997e-05 2024-09-13T10:10:56.235 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.235 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.235 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.236 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:130 4.9 4:998d9bf1:::smithi19530714-33:head [write 1806012~625998] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:56.236 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.811308+0000", 2024-09-13T10:10:56.236 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.3881696669999997, 2024-09-13T10:10:56.236 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012775038000000001, 2024-09-13T10:10:56.236 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.236 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.236 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.236 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.236 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.236 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.236 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 130 2024-09-13T10:10:56.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.236 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811308+0000", 2024-09-13T10:10:56.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811308+0000", 2024-09-13T10:10:56.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811310+0000", 2024-09-13T10:10:56.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9400000000000001e-06 2024-09-13T10:10:56.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811989+0000", 2024-09-13T10:10:56.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00067975999999999996 2024-09-13T10:10:56.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812006+0000", 2024-09-13T10:10:56.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6175999999999999e-05 2024-09-13T10:10:56.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.239 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.239 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.239 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812024+0000", 2024-09-13T10:10:56.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8813000000000001e-05 2024-09-13T10:10:56.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.239 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.239 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.239 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.818639+0000", 2024-09-13T10:10:56.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066148639999999998 2024-09-13T10:10:56.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.239 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.239 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.239 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.818763+0000", 2024-09-13T10:10:56.240 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012372300000000001 2024-09-13T10:10:56.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.240 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.240 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:10:56.240 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.818947+0000", 2024-09-13T10:10:56.240 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184394 2024-09-13T10:10:56.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.240 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.240 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:56.240 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822370+0000", 2024-09-13T10:10:56.240 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034224989999999999 2024-09-13T10:10:56.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.240 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823413+0000", 2024-09-13T10:10:56.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001042739 2024-09-13T10:10:56.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823829+0000", 2024-09-13T10:10:56.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000416477 2024-09-13T10:10:56.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823982+0000", 2024-09-13T10:10:56.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015238599999999999 2024-09-13T10:10:56.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824083+0000", 2024-09-13T10:10:56.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000101267 2024-09-13T10:10:56.242 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.242 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.242 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.242 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:131 4.9 4:998d9bf1:::smithi19530714-33:head [setxattr _header (56) in=7b,truncate 2537824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:56.242 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.812049+0000", 2024-09-13T10:10:56.243 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.3874279090000003, 2024-09-13T10:10:56.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012575461, 2024-09-13T10:10:56.243 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.243 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.243 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.243 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.243 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.243 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.243 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 131 2024-09-13T10:10:56.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.243 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.243 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812049+0000", 2024-09-13T10:10:56.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.244 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812049+0000", 2024-09-13T10:10:56.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.244 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812051+0000", 2024-09-13T10:10:56.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-13T10:10:56.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.245 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.245 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.245 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812060+0000", 2024-09-13T10:10:56.245 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9930000000000001e-06 2024-09-13T10:10:56.245 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.245 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.245 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.245 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812078+0000", 2024-09-13T10:10:56.245 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8266000000000001e-05 2024-09-13T10:10:56.245 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.245 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.245 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.245 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812093+0000", 2024-09-13T10:10:56.246 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5153e-05 2024-09-13T10:10:56.246 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.246 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.246 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.246 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820907+0000", 2024-09-13T10:10:56.246 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088138650000000006 2024-09-13T10:10:56.246 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.246 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.246 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.246 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821046+0000", 2024-09-13T10:10:56.246 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013850900000000001 2024-09-13T10:10:56.246 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.246 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.247 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:10:56.247 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821338+0000", 2024-09-13T10:10:56.247 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029241800000000002 2024-09-13T10:10:56.247 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.247 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.247 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:56.247 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823249+0000", 2024-09-13T10:10:56.247 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019105610000000001 2024-09-13T10:10:56.247 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.247 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.247 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.247 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824157+0000", 2024-09-13T10:10:56.247 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090782899999999997 2024-09-13T10:10:56.248 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.248 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.248 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.248 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824422+0000", 2024-09-13T10:10:56.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026587900000000001 2024-09-13T10:10:56.248 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.248 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.248 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.248 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824497+0000", 2024-09-13T10:10:56.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4443999999999994e-05 2024-09-13T10:10:56.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.249 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824625+0000", 2024-09-13T10:10:56.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012811699999999999 2024-09-13T10:10:56.249 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.249 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.249 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.249 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:132 4.9 4:998d9bf1:::smithi19530714-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:56.250 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.812113+0000", 2024-09-13T10:10:56.250 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.3873638970000002, 2024-09-13T10:10:56.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01281326, 2024-09-13T10:10:56.250 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.250 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.250 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:10:56.250 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.250 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.250 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.250 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 132 2024-09-13T10:10:56.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.250 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.250 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.251 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.251 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812113+0000", 2024-09-13T10:10:56.251 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.251 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.251 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.251 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812113+0000", 2024-09-13T10:10:56.251 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.251 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.251 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.252 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812115+0000", 2024-09-13T10:10:56.252 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.826e-06 2024-09-13T10:10:56.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.252 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812121+0000", 2024-09-13T10:10:56.252 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9540000000000001e-06 2024-09-13T10:10:56.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.252 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812158+0000", 2024-09-13T10:10:56.252 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6619000000000003e-05 2024-09-13T10:10:56.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812175+0000", 2024-09-13T10:10:56.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6662000000000001e-05 2024-09-13T10:10:56.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821791+0000", 2024-09-13T10:10:56.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.009616187 2024-09-13T10:10:56.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:10:56.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821855+0000", 2024-09-13T10:10:56.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4183999999999994e-05 2024-09-13T10:10:56.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.254 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824680+0000", 2024-09-13T10:10:56.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0028252749999999999 2024-09-13T10:10:56.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.254 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824784+0000", 2024-09-13T10:10:56.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010365 2024-09-13T10:10:56.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.255 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824927+0000", 2024-09-13T10:10:56.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000142903 2024-09-13T10:10:56.255 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.255 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.255 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.255 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.255 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:185 4.9 4:9bec18ef:::smithi19530714-47:head [write 773122~559572,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:56.255 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.939228+0000", 2024-09-13T10:10:56.255 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.2602493580000003, 2024-09-13T10:10:56.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0053919969999999999, 2024-09-13T10:10:56.255 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.256 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.256 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.256 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.256 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.256 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.256 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 185 2024-09-13T10:10:56.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.256 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.939228+0000", 2024-09-13T10:10:56.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.939228+0000", 2024-09-13T10:10:56.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.939231+0000", 2024-09-13T10:10:56.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4570000000000001e-06 2024-09-13T10:10:56.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.258 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.939488+0000", 2024-09-13T10:10:56.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025665200000000001 2024-09-13T10:10:56.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.258 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.939522+0000", 2024-09-13T10:10:56.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4119000000000003e-05 2024-09-13T10:10:56.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.258 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.258 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.939538+0000", 2024-09-13T10:10:56.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6220000000000001e-05 2024-09-13T10:10:56.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.939697+0000", 2024-09-13T10:10:56.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015816199999999999 2024-09-13T10:10:56.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.940097+0000", 2024-09-13T10:10:56.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000400601 2024-09-13T10:10:56.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:10:56.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.940560+0000", 2024-09-13T10:10:56.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00046251899999999999 2024-09-13T10:10:56.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:56.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.943691+0000", 2024-09-13T10:10:56.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0031310069999999999 2024-09-13T10:10:56.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.944040+0000", 2024-09-13T10:10:56.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034907999999999998 2024-09-13T10:10:56.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.944334+0000", 2024-09-13T10:10:56.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029425099999999999 2024-09-13T10:10:56.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.944492+0000", 2024-09-13T10:10:56.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000157732 2024-09-13T10:10:56.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.944620+0000", 2024-09-13T10:10:56.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012819700000000001 2024-09-13T10:10:56.262 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.262 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.262 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.262 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:188 4.9 4:9bec18ef:::smithi19530714-47:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:56.262 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.947080+0000", 2024-09-13T10:10:56.262 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.2523969279999996, 2024-09-13T10:10:56.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0052317509999999998, 2024-09-13T10:10:56.262 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.262 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.262 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:10:56.262 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.263 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.263 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.263 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 188 2024-09-13T10:10:56.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.263 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.263 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.263 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.947080+0000", 2024-09-13T10:10:56.263 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.263 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.263 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.947080+0000", 2024-09-13T10:10:56.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.264 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.947081+0000", 2024-09-13T10:10:56.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0599999999999999e-07 2024-09-13T10:10:56.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.264 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.947086+0000", 2024-09-13T10:10:56.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3069999999999996e-06 2024-09-13T10:10:56.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.947098+0000", 2024-09-13T10:10:56.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2466000000000001e-05 2024-09-13T10:10:56.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.947107+0000", 2024-09-13T10:10:56.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7260000000000008e-06 2024-09-13T10:10:56.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.950196+0000", 2024-09-13T10:10:56.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030889989999999998 2024-09-13T10:10:56.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:10:56.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.950262+0000", 2024-09-13T10:10:56.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6460999999999999e-05 2024-09-13T10:10:56.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.952056+0000", 2024-09-13T10:10:56.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00179397 2024-09-13T10:10:56.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.952164+0000", 2024-09-13T10:10:56.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010758 2024-09-13T10:10:56.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.952312+0000", 2024-09-13T10:10:56.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000148336 2024-09-13T10:10:56.267 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.267 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.267 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.267 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:272 4.9 4:96fcf081:::smithi19530714-23:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:56.268 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.226803+0000", 2024-09-13T10:10:56.268 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.9726741839999997, 2024-09-13T10:10:56.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0087344910000000005, 2024-09-13T10:10:56.268 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.268 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.268 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.268 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.268 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.268 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.268 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 272 2024-09-13T10:10:56.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.268 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226803+0000", 2024-09-13T10:10:56.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226803+0000", 2024-09-13T10:10:56.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226807+0000", 2024-09-13T10:10:56.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8829999999999999e-06 2024-09-13T10:10:56.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226823+0000", 2024-09-13T10:10:56.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5979999999999999e-05 2024-09-13T10:10:56.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226859+0000", 2024-09-13T10:10:56.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6331999999999997e-05 2024-09-13T10:10:56.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226882+0000", 2024-09-13T10:10:56.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2452000000000001e-05 2024-09-13T10:10:56.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.227037+0000", 2024-09-13T10:10:56.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000155563 2024-09-13T10:10:56.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.227232+0000", 2024-09-13T10:10:56.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019476500000000001 2024-09-13T10:10:56.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:10:56.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.233676+0000", 2024-09-13T10:10:56.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0064435719999999998 2024-09-13T10:10:56.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235081+0000", 2024-09-13T10:10:56.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001404953 2024-09-13T10:10:56.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:56.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235114+0000", 2024-09-13T10:10:56.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2996000000000001e-05 2024-09-13T10:10:56.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235286+0000", 2024-09-13T10:10:56.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171965 2024-09-13T10:10:56.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235431+0000", 2024-09-13T10:10:56.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000145205 2024-09-13T10:10:56.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235538+0000", 2024-09-13T10:10:56.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010682499999999999 2024-09-13T10:10:56.274 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.274 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.274 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.274 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:282 4.9 4:998d9bf1:::smithi19530714-33:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:56.274 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.240459+0000", 2024-09-13T10:10:56.274 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.9590185289999997, 2024-09-13T10:10:56.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.022877255999999999, 2024-09-13T10:10:56.275 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.275 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.275 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.275 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.275 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.275 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.275 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 282 2024-09-13T10:10:56.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.275 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.275 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240459+0000", 2024-09-13T10:10:56.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240459+0000", 2024-09-13T10:10:56.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240462+0000", 2024-09-13T10:10:56.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.971e-06 2024-09-13T10:10:56.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240479+0000", 2024-09-13T10:10:56.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6937000000000001e-05 2024-09-13T10:10:56.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.277 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.277 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240512+0000", 2024-09-13T10:10:56.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3012999999999997e-05 2024-09-13T10:10:56.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.277 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.277 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240533+0000", 2024-09-13T10:10:56.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1138999999999999e-05 2024-09-13T10:10:56.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240687+0000", 2024-09-13T10:10:56.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000154275 2024-09-13T10:10:56.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240829+0000", 2024-09-13T10:10:56.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000141619 2024-09-13T10:10:56.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:10:56.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.260691+0000", 2024-09-13T10:10:56.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.019861706 2024-09-13T10:10:56.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:56.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.262189+0000", 2024-09-13T10:10:56.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0014983069999999999 2024-09-13T10:10:56.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.262975+0000", 2024-09-13T10:10:56.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00078655900000000004 2024-09-13T10:10:56.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.263100+0000", 2024-09-13T10:10:56.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012431899999999999 2024-09-13T10:10:56.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.263232+0000", 2024-09-13T10:10:56.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013227399999999999 2024-09-13T10:10:56.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.263336+0000", 2024-09-13T10:10:56.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104137 2024-09-13T10:10:56.281 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.281 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.284 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.284 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:296 4.9 4:9bec18ef:::smithi19530714-47:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:56.285 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.261575+0000", 2024-09-13T10:10:56.285 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.937902587, 2024-09-13T10:10:56.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0097761170000000008, 2024-09-13T10:10:56.285 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.285 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.285 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.285 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.285 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.285 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.285 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 296 2024-09-13T10:10:56.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.285 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.286 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.286 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261575+0000", 2024-09-13T10:10:56.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.286 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.286 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.286 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261575+0000", 2024-09-13T10:10:56.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.286 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.286 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.286 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261579+0000", 2024-09-13T10:10:56.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7740000000000002e-06 2024-09-13T10:10:56.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261594+0000", 2024-09-13T10:10:56.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4973999999999999e-05 2024-09-13T10:10:56.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261625+0000", 2024-09-13T10:10:56.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1519000000000001e-05 2024-09-13T10:10:56.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261645+0000", 2024-09-13T10:10:56.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9568e-05 2024-09-13T10:10:56.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261758+0000", 2024-09-13T10:10:56.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011301700000000001 2024-09-13T10:10:56.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261902+0000", 2024-09-13T10:10:56.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014435300000000001 2024-09-13T10:10:56.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:10:56.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.269548+0000", 2024-09-13T10:10:56.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0076456850000000002 2024-09-13T10:10:56.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.270867+0000", 2024-09-13T10:10:56.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013193930000000001 2024-09-13T10:10:56.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:56.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.270884+0000", 2024-09-13T10:10:56.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7116999999999999e-05 2024-09-13T10:10:56.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.271218+0000", 2024-09-13T10:10:56.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033386800000000002 2024-09-13T10:10:56.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.271305+0000", 2024-09-13T10:10:56.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6738999999999997e-05 2024-09-13T10:10:56.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.271351+0000", 2024-09-13T10:10:56.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6109999999999997e-05 2024-09-13T10:10:56.291 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.291 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.291 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.291 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:322 4.9 4:96fcf081:::smithi19530714-23:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:56.291 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.279243+0000", 2024-09-13T10:10:56.291 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.9202348550000004, 2024-09-13T10:10:56.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0076374249999999998, 2024-09-13T10:10:56.292 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.292 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.292 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.292 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.292 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.292 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.292 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 322 2024-09-13T10:10:56.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.292 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.279243+0000", 2024-09-13T10:10:56.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.293 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.279243+0000", 2024-09-13T10:10:56.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.293 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.279244+0000", 2024-09-13T10:10:56.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9539999999999998e-06 2024-09-13T10:10:56.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.279253+0000", 2024-09-13T10:10:56.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0040000000000005e-06 2024-09-13T10:10:56.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.279275+0000", 2024-09-13T10:10:56.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1637e-05 2024-09-13T10:10:56.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.279286+0000", 2024-09-13T10:10:56.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0794e-05 2024-09-13T10:10:56.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.279409+0000", 2024-09-13T10:10:56.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012336500000000001 2024-09-13T10:10:56.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.279591+0000", 2024-09-13T10:10:56.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018197299999999999 2024-09-13T10:10:56.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:10:56.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.284745+0000", 2024-09-13T10:10:56.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0051533200000000003 2024-09-13T10:10:56.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285975+0000", 2024-09-13T10:10:56.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001230128 2024-09-13T10:10:56.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:56.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286521+0000", 2024-09-13T10:10:56.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00054675999999999998 2024-09-13T10:10:56.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286643+0000", 2024-09-13T10:10:56.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000121088 2024-09-13T10:10:56.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286779+0000", 2024-09-13T10:10:56.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136933 2024-09-13T10:10:56.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286880+0000", 2024-09-13T10:10:56.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100469 2024-09-13T10:10:56.298 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.298 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.298 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.298 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:346 4.9 4:9bec18ef:::smithi19530714-47:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:10:56.298 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.285952+0000", 2024-09-13T10:10:56.298 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.9135257379999997, 2024-09-13T10:10:56.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0081934079999999992, 2024-09-13T10:10:56.299 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.299 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.299 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.299 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.299 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.299 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.299 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 346 2024-09-13T10:10:56.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.299 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285952+0000", 2024-09-13T10:10:56.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.300 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.300 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285952+0000", 2024-09-13T10:10:56.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.300 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.300 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285954+0000", 2024-09-13T10:10:56.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3530000000000002e-06 2024-09-13T10:10:56.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.300 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285965+0000", 2024-09-13T10:10:56.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0718e-05 2024-09-13T10:10:56.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285988+0000", 2024-09-13T10:10:56.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3095000000000002e-05 2024-09-13T10:10:56.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286005+0000", 2024-09-13T10:10:56.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7147999999999999e-05 2024-09-13T10:10:56.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286119+0000", 2024-09-13T10:10:56.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000113999 2024-09-13T10:10:56.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286281+0000", 2024-09-13T10:10:56.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000161948 2024-09-13T10:10:56.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:10:56.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.291713+0000", 2024-09-13T10:10:56.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0054324810000000003 2024-09-13T10:10:56.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293743+0000", 2024-09-13T10:10:56.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020298439999999998 2024-09-13T10:10:56.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293904+0000", 2024-09-13T10:10:56.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000160853 2024-09-13T10:10:56.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293952+0000", 2024-09-13T10:10:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7991999999999997e-05 2024-09-13T10:10:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294052+0000", 2024-09-13T10:10:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9893000000000001e-05 2024-09-13T10:10:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294145+0000", 2024-09-13T10:10:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3084000000000005e-05 2024-09-13T10:10:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:372 4.9 4:96fcf081:::smithi19530714-23:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:10:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.298189+0000", 2024-09-13T10:10:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.901288622, 2024-09-13T10:10:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0063382159999999998, 2024-09-13T10:10:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.305 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 372 2024-09-13T10:10:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.298189+0000", 2024-09-13T10:10:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.298189+0000", 2024-09-13T10:10:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.298191+0000", 2024-09-13T10:10:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1799999999999999e-06 2024-09-13T10:10:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.298201+0000", 2024-09-13T10:10:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0353999999999999e-05 2024-09-13T10:10:56.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.298223+0000", 2024-09-13T10:10:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1633000000000001e-05 2024-09-13T10:10:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.298239+0000", 2024-09-13T10:10:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6305999999999999e-05 2024-09-13T10:10:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.298377+0000", 2024-09-13T10:10:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000138052 2024-09-13T10:10:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.298524+0000", 2024-09-13T10:10:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014663799999999999 2024-09-13T10:10:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:10:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.302970+0000", 2024-09-13T10:10:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0044464609999999996 2024-09-13T10:10:56.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.303988+0000", 2024-09-13T10:10:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001017588 2024-09-13T10:10:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.304208+0000", 2024-09-13T10:10:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022030899999999999 2024-09-13T10:10:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.304318+0000", 2024-09-13T10:10:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109819 2024-09-13T10:10:56.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.304444+0000", 2024-09-13T10:10:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000126196 2024-09-13T10:10:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.304527+0000", 2024-09-13T10:10:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2680000000000001e-05 2024-09-13T10:10:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:382 4.9 4:998d9bf1:::smithi19530714-33:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:10:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.320041+0000", 2024-09-13T10:10:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.8794366279999997, 2024-09-13T10:10:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0071818619999999998, 2024-09-13T10:10:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.312 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 382 2024-09-13T10:10:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.320041+0000", 2024-09-13T10:10:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.320041+0000", 2024-09-13T10:10:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.320044+0000", 2024-09-13T10:10:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0170000000000001e-06 2024-09-13T10:10:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.320054+0000", 2024-09-13T10:10:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0607e-05 2024-09-13T10:10:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.320084+0000", 2024-09-13T10:10:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9332999999999999e-05 2024-09-13T10:10:56.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.320098+0000", 2024-09-13T10:10:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4569999999999999e-05 2024-09-13T10:10:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.320214+0000", 2024-09-13T10:10:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115838 2024-09-13T10:10:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.320394+0000", 2024-09-13T10:10:56.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017978699999999999 2024-09-13T10:10:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:10:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.324487+0000", 2024-09-13T10:10:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0040933990000000002 2024-09-13T10:10:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.326640+0000", 2024-09-13T10:10:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021528659999999998 2024-09-13T10:10:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.326718+0000", 2024-09-13T10:10:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7373000000000006e-05 2024-09-13T10:10:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.327005+0000", 2024-09-13T10:10:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000287311 2024-09-13T10:10:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.327139+0000", 2024-09-13T10:10:56.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000133931 2024-09-13T10:10:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.327223+0000", 2024-09-13T10:10:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3830000000000002e-05 2024-09-13T10:10:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:396 4.9 4:9bec18ef:::smithi19530714-47:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:10:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.321357+0000", 2024-09-13T10:10:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.8781207670000004, 2024-09-13T10:10:56.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0062458990000000001, 2024-09-13T10:10:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 396 2024-09-13T10:10:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.321357+0000", 2024-09-13T10:10:56.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.321357+0000", 2024-09-13T10:10:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.321359+0000", 2024-09-13T10:10:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.278e-06 2024-09-13T10:10:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.321370+0000", 2024-09-13T10:10:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1070000000000001e-05 2024-09-13T10:10:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.321398+0000", 2024-09-13T10:10:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7742999999999998e-05 2024-09-13T10:10:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.321416+0000", 2024-09-13T10:10:56.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7872000000000001e-05 2024-09-13T10:10:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.321545+0000", 2024-09-13T10:10:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012977900000000001 2024-09-13T10:10:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.321726+0000", 2024-09-13T10:10:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018042700000000001 2024-09-13T10:10:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:10:56.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.325428+0000", 2024-09-13T10:10:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0037022660000000001 2024-09-13T10:10:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.327380+0000", 2024-09-13T10:10:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019523730000000001 2024-09-13T10:10:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.327404+0000", 2024-09-13T10:10:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4003e-05 2024-09-13T10:10:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.327461+0000", 2024-09-13T10:10:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6986e-05 2024-09-13T10:10:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.327548+0000", 2024-09-13T10:10:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6798999999999993e-05 2024-09-13T10:10:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.327603+0000", 2024-09-13T10:10:56.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4302999999999998e-05 2024-09-13T10:10:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:482 4.9 4:998d9bf1:::smithi19530714-33:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:10:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.342576+0000", 2024-09-13T10:10:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.8569014450000001, 2024-09-13T10:10:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0069407740000000002, 2024-09-13T10:10:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.325 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 482 2024-09-13T10:10:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.342576+0000", 2024-09-13T10:10:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.342576+0000", 2024-09-13T10:10:56.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.342578+0000", 2024-09-13T10:10:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3199999999999998e-06 2024-09-13T10:10:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.342594+0000", 2024-09-13T10:10:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5911999999999999e-05 2024-09-13T10:10:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.342645+0000", 2024-09-13T10:10:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0716e-05 2024-09-13T10:10:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.342669+0000", 2024-09-13T10:10:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4229000000000001e-05 2024-09-13T10:10:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.342786+0000", 2024-09-13T10:10:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116623 2024-09-13T10:10:56.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.342957+0000", 2024-09-13T10:10:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017153399999999999 2024-09-13T10:10:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:10:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.346896+0000", 2024-09-13T10:10:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0039386359999999997 2024-09-13T10:10:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:10:56.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.347829+0000", 2024-09-13T10:10:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00093266400000000004 2024-09-13T10:10:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.348019+0000", 2024-09-13T10:10:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000190638 2024-09-13T10:10:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:10:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.349348+0000", 2024-09-13T10:10:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013287780000000001 2024-09-13T10:10:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.349442+0000", 2024-09-13T10:10:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3916e-05 2024-09-13T10:10:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.349517+0000", 2024-09-13T10:10:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4808000000000006e-05 2024-09-13T10:10:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:801 4.9 4:96fcf081:::smithi19530714-23:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:10:56.331 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.798334+0000", 2024-09-13T10:10:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.4011433799999997, 2024-09-13T10:10:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056549887, 2024-09-13T10:10:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:10:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:10:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:10:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:10:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 801 2024-09-13T10:10:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798334+0000", 2024-09-13T10:10:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798334+0000", 2024-09-13T10:10:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798338+0000", 2024-09-13T10:10:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3239999999999996e-06 2024-09-13T10:10:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798355+0000", 2024-09-13T10:10:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7149000000000001e-05 2024-09-13T10:10:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798403+0000", 2024-09-13T10:10:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7132000000000002e-05 2024-09-13T10:10:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798431+0000", 2024-09-13T10:10:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8365999999999999e-05 2024-09-13T10:10:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798534+0000", 2024-09-13T10:10:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102545 2024-09-13T10:10:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798715+0000", 2024-09-13T10:10:56.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018161800000000001 2024-09-13T10:10:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.854884+0000", 2024-09-13T10:10:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056168753000000002 2024-09-13T10:10:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.336 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.336 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:10:56.337 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T10:10:56.339 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-13T10:10:56.340 INFO:tasks.rados.rados.0.smithi195.stdout:690: finishing write tid 5 to smithi19530714-19 2024-09-13T10:10:56.340 INFO:tasks.rados.rados.0.smithi195.stdout:690: oid 19 updating version 0 to 38 2024-09-13T10:10:56.341 INFO:tasks.rados.rados.0.smithi195.stdout:690: oid 19 updating version 38 to 40 2024-09-13T10:10:56.341 INFO:tasks.rados.rados.0.smithi195.stdout:690: oid 19 version 40 is already newer than 37 2024-09-13T10:10:56.341 INFO:tasks.rados.rados.0.smithi195.stdout:690: oid 19 version 40 is already newer than 39 2024-09-13T10:10:56.341 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 40 (ObjNum 78 snap 0 seq_num 78) dirty exists 2024-09-13T10:10:56.341 INFO:tasks.rados.rados.0.smithi195.stdout:690: left oid 19 (ObjNum 78 snap 0 seq_num 78) 2024-09-13T10:10:56.341 INFO:tasks.rados.rados.0.smithi195.stdout:690: done (7 left) 2024-09-13T10:10:56.341 INFO:tasks.rados.rados.0.smithi195.stdout:691: done (6 left) 2024-09-13T10:10:56.341 INFO:tasks.rados.rados.0.smithi195.stdout:692: done (5 left) 2024-09-13T10:10:56.341 INFO:tasks.rados.rados.0.smithi195.stdout:698: read oid 4 snap -1 2024-09-13T10:10:56.341 INFO:tasks.rados.rados.0.smithi195.stdout:698: expect (ObjNum 58 snap 0 seq_num 58) 2024-09-13T10:10:56.341 INFO:tasks.rados.rados.0.smithi195.stdout:699: write (excl) oid 6 current snap is 0 2024-09-13T10:10:56.341 INFO:tasks.rados.rados.0.smithi195.stdout:693: finishing set_chunk to oid 27 2024-09-13T10:10:56.342 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 27 set_chunk returned error code -95 offset: 483328 length: 295424 2024-09-13T10:10:56.342 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 0 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-13T10:10:56.342 INFO:tasks.rados.rados.0.smithi195.stdout:694: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-13T10:10:56.342 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 427578 2024-09-13T10:10:56.342 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:56.342 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:56.342 INFO:tasks.rados.rados.0.smithi195.stdout:696: got (0) Success 2024-09-13T10:10:56.342 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 0 (ObjNum 63 snap 0 seq_num 63) dirty exists 2024-09-13T10:10:56.342 INFO:tasks.rados.rados.0.smithi195.stdout:695: got (0) Success 2024-09-13T10:10:56.342 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 0 (ObjNum 51 snap 0 seq_num 51) dirty exists 2024-09-13T10:10:56.342 INFO:tasks.rados.rados.0.smithi195.stdout:697: got (0) Success 2024-09-13T10:10:56.342 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 21 (ObjNum 65 snap 0 seq_num 65) dirty exists 2024-09-13T10:10:56.342 INFO:tasks.rados.rados.0.smithi195.stdout:699: seq_num 79 ranges {527753=708201,1856548=737690,3286534=41219} 2024-09-13T10:10:56.342 INFO:tasks.rados.rados.0.smithi195.stdout:699: writing smithi19530714-6 from 527753 to 1235954 tid 1 2024-09-13T10:10:56.343 INFO:tasks.rados.rados.0.smithi195.stdout:699: writing smithi19530714-6 from 1856548 to 2594238 tid 2 2024-09-13T10:10:56.343 INFO:tasks.rados.rados.0.smithi195.stdout:699: writing smithi19530714-6 from 3286534 to 3327753 tid 3 2024-09-13T10:10:56.343 INFO:tasks.rados.rados.0.smithi195.stdout:693: done (6 left) 2024-09-13T10:10:56.343 INFO:tasks.rados.rados.0.smithi195.stdout:694: done (5 left) 2024-09-13T10:10:56.343 INFO:tasks.rados.rados.0.smithi195.stdout:695: done (4 left) 2024-09-13T10:10:56.343 INFO:tasks.rados.rados.0.smithi195.stdout:696: done (3 left) 2024-09-13T10:10:56.343 INFO:tasks.rados.rados.0.smithi195.stdout:697: done (2 left) 2024-09-13T10:10:56.343 INFO:tasks.rados.rados.0.smithi195.stdout:700: set_chunk oid 12 target oid 2024-09-13T10:10:56.343 INFO:tasks.rados.rados.0.smithi195.stdout:700: set_chunk oid 12 offset: 84992 length: 489984 target oid 5d5c23bfc0098014f41970f365eb6399e87d30d693159716cd255b42f8b27dd1 offset: 0 snap -1 2024-09-13T10:10:56.343 INFO:tasks.rados.rados.0.smithi195.stdout:701: set_chunk oid 43 target oid 2024-09-13T10:10:56.343 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:10:56.335+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:10:56.345 INFO:tasks.rados.rados.0.smithi195.stdout:701: set_chunk oid 43 offset: 769536 length: 513536 target oid 23350f9193a45b21569745570c867284eabd73203a3689febb06e99e8b082d0b offset: 0 snap -1 2024-09-13T10:10:56.345 INFO:tasks.rados.rados.0.smithi195.stdout:702: read oid 38 snap -1 2024-09-13T10:10:56.345 INFO:tasks.rados.rados.0.smithi195.stdout:702: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:10:56.345 INFO:tasks.rados.rados.0.smithi195.stdout:703: tier_evict oid 45 2024-09-13T10:10:56.345 INFO:tasks.rados.rados.0.smithi195.stdout:703: tier_evict oid 45 snap -1 2024-09-13T10:10:56.345 INFO:tasks.rados.rados.0.smithi195.stdout:704: write oid 36 current snap is 0 2024-09-13T10:10:56.346 INFO:tasks.rados.rados.0.smithi195.stdout:698: expect (ObjNum 58 snap 0 seq_num 58) 2024-09-13T10:10:56.416 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2717426 2024-09-13T10:10:56.439 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:56.439+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:56.446 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2150648 2024-09-13T10:10:56.459 INFO:tasks.rados.rados.0.smithi195.stdout:699: finishing write tid 1 to smithi19530714-6 2024-09-13T10:10:56.459 INFO:tasks.rados.rados.0.smithi195.stdout:699: finishing write tid 2 to smithi19530714-6 2024-09-13T10:10:56.459 INFO:tasks.rados.rados.0.smithi195.stdout:699: finishing write tid 3 to smithi19530714-6 2024-09-13T10:10:56.459 INFO:tasks.rados.rados.0.smithi195.stdout:699: finishing write tid 4 to smithi19530714-6 2024-09-13T10:10:56.459 INFO:tasks.rados.rados.0.smithi195.stdout:699: finishing write tid 5 to smithi19530714-6 2024-09-13T10:10:56.459 INFO:tasks.rados.rados.0.smithi195.stdout:699: oid 6 updating version 0 to 37 2024-09-13T10:10:56.459 INFO:tasks.rados.rados.0.smithi195.stdout:699: oid 6 version 37 is already newer than 35 2024-09-13T10:10:56.459 INFO:tasks.rados.rados.0.smithi195.stdout:699: oid 6 version 37 is already newer than 36 2024-09-13T10:10:56.459 INFO:tasks.rados.rados.0.smithi195.stdout:699: oid 6 version 37 is already newer than 34 2024-09-13T10:10:56.459 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 37 (ObjNum 79 snap 0 seq_num 79) dirty exists 2024-09-13T10:10:56.459 INFO:tasks.rados.rados.0.smithi195.stdout:699: left oid 6 (ObjNum 79 snap 0 seq_num 79) 2024-09-13T10:10:56.460 INFO:tasks.rados.rados.0.smithi195.stdout:700: finishing set_chunk to oid 12 2024-09-13T10:10:56.460 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 3 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-13T10:10:56.460 INFO:tasks.rados.rados.0.smithi195.stdout:701: finishing set_chunk to oid 43 2024-09-13T10:10:56.460 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 43 set_chunk returned error code -95 offset: 769536 length: 513536 2024-09-13T10:10:56.460 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 0 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-09-13T10:10:56.460 INFO:tasks.rados.rados.0.smithi195.stdout:702: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:10:56.467 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:10:56.467 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:10:56.467 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:10:56.467 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:10:56.485 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-13T10:10:56.543 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:10:56.541+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:10:56.548 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:56.596 INFO:tasks.rados.rados.0.smithi195.stdout:703: got (22) Invalid argument 2024-09-13T10:10:56.596 INFO:tasks.rados.rados.0.smithi195.stdout:704: seq_num 80 ranges {544560=783450,1915030=481348,2797604=479761} 2024-09-13T10:10:56.605 INFO:tasks.rados.rados.0.smithi195.stdout:704: writing smithi19530714-36 from 544560 to 1328010 tid 1 2024-09-13T10:10:56.612 INFO:tasks.rados.rados.0.smithi195.stdout:704: writing smithi19530714-36 from 1915030 to 2396378 tid 2 2024-09-13T10:10:56.618 INFO:tasks.rados.rados.0.smithi195.stdout:704: writing smithi19530714-36 from 2797604 to 3277365 tid 3 2024-09-13T10:10:56.618 INFO:tasks.rados.rados.0.smithi195.stdout:698: done (6 left) 2024-09-13T10:10:56.618 INFO:tasks.rados.rados.0.smithi195.stdout:699: done (5 left) 2024-09-13T10:10:56.618 INFO:tasks.rados.rados.0.smithi195.stdout:700: done (4 left) 2024-09-13T10:10:56.618 INFO:tasks.rados.rados.0.smithi195.stdout:701: done (3 left) 2024-09-13T10:10:56.618 INFO:tasks.rados.rados.0.smithi195.stdout:702: done (2 left) 2024-09-13T10:10:56.618 INFO:tasks.rados.rados.0.smithi195.stdout:703: done (1 left) 2024-09-13T10:10:56.618 INFO:tasks.rados.rados.0.smithi195.stdout:705: tier_promote oid 33 2024-09-13T10:10:56.618 INFO:tasks.rados.rados.0.smithi195.stdout:706: read oid 22 snap -1 2024-09-13T10:10:56.618 INFO:tasks.rados.rados.0.smithi195.stdout:706: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-13T10:10:56.619 INFO:tasks.rados.rados.0.smithi195.stdout:707: set_chunk oid 8 target oid 2024-09-13T10:10:56.625 INFO:tasks.rados.rados.0.smithi195.stdout:707: set_chunk oid 8 offset: 1752576 length: 88064 target oid 542e8f35f2c8e40d98cf7c148f57d7c90c34b85b17903262cd0765a85006619d offset: 0 snap -1 2024-09-13T10:10:56.625 INFO:tasks.rados.rados.0.smithi195.stdout:708: read oid 39 snap -1 2024-09-13T10:10:56.625 INFO:tasks.rados.rados.0.smithi195.stdout:708: expect (ObjNum 38 snap 0 seq_num 38) 2024-09-13T10:10:56.625 INFO:tasks.rados.rados.0.smithi195.stdout:709: write oid 1 current snap is 0 2024-09-13T10:10:56.625 INFO:tasks.rados.rados.0.smithi195.stdout:709: seq_num 81 ranges {561367=458699,1573512=741359,2941380=419987} 2024-09-13T10:10:56.631 INFO:tasks.rados.rados.0.smithi195.stdout:709: writing smithi19530714-1 from 561367 to 1020066 tid 1 2024-09-13T10:10:56.639 INFO:tasks.rados.rados.0.smithi195.stdout:709: writing smithi19530714-1 from 1573512 to 2314871 tid 2 2024-09-13T10:10:56.644 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:56.644+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:10:56.646 INFO:tasks.rados.rados.0.smithi195.stdout:709: writing smithi19530714-1 from 2941380 to 3361367 tid 3 2024-09-13T10:10:56.646 INFO:tasks.rados.rados.0.smithi195.stdout:710: read oid 9 snap -1 2024-09-13T10:10:56.647 INFO:tasks.rados.rados.0.smithi195.stdout:710: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-13T10:10:56.647 INFO:tasks.rados.rados.0.smithi195.stdout:711: read oid 14 snap -1 2024-09-13T10:10:56.647 INFO:tasks.rados.rados.0.smithi195.stdout:711: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-13T10:10:56.647 INFO:tasks.rados.rados.0.smithi195.stdout:712: set_chunk oid 20 target oid 2024-09-13T10:10:56.657 INFO:tasks.rados.rados.0.smithi195.stdout:712: set_chunk oid 20 offset: 929280 length: 245248 target oid ab89a0496833b2acfd4a54aaf7f6de1a32d30c2ae9642a7f30de94b49f08f136 offset: 0 snap -1 2024-09-13T10:10:56.657 INFO:tasks.rados.rados.0.smithi195.stdout:713: tier_promote oid 27 2024-09-13T10:10:56.657 INFO:tasks.rados.rados.0.smithi195.stdout:704: finishing write tid 1 to smithi19530714-36 2024-09-13T10:10:56.657 INFO:tasks.rados.rados.0.smithi195.stdout:714: read oid 45 snap -1 2024-09-13T10:10:56.657 INFO:tasks.rados.rados.0.smithi195.stdout:714: expect (ObjNum 59 snap 0 seq_num 59) 2024-09-13T10:10:56.657 INFO:tasks.rados.rados.0.smithi195.stdout:715: read oid 42 snap -1 2024-09-13T10:10:56.657 INFO:tasks.rados.rados.0.smithi195.stdout:715: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-13T10:10:56.657 INFO:tasks.rados.rados.0.smithi195.stdout:716: read oid 4 snap -1 2024-09-13T10:10:56.658 INFO:tasks.rados.rados.0.smithi195.stdout:716: expect (ObjNum 58 snap 0 seq_num 58) 2024-09-13T10:10:56.658 INFO:tasks.rados.rados.0.smithi195.stdout:717: read oid 43 snap -1 2024-09-13T10:10:56.658 INFO:tasks.rados.rados.0.smithi195.stdout:717: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-13T10:10:56.658 INFO:tasks.rados.rados.0.smithi195.stdout:718: read oid 44 snap -1 2024-09-13T10:10:56.658 INFO:tasks.rados.rados.0.smithi195.stdout:718: expect (ObjNum 72 snap 0 seq_num 72) 2024-09-13T10:10:56.658 INFO:tasks.rados.rados.0.smithi195.stdout:719: set_chunk oid 11 target oid 2024-09-13T10:10:56.670 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:10:56.670 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:10:56.670 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:10:56.670 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:10:56.670 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:10:56.689 INFO:tasks.rados.rados.0.smithi195.stdout:719: set_chunk oid 11 offset: 457216 length: 1419776 target oid 4f893b68aa248b8fb4544a484fa5e966179cdf2e6bafa35a82fbb4198705f32c offset: 0 snap -1 2024-09-13T10:10:56.690 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:56.690 INFO:tasks.rados.rados.0.smithi195.stdout:704: finishing write tid 2 to smithi19530714-36 2024-09-13T10:10:56.690 INFO:tasks.rados.rados.0.smithi195.stdout:707: finishing set_chunk to oid 8 2024-09-13T10:10:56.690 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 8 set_chunk returned error code -95 offset: 1752576 length: 88064 2024-09-13T10:10:56.690 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 0 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-09-13T10:10:56.690 INFO:tasks.rados.rados.0.smithi195.stdout:704: finishing write tid 3 to smithi19530714-36 2024-09-13T10:10:56.690 INFO:tasks.rados.rados.0.smithi195.stdout:707: done (15 left) 2024-09-13T10:10:56.690 INFO:tasks.rados.rados.0.smithi195.stdout:720: write (excl) oid 32 current snap is 0 2024-09-13T10:10:56.690 INFO:tasks.rados.rados.0.smithi195.stdout:720: seq_num 82 ranges {578174=533948,1515641=485017,2736097=502305} 2024-09-13T10:10:56.692 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-13T10:10:56.695 INFO:tasks.rados.rados.0.smithi195.stdout:720: writing smithi19530714-32 from 578174 to 1112122 tid 1 2024-09-13T10:10:56.700 INFO:tasks.rados.rados.0.smithi195.stdout:720: writing smithi19530714-32 from 1515641 to 2000658 tid 2 2024-09-13T10:10:56.707 INFO:tasks.rados.rados.0.smithi195.stdout:720: writing smithi19530714-32 from 2736097 to 3238402 tid 3 2024-09-13T10:10:56.707 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:56.707 INFO:tasks.rados.rados.0.smithi195.stdout:705: got (0) Success 2024-09-13T10:10:56.707 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 22 (ObjNum 64 snap 0 seq_num 64) dirty exists 2024-09-13T10:10:56.707 INFO:tasks.rados.rados.0.smithi195.stdout:706: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-13T10:10:56.745 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:10:56.741+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:10:56.787 INFO:tasks.rados.rados.0.smithi195.stdout:704: finishing write tid 4 to smithi19530714-36 2024-09-13T10:10:56.787 INFO:tasks.rados.rados.0.smithi195.stdout:704: finishing write tid 5 to smithi19530714-36 2024-09-13T10:10:56.787 INFO:tasks.rados.rados.0.smithi195.stdout:704: oid 36 updating version 0 to 16 2024-09-13T10:10:56.787 INFO:tasks.rados.rados.0.smithi195.stdout:704: oid 36 version 16 is already newer than 14 2024-09-13T10:10:56.787 INFO:tasks.rados.rados.0.smithi195.stdout:704: oid 36 updating version 16 to 17 2024-09-13T10:10:56.787 INFO:tasks.rados.rados.0.smithi195.stdout:704: oid 36 version 17 is already newer than 15 2024-09-13T10:10:56.787 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 17 (ObjNum 80 snap 0 seq_num 80) dirty exists 2024-09-13T10:10:56.787 INFO:tasks.rados.rados.0.smithi195.stdout:704: left oid 36 (ObjNum 80 snap 0 seq_num 80) 2024-09-13T10:10:56.787 INFO:tasks.rados.rados.0.smithi195.stdout:708: expect (ObjNum 38 snap 0 seq_num 38) 2024-09-13T10:10:56.788 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:56.812 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:10:56.812 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:10:56.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:10:56.813 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:10:56.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.813 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:161 4.d e38/31 4:b19b932a:::smithi19530714-41:head v 38'1, mlcod=0'0)", 2024-09-13T10:10:56.813 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:10:56.813 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.9557191029999998, 2024-09-13T10:10:56.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056705289999999997, 2024-09-13T10:10:56.813 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.813 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.813 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.813 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:10:56.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:10:56.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:10:56.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0999999999999998e-07 2024-09-13T10:10:56.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856051+0000", 2024-09-13T10:10:56.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020382130000000001 2024-09-13T10:10:56.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856079+0000", 2024-09-13T10:10:56.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8345999999999999e-05 2024-09-13T10:10:56.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856093+0000", 2024-09-13T10:10:56.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4353e-05 2024-09-13T10:10:56.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856232+0000", 2024-09-13T10:10:56.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013922199999999999 2024-09-13T10:10:56.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856267+0000", 2024-09-13T10:10:56.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4823000000000002e-05 2024-09-13T10:10:56.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.859505+0000", 2024-09-13T10:10:56.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032373269999999999 2024-09-13T10:10:56.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.859682+0000", 2024-09-13T10:10:56.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017753499999999999 2024-09-13T10:10:56.817 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.817 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.818 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.818 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:162 4.d e38/31 4:b19b932a:::smithi19530714-41:head v 38'2, mlcod=0'0)", 2024-09-13T10:10:56.818 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.861006+0000", 2024-09-13T10:10:56.818 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.9487248270000004, 2024-09-13T10:10:56.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003007591, 2024-09-13T10:10:56.818 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.818 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.818 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.818 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861006+0000", 2024-09-13T10:10:56.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861006+0000", 2024-09-13T10:10:56.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861008+0000", 2024-09-13T10:10:56.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9860000000000001e-06 2024-09-13T10:10:56.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861797+0000", 2024-09-13T10:10:56.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00078932099999999995 2024-09-13T10:10:56.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861821+0000", 2024-09-13T10:10:56.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4165e-05 2024-09-13T10:10:56.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861841+0000", 2024-09-13T10:10:56.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9383000000000001e-05 2024-09-13T10:10:56.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861960+0000", 2024-09-13T10:10:56.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000118997 2024-09-13T10:10:56.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861996+0000", 2024-09-13T10:10:56.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6099000000000001e-05 2024-09-13T10:10:56.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.863918+0000", 2024-09-13T10:10:56.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019220489999999999 2024-09-13T10:10:56.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.864013+0000", 2024-09-13T10:10:56.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5591e-05 2024-09-13T10:10:56.822 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.822 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.822 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.823 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:163 4.d e38/31 4:b19b932a:::smithi19530714-41:head v 38'3, mlcod=0'0)", 2024-09-13T10:10:56.823 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.862576+0000", 2024-09-13T10:10:56.823 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.9471545470000002, 2024-09-13T10:10:56.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0018310049999999999, 2024-09-13T10:10:56.823 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.823 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.823 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.823 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.862576+0000", 2024-09-13T10:10:56.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.862576+0000", 2024-09-13T10:10:56.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.862577+0000", 2024-09-13T10:10:56.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1200000000000002e-07 2024-09-13T10:10:56.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.862588+0000", 2024-09-13T10:10:56.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0607e-05 2024-09-13T10:10:56.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.862598+0000", 2024-09-13T10:10:56.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9420000000000007e-06 2024-09-13T10:10:56.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.862605+0000", 2024-09-13T10:10:56.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0010000000000004e-06 2024-09-13T10:10:56.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.863254+0000", 2024-09-13T10:10:56.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064926800000000005 2024-09-13T10:10:56.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.863293+0000", 2024-09-13T10:10:56.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8918999999999998e-05 2024-09-13T10:10:56.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.864325+0000", 2024-09-13T10:10:56.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010318390000000001 2024-09-13T10:10:56.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.864407+0000", 2024-09-13T10:10:56.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2616999999999999e-05 2024-09-13T10:10:56.827 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.827 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.827 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.827 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:181 4.d e38/31 4:ba7d2bc5:::smithi19530714-46:head v 38'4, mlcod=38'2)", 2024-09-13T10:10:56.828 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.926727+0000", 2024-09-13T10:10:56.828 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.8830040940000004, 2024-09-13T10:10:56.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0042825909999999997, 2024-09-13T10:10:56.828 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.828 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.828 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.828 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.926727+0000", 2024-09-13T10:10:56.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.926727+0000", 2024-09-13T10:10:56.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.926729+0000", 2024-09-13T10:10:56.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.244e-06 2024-09-13T10:10:56.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.927979+0000", 2024-09-13T10:10:56.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0012504230000000001 2024-09-13T10:10:56.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.928010+0000", 2024-09-13T10:10:56.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1115000000000003e-05 2024-09-13T10:10:56.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.928030+0000", 2024-09-13T10:10:56.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0063000000000001e-05 2024-09-13T10:10:56.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.928185+0000", 2024-09-13T10:10:56.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015451899999999999 2024-09-13T10:10:56.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.928248+0000", 2024-09-13T10:10:56.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3391000000000003e-05 2024-09-13T10:10:56.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.930963+0000", 2024-09-13T10:10:56.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027147170000000002 2024-09-13T10:10:56.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.931009+0000", 2024-09-13T10:10:56.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6118999999999997e-05 2024-09-13T10:10:56.832 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.832 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.832 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.832 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:182 4.d e38/31 4:ba7d2bc5:::smithi19530714-46:head v 38'5, mlcod=38'3)", 2024-09-13T10:10:56.832 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.933744+0000", 2024-09-13T10:10:56.833 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.8759862649999999, 2024-09-13T10:10:56.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0031909019999999998, 2024-09-13T10:10:56.833 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.833 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.833 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.833 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.933744+0000", 2024-09-13T10:10:56.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.933744+0000", 2024-09-13T10:10:56.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.933746+0000", 2024-09-13T10:10:56.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.903e-06 2024-09-13T10:10:56.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.934786+0000", 2024-09-13T10:10:56.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001039683 2024-09-13T10:10:56.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.934812+0000", 2024-09-13T10:10:56.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5902e-05 2024-09-13T10:10:56.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.934826+0000", 2024-09-13T10:10:56.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3696000000000001e-05 2024-09-13T10:10:56.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.934896+0000", 2024-09-13T10:10:56.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0117999999999995e-05 2024-09-13T10:10:56.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.934915+0000", 2024-09-13T10:10:56.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9505e-05 2024-09-13T10:10:56.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.936894+0000", 2024-09-13T10:10:56.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019791990000000001 2024-09-13T10:10:56.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.936935+0000", 2024-09-13T10:10:56.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0896000000000003e-05 2024-09-13T10:10:56.837 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.837 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.837 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.837 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:183 4.d e38/31 4:ba7d2bc5:::smithi19530714-46:head v 38'6, mlcod=38'3)", 2024-09-13T10:10:56.837 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.935517+0000", 2024-09-13T10:10:56.837 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.8742135510000004, 2024-09-13T10:10:56.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001882075, 2024-09-13T10:10:56.838 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.838 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.838 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.838 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.935517+0000", 2024-09-13T10:10:56.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.935517+0000", 2024-09-13T10:10:56.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.935519+0000", 2024-09-13T10:10:56.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8199999999999999e-06 2024-09-13T10:10:56.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.935530+0000", 2024-09-13T10:10:56.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1286e-05 2024-09-13T10:10:56.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.935554+0000", 2024-09-13T10:10:56.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4068999999999999e-05 2024-09-13T10:10:56.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.935574+0000", 2024-09-13T10:10:56.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9986000000000001e-05 2024-09-13T10:10:56.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.935844+0000", 2024-09-13T10:10:56.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026950900000000002 2024-09-13T10:10:56.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.935920+0000", 2024-09-13T10:10:56.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6440999999999995e-05 2024-09-13T10:10:56.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.937367+0000", 2024-09-13T10:10:56.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001446941 2024-09-13T10:10:56.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.937399+0000", 2024-09-13T10:10:56.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2023000000000002e-05 2024-09-13T10:10:56.842 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.842 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.842 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.842 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:290 4.d e38/31 4:b19b932a:::smithi19530714-41:head v 38'7, mlcod=38'5)", 2024-09-13T10:10:56.842 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.273711+0000", 2024-09-13T10:10:56.842 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.5360200930000003, 2024-09-13T10:10:56.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015433770000000001, 2024-09-13T10:10:56.843 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.843 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.843 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.843 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.273711+0000", 2024-09-13T10:10:56.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.273711+0000", 2024-09-13T10:10:56.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.273713+0000", 2024-09-13T10:10:56.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1629999999999999e-06 2024-09-13T10:10:56.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.273732+0000", 2024-09-13T10:10:56.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9211e-05 2024-09-13T10:10:56.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.273767+0000", 2024-09-13T10:10:56.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4610000000000002e-05 2024-09-13T10:10:56.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.273796+0000", 2024-09-13T10:10:56.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.898e-05 2024-09-13T10:10:56.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.273926+0000", 2024-09-13T10:10:56.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000130314 2024-09-13T10:10:56.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.273973+0000", 2024-09-13T10:10:56.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6794e-05 2024-09-13T10:10:56.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.275151+0000", 2024-09-13T10:10:56.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001178503 2024-09-13T10:10:56.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.275254+0000", 2024-09-13T10:10:56.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102802 2024-09-13T10:10:56.847 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.847 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.847 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.847 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:295 4.d e38/31 4:ba7d2bc5:::smithi19530714-46:head v 38'8, mlcod=38'6)", 2024-09-13T10:10:56.847 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.276520+0000", 2024-09-13T10:10:56.847 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.5332103049999999, 2024-09-13T10:10:56.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013610129999999999, 2024-09-13T10:10:56.848 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.848 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.848 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.848 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.276520+0000", 2024-09-13T10:10:56.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.276520+0000", 2024-09-13T10:10:56.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.276523+0000", 2024-09-13T10:10:56.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1629999999999999e-06 2024-09-13T10:10:56.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.276545+0000", 2024-09-13T10:10:56.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2147999999999999e-05 2024-09-13T10:10:56.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.276575+0000", 2024-09-13T10:10:56.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0642000000000002e-05 2024-09-13T10:10:56.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.276608+0000", 2024-09-13T10:10:56.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2996000000000001e-05 2024-09-13T10:10:56.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.276725+0000", 2024-09-13T10:10:56.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011679399999999999 2024-09-13T10:10:56.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.276765+0000", 2024-09-13T10:10:56.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9793999999999999e-05 2024-09-13T10:10:56.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.277828+0000", 2024-09-13T10:10:56.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010630430000000001 2024-09-13T10:10:56.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.277881+0000", 2024-09-13T10:10:56.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3433000000000002e-05 2024-09-13T10:10:56.852 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.852 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.852 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.852 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:340 4.d e38/31 4:b19b932a:::smithi19530714-41:head v 38'9, mlcod=38'7)", 2024-09-13T10:10:56.852 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.291785+0000", 2024-09-13T10:10:56.853 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.5179452829999995, 2024-09-13T10:10:56.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001467363, 2024-09-13T10:10:56.853 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.853 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.853 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.853 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.291785+0000", 2024-09-13T10:10:56.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.291785+0000", 2024-09-13T10:10:56.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.291787+0000", 2024-09-13T10:10:56.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.595e-06 2024-09-13T10:10:56.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.291824+0000", 2024-09-13T10:10:56.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6529000000000002e-05 2024-09-13T10:10:56.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.291858+0000", 2024-09-13T10:10:56.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4211999999999997e-05 2024-09-13T10:10:56.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.291874+0000", 2024-09-13T10:10:56.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6469000000000001e-05 2024-09-13T10:10:56.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.292074+0000", 2024-09-13T10:10:56.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000199311 2024-09-13T10:10:56.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.292100+0000", 2024-09-13T10:10:56.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6849e-05 2024-09-13T10:10:56.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293180+0000", 2024-09-13T10:10:56.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001079418 2024-09-13T10:10:56.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293253+0000", 2024-09-13T10:10:56.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2979999999999996e-05 2024-09-13T10:10:56.858 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.858 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.858 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.858 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:345 4.d e38/31 4:ba7d2bc5:::smithi19530714-46:head v 38'10, mlcod=38'8)", 2024-09-13T10:10:56.859 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.294140+0000", 2024-09-13T10:10:56.859 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.5155907109999998, 2024-09-13T10:10:56.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0014495249999999999, 2024-09-13T10:10:56.859 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.859 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.859 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.859 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294140+0000", 2024-09-13T10:10:56.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294140+0000", 2024-09-13T10:10:56.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294141+0000", 2024-09-13T10:10:56.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.372e-06 2024-09-13T10:10:56.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294153+0000", 2024-09-13T10:10:56.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1192000000000001e-05 2024-09-13T10:10:56.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294175+0000", 2024-09-13T10:10:56.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2259000000000002e-05 2024-09-13T10:10:56.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294189+0000", 2024-09-13T10:10:56.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3747999999999999e-05 2024-09-13T10:10:56.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294335+0000", 2024-09-13T10:10:56.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000146737 2024-09-13T10:10:56.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294380+0000", 2024-09-13T10:10:56.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4966999999999998e-05 2024-09-13T10:10:56.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.295520+0000", 2024-09-13T10:10:56.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0011398020000000001 2024-09-13T10:10:56.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.295590+0000", 2024-09-13T10:10:56.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9448000000000003e-05 2024-09-13T10:10:56.863 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.864 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.864 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.864 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:395 4.d e39/31 4:ba7d2bc5:::smithi19530714-46:head v 39'11, mlcod=38'9)", 2024-09-13T10:10:56.864 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.327859+0000", 2024-09-13T10:10:56.864 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.4818720369999996, 2024-09-13T10:10:56.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001322801, 2024-09-13T10:10:56.864 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.864 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.864 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.864 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.327859+0000", 2024-09-13T10:10:56.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.327859+0000", 2024-09-13T10:10:56.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.327860+0000", 2024-09-13T10:10:56.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0300000000000001e-06 2024-09-13T10:10:56.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.327888+0000", 2024-09-13T10:10:56.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8039e-05 2024-09-13T10:10:56.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.327913+0000", 2024-09-13T10:10:56.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4873000000000001e-05 2024-09-13T10:10:56.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.327927+0000", 2024-09-13T10:10:56.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3903000000000001e-05 2024-09-13T10:10:56.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.328042+0000", 2024-09-13T10:10:56.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115707 2024-09-13T10:10:56.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.328086+0000", 2024-09-13T10:10:56.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4060000000000002e-05 2024-09-13T10:10:56.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.329115+0000", 2024-09-13T10:10:56.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001029056 2024-09-13T10:10:56.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.329181+0000", 2024-09-13T10:10:56.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6132999999999999e-05 2024-09-13T10:10:56.868 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.868 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.869 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.869 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:440 4.d e39/31 4:b19b932a:::smithi19530714-41:head v 39'12, mlcod=38'10)", 2024-09-13T10:10:56.869 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.341544+0000", 2024-09-13T10:10:56.869 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.4681864979999997, 2024-09-13T10:10:56.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001790916, 2024-09-13T10:10:56.869 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.869 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.869 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.869 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.341544+0000", 2024-09-13T10:10:56.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.341544+0000", 2024-09-13T10:10:56.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.341546+0000", 2024-09-13T10:10:56.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6789999999999999e-06 2024-09-13T10:10:56.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.341559+0000", 2024-09-13T10:10:56.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3325e-05 2024-09-13T10:10:56.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.341588+0000", 2024-09-13T10:10:56.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8674e-05 2024-09-13T10:10:56.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.341605+0000", 2024-09-13T10:10:56.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6642000000000001e-05 2024-09-13T10:10:56.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.341776+0000", 2024-09-13T10:10:56.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017120500000000001 2024-09-13T10:10:56.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.341824+0000", 2024-09-13T10:10:56.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8291000000000002e-05 2024-09-13T10:10:56.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.343248+0000", 2024-09-13T10:10:56.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0014242199999999999 2024-09-13T10:10:56.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.343335+0000", 2024-09-13T10:10:56.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6879999999999995e-05 2024-09-13T10:10:56.873 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.873 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.873 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.874 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:595 4.d e39/31 4:ba7d2bc5:::smithi19530714-46:head v 39'13, mlcod=39'11)", 2024-09-13T10:10:56.874 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.376826+0000", 2024-09-13T10:10:56.874 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.432904626, 2024-09-13T10:10:56.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00091676300000000002, 2024-09-13T10:10:56.874 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.874 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.874 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.874 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.376826+0000", 2024-09-13T10:10:56.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.376826+0000", 2024-09-13T10:10:56.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.376828+0000", 2024-09-13T10:10:56.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5230000000000001e-06 2024-09-13T10:10:56.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.376838+0000", 2024-09-13T10:10:56.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0013999999999999e-05 2024-09-13T10:10:56.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.376857+0000", 2024-09-13T10:10:56.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9215000000000002e-05 2024-09-13T10:10:56.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.376868+0000", 2024-09-13T10:10:56.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1336e-05 2024-09-13T10:10:56.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.376978+0000", 2024-09-13T10:10:56.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000110102 2024-09-13T10:10:56.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.377020+0000", 2024-09-13T10:10:56.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1873999999999999e-05 2024-09-13T10:10:56.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.377692+0000", 2024-09-13T10:10:56.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00067159200000000004 2024-09-13T10:10:56.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.377743+0000", 2024-09-13T10:10:56.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1106999999999997e-05 2024-09-13T10:10:56.878 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.878 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.878 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.878 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:645 4.d e39/31 4:ba7d2bc5:::smithi19530714-46:head v 39'14, mlcod=39'12)", 2024-09-13T10:10:56.879 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.386823+0000", 2024-09-13T10:10:56.879 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.4229079559999995, 2024-09-13T10:10:56.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00089980300000000002, 2024-09-13T10:10:56.879 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.879 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.879 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.879 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.386823+0000", 2024-09-13T10:10:56.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.386823+0000", 2024-09-13T10:10:56.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.386825+0000", 2024-09-13T10:10:56.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.793e-06 2024-09-13T10:10:56.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.386832+0000", 2024-09-13T10:10:56.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6340000000000001e-06 2024-09-13T10:10:56.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.386850+0000", 2024-09-13T10:10:56.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7611000000000001e-05 2024-09-13T10:10:56.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.386869+0000", 2024-09-13T10:10:56.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9099000000000001e-05 2024-09-13T10:10:56.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.386972+0000", 2024-09-13T10:10:56.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000103037 2024-09-13T10:10:56.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.387013+0000", 2024-09-13T10:10:56.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0831000000000003e-05 2024-09-13T10:10:56.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.387667+0000", 2024-09-13T10:10:56.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00065462400000000003 2024-09-13T10:10:56.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.387723+0000", 2024-09-13T10:10:56.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5173999999999997e-05 2024-09-13T10:10:56.883 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.883 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.883 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.883 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:786 4.d e39/31 4:b19b932a:::smithi19530714-41:head v 39'15, mlcod=39'13)", 2024-09-13T10:10:56.883 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.620217+0000", 2024-09-13T10:10:56.883 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.1895132899999998, 2024-09-13T10:10:56.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033766159999999998, 2024-09-13T10:10:56.884 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.884 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.884 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.884 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.620217+0000", 2024-09-13T10:10:56.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.620217+0000", 2024-09-13T10:10:56.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.620220+0000", 2024-09-13T10:10:56.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6230000000000001e-06 2024-09-13T10:10:56.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.621490+0000", 2024-09-13T10:10:56.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0012697019999999999 2024-09-13T10:10:56.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.621531+0000", 2024-09-13T10:10:56.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1205999999999998e-05 2024-09-13T10:10:56.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.621552+0000", 2024-09-13T10:10:56.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0948999999999999e-05 2024-09-13T10:10:56.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.621651+0000", 2024-09-13T10:10:56.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9482999999999996e-05 2024-09-13T10:10:56.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.621676+0000", 2024-09-13T10:10:56.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4570999999999999e-05 2024-09-13T10:10:56.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.623532+0000", 2024-09-13T10:10:56.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001856218 2024-09-13T10:10:56.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.623594+0000", 2024-09-13T10:10:56.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1864000000000003e-05 2024-09-13T10:10:56.888 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.888 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.888 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.888 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:787 4.d e39/31 4:b19b932a:::smithi19530714-41:head v 39'16, mlcod=39'14)", 2024-09-13T10:10:56.888 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.628023+0000", 2024-09-13T10:10:56.888 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.1817077989999998, 2024-09-13T10:10:56.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002858612, 2024-09-13T10:10:56.888 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.888 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.889 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.889 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.889 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.628023+0000", 2024-09-13T10:10:56.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.889 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.628023+0000", 2024-09-13T10:10:56.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.628025+0000", 2024-09-13T10:10:56.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9800000000000001e-06 2024-09-13T10:10:56.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.628754+0000", 2024-09-13T10:10:56.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00072891500000000001 2024-09-13T10:10:56.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.628773+0000", 2024-09-13T10:10:56.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8719000000000002e-05 2024-09-13T10:10:56.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.628782+0000", 2024-09-13T10:10:56.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6860000000000001e-06 2024-09-13T10:10:56.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.628902+0000", 2024-09-13T10:10:56.893 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120214 2024-09-13T10:10:56.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.628966+0000", 2024-09-13T10:10:56.893 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3561999999999999e-05 2024-09-13T10:10:56.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.630810+0000", 2024-09-13T10:10:56.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0018439050000000001 2024-09-13T10:10:56.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.894 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.630882+0000", 2024-09-13T10:10:56.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1631000000000006e-05 2024-09-13T10:10:56.894 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.894 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.894 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.894 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:788 4.d e39/31 4:b19b932a:::smithi19530714-41:head v 39'17, mlcod=39'15)", 2024-09-13T10:10:56.895 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.634939+0000", 2024-09-13T10:10:56.895 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.1747920010000001, 2024-09-13T10:10:56.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027623980000000001, 2024-09-13T10:10:56.895 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.895 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.895 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.895 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.634939+0000", 2024-09-13T10:10:56.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.634939+0000", 2024-09-13T10:10:56.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.634955+0000", 2024-09-13T10:10:56.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6209999999999999e-05 2024-09-13T10:10:56.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.635351+0000", 2024-09-13T10:10:56.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000396225 2024-09-13T10:10:56.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.635378+0000", 2024-09-13T10:10:56.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6562000000000001e-05 2024-09-13T10:10:56.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.635397+0000", 2024-09-13T10:10:56.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8836999999999999e-05 2024-09-13T10:10:56.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.898 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.635476+0000", 2024-09-13T10:10:56.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9122999999999994e-05 2024-09-13T10:10:56.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.898 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.635492+0000", 2024-09-13T10:10:56.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6158e-05 2024-09-13T10:10:56.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.637653+0000", 2024-09-13T10:10:56.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021612160000000001 2024-09-13T10:10:56.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.637701+0000", 2024-09-13T10:10:56.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8066999999999999e-05 2024-09-13T10:10:56.899 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.899 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.899 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.899 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:789 4.d e39/31 4:b19b932a:::smithi19530714-41:head v 39'18, mlcod=39'15)", 2024-09-13T10:10:56.900 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.636020+0000", 2024-09-13T10:10:56.900 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.17371085, 2024-09-13T10:10:56.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0022949960000000001, 2024-09-13T10:10:56.900 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.900 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.900 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.900 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.636020+0000", 2024-09-13T10:10:56.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.636020+0000", 2024-09-13T10:10:56.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.636021+0000", 2024-09-13T10:10:56.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9599999999999998e-07 2024-09-13T10:10:56.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.636028+0000", 2024-09-13T10:10:56.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3329999999999999e-06 2024-09-13T10:10:56.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.636038+0000", 2024-09-13T10:10:56.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0141e-05 2024-09-13T10:10:56.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.636046+0000", 2024-09-13T10:10:56.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3119999999999996e-06 2024-09-13T10:10:56.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.636731+0000", 2024-09-13T10:10:56.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000685168 2024-09-13T10:10:56.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.636784+0000", 2024-09-13T10:10:56.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3600999999999997e-05 2024-09-13T10:10:56.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.638211+0000", 2024-09-13T10:10:56.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001426681 2024-09-13T10:10:56.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.638315+0000", 2024-09-13T10:10:56.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010386399999999999 2024-09-13T10:10:56.904 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.904 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.904 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.904 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:1018 4.d e42/31 4:ba7d2bc5:::smithi19530714-46:head v 42'22, mlcod=41'20)", 2024-09-13T10:10:56.905 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:55.673611+0000", 2024-09-13T10:10:56.905 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.1361192579999999, 2024-09-13T10:10:56.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00097575299999999999, 2024-09-13T10:10:56.905 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.905 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.905 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:56.905 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:55.673611+0000", 2024-09-13T10:10:56.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:55.673611+0000", 2024-09-13T10:10:56.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:55.673612+0000", 2024-09-13T10:10:56.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5400000000000002e-07 2024-09-13T10:10:56.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:55.673618+0000", 2024-09-13T10:10:56.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3180000000000003e-06 2024-09-13T10:10:56.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:55.673639+0000", 2024-09-13T10:10:56.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0998000000000001e-05 2024-09-13T10:10:56.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:55.673650+0000", 2024-09-13T10:10:56.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1046999999999999e-05 2024-09-13T10:10:56.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:55.673709+0000", 2024-09-13T10:10:56.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9058999999999998e-05 2024-09-13T10:10:56.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:10:56.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:55.673739+0000", 2024-09-13T10:10:56.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9476999999999999e-05 2024-09-13T10:10:56.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:10:56.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:55.674541+0000", 2024-09-13T10:10:56.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00080267900000000004 2024-09-13T10:10:56.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.909 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:55.674587+0000", 2024-09-13T10:10:56.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5921000000000003e-05 2024-09-13T10:10:56.909 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.909 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.909 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.909 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "pg_update_log_missing(4.d epoch 42/31 rep_tid 57 entries 42'23 (0'0) error 4:b19b932a:::smithi19530714-41:head by client.4607.0:1041 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 41'20 roll_forward_to 41'21)", 2024-09-13T10:10:56.909 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.163783+0000", 2024-09-13T10:10:56.910 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.64594793800000005, 2024-09-13T10:10:56.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010082209999999999, 2024-09-13T10:10:56.910 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:10:56.910 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:10:56.910 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "reached pg", 2024-09-13T10:10:56.910 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:10:56.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:10:56.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.163783+0000", 2024-09-13T10:10:56.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:10:56.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.163783+0000", 2024-09-13T10:10:56.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:10:56.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:10:56.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.163785+0000", 2024-09-13T10:10:56.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.796e-06 2024-09-13T10:10:56.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:10:56.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.163797+0000", 2024-09-13T10:10:56.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1973999999999999e-05 2024-09-13T10:10:56.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:10:56.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.163825+0000", 2024-09-13T10:10:56.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7991999999999999e-05 2024-09-13T10:10:56.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:10:56.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.163839+0000", 2024-09-13T10:10:56.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4558e-05 2024-09-13T10:10:56.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:10:56.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.163981+0000", 2024-09-13T10:10:56.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014152899999999999 2024-09-13T10:10:56.913 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:10:56.913 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:10:56.913 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:10:56.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.164791+0000", 2024-09-13T10:10:56.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00081037199999999996 2024-09-13T10:10:56.913 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.913 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.913 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.913 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:10:56.913 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:10:56.913 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:10:56.914 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:10:56.915 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:56.916 INFO:tasks.rados.rados.0.smithi195.stdout:709: finishing write tid 1 to smithi19530714-1 2024-09-13T10:10:56.916 INFO:tasks.rados.rados.0.smithi195.stdout:709: finishing write tid 2 to smithi19530714-1 2024-09-13T10:10:56.916 INFO:tasks.rados.rados.0.smithi195.stdout:709: finishing write tid 3 to smithi19530714-1 2024-09-13T10:10:56.916 INFO:tasks.rados.rados.0.smithi195.stdout:710: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-13T10:10:56.916 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T10:10:56.917 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:56.911+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:56.921 INFO:tasks.rados.rados.0.smithi195.stdout:709: finishing write tid 4 to smithi19530714-1 2024-09-13T10:10:56.921 INFO:tasks.rados.rados.0.smithi195.stdout:712: finishing set_chunk to oid 20 2024-09-13T10:10:56.921 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 20 set_chunk returned error code -95 offset: 929280 length: 245248 2024-09-13T10:10:56.921 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 0 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-13T10:10:56.922 INFO:tasks.rados.rados.0.smithi195.stdout:709: finishing write tid 5 to smithi19530714-1 2024-09-13T10:10:56.922 INFO:tasks.rados.rados.0.smithi195.stdout:709: oid 1 updating version 0 to 22 2024-09-13T10:10:56.922 INFO:tasks.rados.rados.0.smithi195.stdout:709: oid 1 updating version 22 to 24 2024-09-13T10:10:56.922 INFO:tasks.rados.rados.0.smithi195.stdout:709: oid 1 version 24 is already newer than 23 2024-09-13T10:10:56.922 INFO:tasks.rados.rados.0.smithi195.stdout:709: oid 1 updating version 24 to 25 2024-09-13T10:10:56.922 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 25 (ObjNum 81 snap 0 seq_num 81) dirty exists 2024-09-13T10:10:56.922 INFO:tasks.rados.rados.0.smithi195.stdout:709: left oid 1 (ObjNum 81 snap 0 seq_num 81) 2024-09-13T10:10:56.922 INFO:tasks.rados.rados.0.smithi195.stdout:711: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-13T10:10:56.922 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:56.945 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:56.994 INFO:tasks.rados.rados.0.smithi195.stdout:717: expect (ObjNum 42 snap 0 seq_num 42) 2024-09-13T10:10:57.015 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:10:57.013+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:10:57.022 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:57.045 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:10:57.045 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:10:57.045 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:10:57.045 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:10:57.060 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T10:10:57.069 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:10:57.069 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:10:57.075 INFO:tasks.rados.rados.0.smithi195.stdout:718: expect (ObjNum 72 snap 0 seq_num 72) 2024-09-13T10:10:57.090 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:10:57.090 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:10:57.111 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:10:57.111 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:10:57.116 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:57.116+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:10:57.131 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:10:57.131 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:10:57.148 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:10:57.148 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:10:57.169 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:10:57.169 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:10:57.183 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:57.183 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:57.189 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:10:57.190 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:10:57.194 INFO:tasks.rados.rados.0.smithi195.stdout:714: expect (ObjNum 59 snap 0 seq_num 59) 2024-09-13T10:10:57.197 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T10:10:57.210 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:10:57.210 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:10:57.219 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2639457 2024-09-13T10:10:57.271 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:10:57.272 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:10:57.272 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:57.225+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:10:57.272 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:10:57.273 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:10:57.273 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:10:57.273 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:10:57.283 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2383381 2024-09-13T10:10:57.290 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:10:57.290 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:10:57.292 INFO:tasks.rados.rados.0.smithi195.stdout:713: got (0) Success 2024-09-13T10:10:57.293 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 15 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-09-13T10:10:57.293 INFO:tasks.rados.rados.0.smithi195.stdout:715: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-13T10:10:57.293 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:57.308 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:10:57.308 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:10:57.308 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:10:57.308 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:10:57.308 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:10:57.309 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T10:10:57.309 INFO:tasks.thrashosds.thrasher:Setting osd 12 primary_affinity to 0.000000 2024-09-13T10:10:57.309 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 12 0 2024-09-13T10:10:57.318 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:57.315+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:57.319 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:57.325 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:57.325 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:57.330 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:10:57.330 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:10:57.331 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-13T10:10:57.344 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T10:10:57.352 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:10:57.353 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:10:57.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:10:57.353+0000 7fd250c90640 0 -- 172.21.15.73:0/2135396849 >> v2:172.21.15.195:6805/848686366 conn(0x7fd24c227ae0 msgr2=0x7fd24c229ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:10:57.368 INFO:tasks.rados.rados.0.smithi195.stdout:716: expect (ObjNum 58 snap 0 seq_num 58) 2024-09-13T10:10:57.368 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:57.376 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:10:57.376 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:10:57.409 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:57.418 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:57.415+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:57.480 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:57.480 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:57.489 INFO:tasks.rados.rados.0.smithi195.stdout:719: finishing set_chunk to oid 11 2024-09-13T10:10:57.489 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 21 (ObjNum 76 snap 0 seq_num 76) dirty exists 2024-09-13T10:10:57.489 INFO:tasks.rados.rados.0.smithi195.stdout:720: finishing write tid 1 to smithi19530714-32 2024-09-13T10:10:57.489 INFO:tasks.rados.rados.0.smithi195.stdout:720: finishing write tid 2 to smithi19530714-32 2024-09-13T10:10:57.489 INFO:tasks.rados.rados.0.smithi195.stdout:720: finishing write tid 3 to smithi19530714-32 2024-09-13T10:10:57.489 INFO:tasks.rados.rados.0.smithi195.stdout:720: finishing write tid 4 to smithi19530714-32 2024-09-13T10:10:57.489 INFO:tasks.rados.rados.0.smithi195.stdout:720: finishing write tid 5 to smithi19530714-32 2024-09-13T10:10:57.489 INFO:tasks.rados.rados.0.smithi195.stdout:720: oid 32 updating version 0 to 35 2024-09-13T10:10:57.490 INFO:tasks.rados.rados.0.smithi195.stdout:720: oid 32 version 35 is already newer than 34 2024-09-13T10:10:57.490 INFO:tasks.rados.rados.0.smithi195.stdout:720: oid 32 updating version 35 to 36 2024-09-13T10:10:57.490 INFO:tasks.rados.rados.0.smithi195.stdout:720: oid 32 updating version 36 to 37 2024-09-13T10:10:57.490 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 37 (ObjNum 82 snap 0 seq_num 82) dirty exists 2024-09-13T10:10:57.490 INFO:tasks.rados.rados.0.smithi195.stdout:720: left oid 32 (ObjNum 82 snap 0 seq_num 82) 2024-09-13T10:10:57.490 INFO:tasks.rados.rados.0.smithi195.stdout:704: done (15 left) 2024-09-13T10:10:57.490 INFO:tasks.rados.rados.0.smithi195.stdout:705: done (14 left) 2024-09-13T10:10:57.490 INFO:tasks.rados.rados.0.smithi195.stdout:706: done (13 left) 2024-09-13T10:10:57.490 INFO:tasks.rados.rados.0.smithi195.stdout:708: done (12 left) 2024-09-13T10:10:57.490 INFO:tasks.rados.rados.0.smithi195.stdout:709: done (11 left) 2024-09-13T10:10:57.490 INFO:tasks.rados.rados.0.smithi195.stdout:710: done (10 left) 2024-09-13T10:10:57.490 INFO:tasks.rados.rados.0.smithi195.stdout:711: done (9 left) 2024-09-13T10:10:57.490 INFO:tasks.rados.rados.0.smithi195.stdout:712: done (8 left) 2024-09-13T10:10:57.491 INFO:tasks.rados.rados.0.smithi195.stdout:713: done (7 left) 2024-09-13T10:10:57.491 INFO:tasks.rados.rados.0.smithi195.stdout:714: done (6 left) 2024-09-13T10:10:57.491 INFO:tasks.rados.rados.0.smithi195.stdout:715: done (5 left) 2024-09-13T10:10:57.491 INFO:tasks.rados.rados.0.smithi195.stdout:716: done (4 left) 2024-09-13T10:10:57.491 INFO:tasks.rados.rados.0.smithi195.stdout:717: done (3 left) 2024-09-13T10:10:57.491 INFO:tasks.rados.rados.0.smithi195.stdout:718: done (2 left) 2024-09-13T10:10:57.491 INFO:tasks.rados.rados.0.smithi195.stdout:719: done (1 left) 2024-09-13T10:10:57.491 INFO:tasks.rados.rados.0.smithi195.stdout:720: done (0 left) 2024-09-13T10:10:57.491 INFO:tasks.rados.rados.0.smithi195.stdout:721: read oid 14 snap -1 2024-09-13T10:10:57.491 INFO:tasks.rados.rados.0.smithi195.stdout:721: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-13T10:10:57.491 INFO:tasks.rados.rados.0.smithi195.stdout:722: set_chunk oid 12 target oid 2024-09-13T10:10:57.495 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T10:10:57.508 INFO:tasks.rados.rados.0.smithi195.stdout:722: set_chunk oid 12 offset: 1584128 length: 306176 target oid 29336af06cde79beee061ee52221a9c2e1abb54d64324fe1a6837e94422b2303 offset: 0 snap -1 2024-09-13T10:10:57.508 INFO:tasks.rados.rados.0.smithi195.stdout:723: read oid 22 snap -1 2024-09-13T10:10:57.509 INFO:tasks.rados.rados.0.smithi195.stdout:723: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-13T10:10:57.509 INFO:tasks.rados.rados.0.smithi195.stdout:724: delete oid 23 current snap is 0 2024-09-13T10:10:57.509 INFO:tasks.rados.rados.0.smithi195.stdout:721: expect (ObjNum 13 snap 0 seq_num 13) 2024-09-13T10:10:57.519 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:57.519+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:57.541 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:57.572 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 592083 2024-09-13T10:10:57.594 INFO:tasks.rados.rados.0.smithi195.stdout:722: finishing set_chunk to oid 12 2024-09-13T10:10:57.594 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 12 set_chunk returned error code -95 offset: 1584128 length: 306176 2024-09-13T10:10:57.594 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 0 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-09-13T10:10:57.594 INFO:tasks.rados.rados.0.smithi195.stdout:723: expect (ObjNum 21 snap 0 seq_num 21) 2024-09-13T10:10:57.602 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:57.602 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:57.615 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T10:10:57.620 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:57.619+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:57.622 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2207794 2024-09-13T10:10:57.649 INFO:tasks.rados.rados.0.smithi195.stdout:721: done (3 left) 2024-09-13T10:10:57.649 INFO:tasks.rados.rados.0.smithi195.stdout:722: done (2 left) 2024-09-13T10:10:57.649 INFO:tasks.rados.rados.0.smithi195.stdout:723: done (1 left) 2024-09-13T10:10:57.649 INFO:tasks.rados.rados.0.smithi195.stdout:724: done (0 left) 2024-09-13T10:10:57.649 INFO:tasks.rados.rados.0.smithi195.stdout:725: read oid 24 snap -1 2024-09-13T10:10:57.649 INFO:tasks.rados.rados.0.smithi195.stdout:725: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-13T10:10:57.649 INFO:tasks.rados.rados.0.smithi195.stdout:726: write (excl) oid 10 current snap is 0 2024-09-13T10:10:57.650 INFO:tasks.rados.rados.0.smithi195.stdout:726: seq_num 84 ranges {611788=400799,1748958=488686,2907296=504492} 2024-09-13T10:10:57.662 INFO:tasks.rados.rados.0.smithi195.stdout:726: writing smithi19530714-10 from 611788 to 1012587 tid 1 2024-09-13T10:10:57.669 INFO:tasks.rados.rados.0.smithi195.stdout:726: writing smithi19530714-10 from 1748958 to 2237644 tid 2 2024-09-13T10:10:57.686 INFO:tasks.rados.rados.0.smithi195.stdout:726: writing smithi19530714-10 from 2907296 to 3411788 tid 3 2024-09-13T10:10:57.686 INFO:tasks.rados.rados.0.smithi195.stdout:727: read oid 27 snap -1 2024-09-13T10:10:57.686 INFO:tasks.rados.rados.0.smithi195.stdout:727: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-13T10:10:57.686 INFO:tasks.rados.rados.0.smithi195.stdout:728: tier_evict oid 50 2024-09-13T10:10:57.687 INFO:tasks.rados.rados.0.smithi195.stdout:728: tier_evict oid 50 snap -1 2024-09-13T10:10:57.687 INFO:tasks.rados.rados.0.smithi195.stdout:725: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-13T10:10:57.687 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:57.713 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:57.721 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:10:57.719+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:10:57.740 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:57.740 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:57.753 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-13T10:10:57.754 INFO:tasks.rados.rados.0.smithi195.stdout:726: finishing write tid 1 to smithi19530714-10 2024-09-13T10:10:57.754 INFO:tasks.rados.rados.0.smithi195.stdout:726: finishing write tid 2 to smithi19530714-10 2024-09-13T10:10:57.754 INFO:tasks.rados.rados.0.smithi195.stdout:728: got (22) Invalid argument 2024-09-13T10:10:57.754 INFO:tasks.rados.rados.0.smithi195.stdout:726: finishing write tid 3 to smithi19530714-10 2024-09-13T10:10:57.754 INFO:tasks.rados.rados.0.smithi195.stdout:725: done (3 left) 2024-09-13T10:10:57.755 INFO:tasks.rados.rados.0.smithi195.stdout:728: done (2 left) 2024-09-13T10:10:57.755 INFO:tasks.rados.rados.0.smithi195.stdout:729: write (excl) oid 12 current snap is 0 2024-09-13T10:10:57.755 INFO:tasks.rados.rados.0.smithi195.stdout:729: seq_num 85 ranges {628595=476048,1807440=632344,2934719=452474} 2024-09-13T10:10:57.759 INFO:tasks.rados.rados.0.smithi195.stdout:729: writing smithi19530714-12 from 628595 to 1104643 tid 1 2024-09-13T10:10:57.768 INFO:tasks.rados.rados.0.smithi195.stdout:729: writing smithi19530714-12 from 1807440 to 2439784 tid 2 2024-09-13T10:10:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:729: writing smithi19530714-12 from 2934719 to 3387193 tid 3 2024-09-13T10:10:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:730: tier_evict oid 45 2024-09-13T10:10:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:730: tier_evict oid 45 snap -1 2024-09-13T10:10:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:726: finishing write tid 4 to smithi19530714-10 2024-09-13T10:10:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:726: finishing write tid 5 to smithi19530714-10 2024-09-13T10:10:57.775 INFO:tasks.rados.rados.0.smithi195.stdout:726: oid 10 updating version 0 to 15 2024-09-13T10:10:57.775 INFO:tasks.rados.rados.0.smithi195.stdout:726: oid 10 version 15 is already newer than 14 2024-09-13T10:10:57.775 INFO:tasks.rados.rados.0.smithi195.stdout:726: oid 10 updating version 15 to 17 2024-09-13T10:10:57.775 INFO:tasks.rados.rados.0.smithi195.stdout:726: oid 10 version 17 is already newer than 16 2024-09-13T10:10:57.775 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 17 (ObjNum 84 snap 0 seq_num 84) dirty exists 2024-09-13T10:10:57.775 INFO:tasks.rados.rados.0.smithi195.stdout:726: left oid 10 (ObjNum 84 snap 0 seq_num 84) 2024-09-13T10:10:57.775 INFO:tasks.rados.rados.0.smithi195.stdout:727: expect (ObjNum 26 snap 0 seq_num 26) 2024-09-13T10:10:57.775 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:57.823 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:10:57.820+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:10:57.824 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1660860 2024-09-13T10:10:57.837 INFO:tasks.rados.rados.0.smithi195.stdout:729: finishing write tid 1 to smithi19530714-12 2024-09-13T10:10:57.838 INFO:tasks.rados.rados.0.smithi195.stdout:729: finishing write tid 2 to smithi19530714-12 2024-09-13T10:10:57.838 INFO:tasks.rados.rados.0.smithi195.stdout:730: got (22) Invalid argument 2024-09-13T10:10:57.838 INFO:tasks.rados.rados.0.smithi195.stdout:729: finishing write tid 3 to smithi19530714-12 2024-09-13T10:10:57.838 INFO:tasks.rados.rados.0.smithi195.stdout:729: finishing write tid 4 to smithi19530714-12 2024-09-13T10:10:57.838 INFO:tasks.rados.rados.0.smithi195.stdout:729: finishing write tid 5 to smithi19530714-12 2024-09-13T10:10:57.838 INFO:tasks.rados.rados.0.smithi195.stdout:729: oid 12 updating version 0 to 37 2024-09-13T10:10:57.838 INFO:tasks.rados.rados.0.smithi195.stdout:729: oid 12 updating version 37 to 39 2024-09-13T10:10:57.838 INFO:tasks.rados.rados.0.smithi195.stdout:729: oid 12 version 39 is already newer than 38 2024-09-13T10:10:57.838 INFO:tasks.rados.rados.0.smithi195.stdout:729: oid 12 updating version 39 to 40 2024-09-13T10:10:57.838 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 40 (ObjNum 85 snap 0 seq_num 85) dirty exists 2024-09-13T10:10:57.838 INFO:tasks.rados.rados.0.smithi195.stdout:729: left oid 12 (ObjNum 85 snap 0 seq_num 85) 2024-09-13T10:10:57.839 INFO:tasks.rados.rados.0.smithi195.stdout:726: done (3 left) 2024-09-13T10:10:57.839 INFO:tasks.rados.rados.0.smithi195.stdout:727: done (2 left) 2024-09-13T10:10:57.839 INFO:tasks.rados.rados.0.smithi195.stdout:729: done (1 left) 2024-09-13T10:10:57.839 INFO:tasks.rados.rados.0.smithi195.stdout:730: done (0 left) 2024-09-13T10:10:57.839 INFO:tasks.rados.rados.0.smithi195.stdout:731: write oid 35 current snap is 0 2024-09-13T10:10:57.839 INFO:tasks.rados.rados.0.smithi195.stdout:731: seq_num 86 ranges {645402=551297,1749569=492355,2845789=599613} 2024-09-13T10:10:57.850 INFO:tasks.rados.rados.0.smithi195.stdout:731: writing smithi19530714-35 from 645402 to 1196699 tid 1 2024-09-13T10:10:57.862 INFO:tasks.rados.rados.0.smithi195.stdout:731: writing smithi19530714-35 from 1749569 to 2241924 tid 2 2024-09-13T10:10:57.871 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:57.872 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:57.876 INFO:tasks.rados.rados.0.smithi195.stdout:731: writing smithi19530714-35 from 2845789 to 3445402 tid 3 2024-09-13T10:10:57.876 INFO:tasks.rados.rados.0.smithi195.stdout:732: delete oid 29 current snap is 0 2024-09-13T10:10:57.876 INFO:tasks.rados.rados.0.smithi195.stdout:731: finishing write tid 1 to smithi19530714-35 2024-09-13T10:10:57.877 INFO:tasks.rados.rados.0.smithi195.stdout:731: finishing write tid 2 to smithi19530714-35 2024-09-13T10:10:57.879 INFO:tasks.rados.rados.0.smithi195.stdout:732: done (1 left) 2024-09-13T10:10:57.879 INFO:tasks.rados.rados.0.smithi195.stdout:733: set_chunk oid 6 target oid 2024-09-13T10:10:57.886 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-13T10:10:57.899 INFO:tasks.rados.rados.0.smithi195.stdout:733: set_chunk oid 6 offset: 28672 length: 1549312 target oid ab11fa1c21155891c578c4dad858e1de5d60630e21a626e631b1f95681af7355 offset: 0 snap -1 2024-09-13T10:10:57.899 INFO:tasks.rados.rados.0.smithi195.stdout:734: write oid 11 current snap is 0 2024-09-13T10:10:57.899 INFO:tasks.rados.rados.0.smithi195.stdout:734: seq_num 88 ranges {679016=701795,1866533=496024,2900635=402643} 2024-09-13T10:10:57.914 INFO:tasks.rados.rados.0.smithi195.stdout:734: writing smithi19530714-11 from 679016 to 1380811 tid 1 2024-09-13T10:10:57.923 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:57.920+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:10:57.926 INFO:tasks.rados.rados.0.smithi195.stdout:734: writing smithi19530714-11 from 1866533 to 2362557 tid 2 2024-09-13T10:10:57.937 INFO:tasks.rados.rados.0.smithi195.stdout:734: writing smithi19530714-11 from 2900635 to 3303278 tid 3 2024-09-13T10:10:57.937 INFO:tasks.rados.rados.0.smithi195.stdout:735: read oid 39 snap -1 2024-09-13T10:10:57.937 INFO:tasks.rados.rados.0.smithi195.stdout:735: expect (ObjNum 38 snap 0 seq_num 38) 2024-09-13T10:10:57.937 INFO:tasks.rados.rados.0.smithi195.stdout:736: set_chunk oid 30 target oid 2024-09-13T10:10:57.937 INFO:tasks.rados.rados.0.smithi195.stdout:736: set_chunk oid 30 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:10:57.937 INFO:tasks.rados.rados.0.smithi195.stdout:737: tier_evict oid 28 2024-09-13T10:10:57.937 INFO:tasks.rados.rados.0.smithi195.stdout:737: tier_evict oid 28 snap -1 2024-09-13T10:10:57.937 INFO:tasks.rados.rados.0.smithi195.stdout:731: finishing write tid 3 to smithi19530714-35 2024-09-13T10:10:57.937 INFO:tasks.rados.rados.0.smithi195.stdout:731: finishing write tid 4 to smithi19530714-35 2024-09-13T10:10:57.938 INFO:tasks.rados.rados.0.smithi195.stdout:731: finishing write tid 5 to smithi19530714-35 2024-09-13T10:10:57.938 INFO:tasks.rados.rados.0.smithi195.stdout:731: oid 35 updating version 0 to 40 2024-09-13T10:10:57.938 INFO:tasks.rados.rados.0.smithi195.stdout:731: oid 35 version 40 is already newer than 39 2024-09-13T10:10:57.938 INFO:tasks.rados.rados.0.smithi195.stdout:731: oid 35 updating version 40 to 41 2024-09-13T10:10:57.938 INFO:tasks.rados.rados.0.smithi195.stdout:731: oid 35 version 41 is already newer than 38 2024-09-13T10:10:57.938 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 41 (ObjNum 86 snap 0 seq_num 86) dirty exists 2024-09-13T10:10:57.938 INFO:tasks.rados.rados.0.smithi195.stdout:731: left oid 35 (ObjNum 86 snap 0 seq_num 86) 2024-09-13T10:10:57.938 INFO:tasks.rados.rados.0.smithi195.stdout:733: finishing set_chunk to oid 6 2024-09-13T10:10:57.938 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 6 set_chunk returned error code -95 offset: 28672 length: 1549312 2024-09-13T10:10:57.938 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 0 (ObjNum 79 snap 0 seq_num 79) dirty exists 2024-09-13T10:10:57.938 INFO:tasks.rados.rados.0.smithi195.stdout:731: done (5 left) 2024-09-13T10:10:57.938 INFO:tasks.rados.rados.0.smithi195.stdout:733: done (4 left) 2024-09-13T10:10:57.939 INFO:tasks.rados.rados.0.smithi195.stdout:738: read oid 5 snap -1 2024-09-13T10:10:57.939 INFO:tasks.rados.rados.0.smithi195.stdout:738: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-13T10:10:57.939 INFO:tasks.rados.rados.0.smithi195.stdout:739: read oid 25 snap -1 2024-09-13T10:10:57.939 INFO:tasks.rados.rados.0.smithi195.stdout:739: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-13T10:10:57.939 INFO:tasks.rados.rados.0.smithi195.stdout:740: read oid 44 snap -1 2024-09-13T10:10:57.939 INFO:tasks.rados.rados.0.smithi195.stdout:740: expect (ObjNum 72 snap 0 seq_num 72) 2024-09-13T10:10:57.939 INFO:tasks.rados.rados.0.smithi195.stdout:741: read oid 38 snap -1 2024-09-13T10:10:57.939 INFO:tasks.rados.rados.0.smithi195.stdout:741: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:10:57.939 INFO:tasks.rados.rados.0.smithi195.stdout:742: tier_evict oid 27 2024-09-13T10:10:57.939 INFO:tasks.rados.rados.0.smithi195.stdout:742: tier_evict oid 27 snap -1 2024-09-13T10:10:57.939 INFO:tasks.rados.rados.0.smithi195.stdout:734: finishing write tid 1 to smithi19530714-11 2024-09-13T10:10:57.939 INFO:tasks.rados.rados.0.smithi195.stdout:734: finishing write tid 2 to smithi19530714-11 2024-09-13T10:10:57.939 INFO:tasks.rados.rados.0.smithi195.stdout:743: set_chunk oid 13 target oid 2024-09-13T10:10:57.962 INFO:tasks.rados.rados.0.smithi195.stdout:743: set_chunk oid 13 offset: 172032 length: 474624 target oid 077249b8c0235e9fb37395668d6800145a737f76badf21df47c92f500d86fa5a offset: 0 snap -1 2024-09-13T10:10:57.962 INFO:tasks.rados.rados.0.smithi195.stdout:744: set_chunk oid 19 target oid 2024-09-13T10:10:57.977 INFO:tasks.rados.rados.0.smithi195.stdout:744: set_chunk oid 19 offset: 728064 length: 951808 target oid b9a9147199c386be35108f31c9bba5e89b7805810677d7a231bd60469282d7c2 offset: 0 snap -1 2024-09-13T10:10:57.977 INFO:tasks.rados.rados.0.smithi195.stdout:745: tier_promote oid 23 2024-09-13T10:10:57.977 INFO:tasks.rados.rados.0.smithi195.stdout:736: finishing set_chunk to oid 30 2024-09-13T10:10:57.977 INFO:tasks.rados.rados.0.smithi195.stdout:736: got expected ENOENT (src dne) 2024-09-13T10:10:57.977 INFO:tasks.rados.rados.0.smithi195.stdout:737: got (22) Invalid argument 2024-09-13T10:10:57.977 INFO:tasks.rados.rados.0.smithi195.stdout:742: got (22) Invalid argument 2024-09-13T10:10:57.978 INFO:tasks.rados.rados.0.smithi195.stdout:736: done (11 left) 2024-09-13T10:10:57.978 INFO:tasks.rados.rados.0.smithi195.stdout:737: done (10 left) 2024-09-13T10:10:57.978 INFO:tasks.rados.rados.0.smithi195.stdout:742: done (9 left) 2024-09-13T10:10:57.978 INFO:tasks.rados.rados.0.smithi195.stdout:746: read oid 33 snap -1 2024-09-13T10:10:57.978 INFO:tasks.rados.rados.0.smithi195.stdout:746: expect (ObjNum 64 snap 0 seq_num 64) 2024-09-13T10:10:57.978 INFO:tasks.rados.rados.0.smithi195.stdout:747: write (excl) oid 15 current snap is 0 2024-09-13T10:10:57.978 INFO:tasks.rados.rados.0.smithi195.stdout:747: seq_num 89 ranges {695823=777044,1925015=639682,3328058=167765} 2024-09-13T10:10:57.996 INFO:tasks.rados.rados.0.smithi195.stdout:747: writing smithi19530714-15 from 695823 to 1472867 tid 1 2024-09-13T10:10:57.999 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:57.999 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:58.009 INFO:tasks.rados.rados.0.smithi195.stdout:747: writing smithi19530714-15 from 1925015 to 2564697 tid 2 2024-09-13T10:10:58.014 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-13T10:10:58.019 INFO:tasks.rados.rados.0.smithi195.stdout:747: writing smithi19530714-15 from 3328058 to 3495823 tid 3 2024-09-13T10:10:58.019 INFO:tasks.rados.rados.0.smithi195.stdout:748: read oid 49 snap -1 2024-09-13T10:10:58.019 INFO:tasks.rados.rados.0.smithi195.stdout:748: expect (ObjNum 48 snap 0 seq_num 48) 2024-09-13T10:10:58.019 INFO:tasks.rados.rados.0.smithi195.stdout:749: read oid 30 snap -1 2024-09-13T10:10:58.019 INFO:tasks.rados.rados.0.smithi195.stdout:749: expect deleted 2024-09-13T10:10:58.019 INFO:tasks.rados.rados.0.smithi195.stdout:750: read oid 24 snap -1 2024-09-13T10:10:58.019 INFO:tasks.rados.rados.0.smithi195.stdout:750: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-13T10:10:58.019 INFO:tasks.rados.rados.0.smithi195.stdout:751: set_chunk oid 29 target oid 2024-09-13T10:10:58.019 INFO:tasks.rados.rados.0.smithi195.stdout:751: set_chunk oid 29 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:10:58.019 INFO:tasks.rados.rados.0.smithi195.stdout:752: read oid 17 snap -1 2024-09-13T10:10:58.020 INFO:tasks.rados.rados.0.smithi195.stdout:752: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-13T10:10:58.020 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:10:58.020 INFO:tasks.rados.rados.0.smithi195.stdout:738: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-13T10:10:58.020 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 330943 2024-09-13T10:10:58.024 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:10:58.023+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:10:58.055 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:58.110 INFO:tasks.rados.rados.0.smithi195.stdout:740: expect (ObjNum 72 snap 0 seq_num 72) 2024-09-13T10:10:58.126 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:10:58.124+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:10:58.132 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:58.133 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:58.147 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-13T10:10:58.189 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 380082 2024-09-13T10:10:58.227 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:58.223+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:58.229 INFO:tasks.rados.rados.0.smithi195.stdout:734: finishing write tid 3 to smithi19530714-11 2024-09-13T10:10:58.229 INFO:tasks.rados.rados.0.smithi195.stdout:734: finishing write tid 4 to smithi19530714-11 2024-09-13T10:10:58.229 INFO:tasks.rados.rados.0.smithi195.stdout:734: finishing write tid 5 to smithi19530714-11 2024-09-13T10:10:58.229 INFO:tasks.rados.rados.0.smithi195.stdout:734: oid 11 updating version 0 to 27 2024-09-13T10:10:58.229 INFO:tasks.rados.rados.0.smithi195.stdout:734: oid 11 version 27 is already newer than 26 2024-09-13T10:10:58.229 INFO:tasks.rados.rados.0.smithi195.stdout:734: oid 11 version 27 is already newer than 24 2024-09-13T10:10:58.229 INFO:tasks.rados.rados.0.smithi195.stdout:734: oid 11 version 27 is already newer than 25 2024-09-13T10:10:58.229 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 27 (ObjNum 88 snap 0 seq_num 88) dirty exists 2024-09-13T10:10:58.230 INFO:tasks.rados.rados.0.smithi195.stdout:734: left oid 11 (ObjNum 88 snap 0 seq_num 88) 2024-09-13T10:10:58.230 INFO:tasks.rados.rados.0.smithi195.stdout:739: expect (ObjNum 24 snap 0 seq_num 24) 2024-09-13T10:10:58.230 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:58.265 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:58.265 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:58.278 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 509476 2024-09-13T10:10:58.280 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-13T10:10:58.295 INFO:tasks.rados.rados.0.smithi195.stdout:735: expect (ObjNum 38 snap 0 seq_num 38) 2024-09-13T10:10:58.295 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 442702 2024-09-13T10:10:58.318 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:58.328 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:58.327+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:58.334 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.12 primary-affinity to 0 (802) 2024-09-13T10:10:58.370 INFO:tasks.rados.rados.0.smithi195.stdout:741: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:10:58.408 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:58.408 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:58.423 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-13T10:10:58.423 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1704344 2024-09-13T10:10:58.429 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:58.427+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:58.454 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:58.498 INFO:tasks.rados.rados.0.smithi195.stdout:743: finishing set_chunk to oid 13 2024-09-13T10:10:58.498 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 26 (ObjNum 56 snap 0 seq_num 56) dirty exists 2024-09-13T10:10:58.499 INFO:tasks.rados.rados.0.smithi195.stdout:744: finishing set_chunk to oid 19 2024-09-13T10:10:58.499 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 19 set_chunk returned error code -95 offset: 728064 length: 951808 2024-09-13T10:10:58.499 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 0 (ObjNum 78 snap 0 seq_num 78) dirty exists 2024-09-13T10:10:58.499 INFO:tasks.rados.rados.0.smithi195.stdout:745: got (2) No such file or directory 2024-09-13T10:10:58.499 INFO:tasks.rados.rados.0.smithi195.stdout:745: got expected ENOENT (src dne) 2024-09-13T10:10:58.499 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 23 (ObjNum 129 snap 0 seq_num 643758952) clean dne 2024-09-13T10:10:58.499 INFO:tasks.rados.rados.0.smithi195.stdout:746: expect (ObjNum 64 snap 0 seq_num 64) 2024-09-13T10:10:58.531 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:10:58.527+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:10:58.545 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:58.545 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:58.560 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T10:10:58.624 INFO:tasks.rados.rados.0.smithi195.stdout:747: finishing write tid 1 to smithi19530714-15 2024-09-13T10:10:58.624 INFO:tasks.rados.rados.0.smithi195.stdout:747: finishing write tid 2 to smithi19530714-15 2024-09-13T10:10:58.625 INFO:tasks.rados.rados.0.smithi195.stdout:751: finishing set_chunk to oid 29 2024-09-13T10:10:58.625 INFO:tasks.rados.rados.0.smithi195.stdout:751: got expected ENOENT (src dne) 2024-09-13T10:10:58.625 INFO:tasks.rados.rados.0.smithi195.stdout:752: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-13T10:10:58.632 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:10:58.632+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:10:58.691 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:58.692 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:58.705 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T10:10:58.720 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2599736 2024-09-13T10:10:58.728 INFO:tasks.rados.rados.0.smithi195.stdout:747: finishing write tid 3 to smithi19530714-15 2024-09-13T10:10:58.728 INFO:tasks.rados.rados.0.smithi195.stdout:747: finishing write tid 4 to smithi19530714-15 2024-09-13T10:10:58.728 INFO:tasks.rados.rados.0.smithi195.stdout:747: finishing write tid 5 to smithi19530714-15 2024-09-13T10:10:58.728 INFO:tasks.rados.rados.0.smithi195.stdout:747: oid 15 updating version 0 to 43 2024-09-13T10:10:58.728 INFO:tasks.rados.rados.0.smithi195.stdout:747: oid 15 updating version 43 to 45 2024-09-13T10:10:58.728 INFO:tasks.rados.rados.0.smithi195.stdout:747: oid 15 version 45 is already newer than 44 2024-09-13T10:10:58.728 INFO:tasks.rados.rados.0.smithi195.stdout:747: oid 15 version 45 is already newer than 42 2024-09-13T10:10:58.728 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 45 (ObjNum 89 snap 0 seq_num 89) dirty exists 2024-09-13T10:10:58.728 INFO:tasks.rados.rados.0.smithi195.stdout:747: left oid 15 (ObjNum 89 snap 0 seq_num 89) 2024-09-13T10:10:58.728 INFO:tasks.rados.rados.0.smithi195.stdout:748: expect (ObjNum 48 snap 0 seq_num 48) 2024-09-13T10:10:58.733 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:10:58.732+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:10:58.777 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:58.804 INFO:tasks.rados.rados.0.smithi195.stdout:750: expect (ObjNum 23 snap 0 seq_num 23) 2024-09-13T10:10:58.810 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1156910 2024-09-13T10:10:58.822 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:58.824 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:58.824 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:58.834 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:10:58.829+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:10:58.839 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T10:10:58.846 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:58.870 INFO:tasks.rados.rados.0.smithi195.stdout:734: done (15 left) 2024-09-13T10:10:58.871 INFO:tasks.rados.rados.0.smithi195.stdout:735: done (14 left) 2024-09-13T10:10:58.871 INFO:tasks.rados.rados.0.smithi195.stdout:738: done (13 left) 2024-09-13T10:10:58.871 INFO:tasks.rados.rados.0.smithi195.stdout:739: done (12 left) 2024-09-13T10:10:58.871 INFO:tasks.rados.rados.0.smithi195.stdout:740: done (11 left) 2024-09-13T10:10:58.871 INFO:tasks.rados.rados.0.smithi195.stdout:741: done (10 left) 2024-09-13T10:10:58.871 INFO:tasks.rados.rados.0.smithi195.stdout:743: done (9 left) 2024-09-13T10:10:58.871 INFO:tasks.rados.rados.0.smithi195.stdout:744: done (8 left) 2024-09-13T10:10:58.871 INFO:tasks.rados.rados.0.smithi195.stdout:745: done (7 left) 2024-09-13T10:10:58.871 INFO:tasks.rados.rados.0.smithi195.stdout:746: done (6 left) 2024-09-13T10:10:58.871 INFO:tasks.rados.rados.0.smithi195.stdout:747: done (5 left) 2024-09-13T10:10:58.871 INFO:tasks.rados.rados.0.smithi195.stdout:748: done (4 left) 2024-09-13T10:10:58.871 INFO:tasks.rados.rados.0.smithi195.stdout:749: done (3 left) 2024-09-13T10:10:58.872 INFO:tasks.rados.rados.0.smithi195.stdout:750: done (2 left) 2024-09-13T10:10:58.872 INFO:tasks.rados.rados.0.smithi195.stdout:751: done (1 left) 2024-09-13T10:10:58.872 INFO:tasks.rados.rados.0.smithi195.stdout:752: done (0 left) 2024-09-13T10:10:58.872 INFO:tasks.rados.rados.0.smithi195.stdout:753: read oid 30 snap -1 2024-09-13T10:10:58.872 INFO:tasks.rados.rados.0.smithi195.stdout:753: expect deleted 2024-09-13T10:10:58.872 INFO:tasks.rados.rados.0.smithi195.stdout:754: write oid 23 current snap is 0 2024-09-13T10:10:58.872 INFO:tasks.rados.rados.0.smithi195.stdout:754: seq_num 90 ranges {712630=452293,1583497=499693,2555481=425187} 2024-09-13T10:10:58.889 INFO:tasks.rados.rados.0.smithi195.stdout:754: writing smithi19530714-23 from 712630 to 1164923 tid 1 2024-09-13T10:10:58.904 INFO:tasks.rados.rados.0.smithi195.stdout:754: writing smithi19530714-23 from 1583497 to 2083190 tid 2 2024-09-13T10:10:58.920 INFO:tasks.rados.rados.0.smithi195.stdout:754: writing smithi19530714-23 from 2555481 to 2980668 tid 3 2024-09-13T10:10:58.920 INFO:tasks.rados.rados.0.smithi195.stdout:755: read oid 15 snap -1 2024-09-13T10:10:58.920 INFO:tasks.rados.rados.0.smithi195.stdout:755: expect (ObjNum 89 snap 0 seq_num 89) 2024-09-13T10:10:58.920 INFO:tasks.rados.rados.0.smithi195.stdout:756: read oid 32 snap -1 2024-09-13T10:10:58.920 INFO:tasks.rados.rados.0.smithi195.stdout:756: expect (ObjNum 82 snap 0 seq_num 82) 2024-09-13T10:10:58.920 INFO:tasks.rados.rados.0.smithi195.stdout:757: read oid 29 snap -1 2024-09-13T10:10:58.920 INFO:tasks.rados.rados.0.smithi195.stdout:757: expect deleted 2024-09-13T10:10:58.920 INFO:tasks.rados.rados.0.smithi195.stdout:758: tier_evict oid 7 2024-09-13T10:10:58.920 INFO:tasks.rados.rados.0.smithi195.stdout:758: tier_evict oid 7 snap -1 2024-09-13T10:10:58.920 INFO:tasks.rados.rados.0.smithi195.stdout:754: finishing write tid 1 to smithi19530714-23 2024-09-13T10:10:58.920 INFO:tasks.rados.rados.0.smithi195.stdout:754: finishing write tid 2 to smithi19530714-23 2024-09-13T10:10:58.921 INFO:tasks.rados.rados.0.smithi195.stdout:753: done (5 left) 2024-09-13T10:10:58.921 INFO:tasks.rados.rados.0.smithi195.stdout:759: set_chunk oid 13 target oid 2024-09-13T10:10:58.927 INFO:tasks.rados.rados.0.smithi195.stdout:759: set_chunk oid 13 offset: 2120192 length: 171520 target oid 50a7b108a479e378e8ac854243b90c75dc6363797b5e104bb353a1c22a73c323 offset: 0 snap -1 2024-09-13T10:10:58.928 INFO:tasks.rados.rados.0.smithi195.stdout:760: tier_flush oid 16 2024-09-13T10:10:58.928 INFO:tasks.rados.rados.0.smithi195.stdout:760: tier_flush oid 16 snap -1 2024-09-13T10:10:58.928 INFO:tasks.rados.rados.0.smithi195.stdout:754: finishing write tid 3 to smithi19530714-23 2024-09-13T10:10:58.928 INFO:tasks.rados.rados.0.smithi195.stdout:754: finishing write tid 4 to smithi19530714-23 2024-09-13T10:10:58.928 INFO:tasks.rados.rados.0.smithi195.stdout:754: finishing write tid 5 to smithi19530714-23 2024-09-13T10:10:58.928 INFO:tasks.rados.rados.0.smithi195.stdout:754: oid 23 updating version 0 to 28 2024-09-13T10:10:58.928 INFO:tasks.rados.rados.0.smithi195.stdout:754: oid 23 version 28 is already newer than 25 2024-09-13T10:10:58.928 INFO:tasks.rados.rados.0.smithi195.stdout:754: oid 23 version 28 is already newer than 27 2024-09-13T10:10:58.928 INFO:tasks.rados.rados.0.smithi195.stdout:754: oid 23 version 28 is already newer than 26 2024-09-13T10:10:58.928 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 28 (ObjNum 90 snap 0 seq_num 90) dirty exists 2024-09-13T10:10:58.928 INFO:tasks.rados.rados.0.smithi195.stdout:754: left oid 23 (ObjNum 90 snap 0 seq_num 90) 2024-09-13T10:10:58.928 INFO:tasks.rados.rados.0.smithi195.stdout:754: done (6 left) 2024-09-13T10:10:58.928 INFO:tasks.rados.rados.0.smithi195.stdout:761: tier_promote oid 22 2024-09-13T10:10:58.929 INFO:tasks.rados.rados.0.smithi195.stdout:762: delete oid 31 current snap is 0 2024-09-13T10:10:58.932 INFO:tasks.rados.rados.0.smithi195.stdout:762: done (7 left) 2024-09-13T10:10:58.932 INFO:tasks.rados.rados.0.smithi195.stdout:763: read oid 48 snap -1 2024-09-13T10:10:58.932 INFO:tasks.rados.rados.0.smithi195.stdout:763: expect deleted 2024-09-13T10:10:58.932 INFO:tasks.rados.rados.0.smithi195.stdout:764: write oid 24 current snap is 0 2024-09-13T10:10:58.932 INFO:tasks.rados.rados.0.smithi195.stdout:764: seq_num 92 ranges {746244=719144,2100461=503362,3126680=419564} 2024-09-13T10:10:58.935 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:10:58.932+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:10:58.957 INFO:tasks.rados.rados.0.smithi195.stdout:764: writing smithi19530714-24 from 746244 to 1465388 tid 1 2024-09-13T10:10:58.972 INFO:tasks.rados.rados.0.smithi195.stdout:764: writing smithi19530714-24 from 2100461 to 2603823 tid 2 2024-09-13T10:10:58.982 INFO:tasks.rados.rados.0.smithi195.stdout:764: writing smithi19530714-24 from 3126680 to 3546244 tid 3 2024-09-13T10:10:58.982 INFO:tasks.rados.rados.0.smithi195.stdout:765: tier_flush oid 10 2024-09-13T10:10:58.982 INFO:tasks.rados.rados.0.smithi195.stdout:765: tier_flush oid 10 snap -1 2024-09-13T10:10:58.983 INFO:tasks.rados.rados.0.smithi195.stdout:758: got (2) No such file or directory 2024-09-13T10:10:58.983 INFO:tasks.rados.rados.0.smithi195.stdout:758: got expected ENOENT (src dne) 2024-09-13T10:10:58.983 INFO:tasks.rados.rados.0.smithi195.stdout:759: finishing set_chunk to oid 13 2024-09-13T10:10:58.983 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 26 (ObjNum 56 snap 0 seq_num 56) dirty exists 2024-09-13T10:10:58.983 INFO:tasks.rados.rados.0.smithi195.stdout:756: expect (ObjNum 82 snap 0 seq_num 82) 2024-09-13T10:10:58.983 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:59.018 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:59.018 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:59.035 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T10:10:59.036 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:59.033+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:10:59.087 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 662078 2024-09-13T10:10:59.135 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 689691 2024-09-13T10:10:59.136 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:10:59.135+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:10:59.157 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:59.157 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:59.171 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T10:10:59.177 INFO:tasks.rados.rados.0.smithi195.stdout:761: got (0) Success 2024-09-13T10:10:59.177 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 12 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-09-13T10:10:59.177 INFO:tasks.rados.rados.0.smithi195.stdout:760: got (0) Success 2024-09-13T10:10:59.177 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 0 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-09-13T10:10:59.177 INFO:tasks.rados.rados.0.smithi195.stdout:764: finishing write tid 1 to smithi19530714-24 2024-09-13T10:10:59.177 INFO:tasks.rados.rados.0.smithi195.stdout:764: finishing write tid 2 to smithi19530714-24 2024-09-13T10:10:59.178 INFO:tasks.rados.rados.0.smithi195.stdout:764: finishing write tid 3 to smithi19530714-24 2024-09-13T10:10:59.178 INFO:tasks.rados.rados.0.smithi195.stdout:764: finishing write tid 4 to smithi19530714-24 2024-09-13T10:10:59.178 INFO:tasks.rados.rados.0.smithi195.stdout:764: finishing write tid 5 to smithi19530714-24 2024-09-13T10:10:59.178 INFO:tasks.rados.rados.0.smithi195.stdout:764: oid 24 updating version 0 to 50 2024-09-13T10:10:59.178 INFO:tasks.rados.rados.0.smithi195.stdout:764: oid 24 version 50 is already newer than 48 2024-09-13T10:10:59.178 INFO:tasks.rados.rados.0.smithi195.stdout:764: oid 24 updating version 50 to 51 2024-09-13T10:10:59.178 INFO:tasks.rados.rados.0.smithi195.stdout:764: oid 24 version 51 is already newer than 49 2024-09-13T10:10:59.178 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 51 (ObjNum 92 snap 0 seq_num 92) dirty exists 2024-09-13T10:10:59.178 INFO:tasks.rados.rados.0.smithi195.stdout:764: left oid 24 (ObjNum 92 snap 0 seq_num 92) 2024-09-13T10:10:59.178 INFO:tasks.rados.rados.0.smithi195.stdout:765: got (0) Success 2024-09-13T10:10:59.178 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 0 (ObjNum 84 snap 0 seq_num 84) dirty exists 2024-09-13T10:10:59.178 INFO:tasks.rados.rados.0.smithi195.stdout:756: done (9 left) 2024-09-13T10:10:59.178 INFO:tasks.rados.rados.0.smithi195.stdout:758: done (8 left) 2024-09-13T10:10:59.178 INFO:tasks.rados.rados.0.smithi195.stdout:759: done (7 left) 2024-09-13T10:10:59.179 INFO:tasks.rados.rados.0.smithi195.stdout:760: done (6 left) 2024-09-13T10:10:59.179 INFO:tasks.rados.rados.0.smithi195.stdout:761: done (5 left) 2024-09-13T10:10:59.179 INFO:tasks.rados.rados.0.smithi195.stdout:764: done (4 left) 2024-09-13T10:10:59.179 INFO:tasks.rados.rados.0.smithi195.stdout:765: done (3 left) 2024-09-13T10:10:59.179 INFO:tasks.rados.rados.0.smithi195.stdout:766: read oid 40 snap -1 2024-09-13T10:10:59.179 INFO:tasks.rados.rados.0.smithi195.stdout:766: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-13T10:10:59.179 INFO:tasks.rados.rados.0.smithi195.stdout:767: read oid 19 snap -1 2024-09-13T10:10:59.179 INFO:tasks.rados.rados.0.smithi195.stdout:767: expect (ObjNum 78 snap 0 seq_num 78) 2024-09-13T10:10:59.179 INFO:tasks.rados.rados.0.smithi195.stdout:768: set_chunk oid 21 target oid 2024-09-13T10:10:59.190 INFO:tasks.rados.rados.0.smithi195.stdout:768: set_chunk oid 21 offset: 1850368 length: 606720 target oid 9172c851e6752f1973583eed759c9e93d23b8b8fbb282d5f599f807543b6c148 offset: 0 snap -1 2024-09-13T10:10:59.190 INFO:tasks.rados.rados.0.smithi195.stdout:769: set_chunk oid 18 target oid 2024-09-13T10:10:59.202 INFO:tasks.rados.rados.0.smithi195.stdout:769: set_chunk oid 18 offset: 399360 length: 667648 target oid dad9547cbae480822bcf351e448df4cc112f2a693204ce2fcbf6eec6d9208409 offset: 0 snap -1 2024-09-13T10:10:59.202 INFO:tasks.rados.rados.0.smithi195.stdout:770: read oid 30 snap -1 2024-09-13T10:10:59.202 INFO:tasks.rados.rados.0.smithi195.stdout:770: expect deleted 2024-09-13T10:10:59.202 INFO:tasks.rados.rados.0.smithi195.stdout:771: tier_evict oid 14 2024-09-13T10:10:59.202 INFO:tasks.rados.rados.0.smithi195.stdout:771: tier_evict oid 14 snap -1 2024-09-13T10:10:59.202 INFO:tasks.rados.rados.0.smithi195.stdout:767: expect (ObjNum 78 snap 0 seq_num 78) 2024-09-13T10:10:59.237 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:59.233+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:10:59.289 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:59.289 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:59.303 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T10:10:59.350 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:10:59.349+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:10:59.354 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T10:10:59.378 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2849110 2024-09-13T10:10:59.388 INFO:tasks.rados.rados.0.smithi195.stdout:766: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-13T10:10:59.392 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1797945 2024-09-13T10:10:59.394 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:10:59.424 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:59.424 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:59.435 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:59.439 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:10:59.435+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:10:59.440 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-13T10:10:59.463 INFO:tasks.rados.rados.0.smithi195.stdout:768: finishing set_chunk to oid 21 2024-09-13T10:10:59.463 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 39 (ObjNum 69 snap 0 seq_num 69) dirty exists 2024-09-13T10:10:59.463 INFO:tasks.rados.rados.0.smithi195.stdout:769: finishing set_chunk to oid 18 2024-09-13T10:10:59.463 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 18 set_chunk returned error code -95 offset: 399360 length: 667648 2024-09-13T10:10:59.463 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 0 (ObjNum 51 snap 0 seq_num 51) dirty exists 2024-09-13T10:10:59.463 INFO:tasks.rados.rados.0.smithi195.stdout:755: expect (ObjNum 89 snap 0 seq_num 89) 2024-09-13T10:10:59.463 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 499948 2024-09-13T10:10:59.511 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:10:59.540 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:10:59.539+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:10:59.543 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:10:59.543 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:10:59.563 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:10:59.563 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:10:59.584 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:10:59.584 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:10:59.602 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:10:59.602 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:10:59.604 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:10:59.604 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:10:59.614 INFO:tasks.rados.rados.0.smithi195.stdout:771: got (22) Invalid argument 2024-09-13T10:10:59.614 INFO:tasks.rados.rados.0.smithi195.stdout:755: done (8 left) 2024-09-13T10:10:59.614 INFO:tasks.rados.rados.0.smithi195.stdout:757: done (7 left) 2024-09-13T10:10:59.614 INFO:tasks.rados.rados.0.smithi195.stdout:763: done (6 left) 2024-09-13T10:10:59.614 INFO:tasks.rados.rados.0.smithi195.stdout:766: done (5 left) 2024-09-13T10:10:59.614 INFO:tasks.rados.rados.0.smithi195.stdout:767: done (4 left) 2024-09-13T10:10:59.614 INFO:tasks.rados.rados.0.smithi195.stdout:768: done (3 left) 2024-09-13T10:10:59.615 INFO:tasks.rados.rados.0.smithi195.stdout:769: done (2 left) 2024-09-13T10:10:59.615 INFO:tasks.rados.rados.0.smithi195.stdout:770: done (1 left) 2024-09-13T10:10:59.615 INFO:tasks.rados.rados.0.smithi195.stdout:771: done (0 left) 2024-09-13T10:10:59.615 INFO:tasks.rados.rados.0.smithi195.stdout:772: read oid 34 snap -1 2024-09-13T10:10:59.615 INFO:tasks.rados.rados.0.smithi195.stdout:772: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-13T10:10:59.615 INFO:tasks.rados.rados.0.smithi195.stdout:773: set_chunk oid 38 target oid 2024-09-13T10:10:59.616 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-13T10:10:59.622 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:10:59.622 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:10:59.638 INFO:tasks.rados.rados.0.smithi195.stdout:773: set_chunk oid 38 offset: 1196032 length: 1177088 target oid b2af771e6db7b8a9e10768b1de0e13e3fcd80a249f6aaa1b91aa6989144c89d0 offset: 0 snap -1 2024-09-13T10:10:59.639 INFO:tasks.rados.rados.0.smithi195.stdout:774: write oid 31 current snap is 0 2024-09-13T10:10:59.639 INFO:tasks.rados.rados.0.smithi195.stdout:774: seq_num 93 ranges {763051=794393,2158943=647020,3437750=125301} 2024-09-13T10:10:59.641 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:10:59.641 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:10:59.641 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:10:59.641+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:10:59.661 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:10:59.661 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:10:59.663 INFO:tasks.rados.rados.0.smithi195.stdout:774: writing smithi19530714-31 from 763051 to 1557444 tid 1 2024-09-13T10:10:59.679 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:10:59.680 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:10:59.682 INFO:tasks.rados.rados.0.smithi195.stdout:774: writing smithi19530714-31 from 2158943 to 2805963 tid 2 2024-09-13T10:10:59.692 INFO:tasks.rados.rados.0.smithi195.stdout:774: writing smithi19530714-31 from 3437750 to 3563051 tid 3 2024-09-13T10:10:59.692 INFO:tasks.rados.rados.0.smithi195.stdout:775: write (excl) oid 41 current snap is 0 2024-09-13T10:10:59.692 INFO:tasks.rados.rados.0.smithi195.stdout:775: seq_num 94 ranges {779858=469642,1817425=507031,2781526=586628} 2024-09-13T10:10:59.696 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:10:59.696 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:10:59.705 INFO:tasks.rados.rados.0.smithi195.stdout:775: writing smithi19530714-41 from 779858 to 1249500 tid 1 2024-09-13T10:10:59.714 INFO:tasks.rados.rados.0.smithi195.stdout:775: writing smithi19530714-41 from 1817425 to 2324456 tid 2 2024-09-13T10:10:59.715 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:10:59.715 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:10:59.722 INFO:tasks.rados.rados.0.smithi195.stdout:775: writing smithi19530714-41 from 2781526 to 3368154 tid 3 2024-09-13T10:10:59.722 INFO:tasks.rados.rados.0.smithi195.stdout:776: write (excl) oid 37 current snap is 0 2024-09-13T10:10:59.722 INFO:tasks.rados.rados.0.smithi195.stdout:776: seq_num 95 ranges {796665=544891,1759554=650689,2976243=620422} 2024-09-13T10:10:59.731 INFO:tasks.rados.rados.0.smithi195.stdout:776: writing smithi19530714-37 from 796665 to 1341556 tid 1 2024-09-13T10:10:59.736 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:10:59.736 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:10:59.738 INFO:tasks.rados.rados.0.smithi195.stdout:776: writing smithi19530714-37 from 1759554 to 2410243 tid 2 2024-09-13T10:10:59.739 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:10:59.739 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:10:59.739 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:10:59.739 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:10:59.740 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:10:59.742 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:59.739+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:59.745 INFO:tasks.rados.rados.0.smithi195.stdout:776: writing smithi19530714-37 from 2976243 to 3596665 tid 3 2024-09-13T10:10:59.746 INFO:tasks.rados.rados.0.smithi195.stdout:777: delete oid 16 current snap is 0 2024-09-13T10:10:59.746 INFO:tasks.rados.rados.0.smithi195.stdout:772: expect (ObjNum 50 snap 0 seq_num 50) 2024-09-13T10:10:59.756 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-13T10:10:59.758 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:10:59.759 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:10:59.774 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:10:59.775 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:10:59.791 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:10:59.792 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:10:59.812 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:10:59.813 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:10:59.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:10:59.813+0000 7f56b6d9d640 0 -- 172.21.15.73:0/4014864369 >> v2:172.21.15.195:6805/848686366 conn(0x7f56b8227900 msgr2=0x7f56b8229cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:10:59.836 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:10:59.836 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:10:59.843 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:10:59.840+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:10:59.874 INFO:tasks.rados.rados.0.smithi195.stdout:773: finishing set_chunk to oid 38 2024-09-13T10:10:59.875 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 38 set_chunk returned error code -95 offset: 1196032 length: 1177088 2024-09-13T10:10:59.875 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 0 (ObjNum 71 snap 0 seq_num 71) dirty exists 2024-09-13T10:10:59.875 INFO:tasks.rados.rados.0.smithi195.stdout:774: finishing write tid 1 to smithi19530714-31 2024-09-13T10:10:59.875 INFO:tasks.rados.rados.0.smithi195.stdout:774: finishing write tid 2 to smithi19530714-31 2024-09-13T10:10:59.875 INFO:tasks.rados.rados.0.smithi195.stdout:774: finishing write tid 3 to smithi19530714-31 2024-09-13T10:10:59.875 INFO:tasks.rados.rados.0.smithi195.stdout:774: finishing write tid 4 to smithi19530714-31 2024-09-13T10:10:59.875 INFO:tasks.rados.rados.0.smithi195.stdout:774: finishing write tid 5 to smithi19530714-31 2024-09-13T10:10:59.875 INFO:tasks.rados.rados.0.smithi195.stdout:774: oid 31 updating version 0 to 28 2024-09-13T10:10:59.875 INFO:tasks.rados.rados.0.smithi195.stdout:774: oid 31 version 28 is already newer than 27 2024-09-13T10:10:59.875 INFO:tasks.rados.rados.0.smithi195.stdout:774: oid 31 updating version 28 to 29 2024-09-13T10:10:59.875 INFO:tasks.rados.rados.0.smithi195.stdout:774: oid 31 updating version 29 to 30 2024-09-13T10:10:59.876 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 30 (ObjNum 93 snap 0 seq_num 93) dirty exists 2024-09-13T10:10:59.876 INFO:tasks.rados.rados.0.smithi195.stdout:774: left oid 31 (ObjNum 93 snap 0 seq_num 93) 2024-09-13T10:10:59.876 INFO:tasks.rados.rados.0.smithi195.stdout:775: finishing write tid 1 to smithi19530714-41 2024-09-13T10:10:59.876 INFO:tasks.rados.rados.0.smithi195.stdout:775: finishing write tid 2 to smithi19530714-41 2024-09-13T10:10:59.876 INFO:tasks.rados.rados.0.smithi195.stdout:775: finishing write tid 3 to smithi19530714-41 2024-09-13T10:10:59.876 INFO:tasks.rados.rados.0.smithi195.stdout:775: finishing write tid 4 to smithi19530714-41 2024-09-13T10:10:59.876 INFO:tasks.rados.rados.0.smithi195.stdout:775: finishing write tid 5 to smithi19530714-41 2024-09-13T10:10:59.876 INFO:tasks.rados.rados.0.smithi195.stdout:775: oid 41 updating version 0 to 25 2024-09-13T10:10:59.876 INFO:tasks.rados.rados.0.smithi195.stdout:775: oid 41 version 25 is already newer than 24 2024-09-13T10:10:59.876 INFO:tasks.rados.rados.0.smithi195.stdout:775: oid 41 updating version 25 to 27 2024-09-13T10:10:59.876 INFO:tasks.rados.rados.0.smithi195.stdout:775: oid 41 version 27 is already newer than 26 2024-09-13T10:10:59.876 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 27 (ObjNum 94 snap 0 seq_num 94) dirty exists 2024-09-13T10:10:59.876 INFO:tasks.rados.rados.0.smithi195.stdout:775: left oid 41 (ObjNum 94 snap 0 seq_num 94) 2024-09-13T10:10:59.876 INFO:tasks.rados.rados.0.smithi195.stdout:776: finishing write tid 1 to smithi19530714-37 2024-09-13T10:10:59.876 INFO:tasks.rados.rados.0.smithi195.stdout:776: finishing write tid 2 to smithi19530714-37 2024-09-13T10:10:59.877 INFO:tasks.rados.rados.0.smithi195.stdout:776: finishing write tid 3 to smithi19530714-37 2024-09-13T10:10:59.877 INFO:tasks.rados.rados.0.smithi195.stdout:772: done (5 left) 2024-09-13T10:10:59.877 INFO:tasks.rados.rados.0.smithi195.stdout:773: done (4 left) 2024-09-13T10:10:59.877 INFO:tasks.rados.rados.0.smithi195.stdout:774: done (3 left) 2024-09-13T10:10:59.877 INFO:tasks.rados.rados.0.smithi195.stdout:775: done (2 left) 2024-09-13T10:10:59.877 INFO:tasks.rados.rados.0.smithi195.stdout:777: done (1 left) 2024-09-13T10:10:59.877 INFO:tasks.rados.rados.0.smithi195.stdout:778: tier_flush oid 1 2024-09-13T10:10:59.877 INFO:tasks.rados.rados.0.smithi195.stdout:778: tier_flush oid 1 snap -1 2024-09-13T10:10:59.877 INFO:tasks.rados.rados.0.smithi195.stdout:776: finishing write tid 4 to smithi19530714-37 2024-09-13T10:10:59.877 INFO:tasks.rados.rados.0.smithi195.stdout:776: finishing write tid 5 to smithi19530714-37 2024-09-13T10:10:59.877 INFO:tasks.rados.rados.0.smithi195.stdout:776: oid 37 updating version 0 to 12 2024-09-13T10:10:59.877 INFO:tasks.rados.rados.0.smithi195.stdout:776: oid 37 version 12 is already newer than 11 2024-09-13T10:10:59.877 INFO:tasks.rados.rados.0.smithi195.stdout:776: oid 37 updating version 12 to 14 2024-09-13T10:10:59.877 INFO:tasks.rados.rados.0.smithi195.stdout:776: oid 37 version 14 is already newer than 13 2024-09-13T10:10:59.878 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 14 (ObjNum 95 snap 0 seq_num 95) dirty exists 2024-09-13T10:10:59.878 INFO:tasks.rados.rados.0.smithi195.stdout:776: left oid 37 (ObjNum 95 snap 0 seq_num 95) 2024-09-13T10:10:59.878 INFO:tasks.rados.rados.0.smithi195.stdout:776: done (1 left) 2024-09-13T10:10:59.878 INFO:tasks.rados.rados.0.smithi195.stdout:779: read oid 24 snap -1 2024-09-13T10:10:59.878 INFO:tasks.rados.rados.0.smithi195.stdout:779: expect (ObjNum 92 snap 0 seq_num 92) 2024-09-13T10:10:59.878 INFO:tasks.rados.rados.0.smithi195.stdout:780: write (excl) oid 33 current snap is 0 2024-09-13T10:10:59.878 INFO:tasks.rados.rados.0.smithi195.stdout:780: seq_num 97 ranges {430279=695389,1876518=654358,3031089=536797} 2024-09-13T10:10:59.879 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:10:59.879 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:10:59.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:10:59.879 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:10:59.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.879 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4365.0:5 1.2 e26/24 1:5cea7035:::rbd_info:head v 26'1, mlcod=0'0)", 2024-09-13T10:10:59.879 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:31.142893+0000", 2024-09-13T10:10:59.879 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.734154356000001, 2024-09-13T10:10:59.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024212330000000001, 2024-09-13T10:10:59.879 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.879 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.879 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142893+0000", 2024-09-13T10:10:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142893+0000", 2024-09-13T10:10:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142893+0000", 2024-09-13T10:10:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1699999999999998e-07 2024-09-13T10:10:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142914+0000", 2024-09-13T10:10:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0378999999999999e-05 2024-09-13T10:10:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142951+0000", 2024-09-13T10:10:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7471000000000001e-05 2024-09-13T10:10:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142983+0000", 2024-09-13T10:10:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2422000000000002e-05 2024-09-13T10:10:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.143202+0000", 2024-09-13T10:10:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021875799999999999 2024-09-13T10:10:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.143308+0000", 2024-09-13T10:10:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105487 2024-09-13T10:10:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.145207+0000", 2024-09-13T10:10:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001899848 2024-09-13T10:10:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.145314+0000", 2024-09-13T10:10:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106351 2024-09-13T10:10:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.883 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4365.0:7 1.2 e27/24 1:5cea7035:::rbd_info:head v 27'2, mlcod=0'0)", 2024-09-13T10:10:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:32.145197+0000", 2024-09-13T10:10:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.731850368, 2024-09-13T10:10:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020358360000000001, 2024-09-13T10:10:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145197+0000", 2024-09-13T10:10:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145197+0000", 2024-09-13T10:10:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145197+0000", 2024-09-13T10:10:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5000000000000002e-07 2024-09-13T10:10:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145212+0000", 2024-09-13T10:10:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4619e-05 2024-09-13T10:10:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145241+0000", 2024-09-13T10:10:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8676000000000001e-05 2024-09-13T10:10:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145257+0000", 2024-09-13T10:10:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6155e-05 2024-09-13T10:10:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145397+0000", 2024-09-13T10:10:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014000199999999999 2024-09-13T10:10:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145437+0000", 2024-09-13T10:10:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0615000000000003e-05 2024-09-13T10:10:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.147104+0000", 2024-09-13T10:10:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016668259999999999 2024-09-13T10:10:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.147232+0000", 2024-09-13T10:10:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000128293 2024-09-13T10:10:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.887 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:89 4.9 e38/31 4:96fcf081:::smithi19530714-23:head v 38'1, mlcod=0'0)", 2024-09-13T10:10:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.811705+0000", 2024-09-13T10:10:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0653421109999996, 2024-09-13T10:10:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053533510000000001, 2024-09-13T10:10:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.811705+0000", 2024-09-13T10:10:59.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.811705+0000", 2024-09-13T10:10:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.811706+0000", 2024-09-13T10:10:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7700000000000003e-07 2024-09-13T10:10:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813717+0000", 2024-09-13T10:10:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002011494 2024-09-13T10:10:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813743+0000", 2024-09-13T10:10:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5528999999999999e-05 2024-09-13T10:10:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813759+0000", 2024-09-13T10:10:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6178999999999999e-05 2024-09-13T10:10:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813873+0000", 2024-09-13T10:10:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113863 2024-09-13T10:10:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813922+0000", 2024-09-13T10:10:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9558999999999998e-05 2024-09-13T10:10:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816994+0000", 2024-09-13T10:10:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003071299 2024-09-13T10:10:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817058+0000", 2024-09-13T10:10:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4550999999999999e-05 2024-09-13T10:10:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:90 4.9 e38/31 4:96fcf081:::smithi19530714-23:head v 38'2, mlcod=0'0)", 2024-09-13T10:10:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.815985+0000", 2024-09-13T10:10:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0610617760000007, 2024-09-13T10:10:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00489822, 2024-09-13T10:10:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.892 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815985+0000", 2024-09-13T10:10:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815985+0000", 2024-09-13T10:10:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815987+0000", 2024-09-13T10:10:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.277e-06 2024-09-13T10:10:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817028+0000", 2024-09-13T10:10:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001040664 2024-09-13T10:10:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817055+0000", 2024-09-13T10:10:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6835999999999999e-05 2024-09-13T10:10:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817074+0000", 2024-09-13T10:10:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8629000000000001e-05 2024-09-13T10:10:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817153+0000", 2024-09-13T10:10:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9794000000000002e-05 2024-09-13T10:10:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817183+0000", 2024-09-13T10:10:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9479e-05 2024-09-13T10:10:59.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820783+0000", 2024-09-13T10:10:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035997720000000002 2024-09-13T10:10:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820883+0000", 2024-09-13T10:10:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100769 2024-09-13T10:10:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:91 4.9 e38/31 4:96fcf081:::smithi19530714-23:head v 38'3, mlcod=0'0)", 2024-09-13T10:10:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.817117+0000", 2024-09-13T10:10:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0599295709999996, 2024-09-13T10:10:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038490590000000002, 2024-09-13T10:10:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817117+0000", 2024-09-13T10:10:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817117+0000", 2024-09-13T10:10:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817119+0000", 2024-09-13T10:10:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1999999999999999e-06 2024-09-13T10:10:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817130+0000", 2024-09-13T10:10:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1313e-05 2024-09-13T10:10:59.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817145+0000", 2024-09-13T10:10:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4786e-05 2024-09-13T10:10:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817157+0000", 2024-09-13T10:10:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2775999999999999e-05 2024-09-13T10:10:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818394+0000", 2024-09-13T10:10:59.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012367999999999999 2024-09-13T10:10:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818496+0000", 2024-09-13T10:10:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102098 2024-09-13T10:10:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820895+0000", 2024-09-13T10:10:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023989189999999998 2024-09-13T10:10:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820966+0000", 2024-09-13T10:10:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1167000000000005e-05 2024-09-13T10:10:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:129 4.9 e38/31 4:998d9bf1:::smithi19530714-33:head v 38'4, mlcod=0'0)", 2024-09-13T10:10:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.817768+0000", 2024-09-13T10:10:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0592787670000003, 2024-09-13T10:10:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047213589999999996, 2024-09-13T10:10:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.901 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817768+0000", 2024-09-13T10:10:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817768+0000", 2024-09-13T10:10:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817769+0000", 2024-09-13T10:10:59.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.184e-06 2024-09-13T10:10:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818638+0000", 2024-09-13T10:10:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086842199999999999 2024-09-13T10:10:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818665+0000", 2024-09-13T10:10:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6979e-05 2024-09-13T10:10:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818685+0000", 2024-09-13T10:10:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0063000000000001e-05 2024-09-13T10:10:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818994+0000", 2024-09-13T10:10:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030959500000000002 2024-09-13T10:10:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819019+0000", 2024-09-13T10:10:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4508999999999999e-05 2024-09-13T10:10:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822389+0000", 2024-09-13T10:10:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033701970000000001 2024-09-13T10:10:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822490+0000", 2024-09-13T10:10:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010041 2024-09-13T10:10:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.905 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:130 4.9 e38/31 4:998d9bf1:::smithi19530714-33:head v 38'5, mlcod=0'0)", 2024-09-13T10:10:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.819657+0000", 2024-09-13T10:10:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0573898029999995, 2024-09-13T10:10:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032383609999999999, 2024-09-13T10:10:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819657+0000", 2024-09-13T10:10:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819657+0000", 2024-09-13T10:10:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819659+0000", 2024-09-13T10:10:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.624e-06 2024-09-13T10:10:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820879+0000", 2024-09-13T10:10:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001220184 2024-09-13T10:10:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820906+0000", 2024-09-13T10:10:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6536000000000001e-05 2024-09-13T10:10:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820930+0000", 2024-09-13T10:10:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4552000000000001e-05 2024-09-13T10:10:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821017+0000", 2024-09-13T10:10:59.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6539999999999995e-05 2024-09-13T10:10:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821043+0000", 2024-09-13T10:10:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6018999999999999e-05 2024-09-13T10:10:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822848+0000", 2024-09-13T10:10:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001805673 2024-09-13T10:10:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822896+0000", 2024-09-13T10:10:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7233e-05 2024-09-13T10:10:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:131 4.9 e38/31 4:998d9bf1:::smithi19530714-33:head v 38'6, mlcod=0'0)", 2024-09-13T10:10:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.821886+0000", 2024-09-13T10:10:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0551613329999991, 2024-09-13T10:10:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001783485, 2024-09-13T10:10:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.910 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821886+0000", 2024-09-13T10:10:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821886+0000", 2024-09-13T10:10:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821887+0000", 2024-09-13T10:10:59.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4559999999999999e-06 2024-09-13T10:10:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821897+0000", 2024-09-13T10:10:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5659999999999996e-06 2024-09-13T10:10:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821918+0000", 2024-09-13T10:10:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1223e-05 2024-09-13T10:10:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821935+0000", 2024-09-13T10:10:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7255999999999999e-05 2024-09-13T10:10:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822113+0000", 2024-09-13T10:10:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000177808 2024-09-13T10:10:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822139+0000", 2024-09-13T10:10:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5728000000000002e-05 2024-09-13T10:10:59.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.823606+0000", 2024-09-13T10:10:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014676159999999999 2024-09-13T10:10:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.823669+0000", 2024-09-13T10:10:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2831999999999997e-05 2024-09-13T10:10:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:185 4.9 e38/31 4:9bec18ef:::smithi19530714-47:head v 38'7, mlcod=38'5)", 2024-09-13T10:10:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.941317+0000", 2024-09-13T10:10:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.9357301729999996, 2024-09-13T10:10:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002782858, 2024-09-13T10:10:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.941317+0000", 2024-09-13T10:10:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.941317+0000", 2024-09-13T10:10:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.941319+0000", 2024-09-13T10:10:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9860000000000001e-06 2024-09-13T10:10:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942196+0000", 2024-09-13T10:10:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087739999999999997 2024-09-13T10:10:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942230+0000", 2024-09-13T10:10:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4118000000000001e-05 2024-09-13T10:10:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942252+0000", 2024-09-13T10:10:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2030000000000001e-05 2024-09-13T10:10:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942365+0000", 2024-09-13T10:10:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112689 2024-09-13T10:10:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942398+0000", 2024-09-13T10:10:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2897e-05 2024-09-13T10:10:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.944044+0000", 2024-09-13T10:10:59.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016460800000000001 2024-09-13T10:10:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.944100+0000", 2024-09-13T10:10:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5658000000000001e-05 2024-09-13T10:10:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:186 4.9 e38/31 4:9bec18ef:::smithi19530714-47:head v 38'8, mlcod=38'6)", 2024-09-13T10:10:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.948439+0000", 2024-09-13T10:10:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.9286080999999999, 2024-09-13T10:10:59.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002770018, 2024-09-13T10:10:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.948439+0000", 2024-09-13T10:10:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.948439+0000", 2024-09-13T10:10:59.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.948440+0000", 2024-09-13T10:10:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4440000000000001e-06 2024-09-13T10:10:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949519+0000", 2024-09-13T10:10:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010790039999999999 2024-09-13T10:10:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949537+0000", 2024-09-13T10:10:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7312e-05 2024-09-13T10:10:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949545+0000", 2024-09-13T10:10:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2570000000000007e-06 2024-09-13T10:10:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949615+0000", 2024-09-13T10:10:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9614000000000001e-05 2024-09-13T10:10:59.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949633+0000", 2024-09-13T10:10:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8640000000000001e-05 2024-09-13T10:10:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.951146+0000", 2024-09-13T10:10:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015129570000000001 2024-09-13T10:10:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.951209+0000", 2024-09-13T10:10:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.279e-05 2024-09-13T10:10:59.923 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:187 4.9 e38/31 4:9bec18ef:::smithi19530714-47:head v 38'9, mlcod=38'6)", 2024-09-13T10:10:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.950298+0000", 2024-09-13T10:10:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.9267485499999992, 2024-09-13T10:10:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013826859999999999, 2024-09-13T10:10:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.950298+0000", 2024-09-13T10:10:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.950298+0000", 2024-09-13T10:10:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.950300+0000", 2024-09-13T10:10:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5179999999999999e-06 2024-09-13T10:10:59.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.950309+0000", 2024-09-13T10:10:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9209999999999998e-06 2024-09-13T10:10:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.950327+0000", 2024-09-13T10:10:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8247e-05 2024-09-13T10:10:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.950336+0000", 2024-09-13T10:10:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7309999999999999e-06 2024-09-13T10:10:59.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.950457+0000", 2024-09-13T10:10:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000121116 2024-09-13T10:10:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.950511+0000", 2024-09-13T10:10:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4489e-05 2024-09-13T10:10:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.951629+0000", 2024-09-13T10:10:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011179880000000001 2024-09-13T10:10:59.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.951681+0000", 2024-09-13T10:10:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1675999999999998e-05 2024-09-13T10:10:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:282 4.9 e38/31 4:998d9bf1:::smithi19530714-33:head v 38'11, mlcod=38'9)", 2024-09-13T10:10:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.261272+0000", 2024-09-13T10:10:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.6157754390000001, 2024-09-13T10:10:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016402599999999999, 2024-09-13T10:10:59.928 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261272+0000", 2024-09-13T10:10:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261272+0000", 2024-09-13T10:10:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261274+0000", 2024-09-13T10:10:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2800000000000002e-06 2024-09-13T10:10:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261290+0000", 2024-09-13T10:10:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5923e-05 2024-09-13T10:10:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261321+0000", 2024-09-13T10:10:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1211999999999999e-05 2024-09-13T10:10:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261341+0000", 2024-09-13T10:10:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9853000000000001e-05 2024-09-13T10:10:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261506+0000", 2024-09-13T10:10:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016557799999999999 2024-09-13T10:10:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261554+0000", 2024-09-13T10:10:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7292000000000001e-05 2024-09-13T10:10:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.262796+0000", 2024-09-13T10:10:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012426220000000001 2024-09-13T10:10:59.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.262912+0000", 2024-09-13T10:10:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0001155 2024-09-13T10:10:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:296 4.9 e38/31 4:9bec18ef:::smithi19530714-47:head v 38'12, mlcod=38'10)", 2024-09-13T10:10:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.270077+0000", 2024-09-13T10:10:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.6069702190000008, 2024-09-13T10:10:59.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012488670000000001, 2024-09-13T10:10:59.934 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.934 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.934 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.934 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270077+0000", 2024-09-13T10:10:59.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270077+0000", 2024-09-13T10:10:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270078+0000", 2024-09-13T10:10:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.37e-06 2024-09-13T10:10:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270086+0000", 2024-09-13T10:10:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2719999999999997e-06 2024-09-13T10:10:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270101+0000", 2024-09-13T10:10:59.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4868999999999999e-05 2024-09-13T10:10:59.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270108+0000", 2024-09-13T10:10:59.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2899999999999999e-06 2024-09-13T10:10:59.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270167+0000", 2024-09-13T10:10:59.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9219999999999999e-05 2024-09-13T10:10:59.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270189+0000", 2024-09-13T10:10:59.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2317999999999999e-05 2024-09-13T10:10:59.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.271245+0000", 2024-09-13T10:10:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001056218 2024-09-13T10:10:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.271326+0000", 2024-09-13T10:10:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0309999999999995e-05 2024-09-13T10:10:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:346 4.9 e38/31 4:9bec18ef:::smithi19530714-47:head v 38'14, mlcod=38'12)", 2024-09-13T10:10:59.938 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.292966+0000", 2024-09-13T10:10:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5840810370000007, 2024-09-13T10:10:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085901, 2024-09-13T10:10:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.292966+0000", 2024-09-13T10:10:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.292966+0000", 2024-09-13T10:10:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.292968+0000", 2024-09-13T10:10:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7969999999999999e-06 2024-09-13T10:10:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.292976+0000", 2024-09-13T10:10:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2379999999999997e-06 2024-09-13T10:10:59.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.292993+0000", 2024-09-13T10:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6784999999999999e-05 2024-09-13T10:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.293003+0000", 2024-09-13T10:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0658e-05 2024-09-13T10:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.293081+0000", 2024-09-13T10:10:59.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8028000000000006e-05 2024-09-13T10:10:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.293108+0000", 2024-09-13T10:10:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6299e-05 2024-09-13T10:10:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.293784+0000", 2024-09-13T10:10:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067593600000000003 2024-09-13T10:10:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.293825+0000", 2024-09-13T10:10:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1269000000000001e-05 2024-09-13T10:10:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:382 4.9 e39/31 4:998d9bf1:::smithi19530714-33:head v 39'16, mlcod=38'14)", 2024-09-13T10:10:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.325108+0000", 2024-09-13T10:10:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5519394099999992, 2024-09-13T10:10:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017053599999999999, 2024-09-13T10:10:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.325108+0000", 2024-09-13T10:10:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.325108+0000", 2024-09-13T10:10:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.325110+0000", 2024-09-13T10:10:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.227e-06 2024-09-13T10:10:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.325121+0000", 2024-09-13T10:10:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0917999999999999e-05 2024-09-13T10:10:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.325144+0000", 2024-09-13T10:10:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3184e-05 2024-09-13T10:10:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.325156+0000", 2024-09-13T10:10:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2241e-05 2024-09-13T10:10:59.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.325248+0000", 2024-09-13T10:10:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2248000000000002e-05 2024-09-13T10:10:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.325271+0000", 2024-09-13T10:10:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2682999999999999e-05 2024-09-13T10:10:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326761+0000", 2024-09-13T10:10:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001489588 2024-09-13T10:10:59.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326813+0000", 2024-09-13T10:10:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2271000000000001e-05 2024-09-13T10:10:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:396 4.9 e39/31 4:9bec18ef:::smithi19530714-47:head v 39'17, mlcod=38'14)", 2024-09-13T10:10:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.326336+0000", 2024-09-13T10:10:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5507111099999999, 2024-09-13T10:10:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00090133900000000002, 2024-09-13T10:10:59.947 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326336+0000", 2024-09-13T10:10:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326336+0000", 2024-09-13T10:10:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326337+0000", 2024-09-13T10:10:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6499999999999998e-07 2024-09-13T10:10:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326343+0000", 2024-09-13T10:10:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7649999999999996e-06 2024-09-13T10:10:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326361+0000", 2024-09-13T10:10:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7750000000000001e-05 2024-09-13T10:10:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326371+0000", 2024-09-13T10:10:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0023999999999999e-05 2024-09-13T10:10:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326439+0000", 2024-09-13T10:10:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7812999999999996e-05 2024-09-13T10:10:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.326480+0000", 2024-09-13T10:10:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0778000000000002e-05 2024-09-13T10:10:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.327179+0000", 2024-09-13T10:10:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00069924099999999997 2024-09-13T10:10:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.327237+0000", 2024-09-13T10:10:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8103000000000003e-05 2024-09-13T10:10:59.951 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:482 4.9 e39/31 4:998d9bf1:::smithi19530714-33:head v 39'18, mlcod=39'16)", 2024-09-13T10:10:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.347448+0000", 2024-09-13T10:10:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.5295990390000007, 2024-09-13T10:10:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019968960000000002, 2024-09-13T10:10:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.347448+0000", 2024-09-13T10:10:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.347448+0000", 2024-09-13T10:10:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.347450+0000", 2024-09-13T10:10:59.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6670000000000001e-06 2024-09-13T10:10:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.347459+0000", 2024-09-13T10:10:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9560000000000003e-06 2024-09-13T10:10:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.347477+0000", 2024-09-13T10:10:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8101000000000002e-05 2024-09-13T10:10:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.347486+0000", 2024-09-13T10:10:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7729999999999994e-06 2024-09-13T10:10:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.347559+0000", 2024-09-13T10:10:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2569000000000003e-05 2024-09-13T10:10:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.347581+0000", 2024-09-13T10:10:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1965000000000001e-05 2024-09-13T10:10:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.349412+0000", 2024-09-13T10:10:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018305420000000001 2024-09-13T10:10:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.349445+0000", 2024-09-13T10:10:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3322999999999999e-05 2024-09-13T10:10:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.11.0:18 4.9 e39/31 4:96fcf081:::smithi19530714-23:head v 39'19, mlcod=39'17)", 2024-09-13T10:10:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.854654+0000", 2024-09-13T10:10:59.956 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.0223934579999998, 2024-09-13T10:10:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016285239999999999, 2024-09-13T10:10:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.854654+0000", 2024-09-13T10:10:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.854654+0000", 2024-09-13T10:10:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.854656+0000", 2024-09-13T10:10:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.137e-06 2024-09-13T10:10:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.854668+0000", 2024-09-13T10:10:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2216e-05 2024-09-13T10:10:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.854694+0000", 2024-09-13T10:10:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5772999999999999e-05 2024-09-13T10:10:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.854745+0000", 2024-09-13T10:10:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1103999999999997e-05 2024-09-13T10:10:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.854906+0000", 2024-09-13T10:10:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000161609 2024-09-13T10:10:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.854961+0000", 2024-09-13T10:10:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5043999999999997e-05 2024-09-13T10:10:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.856213+0000", 2024-09-13T10:10:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001251589 2024-09-13T10:10:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.856282+0000", 2024-09-13T10:10:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9052000000000002e-05 2024-09-13T10:10:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.960 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.11.0:22 4.19 e42/41 4:998d9bf1:::smithi19530714-33:head v 42'23, mlcod=0'0)", 2024-09-13T10:10:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.624507+0000", 2024-09-13T10:10:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.2525395690000001, 2024-09-13T10:10:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039005810000000002, 2024-09-13T10:10:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:10:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.624507+0000", 2024-09-13T10:10:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.624507+0000", 2024-09-13T10:10:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.624510+0000", 2024-09-13T10:10:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4770000000000002e-06 2024-09-13T10:10:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.625777+0000", 2024-09-13T10:10:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001267589 2024-09-13T10:10:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.625809+0000", 2024-09-13T10:10:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1288999999999998e-05 2024-09-13T10:10:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.625827+0000", 2024-09-13T10:10:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8312999999999999e-05 2024-09-13T10:10:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626008+0000", 2024-09-13T10:10:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000181421 2024-09-13T10:10:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:10:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626077+0000", 2024-09-13T10:10:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8471000000000002e-05 2024-09-13T10:10:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:10:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.628292+0000", 2024-09-13T10:10:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002214728 2024-09-13T10:10:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.628408+0000", 2024-09-13T10:10:59.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011629300000000001 2024-09-13T10:10:59.968 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.968 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.968 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.969 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "pg_update_log_missing(4.19 epoch 42/41 rep_tid 23 entries 42'24 (0'0) error 4:998d9bf1:::smithi19530714-33:head by client.4607.0:1065 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 40'21 roll_forward_to 0'0)", 2024-09-13T10:10:59.969 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.626821+0000", 2024-09-13T10:10:59.969 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.2502256909999998, 2024-09-13T10:10:59.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022279040000000002, 2024-09-13T10:10:59.969 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:10:59.969 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:10:59.969 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:10:59.969 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:10:59.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:10:59.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626821+0000", 2024-09-13T10:10:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:10:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626821+0000", 2024-09-13T10:10:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:10:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:10:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626823+0000", 2024-09-13T10:10:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.77e-06 2024-09-13T10:10:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:10:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626840+0000", 2024-09-13T10:10:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7272999999999999e-05 2024-09-13T10:10:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:10:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626882+0000", 2024-09-13T10:10:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1936e-05 2024-09-13T10:10:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:10:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626930+0000", 2024-09-13T10:10:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7284000000000003e-05 2024-09-13T10:10:59.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:10:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.627490+0000", 2024-09-13T10:10:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056048399999999998 2024-09-13T10:10:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:10:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:10:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:10:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.629049+0000", 2024-09-13T10:10:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001559157 2024-09-13T10:10:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:10:59.972 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:10:59.973 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:10:59.975 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-13T10:10:59.976 INFO:tasks.rados.rados.0.smithi195.stdout:780: writing smithi19530714-33 from 430279 to 1125668 tid 1 2024-09-13T10:10:59.976 INFO:tasks.rados.rados.0.smithi195.stdout:780: writing smithi19530714-33 from 1876518 to 2530876 tid 2 2024-09-13T10:10:59.976 INFO:tasks.rados.rados.0.smithi195.stdout:780: writing smithi19530714-33 from 3031089 to 3567886 tid 3 2024-09-13T10:10:59.976 INFO:tasks.rados.rados.0.smithi195.stdout:781: read oid 15 snap -1 2024-09-13T10:10:59.977 INFO:tasks.rados.rados.0.smithi195.stdout:781: expect (ObjNum 89 snap 0 seq_num 89) 2024-09-13T10:10:59.977 INFO:tasks.rados.rados.0.smithi195.stdout:782: set_chunk oid 50 target oid 2024-09-13T10:10:59.977 INFO:tasks.rados.rados.0.smithi195.stdout:782: set_chunk oid 50 offset: 807936 length: 478720 target oid 1993d4762cba046f1b0776dd99cca27266065be4e86d857fda24d687b83032f8 offset: 0 snap -1 2024-09-13T10:10:59.977 INFO:tasks.rados.rados.0.smithi195.stdout:783: write oid 43 current snap is 0 2024-09-13T10:10:59.977 INFO:tasks.rados.rados.0.smithi195.stdout:783: seq_num 98 ranges {447086=770638,1935000=798016,3458512=188574} 2024-09-13T10:10:59.977 INFO:tasks.rados.rados.0.smithi195.stdout:783: writing smithi19530714-43 from 447086 to 1217724 tid 1 2024-09-13T10:10:59.977 INFO:tasks.rados.rados.0.smithi195.stdout:783: writing smithi19530714-43 from 1935000 to 2733016 tid 2 2024-09-13T10:10:59.977 INFO:tasks.rados.rados.0.smithi195.stdout:783: writing smithi19530714-43 from 3458512 to 3647086 tid 3 2024-09-13T10:10:59.977 INFO:tasks.rados.rados.0.smithi195.stdout:784: write (excl) oid 28 current snap is 0 2024-09-13T10:10:59.977 INFO:tasks.rados.rados.0.smithi195.stdout:784: seq_num 99 ranges {463893=562240,1593482=658027,2685935=559341} 2024-09-13T10:10:59.977 INFO:tasks.rados.rados.0.smithi195.stdout:784: writing smithi19530714-28 from 463893 to 1026133 tid 1 2024-09-13T10:10:59.977 INFO:tasks.rados.rados.0.smithi195.stdout:784: writing smithi19530714-28 from 1593482 to 2251509 tid 2 2024-09-13T10:10:59.977 INFO:tasks.rados.rados.0.smithi195.stdout:784: writing smithi19530714-28 from 2685935 to 3245276 tid 3 2024-09-13T10:10:59.978 INFO:tasks.rados.rados.0.smithi195.stdout:785: write oid 27 current snap is 0 2024-09-13T10:10:59.978 INFO:tasks.rados.rados.0.smithi195.stdout:785: seq_num 100 ranges {480700=637489,1651964=401685,2713358=770613} 2024-09-13T10:10:59.978 INFO:tasks.rados.rados.0.smithi195.stdout:785: writing smithi19530714-27 from 480700 to 1118189 tid 1 2024-09-13T10:10:59.978 INFO:tasks.rados.rados.0.smithi195.stdout:785: writing smithi19530714-27 from 1651964 to 2053649 tid 2 2024-09-13T10:10:59.978 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:10:59.971+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:10:59.979 INFO:tasks.rados.rados.0.smithi195.stdout:785: writing smithi19530714-27 from 2713358 to 3483971 tid 3 2024-09-13T10:10:59.979 INFO:tasks.rados.rados.0.smithi195.stdout:786: tier_evict oid 4 2024-09-13T10:10:59.979 INFO:tasks.rados.rados.0.smithi195.stdout:786: tier_evict oid 4 snap -1 2024-09-13T10:10:59.979 INFO:tasks.rados.rados.0.smithi195.stdout:780: finishing write tid 1 to smithi19530714-33 2024-09-13T10:10:59.979 INFO:tasks.rados.rados.0.smithi195.stdout:779: expect (ObjNum 92 snap 0 seq_num 92) 2024-09-13T10:11:00.046 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3326420 2024-09-13T10:11:00.048 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:00.075 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:00.072+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:00.104 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:00.105 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:00.105 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:11:00.105 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:00.120 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-13T10:11:00.155 INFO:tasks.rados.rados.0.smithi195.stdout:780: finishing write tid 2 to smithi19530714-33 2024-09-13T10:11:00.155 INFO:tasks.rados.rados.0.smithi195.stdout:780: finishing write tid 3 to smithi19530714-33 2024-09-13T10:11:00.156 INFO:tasks.rados.rados.0.smithi195.stdout:780: finishing write tid 4 to smithi19530714-33 2024-09-13T10:11:00.156 INFO:tasks.rados.rados.0.smithi195.stdout:780: finishing write tid 5 to smithi19530714-33 2024-09-13T10:11:00.156 INFO:tasks.rados.rados.0.smithi195.stdout:780: oid 33 updating version 0 to 27 2024-09-13T10:11:00.156 INFO:tasks.rados.rados.0.smithi195.stdout:780: oid 33 updating version 27 to 28 2024-09-13T10:11:00.156 INFO:tasks.rados.rados.0.smithi195.stdout:780: oid 33 version 28 is already newer than 26 2024-09-13T10:11:00.156 INFO:tasks.rados.rados.0.smithi195.stdout:780: oid 33 version 28 is already newer than 25 2024-09-13T10:11:00.156 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 28 (ObjNum 97 snap 0 seq_num 97) dirty exists 2024-09-13T10:11:00.156 INFO:tasks.rados.rados.0.smithi195.stdout:780: left oid 33 (ObjNum 97 snap 0 seq_num 97) 2024-09-13T10:11:00.156 INFO:tasks.rados.rados.0.smithi195.stdout:781: expect (ObjNum 89 snap 0 seq_num 89) 2024-09-13T10:11:00.176 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:00.175+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:00.241 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:00.241 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:00.241 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:11:00.241 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:11:00.241 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:00.255 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-13T10:11:00.260 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2357529 2024-09-13T10:11:00.276 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:00.276+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:00.284 INFO:tasks.rados.rados.0.smithi195.stdout:782: finishing set_chunk to oid 50 2024-09-13T10:11:00.284 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 50 set_chunk returned error code -95 offset: 807936 length: 478720 2024-09-13T10:11:00.284 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 0 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-09-13T10:11:00.284 INFO:tasks.rados.rados.0.smithi195.stdout:778: got (0) Success 2024-09-13T10:11:00.284 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 0 (ObjNum 81 snap 0 seq_num 81) dirty exists 2024-09-13T10:11:00.284 INFO:tasks.rados.rados.0.smithi195.stdout:783: finishing write tid 1 to smithi19530714-43 2024-09-13T10:11:00.284 INFO:tasks.rados.rados.0.smithi195.stdout:783: finishing write tid 2 to smithi19530714-43 2024-09-13T10:11:00.285 INFO:tasks.rados.rados.0.smithi195.stdout:783: finishing write tid 3 to smithi19530714-43 2024-09-13T10:11:00.285 INFO:tasks.rados.rados.0.smithi195.stdout:783: finishing write tid 4 to smithi19530714-43 2024-09-13T10:11:00.285 INFO:tasks.rados.rados.0.smithi195.stdout:783: finishing write tid 5 to smithi19530714-43 2024-09-13T10:11:00.285 INFO:tasks.rados.rados.0.smithi195.stdout:783: oid 43 updating version 0 to 31 2024-09-13T10:11:00.285 INFO:tasks.rados.rados.0.smithi195.stdout:783: oid 43 updating version 31 to 34 2024-09-13T10:11:00.285 INFO:tasks.rados.rados.0.smithi195.stdout:783: oid 43 version 34 is already newer than 33 2024-09-13T10:11:00.285 INFO:tasks.rados.rados.0.smithi195.stdout:783: oid 43 updating version 34 to 35 2024-09-13T10:11:00.285 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 35 (ObjNum 98 snap 0 seq_num 98) dirty exists 2024-09-13T10:11:00.285 INFO:tasks.rados.rados.0.smithi195.stdout:783: left oid 43 (ObjNum 98 snap 0 seq_num 98) 2024-09-13T10:11:00.285 INFO:tasks.rados.rados.0.smithi195.stdout:784: finishing write tid 1 to smithi19530714-28 2024-09-13T10:11:00.285 INFO:tasks.rados.rados.0.smithi195.stdout:784: finishing write tid 2 to smithi19530714-28 2024-09-13T10:11:00.285 INFO:tasks.rados.rados.0.smithi195.stdout:784: finishing write tid 3 to smithi19530714-28 2024-09-13T10:11:00.285 INFO:tasks.rados.rados.0.smithi195.stdout:778: done (8 left) 2024-09-13T10:11:00.285 INFO:tasks.rados.rados.0.smithi195.stdout:779: done (7 left) 2024-09-13T10:11:00.286 INFO:tasks.rados.rados.0.smithi195.stdout:780: done (6 left) 2024-09-13T10:11:00.286 INFO:tasks.rados.rados.0.smithi195.stdout:781: done (5 left) 2024-09-13T10:11:00.286 INFO:tasks.rados.rados.0.smithi195.stdout:782: done (4 left) 2024-09-13T10:11:00.286 INFO:tasks.rados.rados.0.smithi195.stdout:783: done (3 left) 2024-09-13T10:11:00.286 INFO:tasks.rados.rados.0.smithi195.stdout:787: read oid 50 snap -1 2024-09-13T10:11:00.286 INFO:tasks.rados.rados.0.smithi195.stdout:787: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-13T10:11:00.286 INFO:tasks.rados.rados.0.smithi195.stdout:788: read oid 20 snap -1 2024-09-13T10:11:00.286 INFO:tasks.rados.rados.0.smithi195.stdout:788: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-13T10:11:00.286 INFO:tasks.rados.rados.0.smithi195.stdout:789: set_chunk oid 29 target oid 2024-09-13T10:11:00.286 INFO:tasks.rados.rados.0.smithi195.stdout:789: set_chunk oid 29 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:00.286 INFO:tasks.rados.rados.0.smithi195.stdout:790: write oid 43 current snap is 0 2024-09-13T10:11:00.286 INFO:tasks.rados.rados.0.smithi195.stdout:784: finishing write tid 4 to smithi19530714-28 2024-09-13T10:11:00.286 INFO:tasks.rados.rados.0.smithi195.stdout:790: seq_num 101 ranges {497507=712738,1710446=661696,3140781=556726} 2024-09-13T10:11:00.293 INFO:tasks.rados.rados.0.smithi195.stdout:790: writing smithi19530714-43 from 497507 to 1210245 tid 1 2024-09-13T10:11:00.302 INFO:tasks.rados.rados.0.smithi195.stdout:790: writing smithi19530714-43 from 1710446 to 2372142 tid 2 2024-09-13T10:11:00.312 INFO:tasks.rados.rados.0.smithi195.stdout:790: writing smithi19530714-43 from 3140781 to 3697507 tid 3 2024-09-13T10:11:00.312 INFO:tasks.rados.rados.0.smithi195.stdout:791: write (excl) oid 30 current snap is 0 2024-09-13T10:11:00.312 INFO:tasks.rados.rados.0.smithi195.stdout:791: seq_num 102 ranges {514314=787987,1768928=405354,2768204=509510} 2024-09-13T10:11:00.322 INFO:tasks.rados.rados.0.smithi195.stdout:791: writing smithi19530714-30 from 514314 to 1302301 tid 1 2024-09-13T10:11:00.328 INFO:tasks.rados.rados.0.smithi195.stdout:791: writing smithi19530714-30 from 1768928 to 2174282 tid 2 2024-09-13T10:11:00.336 INFO:tasks.rados.rados.0.smithi195.stdout:791: writing smithi19530714-30 from 2768204 to 3277714 tid 3 2024-09-13T10:11:00.336 INFO:tasks.rados.rados.0.smithi195.stdout:792: read oid 9 snap -1 2024-09-13T10:11:00.336 INFO:tasks.rados.rados.0.smithi195.stdout:792: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-13T10:11:00.336 INFO:tasks.rados.rados.0.smithi195.stdout:793: write oid 14 current snap is 0 2024-09-13T10:11:00.336 INFO:tasks.rados.rados.0.smithi195.stdout:793: seq_num 103 ranges {531121=463236,1711057=665365,3079274=651847} 2024-09-13T10:11:00.343 INFO:tasks.rados.rados.0.smithi195.stdout:793: writing smithi19530714-14 from 531121 to 994357 tid 1 2024-09-13T10:11:00.352 INFO:tasks.rados.rados.0.smithi195.stdout:793: writing smithi19530714-14 from 1711057 to 2376422 tid 2 2024-09-13T10:11:00.352 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-13T10:11:00.360 INFO:tasks.rados.rados.0.smithi195.stdout:793: writing smithi19530714-14 from 3079274 to 3731121 tid 3 2024-09-13T10:11:00.361 INFO:tasks.rados.rados.0.smithi195.stdout:794: read oid 2 snap -1 2024-09-13T10:11:00.361 INFO:tasks.rados.rados.0.smithi195.stdout:794: expect (ObjNum 68 snap 0 seq_num 68) 2024-09-13T10:11:00.361 INFO:tasks.rados.rados.0.smithi195.stdout:795: read oid 15 snap -1 2024-09-13T10:11:00.361 INFO:tasks.rados.rados.0.smithi195.stdout:795: expect (ObjNum 89 snap 0 seq_num 89) 2024-09-13T10:11:00.361 INFO:tasks.rados.rados.0.smithi195.stdout:796: set_chunk oid 21 target oid 2024-09-13T10:11:00.378 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:00.373+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:00.385 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:00.385 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:11:00.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:11:00.385 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:11:00.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4365.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:30.119900+0000", 2024-09-13T10:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.261631692999998, 2024-09-13T10:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020673093999999999, 2024-09-13T10:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4365", 2024-09-13T10:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.73:0/186187721", 2024-09-13T10:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-13T10:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.386 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119900+0000", 2024-09-13T10:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119900+0000", 2024-09-13T10:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119903+0000", 2024-09-13T10:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0929999999999999e-06 2024-09-13T10:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119919+0000", 2024-09-13T10:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6082999999999998e-05 2024-09-13T10:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119954+0000", 2024-09-13T10:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4944000000000003e-05 2024-09-13T10:11:00.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119983+0000", 2024-09-13T10:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8932e-05 2024-09-13T10:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.120157+0000", 2024-09-13T10:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017421 2024-09-13T10:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-13T10:11:00.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.120180+0000", 2024-09-13T10:11:00.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3039000000000001e-05 2024-09-13T10:11:00.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.121820+0000", 2024-09-13T10:11:00.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016397479999999999 2024-09-13T10:11:00.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:00.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.121842+0000", 2024-09-13T10:11:00.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2612e-05 2024-09-13T10:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.125236+0000", 2024-09-13T10:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003394181 2024-09-13T10:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.125597+0000", 2024-09-13T10:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036106199999999999 2024-09-13T10:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:00.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.125908+0000", 2024-09-13T10:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031034200000000002 2024-09-13T10:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.139886+0000", 2024-09-13T10:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013977824 2024-09-13T10:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140020+0000", 2024-09-13T10:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013476199999999999 2024-09-13T10:11:00.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140357+0000", 2024-09-13T10:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033711 2024-09-13T10:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140464+0000", 2024-09-13T10:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106028 2024-09-13T10:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140573+0000", 2024-09-13T10:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010912399999999999 2024-09-13T10:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:203 3.1 3:b033e801:::smithi19530714-4 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3900676510000007, 2024-09-13T10:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035547764000000003, 2024-09-13T10:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.394 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 203 2024-09-13T10:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991467+0000", 2024-09-13T10:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5030000000000002e-06 2024-09-13T10:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991483+0000", 2024-09-13T10:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5503e-05 2024-09-13T10:11:00.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991522+0000", 2024-09-13T10:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9399e-05 2024-09-13T10:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991544+0000", 2024-09-13T10:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2408999999999998e-05 2024-09-13T10:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991646+0000", 2024-09-13T10:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101798 2024-09-13T10:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991818+0000", 2024-09-13T10:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017204300000000001 2024-09-13T10:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.001618+0000", 2024-09-13T10:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097993490000000006 2024-09-13T10:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026785+0000", 2024-09-13T10:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025167446 2024-09-13T10:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026856+0000", 2024-09-13T10:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0575999999999996e-05 2024-09-13T10:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026915+0000", 2024-09-13T10:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9258000000000001e-05 2024-09-13T10:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026964+0000", 2024-09-13T10:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000003e-05 2024-09-13T10:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027011+0000", 2024-09-13T10:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7200000000000002e-05 2024-09-13T10:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:204 3.1 3:babcb876:::smithi19530714-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3899431349999993, 2024-09-13T10:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.22268644700000001, 2024-09-13T10:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 204 2024-09-13T10:11:00.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991589+0000", 2024-09-13T10:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3859999999999999e-06 2024-09-13T10:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991598+0000", 2024-09-13T10:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7669999999999992e-06 2024-09-13T10:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991616+0000", 2024-09-13T10:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7589e-05 2024-09-13T10:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991631+0000", 2024-09-13T10:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.47e-05 2024-09-13T10:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992050+0000", 2024-09-13T10:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041913399999999999 2024-09-13T10:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992229+0000", 2024-09-13T10:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017956499999999999 2024-09-13T10:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.203594+0000", 2024-09-13T10:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.211364303 2024-09-13T10:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.208526+0000", 2024-09-13T10:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049325020000000001 2024-09-13T10:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214019+0000", 2024-09-13T10:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054933270000000001 2024-09-13T10:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214108+0000", 2024-09-13T10:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8517000000000006e-05 2024-09-13T10:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214202+0000", 2024-09-13T10:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3763000000000003e-05 2024-09-13T10:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214275+0000", 2024-09-13T10:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2893999999999997e-05 2024-09-13T10:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:205 3.1 3:aec11a92:::smithi19530714-6 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3898674809999996, 2024-09-13T10:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056950259000000003, 2024-09-13T10:11:00.407 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 205 2024-09-13T10:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991665+0000", 2024-09-13T10:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1400000000000001e-06 2024-09-13T10:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991673+0000", 2024-09-13T10:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5730000000000008e-06 2024-09-13T10:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991689+0000", 2024-09-13T10:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5956000000000001e-05 2024-09-13T10:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991705+0000", 2024-09-13T10:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5539000000000001e-05 2024-09-13T10:11:00.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992393+0000", 2024-09-13T10:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068836600000000004 2024-09-13T10:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992509+0000", 2024-09-13T10:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115563 2024-09-13T10:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:00.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.011894+0000", 2024-09-13T10:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019385594999999999 2024-09-13T10:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027029+0000", 2024-09-13T10:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015134664000000001 2024-09-13T10:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.031277+0000", 2024-09-13T10:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004247804 2024-09-13T10:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048542+0000", 2024-09-13T10:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017265401999999999 2024-09-13T10:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048579+0000", 2024-09-13T10:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6238000000000001e-05 2024-09-13T10:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048614+0000", 2024-09-13T10:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5419000000000001e-05 2024-09-13T10:11:00.413 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:209 3.1 3:9798d58d:::smithi19530714-10 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3897392659999994, 2024-09-13T10:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.069824676000000002, 2024-09-13T10:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:00.414 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 209 2024-09-13T10:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991793+0000", 2024-09-13T10:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.057e-06 2024-09-13T10:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991803+0000", 2024-09-13T10:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7860000000000008e-06 2024-09-13T10:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991817+0000", 2024-09-13T10:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4066000000000001e-05 2024-09-13T10:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991828+0000", 2024-09-13T10:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1097000000000001e-05 2024-09-13T10:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992666+0000", 2024-09-13T10:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083755799999999997 2024-09-13T10:11:00.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992792+0000", 2024-09-13T10:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000126334 2024-09-13T10:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027675+0000", 2024-09-13T10:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034883299 2024-09-13T10:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.035648+0000", 2024-09-13T10:11:00.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007972972 2024-09-13T10:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.045764+0000", 2024-09-13T10:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010115832 2024-09-13T10:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061460+0000", 2024-09-13T10:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015695563999999999 2024-09-13T10:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061544+0000", 2024-09-13T10:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4866999999999999e-05 2024-09-13T10:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061617+0000", 2024-09-13T10:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2243999999999995e-05 2024-09-13T10:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.420 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:213 3.1 3:accf4e56:::smithi19530714-14 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3889653230000008, 2024-09-13T10:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.060593045999999998, 2024-09-13T10:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 213 2024-09-13T10:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992568+0000", 2024-09-13T10:11:00.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3640000000000002e-06 2024-09-13T10:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992580+0000", 2024-09-13T10:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1796e-05 2024-09-13T10:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992662+0000", 2024-09-13T10:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1814e-05 2024-09-13T10:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992683+0000", 2024-09-13T10:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0746000000000001e-05 2024-09-13T10:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.993024+0000", 2024-09-13T10:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034141299999999999 2024-09-13T10:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.993182+0000", 2024-09-13T10:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015817699999999999 2024-09-13T10:11:00.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.019852+0000", 2024-09-13T10:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026669470000000001 2024-09-13T10:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.032551+0000", 2024-09-13T10:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012699729999999999 2024-09-13T10:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.034351+0000", 2024-09-13T10:11:00.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017998269999999999 2024-09-13T10:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053106+0000", 2024-09-13T10:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018755236000000002 2024-09-13T10:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053131+0000", 2024-09-13T10:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4729e-05 2024-09-13T10:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053159+0000", 2024-09-13T10:11:00.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7744000000000001e-05 2024-09-13T10:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:219 3.1 3:87bfc7b1:::smithi19530714-20 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3485162499999994, 2024-09-13T10:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.037494069999999997, 2024-09-13T10:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.427 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 219 2024-09-13T10:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:11:00.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033019+0000", 2024-09-13T10:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.726e-06 2024-09-13T10:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033036+0000", 2024-09-13T10:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7329999999999998e-05 2024-09-13T10:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033060+0000", 2024-09-13T10:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4286000000000001e-05 2024-09-13T10:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033079+0000", 2024-09-13T10:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8872000000000002e-05 2024-09-13T10:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033155+0000", 2024-09-13T10:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6118999999999994e-05 2024-09-13T10:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033297+0000", 2024-09-13T10:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014154700000000001 2024-09-13T10:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.041666+0000", 2024-09-13T10:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083696180000000005 2024-09-13T10:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046621+0000", 2024-09-13T10:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049545759999999996 2024-09-13T10:11:00.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.052333+0000", 2024-09-13T10:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057120160000000003 2024-09-13T10:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070340+0000", 2024-09-13T10:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018006859 2024-09-13T10:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.432 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070447+0000", 2024-09-13T10:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106637 2024-09-13T10:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070509+0000", 2024-09-13T10:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2483999999999993e-05 2024-09-13T10:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.433 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:222 3.1 3:bdac4e61:::smithi19530714-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3348282759999996, 2024-09-13T10:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041434034000000002, 2024-09-13T10:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 222 2024-09-13T10:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046706+0000", 2024-09-13T10:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0089999999999999e-06 2024-09-13T10:11:00.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046719+0000", 2024-09-13T10:11:00.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2565000000000001e-05 2024-09-13T10:11:00.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046740+0000", 2024-09-13T10:11:00.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0979e-05 2024-09-13T10:11:00.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046752+0000", 2024-09-13T10:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2917e-05 2024-09-13T10:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046803+0000", 2024-09-13T10:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0417000000000002e-05 2024-09-13T10:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046908+0000", 2024-09-13T10:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105039 2024-09-13T10:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.066097+0000", 2024-09-13T10:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019189277000000001 2024-09-13T10:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070969+0000", 2024-09-13T10:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048721429999999998 2024-09-13T10:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.087852+0000", 2024-09-13T10:11:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016882489000000001 2024-09-13T10:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.087942+0000", 2024-09-13T10:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9738e-05 2024-09-13T10:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088076+0000", 2024-09-13T10:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013400700000000001 2024-09-13T10:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088137+0000", 2024-09-13T10:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1453999999999998e-05 2024-09-13T10:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:223 3.1 3:8759f4e0:::smithi19530714-24 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3221514069999998, 2024-09-13T10:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053691092000000003, 2024-09-13T10:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.440 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 223 2024-09-13T10:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059383+0000", 2024-09-13T10:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.847e-06 2024-09-13T10:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059399+0000", 2024-09-13T10:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6739e-05 2024-09-13T10:11:00.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059433+0000", 2024-09-13T10:11:00.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3946000000000003e-05 2024-09-13T10:11:00.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059456+0000", 2024-09-13T10:11:00.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2163000000000001e-05 2024-09-13T10:11:00.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059621+0000", 2024-09-13T10:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016546700000000001 2024-09-13T10:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.060003+0000", 2024-09-13T10:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000381982 2024-09-13T10:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.085339+0000", 2024-09-13T10:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025335610000000001 2024-09-13T10:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.089896+0000", 2024-09-13T10:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045575930000000004 2024-09-13T10:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.102515+0000", 2024-09-13T10:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012618664999999999 2024-09-13T10:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113009+0000", 2024-09-13T10:11:00.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010493784000000001 2024-09-13T10:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113044+0000", 2024-09-13T10:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5543999999999997e-05 2024-09-13T10:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113071+0000", 2024-09-13T10:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6752000000000001e-05 2024-09-13T10:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.448 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:231 3.1 3:9d247594:::smithi19530714-32 low_tier:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.065612+0000", 2024-09-13T10:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3159187489999997, 2024-09-13T10:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031259105000000002, 2024-09-13T10:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.449 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 231 2024-09-13T10:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065612+0000", 2024-09-13T10:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065612+0000", 2024-09-13T10:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065614+0000", 2024-09-13T10:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7370000000000001e-06 2024-09-13T10:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065622+0000", 2024-09-13T10:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1110000000000008e-06 2024-09-13T10:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065640+0000", 2024-09-13T10:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7499999999999998e-05 2024-09-13T10:11:00.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065655+0000", 2024-09-13T10:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5021999999999999e-05 2024-09-13T10:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069879+0000", 2024-09-13T10:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042246360000000004 2024-09-13T10:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070080+0000", 2024-09-13T10:11:00.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020099199999999999 2024-09-13T10:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.075971+0000", 2024-09-13T10:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058906030000000003 2024-09-13T10:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088156+0000", 2024-09-13T10:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012184868999999999 2024-09-13T10:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.095237+0000", 2024-09-13T10:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0070814279999999999 2024-09-13T10:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.096766+0000", 2024-09-13T10:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001529007 2024-09-13T10:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.096823+0000", 2024-09-13T10:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6789999999999997e-05 2024-09-13T10:11:00.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.096872+0000", 2024-09-13T10:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8409999999999999e-05 2024-09-13T10:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:234 3.1 3:bdce9caa:::smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.3124321460000008, 2024-09-13T10:11:00.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043032263000000001, 2024-09-13T10:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 234 2024-09-13T10:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:11:00.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069102+0000", 2024-09-13T10:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.543e-06 2024-09-13T10:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069112+0000", 2024-09-13T10:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0091e-05 2024-09-13T10:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069133+0000", 2024-09-13T10:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1188000000000001e-05 2024-09-13T10:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069143+0000", 2024-09-13T10:11:00.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0456e-05 2024-09-13T10:11:00.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070588+0000", 2024-09-13T10:11:00.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001445046 2024-09-13T10:11:00.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070801+0000", 2024-09-13T10:11:00.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000212151 2024-09-13T10:11:00.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.461 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.461 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:00.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.082362+0000", 2024-09-13T10:11:00.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561781 2024-09-13T10:11:00.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088161+0000", 2024-09-13T10:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057986119999999999 2024-09-13T10:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.101972+0000", 2024-09-13T10:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013811498 2024-09-13T10:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.462 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112028+0000", 2024-09-13T10:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010055530999999999 2024-09-13T10:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112082+0000", 2024-09-13T10:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3974999999999997e-05 2024-09-13T10:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112131+0000", 2024-09-13T10:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9391000000000002e-05 2024-09-13T10:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.463 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.464 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:243 3.1 3:a4dd4282:::smithi19530714-44 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.464 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:11:00.464 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.2842244409999992, 2024-09-13T10:11:00.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036759694000000002, 2024-09-13T10:11:00.464 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.464 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.464 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.464 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.464 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 243 2024-09-13T10:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:11:00.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097308+0000", 2024-09-13T10:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2099999999999995e-07 2024-09-13T10:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097314+0000", 2024-09-13T10:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9989999999999997e-06 2024-09-13T10:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097326+0000", 2024-09-13T10:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1787000000000001e-05 2024-09-13T10:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097332+0000", 2024-09-13T10:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6950000000000003e-06 2024-09-13T10:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097378+0000", 2024-09-13T10:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5744000000000001e-05 2024-09-13T10:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097489+0000", 2024-09-13T10:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111352 2024-09-13T10:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.118571+0000", 2024-09-13T10:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021081249999999999 2024-09-13T10:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.128494+0000", 2024-09-13T10:11:00.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099237270000000002 2024-09-13T10:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.128775+0000", 2024-09-13T10:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028027299999999999 2024-09-13T10:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.133963+0000", 2024-09-13T10:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005188191 2024-09-13T10:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134015+0000", 2024-09-13T10:11:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2438000000000002e-05 2024-09-13T10:11:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134066+0000", 2024-09-13T10:11:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1316999999999997e-05 2024-09-13T10:11:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.470 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:248 3.1 3:b0d90ce0:::smithi19530714-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 18] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.114744+0000", 2024-09-13T10:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.2667875399999993, 2024-09-13T10:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028886754000000001, 2024-09-13T10:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.471 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 248 2024-09-13T10:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114744+0000", 2024-09-13T10:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114744+0000", 2024-09-13T10:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114745+0000", 2024-09-13T10:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.55e-06 2024-09-13T10:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114752+0000", 2024-09-13T10:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1670000000000002e-06 2024-09-13T10:11:00.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114768+0000", 2024-09-13T10:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5471000000000001e-05 2024-09-13T10:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114775+0000", 2024-09-13T10:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3799999999999996e-06 2024-09-13T10:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114826+0000", 2024-09-13T10:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0909999999999999e-05 2024-09-13T10:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114981+0000", 2024-09-13T10:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015455699999999999 2024-09-13T10:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.125745+0000", 2024-09-13T10:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010764404 2024-09-13T10:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.129548+0000", 2024-09-13T10:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038027019999999998 2024-09-13T10:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.140602+0000", 2024-09-13T10:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011054082 2024-09-13T10:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.143523+0000", 2024-09-13T10:11:00.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002920962 2024-09-13T10:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.143586+0000", 2024-09-13T10:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2866000000000004e-05 2024-09-13T10:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.143630+0000", 2024-09-13T10:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4703000000000002e-05 2024-09-13T10:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.477 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:280 4.f 4:fe60987c:::smithi19530714-31:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.236614+0000", 2024-09-13T10:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1449176520000002, 2024-09-13T10:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021117787999999998, 2024-09-13T10:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.478 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280 2024-09-13T10:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236614+0000", 2024-09-13T10:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236614+0000", 2024-09-13T10:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236616+0000", 2024-09-13T10:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4700000000000001e-06 2024-09-13T10:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236627+0000", 2024-09-13T10:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1226000000000001e-05 2024-09-13T10:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236654+0000", 2024-09-13T10:11:00.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6876000000000002e-05 2024-09-13T10:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236675+0000", 2024-09-13T10:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1199000000000002e-05 2024-09-13T10:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236772+0000", 2024-09-13T10:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6359000000000004e-05 2024-09-13T10:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236873+0000", 2024-09-13T10:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010113199999999999 2024-09-13T10:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2,7", 2024-09-13T10:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.254958+0000", 2024-09-13T10:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018085127999999999 2024-09-13T10:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.256855+0000", 2024-09-13T10:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001896888 2024-09-13T10:11:00.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257456+0000", 2024-09-13T10:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060094800000000002 2024-09-13T10:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257633+0000", 2024-09-13T10:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017736499999999999 2024-09-13T10:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257684+0000", 2024-09-13T10:11:00.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0812000000000001e-05 2024-09-13T10:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257731+0000", 2024-09-13T10:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7385000000000001e-05 2024-09-13T10:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:292 4.f 4:f6a0e8ce:::smithi19530714-43:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.484 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.257863+0000", 2024-09-13T10:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1236683200000002, 2024-09-13T10:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022747474, 2024-09-13T10:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 292 2024-09-13T10:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257863+0000", 2024-09-13T10:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257863+0000", 2024-09-13T10:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257865+0000", 2024-09-13T10:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.883e-06 2024-09-13T10:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257874+0000", 2024-09-13T10:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0699999999999996e-06 2024-09-13T10:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257894+0000", 2024-09-13T10:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0349000000000001e-05 2024-09-13T10:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257925+0000", 2024-09-13T10:11:00.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1092e-05 2024-09-13T10:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258040+0000", 2024-09-13T10:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114474 2024-09-13T10:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258126+0000", 2024-09-13T10:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6479999999999999e-05 2024-09-13T10:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2,7", 2024-09-13T10:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.273691+0000", 2024-09-13T10:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015564626999999999 2024-09-13T10:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.275091+0000", 2024-09-13T10:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001400079 2024-09-13T10:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.276990+0000", 2024-09-13T10:11:00.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018994179999999999 2024-09-13T10:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.280333+0000", 2024-09-13T10:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033424320000000002 2024-09-13T10:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.280486+0000", 2024-09-13T10:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015310500000000001 2024-09-13T10:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.280610+0000", 2024-09-13T10:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012446500000000001 2024-09-13T10:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:300 4.f 4:f7ad6ef6:::smithi19530714-1:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.264825+0000", 2024-09-13T10:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1167060180000004, 2024-09-13T10:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018261785999999999, 2024-09-13T10:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.491 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 300 2024-09-13T10:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264825+0000", 2024-09-13T10:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264825+0000", 2024-09-13T10:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264827+0000", 2024-09-13T10:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.204e-06 2024-09-13T10:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264838+0000", 2024-09-13T10:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0212e-05 2024-09-13T10:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264859+0000", 2024-09-13T10:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1613000000000001e-05 2024-09-13T10:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264873+0000", 2024-09-13T10:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3893000000000001e-05 2024-09-13T10:11:00.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264941+0000", 2024-09-13T10:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8355000000000005e-05 2024-09-13T10:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.265012+0000", 2024-09-13T10:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0759999999999993e-05 2024-09-13T10:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2,7", 2024-09-13T10:11:00.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.276117+0000", 2024-09-13T10:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011104283 2024-09-13T10:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.278041+0000", 2024-09-13T10:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001924427 2024-09-13T10:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.278343+0000", 2024-09-13T10:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030249900000000002 2024-09-13T10:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.283022+0000", 2024-09-13T10:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046781840000000002 2024-09-13T10:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.283053+0000", 2024-09-13T10:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1696999999999998e-05 2024-09-13T10:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.283087+0000", 2024-09-13T10:11:00.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3658999999999997e-05 2024-09-13T10:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.1.22:6 3.1 3:b033e801:::smithi19530714-4 low_tier:head [call cas.chunk_get_ref in=16b] snapc 0=[] RETRY=1 ondisk+retry+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.270563+0000", 2024-09-13T10:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.110968626, 2024-09-13T10:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012386094, 2024-09-13T10:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.498 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270563+0000", 2024-09-13T10:11:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270563+0000", 2024-09-13T10:11:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270565+0000", 2024-09-13T10:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2259999999999999e-06 2024-09-13T10:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270577+0000", 2024-09-13T10:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.186e-05 2024-09-13T10:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270603+0000", 2024-09-13T10:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6336e-05 2024-09-13T10:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270625+0000", 2024-09-13T10:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2025999999999999e-05 2024-09-13T10:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.270945+0000", 2024-09-13T10:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032027299999999999 2024-09-13T10:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.271163+0000", 2024-09-13T10:11:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021806700000000001 2024-09-13T10:11:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.271630+0000", 2024-09-13T10:11:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046686200000000002 2024-09-13T10:11:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.272547+0000", 2024-09-13T10:11:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00091682500000000002 2024-09-13T10:11:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.274201+0000", 2024-09-13T10:11:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00165398 2024-09-13T10:11:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.282877+0000", 2024-09-13T10:11:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086762560000000002 2024-09-13T10:11:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.282918+0000", 2024-09-13T10:11:00.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0666e-05 2024-09-13T10:11:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.282949+0000", 2024-09-13T10:11:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0716999999999997e-05 2024-09-13T10:11:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.1.22:23 3.1 3:969e51ee:::93c498e92137985868c9dcdff823d39ad7f943e0cf15bbbe4a7d3098e4bc2da6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:00.504 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.128192+0000", 2024-09-13T10:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.2533395880000002, 2024-09-13T10:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016280833000000001, 2024-09-13T10:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128192+0000", 2024-09-13T10:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128192+0000", 2024-09-13T10:11:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128193+0000", 2024-09-13T10:11:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0139999999999999e-06 2024-09-13T10:11:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128359+0000", 2024-09-13T10:11:00.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000166455 2024-09-13T10:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128372+0000", 2024-09-13T10:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3029e-05 2024-09-13T10:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128382+0000", 2024-09-13T10:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.55e-06 2024-09-13T10:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.137523+0000", 2024-09-13T10:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091413320000000003 2024-09-13T10:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.137721+0000", 2024-09-13T10:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000197777 2024-09-13T10:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.138223+0000", 2024-09-13T10:11:00.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050221499999999997 2024-09-13T10:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.144219+0000", 2024-09-13T10:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059962310000000003 2024-09-13T10:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.144284+0000", 2024-09-13T10:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4734999999999996e-05 2024-09-13T10:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.144319+0000", 2024-09-13T10:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5271999999999997e-05 2024-09-13T10:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.144403+0000", 2024-09-13T10:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3792e-05 2024-09-13T10:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.144472+0000", 2024-09-13T10:11:00.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9431000000000007e-05 2024-09-13T10:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.1.22:26 3.1 3:81602715:::0cd1941c9b5b3a2b092b213a045ec3831dc9f094ae77b0454d92fa84833c395e:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.128673+0000", 2024-09-13T10:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.2528578189999999, 2024-09-13T10:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017705240000000001, 2024-09-13T10:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128673+0000", 2024-09-13T10:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128673+0000", 2024-09-13T10:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128676+0000", 2024-09-13T10:11:00.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9129999999999999e-06 2024-09-13T10:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.129361+0000", 2024-09-13T10:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068506999999999997 2024-09-13T10:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.129403+0000", 2024-09-13T10:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1776000000000001e-05 2024-09-13T10:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.129420+0000", 2024-09-13T10:11:00.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7125e-05 2024-09-13T10:11:00.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.141718+0000", 2024-09-13T10:11:00.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012297439 2024-09-13T10:11:00.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.141913+0000", 2024-09-13T10:11:00.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019493000000000001 2024-09-13T10:11:00.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:00.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.142410+0000", 2024-09-13T10:11:00.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049744199999999996 2024-09-13T10:11:00.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:00.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.145111+0000", 2024-09-13T10:11:00.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027010329999999998 2024-09-13T10:11:00.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.145797+0000", 2024-09-13T10:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068605300000000005 2024-09-13T10:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.146186+0000", 2024-09-13T10:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038835200000000001 2024-09-13T10:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.146290+0000", 2024-09-13T10:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010416000000000001 2024-09-13T10:11:00.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.146379+0000", 2024-09-13T10:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8947000000000001e-05 2024-09-13T10:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1272 4.f 4:f7ad6ef6:::smithi19530714-1:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e45)", 2024-09-13T10:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.50599626600000003, 2024-09-13T10:11:00.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.050041173000000001, 2024-09-13T10:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1272 2024-09-13T10:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:00.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875537+0000", 2024-09-13T10:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2689999999999998e-06 2024-09-13T10:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875548+0000", 2024-09-13T10:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0291e-05 2024-09-13T10:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875567+0000", 2024-09-13T10:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9284999999999999e-05 2024-09-13T10:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875578+0000", 2024-09-13T10:11:00.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1256e-05 2024-09-13T10:11:00.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:00.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875763+0000", 2024-09-13T10:11:00.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000185396 2024-09-13T10:11:00.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:00.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875908+0000", 2024-09-13T10:11:00.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014412299999999999 2024-09-13T10:11:00.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:00.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:00.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:00.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.925576+0000", 2024-09-13T10:11:00.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049668552999999997 2024-09-13T10:11:00.522 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.522 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.522 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.522 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:00.522 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:00.522 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:00.523 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T10:11:00.525 INFO:tasks.rados.rados.0.smithi195.stdout:796: set_chunk oid 21 offset: 1053184 length: 987136 target oid a568948fd3887fbe65fa504131d2f31322eedc11dcb33f8056d8defabd530d5a offset: 0 snap -1 2024-09-13T10:11:00.525 INFO:tasks.rados.rados.0.smithi195.stdout:797: tier_evict oid 41 2024-09-13T10:11:00.526 INFO:tasks.rados.rados.0.smithi195.stdout:797: tier_evict oid 41 snap -1 2024-09-13T10:11:00.526 INFO:tasks.rados.rados.0.smithi195.stdout:798: write oid 8 current snap is 0 2024-09-13T10:11:00.526 INFO:tasks.rados.rados.0.smithi195.stdout:798: seq_num 104 ranges {547928=538485,1769539=409023,2706697=532054} 2024-09-13T10:11:00.526 INFO:tasks.rados.rados.0.smithi195.stdout:798: writing smithi19530714-8 from 547928 to 1086413 tid 1 2024-09-13T10:11:00.526 INFO:tasks.rados.rados.0.smithi195.stdout:798: writing smithi19530714-8 from 1769539 to 2178562 tid 2 2024-09-13T10:11:00.526 INFO:tasks.rados.rados.0.smithi195.stdout:798: writing smithi19530714-8 from 2706697 to 3238751 tid 3 2024-09-13T10:11:00.526 INFO:tasks.rados.rados.0.smithi195.stdout:799: tier_promote oid 38 2024-09-13T10:11:00.526 INFO:tasks.rados.rados.0.smithi195.stdout:784: finishing write tid 5 to smithi19530714-28 2024-09-13T10:11:00.526 INFO:tasks.rados.rados.0.smithi195.stdout:784: oid 28 updating version 0 to 52 2024-09-13T10:11:00.526 INFO:tasks.rados.rados.0.smithi195.stdout:784: oid 28 updating version 52 to 55 2024-09-13T10:11:00.526 INFO:tasks.rados.rados.0.smithi195.stdout:784: oid 28 version 55 is already newer than 53 2024-09-13T10:11:00.526 INFO:tasks.rados.rados.0.smithi195.stdout:784: oid 28 version 55 is already newer than 54 2024-09-13T10:11:00.526 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 55 (ObjNum 99 snap 0 seq_num 99) dirty exists 2024-09-13T10:11:00.526 INFO:tasks.rados.rados.0.smithi195.stdout:784: left oid 28 (ObjNum 99 snap 0 seq_num 99) 2024-09-13T10:11:00.527 INFO:tasks.rados.rados.0.smithi195.stdout:785: finishing write tid 1 to smithi19530714-27 2024-09-13T10:11:00.527 INFO:tasks.rados.rados.0.smithi195.stdout:785: finishing write tid 2 to smithi19530714-27 2024-09-13T10:11:00.527 INFO:tasks.rados.rados.0.smithi195.stdout:786: got (22) Invalid argument 2024-09-13T10:11:00.527 INFO:tasks.rados.rados.0.smithi195.stdout:785: finishing write tid 3 to smithi19530714-27 2024-09-13T10:11:00.527 INFO:tasks.rados.rados.0.smithi195.stdout:785: finishing write tid 4 to smithi19530714-27 2024-09-13T10:11:00.527 INFO:tasks.rados.rados.0.smithi195.stdout:785: finishing write tid 5 to smithi19530714-27 2024-09-13T10:11:00.527 INFO:tasks.rados.rados.0.smithi195.stdout:785: oid 27 updating version 0 to 46 2024-09-13T10:11:00.527 INFO:tasks.rados.rados.0.smithi195.stdout:785: oid 27 version 46 is already newer than 45 2024-09-13T10:11:00.527 INFO:tasks.rados.rados.0.smithi195.stdout:785: oid 27 updating version 46 to 48 2024-09-13T10:11:00.527 INFO:tasks.rados.rados.0.smithi195.stdout:785: oid 27 version 48 is already newer than 47 2024-09-13T10:11:00.527 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 48 (ObjNum 100 snap 0 seq_num 100) dirty exists 2024-09-13T10:11:00.527 INFO:tasks.rados.rados.0.smithi195.stdout:785: left oid 27 (ObjNum 100 snap 0 seq_num 100) 2024-09-13T10:11:00.527 INFO:tasks.rados.rados.0.smithi195.stdout:784: done (15 left) 2024-09-13T10:11:00.527 INFO:tasks.rados.rados.0.smithi195.stdout:785: done (14 left) 2024-09-13T10:11:00.527 INFO:tasks.rados.rados.0.smithi195.stdout:786: done (13 left) 2024-09-13T10:11:00.528 INFO:tasks.rados.rados.0.smithi195.stdout:800: read oid 36 snap -1 2024-09-13T10:11:00.528 INFO:tasks.rados.rados.0.smithi195.stdout:800: expect (ObjNum 80 snap 0 seq_num 80) 2024-09-13T10:11:00.528 INFO:tasks.rados.rados.0.smithi195.stdout:801: write oid 19 current snap is 0 2024-09-13T10:11:00.528 INFO:tasks.rados.rados.0.smithi195.stdout:789: finishing set_chunk to oid 29 2024-09-13T10:11:00.528 INFO:tasks.rados.rados.0.smithi195.stdout:789: got expected ENOENT (src dne) 2024-09-13T10:11:00.528 INFO:tasks.rados.rados.0.smithi195.stdout:801: seq_num 105 ranges {564735=613734,1828021=669034,3134120=630615} 2024-09-13T10:11:00.528 INFO:tasks.rados.rados.0.smithi195.stdout:801: writing smithi19530714-19 from 564735 to 1178469 tid 1 2024-09-13T10:11:00.528 INFO:tasks.rados.rados.0.smithi195.stdout:801: writing smithi19530714-19 from 1828021 to 2497055 tid 2 2024-09-13T10:11:00.528 INFO:tasks.rados.rados.0.smithi195.stdout:801: writing smithi19530714-19 from 3134120 to 3764735 tid 3 2024-09-13T10:11:00.528 INFO:tasks.rados.rados.0.smithi195.stdout:789: done (14 left) 2024-09-13T10:11:00.528 INFO:tasks.rados.rados.0.smithi195.stdout:802: write (excl) oid 49 current snap is 0 2024-09-13T10:11:00.528 INFO:tasks.rados.rados.0.smithi195.stdout:802: seq_num 106 ranges {581542=688983,1886503=412692,2761543=670951} 2024-09-13T10:11:00.528 INFO:tasks.rados.rados.0.smithi195.stdout:802: writing smithi19530714-49 from 581542 to 1270525 tid 1 2024-09-13T10:11:00.528 INFO:tasks.rados.rados.0.smithi195.stdout:802: writing smithi19530714-49 from 1886503 to 2299195 tid 2 2024-09-13T10:11:00.529 INFO:tasks.rados.rados.0.smithi195.stdout:802: writing smithi19530714-49 from 2761543 to 3432494 tid 3 2024-09-13T10:11:00.529 INFO:tasks.rados.rados.0.smithi195.stdout:803: delete oid 25 current snap is 0 2024-09-13T10:11:00.529 INFO:tasks.rados.rados.0.smithi195.stdout:787: expect (ObjNum 49 snap 0 seq_num 49) 2024-09-13T10:11:00.529 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:00.524+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:00.576 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1890591 2024-09-13T10:11:00.584 INFO:tasks.rados.rados.0.smithi195.stdout:788: expect (ObjNum 19 snap 0 seq_num 19) 2024-09-13T10:11:00.589 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 868032 2024-09-13T10:11:00.605 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 129906 2024-09-13T10:11:00.627 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:00.623+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:00.648 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:00.648 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:00.648 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:11:00.649 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:00.655 INFO:tasks.rados.rados.0.smithi195.stdout:790: finishing write tid 1 to smithi19530714-43 2024-09-13T10:11:00.655 INFO:tasks.rados.rados.0.smithi195.stdout:790: finishing write tid 2 to smithi19530714-43 2024-09-13T10:11:00.655 INFO:tasks.rados.rados.0.smithi195.stdout:790: finishing write tid 3 to smithi19530714-43 2024-09-13T10:11:00.655 INFO:tasks.rados.rados.0.smithi195.stdout:787: done (15 left) 2024-09-13T10:11:00.655 INFO:tasks.rados.rados.0.smithi195.stdout:788: done (14 left) 2024-09-13T10:11:00.655 INFO:tasks.rados.rados.0.smithi195.stdout:803: done (13 left) 2024-09-13T10:11:00.655 INFO:tasks.rados.rados.0.smithi195.stdout:804: write oid 7 current snap is 0 2024-09-13T10:11:00.656 INFO:tasks.rados.rados.0.smithi195.stdout:790: finishing write tid 4 to smithi19530714-43 2024-09-13T10:11:00.656 INFO:tasks.rados.rados.0.smithi195.stdout:790: finishing write tid 5 to smithi19530714-43 2024-09-13T10:11:00.656 INFO:tasks.rados.rados.0.smithi195.stdout:790: oid 43 updating version 0 to 39 2024-09-13T10:11:00.656 INFO:tasks.rados.rados.0.smithi195.stdout:790: oid 43 version 39 is already newer than 37 2024-09-13T10:11:00.656 INFO:tasks.rados.rados.0.smithi195.stdout:790: oid 43 version 39 is already newer than 38 2024-09-13T10:11:00.656 INFO:tasks.rados.rados.0.smithi195.stdout:790: oid 43 version 39 is already newer than 36 2024-09-13T10:11:00.656 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 39 (ObjNum 101 snap 0 seq_num 101) dirty exists 2024-09-13T10:11:00.656 INFO:tasks.rados.rados.0.smithi195.stdout:790: left oid 43 (ObjNum 101 snap 0 seq_num 101) 2024-09-13T10:11:00.656 INFO:tasks.rados.rados.0.smithi195.stdout:791: finishing write tid 1 to smithi19530714-30 2024-09-13T10:11:00.656 INFO:tasks.rados.rados.0.smithi195.stdout:791: finishing write tid 2 to smithi19530714-30 2024-09-13T10:11:00.656 INFO:tasks.rados.rados.0.smithi195.stdout:791: finishing write tid 3 to smithi19530714-30 2024-09-13T10:11:00.656 INFO:tasks.rados.rados.0.smithi195.stdout:804: seq_num 108 ranges {615156=555834,1603467=416361,2816389=693495} 2024-09-13T10:11:00.662 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T10:11:00.666 INFO:tasks.rados.rados.0.smithi195.stdout:804: writing smithi19530714-7 from 615156 to 1170990 tid 1 2024-09-13T10:11:00.675 INFO:tasks.rados.rados.0.smithi195.stdout:804: writing smithi19530714-7 from 1603467 to 2019828 tid 2 2024-09-13T10:11:00.690 INFO:tasks.rados.rados.0.smithi195.stdout:804: writing smithi19530714-7 from 2816389 to 3509884 tid 3 2024-09-13T10:11:00.690 INFO:tasks.rados.rados.0.smithi195.stdout:790: done (13 left) 2024-09-13T10:11:00.690 INFO:tasks.rados.rados.0.smithi195.stdout:805: tier_evict oid 50 2024-09-13T10:11:00.690 INFO:tasks.rados.rados.0.smithi195.stdout:805: tier_evict oid 50 snap -1 2024-09-13T10:11:00.690 INFO:tasks.rados.rados.0.smithi195.stdout:791: finishing write tid 4 to smithi19530714-30 2024-09-13T10:11:00.690 INFO:tasks.rados.rados.0.smithi195.stdout:791: finishing write tid 5 to smithi19530714-30 2024-09-13T10:11:00.690 INFO:tasks.rados.rados.0.smithi195.stdout:791: oid 30 updating version 0 to 33 2024-09-13T10:11:00.690 INFO:tasks.rados.rados.0.smithi195.stdout:791: oid 30 updating version 33 to 34 2024-09-13T10:11:00.690 INFO:tasks.rados.rados.0.smithi195.stdout:791: oid 30 updating version 34 to 36 2024-09-13T10:11:00.690 INFO:tasks.rados.rados.0.smithi195.stdout:791: oid 30 version 36 is already newer than 35 2024-09-13T10:11:00.690 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 36 (ObjNum 102 snap 0 seq_num 102) dirty exists 2024-09-13T10:11:00.690 INFO:tasks.rados.rados.0.smithi195.stdout:791: left oid 30 (ObjNum 102 snap 0 seq_num 102) 2024-09-13T10:11:00.691 INFO:tasks.rados.rados.0.smithi195.stdout:792: expect (ObjNum 8 snap 0 seq_num 8) 2024-09-13T10:11:00.693 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1815596 2024-09-13T10:11:00.727 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:00.725+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:00.729 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:00.754 INFO:tasks.rados.rados.0.smithi195.stdout:793: finishing write tid 1 to smithi19530714-14 2024-09-13T10:11:00.754 INFO:tasks.rados.rados.0.smithi195.stdout:793: finishing write tid 2 to smithi19530714-14 2024-09-13T10:11:00.754 INFO:tasks.rados.rados.0.smithi195.stdout:793: finishing write tid 3 to smithi19530714-14 2024-09-13T10:11:00.754 INFO:tasks.rados.rados.0.smithi195.stdout:795: expect (ObjNum 89 snap 0 seq_num 89) 2024-09-13T10:11:00.764 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1216115 2024-09-13T10:11:00.785 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:00.785 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:00.785 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:11:00.785 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:11:00.785 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:00.800 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T10:11:00.829 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:00.827+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:00.895 INFO:tasks.rados.rados.0.smithi195.stdout:794: expect (ObjNum 68 snap 0 seq_num 68) 2024-09-13T10:11:00.926 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:00.926 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:11:00.926 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:11:00.926 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:11:00.926 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.926 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4365.0:5 1.2 1:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T10:11:00.926 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:31.135075+0000", 2024-09-13T10:11:00.926 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 29.789404140999999, 2024-09-13T10:11:00.926 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010617898000000001, 2024-09-13T10:11:00.927 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:00.927 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:00.927 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.927 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:00.927 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4365", 2024-09-13T10:11:00.927 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.73:0/186187721", 2024-09-13T10:11:00.927 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 5 2024-09-13T10:11:00.927 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.927 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:00.927 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.927 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:00.927 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135075+0000", 2024-09-13T10:11:00.927 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.927 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.928 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.928 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:00.928 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135075+0000", 2024-09-13T10:11:00.928 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.928 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.928 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.928 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:00.928 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135089+0000", 2024-09-13T10:11:00.928 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3512e-05 2024-09-13T10:11:00.928 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.928 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.928 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:00.929 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135113+0000", 2024-09-13T10:11:00.929 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4006e-05 2024-09-13T10:11:00.929 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.929 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.929 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:00.929 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135150+0000", 2024-09-13T10:11:00.929 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6838999999999997e-05 2024-09-13T10:11:00.929 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.929 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.929 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.929 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135169+0000", 2024-09-13T10:11:00.929 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9499000000000001e-05 2024-09-13T10:11:00.929 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.930 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.930 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:00.930 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135393+0000", 2024-09-13T10:11:00.930 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000224081 2024-09-13T10:11:00.930 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.930 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.930 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op must wait for map", 2024-09-13T10:11:00.930 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135412+0000", 2024-09-13T10:11:00.930 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8989000000000001e-05 2024-09-13T10:11:00.930 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.930 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.930 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:00.930 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.138668+0000", 2024-09-13T10:11:00.931 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0032558909999999999 2024-09-13T10:11:00.931 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.931 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.931 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:00.931 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.138896+0000", 2024-09-13T10:11:00.931 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022737699999999999 2024-09-13T10:11:00.931 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.931 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.931 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 6,9", 2024-09-13T10:11:00.931 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.139159+0000", 2024-09-13T10:11:00.931 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026282399999999999 2024-09-13T10:11:00.931 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.931 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.931 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:00.931 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.143646+0000", 2024-09-13T10:11:00.932 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044877249999999997 2024-09-13T10:11:00.932 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.932 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.932 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.932 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.144479+0000", 2024-09-13T10:11:00.932 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00083277000000000004 2024-09-13T10:11:00.932 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.932 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.932 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.932 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145487+0000", 2024-09-13T10:11:00.932 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001008246 2024-09-13T10:11:00.932 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.932 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.933 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:00.933 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145571+0000", 2024-09-13T10:11:00.933 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.4040000000000002e-05 2024-09-13T10:11:00.933 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.933 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.933 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:00.933 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145693+0000", 2024-09-13T10:11:00.933 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000122099 2024-09-13T10:11:00.933 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.933 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:00.933 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.933 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.933 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.933 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:37 4.1 4:85f1e656:::smithi19530714-10:head [write 551263~793594,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.934 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.445403+0000", 2024-09-13T10:11:00.934 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.479076170000001, 2024-09-13T10:11:00.934 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010589464, 2024-09-13T10:11:00.934 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:00.934 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:00.934 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.934 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:00.934 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:00.934 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.934 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 37 2024-09-13T10:11:00.934 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.934 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:00.934 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.935 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:00.935 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.445403+0000", 2024-09-13T10:11:00.935 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.935 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.935 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.935 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:00.935 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.445403+0000", 2024-09-13T10:11:00.935 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.935 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.935 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.935 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:00.935 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.445406+0000", 2024-09-13T10:11:00.935 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.993e-06 2024-09-13T10:11:00.936 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.936 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.936 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:00.936 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.446815+0000", 2024-09-13T10:11:00.936 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001408687 2024-09-13T10:11:00.936 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.936 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.936 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:00.936 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.446840+0000", 2024-09-13T10:11:00.936 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4426e-05 2024-09-13T10:11:00.936 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.936 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.936 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.936 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.446856+0000", 2024-09-13T10:11:00.937 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6073e-05 2024-09-13T10:11:00.937 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.937 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.937 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:00.937 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.446986+0000", 2024-09-13T10:11:00.937 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000130746 2024-09-13T10:11:00.937 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.937 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.937 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:00.937 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.447206+0000", 2024-09-13T10:11:00.937 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021936199999999999 2024-09-13T10:11:00.937 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.938 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.938 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:00.938 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.447450+0000", 2024-09-13T10:11:00.938 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024457999999999999 2024-09-13T10:11:00.938 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.938 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.938 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.938 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.455772+0000", 2024-09-13T10:11:00.938 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.008322009 2024-09-13T10:11:00.938 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.938 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.938 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:00.938 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.455796+0000", 2024-09-13T10:11:00.938 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4110000000000001e-05 2024-09-13T10:11:00.939 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.939 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.939 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.939 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.455846+0000", 2024-09-13T10:11:00.939 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.9672000000000001e-05 2024-09-13T10:11:00.939 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.939 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.939 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:00.939 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.455927+0000", 2024-09-13T10:11:00.939 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.0927000000000006e-05 2024-09-13T10:11:00.939 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.939 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.939 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:00.939 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.455993+0000", 2024-09-13T10:11:00.940 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.5878999999999997e-05 2024-09-13T10:11:00.940 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.940 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:00.940 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.940 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.940 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.940 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:39 4.1 4:85f1e656:::smithi19530714-10:head [setxattr _header (56) in=7b,truncate 2151263] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.940 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.449473+0000", 2024-09-13T10:11:00.940 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.475007099000001, 2024-09-13T10:11:00.940 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011271674000000001, 2024-09-13T10:11:00.940 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:00.940 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:00.941 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.941 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:00.941 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:00.941 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.941 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 39 2024-09-13T10:11:00.941 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.941 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:00.941 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.941 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:00.941 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449473+0000", 2024-09-13T10:11:00.941 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.941 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.941 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.942 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:00.942 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449473+0000", 2024-09-13T10:11:00.942 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.942 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.942 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.942 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:00.942 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449474+0000", 2024-09-13T10:11:00.942 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5740000000000001e-06 2024-09-13T10:11:00.942 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.942 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.942 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:00.942 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449486+0000", 2024-09-13T10:11:00.942 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1545999999999999e-05 2024-09-13T10:11:00.943 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.943 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.943 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:00.943 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449504+0000", 2024-09-13T10:11:00.943 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8502999999999999e-05 2024-09-13T10:11:00.943 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.943 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.943 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.943 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449517+0000", 2024-09-13T10:11:00.943 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2753e-05 2024-09-13T10:11:00.943 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.943 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.944 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:00.944 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.454948+0000", 2024-09-13T10:11:00.944 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0054313540000000002 2024-09-13T10:11:00.944 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.944 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.944 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:00.944 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.455084+0000", 2024-09-13T10:11:00.944 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013585 2024-09-13T10:11:00.944 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.944 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.944 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:00.944 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.455296+0000", 2024-09-13T10:11:00.944 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021145100000000001 2024-09-13T10:11:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.456431+0000", 2024-09-13T10:11:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011352210000000001 2024-09-13T10:11:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.456490+0000", 2024-09-13T10:11:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9457999999999999e-05 2024-09-13T10:11:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.460547+0000", 2024-09-13T10:11:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0040571690000000002 2024-09-13T10:11:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.460663+0000", 2024-09-13T10:11:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000115691 2024-09-13T10:11:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.460744+0000", 2024-09-13T10:11:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.1104000000000001e-05 2024-09-13T10:11:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:40 4.1 4:85f1e656:::smithi19530714-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.449537+0000", 2024-09-13T10:11:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.474942453000001, 2024-09-13T10:11:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011651083, 2024-09-13T10:11:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 40 2024-09-13T10:11:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449537+0000", 2024-09-13T10:11:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449537+0000", 2024-09-13T10:11:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449538+0000", 2024-09-13T10:11:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0070000000000001e-06 2024-09-13T10:11:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449543+0000", 2024-09-13T10:11:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.3560000000000003e-06 2024-09-13T10:11:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449554+0000", 2024-09-13T10:11:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1263e-05 2024-09-13T10:11:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449566+0000", 2024-09-13T10:11:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2383e-05 2024-09-13T10:11:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.456239+0000", 2024-09-13T10:11:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0066730080000000002 2024-09-13T10:11:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.456350+0000", 2024-09-13T10:11:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000111214 2024-09-13T10:11:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.460834+0000", 2024-09-13T10:11:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044834080000000004 2024-09-13T10:11:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.460956+0000", 2024-09-13T10:11:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000121934 2024-09-13T10:11:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.461188+0000", 2024-09-13T10:11:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023251000000000001 2024-09-13T10:11:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:166 4.1 4:8e413aa3:::smithi19530714-42:head [write 1699644~784332] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.873691+0000", 2024-09-13T10:11:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.050788462, 2024-09-13T10:11:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0093888319999999997, 2024-09-13T10:11:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 166 2024-09-13T10:11:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.873691+0000", 2024-09-13T10:11:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.873691+0000", 2024-09-13T10:11:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.873693+0000", 2024-09-13T10:11:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1169999999999998e-06 2024-09-13T10:11:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.874601+0000", 2024-09-13T10:11:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00090731299999999998 2024-09-13T10:11:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.874626+0000", 2024-09-13T10:11:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5296e-05 2024-09-13T10:11:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.874639+0000", 2024-09-13T10:11:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3332999999999999e-05 2024-09-13T10:11:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.874783+0000", 2024-09-13T10:11:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014395599999999999 2024-09-13T10:11:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.874999+0000", 2024-09-13T10:11:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000215675 2024-09-13T10:11:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.875391+0000", 2024-09-13T10:11:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039174700000000002 2024-09-13T10:11:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.879556+0000", 2024-09-13T10:11:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0041654559999999997 2024-09-13T10:11:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.882494+0000", 2024-09-13T10:11:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0029378939999999999 2024-09-13T10:11:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.882784+0000", 2024-09-13T10:11:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00029026600000000002 2024-09-13T10:11:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.882941+0000", 2024-09-13T10:11:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000156739 2024-09-13T10:11:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.883080+0000", 2024-09-13T10:11:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013904 2024-09-13T10:11:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:259 4.1 4:85f1e656:::smithi19530714-10:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:11:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.7093130340000009, 2024-09-13T10:11:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014693513, 2024-09-13T10:11:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 259 2024-09-13T10:11:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:11:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:11:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215171+0000", 2024-09-13T10:11:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.3329999999999998e-06 2024-09-13T10:11:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215190+0000", 2024-09-13T10:11:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8672999999999999e-05 2024-09-13T10:11:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215229+0000", 2024-09-13T10:11:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9076000000000003e-05 2024-09-13T10:11:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215287+0000", 2024-09-13T10:11:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8142e-05 2024-09-13T10:11:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215435+0000", 2024-09-13T10:11:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014800600000000001 2024-09-13T10:11:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215593+0000", 2024-09-13T10:11:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015864300000000001 2024-09-13T10:11:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226037+0000", 2024-09-13T10:11:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010443948 2024-09-13T10:11:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228312+0000", 2024-09-13T10:11:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022743009999999998 2024-09-13T10:11:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228708+0000", 2024-09-13T10:11:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039651999999999998 2024-09-13T10:11:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229625+0000", 2024-09-13T10:11:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00091692300000000004 2024-09-13T10:11:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229750+0000", 2024-09-13T10:11:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012532099999999999 2024-09-13T10:11:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229860+0000", 2024-09-13T10:11:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000109627 2024-09-13T10:11:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:260 4.3 4:c426503f:::smithi19530714-11:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:11:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.7091450750000003, 2024-09-13T10:11:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041961871999999997, 2024-09-13T10:11:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 260 2024-09-13T10:11:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:11:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:11:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215336+0000", 2024-09-13T10:11:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9069999999999999e-06 2024-09-13T10:11:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215345+0000", 2024-09-13T10:11:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.6659999999999997e-06 2024-09-13T10:11:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215363+0000", 2024-09-13T10:11:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8153000000000001e-05 2024-09-13T10:11:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215388+0000", 2024-09-13T10:11:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4556e-05 2024-09-13T10:11:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215542+0000", 2024-09-13T10:11:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000154637 2024-09-13T10:11:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215825+0000", 2024-09-13T10:11:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028208600000000002 2024-09-13T10:11:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254572+0000", 2024-09-13T10:11:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.038747464000000002 2024-09-13T10:11:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.255694+0000", 2024-09-13T10:11:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011222929999999999 2024-09-13T10:11:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.256931+0000", 2024-09-13T10:11:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0012370980000000001 2024-09-13T10:11:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257080+0000", 2024-09-13T10:11:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014874800000000001 2024-09-13T10:11:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257211+0000", 2024-09-13T10:11:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000130674 2024-09-13T10:11:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257296+0000", 2024-09-13T10:11:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5589999999999999e-05 2024-09-13T10:11:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:269 4.3 4:c5b5ff75:::smithi19530714-20:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:11:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.7086408370000008, 2024-09-13T10:11:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014988243, 2024-09-13T10:11:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 269 2024-09-13T10:11:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:11:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:11:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215841+0000", 2024-09-13T10:11:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0829999999999998e-06 2024-09-13T10:11:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215854+0000", 2024-09-13T10:11:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2976000000000001e-05 2024-09-13T10:11:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215875+0000", 2024-09-13T10:11:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1100000000000001e-05 2024-09-13T10:11:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215892+0000", 2024-09-13T10:11:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7388999999999999e-05 2024-09-13T10:11:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216353+0000", 2024-09-13T10:11:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00046100900000000002 2024-09-13T10:11:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216611+0000", 2024-09-13T10:11:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025716299999999998 2024-09-13T10:11:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.227177+0000", 2024-09-13T10:11:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010566676000000001 2024-09-13T10:11:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229574+0000", 2024-09-13T10:11:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023964469999999999 2024-09-13T10:11:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230619+0000", 2024-09-13T10:11:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0010455950000000001 2024-09-13T10:11:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230697+0000", 2024-09-13T10:11:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8021000000000004e-05 2024-09-13T10:11:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230767+0000", 2024-09-13T10:11:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9544999999999999e-05 2024-09-13T10:11:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230827+0000", 2024-09-13T10:11:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.0238999999999997e-05 2024-09-13T10:11:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:288 4.3 4:cceea2a7:::smithi19530714-39:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.249445+0000", 2024-09-13T10:11:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.6750345459999991, 2024-09-13T10:11:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012024603, 2024-09-13T10:11:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 288 2024-09-13T10:11:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249445+0000", 2024-09-13T10:11:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249445+0000", 2024-09-13T10:11:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249448+0000", 2024-09-13T10:11:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0350000000000002e-06 2024-09-13T10:11:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249460+0000", 2024-09-13T10:11:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2214e-05 2024-09-13T10:11:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249490+0000", 2024-09-13T10:11:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0130000000000001e-05 2024-09-13T10:11:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249510+0000", 2024-09-13T10:11:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9757000000000001e-05 2024-09-13T10:11:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249684+0000", 2024-09-13T10:11:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000174086 2024-09-13T10:11:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249913+0000", 2024-09-13T10:11:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022833400000000001 2024-09-13T10:11:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257665+0000", 2024-09-13T10:11:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0077526330000000001 2024-09-13T10:11:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.259621+0000", 2024-09-13T10:11:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019558610000000001 2024-09-13T10:11:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260048+0000", 2024-09-13T10:11:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00042668799999999999 2024-09-13T10:11:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.261264+0000", 2024-09-13T10:11:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001216445 2024-09-13T10:11:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.261374+0000", 2024-09-13T10:11:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110117 2024-09-13T10:11:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.261470+0000", 2024-09-13T10:11:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5303000000000006e-05 2024-09-13T10:11:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:291 4.1 4:8e413aa3:::smithi19530714-42:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.257868+0000", 2024-09-13T10:11:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.6666117689999993, 2024-09-13T10:11:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010494850999999999, 2024-09-13T10:11:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 291 2024-09-13T10:11:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257868+0000", 2024-09-13T10:11:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257868+0000", 2024-09-13T10:11:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257870+0000", 2024-09-13T10:11:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3800000000000001e-06 2024-09-13T10:11:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257882+0000", 2024-09-13T10:11:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1522000000000001e-05 2024-09-13T10:11:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257901+0000", 2024-09-13T10:11:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9672000000000001e-05 2024-09-13T10:11:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257918+0000", 2024-09-13T10:11:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6407000000000001e-05 2024-09-13T10:11:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.258034+0000", 2024-09-13T10:11:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011616399999999999 2024-09-13T10:11:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.258174+0000", 2024-09-13T10:11:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014034 2024-09-13T10:11:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.265104+0000", 2024-09-13T10:11:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0069297889999999996 2024-09-13T10:11:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.266805+0000", 2024-09-13T10:11:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001700685 2024-09-13T10:11:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267613+0000", 2024-09-13T10:11:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00080784300000000004 2024-09-13T10:11:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.268164+0000", 2024-09-13T10:11:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00055089600000000005 2024-09-13T10:11:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.268274+0000", 2024-09-13T10:11:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110927 2024-09-13T10:11:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.268363+0000", 2024-09-13T10:11:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8226000000000005e-05 2024-09-13T10:11:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:310 4.3 4:c426503f:::smithi19530714-11:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:11:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.6513447639999992, 2024-09-13T10:11:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013782279999999999, 2024-09-13T10:11:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 310 2024-09-13T10:11:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:11:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:11:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273138+0000", 2024-09-13T10:11:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8459999999999999e-06 2024-09-13T10:11:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273150+0000", 2024-09-13T10:11:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2133999999999999e-05 2024-09-13T10:11:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273175+0000", 2024-09-13T10:11:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5321999999999999e-05 2024-09-13T10:11:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273192+0000", 2024-09-13T10:11:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.681e-05 2024-09-13T10:11:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273314+0000", 2024-09-13T10:11:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012228 2024-09-13T10:11:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273474+0000", 2024-09-13T10:11:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016019799999999999 2024-09-13T10:11:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.284136+0000", 2024-09-13T10:11:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010661761000000001 2024-09-13T10:11:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286085+0000", 2024-09-13T10:11:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019484929999999999 2024-09-13T10:11:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286111+0000", 2024-09-13T10:11:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5834e-05 2024-09-13T10:11:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286721+0000", 2024-09-13T10:11:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00061038599999999998 2024-09-13T10:11:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286838+0000", 2024-09-13T10:11:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117064 2024-09-13T10:11:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286917+0000", 2024-09-13T10:11:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9152000000000004e-05 2024-09-13T10:11:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:360 4.3 4:c426503f:::smithi19530714-11:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.294403+0000", 2024-09-13T10:11:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.6300770399999998, 2024-09-13T10:11:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010710917, 2024-09-13T10:11:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 360 2024-09-13T10:11:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294403+0000", 2024-09-13T10:11:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294403+0000", 2024-09-13T10:11:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294404+0000", 2024-09-13T10:11:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.257e-06 2024-09-13T10:11:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294413+0000", 2024-09-13T10:11:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7830000000000004e-06 2024-09-13T10:11:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294436+0000", 2024-09-13T10:11:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3468e-05 2024-09-13T10:11:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294447+0000", 2024-09-13T10:11:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0908e-05 2024-09-13T10:11:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294521+0000", 2024-09-13T10:11:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.4468999999999995e-05 2024-09-13T10:11:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op must wait for map", 2024-09-13T10:11:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294528+0000", 2024-09-13T10:11:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.2750000000000001e-06 2024-09-13T10:11:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.297057+0000", 2024-09-13T10:11:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025294499999999999 2024-09-13T10:11:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.297196+0000", 2024-09-13T10:11:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000138646 2024-09-13T10:11:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.303520+0000", 2024-09-13T10:11:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00632381 2024-09-13T10:11:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.304331+0000", 2024-09-13T10:11:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00081096399999999996 2024-09-13T10:11:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.304707+0000", 2024-09-13T10:11:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037599399999999999 2024-09-13T10:11:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.305024+0000", 2024-09-13T10:11:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00031715999999999998 2024-09-13T10:11:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.305073+0000", 2024-09-13T10:11:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.9635000000000002e-05 2024-09-13T10:11:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.305114+0000", 2024-09-13T10:11:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0098000000000001e-05 2024-09-13T10:11:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:391 4.1 4:8e413aa3:::smithi19530714-42:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.321223+0000", 2024-09-13T10:11:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.6032567570000005, 2024-09-13T10:11:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011654998999999999, 2024-09-13T10:11:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 391 2024-09-13T10:11:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321223+0000", 2024-09-13T10:11:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321223+0000", 2024-09-13T10:11:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321225+0000", 2024-09-13T10:11:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0360000000000001e-06 2024-09-13T10:11:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321236+0000", 2024-09-13T10:11:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1419e-05 2024-09-13T10:11:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321262+0000", 2024-09-13T10:11:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6035000000000001e-05 2024-09-13T10:11:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321280+0000", 2024-09-13T10:11:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8022000000000001e-05 2024-09-13T10:11:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321388+0000", 2024-09-13T10:11:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010752900000000001 2024-09-13T10:11:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321518+0000", 2024-09-13T10:11:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000129901 2024-09-13T10:11:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.329954+0000", 2024-09-13T10:11:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0084364639999999994 2024-09-13T10:11:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.331173+0000", 2024-09-13T10:11:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001218968 2024-09-13T10:11:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.331209+0000", 2024-09-13T10:11:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6018e-05 2024-09-13T10:11:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.332764+0000", 2024-09-13T10:11:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001554466 2024-09-13T10:11:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.332838+0000", 2024-09-13T10:11:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.3779e-05 2024-09-13T10:11:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.332878+0000", 2024-09-13T10:11:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0361999999999997e-05 2024-09-13T10:11:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:993 4.3 4:c5b5ff75:::smithi19530714-20:head [read 0~2319333,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.181393+0000", 2024-09-13T10:11:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 5.7430868720000001, 2024-09-13T10:11:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0095355430000000005, 2024-09-13T10:11:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 993 2024-09-13T10:11:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.181393+0000", 2024-09-13T10:11:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.181393+0000", 2024-09-13T10:11:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.181394+0000", 2024-09-13T10:11:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4300000000000001e-06 2024-09-13T10:11:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.181403+0000", 2024-09-13T10:11:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1460000000000002e-06 2024-09-13T10:11:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.181422+0000", 2024-09-13T10:11:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8916999999999999e-05 2024-09-13T10:11:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.181438+0000", 2024-09-13T10:11:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6146000000000001e-05 2024-09-13T10:11:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.187420+0000", 2024-09-13T10:11:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0059811339999999999 2024-09-13T10:11:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.187640+0000", 2024-09-13T10:11:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022068500000000001 2024-09-13T10:11:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.190928+0000", 2024-09-13T10:11:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0032880850000000001 2024-09-13T10:11:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1006 4.3 4:c5b5ff75:::smithi19530714-20:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:11:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 5.2704315160000004, 2024-09-13T10:11:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.047764635, 2024-09-13T10:11:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1006 2024-09-13T10:11:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:11:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:11:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654049+0000", 2024-09-13T10:11:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5600000000000005e-07 2024-09-13T10:11:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654058+0000", 2024-09-13T10:11:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8200000000000003e-06 2024-09-13T10:11:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654067+0000", 2024-09-13T10:11:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.4599999999999992e-06 2024-09-13T10:11:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654075+0000", 2024-09-13T10:11:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.1070000000000005e-06 2024-09-13T10:11:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654188+0000", 2024-09-13T10:11:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000112734 2024-09-13T10:11:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654395+0000", 2024-09-13T10:11:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000207235 2024-09-13T10:11:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.701813+0000", 2024-09-13T10:11:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.047417523000000003 2024-09-13T10:11:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1013 4.3 4:c426503f:::smithi19530714-11:head [write 477332~766101] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:11:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 5.2631102619999997, 2024-09-13T10:11:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027363838000000001, 2024-09-13T10:11:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1013 2024-09-13T10:11:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:11:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:11:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661374+0000", 2024-09-13T10:11:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.0930000000000002e-06 2024-09-13T10:11:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665828+0000", 2024-09-13T10:11:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044532119999999998 2024-09-13T10:11:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665864+0000", 2024-09-13T10:11:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5979000000000002e-05 2024-09-13T10:11:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665881+0000", 2024-09-13T10:11:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7836e-05 2024-09-13T10:11:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.678976+0000", 2024-09-13T10:11:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.01309448 2024-09-13T10:11:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.679188+0000", 2024-09-13T10:11:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000211945 2024-09-13T10:11:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.679468+0000", 2024-09-13T10:11:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028034000000000002 2024-09-13T10:11:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688233+0000", 2024-09-13T10:11:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0087645369999999993 2024-09-13T10:11:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688388+0000", 2024-09-13T10:11:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015488000000000001 2024-09-13T10:11:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688505+0000", 2024-09-13T10:11:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117114 2024-09-13T10:11:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688612+0000", 2024-09-13T10:11:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010741100000000001 2024-09-13T10:11:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688733+0000", 2024-09-13T10:11:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000121011 2024-09-13T10:11:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1014 4.3 4:c426503f:::smithi19530714-11:head [write 1681102~474010] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:11:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 5.2578368759999998, 2024-09-13T10:11:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022694374, 2024-09-13T10:11:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1014 2024-09-13T10:11:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:11:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:11:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666645+0000", 2024-09-13T10:11:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2970000000000002e-06 2024-09-13T10:11:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667326+0000", 2024-09-13T10:11:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00068073199999999997 2024-09-13T10:11:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667346+0000", 2024-09-13T10:11:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0188999999999999e-05 2024-09-13T10:11:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667361+0000", 2024-09-13T10:11:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5299999999999999e-05 2024-09-13T10:11:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.683468+0000", 2024-09-13T10:11:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016106944000000002 2024-09-13T10:11:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.683714+0000", 2024-09-13T10:11:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024595699999999999 2024-09-13T10:11:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.684132+0000", 2024-09-13T10:11:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00041775800000000001 2024-09-13T10:11:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688771+0000", 2024-09-13T10:11:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0046394540000000003 2024-09-13T10:11:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688920+0000", 2024-09-13T10:11:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014886000000000001 2024-09-13T10:11:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689085+0000", 2024-09-13T10:11:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000164539 2024-09-13T10:11:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689230+0000", 2024-09-13T10:11:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000145394 2024-09-13T10:11:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689337+0000", 2024-09-13T10:11:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010695 2024-09-13T10:11:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1015 4.3 4:c426503f:::smithi19530714-11:head [write 2687912~589420] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:11:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 5.2515201989999998, 2024-09-13T10:11:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.018489873, 2024-09-13T10:11:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1015 2024-09-13T10:11:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:11:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:11:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672962+0000", 2024-09-13T10:11:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.277e-06 2024-09-13T10:11:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673758+0000", 2024-09-13T10:11:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00079599100000000002 2024-09-13T10:11:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673787+0000", 2024-09-13T10:11:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9280000000000001e-05 2024-09-13T10:11:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673801+0000", 2024-09-13T10:11:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4493e-05 2024-09-13T10:11:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.685885+0000", 2024-09-13T10:11:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012083692 2024-09-13T10:11:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.686057+0000", 2024-09-13T10:11:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017133900000000001 2024-09-13T10:11:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.686396+0000", 2024-09-13T10:11:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000339809 2024-09-13T10:11:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691040+0000", 2024-09-13T10:11:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.004643215 2024-09-13T10:11:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691082+0000", 2024-09-13T10:11:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.2499000000000001e-05 2024-09-13T10:11:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691179+0000", 2024-09-13T10:11:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7044000000000001e-05 2024-09-13T10:11:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691324+0000", 2024-09-13T10:11:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014532299999999999 2024-09-13T10:11:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691449+0000", 2024-09-13T10:11:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012491100000000001 2024-09-13T10:11:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1016 4.3 4:c426503f:::smithi19530714-11:head [setxattr _header (56) in=7b,truncate 3277332] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:11:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 5.250651242, 2024-09-13T10:11:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.019820562999999999, 2024-09-13T10:11:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1016 2024-09-13T10:11:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:11:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:11:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673830+0000", 2024-09-13T10:11:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1170000000000001e-06 2024-09-13T10:11:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673835+0000", 2024-09-13T10:11:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5559999999999998e-06 2024-09-13T10:11:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673845+0000", 2024-09-13T10:11:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6330000000000008e-06 2024-09-13T10:11:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673856+0000", 2024-09-13T10:11:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0963e-05 2024-09-13T10:11:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689480+0000", 2024-09-13T10:11:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015624605 2024-09-13T10:11:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689739+0000", 2024-09-13T10:11:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025878700000000001 2024-09-13T10:11:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.690154+0000", 2024-09-13T10:11:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00041533800000000001 2024-09-13T10:11:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691858+0000", 2024-09-13T10:11:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001703811 2024-09-13T10:11:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691880+0000", 2024-09-13T10:11:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1997e-05 2024-09-13T10:11:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693384+0000", 2024-09-13T10:11:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0015040190000000001 2024-09-13T10:11:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693541+0000", 2024-09-13T10:11:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015723899999999999 2024-09-13T10:11:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693649+0000", 2024-09-13T10:11:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010749800000000001 2024-09-13T10:11:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1017 4.3 4:c426503f:::smithi19530714-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:11:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 5.2506112299999996, 2024-09-13T10:11:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020166488999999999, 2024-09-13T10:11:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1017 2024-09-13T10:11:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:11:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:11:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673869+0000", 2024-09-13T10:11:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9699999999999995e-07 2024-09-13T10:11:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673873+0000", 2024-09-13T10:11:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.4970000000000001e-06 2024-09-13T10:11:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673885+0000", 2024-09-13T10:11:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2398999999999999e-05 2024-09-13T10:11:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673893+0000", 2024-09-13T10:11:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7530000000000001e-06 2024-09-13T10:11:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691638+0000", 2024-09-13T10:11:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017745007 2024-09-13T10:11:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691789+0000", 2024-09-13T10:11:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015133400000000001 2024-09-13T10:11:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693731+0000", 2024-09-13T10:11:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019422459999999999 2024-09-13T10:11:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693870+0000", 2024-09-13T10:11:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013820299999999999 2024-09-13T10:11:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.694035+0000", 2024-09-13T10:11:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000165253 2024-09-13T10:11:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.057 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:01.060 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T10:11:01.060 INFO:tasks.rados.rados.0.smithi195.stdout:793: finishing write tid 4 to smithi19530714-14 2024-09-13T10:11:01.060 INFO:tasks.rados.rados.0.smithi195.stdout:793: finishing write tid 5 to smithi19530714-14 2024-09-13T10:11:01.061 INFO:tasks.rados.rados.0.smithi195.stdout:793: oid 14 updating version 0 to 47 2024-09-13T10:11:01.061 INFO:tasks.rados.rados.0.smithi195.stdout:793: oid 14 updating version 47 to 49 2024-09-13T10:11:01.061 INFO:tasks.rados.rados.0.smithi195.stdout:793: oid 14 version 49 is already newer than 48 2024-09-13T10:11:01.061 INFO:tasks.rados.rados.0.smithi195.stdout:793: oid 14 version 49 is already newer than 46 2024-09-13T10:11:01.061 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 49 (ObjNum 103 snap 0 seq_num 103) dirty exists 2024-09-13T10:11:01.061 INFO:tasks.rados.rados.0.smithi195.stdout:793: left oid 14 (ObjNum 103 snap 0 seq_num 103) 2024-09-13T10:11:01.061 INFO:tasks.rados.rados.0.smithi195.stdout:797: got (22) Invalid argument 2024-09-13T10:11:01.061 INFO:tasks.rados.rados.0.smithi195.stdout:796: finishing set_chunk to oid 21 2024-09-13T10:11:01.061 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 21 set_chunk returned error code -95 offset: 1053184 length: 987136 2024-09-13T10:11:01.061 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 0 (ObjNum 69 snap 0 seq_num 69) dirty exists 2024-09-13T10:11:01.061 INFO:tasks.rados.rados.0.smithi195.stdout:798: finishing write tid 1 to smithi19530714-8 2024-09-13T10:11:01.061 INFO:tasks.rados.rados.0.smithi195.stdout:798: finishing write tid 2 to smithi19530714-8 2024-09-13T10:11:01.061 INFO:tasks.rados.rados.0.smithi195.stdout:798: finishing write tid 3 to smithi19530714-8 2024-09-13T10:11:01.062 INFO:tasks.rados.rados.0.smithi195.stdout:791: done (13 left) 2024-09-13T10:11:01.062 INFO:tasks.rados.rados.0.smithi195.stdout:792: done (12 left) 2024-09-13T10:11:01.062 INFO:tasks.rados.rados.0.smithi195.stdout:793: done (11 left) 2024-09-13T10:11:01.062 INFO:tasks.rados.rados.0.smithi195.stdout:794: done (10 left) 2024-09-13T10:11:01.062 INFO:tasks.rados.rados.0.smithi195.stdout:795: done (9 left) 2024-09-13T10:11:01.062 INFO:tasks.rados.rados.0.smithi195.stdout:796: done (8 left) 2024-09-13T10:11:01.062 INFO:tasks.rados.rados.0.smithi195.stdout:797: done (7 left) 2024-09-13T10:11:01.062 INFO:tasks.rados.rados.0.smithi195.stdout:806: read oid 40 snap -1 2024-09-13T10:11:01.062 INFO:tasks.rados.rados.0.smithi195.stdout:806: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-13T10:11:01.062 INFO:tasks.rados.rados.0.smithi195.stdout:807: tier_promote oid 46 2024-09-13T10:11:01.062 INFO:tasks.rados.rados.0.smithi195.stdout:798: finishing write tid 4 to smithi19530714-8 2024-09-13T10:11:01.062 INFO:tasks.rados.rados.0.smithi195.stdout:798: finishing write tid 5 to smithi19530714-8 2024-09-13T10:11:01.062 INFO:tasks.rados.rados.0.smithi195.stdout:798: oid 8 updating version 0 to 6 2024-09-13T10:11:01.062 INFO:tasks.rados.rados.0.smithi195.stdout:798: oid 8 updating version 6 to 7 2024-09-13T10:11:01.063 INFO:tasks.rados.rados.0.smithi195.stdout:798: oid 8 updating version 7 to 8 2024-09-13T10:11:01.063 INFO:tasks.rados.rados.0.smithi195.stdout:798: oid 8 updating version 8 to 9 2024-09-13T10:11:01.063 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 9 (ObjNum 104 snap 0 seq_num 104) dirty exists 2024-09-13T10:11:01.063 INFO:tasks.rados.rados.0.smithi195.stdout:798: left oid 8 (ObjNum 104 snap 0 seq_num 104) 2024-09-13T10:11:01.063 INFO:tasks.rados.rados.0.smithi195.stdout:800: expect (ObjNum 80 snap 0 seq_num 80) 2024-09-13T10:11:01.063 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 385174 2024-09-13T10:11:01.063 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:01.057+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:01.125 INFO:tasks.rados.rados.0.smithi195.stdout:799: got (0) Success 2024-09-13T10:11:01.125 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 46 (ObjNum 71 snap 0 seq_num 71) dirty exists 2024-09-13T10:11:01.125 INFO:tasks.rados.rados.0.smithi195.stdout:801: finishing write tid 1 to smithi19530714-19 2024-09-13T10:11:01.125 INFO:tasks.rados.rados.0.smithi195.stdout:801: finishing write tid 2 to smithi19530714-19 2024-09-13T10:11:01.125 INFO:tasks.rados.rados.0.smithi195.stdout:801: finishing write tid 3 to smithi19530714-19 2024-09-13T10:11:01.125 INFO:tasks.rados.rados.0.smithi195.stdout:801: finishing write tid 4 to smithi19530714-19 2024-09-13T10:11:01.125 INFO:tasks.rados.rados.0.smithi195.stdout:801: finishing write tid 5 to smithi19530714-19 2024-09-13T10:11:01.125 INFO:tasks.rados.rados.0.smithi195.stdout:801: oid 19 updating version 0 to 41 2024-09-13T10:11:01.125 INFO:tasks.rados.rados.0.smithi195.stdout:801: oid 19 updating version 41 to 43 2024-09-13T10:11:01.125 INFO:tasks.rados.rados.0.smithi195.stdout:801: oid 19 version 43 is already newer than 42 2024-09-13T10:11:01.126 INFO:tasks.rados.rados.0.smithi195.stdout:801: oid 19 updating version 43 to 44 2024-09-13T10:11:01.126 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 44 (ObjNum 105 snap 0 seq_num 105) dirty exists 2024-09-13T10:11:01.126 INFO:tasks.rados.rados.0.smithi195.stdout:801: left oid 19 (ObjNum 105 snap 0 seq_num 105) 2024-09-13T10:11:01.126 INFO:tasks.rados.rados.0.smithi195.stdout:802: finishing write tid 1 to smithi19530714-49 2024-09-13T10:11:01.126 INFO:tasks.rados.rados.0.smithi195.stdout:802: finishing write tid 2 to smithi19530714-49 2024-09-13T10:11:01.126 INFO:tasks.rados.rados.0.smithi195.stdout:802: finishing write tid 3 to smithi19530714-49 2024-09-13T10:11:01.126 INFO:tasks.rados.rados.0.smithi195.stdout:802: finishing write tid 4 to smithi19530714-49 2024-09-13T10:11:01.126 INFO:tasks.rados.rados.0.smithi195.stdout:802: finishing write tid 5 to smithi19530714-49 2024-09-13T10:11:01.126 INFO:tasks.rados.rados.0.smithi195.stdout:802: oid 49 updating version 0 to 61 2024-09-13T10:11:01.126 INFO:tasks.rados.rados.0.smithi195.stdout:802: oid 49 version 61 is already newer than 60 2024-09-13T10:11:01.126 INFO:tasks.rados.rados.0.smithi195.stdout:802: oid 49 version 61 is already newer than 59 2024-09-13T10:11:01.126 INFO:tasks.rados.rados.0.smithi195.stdout:802: oid 49 version 61 is already newer than 58 2024-09-13T10:11:01.126 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 61 (ObjNum 106 snap 0 seq_num 106) dirty exists 2024-09-13T10:11:01.127 INFO:tasks.rados.rados.0.smithi195.stdout:802: left oid 49 (ObjNum 106 snap 0 seq_num 106) 2024-09-13T10:11:01.127 INFO:tasks.rados.rados.0.smithi195.stdout:804: finishing write tid 1 to smithi19530714-7 2024-09-13T10:11:01.127 INFO:tasks.rados.rados.0.smithi195.stdout:804: finishing write tid 2 to smithi19530714-7 2024-09-13T10:11:01.127 INFO:tasks.rados.rados.0.smithi195.stdout:805: got (22) Invalid argument 2024-09-13T10:11:01.127 INFO:tasks.rados.rados.0.smithi195.stdout:804: finishing write tid 3 to smithi19530714-7 2024-09-13T10:11:01.127 INFO:tasks.rados.rados.0.smithi195.stdout:804: finishing write tid 4 to smithi19530714-7 2024-09-13T10:11:01.127 INFO:tasks.rados.rados.0.smithi195.stdout:804: finishing write tid 5 to smithi19530714-7 2024-09-13T10:11:01.127 INFO:tasks.rados.rados.0.smithi195.stdout:804: oid 7 updating version 0 to 35 2024-09-13T10:11:01.127 INFO:tasks.rados.rados.0.smithi195.stdout:804: oid 7 updating version 35 to 38 2024-09-13T10:11:01.127 INFO:tasks.rados.rados.0.smithi195.stdout:804: oid 7 version 38 is already newer than 36 2024-09-13T10:11:01.127 INFO:tasks.rados.rados.0.smithi195.stdout:804: oid 7 version 38 is already newer than 37 2024-09-13T10:11:01.127 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 38 (ObjNum 108 snap 0 seq_num 108) dirty exists 2024-09-13T10:11:01.127 INFO:tasks.rados.rados.0.smithi195.stdout:804: left oid 7 (ObjNum 108 snap 0 seq_num 108) 2024-09-13T10:11:01.128 INFO:tasks.rados.rados.0.smithi195.stdout:807: got (2) No such file or directory 2024-09-13T10:11:01.128 INFO:tasks.rados.rados.0.smithi195.stdout:807: got expected ENOENT (src dne) 2024-09-13T10:11:01.128 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 27 (ObjNum 4 snap 0 seq_num 304) clean dne 2024-09-13T10:11:01.128 INFO:tasks.rados.rados.0.smithi195.stdout:806: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-13T10:11:01.159 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:01.160+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:01.164 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2162795 2024-09-13T10:11:01.191 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:01.191 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:01.191 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:11:01.192 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:01.197 INFO:tasks.rados.rados.0.smithi195.stdout:798: done (8 left) 2024-09-13T10:11:01.197 INFO:tasks.rados.rados.0.smithi195.stdout:799: done (7 left) 2024-09-13T10:11:01.197 INFO:tasks.rados.rados.0.smithi195.stdout:800: done (6 left) 2024-09-13T10:11:01.197 INFO:tasks.rados.rados.0.smithi195.stdout:801: done (5 left) 2024-09-13T10:11:01.198 INFO:tasks.rados.rados.0.smithi195.stdout:802: done (4 left) 2024-09-13T10:11:01.198 INFO:tasks.rados.rados.0.smithi195.stdout:804: done (3 left) 2024-09-13T10:11:01.198 INFO:tasks.rados.rados.0.smithi195.stdout:805: done (2 left) 2024-09-13T10:11:01.198 INFO:tasks.rados.rados.0.smithi195.stdout:806: done (1 left) 2024-09-13T10:11:01.198 INFO:tasks.rados.rados.0.smithi195.stdout:807: done (0 left) 2024-09-13T10:11:01.198 INFO:tasks.rados.rados.0.smithi195.stdout:808: read oid 38 snap -1 2024-09-13T10:11:01.198 INFO:tasks.rados.rados.0.smithi195.stdout:808: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:11:01.198 INFO:tasks.rados.rados.0.smithi195.stdout:809: write (excl) oid 44 current snap is 0 2024-09-13T10:11:01.198 INFO:tasks.rados.rados.0.smithi195.stdout:809: seq_num 109 ranges {631963=631083,2061949=560019,3243812=504767} 2024-09-13T10:11:01.208 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T10:11:01.218 INFO:tasks.rados.rados.0.smithi195.stdout:809: writing smithi19530714-44 from 631963 to 1263046 tid 1 2024-09-13T10:11:01.240 INFO:tasks.rados.rados.0.smithi195.stdout:809: writing smithi19530714-44 from 2061949 to 2621968 tid 2 2024-09-13T10:11:01.256 INFO:tasks.rados.rados.0.smithi195.stdout:809: writing smithi19530714-44 from 3243812 to 3748579 tid 3 2024-09-13T10:11:01.257 INFO:tasks.rados.rados.0.smithi195.stdout:810: tier_evict oid 10 2024-09-13T10:11:01.257 INFO:tasks.rados.rados.0.smithi195.stdout:810: tier_evict oid 10 snap -1 2024-09-13T10:11:01.257 INFO:tasks.rados.rados.0.smithi195.stdout:808: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:11:01.292 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:01.300 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:01.300+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:01.340 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:01.361 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:01.357+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:01.378 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:01.378 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:01.378 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:11:01.378 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:11:01.378 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:01.388 INFO:tasks.rados.rados.0.smithi195.stdout:809: finishing write tid 1 to smithi19530714-44 2024-09-13T10:11:01.388 INFO:tasks.rados.rados.0.smithi195.stdout:809: finishing write tid 2 to smithi19530714-44 2024-09-13T10:11:01.388 INFO:tasks.rados.rados.0.smithi195.stdout:810: got (22) Invalid argument 2024-09-13T10:11:01.389 INFO:tasks.rados.rados.0.smithi195.stdout:809: finishing write tid 3 to smithi19530714-44 2024-09-13T10:11:01.389 INFO:tasks.rados.rados.0.smithi195.stdout:809: finishing write tid 4 to smithi19530714-44 2024-09-13T10:11:01.389 INFO:tasks.rados.rados.0.smithi195.stdout:809: finishing write tid 5 to smithi19530714-44 2024-09-13T10:11:01.389 INFO:tasks.rados.rados.0.smithi195.stdout:809: oid 44 updating version 0 to 36 2024-09-13T10:11:01.389 INFO:tasks.rados.rados.0.smithi195.stdout:809: oid 44 version 36 is already newer than 34 2024-09-13T10:11:01.389 INFO:tasks.rados.rados.0.smithi195.stdout:809: oid 44 version 36 is already newer than 33 2024-09-13T10:11:01.389 INFO:tasks.rados.rados.0.smithi195.stdout:809: oid 44 version 36 is already newer than 35 2024-09-13T10:11:01.389 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 36 (ObjNum 109 snap 0 seq_num 109) dirty exists 2024-09-13T10:11:01.389 INFO:tasks.rados.rados.0.smithi195.stdout:809: left oid 44 (ObjNum 109 snap 0 seq_num 109) 2024-09-13T10:11:01.389 INFO:tasks.rados.rados.0.smithi195.stdout:808: done (2 left) 2024-09-13T10:11:01.389 INFO:tasks.rados.rados.0.smithi195.stdout:809: done (1 left) 2024-09-13T10:11:01.389 INFO:tasks.rados.rados.0.smithi195.stdout:810: done (0 left) 2024-09-13T10:11:01.390 INFO:tasks.rados.rados.0.smithi195.stdout:811: read oid 16 snap -1 2024-09-13T10:11:01.390 INFO:tasks.rados.rados.0.smithi195.stdout:811: expect deleted 2024-09-13T10:11:01.390 INFO:tasks.rados.rados.0.smithi195.stdout:812: read oid 28 snap -1 2024-09-13T10:11:01.390 INFO:tasks.rados.rados.0.smithi195.stdout:812: expect (ObjNum 99 snap 0 seq_num 99) 2024-09-13T10:11:01.390 INFO:tasks.rados.rados.0.smithi195.stdout:813: read oid 44 snap -1 2024-09-13T10:11:01.390 INFO:tasks.rados.rados.0.smithi195.stdout:813: expect (ObjNum 109 snap 0 seq_num 109) 2024-09-13T10:11:01.390 INFO:tasks.rados.rados.0.smithi195.stdout:814: read oid 3 snap -1 2024-09-13T10:11:01.390 INFO:tasks.rados.rados.0.smithi195.stdout:814: expect (ObjNum 60 snap 0 seq_num 60) 2024-09-13T10:11:01.390 INFO:tasks.rados.rados.0.smithi195.stdout:815: write (excl) oid 4 current snap is 0 2024-09-13T10:11:01.390 INFO:tasks.rados.rados.0.smithi195.stdout:815: seq_num 110 ranges {648770=706332,2120431=420030,3271235=432392} 2024-09-13T10:11:01.393 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T10:11:01.405 INFO:tasks.rados.rados.0.smithi195.stdout:815: writing smithi19530714-4 from 648770 to 1355102 tid 1 2024-09-13T10:11:01.419 INFO:tasks.rados.rados.0.smithi195.stdout:815: writing smithi19530714-4 from 2120431 to 2540461 tid 2 2024-09-13T10:11:01.432 INFO:tasks.rados.rados.0.smithi195.stdout:815: writing smithi19530714-4 from 3271235 to 3703627 tid 3 2024-09-13T10:11:01.433 INFO:tasks.rados.rados.0.smithi195.stdout:816: write (excl) oid 22 current snap is 0 2024-09-13T10:11:01.433 INFO:tasks.rados.rados.0.smithi195.stdout:816: seq_num 111 ranges {665577=781581,2062560=563688,3182305=643664} 2024-09-13T10:11:01.447 INFO:tasks.rados.rados.0.smithi195.stdout:816: writing smithi19530714-22 from 665577 to 1447158 tid 1 2024-09-13T10:11:01.458 INFO:tasks.rados.rados.0.smithi195.stdout:816: writing smithi19530714-22 from 2062560 to 2626248 tid 2 2024-09-13T10:11:01.462 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:01.461+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:01.467 INFO:tasks.rados.rados.0.smithi195.stdout:816: writing smithi19530714-22 from 3182305 to 3825969 tid 3 2024-09-13T10:11:01.467 INFO:tasks.rados.rados.0.smithi195.stdout:817: write oid 27 current snap is 0 2024-09-13T10:11:01.467 INFO:tasks.rados.rados.0.smithi195.stdout:817: seq_num 112 ranges {682384=456830,1721042=423699,2809728=454936,3731357=151027} 2024-09-13T10:11:01.474 INFO:tasks.rados.rados.0.smithi195.stdout:817: writing smithi19530714-27 from 682384 to 1139214 tid 1 2024-09-13T10:11:01.481 INFO:tasks.rados.rados.0.smithi195.stdout:817: writing smithi19530714-27 from 1721042 to 2144741 tid 2 2024-09-13T10:11:01.488 INFO:tasks.rados.rados.0.smithi195.stdout:817: writing smithi19530714-27 from 2809728 to 3264664 tid 3 2024-09-13T10:11:01.491 INFO:tasks.rados.rados.0.smithi195.stdout:817: writing smithi19530714-27 from 3731357 to 3882384 tid 4 2024-09-13T10:11:01.491 INFO:tasks.rados.rados.0.smithi195.stdout:818: tier_promote oid 20 2024-09-13T10:11:01.491 INFO:tasks.rados.rados.0.smithi195.stdout:819: tier_evict oid 24 2024-09-13T10:11:01.491 INFO:tasks.rados.rados.0.smithi195.stdout:819: tier_evict oid 24 snap -1 2024-09-13T10:11:01.492 INFO:tasks.rados.rados.0.smithi195.stdout:814: expect (ObjNum 60 snap 0 seq_num 60) 2024-09-13T10:11:01.563 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:01.560+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:01.566 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:01.566 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:11:01.566 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:11:01.566 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:11:01.566 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.566 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:105 4.5 4:a7549961:::smithi19530714-27:head [write 436982~705533,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.566 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.660565+0000", 2024-09-13T10:11:01.566 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.904119074, 2024-09-13T10:11:01.566 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012929889999999999, 2024-09-13T10:11:01.566 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.567 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.567 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.567 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.567 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.567 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.567 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 105 2024-09-13T10:11:01.567 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.567 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.567 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.567 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.567 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.660565+0000", 2024-09-13T10:11:01.567 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.567 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.567 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.568 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.568 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.660565+0000", 2024-09-13T10:11:01.568 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.568 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.568 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.568 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.568 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.660599+0000", 2024-09-13T10:11:01.568 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3874999999999997e-05 2024-09-13T10:11:01.568 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.568 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.568 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.568 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.661419+0000", 2024-09-13T10:11:01.568 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00081945599999999998 2024-09-13T10:11:01.569 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.569 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.569 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.569 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.661442+0000", 2024-09-13T10:11:01.569 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3876e-05 2024-09-13T10:11:01.569 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.569 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.569 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.569 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.661456+0000", 2024-09-13T10:11:01.569 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3679999999999999e-05 2024-09-13T10:11:01.569 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.569 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.569 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.569 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.661579+0000", 2024-09-13T10:11:01.570 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012293699999999999 2024-09-13T10:11:01.570 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.570 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.570 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.570 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.661774+0000", 2024-09-13T10:11:01.570 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000195045 2024-09-13T10:11:01.570 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.570 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.570 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,10", 2024-09-13T10:11:01.570 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.662045+0000", 2024-09-13T10:11:01.570 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000270599 2024-09-13T10:11:01.570 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.570 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.571 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.571 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.665296+0000", 2024-09-13T10:11:01.571 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003251313 2024-09-13T10:11:01.571 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.571 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.571 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.571 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.666469+0000", 2024-09-13T10:11:01.571 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011733900000000001 2024-09-13T10:11:01.571 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.571 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.571 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.571 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.673231+0000", 2024-09-13T10:11:01.571 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0067615360000000003 2024-09-13T10:11:01.572 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.572 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.572 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.572 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.673399+0000", 2024-09-13T10:11:01.572 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000167756 2024-09-13T10:11:01.572 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.572 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.572 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.572 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.673495+0000", 2024-09-13T10:11:01.572 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6427000000000004e-05 2024-09-13T10:11:01.572 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.572 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.572 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.572 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.573 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.573 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:251 4.5 4:a71ca386:::smithi19530714-2:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.573 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:11:01.573 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.446695765999999, 2024-09-13T10:11:01.573 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041813326999999997, 2024-09-13T10:11:01.573 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.573 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.573 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.573 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.573 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.573 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.573 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 251 2024-09-13T10:11:01.573 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.573 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.574 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.574 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.574 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:11:01.574 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.574 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.574 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.574 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:11:01.575 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.575 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.575 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.575 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117990+0000", 2024-09-13T10:11:01.575 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.203e-06 2024-09-13T10:11:01.575 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.575 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.575 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117997+0000", 2024-09-13T10:11:01.575 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9750000000000001e-06 2024-09-13T10:11:01.575 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.576 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.576 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118009+0000", 2024-09-13T10:11:01.576 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2478e-05 2024-09-13T10:11:01.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.576 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.576 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.576 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118017+0000", 2024-09-13T10:11:01.576 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6909999999999997e-06 2024-09-13T10:11:01.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.576 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.576 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.576 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118106+0000", 2024-09-13T10:11:01.576 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.9331000000000002e-05 2024-09-13T10:11:01.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.577 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.577 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.577 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118184+0000", 2024-09-13T10:11:01.577 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7448999999999997e-05 2024-09-13T10:11:01.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.577 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.577 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,10", 2024-09-13T10:11:01.577 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158604+0000", 2024-09-13T10:11:01.577 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.040419954000000001 2024-09-13T10:11:01.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.577 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.577 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.578 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159306+0000", 2024-09-13T10:11:01.578 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00070229299999999999 2024-09-13T10:11:01.578 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.578 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.578 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.578 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159539+0000", 2024-09-13T10:11:01.578 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023353600000000001 2024-09-13T10:11:01.578 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.578 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.578 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.578 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159696+0000", 2024-09-13T10:11:01.578 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015616 2024-09-13T10:11:01.578 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.578 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.579 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.579 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159754+0000", 2024-09-13T10:11:01.579 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8513e-05 2024-09-13T10:11:01.579 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.579 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.579 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.579 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159802+0000", 2024-09-13T10:11:01.579 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.7744000000000002e-05 2024-09-13T10:11:01.579 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.579 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.579 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.579 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.579 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.580 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:270 4.5 4:a59e88f7:::smithi19530714-21:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.580 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:11:01.580 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.33864964, 2024-09-13T10:11:01.580 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041472594000000002, 2024-09-13T10:11:01.580 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.580 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.580 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.580 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.580 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.580 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.580 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 270 2024-09-13T10:11:01.580 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.580 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.581 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.581 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:11:01.581 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.581 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.581 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.581 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:11:01.581 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.581 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.581 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.582 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226037+0000", 2024-09-13T10:11:01.582 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6560000000000001e-06 2024-09-13T10:11:01.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.582 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.582 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226048+0000", 2024-09-13T10:11:01.582 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0893000000000001e-05 2024-09-13T10:11:01.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.582 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.582 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226079+0000", 2024-09-13T10:11:01.582 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0412999999999998e-05 2024-09-13T10:11:01.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.583 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.583 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226106+0000", 2024-09-13T10:11:01.583 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7160000000000001e-05 2024-09-13T10:11:01.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.583 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.583 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226234+0000", 2024-09-13T10:11:01.583 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000128393 2024-09-13T10:11:01.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.584 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.584 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226388+0000", 2024-09-13T10:11:01.584 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015424300000000001 2024-09-13T10:11:01.584 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.584 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.584 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,10", 2024-09-13T10:11:01.584 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262647+0000", 2024-09-13T10:11:01.584 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036258460999999999 2024-09-13T10:11:01.584 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.584 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.584 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.584 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264690+0000", 2024-09-13T10:11:01.584 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020429889999999998 2024-09-13T10:11:01.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.585 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.585 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.585 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.266720+0000", 2024-09-13T10:11:01.585 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002029669 2024-09-13T10:11:01.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.585 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.585 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.585 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267350+0000", 2024-09-13T10:11:01.585 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00063076699999999996 2024-09-13T10:11:01.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.585 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.586 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.586 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267409+0000", 2024-09-13T10:11:01.586 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8366000000000003e-05 2024-09-13T10:11:01.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.586 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.586 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267507+0000", 2024-09-13T10:11:01.586 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.8584000000000003e-05 2024-09-13T10:11:01.586 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.586 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.586 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.586 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:271 4.5 4:a3098556:::smithi19530714-22:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.587 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.226458+0000", 2024-09-13T10:11:01.587 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.338226767, 2024-09-13T10:11:01.587 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.01416453, 2024-09-13T10:11:01.587 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.587 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.587 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.587 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.587 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.587 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.587 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 271 2024-09-13T10:11:01.587 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.587 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.587 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.587 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.588 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226458+0000", 2024-09-13T10:11:01.588 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.588 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.588 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.588 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226458+0000", 2024-09-13T10:11:01.588 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.588 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.588 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.588 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226460+0000", 2024-09-13T10:11:01.588 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3029999999999998e-06 2024-09-13T10:11:01.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.589 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.589 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226472+0000", 2024-09-13T10:11:01.589 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.259e-05 2024-09-13T10:11:01.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.589 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.589 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226515+0000", 2024-09-13T10:11:01.589 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.2818999999999998e-05 2024-09-13T10:11:01.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.589 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.589 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226552+0000", 2024-09-13T10:11:01.589 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6495000000000002e-05 2024-09-13T10:11:01.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.590 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.590 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.590 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226773+0000", 2024-09-13T10:11:01.590 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000221105 2024-09-13T10:11:01.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.590 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.590 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.590 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226889+0000", 2024-09-13T10:11:01.590 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011633699999999999 2024-09-13T10:11:01.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.590 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.590 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,10", 2024-09-13T10:11:01.591 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.235481+0000", 2024-09-13T10:11:01.591 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0085921780000000007 2024-09-13T10:11:01.591 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.591 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.591 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.591 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.238047+0000", 2024-09-13T10:11:01.591 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025656229999999999 2024-09-13T10:11:01.591 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.591 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.591 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.591 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.238494+0000", 2024-09-13T10:11:01.591 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00044702899999999999 2024-09-13T10:11:01.591 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.592 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.592 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.240335+0000", 2024-09-13T10:11:01.592 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001841224 2024-09-13T10:11:01.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.592 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.592 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.240458+0000", 2024-09-13T10:11:01.592 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012293100000000001 2024-09-13T10:11:01.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.592 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.592 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.240622+0000", 2024-09-13T10:11:01.593 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016389599999999999 2024-09-13T10:11:01.593 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.593 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.593 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.593 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.593 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:273 4.4 4:244eab67:::smithi19530714-24:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.593 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.228868+0000", 2024-09-13T10:11:01.593 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.335816454, 2024-09-13T10:11:01.593 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013467801999999999, 2024-09-13T10:11:01.593 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.593 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.593 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.594 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.594 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.594 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.594 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 273 2024-09-13T10:11:01.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.594 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.594 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.594 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.594 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228868+0000", 2024-09-13T10:11:01.594 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.594 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.594 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.594 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228868+0000", 2024-09-13T10:11:01.595 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.595 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.595 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.595 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228870+0000", 2024-09-13T10:11:01.595 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.103e-06 2024-09-13T10:11:01.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.595 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.595 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.595 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228882+0000", 2024-09-13T10:11:01.595 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1637000000000001e-05 2024-09-13T10:11:01.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.596 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.596 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228908+0000", 2024-09-13T10:11:01.596 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6539000000000001e-05 2024-09-13T10:11:01.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.596 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.596 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228925+0000", 2024-09-13T10:11:01.596 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6759999999999999e-05 2024-09-13T10:11:01.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.596 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.597 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229072+0000", 2024-09-13T10:11:01.597 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000147553 2024-09-13T10:11:01.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.597 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.597 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.597 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229291+0000", 2024-09-13T10:11:01.597 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021820800000000001 2024-09-13T10:11:01.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.597 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.597 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:01.600 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.238661+0000", 2024-09-13T10:11:01.600 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0093704249999999999 2024-09-13T10:11:01.600 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.600 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.600 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.600 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.240860+0000", 2024-09-13T10:11:01.600 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002198471 2024-09-13T10:11:01.600 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.600 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.601 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.601 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.240993+0000", 2024-09-13T10:11:01.601 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013337399999999999 2024-09-13T10:11:01.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.601 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.601 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.601 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.242106+0000", 2024-09-13T10:11:01.601 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001112692 2024-09-13T10:11:01.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.601 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.601 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.601 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.242213+0000", 2024-09-13T10:11:01.601 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000106895 2024-09-13T10:11:01.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.602 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.602 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.602 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.242336+0000", 2024-09-13T10:11:01.602 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012314499999999999 2024-09-13T10:11:01.602 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.602 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.602 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.602 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.602 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:275 4.2 4:4ac406d9:::smithi19530714-26:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.602 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:11:01.602 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.334902576999999, 2024-09-13T10:11:01.603 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024603505000000001, 2024-09-13T10:11:01.603 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.603 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.603 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.603 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.603 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.603 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.603 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 275 2024-09-13T10:11:01.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.603 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.603 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.603 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.603 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:11:01.604 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.604 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.604 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:11:01.604 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.604 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.604 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229783+0000", 2024-09-13T10:11:01.604 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.44e-06 2024-09-13T10:11:01.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.605 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.605 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229793+0000", 2024-09-13T10:11:01.605 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0163000000000001e-05 2024-09-13T10:11:01.605 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.605 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.605 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.605 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229814+0000", 2024-09-13T10:11:01.605 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0564000000000002e-05 2024-09-13T10:11:01.605 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.605 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.605 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.605 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229831+0000", 2024-09-13T10:11:01.605 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7142e-05 2024-09-13T10:11:01.605 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.606 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.606 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.606 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229949+0000", 2024-09-13T10:11:01.606 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000118014 2024-09-13T10:11:01.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.606 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.606 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.606 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230081+0000", 2024-09-13T10:11:01.606 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013181299999999999 2024-09-13T10:11:01.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.606 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.606 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,14", 2024-09-13T10:11:01.606 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.236541+0000", 2024-09-13T10:11:01.607 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0064597680000000003 2024-09-13T10:11:01.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.607 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.607 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.607 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.238773+0000", 2024-09-13T10:11:01.607 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022326680000000002 2024-09-13T10:11:01.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.607 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.607 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.607 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.241978+0000", 2024-09-13T10:11:01.607 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00320455 2024-09-13T10:11:01.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.607 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.608 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.608 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254159+0000", 2024-09-13T10:11:01.608 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012181149 2024-09-13T10:11:01.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.608 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.608 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254259+0000", 2024-09-13T10:11:01.608 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000100184 2024-09-13T10:11:01.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.608 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.608 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254385+0000", 2024-09-13T10:11:01.608 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012605 2024-09-13T10:11:01.609 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.609 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.609 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.609 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.609 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:270 4.5 4:a59e88f7:::smithi19530714-21:head [assert-version v9,set-chunk] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.609 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:11:01.609 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.331248147, 2024-09-13T10:11:01.609 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033985986000000003, 2024-09-13T10:11:01.609 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.609 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.609 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "waiting for ondisk", 2024-09-13T10:11:01.609 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.609 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.610 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.610 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 270 2024-09-13T10:11:01.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.610 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.610 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.610 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.610 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:11:01.610 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.610 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.610 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.610 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:11:01.610 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.611 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.611 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.611 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.611 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233438+0000", 2024-09-13T10:11:01.611 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.187e-06 2024-09-13T10:11:01.611 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.611 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.611 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.611 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233449+0000", 2024-09-13T10:11:01.611 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0926e-05 2024-09-13T10:11:01.611 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.611 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.611 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.611 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233477+0000", 2024-09-13T10:11:01.612 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7515999999999999e-05 2024-09-13T10:11:01.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.612 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.612 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.612 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233499+0000", 2024-09-13T10:11:01.612 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1719999999999999e-05 2024-09-13T10:11:01.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.612 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.612 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.612 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233670+0000", 2024-09-13T10:11:01.612 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000171146 2024-09-13T10:11:01.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.612 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.613 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for blocked object", 2024-09-13T10:11:01.613 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233823+0000", 2024-09-13T10:11:01.613 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015309300000000001 2024-09-13T10:11:01.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.613 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.613 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.263191+0000", 2024-09-13T10:11:01.613 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029368157999999998 2024-09-13T10:11:01.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.613 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for ondisk", 2024-09-13T10:11:01.613 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.263301+0000", 2024-09-13T10:11:01.613 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110214 2024-09-13T10:11:01.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.614 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.614 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.614 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267422+0000", 2024-09-13T10:11:01.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0041210259999999999 2024-09-13T10:11:01.614 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.614 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.614 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.614 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.614 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:275 4.2 4:4ac406d9:::smithi19530714-26:head [assert-version v3,set-chunk] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.614 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.233895+0000", 2024-09-13T10:11:01.614 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.330789761, 2024-09-13T10:11:01.615 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020388462, 2024-09-13T10:11:01.615 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.615 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.615 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "waiting for ondisk", 2024-09-13T10:11:01.615 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.615 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.615 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.615 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 275 2024-09-13T10:11:01.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.615 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.615 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.615 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.615 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233895+0000", 2024-09-13T10:11:01.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.616 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.616 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233895+0000", 2024-09-13T10:11:01.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.616 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.616 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233896+0000", 2024-09-13T10:11:01.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3230000000000001e-06 2024-09-13T10:11:01.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.616 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233904+0000", 2024-09-13T10:11:01.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6930000000000007e-06 2024-09-13T10:11:01.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.617 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233921+0000", 2024-09-13T10:11:01.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7739000000000001e-05 2024-09-13T10:11:01.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.617 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233937+0000", 2024-09-13T10:11:01.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5444e-05 2024-09-13T10:11:01.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.618 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.618 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.618 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234067+0000", 2024-09-13T10:11:01.618 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013029 2024-09-13T10:11:01.618 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.618 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.618 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for blocked object", 2024-09-13T10:11:01.618 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234173+0000", 2024-09-13T10:11:01.618 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010631 2024-09-13T10:11:01.618 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.618 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.618 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.618 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.237380+0000", 2024-09-13T10:11:01.619 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0032067369999999999 2024-09-13T10:11:01.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.619 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.619 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for ondisk", 2024-09-13T10:11:01.619 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.237533+0000", 2024-09-13T10:11:01.619 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000152748 2024-09-13T10:11:01.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.619 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.619 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.619 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254283+0000", 2024-09-13T10:11:01.619 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016750178000000001 2024-09-13T10:11:01.619 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.619 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.619 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.620 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.620 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:277 4.4 4:20c7201f:::smithi19530714-28:head [assert-version v9,set-chunk] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.620 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:11:01.620 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.330642062000001, 2024-09-13T10:11:01.620 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.042430906999999997, 2024-09-13T10:11:01.620 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.620 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.620 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.620 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.620 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.620 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.620 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 277 2024-09-13T10:11:01.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.621 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.621 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.621 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.621 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:11:01.621 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.621 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.621 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.621 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:11:01.621 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.621 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.622 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.622 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234043+0000", 2024-09-13T10:11:01.622 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.13e-06 2024-09-13T10:11:01.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.622 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.622 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.622 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234050+0000", 2024-09-13T10:11:01.622 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.263e-06 2024-09-13T10:11:01.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.622 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.622 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.622 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234065+0000", 2024-09-13T10:11:01.622 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5207e-05 2024-09-13T10:11:01.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.623 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.623 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.623 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234078+0000", 2024-09-13T10:11:01.623 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3643e-05 2024-09-13T10:11:01.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.623 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.623 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.623 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.235383+0000", 2024-09-13T10:11:01.623 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0013048269999999999 2024-09-13T10:11:01.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.623 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.623 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.623 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.235604+0000", 2024-09-13T10:11:01.624 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022116900000000001 2024-09-13T10:11:01.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.624 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:01.624 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.271455+0000", 2024-09-13T10:11:01.624 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035850173999999999 2024-09-13T10:11:01.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.624 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.624 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273083+0000", 2024-09-13T10:11:01.624 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0016287649999999999 2024-09-13T10:11:01.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.625 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.625 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.275756+0000", 2024-09-13T10:11:01.625 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0026721140000000002 2024-09-13T10:11:01.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.625 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.625 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.625 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276255+0000", 2024-09-13T10:11:01.625 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00049961600000000004 2024-09-13T10:11:01.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.625 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.625 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.625 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276401+0000", 2024-09-13T10:11:01.625 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014584099999999999 2024-09-13T10:11:01.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.626 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.626 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276473+0000", 2024-09-13T10:11:01.626 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2157999999999996e-05 2024-09-13T10:11:01.626 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.626 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.626 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.626 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:285 4.2 4:4a24ecac:::smithi19530714-36:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.626 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.245503+0000", 2024-09-13T10:11:01.626 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.319181698, 2024-09-13T10:11:01.626 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015177338, 2024-09-13T10:11:01.627 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.627 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.627 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.627 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.627 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.627 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.627 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 285 2024-09-13T10:11:01.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.627 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.627 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.627 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.627 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.245503+0000", 2024-09-13T10:11:01.627 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.628 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.628 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.628 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.245503+0000", 2024-09-13T10:11:01.628 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.628 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.628 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.628 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.245505+0000", 2024-09-13T10:11:01.628 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2529999999999999e-06 2024-09-13T10:11:01.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.628 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.628 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.629 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.245515+0000", 2024-09-13T10:11:01.629 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0356999999999999e-05 2024-09-13T10:11:01.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.629 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.629 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.245540+0000", 2024-09-13T10:11:01.629 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4411999999999999e-05 2024-09-13T10:11:01.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.629 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.629 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.245555+0000", 2024-09-13T10:11:01.629 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.503e-05 2024-09-13T10:11:01.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.630 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.245656+0000", 2024-09-13T10:11:01.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000101388 2024-09-13T10:11:01.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.630 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.245775+0000", 2024-09-13T10:11:01.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011937199999999999 2024-09-13T10:11:01.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.630 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,14", 2024-09-13T10:11:01.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249845+0000", 2024-09-13T10:11:01.631 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0040691369999999996 2024-09-13T10:11:01.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.631 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.631 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.251668+0000", 2024-09-13T10:11:01.631 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0018239199999999999 2024-09-13T10:11:01.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.631 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.631 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.252433+0000", 2024-09-13T10:11:01.631 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00076494400000000004 2024-09-13T10:11:01.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.632 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.632 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260473+0000", 2024-09-13T10:11:01.632 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0080398180000000007 2024-09-13T10:11:01.632 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.632 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.632 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.632 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260599+0000", 2024-09-13T10:11:01.632 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012575100000000001 2024-09-13T10:11:01.632 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.632 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.632 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.632 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260680+0000", 2024-09-13T10:11:01.632 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.0956000000000002e-05 2024-09-13T10:11:01.633 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.633 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.633 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.633 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:294 4.4 4:25c904b0:::smithi19530714-45:head [assert-version v15,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.633 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.260461+0000", 2024-09-13T10:11:01.633 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.304223234, 2024-09-13T10:11:01.633 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020509151, 2024-09-13T10:11:01.633 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.633 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.633 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.633 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.634 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.634 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.634 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 294 2024-09-13T10:11:01.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.634 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.634 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.634 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.634 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260461+0000", 2024-09-13T10:11:01.634 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.634 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.634 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.634 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260461+0000", 2024-09-13T10:11:01.634 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.635 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.635 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.635 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260464+0000", 2024-09-13T10:11:01.635 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4279999999999999e-06 2024-09-13T10:11:01.635 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.635 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.635 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260474+0000", 2024-09-13T10:11:01.635 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.031e-05 2024-09-13T10:11:01.635 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.635 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.636 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260494+0000", 2024-09-13T10:11:01.636 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9894e-05 2024-09-13T10:11:01.636 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.636 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.636 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.636 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260509+0000", 2024-09-13T10:11:01.636 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5135e-05 2024-09-13T10:11:01.636 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.636 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.636 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.636 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260654+0000", 2024-09-13T10:11:01.636 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014543100000000001 2024-09-13T10:11:01.636 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.637 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.637 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.637 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260980+0000", 2024-09-13T10:11:01.637 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00032611599999999999 2024-09-13T10:11:01.637 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.637 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.637 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:01.637 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.274012+0000", 2024-09-13T10:11:01.637 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013031205000000001 2024-09-13T10:11:01.637 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.637 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.637 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.637 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.275836+0000", 2024-09-13T10:11:01.638 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0018243630000000001 2024-09-13T10:11:01.638 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.638 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.638 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.638 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280160+0000", 2024-09-13T10:11:01.638 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0043236020000000002 2024-09-13T10:11:01.638 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.638 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.638 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.638 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280818+0000", 2024-09-13T10:11:01.638 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00065824600000000003 2024-09-13T10:11:01.638 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.638 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.638 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.639 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280897+0000", 2024-09-13T10:11:01.639 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9623000000000007e-05 2024-09-13T10:11:01.639 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.639 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.639 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.639 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280970+0000", 2024-09-13T10:11:01.639 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2798000000000003e-05 2024-09-13T10:11:01.639 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.639 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.639 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.639 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.639 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.639 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:298 4.4 4:24da8222:::smithi19530714-49:head [assert-version v18,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.639 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.262102+0000", 2024-09-13T10:11:01.640 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.302582678, 2024-09-13T10:11:01.640 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023401737999999998, 2024-09-13T10:11:01.640 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.640 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.640 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.640 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.640 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.640 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.640 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 298 2024-09-13T10:11:01.640 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.640 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.640 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.640 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.641 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262102+0000", 2024-09-13T10:11:01.641 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.641 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.641 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.641 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262102+0000", 2024-09-13T10:11:01.641 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.641 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.641 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.641 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262103+0000", 2024-09-13T10:11:01.641 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7719999999999999e-06 2024-09-13T10:11:01.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.642 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.642 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.642 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262112+0000", 2024-09-13T10:11:01.642 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.242e-06 2024-09-13T10:11:01.642 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.642 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.642 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.642 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262130+0000", 2024-09-13T10:11:01.642 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8057999999999999e-05 2024-09-13T10:11:01.642 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.642 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.642 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.643 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262141+0000", 2024-09-13T10:11:01.643 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0893000000000001e-05 2024-09-13T10:11:01.643 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.643 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.643 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.643 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262226+0000", 2024-09-13T10:11:01.643 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5694999999999999e-05 2024-09-13T10:11:01.643 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.643 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.643 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.644 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262321+0000", 2024-09-13T10:11:01.644 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.4394999999999993e-05 2024-09-13T10:11:01.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.644 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.644 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:01.644 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.274901+0000", 2024-09-13T10:11:01.644 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012580071999999999 2024-09-13T10:11:01.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.644 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.644 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.644 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276773+0000", 2024-09-13T10:11:01.644 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0018726579999999999 2024-09-13T10:11:01.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.645 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.645 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.645 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283725+0000", 2024-09-13T10:11:01.645 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0069512460000000003 2024-09-13T10:11:01.645 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.645 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.645 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.645 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.285294+0000", 2024-09-13T10:11:01.645 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0015694859999999999 2024-09-13T10:11:01.645 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.645 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.645 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.645 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.285420+0000", 2024-09-13T10:11:01.646 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012607699999999999 2024-09-13T10:11:01.646 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.646 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.646 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.646 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.285503+0000", 2024-09-13T10:11:01.646 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.3144000000000002e-05 2024-09-13T10:11:01.646 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.646 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.646 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.646 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.646 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.646 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:308 4.5 4:a51b04a4:::smithi19530714-9:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.646 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.270891+0000", 2024-09-13T10:11:01.647 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.293793745, 2024-09-13T10:11:01.647 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014235486, 2024-09-13T10:11:01.647 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.647 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.647 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.647 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.647 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.647 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.647 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 308 2024-09-13T10:11:01.647 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.647 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.647 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.647 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.647 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.270891+0000", 2024-09-13T10:11:01.648 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.648 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.648 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.648 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.648 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.270891+0000", 2024-09-13T10:11:01.648 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.648 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.648 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.648 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.648 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.270893+0000", 2024-09-13T10:11:01.648 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.979e-06 2024-09-13T10:11:01.648 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.649 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.649 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.649 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.270906+0000", 2024-09-13T10:11:01.649 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3655999999999999e-05 2024-09-13T10:11:01.649 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.649 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.649 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.649 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.270931+0000", 2024-09-13T10:11:01.649 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5035999999999999e-05 2024-09-13T10:11:01.649 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.649 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.649 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.649 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.270947+0000", 2024-09-13T10:11:01.649 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5549e-05 2024-09-13T10:11:01.650 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.650 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.650 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.650 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.271077+0000", 2024-09-13T10:11:01.650 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013000199999999999 2024-09-13T10:11:01.650 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.650 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.650 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.650 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.271250+0000", 2024-09-13T10:11:01.650 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017361000000000001 2024-09-13T10:11:01.650 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.650 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.650 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,10", 2024-09-13T10:11:01.651 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.277373+0000", 2024-09-13T10:11:01.651 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0061230349999999998 2024-09-13T10:11:01.651 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.651 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.651 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.651 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.279233+0000", 2024-09-13T10:11:01.651 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001859131 2024-09-13T10:11:01.651 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.651 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.651 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.651 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283439+0000", 2024-09-13T10:11:01.651 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0042059419999999998 2024-09-13T10:11:01.651 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.652 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.652 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.652 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.284971+0000", 2024-09-13T10:11:01.652 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001532521 2024-09-13T10:11:01.652 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.652 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.652 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.652 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.285052+0000", 2024-09-13T10:11:01.652 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.0580000000000004e-05 2024-09-13T10:11:01.652 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.652 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.652 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.652 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.285126+0000", 2024-09-13T10:11:01.652 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.4444999999999996e-05 2024-09-13T10:11:01.653 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.653 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.653 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.653 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.653 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.653 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:324 4.4 4:26b855f3:::smithi19530714-25:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.653 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:11:01.653 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.283979816, 2024-09-13T10:11:01.653 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033888489000000001, 2024-09-13T10:11:01.653 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.653 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.653 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.653 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.654 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.654 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.654 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 324 2024-09-13T10:11:01.654 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.654 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.654 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.654 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.654 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:11:01.654 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.654 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.654 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.654 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.654 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:11:01.655 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.655 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.655 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.655 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280706+0000", 2024-09-13T10:11:01.655 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6479999999999999e-06 2024-09-13T10:11:01.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.655 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.655 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.655 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280715+0000", 2024-09-13T10:11:01.655 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1500000000000005e-06 2024-09-13T10:11:01.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.655 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.655 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.656 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280733+0000", 2024-09-13T10:11:01.656 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8168e-05 2024-09-13T10:11:01.656 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.656 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.656 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.656 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280747+0000", 2024-09-13T10:11:01.656 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3661e-05 2024-09-13T10:11:01.656 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.656 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.656 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.656 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281013+0000", 2024-09-13T10:11:01.656 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026565099999999999 2024-09-13T10:11:01.656 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.657 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.657 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.657 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281124+0000", 2024-09-13T10:11:01.657 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000111715 2024-09-13T10:11:01.657 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.657 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.657 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:01.657 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286502+0000", 2024-09-13T10:11:01.657 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0053775259999999997 2024-09-13T10:11:01.657 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.657 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.657 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.657 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.289334+0000", 2024-09-13T10:11:01.658 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0028316779999999998 2024-09-13T10:11:01.658 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.658 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.658 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.658 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.290331+0000", 2024-09-13T10:11:01.658 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0009974929999999999 2024-09-13T10:11:01.658 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.658 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.658 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.658 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314453+0000", 2024-09-13T10:11:01.658 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024121417999999999 2024-09-13T10:11:01.658 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.658 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.659 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.659 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314531+0000", 2024-09-13T10:11:01.659 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8841e-05 2024-09-13T10:11:01.659 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.659 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.659 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.659 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314593+0000", 2024-09-13T10:11:01.659 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.1539999999999997e-05 2024-09-13T10:11:01.659 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.659 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.659 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.659 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.659 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.660 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:327 4.4 4:20c7201f:::smithi19530714-28:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.660 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:11:01.660 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.283511987000001, 2024-09-13T10:11:01.660 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035855512999999999, 2024-09-13T10:11:01.660 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.660 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.660 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.660 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.660 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.660 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.660 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 327 2024-09-13T10:11:01.660 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.661 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.661 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.661 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.661 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:11:01.661 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.661 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.661 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.661 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.661 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:11:01.661 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.661 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.661 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.661 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.662 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281173+0000", 2024-09-13T10:11:01.662 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0020000000000001e-06 2024-09-13T10:11:01.662 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.662 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.662 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.662 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281180+0000", 2024-09-13T10:11:01.662 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.8260000000000004e-06 2024-09-13T10:11:01.662 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.662 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.662 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.662 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281192+0000", 2024-09-13T10:11:01.662 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1725e-05 2024-09-13T10:11:01.662 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.663 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.663 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.663 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281202+0000", 2024-09-13T10:11:01.663 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0492000000000001e-05 2024-09-13T10:11:01.663 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.663 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.663 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.663 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281396+0000", 2024-09-13T10:11:01.663 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019404200000000001 2024-09-13T10:11:01.663 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.663 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.663 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.663 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281533+0000", 2024-09-13T10:11:01.663 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000136267 2024-09-13T10:11:01.664 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.664 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.664 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:01.664 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.313834+0000", 2024-09-13T10:11:01.664 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032301491000000002 2024-09-13T10:11:01.664 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.664 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.664 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.664 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314993+0000", 2024-09-13T10:11:01.664 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001158585 2024-09-13T10:11:01.664 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.664 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.664 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.665 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.315097+0000", 2024-09-13T10:11:01.665 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000104685 2024-09-13T10:11:01.665 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.665 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.665 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.665 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.316808+0000", 2024-09-13T10:11:01.665 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0017110529999999999 2024-09-13T10:11:01.665 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.665 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.665 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.665 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.316946+0000", 2024-09-13T10:11:01.665 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000137737 2024-09-13T10:11:01.665 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.666 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.666 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.666 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.317028+0000", 2024-09-13T10:11:01.666 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.1607999999999995e-05 2024-09-13T10:11:01.666 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.666 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.666 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.666 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.666 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.666 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:335 4.2 4:4a24ecac:::smithi19530714-36:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.666 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:11:01.666 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.282193241, 2024-09-13T10:11:01.666 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032260373000000002, 2024-09-13T10:11:01.667 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.667 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.667 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.667 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.667 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.667 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.667 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 335 2024-09-13T10:11:01.667 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.667 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.667 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.667 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.667 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:11:01.667 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.668 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.668 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.668 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.668 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:11:01.668 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.668 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.668 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.668 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.668 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282494+0000", 2024-09-13T10:11:01.668 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5390000000000002e-06 2024-09-13T10:11:01.668 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.668 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.668 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.669 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282506+0000", 2024-09-13T10:11:01.669 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2646000000000001e-05 2024-09-13T10:11:01.669 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.669 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.669 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.669 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282539+0000", 2024-09-13T10:11:01.669 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3176000000000002e-05 2024-09-13T10:11:01.669 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.669 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.669 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.669 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282558+0000", 2024-09-13T10:11:01.669 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9065000000000001e-05 2024-09-13T10:11:01.669 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.670 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.670 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.670 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282697+0000", 2024-09-13T10:11:01.670 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013857999999999999 2024-09-13T10:11:01.670 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.670 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.670 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.670 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282874+0000", 2024-09-13T10:11:01.670 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017656900000000001 2024-09-13T10:11:01.670 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.670 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.670 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,14", 2024-09-13T10:11:01.670 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.287911+0000", 2024-09-13T10:11:01.671 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0050369220000000001 2024-09-13T10:11:01.671 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.671 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.671 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.671 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.290491+0000", 2024-09-13T10:11:01.671 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025802500000000001 2024-09-13T10:11:01.671 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.671 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.671 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.671 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.291374+0000", 2024-09-13T10:11:01.671 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00088367800000000005 2024-09-13T10:11:01.671 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.672 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.672 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.672 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314557+0000", 2024-09-13T10:11:01.672 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023182094 2024-09-13T10:11:01.672 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.672 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.672 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.672 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314664+0000", 2024-09-13T10:11:01.672 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010751099999999999 2024-09-13T10:11:01.672 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.672 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.672 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.672 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314751+0000", 2024-09-13T10:11:01.673 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7342999999999994e-05 2024-09-13T10:11:01.673 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.673 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.673 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.673 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.673 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.673 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:337 4.4 4:24275569:::smithi19530714-38:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:01.673 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:11:01.673 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.281536689999999, 2024-09-13T10:11:01.673 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.031660147, 2024-09-13T10:11:01.673 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.673 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.674 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:01.674 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.674 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.674 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.674 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 337 2024-09-13T10:11:01.674 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.674 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.674 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.674 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.674 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:11:01.674 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.674 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.675 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.675 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.675 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:11:01.675 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.675 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.675 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.675 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.675 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283149+0000", 2024-09-13T10:11:01.675 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.655e-06 2024-09-13T10:11:01.675 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.676 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.676 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.676 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283159+0000", 2024-09-13T10:11:01.676 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.9529999999999994e-06 2024-09-13T10:11:01.676 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.676 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.676 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.676 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283177+0000", 2024-09-13T10:11:01.676 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8168e-05 2024-09-13T10:11:01.676 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.676 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.676 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.677 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283193+0000", 2024-09-13T10:11:01.677 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5563e-05 2024-09-13T10:11:01.677 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.677 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.677 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.677 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283317+0000", 2024-09-13T10:11:01.677 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012421700000000001 2024-09-13T10:11:01.677 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.677 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.677 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.677 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283478+0000", 2024-09-13T10:11:01.677 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000160378 2024-09-13T10:11:01.678 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.678 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.678 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:01.678 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.289593+0000", 2024-09-13T10:11:01.678 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0061152200000000002 2024-09-13T10:11:01.678 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.678 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.678 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.678 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.291636+0000", 2024-09-13T10:11:01.678 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020435430000000001 2024-09-13T10:11:01.678 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.678 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.678 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:01.679 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294186+0000", 2024-09-13T10:11:01.679 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025494760000000002 2024-09-13T10:11:01.679 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.679 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.679 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:01.679 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314672+0000", 2024-09-13T10:11:01.679 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020486392999999999 2024-09-13T10:11:01.679 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.679 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.679 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:01.679 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314741+0000", 2024-09-13T10:11:01.679 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9003000000000003e-05 2024-09-13T10:11:01.679 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.679 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.680 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.680 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314808+0000", 2024-09-13T10:11:01.680 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.6577999999999999e-05 2024-09-13T10:11:01.680 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.680 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.680 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.680 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.680 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.680 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:774 4.4 4:244eab67:::smithi19530714-24:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:01.680 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.565689+0000", 2024-09-13T10:11:01.680 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.9989950689999993, 2024-09-13T10:11:01.680 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015688112000000001, 2024-09-13T10:11:01.681 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.681 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.681 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:01.681 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.681 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.681 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.681 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 774 2024-09-13T10:11:01.681 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.681 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.681 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.681 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.681 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.565689+0000", 2024-09-13T10:11:01.681 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.682 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.682 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.682 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.682 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.565689+0000", 2024-09-13T10:11:01.682 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.682 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.682 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.682 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.682 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.565690+0000", 2024-09-13T10:11:01.682 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.6000000000000002e-07 2024-09-13T10:11:01.682 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.682 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.682 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.682 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.565694+0000", 2024-09-13T10:11:01.683 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.8299999999999998e-06 2024-09-13T10:11:01.683 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.683 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.683 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.683 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.565704+0000", 2024-09-13T10:11:01.683 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0326000000000001e-05 2024-09-13T10:11:01.683 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.683 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.683 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.683 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.565712+0000", 2024-09-13T10:11:01.683 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9359999999999999e-06 2024-09-13T10:11:01.683 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.683 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.683 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.565804+0000", 2024-09-13T10:11:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1402000000000003e-05 2024-09-13T10:11:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.565937+0000", 2024-09-13T10:11:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013338 2024-09-13T10:11:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.581377+0000", 2024-09-13T10:11:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015440377999999999 2024-09-13T10:11:01.684 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:807 4.5 4:ab73fdc2:::smithi19530714-30:head [sparse-read 0~1009951] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.799045+0000", 2024-09-13T10:11:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.7656395380000003, 2024-09-13T10:11:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013638300000000001, 2024-09-13T10:11:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:01.685 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 807 2024-09-13T10:11:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.799045+0000", 2024-09-13T10:11:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.799045+0000", 2024-09-13T10:11:01.686 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.799046+0000", 2024-09-13T10:11:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.0699999999999996e-07 2024-09-13T10:11:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.799049+0000", 2024-09-13T10:11:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6660000000000001e-06 2024-09-13T10:11:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.687 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.799060+0000", 2024-09-13T10:11:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0254e-05 2024-09-13T10:11:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.799067+0000", 2024-09-13T10:11:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2559999999999996e-06 2024-09-13T10:11:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.811866+0000", 2024-09-13T10:11:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012798756999999999 2024-09-13T10:11:01.688 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.812100+0000", 2024-09-13T10:11:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023488400000000001 2024-09-13T10:11:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.812683+0000", 2024-09-13T10:11:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00058267599999999996 2024-09-13T10:11:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.689 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1089 4.5 4:a7549961:::smithi19530714-27:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:56.657202+0000", 2024-09-13T10:11:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 4.9074821350000004, 2024-09-13T10:11:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020240790000000002, 2024-09-13T10:11:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1089 2024-09-13T10:11:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.690 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657202+0000", 2024-09-13T10:11:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657202+0000", 2024-09-13T10:11:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:01.691 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657205+0000", 2024-09-13T10:11:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.373e-06 2024-09-13T10:11:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657215+0000", 2024-09-13T10:11:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0563e-05 2024-09-13T10:11:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657243+0000", 2024-09-13T10:11:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8065999999999999e-05 2024-09-13T10:11:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.692 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657259+0000", 2024-09-13T10:11:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5415999999999999e-05 2024-09-13T10:11:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657404+0000", 2024-09-13T10:11:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014504000000000001 2024-09-13T10:11:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657619+0000", 2024-09-13T10:11:01.693 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021532499999999999 2024-09-13T10:11:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.677443+0000", 2024-09-13T10:11:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.019824007000000001 2024-09-13T10:11:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:01.694 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:01.694 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:01.695 INFO:tasks.rados.rados.0.smithi195.stdout:813: expect (ObjNum 109 snap 0 seq_num 109) 2024-09-13T10:11:01.696 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-13T10:11:01.697 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:01.693+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:01.735 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1176237 2024-09-13T10:11:01.795 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 182155 2024-09-13T10:11:01.796 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:01.796+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:01.854 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:11:01.865 INFO:tasks.rados.rados.0.smithi195.stdout:812: expect (ObjNum 99 snap 0 seq_num 99) 2024-09-13T10:11:01.865 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:01.869 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:01.869 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:01.869 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:11:01.869 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:01.883 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-13T10:11:01.896 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:01.893+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:01.920 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:01.976 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:01.997 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:01.997+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:02.011 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:02.011 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:02.032 INFO:tasks.rados.rados.0.smithi195.stdout:815: finishing write tid 1 to smithi19530714-4 2024-09-13T10:11:02.032 INFO:tasks.rados.rados.0.smithi195.stdout:815: finishing write tid 2 to smithi19530714-4 2024-09-13T10:11:02.032 INFO:tasks.rados.rados.0.smithi195.stdout:815: finishing write tid 3 to smithi19530714-4 2024-09-13T10:11:02.032 INFO:tasks.rados.rados.0.smithi195.stdout:815: finishing write tid 4 to smithi19530714-4 2024-09-13T10:11:02.033 INFO:tasks.rados.rados.0.smithi195.stdout:815: finishing write tid 5 to smithi19530714-4 2024-09-13T10:11:02.033 INFO:tasks.rados.rados.0.smithi195.stdout:815: oid 4 updating version 0 to 39 2024-09-13T10:11:02.033 INFO:tasks.rados.rados.0.smithi195.stdout:815: oid 4 updating version 39 to 42 2024-09-13T10:11:02.033 INFO:tasks.rados.rados.0.smithi195.stdout:815: oid 4 version 42 is already newer than 41 2024-09-13T10:11:02.033 INFO:tasks.rados.rados.0.smithi195.stdout:815: oid 4 version 42 is already newer than 40 2024-09-13T10:11:02.033 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 42 (ObjNum 110 snap 0 seq_num 110) dirty exists 2024-09-13T10:11:02.033 INFO:tasks.rados.rados.0.smithi195.stdout:815: left oid 4 (ObjNum 110 snap 0 seq_num 110) 2024-09-13T10:11:02.033 INFO:tasks.rados.rados.0.smithi195.stdout:816: finishing write tid 1 to smithi19530714-22 2024-09-13T10:11:02.033 INFO:tasks.rados.rados.0.smithi195.stdout:816: finishing write tid 2 to smithi19530714-22 2024-09-13T10:11:02.033 INFO:tasks.rados.rados.0.smithi195.stdout:816: finishing write tid 3 to smithi19530714-22 2024-09-13T10:11:02.033 INFO:tasks.rados.rados.0.smithi195.stdout:816: finishing write tid 4 to smithi19530714-22 2024-09-13T10:11:02.033 INFO:tasks.rados.rados.0.smithi195.stdout:816: finishing write tid 5 to smithi19530714-22 2024-09-13T10:11:02.033 INFO:tasks.rados.rados.0.smithi195.stdout:816: oid 22 updating version 0 to 49 2024-09-13T10:11:02.034 INFO:tasks.rados.rados.0.smithi195.stdout:816: oid 22 updating version 49 to 50 2024-09-13T10:11:02.034 INFO:tasks.rados.rados.0.smithi195.stdout:816: oid 22 updating version 50 to 52 2024-09-13T10:11:02.034 INFO:tasks.rados.rados.0.smithi195.stdout:816: oid 22 version 52 is already newer than 51 2024-09-13T10:11:02.034 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 52 (ObjNum 111 snap 0 seq_num 111) dirty exists 2024-09-13T10:11:02.034 INFO:tasks.rados.rados.0.smithi195.stdout:816: left oid 22 (ObjNum 111 snap 0 seq_num 111) 2024-09-13T10:11:02.034 INFO:tasks.rados.rados.0.smithi195.stdout:817: finishing write tid 1 to smithi19530714-27 2024-09-13T10:11:02.034 INFO:tasks.rados.rados.0.smithi195.stdout:817: finishing write tid 2 to smithi19530714-27 2024-09-13T10:11:02.034 INFO:tasks.rados.rados.0.smithi195.stdout:811: done (8 left) 2024-09-13T10:11:02.034 INFO:tasks.rados.rados.0.smithi195.stdout:812: done (7 left) 2024-09-13T10:11:02.034 INFO:tasks.rados.rados.0.smithi195.stdout:813: done (6 left) 2024-09-13T10:11:02.034 INFO:tasks.rados.rados.0.smithi195.stdout:814: done (5 left) 2024-09-13T10:11:02.034 INFO:tasks.rados.rados.0.smithi195.stdout:815: done (4 left) 2024-09-13T10:11:02.035 INFO:tasks.rados.rados.0.smithi195.stdout:816: done (3 left) 2024-09-13T10:11:02.035 INFO:tasks.rados.rados.0.smithi195.stdout:820: read oid 37 snap -1 2024-09-13T10:11:02.035 INFO:tasks.rados.rados.0.smithi195.stdout:820: expect (ObjNum 95 snap 0 seq_num 95) 2024-09-13T10:11:02.035 INFO:tasks.rados.rados.0.smithi195.stdout:821: read oid 19 snap -1 2024-09-13T10:11:02.035 INFO:tasks.rados.rados.0.smithi195.stdout:821: expect (ObjNum 105 snap 0 seq_num 105) 2024-09-13T10:11:02.035 INFO:tasks.rados.rados.0.smithi195.stdout:822: write oid 9 current snap is 0 2024-09-13T10:11:02.035 INFO:tasks.rados.rados.0.smithi195.stdout:819: got (22) Invalid argument 2024-09-13T10:11:02.035 INFO:tasks.rados.rados.0.smithi195.stdout:817: finishing write tid 3 to smithi19530714-27 2024-09-13T10:11:02.035 INFO:tasks.rados.rados.0.smithi195.stdout:817: finishing write tid 4 to smithi19530714-27 2024-09-13T10:11:02.035 INFO:tasks.rados.rados.0.smithi195.stdout:817: finishing write tid 5 to smithi19530714-27 2024-09-13T10:11:02.035 INFO:tasks.rados.rados.0.smithi195.stdout:817: finishing write tid 6 to smithi19530714-27 2024-09-13T10:11:02.035 INFO:tasks.rados.rados.0.smithi195.stdout:817: oid 27 updating version 0 to 56 2024-09-13T10:11:02.035 INFO:tasks.rados.rados.0.smithi195.stdout:817: oid 27 version 56 is already newer than 55 2024-09-13T10:11:02.035 INFO:tasks.rados.rados.0.smithi195.stdout:817: oid 27 version 56 is already newer than 54 2024-09-13T10:11:02.036 INFO:tasks.rados.rados.0.smithi195.stdout:817: oid 27 updating version 56 to 57 2024-09-13T10:11:02.036 INFO:tasks.rados.rados.0.smithi195.stdout:817: oid 27 version 57 is already newer than 53 2024-09-13T10:11:02.036 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 57 (ObjNum 112 snap 0 seq_num 112) dirty exists 2024-09-13T10:11:02.036 INFO:tasks.rados.rados.0.smithi195.stdout:817: left oid 27 (ObjNum 112 snap 0 seq_num 112) 2024-09-13T10:11:02.036 INFO:tasks.rados.rados.0.smithi195.stdout:818: got (0) Success 2024-09-13T10:11:02.036 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 6 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-09-13T10:11:02.036 INFO:tasks.rados.rados.0.smithi195.stdout:822: seq_num 113 ranges {699191=532079,1779524=567357,2837151=666208} 2024-09-13T10:11:02.036 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:02.037 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:02.046 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:02.046 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:02.046 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:11:02.046 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:11:02.046 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:02.047 INFO:tasks.rados.rados.0.smithi195.stdout:822: writing smithi19530714-9 from 699191 to 1231270 tid 1 2024-09-13T10:11:02.059 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:02.059 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:02.059 INFO:tasks.rados.rados.0.smithi195.stdout:822: writing smithi19530714-9 from 1779524 to 2346881 tid 2 2024-09-13T10:11:02.064 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-13T10:11:02.073 INFO:tasks.rados.rados.0.smithi195.stdout:822: writing smithi19530714-9 from 2837151 to 3503359 tid 3 2024-09-13T10:11:02.073 INFO:tasks.rados.rados.0.smithi195.stdout:817: done (5 left) 2024-09-13T10:11:02.073 INFO:tasks.rados.rados.0.smithi195.stdout:818: done (4 left) 2024-09-13T10:11:02.073 INFO:tasks.rados.rados.0.smithi195.stdout:819: done (3 left) 2024-09-13T10:11:02.073 INFO:tasks.rados.rados.0.smithi195.stdout:823: write (excl) oid 22 current snap is 0 2024-09-13T10:11:02.073 INFO:tasks.rados.rados.0.smithi195.stdout:823: seq_num 114 ranges {715998=607328,1838006=427368,2864574=477480,3863835=52163} 2024-09-13T10:11:02.080 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:02.080 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:02.087 INFO:tasks.rados.rados.0.smithi195.stdout:823: writing smithi19530714-22 from 715998 to 1323326 tid 1 2024-09-13T10:11:02.096 INFO:tasks.rados.rados.0.smithi195.stdout:823: writing smithi19530714-22 from 1838006 to 2265374 tid 2 2024-09-13T10:11:02.097 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:02.097 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:02.105 INFO:tasks.rados.rados.0.smithi195.stdout:823: writing smithi19530714-22 from 2864574 to 3342054 tid 3 2024-09-13T10:11:02.107 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:02.108+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:02.109 INFO:tasks.rados.rados.0.smithi195.stdout:823: writing smithi19530714-22 from 3863835 to 3915998 tid 4 2024-09-13T10:11:02.109 INFO:tasks.rados.rados.0.smithi195.stdout:824: write (excl) oid 4 current snap is 0 2024-09-13T10:11:02.109 INFO:tasks.rados.rados.0.smithi195.stdout:824: seq_num 115 ranges {732805=798930,2296488=571026,3291997=405105} 2024-09-13T10:11:02.119 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:02.119 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:02.120 INFO:tasks.rados.rados.0.smithi195.stdout:824: writing smithi19530714-4 from 732805 to 1531735 tid 1 2024-09-13T10:11:02.128 INFO:tasks.rados.rados.0.smithi195.stdout:824: writing smithi19530714-4 from 2296488 to 2867514 tid 2 2024-09-13T10:11:02.134 INFO:tasks.rados.rados.0.smithi195.stdout:824: writing smithi19530714-4 from 3291997 to 3697102 tid 3 2024-09-13T10:11:02.134 INFO:tasks.rados.rados.0.smithi195.stdout:825: read oid 26 snap -1 2024-09-13T10:11:02.134 INFO:tasks.rados.rados.0.smithi195.stdout:825: expect (ObjNum 25 snap 0 seq_num 25) 2024-09-13T10:11:02.134 INFO:tasks.rados.rados.0.smithi195.stdout:826: read oid 14 snap -1 2024-09-13T10:11:02.134 INFO:tasks.rados.rados.0.smithi195.stdout:826: expect (ObjNum 103 snap 0 seq_num 103) 2024-09-13T10:11:02.134 INFO:tasks.rados.rados.0.smithi195.stdout:827: read oid 38 snap -1 2024-09-13T10:11:02.134 INFO:tasks.rados.rados.0.smithi195.stdout:827: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:11:02.134 INFO:tasks.rados.rados.0.smithi195.stdout:828: read oid 47 snap -1 2024-09-13T10:11:02.134 INFO:tasks.rados.rados.0.smithi195.stdout:828: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-13T10:11:02.134 INFO:tasks.rados.rados.0.smithi195.stdout:829: read oid 36 snap -1 2024-09-13T10:11:02.134 INFO:tasks.rados.rados.0.smithi195.stdout:829: expect (ObjNum 80 snap 0 seq_num 80) 2024-09-13T10:11:02.135 INFO:tasks.rados.rados.0.smithi195.stdout:830: tier_flush oid 18 2024-09-13T10:11:02.135 INFO:tasks.rados.rados.0.smithi195.stdout:830: tier_flush oid 18 snap -1 2024-09-13T10:11:02.135 INFO:tasks.rados.rados.0.smithi195.stdout:831: tier_evict oid 45 2024-09-13T10:11:02.135 INFO:tasks.rados.rados.0.smithi195.stdout:831: tier_evict oid 45 snap -1 2024-09-13T10:11:02.135 INFO:tasks.rados.rados.0.smithi195.stdout:832: write oid 49 current snap is 0 2024-09-13T10:11:02.135 INFO:tasks.rados.rados.0.smithi195.stdout:821: expect (ObjNum 105 snap 0 seq_num 105) 2024-09-13T10:11:02.138 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:02.138 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:02.160 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:02.160 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:02.177 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:02.177 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:02.199 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:02.199 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:11:02.199 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:11:02.199 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:11:02.199 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.199 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:201 3.3 3:d4fb61be:::smithi19530714-2 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:02.199 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:11:02.200 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 11.202255863, 2024-09-13T10:11:02.200 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.052937043000000003, 2024-09-13T10:11:02.200 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.200 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.200 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.200 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:02.200 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:02.200 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:02.200 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 201 2024-09-13T10:11:02.200 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.200 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.200 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.201 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.201 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:11:02.201 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.201 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.201 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.201 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.201 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:11:02.201 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.201 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.201 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.201 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.201 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993192+0000", 2024-09-13T10:11:02.202 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8559999999999996e-06 2024-09-13T10:11:02.202 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.202 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.202 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.202 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993203+0000", 2024-09-13T10:11:02.202 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1113e-05 2024-09-13T10:11:02.202 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.202 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.202 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.202 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993225+0000", 2024-09-13T10:11:02.202 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2793e-05 2024-09-13T10:11:02.202 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.202 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.203 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.203 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993241+0000", 2024-09-13T10:11:02.203 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5806000000000001e-05 2024-09-13T10:11:02.203 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.203 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.203 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.203 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993401+0000", 2024-09-13T10:11:02.203 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000160149 2024-09-13T10:11:02.203 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.203 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.203 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.203 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994047+0000", 2024-09-13T10:11:02.203 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00064594100000000003 2024-09-13T10:11:02.204 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.204 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.204 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.204 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.011243+0000", 2024-09-13T10:11:02.204 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017196026999999999 2024-09-13T10:11:02.204 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.204 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.204 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.204 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029096+0000", 2024-09-13T10:11:02.204 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017852115000000002 2024-09-13T10:11:02.204 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.204 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.204 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.205 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029271+0000", 2024-09-13T10:11:02.205 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017589299999999999 2024-09-13T10:11:02.205 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.205 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.205 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.205 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.045867+0000", 2024-09-13T10:11:02.205 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016595625999999999 2024-09-13T10:11:02.205 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.205 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.205 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.205 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.046017+0000", 2024-09-13T10:11:02.205 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014974199999999999 2024-09-13T10:11:02.205 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.206 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.206 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.206 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.046120+0000", 2024-09-13T10:11:02.206 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000102982 2024-09-13T10:11:02.206 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.206 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.206 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.206 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.206 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.206 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:208 3.3 3:dbb0d8aa:::smithi19530714-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:02.206 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:11:02.206 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 11.202160053, 2024-09-13T10:11:02.206 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.071216475000000001, 2024-09-13T10:11:02.207 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.207 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.207 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.207 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:02.207 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:02.207 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:02.207 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 208 2024-09-13T10:11:02.207 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.207 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.207 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.207 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.207 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:11:02.207 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.208 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.208 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.208 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:11:02.208 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.208 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.208 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.208 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993280+0000", 2024-09-13T10:11:02.208 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1999999999999999e-06 2024-09-13T10:11:02.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.208 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.209 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.209 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993289+0000", 2024-09-13T10:11:02.209 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8200000000000003e-06 2024-09-13T10:11:02.209 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.209 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.209 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.209 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994000+0000", 2024-09-13T10:11:02.209 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00071192499999999997 2024-09-13T10:11:02.209 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.209 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.209 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.209 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994024+0000", 2024-09-13T10:11:02.209 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3578999999999999e-05 2024-09-13T10:11:02.210 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.210 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.210 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.210 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994468+0000", 2024-09-13T10:11:02.210 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00044410599999999999 2024-09-13T10:11:02.210 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.210 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.210 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.210 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994842+0000", 2024-09-13T10:11:02.210 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037380300000000003 2024-09-13T10:11:02.210 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.210 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.210 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.211 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031572+0000", 2024-09-13T10:11:02.211 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036730479000000003 2024-09-13T10:11:02.211 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.211 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.211 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.211 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.039007+0000", 2024-09-13T10:11:02.211 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.007434871 2024-09-13T10:11:02.211 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.211 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.211 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.211 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.041001+0000", 2024-09-13T10:11:02.211 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001994104 2024-09-13T10:11:02.211 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.211 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.212 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.212 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064330+0000", 2024-09-13T10:11:02.212 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023328912 2024-09-13T10:11:02.212 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.212 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.212 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.212 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064401+0000", 2024-09-13T10:11:02.212 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.1123000000000003e-05 2024-09-13T10:11:02.212 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.212 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.212 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.212 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064495+0000", 2024-09-13T10:11:02.212 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.3553000000000004e-05 2024-09-13T10:11:02.213 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.213 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.213 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.213 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.213 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.213 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:215 3.3 3:ed6c0a17:::smithi19530714-16 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:02.213 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:11:02.213 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 11.201379799, 2024-09-13T10:11:02.213 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.055806517999999999, 2024-09-13T10:11:02.213 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.213 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.213 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.213 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:02.214 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:02.214 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:02.214 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 215 2024-09-13T10:11:02.214 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.214 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.214 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.214 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.214 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:11:02.214 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.214 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.214 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.214 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.214 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:11:02.214 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.215 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.215 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.215 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994060+0000", 2024-09-13T10:11:02.215 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5030000000000001e-06 2024-09-13T10:11:02.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.215 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.215 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.215 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994070+0000", 2024-09-13T10:11:02.215 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0046e-05 2024-09-13T10:11:02.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.215 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.215 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.216 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994087+0000", 2024-09-13T10:11:02.216 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6283999999999999e-05 2024-09-13T10:11:02.216 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.216 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.216 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.216 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994096+0000", 2024-09-13T10:11:02.216 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.969e-06 2024-09-13T10:11:02.216 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.216 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.216 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.216 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.995226+0000", 2024-09-13T10:11:02.216 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011299540000000001 2024-09-13T10:11:02.216 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.217 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.217 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.217 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.995548+0000", 2024-09-13T10:11:02.217 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00032266199999999998 2024-09-13T10:11:02.217 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.217 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.217 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.217 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.018922+0000", 2024-09-13T10:11:02.217 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023373485999999999 2024-09-13T10:11:02.217 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.217 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.217 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.217 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029400+0000", 2024-09-13T10:11:02.218 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010478721 2024-09-13T10:11:02.218 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.218 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.218 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.218 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038932+0000", 2024-09-13T10:11:02.218 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0095316859999999993 2024-09-13T10:11:02.218 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.218 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.218 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.218 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049620+0000", 2024-09-13T10:11:02.218 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010687688000000001 2024-09-13T10:11:02.218 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.218 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.218 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.219 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049754+0000", 2024-09-13T10:11:02.219 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013457700000000001 2024-09-13T10:11:02.219 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.219 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.219 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.219 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049865+0000", 2024-09-13T10:11:02.219 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110942 2024-09-13T10:11:02.219 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.219 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.219 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.219 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.219 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.219 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:224 3.3 3:f941658c:::smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:02.220 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:11:02.220 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 11.144871761999999, 2024-09-13T10:11:02.220 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.025014027000000001, 2024-09-13T10:11:02.220 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.220 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.220 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.220 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:02.220 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:02.220 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:02.220 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 224 2024-09-13T10:11:02.220 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.220 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.220 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.221 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.221 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:11:02.221 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.221 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.221 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.221 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.221 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:11:02.221 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.221 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.221 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.221 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.221 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050570+0000", 2024-09-13T10:11:02.221 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9799999999999998e-06 2024-09-13T10:11:02.222 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.222 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.222 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.222 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050582+0000", 2024-09-13T10:11:02.222 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2170999999999999e-05 2024-09-13T10:11:02.222 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.222 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.222 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.222 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050605+0000", 2024-09-13T10:11:02.222 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3543000000000001e-05 2024-09-13T10:11:02.222 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.222 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.222 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.222 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050619+0000", 2024-09-13T10:11:02.223 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3478999999999999e-05 2024-09-13T10:11:02.223 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.223 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.223 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.223 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050725+0000", 2024-09-13T10:11:02.223 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000105585 2024-09-13T10:11:02.223 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.223 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.223 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.223 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.051086+0000", 2024-09-13T10:11:02.223 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036162100000000001 2024-09-13T10:11:02.223 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.223 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.224 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.224 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.058628+0000", 2024-09-13T10:11:02.224 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075415569999999999 2024-09-13T10:11:02.224 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.224 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.224 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.224 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065344+0000", 2024-09-13T10:11:02.224 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0067161740000000001 2024-09-13T10:11:02.224 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.224 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.224 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.224 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.066222+0000", 2024-09-13T10:11:02.224 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00087765599999999999 2024-09-13T10:11:02.225 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.225 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.225 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.225 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075372+0000", 2024-09-13T10:11:02.225 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0091499609999999999 2024-09-13T10:11:02.225 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.225 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.225 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.225 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075489+0000", 2024-09-13T10:11:02.225 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011718300000000001 2024-09-13T10:11:02.225 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.225 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.225 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.225 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075581+0000", 2024-09-13T10:11:02.226 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.2116999999999999e-05 2024-09-13T10:11:02.226 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.226 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.226 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.226 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.226 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.226 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:226 3.3 3:fe5bfa3a:::smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:02.226 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:11:02.226 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 11.135338752999999, 2024-09-13T10:11:02.226 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051515848000000003, 2024-09-13T10:11:02.226 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.226 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.226 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.227 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:02.227 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:02.227 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:02.227 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 226 2024-09-13T10:11:02.227 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.227 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.227 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.227 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.227 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:11:02.227 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.227 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.227 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.227 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.228 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:11:02.228 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.228 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.228 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.228 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.228 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060102+0000", 2024-09-13T10:11:02.228 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0480000000000001e-06 2024-09-13T10:11:02.228 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.228 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.228 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.228 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060112+0000", 2024-09-13T10:11:02.228 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6649999999999998e-06 2024-09-13T10:11:02.228 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.229 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.229 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.229 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060132+0000", 2024-09-13T10:11:02.229 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0262e-05 2024-09-13T10:11:02.229 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.229 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.229 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.229 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060143+0000", 2024-09-13T10:11:02.229 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1420000000000001e-05 2024-09-13T10:11:02.229 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.229 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.229 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.229 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064563+0000", 2024-09-13T10:11:02.230 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044199119999999998 2024-09-13T10:11:02.230 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.230 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.230 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.230 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064882+0000", 2024-09-13T10:11:02.230 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000318735 2024-09-13T10:11:02.230 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.230 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.230 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.230 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.078858+0000", 2024-09-13T10:11:02.230 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013976455 2024-09-13T10:11:02.230 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.230 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.231 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.231 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088977+0000", 2024-09-13T10:11:02.231 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010118544 2024-09-13T10:11:02.231 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.231 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.231 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.231 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.090451+0000", 2024-09-13T10:11:02.231 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001474026 2024-09-13T10:11:02.231 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.231 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.231 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.231 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111539+0000", 2024-09-13T10:11:02.231 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.02108786 2024-09-13T10:11:02.232 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.232 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.232 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.232 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111580+0000", 2024-09-13T10:11:02.232 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1245000000000002e-05 2024-09-13T10:11:02.232 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.232 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.232 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.232 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111616+0000", 2024-09-13T10:11:02.232 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5676000000000002e-05 2024-09-13T10:11:02.232 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.232 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.232 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.232 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.233 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.233 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:235 3.3 3:e217e96e:::smithi19530714-36 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:02.233 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:11:02.233 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 11.124561124, 2024-09-13T10:11:02.233 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051184682000000002, 2024-09-13T10:11:02.233 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.233 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.233 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.233 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:02.233 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:02.233 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:02.233 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 235 2024-09-13T10:11:02.233 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.234 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.234 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.234 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.234 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:11:02.234 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.234 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.234 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.234 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.234 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:11:02.234 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.234 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.234 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.234 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.235 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070881+0000", 2024-09-13T10:11:02.235 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0579999999999999e-06 2024-09-13T10:11:02.235 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.235 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.235 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.235 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070897+0000", 2024-09-13T10:11:02.235 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5967000000000001e-05 2024-09-13T10:11:02.235 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.235 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.235 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.235 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070942+0000", 2024-09-13T10:11:02.235 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4947000000000002e-05 2024-09-13T10:11:02.235 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.235 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.236 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.236 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070963+0000", 2024-09-13T10:11:02.236 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0098e-05 2024-09-13T10:11:02.236 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.236 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.236 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.236 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.071099+0000", 2024-09-13T10:11:02.236 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013697199999999999 2024-09-13T10:11:02.236 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.236 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.236 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.236 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.071330+0000", 2024-09-13T10:11:02.236 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000230881 2024-09-13T10:11:02.237 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.237 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.237 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.237 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.084668+0000", 2024-09-13T10:11:02.237 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013337663 2024-09-13T10:11:02.237 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.237 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.237 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.237 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.092656+0000", 2024-09-13T10:11:02.237 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0079878880000000003 2024-09-13T10:11:02.237 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.237 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.237 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.238 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.100238+0000", 2024-09-13T10:11:02.238 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075819449999999997 2024-09-13T10:11:02.238 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.238 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.238 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.238 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121877+0000", 2024-09-13T10:11:02.238 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021639104999999999 2024-09-13T10:11:02.238 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.238 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.238 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.238 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121978+0000", 2024-09-13T10:11:02.238 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000101363 2024-09-13T10:11:02.238 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.238 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.239 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.239 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.122062+0000", 2024-09-13T10:11:02.239 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.3795000000000007e-05 2024-09-13T10:11:02.239 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.239 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.239 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.239 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.239 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.239 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:237 3.3 3:de40747e:::smithi19530714-38 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:02.239 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:11:02.239 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 11.116371920000001, 2024-09-13T10:11:02.239 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.073341331999999995, 2024-09-13T10:11:02.239 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.240 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.240 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.240 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:02.240 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:02.240 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:02.240 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 237 2024-09-13T10:11:02.240 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.240 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.240 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.240 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.240 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:11:02.240 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.240 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.240 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.241 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.241 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:11:02.241 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.241 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.241 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.241 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.241 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079069+0000", 2024-09-13T10:11:02.241 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6979999999999998e-06 2024-09-13T10:11:02.241 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.241 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.241 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.241 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079083+0000", 2024-09-13T10:11:02.241 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.384e-05 2024-09-13T10:11:02.242 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.242 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.242 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.242 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079105+0000", 2024-09-13T10:11:02.242 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1736e-05 2024-09-13T10:11:02.242 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.242 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.242 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.242 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079123+0000", 2024-09-13T10:11:02.242 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7889000000000001e-05 2024-09-13T10:11:02.242 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.242 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.242 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.243 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088459+0000", 2024-09-13T10:11:02.243 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0093359700000000007 2024-09-13T10:11:02.243 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.243 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.243 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.243 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088726+0000", 2024-09-13T10:11:02.243 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026732299999999997 2024-09-13T10:11:02.243 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.243 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.243 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.243 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.097522+0000", 2024-09-13T10:11:02.243 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0087962630000000003 2024-09-13T10:11:02.243 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.243 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.244 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.244 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.103972+0000", 2024-09-13T10:11:02.244 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0064494840000000001 2024-09-13T10:11:02.244 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.244 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.244 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.244 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.104160+0000", 2024-09-13T10:11:02.244 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00018782200000000001 2024-09-13T10:11:02.244 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.244 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.244 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.244 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152294+0000", 2024-09-13T10:11:02.244 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048134458999999997 2024-09-13T10:11:02.245 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.245 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.245 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.245 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152373+0000", 2024-09-13T10:11:02.245 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8574999999999999e-05 2024-09-13T10:11:02.245 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.245 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.245 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.245 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152408+0000", 2024-09-13T10:11:02.245 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5272999999999999e-05 2024-09-13T10:11:02.245 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.245 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.245 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.246 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.246 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.246 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:238 3.3 3:c43dd60b:::smithi19530714-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:02.246 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:11:02.246 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 11.106786273000001, 2024-09-13T10:11:02.246 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.069426360000000006, 2024-09-13T10:11:02.246 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.246 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.246 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.246 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:02.246 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:02.246 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:02.246 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 238 2024-09-13T10:11:02.247 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.247 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.247 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.247 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.247 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:11:02.247 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.247 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.247 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.247 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.247 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:11:02.247 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.247 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.247 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.247 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.248 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088655+0000", 2024-09-13T10:11:02.248 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2359999999999999e-06 2024-09-13T10:11:02.248 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.248 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.248 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.248 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088663+0000", 2024-09-13T10:11:02.248 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7800000000000006e-06 2024-09-13T10:11:02.248 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.248 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.248 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.248 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088688+0000", 2024-09-13T10:11:02.248 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4507000000000001e-05 2024-09-13T10:11:02.248 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.249 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.249 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.249 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088717+0000", 2024-09-13T10:11:02.249 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9204e-05 2024-09-13T10:11:02.249 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.249 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.249 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.249 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.089038+0000", 2024-09-13T10:11:02.249 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00032087199999999999 2024-09-13T10:11:02.249 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.249 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.249 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.249 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.089435+0000", 2024-09-13T10:11:02.250 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039756600000000002 2024-09-13T10:11:02.250 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.250 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.250 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.250 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.101672+0000", 2024-09-13T10:11:02.250 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012236528 2024-09-13T10:11:02.250 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.250 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.250 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.250 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.105256+0000", 2024-09-13T10:11:02.250 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0035837159999999998 2024-09-13T10:11:02.250 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.250 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.250 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.251 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111504+0000", 2024-09-13T10:11:02.251 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.006248577 2024-09-13T10:11:02.251 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.251 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.251 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.251 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.157950+0000", 2024-09-13T10:11:02.251 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.046446031999999998 2024-09-13T10:11:02.251 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.251 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.251 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.251 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158027+0000", 2024-09-13T10:11:02.251 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6402999999999993e-05 2024-09-13T10:11:02.251 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.251 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.252 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.252 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158079+0000", 2024-09-13T10:11:02.252 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.1938999999999999e-05 2024-09-13T10:11:02.252 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.252 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.252 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.252 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.252 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.252 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:246 3.3 3:f1be28e7:::smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:02.252 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:11:02.252 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 11.082075526000001, 2024-09-13T10:11:02.252 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051630810999999999, 2024-09-13T10:11:02.252 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.253 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.253 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.253 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:02.253 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:02.253 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:02.253 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 246 2024-09-13T10:11:02.253 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.253 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.253 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.253 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.253 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:11:02.253 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.253 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.254 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.254 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.254 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:11:02.254 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.254 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.254 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.254 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.254 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113365+0000", 2024-09-13T10:11:02.254 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.762e-06 2024-09-13T10:11:02.254 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.254 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.254 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.254 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113372+0000", 2024-09-13T10:11:02.254 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5020000000000003e-06 2024-09-13T10:11:02.255 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.255 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.255 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.255 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113390+0000", 2024-09-13T10:11:02.255 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8056000000000001e-05 2024-09-13T10:11:02.255 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.255 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.255 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.255 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113400+0000", 2024-09-13T10:11:02.255 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5119999999999997e-06 2024-09-13T10:11:02.255 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.255 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.255 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.256 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113456+0000", 2024-09-13T10:11:02.256 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.6221999999999998e-05 2024-09-13T10:11:02.256 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.256 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.256 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.256 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113576+0000", 2024-09-13T10:11:02.256 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000119466 2024-09-13T10:11:02.256 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.256 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.256 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.256 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.129127+0000", 2024-09-13T10:11:02.256 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015551003000000001 2024-09-13T10:11:02.256 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.257 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.257 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.257 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142482+0000", 2024-09-13T10:11:02.257 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013355104 2024-09-13T10:11:02.257 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.257 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.257 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.257 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142518+0000", 2024-09-13T10:11:02.257 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6532000000000002e-05 2024-09-13T10:11:02.257 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.257 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.257 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.257 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164919+0000", 2024-09-13T10:11:02.258 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022401038000000002 2024-09-13T10:11:02.258 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.258 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.258 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.258 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164962+0000", 2024-09-13T10:11:02.258 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.2329000000000001e-05 2024-09-13T10:11:02.258 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.258 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.258 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.258 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164994+0000", 2024-09-13T10:11:02.258 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2285e-05 2024-09-13T10:11:02.258 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.258 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.259 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.259 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.259 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.259 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:249 3.3 3:f565152a:::smithi19530714-50 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:02.259 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:11:02.259 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 11.080473049, 2024-09-13T10:11:02.259 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051233251, 2024-09-13T10:11:02.259 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.259 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.259 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.259 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:02.259 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:02.259 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:02.260 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 249 2024-09-13T10:11:02.260 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.260 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.260 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.260 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.260 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:11:02.260 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.260 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.260 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.260 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.260 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:11:02.260 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.260 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.261 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.261 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.261 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114967+0000", 2024-09-13T10:11:02.261 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.313e-06 2024-09-13T10:11:02.261 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.261 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.261 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.261 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114972+0000", 2024-09-13T10:11:02.261 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5620000000000003e-06 2024-09-13T10:11:02.261 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.261 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.261 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.261 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114997+0000", 2024-09-13T10:11:02.262 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4845999999999999e-05 2024-09-13T10:11:02.262 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.262 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.262 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.262 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115007+0000", 2024-09-13T10:11:02.262 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7200000000000001e-06 2024-09-13T10:11:02.262 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.262 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.262 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.262 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115076+0000", 2024-09-13T10:11:02.262 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9473000000000004e-05 2024-09-13T10:11:02.262 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.262 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.263 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.263 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115221+0000", 2024-09-13T10:11:02.263 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000144604 2024-09-13T10:11:02.263 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.263 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.263 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.263 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.136862+0000", 2024-09-13T10:11:02.263 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021641365999999999 2024-09-13T10:11:02.263 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.263 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.263 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.263 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.145775+0000", 2024-09-13T10:11:02.263 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0089130649999999995 2024-09-13T10:11:02.264 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.264 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.264 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.264 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.150578+0000", 2024-09-13T10:11:02.264 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0048025999999999998 2024-09-13T10:11:02.264 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.264 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.264 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.264 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166133+0000", 2024-09-13T10:11:02.264 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015554853 2024-09-13T10:11:02.264 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.264 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.264 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.264 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166167+0000", 2024-09-13T10:11:02.265 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3973999999999997e-05 2024-09-13T10:11:02.265 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.265 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.265 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.265 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166199+0000", 2024-09-13T10:11:02.265 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1875000000000002e-05 2024-09-13T10:11:02.265 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.265 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.265 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.265 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.265 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.265 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.6.23:1 3.3 3:d4fb61be:::smithi19530714-2 low_tier:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:02.266 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:11:02.266 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 11.07531, 2024-09-13T10:11:02.266 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.038168746000000003, 2024-09-13T10:11:02.266 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.266 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.266 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.266 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.266 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.266 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.266 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:11:02.266 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.266 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.266 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.266 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.267 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:11:02.267 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.267 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.267 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.267 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.267 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120131+0000", 2024-09-13T10:11:02.267 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5620000000000002e-06 2024-09-13T10:11:02.267 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.267 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.267 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.267 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120140+0000", 2024-09-13T10:11:02.267 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1889999999999996e-06 2024-09-13T10:11:02.267 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.268 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.268 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.268 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120161+0000", 2024-09-13T10:11:02.268 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0485000000000001e-05 2024-09-13T10:11:02.268 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.268 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.268 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.268 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120200+0000", 2024-09-13T10:11:02.268 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9648999999999999e-05 2024-09-13T10:11:02.268 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.268 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.268 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.268 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120285+0000", 2024-09-13T10:11:02.269 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.4488999999999997e-05 2024-09-13T10:11:02.269 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.269 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.269 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.269 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120399+0000", 2024-09-13T10:11:02.269 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000113857 2024-09-13T10:11:02.269 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.269 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.269 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.269 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120640+0000", 2024-09-13T10:11:02.269 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024112600000000001 2024-09-13T10:11:02.269 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.270 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.270 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.270 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121715+0000", 2024-09-13T10:11:02.270 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001074885 2024-09-13T10:11:02.270 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.270 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.270 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.270 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.122870+0000", 2024-09-13T10:11:02.270 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00115504 2024-09-13T10:11:02.270 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.270 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.270 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.270 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158160+0000", 2024-09-13T10:11:02.271 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035289760000000003 2024-09-13T10:11:02.271 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.271 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.271 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.271 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158211+0000", 2024-09-13T10:11:02.271 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.1005999999999999e-05 2024-09-13T10:11:02.271 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.271 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.271 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.271 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158297+0000", 2024-09-13T10:11:02.271 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.6698000000000002e-05 2024-09-13T10:11:02.271 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.271 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.271 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.272 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.272 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.272 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.5.22:3 3.3 3:f565152a:::smithi19530714-50 low_tier:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:02.272 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.264828+0000", 2024-09-13T10:11:02.272 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.930610981999999, 2024-09-13T10:11:02.272 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011635411, 2024-09-13T10:11:02.272 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.272 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.272 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.272 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.272 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.272 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.273 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264828+0000", 2024-09-13T10:11:02.273 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.273 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.273 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.273 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.273 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264828+0000", 2024-09-13T10:11:02.273 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.273 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.273 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.273 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.273 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264830+0000", 2024-09-13T10:11:02.273 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.283e-06 2024-09-13T10:11:02.273 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.274 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.274 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.274 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264841+0000", 2024-09-13T10:11:02.274 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0893999999999999e-05 2024-09-13T10:11:02.274 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.274 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.274 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.274 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264865+0000", 2024-09-13T10:11:02.274 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4516999999999999e-05 2024-09-13T10:11:02.274 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.274 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.274 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.274 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264883+0000", 2024-09-13T10:11:02.274 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7759000000000001e-05 2024-09-13T10:11:02.275 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.275 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.275 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.275 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.265917+0000", 2024-09-13T10:11:02.275 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0010340390000000001 2024-09-13T10:11:02.275 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.275 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.275 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.275 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.266107+0000", 2024-09-13T10:11:02.275 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000189435 2024-09-13T10:11:02.275 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.275 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.275 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.276 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.266566+0000", 2024-09-13T10:11:02.276 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00045961599999999999 2024-09-13T10:11:02.276 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.276 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.276 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.276 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.268500+0000", 2024-09-13T10:11:02.276 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019334549999999999 2024-09-13T10:11:02.276 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.276 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.276 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.276 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.269782+0000", 2024-09-13T10:11:02.276 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001282444 2024-09-13T10:11:02.276 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.277 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.277 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.277 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276234+0000", 2024-09-13T10:11:02.277 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0064523580000000001 2024-09-13T10:11:02.277 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.277 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.277 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.277 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276372+0000", 2024-09-13T10:11:02.277 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013788099999999999 2024-09-13T10:11:02.277 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.277 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.277 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.278 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276463+0000", 2024-09-13T10:11:02.278 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.0729999999999994e-05 2024-09-13T10:11:02.278 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.278 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.278 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.278 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.278 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.278 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_repop(client.4607.0:298 4.4 e38/31 4:24da8222:::smithi19530714-49:head v 38'24, mlcod=38'20)", 2024-09-13T10:11:02.278 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.275926+0000", 2024-09-13T10:11:02.278 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.919512502, 2024-09-13T10:11:02.278 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0085803330000000008, 2024-09-13T10:11:02.278 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.279 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.279 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.279 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.279 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.279 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.279 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.275926+0000", 2024-09-13T10:11:02.279 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.279 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.279 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.279 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.279 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.275926+0000", 2024-09-13T10:11:02.279 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.279 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.279 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.280 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.280 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.275927+0000", 2024-09-13T10:11:02.280 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1039999999999999e-06 2024-09-13T10:11:02.280 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.280 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.280 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.280 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.275938+0000", 2024-09-13T10:11:02.280 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0603e-05 2024-09-13T10:11:02.280 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.280 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.280 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.280 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.275953+0000", 2024-09-13T10:11:02.280 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4842e-05 2024-09-13T10:11:02.281 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.281 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.281 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.281 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.275962+0000", 2024-09-13T10:11:02.281 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6039999999999999e-06 2024-09-13T10:11:02.281 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.281 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.281 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.281 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276060+0000", 2024-09-13T10:11:02.281 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7508000000000002e-05 2024-09-13T10:11:02.281 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.281 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.281 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.282 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276097+0000", 2024-09-13T10:11:02.282 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.7344e-05 2024-09-13T10:11:02.282 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.282 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.282 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.282 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.284428+0000", 2024-09-13T10:11:02.282 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0083306470000000001 2024-09-13T10:11:02.282 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.282 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.282 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.282 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.284506+0000", 2024-09-13T10:11:02.282 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8681000000000001e-05 2024-09-13T10:11:02.282 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.282 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.283 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.283 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.283 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.283 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_repop(client.4607.0:762 4.7 e39/31 4:ea79a1ec:::smithi19530714-18:head v 39'24, mlcod=39'22)", 2024-09-13T10:11:02.283 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.450784+0000", 2024-09-13T10:11:02.283 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.744654553, 2024-09-13T10:11:02.283 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0095149940000000006, 2024-09-13T10:11:02.283 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.283 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.283 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.283 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.283 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.283 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.284 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.450784+0000", 2024-09-13T10:11:02.284 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.284 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.284 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.284 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.284 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.450784+0000", 2024-09-13T10:11:02.284 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.284 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.284 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.284 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.284 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.450787+0000", 2024-09-13T10:11:02.284 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9059999999999998e-06 2024-09-13T10:11:02.285 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.285 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.285 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.285 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.451730+0000", 2024-09-13T10:11:02.285 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00094282000000000003 2024-09-13T10:11:02.285 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.285 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.285 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.285 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.451765+0000", 2024-09-13T10:11:02.285 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5278999999999999e-05 2024-09-13T10:11:02.285 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.285 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.285 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.286 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.451780+0000", 2024-09-13T10:11:02.286 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4936999999999999e-05 2024-09-13T10:11:02.286 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.286 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.286 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.286 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.451878+0000", 2024-09-13T10:11:02.286 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7787000000000004e-05 2024-09-13T10:11:02.286 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.286 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.286 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.286 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.451907+0000", 2024-09-13T10:11:02.286 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9505999999999998e-05 2024-09-13T10:11:02.287 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.287 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.287 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.287 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.460238+0000", 2024-09-13T10:11:02.287 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0083305789999999994 2024-09-13T10:11:02.287 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.287 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.287 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.287 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.460299+0000", 2024-09-13T10:11:02.287 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.1179999999999994e-05 2024-09-13T10:11:02.287 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.287 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.287 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.288 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.288 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.288 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:978 3.3 3:ff3c63cb:::0ce0556d49e3d107a4b09599016178b0eacb474223b61814958b982e2164bc90:head [write 0~1038336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:02.288 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.130123+0000", 2024-09-13T10:11:02.288 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 7.065315075, 2024-09-13T10:11:02.288 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010302242, 2024-09-13T10:11:02.288 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.288 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.288 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.288 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:02.288 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:02.288 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:02.288 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 978 2024-09-13T10:11:02.289 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.289 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.289 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.289 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.289 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.130123+0000", 2024-09-13T10:11:02.289 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.289 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.289 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.289 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.289 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.130123+0000", 2024-09-13T10:11:02.289 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.289 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.289 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.290 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.290 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.130127+0000", 2024-09-13T10:11:02.290 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.529e-06 2024-09-13T10:11:02.290 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.290 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.290 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.290 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.131520+0000", 2024-09-13T10:11:02.290 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001392496 2024-09-13T10:11:02.290 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.290 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.290 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.290 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.131557+0000", 2024-09-13T10:11:02.290 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.7734999999999997e-05 2024-09-13T10:11:02.291 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.291 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.291 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.291 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.131574+0000", 2024-09-13T10:11:02.291 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6860000000000001e-05 2024-09-13T10:11:02.291 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.291 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.291 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.291 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.131709+0000", 2024-09-13T10:11:02.291 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013520399999999999 2024-09-13T10:11:02.291 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.291 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.291 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.291 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.131932+0000", 2024-09-13T10:11:02.292 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022303500000000001 2024-09-13T10:11:02.292 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.292 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.292 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.292 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.132173+0000", 2024-09-13T10:11:02.292 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000240857 2024-09-13T10:11:02.292 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.292 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.292 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.292 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.136693+0000", 2024-09-13T10:11:02.292 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0045199840000000003 2024-09-13T10:11:02.292 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.293 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.293 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.293 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.137803+0000", 2024-09-13T10:11:02.293 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011101749999999999 2024-09-13T10:11:02.293 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.293 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.293 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.293 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.140141+0000", 2024-09-13T10:11:02.293 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023376569999999999 2024-09-13T10:11:02.293 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.293 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.293 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.293 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.140299+0000", 2024-09-13T10:11:02.294 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015800300000000001 2024-09-13T10:11:02.294 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.294 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.294 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.294 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.140426+0000", 2024-09-13T10:11:02.294 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012670699999999999 2024-09-13T10:11:02.294 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.294 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.294 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.294 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.294 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.294 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.7.22:23 3.3 3:f074185b:::60e8f4cccf4a419a087d28a5b7eaa0b3caf3bd0b2896a08d59fadb32a1496aff:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:02.294 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:56.114343+0000", 2024-09-13T10:11:02.294 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 6.0810958570000002, 2024-09-13T10:11:02.295 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0086448110000000005, 2024-09-13T10:11:02.295 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.295 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.295 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.295 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.295 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.295 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.295 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.114343+0000", 2024-09-13T10:11:02.295 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.295 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.295 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.295 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.295 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.114343+0000", 2024-09-13T10:11:02.295 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.295 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.296 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.296 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.296 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.114479+0000", 2024-09-13T10:11:02.296 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013634699999999999 2024-09-13T10:11:02.296 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.296 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.296 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.296 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.116701+0000", 2024-09-13T10:11:02.296 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022223999999999998 2024-09-13T10:11:02.296 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.296 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.296 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.296 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.116739+0000", 2024-09-13T10:11:02.297 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.7781999999999998e-05 2024-09-13T10:11:02.297 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.297 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.297 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.297 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.116762+0000", 2024-09-13T10:11:02.297 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3053000000000001e-05 2024-09-13T10:11:02.297 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.297 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.297 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.297 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.116901+0000", 2024-09-13T10:11:02.297 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013871000000000001 2024-09-13T10:11:02.297 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.297 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.297 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.298 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.117133+0000", 2024-09-13T10:11:02.298 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023221200000000001 2024-09-13T10:11:02.298 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.298 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.298 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.298 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.117691+0000", 2024-09-13T10:11:02.298 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00055798700000000002 2024-09-13T10:11:02.298 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.298 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.298 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.298 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.122439+0000", 2024-09-13T10:11:02.298 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0047479549999999999 2024-09-13T10:11:02.298 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.299 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.299 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.299 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.122577+0000", 2024-09-13T10:11:02.299 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000137797 2024-09-13T10:11:02.299 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.299 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.299 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.299 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.122756+0000", 2024-09-13T10:11:02.299 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017904299999999999 2024-09-13T10:11:02.299 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.299 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.299 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.299 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.122887+0000", 2024-09-13T10:11:02.300 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013131699999999999 2024-09-13T10:11:02.300 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.300 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.300 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.300 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.122988+0000", 2024-09-13T10:11:02.300 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000100208 2024-09-13T10:11:02.300 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.300 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.300 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.300 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.300 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.300 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_repop(osd.6.0:104 4.5 e42/41 4:a7549961:::smithi19530714-27:head v 42'40, mlcod=42'38)", 2024-09-13T10:11:02.300 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:56.665478+0000", 2024-09-13T10:11:02.301 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 5.5299603240000001, 2024-09-13T10:11:02.301 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011125397, 2024-09-13T10:11:02.301 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.301 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.301 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.301 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.301 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.301 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.301 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.665478+0000", 2024-09-13T10:11:02.301 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.301 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.301 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.301 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.302 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.665478+0000", 2024-09-13T10:11:02.302 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.302 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.302 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.302 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.302 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.665481+0000", 2024-09-13T10:11:02.302 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6560000000000001e-06 2024-09-13T10:11:02.302 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.302 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.302 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.302 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.667702+0000", 2024-09-13T10:11:02.302 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022206629999999999 2024-09-13T10:11:02.302 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.303 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.303 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.303 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.667736+0000", 2024-09-13T10:11:02.303 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.4795999999999997e-05 2024-09-13T10:11:02.303 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.303 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.303 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.303 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.667759+0000", 2024-09-13T10:11:02.303 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2180000000000001e-05 2024-09-13T10:11:02.303 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.303 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.303 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.303 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.667954+0000", 2024-09-13T10:11:02.304 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019529500000000001 2024-09-13T10:11:02.304 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.304 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.304 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.304 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.668000+0000", 2024-09-13T10:11:02.304 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.5799e-05 2024-09-13T10:11:02.304 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.304 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.304 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.304 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.676441+0000", 2024-09-13T10:11:02.304 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0084409020000000001 2024-09-13T10:11:02.304 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.304 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.304 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.305 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.676604+0000", 2024-09-13T10:11:02.305 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016310599999999999 2024-09-13T10:11:02.305 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.305 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.305 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.305 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.305 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.305 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1106 3.3 3:fe41cd1b:::4f893b68aa248b8fb4544a484fa5e966179cdf2e6bafa35a82fbb4198705f32c:head [write 0~1419776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:02.305 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:56.680017+0000", 2024-09-13T10:11:02.305 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 5.5154210910000003, 2024-09-13T10:11:02.305 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010313045, 2024-09-13T10:11:02.305 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.305 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.306 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.306 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:02.306 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:02.306 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:02.306 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1106 2024-09-13T10:11:02.306 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.306 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.306 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.306 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.306 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.680017+0000", 2024-09-13T10:11:02.306 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.306 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.306 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.307 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.307 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.680017+0000", 2024-09-13T10:11:02.307 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.307 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.307 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.307 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.307 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.680028+0000", 2024-09-13T10:11:02.307 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0509999999999999e-05 2024-09-13T10:11:02.307 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.307 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.307 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.307 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.681990+0000", 2024-09-13T10:11:02.307 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019622519999999998 2024-09-13T10:11:02.308 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.308 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.308 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.308 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.682022+0000", 2024-09-13T10:11:02.308 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1296e-05 2024-09-13T10:11:02.308 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.308 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.308 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.308 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.682037+0000", 2024-09-13T10:11:02.308 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.501e-05 2024-09-13T10:11:02.308 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.308 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.308 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.309 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.682150+0000", 2024-09-13T10:11:02.309 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000113473 2024-09-13T10:11:02.309 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.309 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.309 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.309 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.682422+0000", 2024-09-13T10:11:02.309 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00027188100000000002 2024-09-13T10:11:02.309 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.309 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.309 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.309 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.682673+0000", 2024-09-13T10:11:02.309 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000251074 2024-09-13T10:11:02.309 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.310 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.310 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.310 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.689336+0000", 2024-09-13T10:11:02.310 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0066627789999999998 2024-09-13T10:11:02.310 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.310 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.310 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.310 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.689908+0000", 2024-09-13T10:11:02.310 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000572267 2024-09-13T10:11:02.310 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.310 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.310 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.310 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.690173+0000", 2024-09-13T10:11:02.311 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026483399999999998 2024-09-13T10:11:02.311 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.311 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.311 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.311 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.690257+0000", 2024-09-13T10:11:02.311 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.3881000000000005e-05 2024-09-13T10:11:02.311 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.311 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.311 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.311 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.690331+0000", 2024-09-13T10:11:02.311 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.3788000000000006e-05 2024-09-13T10:11:02.311 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.311 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.312 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.312 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.312 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.312 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.13.21:28 3.3 3:e0983e13:::5461c8cdc9cef4176ad079aec63fd1e488535874105627b16205f8e2e653cf41:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e45)", 2024-09-13T10:11:02.312 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:59.914541+0000", 2024-09-13T10:11:02.312 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 2.280897478, 2024-09-13T10:11:02.312 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0085498629999999996, 2024-09-13T10:11:02.312 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.312 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.312 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.312 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.312 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.312 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.313 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:59.914541+0000", 2024-09-13T10:11:02.313 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.313 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.313 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.313 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.313 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:59.914541+0000", 2024-09-13T10:11:02.313 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.313 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.313 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.313 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.313 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:59.914548+0000", 2024-09-13T10:11:02.313 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.6470000000000001e-06 2024-09-13T10:11:02.313 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.314 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.314 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.314 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:59.916303+0000", 2024-09-13T10:11:02.314 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001754893 2024-09-13T10:11:02.314 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.314 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.314 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.314 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:59.916346+0000", 2024-09-13T10:11:02.314 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.3375999999999999e-05 2024-09-13T10:11:02.314 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.314 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.314 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.314 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:59.916372+0000", 2024-09-13T10:11:02.315 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6205999999999999e-05 2024-09-13T10:11:02.315 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.315 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.315 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.315 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:59.916601+0000", 2024-09-13T10:11:02.315 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022833900000000001 2024-09-13T10:11:02.315 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.315 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.315 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.315 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:59.916866+0000", 2024-09-13T10:11:02.315 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026582100000000002 2024-09-13T10:11:02.315 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.315 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.316 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.316 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:59.917387+0000", 2024-09-13T10:11:02.316 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00052028299999999997 2024-09-13T10:11:02.316 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.316 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.316 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.316 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:59.922679+0000", 2024-09-13T10:11:02.316 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0052926029999999999 2024-09-13T10:11:02.316 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.316 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.316 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.316 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:59.922809+0000", 2024-09-13T10:11:02.317 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012950400000000001 2024-09-13T10:11:02.317 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.317 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.317 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.317 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:59.922832+0000", 2024-09-13T10:11:02.317 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3090000000000001e-05 2024-09-13T10:11:02.317 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.317 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.317 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.317 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:59.922962+0000", 2024-09-13T10:11:02.317 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012988999999999999 2024-09-13T10:11:02.317 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.317 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.318 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.318 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:59.923091+0000", 2024-09-13T10:11:02.318 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012921100000000001 2024-09-13T10:11:02.318 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.318 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.318 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.318 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.318 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.318 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.6.23:70 3.3 3:f941658c:::smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e46)", 2024-09-13T10:11:02.318 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:00.468555+0000", 2024-09-13T10:11:02.318 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 1.72688318, 2024-09-13T10:11:02.318 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0096087080000000005, 2024-09-13T10:11:02.318 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:02.319 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:02.319 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:02.319 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:02.319 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.319 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:02.319 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:00.468555+0000", 2024-09-13T10:11:02.319 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.319 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.319 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.319 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:02.319 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:00.468555+0000", 2024-09-13T10:11:02.319 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:02.319 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.319 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.320 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:02.320 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:00.468556+0000", 2024-09-13T10:11:02.320 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5300000000000003e-07 2024-09-13T10:11:02.320 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.320 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.320 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:02.320 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:00.468568+0000", 2024-09-13T10:11:02.320 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2037e-05 2024-09-13T10:11:02.320 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.320 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.320 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:02.320 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:00.468596+0000", 2024-09-13T10:11:02.320 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8068999999999998e-05 2024-09-13T10:11:02.321 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.321 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.321 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:02.321 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:00.468605+0000", 2024-09-13T10:11:02.321 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.1829999999999994e-06 2024-09-13T10:11:02.321 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.321 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.321 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:02.321 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:00.471988+0000", 2024-09-13T10:11:02.321 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003383217 2024-09-13T10:11:02.321 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.321 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.321 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:02.322 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:00.472160+0000", 2024-09-13T10:11:02.322 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000172483 2024-09-13T10:11:02.322 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.322 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.322 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:02.322 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:00.472461+0000", 2024-09-13T10:11:02.322 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00030117399999999997 2024-09-13T10:11:02.322 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.322 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.322 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:02.322 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:00.474006+0000", 2024-09-13T10:11:02.322 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0015445540000000001 2024-09-13T10:11:02.322 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.322 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.323 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.323 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:00.474370+0000", 2024-09-13T10:11:02.323 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036372999999999999 2024-09-13T10:11:02.323 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.323 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.323 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:02.323 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:00.478042+0000", 2024-09-13T10:11:02.323 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0036723469999999999 2024-09-13T10:11:02.323 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.323 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.323 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:02.323 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:00.478114+0000", 2024-09-13T10:11:02.323 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2121999999999998e-05 2024-09-13T10:11:02.324 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:02.324 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:02.324 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:02.324 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:00.478164+0000", 2024-09-13T10:11:02.324 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.9938999999999997e-05 2024-09-13T10:11:02.324 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.324 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.324 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.324 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:02.324 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:02.324 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:02.326 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:02.326 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:02.326 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:02.197+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:02.326 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:02.323+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:02.327 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:02.327 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:02.328 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:02.328 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-13T10:11:02.328 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:02.329 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:02.329 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:02.329 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:02.330 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:02.330 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:02.330 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:02.330 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:02.331 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:02.331 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:02.379 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T10:11:02.401 INFO:tasks.rados.rados.0.smithi195.stdout:820: expect (ObjNum 95 snap 0 seq_num 95) 2024-09-13T10:11:02.401 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:02.427 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:02.425+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:02.501 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:02.501 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:02.515 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-13T10:11:02.560 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:02.560+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:02.580 INFO:tasks.rados.rados.0.smithi195.stdout:822: finishing write tid 1 to smithi19530714-9 2024-09-13T10:11:02.580 INFO:tasks.rados.rados.0.smithi195.stdout:822: finishing write tid 2 to smithi19530714-9 2024-09-13T10:11:02.580 INFO:tasks.rados.rados.0.smithi195.stdout:822: finishing write tid 3 to smithi19530714-9 2024-09-13T10:11:02.580 INFO:tasks.rados.rados.0.smithi195.stdout:822: finishing write tid 4 to smithi19530714-9 2024-09-13T10:11:02.580 INFO:tasks.rados.rados.0.smithi195.stdout:822: finishing write tid 5 to smithi19530714-9 2024-09-13T10:11:02.580 INFO:tasks.rados.rados.0.smithi195.stdout:822: oid 9 updating version 0 to 59 2024-09-13T10:11:02.580 INFO:tasks.rados.rados.0.smithi195.stdout:822: oid 9 updating version 59 to 60 2024-09-13T10:11:02.581 INFO:tasks.rados.rados.0.smithi195.stdout:822: oid 9 updating version 60 to 61 2024-09-13T10:11:02.581 INFO:tasks.rados.rados.0.smithi195.stdout:822: oid 9 version 61 is already newer than 58 2024-09-13T10:11:02.581 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 61 (ObjNum 113 snap 0 seq_num 113) dirty exists 2024-09-13T10:11:02.581 INFO:tasks.rados.rados.0.smithi195.stdout:822: left oid 9 (ObjNum 113 snap 0 seq_num 113) 2024-09-13T10:11:02.581 INFO:tasks.rados.rados.0.smithi195.stdout:823: finishing write tid 1 to smithi19530714-22 2024-09-13T10:11:02.581 INFO:tasks.rados.rados.0.smithi195.stdout:823: finishing write tid 2 to smithi19530714-22 2024-09-13T10:11:02.581 INFO:tasks.rados.rados.0.smithi195.stdout:823: finishing write tid 3 to smithi19530714-22 2024-09-13T10:11:02.581 INFO:tasks.rados.rados.0.smithi195.stdout:823: finishing write tid 4 to smithi19530714-22 2024-09-13T10:11:02.581 INFO:tasks.rados.rados.0.smithi195.stdout:823: finishing write tid 5 to smithi19530714-22 2024-09-13T10:11:02.581 INFO:tasks.rados.rados.0.smithi195.stdout:823: finishing write tid 6 to smithi19530714-22 2024-09-13T10:11:02.581 INFO:tasks.rados.rados.0.smithi195.stdout:823: oid 22 updating version 0 to 65 2024-09-13T10:11:02.581 INFO:tasks.rados.rados.0.smithi195.stdout:823: oid 22 version 65 is already newer than 63 2024-09-13T10:11:02.582 INFO:tasks.rados.rados.0.smithi195.stdout:823: oid 22 version 65 is already newer than 62 2024-09-13T10:11:02.582 INFO:tasks.rados.rados.0.smithi195.stdout:823: oid 22 updating version 65 to 66 2024-09-13T10:11:02.582 INFO:tasks.rados.rados.0.smithi195.stdout:823: oid 22 version 66 is already newer than 64 2024-09-13T10:11:02.582 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 66 (ObjNum 114 snap 0 seq_num 114) dirty exists 2024-09-13T10:11:02.582 INFO:tasks.rados.rados.0.smithi195.stdout:823: left oid 22 (ObjNum 114 snap 0 seq_num 114) 2024-09-13T10:11:02.582 INFO:tasks.rados.rados.0.smithi195.stdout:824: finishing write tid 1 to smithi19530714-4 2024-09-13T10:11:02.582 INFO:tasks.rados.rados.0.smithi195.stdout:824: finishing write tid 2 to smithi19530714-4 2024-09-13T10:11:02.582 INFO:tasks.rados.rados.0.smithi195.stdout:830: got (0) Success 2024-09-13T10:11:02.582 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 26 (ObjNum 51 snap 0 seq_num 51) dirty exists 2024-09-13T10:11:02.582 INFO:tasks.rados.rados.0.smithi195.stdout:824: finishing write tid 3 to smithi19530714-4 2024-09-13T10:11:02.582 INFO:tasks.rados.rados.0.smithi195.stdout:824: finishing write tid 4 to smithi19530714-4 2024-09-13T10:11:02.582 INFO:tasks.rados.rados.0.smithi195.stdout:824: finishing write tid 5 to smithi19530714-4 2024-09-13T10:11:02.582 INFO:tasks.rados.rados.0.smithi195.stdout:824: oid 4 updating version 0 to 45 2024-09-13T10:11:02.582 INFO:tasks.rados.rados.0.smithi195.stdout:824: oid 4 updating version 45 to 46 2024-09-13T10:11:02.583 INFO:tasks.rados.rados.0.smithi195.stdout:824: oid 4 version 46 is already newer than 44 2024-09-13T10:11:02.583 INFO:tasks.rados.rados.0.smithi195.stdout:824: oid 4 version 46 is already newer than 43 2024-09-13T10:11:02.583 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 46 (ObjNum 115 snap 0 seq_num 115) dirty exists 2024-09-13T10:11:02.583 INFO:tasks.rados.rados.0.smithi195.stdout:824: left oid 4 (ObjNum 115 snap 0 seq_num 115) 2024-09-13T10:11:02.583 INFO:tasks.rados.rados.0.smithi195.stdout:828: expect (ObjNum 46 snap 0 seq_num 46) 2024-09-13T10:11:02.583 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 345377 2024-09-13T10:11:02.629 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:02.627+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:02.631 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:02.638 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:02.638 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:02.652 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-13T10:11:02.661 INFO:tasks.rados.rados.0.smithi195.stdout:825: expect (ObjNum 25 snap 0 seq_num 25) 2024-09-13T10:11:02.684 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1226504 2024-09-13T10:11:02.717 INFO:tasks.rados.rados.0.smithi195.stdout:829: expect (ObjNum 80 snap 0 seq_num 80) 2024-09-13T10:11:02.717 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:02.730 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:02.727+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:02.782 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:02.814 INFO:tasks.rados.rados.0.smithi195.stdout:826: expect (ObjNum 103 snap 0 seq_num 103) 2024-09-13T10:11:02.825 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:02.825 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:02.830 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:02.828+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:02.837 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T10:11:02.862 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:02.910 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:02.932 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:02.929+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:02.957 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:02.957 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:02.959 INFO:tasks.rados.rados.0.smithi195.stdout:827: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:11:02.971 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T10:11:02.973 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1888524 2024-09-13T10:11:03.033 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:03.032+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:03.057 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2053668 2024-09-13T10:11:03.078 INFO:tasks.rados.rados.0.smithi195.stdout:831: got (22) Invalid argument 2024-09-13T10:11:03.079 INFO:tasks.rados.rados.0.smithi195.stdout:832: seq_num 116 ranges {749612=474179,1954970=431037,2919420=616377} 2024-09-13T10:11:03.088 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:03.089 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:03.098 INFO:tasks.rados.rados.0.smithi195.stdout:832: writing smithi19530714-49 from 749612 to 1223791 tid 1 2024-09-13T10:11:03.102 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T10:11:03.117 INFO:tasks.rados.rados.0.smithi195.stdout:832: writing smithi19530714-49 from 1954970 to 2386007 tid 2 2024-09-13T10:11:03.134 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:03.132+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:03.135 INFO:tasks.rados.rados.0.smithi195.stdout:832: writing smithi19530714-49 from 2919420 to 3535797 tid 3 2024-09-13T10:11:03.135 INFO:tasks.rados.rados.0.smithi195.stdout:820: done (12 left) 2024-09-13T10:11:03.135 INFO:tasks.rados.rados.0.smithi195.stdout:821: done (11 left) 2024-09-13T10:11:03.135 INFO:tasks.rados.rados.0.smithi195.stdout:822: done (10 left) 2024-09-13T10:11:03.135 INFO:tasks.rados.rados.0.smithi195.stdout:823: done (9 left) 2024-09-13T10:11:03.135 INFO:tasks.rados.rados.0.smithi195.stdout:824: done (8 left) 2024-09-13T10:11:03.136 INFO:tasks.rados.rados.0.smithi195.stdout:825: done (7 left) 2024-09-13T10:11:03.136 INFO:tasks.rados.rados.0.smithi195.stdout:826: done (6 left) 2024-09-13T10:11:03.136 INFO:tasks.rados.rados.0.smithi195.stdout:827: done (5 left) 2024-09-13T10:11:03.136 INFO:tasks.rados.rados.0.smithi195.stdout:828: done (4 left) 2024-09-13T10:11:03.136 INFO:tasks.rados.rados.0.smithi195.stdout:829: done (3 left) 2024-09-13T10:11:03.136 INFO:tasks.rados.rados.0.smithi195.stdout:830: done (2 left) 2024-09-13T10:11:03.136 INFO:tasks.rados.rados.0.smithi195.stdout:831: done (1 left) 2024-09-13T10:11:03.136 INFO:tasks.rados.rados.0.smithi195.stdout:833: set_chunk oid 23 target oid 2024-09-13T10:11:03.153 INFO:tasks.rados.rados.0.smithi195.stdout:833: set_chunk oid 23 offset: 1484800 length: 998400 target oid c3ae356ca039175b1900e044f1a56c11677601b30c3ce5423445050e0a912178 offset: 0 snap -1 2024-09-13T10:11:03.153 INFO:tasks.rados.rados.0.smithi195.stdout:834: tier_evict oid 37 2024-09-13T10:11:03.153 INFO:tasks.rados.rados.0.smithi195.stdout:834: tier_evict oid 37 snap -1 2024-09-13T10:11:03.153 INFO:tasks.rados.rados.0.smithi195.stdout:832: finishing write tid 1 to smithi19530714-49 2024-09-13T10:11:03.153 INFO:tasks.rados.rados.0.smithi195.stdout:832: finishing write tid 2 to smithi19530714-49 2024-09-13T10:11:03.153 INFO:tasks.rados.rados.0.smithi195.stdout:832: finishing write tid 3 to smithi19530714-49 2024-09-13T10:11:03.153 INFO:tasks.rados.rados.0.smithi195.stdout:832: finishing write tid 4 to smithi19530714-49 2024-09-13T10:11:03.154 INFO:tasks.rados.rados.0.smithi195.stdout:832: finishing write tid 5 to smithi19530714-49 2024-09-13T10:11:03.154 INFO:tasks.rados.rados.0.smithi195.stdout:832: oid 49 updating version 0 to 64 2024-09-13T10:11:03.154 INFO:tasks.rados.rados.0.smithi195.stdout:832: oid 49 version 64 is already newer than 63 2024-09-13T10:11:03.154 INFO:tasks.rados.rados.0.smithi195.stdout:832: oid 49 updating version 64 to 66 2024-09-13T10:11:03.154 INFO:tasks.rados.rados.0.smithi195.stdout:832: oid 49 version 66 is already newer than 65 2024-09-13T10:11:03.154 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 66 (ObjNum 116 snap 0 seq_num 116) dirty exists 2024-09-13T10:11:03.154 INFO:tasks.rados.rados.0.smithi195.stdout:832: left oid 49 (ObjNum 116 snap 0 seq_num 116) 2024-09-13T10:11:03.154 INFO:tasks.rados.rados.0.smithi195.stdout:832: done (2 left) 2024-09-13T10:11:03.154 INFO:tasks.rados.rados.0.smithi195.stdout:835: read oid 43 snap -1 2024-09-13T10:11:03.154 INFO:tasks.rados.rados.0.smithi195.stdout:835: expect (ObjNum 101 snap 0 seq_num 101) 2024-09-13T10:11:03.154 INFO:tasks.rados.rados.0.smithi195.stdout:836: read oid 29 snap -1 2024-09-13T10:11:03.154 INFO:tasks.rados.rados.0.smithi195.stdout:836: expect deleted 2024-09-13T10:11:03.154 INFO:tasks.rados.rados.0.smithi195.stdout:837: set_chunk oid 8 target oid 2024-09-13T10:11:03.172 INFO:tasks.rados.rados.0.smithi195.stdout:837: set_chunk oid 8 offset: 2256384 length: 872448 target oid fa7b494997ac90334f2eb2075334bae225f7bc34bf9ca7b6d92f26af2ccc570d offset: 0 snap -1 2024-09-13T10:11:03.172 INFO:tasks.rados.rados.0.smithi195.stdout:838: write (excl) oid 50 current snap is 0 2024-09-13T10:11:03.172 INFO:tasks.rados.rados.0.smithi195.stdout:838: seq_num 117 ranges {766419=549428,2013452=574695,3346843=427649} 2024-09-13T10:11:03.184 INFO:tasks.rados.rados.0.smithi195.stdout:838: writing smithi19530714-50 from 766419 to 1315847 tid 1 2024-09-13T10:11:03.193 INFO:tasks.rados.rados.0.smithi195.stdout:838: writing smithi19530714-50 from 2013452 to 2588147 tid 2 2024-09-13T10:11:03.202 INFO:tasks.rados.rados.0.smithi195.stdout:838: writing smithi19530714-50 from 3346843 to 3774492 tid 3 2024-09-13T10:11:03.202 INFO:tasks.rados.rados.0.smithi195.stdout:839: write oid 22 current snap is 0 2024-09-13T10:11:03.202 INFO:tasks.rados.rados.0.smithi195.stdout:839: seq_num 118 ranges {783226=624677,2071934=434706,2974266=638921} 2024-09-13T10:11:03.212 INFO:tasks.rados.rados.0.smithi195.stdout:839: writing smithi19530714-22 from 783226 to 1407903 tid 1 2024-09-13T10:11:03.220 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:03.220 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:03.221 INFO:tasks.rados.rados.0.smithi195.stdout:839: writing smithi19530714-22 from 2071934 to 2506640 tid 2 2024-09-13T10:11:03.229 INFO:tasks.rados.rados.0.smithi195.stdout:839: writing smithi19530714-22 from 2974266 to 3613187 tid 3 2024-09-13T10:11:03.229 INFO:tasks.rados.rados.0.smithi195.stdout:840: delete oid 3 current snap is 0 2024-09-13T10:11:03.229 INFO:tasks.rados.rados.0.smithi195.stdout:834: got (22) Invalid argument 2024-09-13T10:11:03.229 INFO:tasks.rados.rados.0.smithi195.stdout:833: finishing set_chunk to oid 23 2024-09-13T10:11:03.229 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 28 (ObjNum 90 snap 0 seq_num 90) dirty exists 2024-09-13T10:11:03.229 INFO:tasks.rados.rados.0.smithi195.stdout:835: expect (ObjNum 101 snap 0 seq_num 101) 2024-09-13T10:11:03.235 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T10:11:03.235 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:03.231+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:03.245 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1412513 2024-09-13T10:11:03.289 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:03.335 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:03.333+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:03.357 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:03.358 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:03.358 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:03.358 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:11:03.358 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:11:03.358 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:11:03.359 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-13T10:11:03.359 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-13T10:11:03.372 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T10:11:03.378 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-13T10:11:03.422 INFO:tasks.rados.rados.0.smithi195.stdout:837: finishing set_chunk to oid 8 2024-09-13T10:11:03.422 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 9 (ObjNum 104 snap 0 seq_num 104) dirty exists 2024-09-13T10:11:03.422 INFO:tasks.rados.rados.0.smithi195.stdout:838: finishing write tid 1 to smithi19530714-50 2024-09-13T10:11:03.422 INFO:tasks.rados.rados.0.smithi195.stdout:838: finishing write tid 2 to smithi19530714-50 2024-09-13T10:11:03.422 INFO:tasks.rados.rados.0.smithi195.stdout:838: finishing write tid 3 to smithi19530714-50 2024-09-13T10:11:03.422 INFO:tasks.rados.rados.0.smithi195.stdout:838: finishing write tid 4 to smithi19530714-50 2024-09-13T10:11:03.423 INFO:tasks.rados.rados.0.smithi195.stdout:838: finishing write tid 5 to smithi19530714-50 2024-09-13T10:11:03.423 INFO:tasks.rados.rados.0.smithi195.stdout:838: oid 50 updating version 0 to 20 2024-09-13T10:11:03.423 INFO:tasks.rados.rados.0.smithi195.stdout:838: oid 50 version 20 is already newer than 17 2024-09-13T10:11:03.423 INFO:tasks.rados.rados.0.smithi195.stdout:838: oid 50 version 20 is already newer than 19 2024-09-13T10:11:03.423 INFO:tasks.rados.rados.0.smithi195.stdout:838: oid 50 version 20 is already newer than 18 2024-09-13T10:11:03.423 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 20 (ObjNum 117 snap 0 seq_num 117) dirty exists 2024-09-13T10:11:03.423 INFO:tasks.rados.rados.0.smithi195.stdout:838: left oid 50 (ObjNum 117 snap 0 seq_num 117) 2024-09-13T10:11:03.423 INFO:tasks.rados.rados.0.smithi195.stdout:839: finishing write tid 1 to smithi19530714-22 2024-09-13T10:11:03.423 INFO:tasks.rados.rados.0.smithi195.stdout:839: finishing write tid 2 to smithi19530714-22 2024-09-13T10:11:03.423 INFO:tasks.rados.rados.0.smithi195.stdout:839: finishing write tid 3 to smithi19530714-22 2024-09-13T10:11:03.423 INFO:tasks.rados.rados.0.smithi195.stdout:839: finishing write tid 4 to smithi19530714-22 2024-09-13T10:11:03.423 INFO:tasks.rados.rados.0.smithi195.stdout:839: finishing write tid 5 to smithi19530714-22 2024-09-13T10:11:03.423 INFO:tasks.rados.rados.0.smithi195.stdout:839: oid 22 updating version 0 to 67 2024-09-13T10:11:03.423 INFO:tasks.rados.rados.0.smithi195.stdout:839: oid 22 updating version 67 to 70 2024-09-13T10:11:03.424 INFO:tasks.rados.rados.0.smithi195.stdout:839: oid 22 version 70 is already newer than 69 2024-09-13T10:11:03.424 INFO:tasks.rados.rados.0.smithi195.stdout:839: oid 22 version 70 is already newer than 68 2024-09-13T10:11:03.424 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 70 (ObjNum 118 snap 0 seq_num 118) dirty exists 2024-09-13T10:11:03.424 INFO:tasks.rados.rados.0.smithi195.stdout:839: left oid 22 (ObjNum 118 snap 0 seq_num 118) 2024-09-13T10:11:03.424 INFO:tasks.rados.rados.0.smithi195.stdout:833: done (7 left) 2024-09-13T10:11:03.424 INFO:tasks.rados.rados.0.smithi195.stdout:834: done (6 left) 2024-09-13T10:11:03.424 INFO:tasks.rados.rados.0.smithi195.stdout:835: done (5 left) 2024-09-13T10:11:03.424 INFO:tasks.rados.rados.0.smithi195.stdout:836: done (4 left) 2024-09-13T10:11:03.425 INFO:tasks.rados.rados.0.smithi195.stdout:837: done (3 left) 2024-09-13T10:11:03.425 INFO:tasks.rados.rados.0.smithi195.stdout:838: done (2 left) 2024-09-13T10:11:03.425 INFO:tasks.rados.rados.0.smithi195.stdout:839: done (1 left) 2024-09-13T10:11:03.425 INFO:tasks.rados.rados.0.smithi195.stdout:840: done (0 left) 2024-09-13T10:11:03.425 INFO:tasks.rados.rados.0.smithi195.stdout:841: read oid 17 snap -1 2024-09-13T10:11:03.425 INFO:tasks.rados.rados.0.smithi195.stdout:841: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-13T10:11:03.425 INFO:tasks.rados.rados.0.smithi195.stdout:842: read oid 13 snap -1 2024-09-13T10:11:03.425 INFO:tasks.rados.rados.0.smithi195.stdout:842: expect (ObjNum 56 snap 0 seq_num 56) 2024-09-13T10:11:03.425 INFO:tasks.rados.rados.0.smithi195.stdout:843: write (excl) oid 44 current snap is 0 2024-09-13T10:11:03.426 INFO:tasks.rados.rados.0.smithi195.stdout:843: seq_num 120 ranges {416840=775175,1672545=344295} 2024-09-13T10:11:03.433 INFO:tasks.rados.rados.0.smithi195.stdout:843: writing smithi19530714-44 from 416840 to 1192015 tid 1 2024-09-13T10:11:03.436 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:03.436+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:03.441 INFO:tasks.rados.rados.0.smithi195.stdout:843: writing smithi19530714-44 from 1672545 to 2016840 tid 2 2024-09-13T10:11:03.441 INFO:tasks.rados.rados.0.smithi195.stdout:844: read oid 35 snap -1 2024-09-13T10:11:03.442 INFO:tasks.rados.rados.0.smithi195.stdout:844: expect (ObjNum 86 snap 0 seq_num 86) 2024-09-13T10:11:03.442 INFO:tasks.rados.rados.0.smithi195.stdout:845: read oid 25 snap -1 2024-09-13T10:11:03.442 INFO:tasks.rados.rados.0.smithi195.stdout:845: expect deleted 2024-09-13T10:11:03.442 INFO:tasks.rados.rados.0.smithi195.stdout:846: tier_flush oid 41 2024-09-13T10:11:03.442 INFO:tasks.rados.rados.0.smithi195.stdout:846: tier_flush oid 41 snap -1 2024-09-13T10:11:03.442 INFO:tasks.rados.rados.0.smithi195.stdout:842: expect (ObjNum 56 snap 0 seq_num 56) 2024-09-13T10:11:03.537 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:03.536+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:03.541 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-13T10:11:03.541 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-13T10:11:03.542 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-13T10:11:03.542 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-13T10:11:03.542 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-13T10:11:03.542 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-13T10:11:03.542 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-09-13T10:11:03.542 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-13T10:11:03.542 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-13T10:11:03.542 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-13T10:11:03.542 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-13T10:11:03.542 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-09-13T10:11:03.544 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T10:11:03.545 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-13T10:11:03.545 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-13T10:11:03.546 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:03.546 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:03.558 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T10:11:03.576 INFO:tasks.rados.rados.0.smithi195.stdout:841: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-13T10:11:03.617 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:03.638 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:03.636+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:03.668 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1280103 2024-09-13T10:11:03.692 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:03.692 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:03.695 INFO:tasks.rados.rados.0.smithi195.stdout:843: finishing write tid 1 to smithi19530714-44 2024-09-13T10:11:03.695 INFO:tasks.rados.rados.0.smithi195.stdout:843: finishing write tid 2 to smithi19530714-44 2024-09-13T10:11:03.695 INFO:tasks.rados.rados.0.smithi195.stdout:843: finishing write tid 3 to smithi19530714-44 2024-09-13T10:11:03.695 INFO:tasks.rados.rados.0.smithi195.stdout:844: expect (ObjNum 86 snap 0 seq_num 86) 2024-09-13T10:11:03.695 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:03.696 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-13T10:11:03.696 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-13T10:11:03.697 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-13T10:11:03.697 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-13T10:11:03.697 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-13T10:11:03.697 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-13T10:11:03.697 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-09-13T10:11:03.697 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-13T10:11:03.697 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-13T10:11:03.697 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-13T10:11:03.697 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-13T10:11:03.697 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-09-13T10:11:03.700 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T10:11:03.700 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-13T10:11:03.700 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-13T10:11:03.706 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-13T10:11:03.738 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:03.736+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:03.753 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:03.811 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:03.839 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:03.837+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:03.857 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-13T10:11:03.857 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-13T10:11:03.857 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-13T10:11:03.857 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-13T10:11:03.857 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-13T10:11:03.857 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-13T10:11:03.857 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-09-13T10:11:03.857 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-13T10:11:03.857 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-13T10:11:03.857 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-13T10:11:03.857 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-13T10:11:03.857 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-09-13T10:11:03.858 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:03.858 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:03.860 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T10:11:03.860 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-13T10:11:03.860 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-13T10:11:03.869 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-13T10:11:03.870 INFO:tasks.rados.rados.0.smithi195.stdout:843: finishing write tid 4 to smithi19530714-44 2024-09-13T10:11:03.870 INFO:tasks.rados.rados.0.smithi195.stdout:843: oid 44 updating version 0 to 39 2024-09-13T10:11:03.870 INFO:tasks.rados.rados.0.smithi195.stdout:843: oid 44 version 39 is already newer than 37 2024-09-13T10:11:03.870 INFO:tasks.rados.rados.0.smithi195.stdout:843: oid 44 version 39 is already newer than 38 2024-09-13T10:11:03.870 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 39 (ObjNum 120 snap 0 seq_num 120) dirty exists 2024-09-13T10:11:03.871 INFO:tasks.rados.rados.0.smithi195.stdout:843: left oid 44 (ObjNum 120 snap 0 seq_num 120) 2024-09-13T10:11:03.871 INFO:tasks.rados.rados.0.smithi195.stdout:846: got (0) Success 2024-09-13T10:11:03.871 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 0 (ObjNum 94 snap 0 seq_num 94) dirty exists 2024-09-13T10:11:03.871 INFO:tasks.rados.rados.0.smithi195.stdout:841: done (5 left) 2024-09-13T10:11:03.871 INFO:tasks.rados.rados.0.smithi195.stdout:842: done (4 left) 2024-09-13T10:11:03.871 INFO:tasks.rados.rados.0.smithi195.stdout:843: done (3 left) 2024-09-13T10:11:03.871 INFO:tasks.rados.rados.0.smithi195.stdout:844: done (2 left) 2024-09-13T10:11:03.871 INFO:tasks.rados.rados.0.smithi195.stdout:845: done (1 left) 2024-09-13T10:11:03.871 INFO:tasks.rados.rados.0.smithi195.stdout:846: done (0 left) 2024-09-13T10:11:03.871 INFO:tasks.rados.rados.0.smithi195.stdout:847: write (excl) oid 24 current snap is 0 2024-09-13T10:11:03.871 INFO:tasks.rados.rados.0.smithi195.stdout:847: seq_num 121 ranges {433647=450424,1331027=582033} 2024-09-13T10:11:03.874 INFO:tasks.rados.rados.0.smithi195.stdout:847: writing smithi19530714-24 from 433647 to 884071 tid 1 2024-09-13T10:11:03.880 INFO:tasks.rados.rados.0.smithi195.stdout:847: writing smithi19530714-24 from 1331027 to 1913060 tid 2 2024-09-13T10:11:03.880 INFO:tasks.rados.rados.0.smithi195.stdout:848: write (excl) oid 26 current snap is 0 2024-09-13T10:11:03.880 INFO:tasks.rados.rados.0.smithi195.stdout:848: seq_num 122 ranges {450454=642026,1505862=544592} 2024-09-13T10:11:03.886 INFO:tasks.rados.rados.0.smithi195.stdout:848: writing smithi19530714-26 from 450454 to 1092480 tid 1 2024-09-13T10:11:03.892 INFO:tasks.rados.rados.0.smithi195.stdout:848: writing smithi19530714-26 from 1505862 to 2050454 tid 2 2024-09-13T10:11:03.892 INFO:tasks.rados.rados.0.smithi195.stdout:849: read oid 40 snap -1 2024-09-13T10:11:03.892 INFO:tasks.rados.rados.0.smithi195.stdout:849: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-13T10:11:03.892 INFO:tasks.rados.rados.0.smithi195.stdout:850: read oid 41 snap -1 2024-09-13T10:11:03.892 INFO:tasks.rados.rados.0.smithi195.stdout:850: expect (ObjNum 94 snap 0 seq_num 94) 2024-09-13T10:11:03.892 INFO:tasks.rados.rados.0.smithi195.stdout:851: tier_promote oid 43 2024-09-13T10:11:03.892 INFO:tasks.rados.rados.0.smithi195.stdout:847: finishing write tid 1 to smithi19530714-24 2024-09-13T10:11:03.892 INFO:tasks.rados.rados.0.smithi195.stdout:847: finishing write tid 2 to smithi19530714-24 2024-09-13T10:11:03.892 INFO:tasks.rados.rados.0.smithi195.stdout:852: read oid 28 snap -1 2024-09-13T10:11:03.892 INFO:tasks.rados.rados.0.smithi195.stdout:852: expect (ObjNum 99 snap 0 seq_num 99) 2024-09-13T10:11:03.892 INFO:tasks.rados.rados.0.smithi195.stdout:853: write oid 48 current snap is 0 2024-09-13T10:11:03.893 INFO:tasks.rados.rados.0.smithi195.stdout:853: seq_num 123 ranges {467261=717275,1847991=219270} 2024-09-13T10:11:03.899 INFO:tasks.rados.rados.0.smithi195.stdout:853: writing smithi19530714-48 from 467261 to 1184536 tid 1 2024-09-13T10:11:03.904 INFO:tasks.rados.rados.0.smithi195.stdout:853: writing smithi19530714-48 from 1847991 to 2067261 tid 2 2024-09-13T10:11:03.904 INFO:tasks.rados.rados.0.smithi195.stdout:854: tier_promote oid 18 2024-09-13T10:11:03.905 INFO:tasks.rados.rados.0.smithi195.stdout:847: finishing write tid 3 to smithi19530714-24 2024-09-13T10:11:03.905 INFO:tasks.rados.rados.0.smithi195.stdout:847: finishing write tid 4 to smithi19530714-24 2024-09-13T10:11:03.905 INFO:tasks.rados.rados.0.smithi195.stdout:847: oid 24 updating version 0 to 69 2024-09-13T10:11:03.905 INFO:tasks.rados.rados.0.smithi195.stdout:847: oid 24 version 69 is already newer than 67 2024-09-13T10:11:03.905 INFO:tasks.rados.rados.0.smithi195.stdout:847: oid 24 version 69 is already newer than 68 2024-09-13T10:11:03.905 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 69 (ObjNum 121 snap 0 seq_num 121) dirty exists 2024-09-13T10:11:03.905 INFO:tasks.rados.rados.0.smithi195.stdout:847: left oid 24 (ObjNum 121 snap 0 seq_num 121) 2024-09-13T10:11:03.905 INFO:tasks.rados.rados.0.smithi195.stdout:851: got (0) Success 2024-09-13T10:11:03.905 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 39 (ObjNum 101 snap 0 seq_num 101) dirty exists 2024-09-13T10:11:03.905 INFO:tasks.rados.rados.0.smithi195.stdout:850: expect (ObjNum 94 snap 0 seq_num 94) 2024-09-13T10:11:03.939 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:03.939+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:03.976 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:03.990 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:03.990 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:04.004 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-13T10:11:04.019 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-13T10:11:04.019 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-13T10:11:04.019 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-13T10:11:04.019 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-13T10:11:04.019 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-13T10:11:04.019 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-13T10:11:04.019 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-09-13T10:11:04.019 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-13T10:11:04.019 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-13T10:11:04.020 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-13T10:11:04.020 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-13T10:11:04.020 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-09-13T10:11:04.022 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T10:11:04.022 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-13T10:11:04.022 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-13T10:11:04.047 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:04.047+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:04.118 INFO:tasks.rados.rados.0.smithi195.stdout:848: finishing write tid 1 to smithi19530714-26 2024-09-13T10:11:04.118 INFO:tasks.rados.rados.0.smithi195.stdout:848: finishing write tid 2 to smithi19530714-26 2024-09-13T10:11:04.118 INFO:tasks.rados.rados.0.smithi195.stdout:848: finishing write tid 3 to smithi19530714-26 2024-09-13T10:11:04.118 INFO:tasks.rados.rados.0.smithi195.stdout:849: expect (ObjNum 39 snap 0 seq_num 39) 2024-09-13T10:11:04.141 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:04.139+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:04.146 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:04.155 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:04.155 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-13T10:11:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-13T10:11:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-13T10:11:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-13T10:11:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-13T10:11:04.166 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-13T10:11:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-09-13T10:11:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-13T10:11:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-13T10:11:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-13T10:11:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-13T10:11:04.167 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-09-13T10:11:04.168 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-13T10:11:04.169 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T10:11:04.169 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-13T10:11:04.169 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-13T10:11:04.173 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:04.200 INFO:tasks.rados.rados.0.smithi195.stdout:848: finishing write tid 4 to smithi19530714-26 2024-09-13T10:11:04.200 INFO:tasks.rados.rados.0.smithi195.stdout:848: oid 26 updating version 0 to 18 2024-09-13T10:11:04.200 INFO:tasks.rados.rados.0.smithi195.stdout:848: oid 26 updating version 18 to 20 2024-09-13T10:11:04.200 INFO:tasks.rados.rados.0.smithi195.stdout:848: oid 26 version 20 is already newer than 19 2024-09-13T10:11:04.200 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 20 (ObjNum 122 snap 0 seq_num 122) dirty exists 2024-09-13T10:11:04.201 INFO:tasks.rados.rados.0.smithi195.stdout:848: left oid 26 (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:04.201 INFO:tasks.rados.rados.0.smithi195.stdout:852: expect (ObjNum 99 snap 0 seq_num 99) 2024-09-13T10:11:04.207 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 936205 2024-09-13T10:11:04.242 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:04.241+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:04.290 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:04.291 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:04.303 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-13T10:11:04.314 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-13T10:11:04.315 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-13T10:11:04.315 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-13T10:11:04.315 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-13T10:11:04.315 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-13T10:11:04.315 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-13T10:11:04.315 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-09-13T10:11:04.315 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-13T10:11:04.315 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-13T10:11:04.315 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-13T10:11:04.315 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-13T10:11:04.315 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-09-13T10:11:04.318 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T10:11:04.318 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-13T10:11:04.318 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-09-13T10:11:04.342 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:11:04.343 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:04.341+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:04.345 INFO:tasks.rados.rados.0.smithi195.stdout:853: finishing write tid 1 to smithi19530714-48 2024-09-13T10:11:04.345 INFO:tasks.rados.rados.0.smithi195.stdout:853: finishing write tid 2 to smithi19530714-48 2024-09-13T10:11:04.345 INFO:tasks.rados.rados.0.smithi195.stdout:853: finishing write tid 3 to smithi19530714-48 2024-09-13T10:11:04.345 INFO:tasks.rados.rados.0.smithi195.stdout:853: finishing write tid 4 to smithi19530714-48 2024-09-13T10:11:04.345 INFO:tasks.rados.rados.0.smithi195.stdout:853: oid 48 updating version 0 to 51 2024-09-13T10:11:04.345 INFO:tasks.rados.rados.0.smithi195.stdout:853: oid 48 updating version 51 to 52 2024-09-13T10:11:04.346 INFO:tasks.rados.rados.0.smithi195.stdout:853: oid 48 version 52 is already newer than 50 2024-09-13T10:11:04.346 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 52 (ObjNum 123 snap 0 seq_num 123) dirty exists 2024-09-13T10:11:04.346 INFO:tasks.rados.rados.0.smithi195.stdout:853: left oid 48 (ObjNum 123 snap 0 seq_num 123) 2024-09-13T10:11:04.346 INFO:tasks.rados.rados.0.smithi195.stdout:854: got (0) Success 2024-09-13T10:11:04.346 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 26 (ObjNum 51 snap 0 seq_num 51) dirty exists 2024-09-13T10:11:04.346 INFO:tasks.rados.rados.0.smithi195.stdout:847: done (7 left) 2024-09-13T10:11:04.346 INFO:tasks.rados.rados.0.smithi195.stdout:848: done (6 left) 2024-09-13T10:11:04.346 INFO:tasks.rados.rados.0.smithi195.stdout:849: done (5 left) 2024-09-13T10:11:04.346 INFO:tasks.rados.rados.0.smithi195.stdout:850: done (4 left) 2024-09-13T10:11:04.346 INFO:tasks.rados.rados.0.smithi195.stdout:851: done (3 left) 2024-09-13T10:11:04.346 INFO:tasks.rados.rados.0.smithi195.stdout:852: done (2 left) 2024-09-13T10:11:04.346 INFO:tasks.rados.rados.0.smithi195.stdout:853: done (1 left) 2024-09-13T10:11:04.346 INFO:tasks.rados.rados.0.smithi195.stdout:854: done (0 left) 2024-09-13T10:11:04.346 INFO:tasks.rados.rados.0.smithi195.stdout:855: set_chunk oid 46 target oid 2024-09-13T10:11:04.346 INFO:tasks.rados.rados.0.smithi195.stdout:855: set_chunk oid 46 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:04.347 INFO:tasks.rados.rados.0.smithi195.stdout:856: write oid 22 current snap is 0 2024-09-13T10:11:04.347 INFO:tasks.rados.rados.0.smithi195.stdout:856: seq_num 124 ranges {484068=792524,1906473=177595} 2024-09-13T10:11:04.354 INFO:tasks.rados.rados.0.smithi195.stdout:856: writing smithi19530714-22 from 484068 to 1276592 tid 1 2024-09-13T10:11:04.361 INFO:tasks.rados.rados.0.smithi195.stdout:856: writing smithi19530714-22 from 1906473 to 2084068 tid 2 2024-09-13T10:11:04.361 INFO:tasks.rados.rados.0.smithi195.stdout:857: read oid 48 snap -1 2024-09-13T10:11:04.361 INFO:tasks.rados.rados.0.smithi195.stdout:857: expect (ObjNum 123 snap 0 seq_num 123) 2024-09-13T10:11:04.361 INFO:tasks.rados.rados.0.smithi195.stdout:858: read oid 17 snap -1 2024-09-13T10:11:04.361 INFO:tasks.rados.rados.0.smithi195.stdout:858: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-13T10:11:04.361 INFO:tasks.rados.rados.0.smithi195.stdout:859: tier_evict oid 39 2024-09-13T10:11:04.361 INFO:tasks.rados.rados.0.smithi195.stdout:859: tier_evict oid 39 snap -1 2024-09-13T10:11:04.361 INFO:tasks.rados.rados.0.smithi195.stdout:855: finishing set_chunk to oid 46 2024-09-13T10:11:04.361 INFO:tasks.rados.rados.0.smithi195.stdout:855: got expected ENOENT (src dne) 2024-09-13T10:11:04.361 INFO:tasks.rados.rados.0.smithi195.stdout:855: done (4 left) 2024-09-13T10:11:04.361 INFO:tasks.rados.rados.0.smithi195.stdout:860: delete oid 47 current snap is 0 2024-09-13T10:11:04.362 INFO:tasks.rados.rados.0.smithi195.stdout:859: got (22) Invalid argument 2024-09-13T10:11:04.365 INFO:tasks.rados.rados.0.smithi195.stdout:859: done (4 left) 2024-09-13T10:11:04.365 INFO:tasks.rados.rados.0.smithi195.stdout:860: done (3 left) 2024-09-13T10:11:04.365 INFO:tasks.rados.rados.0.smithi195.stdout:861: read oid 26 snap -1 2024-09-13T10:11:04.365 INFO:tasks.rados.rados.0.smithi195.stdout:861: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:04.365 INFO:tasks.rados.rados.0.smithi195.stdout:862: write oid 39 current snap is 0 2024-09-13T10:11:04.365 INFO:tasks.rados.rados.0.smithi195.stdout:862: seq_num 126 ranges {517682=543022,1623437=494245} 2024-09-13T10:11:04.372 INFO:tasks.rados.rados.0.smithi195.stdout:862: writing smithi19530714-39 from 517682 to 1060704 tid 1 2024-09-13T10:11:04.381 INFO:tasks.rados.rados.0.smithi195.stdout:862: writing smithi19530714-39 from 1623437 to 2117682 tid 2 2024-09-13T10:11:04.381 INFO:tasks.rados.rados.0.smithi195.stdout:863: read oid 11 snap -1 2024-09-13T10:11:04.381 INFO:tasks.rados.rados.0.smithi195.stdout:863: expect (ObjNum 88 snap 0 seq_num 88) 2024-09-13T10:11:04.381 INFO:tasks.rados.rados.0.smithi195.stdout:864: write oid 20 current snap is 0 2024-09-13T10:11:04.381 INFO:tasks.rados.rados.0.smithi195.stdout:864: seq_num 127 ranges {534489=618271,1565566=568923} 2024-09-13T10:11:04.389 INFO:tasks.rados.rados.0.smithi195.stdout:864: writing smithi19530714-20 from 534489 to 1152760 tid 1 2024-09-13T10:11:04.397 INFO:tasks.rados.rados.0.smithi195.stdout:864: writing smithi19530714-20 from 1565566 to 2134489 tid 2 2024-09-13T10:11:04.398 INFO:tasks.rados.rados.0.smithi195.stdout:865: read oid 50 snap -1 2024-09-13T10:11:04.398 INFO:tasks.rados.rados.0.smithi195.stdout:865: expect (ObjNum 117 snap 0 seq_num 117) 2024-09-13T10:11:04.398 INFO:tasks.rados.rados.0.smithi195.stdout:866: read oid 49 snap -1 2024-09-13T10:11:04.398 INFO:tasks.rados.rados.0.smithi195.stdout:866: expect (ObjNum 116 snap 0 seq_num 116) 2024-09-13T10:11:04.398 INFO:tasks.rados.rados.0.smithi195.stdout:867: read oid 29 snap -1 2024-09-13T10:11:04.398 INFO:tasks.rados.rados.0.smithi195.stdout:867: expect deleted 2024-09-13T10:11:04.398 INFO:tasks.rados.rados.0.smithi195.stdout:868: set_chunk oid 25 target oid 2024-09-13T10:11:04.398 INFO:tasks.rados.rados.0.smithi195.stdout:868: set_chunk oid 25 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:04.398 INFO:tasks.rados.rados.0.smithi195.stdout:869: write (excl) oid 16 current snap is 0 2024-09-13T10:11:04.398 INFO:tasks.rados.rados.0.smithi195.stdout:869: seq_num 128 ranges {551296=693520,2024048=127248} 2024-09-13T10:11:04.407 INFO:tasks.rados.rados.0.smithi195.stdout:869: writing smithi19530714-16 from 551296 to 1244816 tid 1 2024-09-13T10:11:04.414 INFO:tasks.rados.rados.0.smithi195.stdout:869: writing smithi19530714-16 from 2024048 to 2151296 tid 2 2024-09-13T10:11:04.414 INFO:tasks.rados.rados.0.smithi195.stdout:870: set_chunk oid 13 target oid 2024-09-13T10:11:04.426 INFO:tasks.rados.rados.0.smithi195.stdout:870: set_chunk oid 13 offset: 361984 length: 682496 target oid 9e5daa7c53ce625d7d00c3858128921c2e31dc3ce1e8c5e76043647ce2589d56 offset: 0 snap -1 2024-09-13T10:11:04.427 INFO:tasks.rados.rados.0.smithi195.stdout:871: write (excl) oid 6 current snap is 0 2024-09-13T10:11:04.427 INFO:tasks.rados.rados.0.smithi195.stdout:871: seq_num 129 ranges {568103=768769,2082530=85573} 2024-09-13T10:11:04.436 INFO:tasks.rados.rados.0.smithi195.stdout:871: writing smithi19530714-6 from 568103 to 1336872 tid 1 2024-09-13T10:11:04.442 INFO:tasks.rados.rados.0.smithi195.stdout:871: writing smithi19530714-6 from 2082530 to 2168103 tid 2 2024-09-13T10:11:04.442 INFO:tasks.rados.rados.0.smithi195.stdout:872: read oid 38 snap -1 2024-09-13T10:11:04.442 INFO:tasks.rados.rados.0.smithi195.stdout:872: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:11:04.442 INFO:tasks.rados.rados.0.smithi195.stdout:873: write (excl) oid 18 current snap is 0 2024-09-13T10:11:04.442 INFO:tasks.rados.rados.0.smithi195.stdout:873: seq_num 130 ranges {584910=560371,1857365=327545} 2024-09-13T10:11:04.444 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:04.444+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:04.451 INFO:tasks.rados.rados.0.smithi195.stdout:873: writing smithi19530714-18 from 584910 to 1145281 tid 1 2024-09-13T10:11:04.460 INFO:tasks.rados.rados.0.smithi195.stdout:873: writing smithi19530714-18 from 1857365 to 2184910 tid 2 2024-09-13T10:11:04.461 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:04.461 INFO:tasks.rados.rados.0.smithi195.stdout:858: expect (ObjNum 53 snap 0 seq_num 53) 2024-09-13T10:11:04.467 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:04.467 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:04.481 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-13T10:11:04.502 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:04.502 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:04.520 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-13T10:11:04.521 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-13T10:11:04.521 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-13T10:11:04.521 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-13T10:11:04.521 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-13T10:11:04.521 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-13T10:11:04.521 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-09-13T10:11:04.521 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-13T10:11:04.521 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-13T10:11:04.521 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-13T10:11:04.521 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-13T10:11:04.521 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-09-13T10:11:04.522 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:04.522 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:04.524 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T10:11:04.524 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-13T10:11:04.524 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-09-13T10:11:04.542 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:04.542 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:04.561 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:04.561 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:04.567 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:04.567+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:04.576 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:04.577 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:04.593 INFO:tasks.rados.rados.0.smithi195.stdout:861: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:04.597 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:04.597 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:04.617 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:04.617 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:04.634 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:04.635 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:04.646 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:04.644+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:04.647 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:04.647 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:04.650 INFO:tasks.rados.rados.0.smithi195.stdout:856: finishing write tid 1 to smithi19530714-22 2024-09-13T10:11:04.650 INFO:tasks.rados.rados.0.smithi195.stdout:856: finishing write tid 2 to smithi19530714-22 2024-09-13T10:11:04.651 INFO:tasks.rados.rados.0.smithi195.stdout:857: expect (ObjNum 123 snap 0 seq_num 123) 2024-09-13T10:11:04.651 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:04.651 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:04.660 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T10:11:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-13T10:11:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-13T10:11:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-13T10:11:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-13T10:11:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-13T10:11:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-13T10:11:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-09-13T10:11:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-13T10:11:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-13T10:11:04.669 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-13T10:11:04.670 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-13T10:11:04.670 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-09-13T10:11:04.671 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T10:11:04.672 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-13T10:11:04.672 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 'trim stale osdmaps' 2024-09-13T10:11:04.672 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:04.673 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:04.703 INFO:tasks.rados.rados.0.smithi195.stdout:856: finishing write tid 3 to smithi19530714-22 2024-09-13T10:11:04.703 INFO:tasks.rados.rados.0.smithi195.stdout:856: finishing write tid 4 to smithi19530714-22 2024-09-13T10:11:04.704 INFO:tasks.rados.rados.0.smithi195.stdout:856: oid 22 updating version 0 to 71 2024-09-13T10:11:04.704 INFO:tasks.rados.rados.0.smithi195.stdout:856: oid 22 updating version 71 to 72 2024-09-13T10:11:04.704 INFO:tasks.rados.rados.0.smithi195.stdout:856: oid 22 updating version 72 to 73 2024-09-13T10:11:04.704 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 73 (ObjNum 124 snap 0 seq_num 124) dirty exists 2024-09-13T10:11:04.704 INFO:tasks.rados.rados.0.smithi195.stdout:856: left oid 22 (ObjNum 124 snap 0 seq_num 124) 2024-09-13T10:11:04.704 INFO:tasks.rados.rados.0.smithi195.stdout:862: finishing write tid 1 to smithi19530714-39 2024-09-13T10:11:04.704 INFO:tasks.rados.rados.0.smithi195.stdout:856: done (15 left) 2024-09-13T10:11:04.704 INFO:tasks.rados.rados.0.smithi195.stdout:857: done (14 left) 2024-09-13T10:11:04.704 INFO:tasks.rados.rados.0.smithi195.stdout:858: done (13 left) 2024-09-13T10:11:04.704 INFO:tasks.rados.rados.0.smithi195.stdout:861: done (12 left) 2024-09-13T10:11:04.704 INFO:tasks.rados.rados.0.smithi195.stdout:874: read oid 5 snap -1 2024-09-13T10:11:04.704 INFO:tasks.rados.rados.0.smithi195.stdout:874: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-13T10:11:04.704 INFO:tasks.rados.rados.0.smithi195.stdout:875: write oid 32 current snap is 0 2024-09-13T10:11:04.705 INFO:tasks.rados.rados.0.smithi195.stdout:862: finishing write tid 2 to smithi19530714-39 2024-09-13T10:11:04.705 INFO:tasks.rados.rados.0.smithi195.stdout:875: seq_num 131 ranges {601717=635620,1799494=402223} 2024-09-13T10:11:04.707 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:04.708 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:04.710 INFO:tasks.rados.rados.0.smithi195.stdout:875: writing smithi19530714-32 from 601717 to 1237337 tid 1 2024-09-13T10:11:04.716 INFO:tasks.rados.rados.0.smithi195.stdout:875: writing smithi19530714-32 from 1799494 to 2201717 tid 2 2024-09-13T10:11:04.716 INFO:tasks.rados.rados.0.smithi195.stdout:876: read oid 21 snap -1 2024-09-13T10:11:04.716 INFO:tasks.rados.rados.0.smithi195.stdout:876: expect (ObjNum 69 snap 0 seq_num 69) 2024-09-13T10:11:04.716 INFO:tasks.rados.rados.0.smithi195.stdout:877: write (excl) oid 7 current snap is 0 2024-09-13T10:11:04.716 INFO:tasks.rados.rados.0.smithi195.stdout:877: seq_num 132 ranges {618524=710869,1857976=360548} 2024-09-13T10:11:04.723 INFO:tasks.rados.rados.0.smithi195.stdout:877: writing smithi19530714-7 from 618524 to 1329393 tid 1 2024-09-13T10:11:04.728 INFO:tasks.rados.rados.0.smithi195.stdout:877: writing smithi19530714-7 from 1857976 to 2218524 tid 2 2024-09-13T10:11:04.728 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:04.728 INFO:tasks.rados.rados.0.smithi195.stdout:862: finishing write tid 3 to smithi19530714-39 2024-09-13T10:11:04.728 INFO:tasks.rados.rados.0.smithi195.stdout:866: expect (ObjNum 116 snap 0 seq_num 116) 2024-09-13T10:11:04.741 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:04.742 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:04.747 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:04.743+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:04.757 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:04.758 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:04.792 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:04.792 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:04.801 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:04.801 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:04.801 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:11:04.801 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:04.812 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:04.813 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:04.813 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:04.809+0000 7f27d2d9d640 0 -- 172.21.15.73:0/2177770577 >> v2:172.21.15.195:6805/848686366 conn(0x7f27d4222dd0 msgr2=0x7f27d42251c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:11:04.814 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T10:11:04.849 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:04.845+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:04.849 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-13T10:11:04.849 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-13T10:11:04.850 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-13T10:11:04.850 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-13T10:11:04.850 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-13T10:11:04.850 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-13T10:11:04.850 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-09-13T10:11:04.850 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-13T10:11:04.850 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-13T10:11:04.850 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-13T10:11:04.850 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-13T10:11:04.850 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-09-13T10:11:04.852 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T10:11:04.853 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-13T10:11:04.853 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 'trim stale osdmaps' 2024-09-13T10:11:04.949 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:04.949+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:04.953 INFO:tasks.rados.rados.0.smithi195.stdout:868: finishing set_chunk to oid 25 2024-09-13T10:11:04.953 INFO:tasks.rados.rados.0.smithi195.stdout:868: got expected ENOENT (src dne) 2024-09-13T10:11:04.954 INFO:tasks.rados.rados.0.smithi195.stdout:865: expect (ObjNum 117 snap 0 seq_num 117) 2024-09-13T10:11:04.959 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:04.959 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:04.959 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:11:04.959 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:11:04.959 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:04.972 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T10:11:04.981 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2964255 2024-09-13T10:11:05.004 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 520797 2024-09-13T10:11:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-13T10:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-13T10:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-13T10:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-13T10:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-13T10:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-13T10:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-09-13T10:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-13T10:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-13T10:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-13T10:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-13T10:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-09-13T10:11:05.035 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:05.035 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:05.037 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T10:11:05.037 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-13T10:11:05.037 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 'trim stale osdmaps' 2024-09-13T10:11:05.050 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:05.047+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:05.116 INFO:tasks.rados.rados.0.smithi195.stdout:869: finishing write tid 1 to smithi19530714-16 2024-09-13T10:11:05.117 INFO:tasks.rados.rados.0.smithi195.stdout:869: finishing write tid 2 to smithi19530714-16 2024-09-13T10:11:05.117 INFO:tasks.rados.rados.0.smithi195.stdout:865: done (15 left) 2024-09-13T10:11:05.117 INFO:tasks.rados.rados.0.smithi195.stdout:866: done (14 left) 2024-09-13T10:11:05.117 INFO:tasks.rados.rados.0.smithi195.stdout:867: done (13 left) 2024-09-13T10:11:05.117 INFO:tasks.rados.rados.0.smithi195.stdout:868: done (12 left) 2024-09-13T10:11:05.117 INFO:tasks.rados.rados.0.smithi195.stdout:878: read oid 23 snap -1 2024-09-13T10:11:05.117 INFO:tasks.rados.rados.0.smithi195.stdout:878: expect (ObjNum 90 snap 0 seq_num 90) 2024-09-13T10:11:05.117 INFO:tasks.rados.rados.0.smithi195.stdout:879: write (excl) oid 49 current snap is 0 2024-09-13T10:11:05.117 INFO:tasks.rados.rados.0.smithi195.stdout:869: finishing write tid 3 to smithi19530714-16 2024-09-13T10:11:05.117 INFO:tasks.rados.rados.0.smithi195.stdout:879: seq_num 133 ranges {635331=786118,1916458=318873} 2024-09-13T10:11:05.124 INFO:tasks.rados.rados.0.smithi195.stdout:879: writing smithi19530714-49 from 635331 to 1421449 tid 1 2024-09-13T10:11:05.125 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:05.125 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:11:05.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:11:05.125 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:11:05.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:202 3.2 e38/30 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:11:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.122330243, 2024-09-13T10:11:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016741669000000001, 2024-09-13T10:11:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:11:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:11:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002368+0000", 2024-09-13T10:11:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-13T10:11:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010047+0000", 2024-09-13T10:11:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007678748 2024-09-13T10:11:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010081+0000", 2024-09-13T10:11:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4011999999999999e-05 2024-09-13T10:11:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010098+0000", 2024-09-13T10:11:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7125999999999998e-05 2024-09-13T10:11:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010253+0000", 2024-09-13T10:11:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000154881 2024-09-13T10:11:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010294+0000", 2024-09-13T10:11:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1746000000000003e-05 2024-09-13T10:11:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.018922+0000", 2024-09-13T10:11:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0086279619999999994 2024-09-13T10:11:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.019109+0000", 2024-09-13T10:11:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000186057 2024-09-13T10:11:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:206 3.2 e38/30 3:4cbdbe9e:::smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:11:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.113945512000001, 2024-09-13T10:11:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011224615, 2024-09-13T10:11:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:11:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:11:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010754+0000", 2024-09-13T10:11:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9999999999999999e-06 2024-09-13T10:11:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016750+0000", 2024-09-13T10:11:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059966480000000003 2024-09-13T10:11:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016782+0000", 2024-09-13T10:11:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2218999999999998e-05 2024-09-13T10:11:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016799+0000", 2024-09-13T10:11:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6685999999999999e-05 2024-09-13T10:11:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016923+0000", 2024-09-13T10:11:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000124122 2024-09-13T10:11:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016962+0000", 2024-09-13T10:11:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8748999999999998e-05 2024-09-13T10:11:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021869+0000", 2024-09-13T10:11:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049073249999999997 2024-09-13T10:11:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021976+0000", 2024-09-13T10:11:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106866 2024-09-13T10:11:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:214 3.2 e38/30 3:68ba95ba:::smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'3, mlcod=0'0)", 2024-09-13T10:11:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:11:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.104242856000001, 2024-09-13T10:11:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011518149, 2024-09-13T10:11:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:11:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:11:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020456+0000", 2024-09-13T10:11:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0430000000000002e-06 2024-09-13T10:11:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026998+0000", 2024-09-13T10:11:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065420169999999998 2024-09-13T10:11:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027030+0000", 2024-09-13T10:11:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1976e-05 2024-09-13T10:11:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027048+0000", 2024-09-13T10:11:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7615e-05 2024-09-13T10:11:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027206+0000", 2024-09-13T10:11:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015788900000000001 2024-09-13T10:11:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027283+0000", 2024-09-13T10:11:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6942999999999998e-05 2024-09-13T10:11:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.031808+0000", 2024-09-13T10:11:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045256940000000002 2024-09-13T10:11:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.031972+0000", 2024-09-13T10:11:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016397199999999999 2024-09-13T10:11:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:210 3.2 e38/30 3:46ab5247:::smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:11:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:11:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.096377137999999, 2024-09-13T10:11:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015681245, 2024-09-13T10:11:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:11:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:11:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028322+0000", 2024-09-13T10:11:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0200000000000001e-06 2024-09-13T10:11:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037903+0000", 2024-09-13T10:11:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095811639999999997 2024-09-13T10:11:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037945+0000", 2024-09-13T10:11:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1891000000000003e-05 2024-09-13T10:11:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037971+0000", 2024-09-13T10:11:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5829e-05 2024-09-13T10:11:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038125+0000", 2024-09-13T10:11:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000153852 2024-09-13T10:11:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038193+0000", 2024-09-13T10:11:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7995000000000002e-05 2024-09-13T10:11:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.043904+0000", 2024-09-13T10:11:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0057110340000000002 2024-09-13T10:11:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.044001+0000", 2024-09-13T10:11:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7460000000000005e-05 2024-09-13T10:11:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:218 3.2 e38/30 3:5915c927:::smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'5, mlcod=38'2)", 2024-09-13T10:11:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:11:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.083014552, 2024-09-13T10:11:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012988942, 2024-09-13T10:11:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:11:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:11:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041685+0000", 2024-09-13T10:11:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7769999999999998e-06 2024-09-13T10:11:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046575+0000", 2024-09-13T10:11:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048899520000000004 2024-09-13T10:11:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046633+0000", 2024-09-13T10:11:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7561e-05 2024-09-13T10:11:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046657+0000", 2024-09-13T10:11:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4223000000000002e-05 2024-09-13T10:11:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046819+0000", 2024-09-13T10:11:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016167699999999999 2024-09-13T10:11:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046882+0000", 2024-09-13T10:11:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3467999999999996e-05 2024-09-13T10:11:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054529+0000", 2024-09-13T10:11:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0076467289999999997 2024-09-13T10:11:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054671+0000", 2024-09-13T10:11:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014255499999999999 2024-09-13T10:11:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:217 3.2 e38/30 3:554c406d:::smithi19530714-18 low_tier:head v 38'6, mlcod=38'2)", 2024-09-13T10:11:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:11:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.077967762, 2024-09-13T10:11:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013473855999999999, 2024-09-13T10:11:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:11:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:11:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046731+0000", 2024-09-13T10:11:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.813e-06 2024-09-13T10:11:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054994+0000", 2024-09-13T10:11:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0082631279999999998 2024-09-13T10:11:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055024+0000", 2024-09-13T10:11:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9839e-05 2024-09-13T10:11:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055044+0000", 2024-09-13T10:11:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9507000000000001e-05 2024-09-13T10:11:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055186+0000", 2024-09-13T10:11:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014243099999999999 2024-09-13T10:11:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055240+0000", 2024-09-13T10:11:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4444999999999998e-05 2024-09-13T10:11:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060135+0000", 2024-09-13T10:11:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048942949999999999 2024-09-13T10:11:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060203+0000", 2024-09-13T10:11:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8398000000000005e-05 2024-09-13T10:11:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:220 3.2 e38/30 3:753d5785:::smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'7, mlcod=38'3)", 2024-09-13T10:11:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:11:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.069586493999999, 2024-09-13T10:11:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.009287455, 2024-09-13T10:11:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:11:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:11:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055112+0000", 2024-09-13T10:11:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6589999999999999e-06 2024-09-13T10:11:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057327+0000", 2024-09-13T10:11:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0022145229999999999 2024-09-13T10:11:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057354+0000", 2024-09-13T10:11:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7642000000000001e-05 2024-09-13T10:11:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057373+0000", 2024-09-13T10:11:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8768999999999999e-05 2024-09-13T10:11:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057511+0000", 2024-09-13T10:11:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013793900000000001 2024-09-13T10:11:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057550+0000", 2024-09-13T10:11:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9301999999999997e-05 2024-09-13T10:11:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.064324+0000", 2024-09-13T10:11:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0067737429999999996 2024-09-13T10:11:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.064398+0000", 2024-09-13T10:11:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3878e-05 2024-09-13T10:11:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:227 3.2 e38/30 3:6ac7800c:::smithi19530714-28 low_tier:head v 38'8, mlcod=38'6)", 2024-09-13T10:11:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:11:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.049995844, 2024-09-13T10:11:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018812816, 2024-09-13T10:11:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:11:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:11:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074708+0000", 2024-09-13T10:11:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.968e-06 2024-09-13T10:11:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085262+0000", 2024-09-13T10:11:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010553842000000001 2024-09-13T10:11:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085289+0000", 2024-09-13T10:11:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7288999999999999e-05 2024-09-13T10:11:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085305+0000", 2024-09-13T10:11:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5925000000000001e-05 2024-09-13T10:11:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085418+0000", 2024-09-13T10:11:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112584 2024-09-13T10:11:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085451+0000", 2024-09-13T10:11:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3605000000000001e-05 2024-09-13T10:11:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093417+0000", 2024-09-13T10:11:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0079654249999999999 2024-09-13T10:11:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093514+0000", 2024-09-13T10:11:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7177999999999997e-05 2024-09-13T10:11:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:230 3.2 e38/30 3:5d108080:::smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'9, mlcod=38'6)", 2024-09-13T10:11:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:11:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.035712814, 2024-09-13T10:11:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01953365, 2024-09-13T10:11:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:11:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:11:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088987+0000", 2024-09-13T10:11:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4600000000000002e-06 2024-09-13T10:11:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095842+0000", 2024-09-13T10:11:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0068548760000000002 2024-09-13T10:11:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095872+0000", 2024-09-13T10:11:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0476000000000001e-05 2024-09-13T10:11:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095890+0000", 2024-09-13T10:11:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7554000000000002e-05 2024-09-13T10:11:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.096006+0000", 2024-09-13T10:11:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011597199999999999 2024-09-13T10:11:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.096058+0000", 2024-09-13T10:11:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2274000000000001e-05 2024-09-13T10:11:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.108451+0000", 2024-09-13T10:11:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012392815 2024-09-13T10:11:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.108518+0000", 2024-09-13T10:11:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7223000000000003e-05 2024-09-13T10:11:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:232 3.2 e38/30 3:6538713c:::smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'10, mlcod=38'6)", 2024-09-13T10:11:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:11:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.028761933, 2024-09-13T10:11:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015307497, 2024-09-13T10:11:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:11:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:11:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095937+0000", 2024-09-13T10:11:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.367e-06 2024-09-13T10:11:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099841+0000", 2024-09-13T10:11:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0039043960000000001 2024-09-13T10:11:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099865+0000", 2024-09-13T10:11:05.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4538000000000001e-05 2024-09-13T10:11:05.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099881+0000", 2024-09-13T10:11:05.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5809000000000001e-05 2024-09-13T10:11:05.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099995+0000", 2024-09-13T10:11:05.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113843 2024-09-13T10:11:05.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.100043+0000", 2024-09-13T10:11:05.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8291999999999998e-05 2024-09-13T10:11:05.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111200+0000", 2024-09-13T10:11:05.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011156291 2024-09-13T10:11:05.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111243+0000", 2024-09-13T10:11:05.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2960999999999997e-05 2024-09-13T10:11:05.173 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.173 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.173 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.173 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:236 3.2 e38/30 3:73c2893b:::smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'11, mlcod=38'7)", 2024-09-13T10:11:05.173 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:11:05.173 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.019261815, 2024-09-13T10:11:05.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0098707670000000008, 2024-09-13T10:11:05.173 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.173 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.173 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.173 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:11:05.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:11:05.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105438+0000", 2024-09-13T10:11:05.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3010000000000001e-06 2024-09-13T10:11:05.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112262+0000", 2024-09-13T10:11:05.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.006824793 2024-09-13T10:11:05.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112279+0000", 2024-09-13T10:11:05.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6829000000000001e-05 2024-09-13T10:11:05.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112293+0000", 2024-09-13T10:11:05.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3801000000000001e-05 2024-09-13T10:11:05.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112381+0000", 2024-09-13T10:11:05.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7825999999999996e-05 2024-09-13T10:11:05.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112411+0000", 2024-09-13T10:11:05.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0434e-05 2024-09-13T10:11:05.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.115246+0000", 2024-09-13T10:11:05.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028352360000000001 2024-09-13T10:11:05.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.115306+0000", 2024-09-13T10:11:05.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9546999999999998e-05 2024-09-13T10:11:05.177 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.177 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.177 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.178 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:240 3.2 e38/30 3:7eee2f4a:::smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'12, mlcod=38'7)", 2024-09-13T10:11:05.178 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:11:05.178 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.012352028, 2024-09-13T10:11:05.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011743134000000001, 2024-09-13T10:11:05.178 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.178 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.178 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.178 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:11:05.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:11:05.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112346+0000", 2024-09-13T10:11:05.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.037e-06 2024-09-13T10:11:05.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120152+0000", 2024-09-13T10:11:05.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0078054700000000001 2024-09-13T10:11:05.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120173+0000", 2024-09-13T10:11:05.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1498999999999999e-05 2024-09-13T10:11:05.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120188+0000", 2024-09-13T10:11:05.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5159e-05 2024-09-13T10:11:05.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120313+0000", 2024-09-13T10:11:05.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012440199999999999 2024-09-13T10:11:05.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120358+0000", 2024-09-13T10:11:05.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-13T10:11:05.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.123945+0000", 2024-09-13T10:11:05.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035865490000000001 2024-09-13T10:11:05.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.124088+0000", 2024-09-13T10:11:05.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014318999999999999 2024-09-13T10:11:05.182 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.182 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.182 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.183 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:242 3.2 e38/30 3:501d22a1:::smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'13, mlcod=38'7)", 2024-09-13T10:11:05.183 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:11:05.183 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.004459020000001, 2024-09-13T10:11:05.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0088394330000000007, 2024-09-13T10:11:05.183 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.183 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.183 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.183 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:11:05.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:11:05.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120240+0000", 2024-09-13T10:11:05.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.731e-06 2024-09-13T10:11:05.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125395+0000", 2024-09-13T10:11:05.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0051547490000000001 2024-09-13T10:11:05.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125412+0000", 2024-09-13T10:11:05.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7833e-05 2024-09-13T10:11:05.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125426+0000", 2024-09-13T10:11:05.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3237999999999999e-05 2024-09-13T10:11:05.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125541+0000", 2024-09-13T10:11:05.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011566299999999999 2024-09-13T10:11:05.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125578+0000", 2024-09-13T10:11:05.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7197000000000003e-05 2024-09-13T10:11:05.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.128885+0000", 2024-09-13T10:11:05.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0033067119999999998 2024-09-13T10:11:05.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.129077+0000", 2024-09-13T10:11:05.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019231000000000001 2024-09-13T10:11:05.187 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.187 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.187 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.187 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:244 3.2 e38/30 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'14, mlcod=38'10)", 2024-09-13T10:11:05.187 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:11:05.187 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 13.998439652, 2024-09-13T10:11:05.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01536599, 2024-09-13T10:11:05.187 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.187 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.187 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.188 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:11:05.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:11:05.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126259+0000", 2024-09-13T10:11:05.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.528e-06 2024-09-13T10:11:05.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134336+0000", 2024-09-13T10:11:05.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080767410000000001 2024-09-13T10:11:05.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134356+0000", 2024-09-13T10:11:05.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9936999999999999e-05 2024-09-13T10:11:05.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134367+0000", 2024-09-13T10:11:05.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1352000000000001e-05 2024-09-13T10:11:05.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134532+0000", 2024-09-13T10:11:05.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001651 2024-09-13T10:11:05.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134578+0000", 2024-09-13T10:11:05.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5442000000000003e-05 2024-09-13T10:11:05.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.141521+0000", 2024-09-13T10:11:05.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069432720000000003 2024-09-13T10:11:05.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.141623+0000", 2024-09-13T10:11:05.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102618 2024-09-13T10:11:05.192 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.192 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.192 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.192 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:250 3.2 e38/30 3:78d40eb9:::smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'15, mlcod=38'10)", 2024-09-13T10:11:05.192 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:11:05.192 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 13.990243181, 2024-09-13T10:11:05.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095447210000000008, 2024-09-13T10:11:05.192 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.192 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.192 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.192 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:11:05.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:11:05.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134455+0000", 2024-09-13T10:11:05.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2699999999999999e-07 2024-09-13T10:11:05.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139352+0000", 2024-09-13T10:11:05.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048977810000000004 2024-09-13T10:11:05.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139374+0000", 2024-09-13T10:11:05.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1339e-05 2024-09-13T10:11:05.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139384+0000", 2024-09-13T10:11:05.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0183999999999999e-05 2024-09-13T10:11:05.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139477+0000", 2024-09-13T10:11:05.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2590999999999995e-05 2024-09-13T10:11:05.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139515+0000", 2024-09-13T10:11:05.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8624999999999997e-05 2024-09-13T10:11:05.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.143939+0000", 2024-09-13T10:11:05.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0044241189999999998 2024-09-13T10:11:05.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.143999+0000", 2024-09-13T10:11:05.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9354999999999997e-05 2024-09-13T10:11:05.196 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.196 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.196 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.197 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:245 3.2 e38/30 3:7837e849:::smithi19530714-46 low_tier:head v 38'16, mlcod=38'10)", 2024-09-13T10:11:05.197 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:11:05.197 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 13.985286826999999, 2024-09-13T10:11:05.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012687423999999999, 2024-09-13T10:11:05.197 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.197 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.197 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.197 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:11:05.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:11:05.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139411+0000", 2024-09-13T10:11:05.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6300000000000005e-07 2024-09-13T10:11:05.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144635+0000", 2024-09-13T10:11:05.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0052242169999999997 2024-09-13T10:11:05.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144651+0000", 2024-09-13T10:11:05.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6371e-05 2024-09-13T10:11:05.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144662+0000", 2024-09-13T10:11:05.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0003000000000001e-05 2024-09-13T10:11:05.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144741+0000", 2024-09-13T10:11:05.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9614e-05 2024-09-13T10:11:05.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144768+0000", 2024-09-13T10:11:05.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6826999999999999e-05 2024-09-13T10:11:05.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152015+0000", 2024-09-13T10:11:05.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0072470879999999996 2024-09-13T10:11:05.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152098+0000", 2024-09-13T10:11:05.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2640999999999997e-05 2024-09-13T10:11:05.201 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.201 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.201 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.202 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(osd.7.22:1 3.2 e38/30 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'17, mlcod=38'12)", 2024-09-13T10:11:05.202 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.144693+0000", 2024-09-13T10:11:05.202 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 13.980004055, 2024-09-13T10:11:05.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0075754180000000004, 2024-09-13T10:11:05.202 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.202 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.202 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.202 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144693+0000", 2024-09-13T10:11:05.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144693+0000", 2024-09-13T10:11:05.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144694+0000", 2024-09-13T10:11:05.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6500000000000008e-07 2024-09-13T10:11:05.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144699+0000", 2024-09-13T10:11:05.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6820000000000002e-06 2024-09-13T10:11:05.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144704+0000", 2024-09-13T10:11:05.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7869999999999998e-06 2024-09-13T10:11:05.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144710+0000", 2024-09-13T10:11:05.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3079999999999998e-06 2024-09-13T10:11:05.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.146857+0000", 2024-09-13T10:11:05.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021475209999999999 2024-09-13T10:11:05.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.146883+0000", 2024-09-13T10:11:05.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5947000000000001e-05 2024-09-13T10:11:05.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152225+0000", 2024-09-13T10:11:05.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053421450000000004 2024-09-13T10:11:05.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152268+0000", 2024-09-13T10:11:05.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3062999999999997e-05 2024-09-13T10:11:05.206 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.206 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.206 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.206 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:779 3.2 e39/30 3:7cce43d8:::4d6751ba15e95c360cd3577cdb9af7c034dc546188a4da3e77ae41469aed654d:head v 39'67, mlcod=39'65)", 2024-09-13T10:11:05.206 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.575055+0000", 2024-09-13T10:11:05.206 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 13.549642376, 2024-09-13T10:11:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0063294359999999999, 2024-09-13T10:11:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.575055+0000", 2024-09-13T10:11:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.575055+0000", 2024-09-13T10:11:05.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.575057+0000", 2024-09-13T10:11:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5830000000000001e-06 2024-09-13T10:11:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.577075+0000", 2024-09-13T10:11:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0020175879999999998 2024-09-13T10:11:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.577108+0000", 2024-09-13T10:11:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2659e-05 2024-09-13T10:11:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.577128+0000", 2024-09-13T10:11:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0585999999999999e-05 2024-09-13T10:11:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.577228+0000", 2024-09-13T10:11:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9925000000000003e-05 2024-09-13T10:11:05.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.577252+0000", 2024-09-13T10:11:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4450000000000001e-05 2024-09-13T10:11:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.581303+0000", 2024-09-13T10:11:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040503079999999999 2024-09-13T10:11:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.581384+0000", 2024-09-13T10:11:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1336999999999997e-05 2024-09-13T10:11:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:943 4.5 e42/41 4:a59e88f7:::smithi19530714-21:head v 42'36, mlcod=42'34)", 2024-09-13T10:11:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:54.640469+0000", 2024-09-13T10:11:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.484228383, 2024-09-13T10:11:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053006540000000001, 2024-09-13T10:11:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.211 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.640469+0000", 2024-09-13T10:11:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.640469+0000", 2024-09-13T10:11:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.640471+0000", 2024-09-13T10:11:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9369999999999998e-06 2024-09-13T10:11:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.641389+0000", 2024-09-13T10:11:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00091834700000000002 2024-09-13T10:11:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.641418+0000", 2024-09-13T10:11:05.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8682000000000001e-05 2024-09-13T10:11:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.641431+0000", 2024-09-13T10:11:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3349999999999999e-05 2024-09-13T10:11:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.641572+0000", 2024-09-13T10:11:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014117000000000001 2024-09-13T10:11:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.641593+0000", 2024-09-13T10:11:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0647000000000001e-05 2024-09-13T10:11:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.645732+0000", 2024-09-13T10:11:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00413949 2024-09-13T10:11:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.645769+0000", 2024-09-13T10:11:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7030999999999998e-05 2024-09-13T10:11:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.215 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(osd.6.0:104 4.5 e42/41 4:a7549961:::smithi19530714-27:head v 42'40, mlcod=42'38)", 2024-09-13T10:11:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:56.665769+0000", 2024-09-13T10:11:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 8.4589284899999999, 2024-09-13T10:11:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065613520000000003, 2024-09-13T10:11:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.665769+0000", 2024-09-13T10:11:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.665769+0000", 2024-09-13T10:11:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.665771+0000", 2024-09-13T10:11:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.317e-06 2024-09-13T10:11:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.669348+0000", 2024-09-13T10:11:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035773689999999999 2024-09-13T10:11:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.669365+0000", 2024-09-13T10:11:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6623999999999999e-05 2024-09-13T10:11:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.669373+0000", 2024-09-13T10:11:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1820000000000006e-06 2024-09-13T10:11:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.669438+0000", 2024-09-13T10:11:05.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5265999999999995e-05 2024-09-13T10:11:05.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.669462+0000", 2024-09-13T10:11:05.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3555000000000001e-05 2024-09-13T10:11:05.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.672241+0000", 2024-09-13T10:11:05.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0027788650000000002 2024-09-13T10:11:05.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.672330+0000", 2024-09-13T10:11:05.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9173999999999997e-05 2024-09-13T10:11:05.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.220 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.220 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.220 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:05.222 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T10:11:05.223 INFO:tasks.rados.rados.0.smithi195.stdout:879: writing smithi19530714-49 from 1916458 to 2235331 tid 2 2024-09-13T10:11:05.223 INFO:tasks.rados.rados.0.smithi195.stdout:880: read oid 25 snap -1 2024-09-13T10:11:05.223 INFO:tasks.rados.rados.0.smithi195.stdout:880: expect deleted 2024-09-13T10:11:05.223 INFO:tasks.rados.rados.0.smithi195.stdout:881: read oid 37 snap -1 2024-09-13T10:11:05.224 INFO:tasks.rados.rados.0.smithi195.stdout:881: expect (ObjNum 95 snap 0 seq_num 95) 2024-09-13T10:11:05.224 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:05.224 INFO:tasks.rados.rados.0.smithi195.stdout:869: finishing write tid 4 to smithi19530714-16 2024-09-13T10:11:05.224 INFO:tasks.rados.rados.0.smithi195.stdout:869: oid 16 updating version 0 to 23 2024-09-13T10:11:05.224 INFO:tasks.rados.rados.0.smithi195.stdout:869: oid 16 version 23 is already newer than 22 2024-09-13T10:11:05.224 INFO:tasks.rados.rados.0.smithi195.stdout:869: oid 16 version 23 is already newer than 21 2024-09-13T10:11:05.224 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 23 (ObjNum 128 snap 0 seq_num 128) dirty exists 2024-09-13T10:11:05.224 INFO:tasks.rados.rados.0.smithi195.stdout:869: left oid 16 (ObjNum 128 snap 0 seq_num 128) 2024-09-13T10:11:05.224 INFO:tasks.rados.rados.0.smithi195.stdout:869: done (15 left) 2024-09-13T10:11:05.224 INFO:tasks.rados.rados.0.smithi195.stdout:882: read oid 35 snap -1 2024-09-13T10:11:05.224 INFO:tasks.rados.rados.0.smithi195.stdout:882: expect (ObjNum 86 snap 0 seq_num 86) 2024-09-13T10:11:05.224 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:05.225 INFO:tasks.rados.rados.0.smithi195.stdout:870: finishing set_chunk to oid 13 2024-09-13T10:11:05.225 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 13 set_chunk returned error code -95 offset: 361984 length: 682496 2024-09-13T10:11:05.225 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 0 (ObjNum 56 snap 0 seq_num 56) dirty exists 2024-09-13T10:11:05.225 INFO:tasks.rados.rados.0.smithi195.stdout:863: expect (ObjNum 88 snap 0 seq_num 88) 2024-09-13T10:11:05.225 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:05.225 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:05.219+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:05.237 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-13T10:11:05.237 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-13T10:11:05.237 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-13T10:11:05.237 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-13T10:11:05.237 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-13T10:11:05.237 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-13T10:11:05.237 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-09-13T10:11:05.237 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-13T10:11:05.237 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-13T10:11:05.238 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-13T10:11:05.238 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-13T10:11:05.238 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-09-13T10:11:05.240 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T10:11:05.240 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-13T10:11:05.241 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 'trim stale osdmaps' 2024-09-13T10:11:05.281 INFO:tasks.rados.rados.0.smithi195.stdout:862: finishing write tid 4 to smithi19530714-39 2024-09-13T10:11:05.282 INFO:tasks.rados.rados.0.smithi195.stdout:862: oid 39 updating version 0 to 19 2024-09-13T10:11:05.282 INFO:tasks.rados.rados.0.smithi195.stdout:862: oid 39 updating version 19 to 20 2024-09-13T10:11:05.282 INFO:tasks.rados.rados.0.smithi195.stdout:862: oid 39 version 20 is already newer than 18 2024-09-13T10:11:05.282 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 20 (ObjNum 126 snap 0 seq_num 126) dirty exists 2024-09-13T10:11:05.282 INFO:tasks.rados.rados.0.smithi195.stdout:862: left oid 39 (ObjNum 126 snap 0 seq_num 126) 2024-09-13T10:11:05.282 INFO:tasks.rados.rados.0.smithi195.stdout:864: finishing write tid 1 to smithi19530714-20 2024-09-13T10:11:05.282 INFO:tasks.rados.rados.0.smithi195.stdout:864: finishing write tid 2 to smithi19530714-20 2024-09-13T10:11:05.282 INFO:tasks.rados.rados.0.smithi195.stdout:864: finishing write tid 3 to smithi19530714-20 2024-09-13T10:11:05.282 INFO:tasks.rados.rados.0.smithi195.stdout:864: finishing write tid 4 to smithi19530714-20 2024-09-13T10:11:05.282 INFO:tasks.rados.rados.0.smithi195.stdout:864: oid 20 updating version 0 to 32 2024-09-13T10:11:05.282 INFO:tasks.rados.rados.0.smithi195.stdout:864: oid 20 version 32 is already newer than 30 2024-09-13T10:11:05.282 INFO:tasks.rados.rados.0.smithi195.stdout:864: oid 20 version 32 is already newer than 31 2024-09-13T10:11:05.283 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 32 (ObjNum 127 snap 0 seq_num 127) dirty exists 2024-09-13T10:11:05.283 INFO:tasks.rados.rados.0.smithi195.stdout:864: left oid 20 (ObjNum 127 snap 0 seq_num 127) 2024-09-13T10:11:05.283 INFO:tasks.rados.rados.0.smithi195.stdout:871: finishing write tid 1 to smithi19530714-6 2024-09-13T10:11:05.283 INFO:tasks.rados.rados.0.smithi195.stdout:871: finishing write tid 2 to smithi19530714-6 2024-09-13T10:11:05.283 INFO:tasks.rados.rados.0.smithi195.stdout:871: finishing write tid 3 to smithi19530714-6 2024-09-13T10:11:05.283 INFO:tasks.rados.rados.0.smithi195.stdout:871: finishing write tid 4 to smithi19530714-6 2024-09-13T10:11:05.283 INFO:tasks.rados.rados.0.smithi195.stdout:871: oid 6 updating version 0 to 44 2024-09-13T10:11:05.283 INFO:tasks.rados.rados.0.smithi195.stdout:871: oid 6 version 44 is already newer than 42 2024-09-13T10:11:05.283 INFO:tasks.rados.rados.0.smithi195.stdout:871: oid 6 version 44 is already newer than 43 2024-09-13T10:11:05.283 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 44 (ObjNum 129 snap 0 seq_num 129) dirty exists 2024-09-13T10:11:05.283 INFO:tasks.rados.rados.0.smithi195.stdout:871: left oid 6 (ObjNum 129 snap 0 seq_num 129) 2024-09-13T10:11:05.283 INFO:tasks.rados.rados.0.smithi195.stdout:872: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:11:05.323 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:05.320+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:05.343 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:05.343 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:05.343 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:11:05.344 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:05.355 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:05.357 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T10:11:05.402 INFO:tasks.rados.rados.0.smithi195.stdout:873: finishing write tid 1 to smithi19530714-18 2024-09-13T10:11:05.402 INFO:tasks.rados.rados.0.smithi195.stdout:873: finishing write tid 2 to smithi19530714-18 2024-09-13T10:11:05.402 INFO:tasks.rados.rados.0.smithi195.stdout:873: finishing write tid 3 to smithi19530714-18 2024-09-13T10:11:05.402 INFO:tasks.rados.rados.0.smithi195.stdout:873: finishing write tid 4 to smithi19530714-18 2024-09-13T10:11:05.402 INFO:tasks.rados.rados.0.smithi195.stdout:873: oid 18 updating version 0 to 51 2024-09-13T10:11:05.402 INFO:tasks.rados.rados.0.smithi195.stdout:873: oid 18 version 51 is already newer than 50 2024-09-13T10:11:05.402 INFO:tasks.rados.rados.0.smithi195.stdout:873: oid 18 version 51 is already newer than 49 2024-09-13T10:11:05.402 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 51 (ObjNum 130 snap 0 seq_num 130) dirty exists 2024-09-13T10:11:05.402 INFO:tasks.rados.rados.0.smithi195.stdout:873: left oid 18 (ObjNum 130 snap 0 seq_num 130) 2024-09-13T10:11:05.403 INFO:tasks.rados.rados.0.smithi195.stdout:874: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-13T10:11:05.403 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T10:11:05.423 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:05.421+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:05.423 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:05.426 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-13T10:11:05.426 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-13T10:11:05.426 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-13T10:11:05.426 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-13T10:11:05.426 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-13T10:11:05.426 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-13T10:11:05.426 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-09-13T10:11:05.426 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-13T10:11:05.426 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-13T10:11:05.427 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-13T10:11:05.427 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-13T10:11:05.427 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-09-13T10:11:05.429 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T10:11:05.429 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-13T10:11:05.429 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 'trim stale osdmaps' 2024-09-13T10:11:05.456 INFO:tasks.rados.rados.0.smithi195.stdout:875: finishing write tid 1 to smithi19530714-32 2024-09-13T10:11:05.457 INFO:tasks.rados.rados.0.smithi195.stdout:875: finishing write tid 2 to smithi19530714-32 2024-09-13T10:11:05.457 INFO:tasks.rados.rados.0.smithi195.stdout:876: expect (ObjNum 69 snap 0 seq_num 69) 2024-09-13T10:11:05.479 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:05.479 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:05.479 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:11:05.479 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:11:05.479 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:05.497 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T10:11:05.504 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 826538 2024-09-13T10:11:05.524 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:05.521+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:05.573 INFO:tasks.rados.rados.0.smithi195.stdout:875: finishing write tid 3 to smithi19530714-32 2024-09-13T10:11:05.573 INFO:tasks.rados.rados.0.smithi195.stdout:875: finishing write tid 4 to smithi19530714-32 2024-09-13T10:11:05.573 INFO:tasks.rados.rados.0.smithi195.stdout:875: oid 32 updating version 0 to 39 2024-09-13T10:11:05.573 INFO:tasks.rados.rados.0.smithi195.stdout:875: oid 32 updating version 39 to 40 2024-09-13T10:11:05.573 INFO:tasks.rados.rados.0.smithi195.stdout:875: oid 32 version 40 is already newer than 38 2024-09-13T10:11:05.573 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 40 (ObjNum 131 snap 0 seq_num 131) dirty exists 2024-09-13T10:11:05.573 INFO:tasks.rados.rados.0.smithi195.stdout:875: left oid 32 (ObjNum 131 snap 0 seq_num 131) 2024-09-13T10:11:05.573 INFO:tasks.rados.rados.0.smithi195.stdout:877: finishing write tid 1 to smithi19530714-7 2024-09-13T10:11:05.573 INFO:tasks.rados.rados.0.smithi195.stdout:877: finishing write tid 2 to smithi19530714-7 2024-09-13T10:11:05.574 INFO:tasks.rados.rados.0.smithi195.stdout:877: finishing write tid 3 to smithi19530714-7 2024-09-13T10:11:05.574 INFO:tasks.rados.rados.0.smithi195.stdout:877: finishing write tid 4 to smithi19530714-7 2024-09-13T10:11:05.574 INFO:tasks.rados.rados.0.smithi195.stdout:877: oid 7 updating version 0 to 53 2024-09-13T10:11:05.574 INFO:tasks.rados.rados.0.smithi195.stdout:877: oid 7 version 53 is already newer than 52 2024-09-13T10:11:05.574 INFO:tasks.rados.rados.0.smithi195.stdout:877: oid 7 updating version 53 to 54 2024-09-13T10:11:05.574 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 54 (ObjNum 132 snap 0 seq_num 132) dirty exists 2024-09-13T10:11:05.574 INFO:tasks.rados.rados.0.smithi195.stdout:877: left oid 7 (ObjNum 132 snap 0 seq_num 132) 2024-09-13T10:11:05.574 INFO:tasks.rados.rados.0.smithi195.stdout:878: expect (ObjNum 90 snap 0 seq_num 90) 2024-09-13T10:11:05.576 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1302991 2024-09-13T10:11:05.618 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-13T10:11:05.618 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-13T10:11:05.618 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-13T10:11:05.618 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-13T10:11:05.618 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-13T10:11:05.618 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-13T10:11:05.618 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-09-13T10:11:05.618 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-13T10:11:05.619 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-13T10:11:05.619 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-13T10:11:05.619 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-13T10:11:05.619 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-09-13T10:11:05.621 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T10:11:05.621 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-13T10:11:05.621 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 'trim stale osdmaps' 2024-09-13T10:11:05.638 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1796907 2024-09-13T10:11:05.652 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:05.652 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:11:05.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:11:05.652 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:11:05.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.653 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:203 3.1 e38/30 3:b033e801:::smithi19530714-4 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:05.653 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.002504+0000", 2024-09-13T10:11:05.653 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.649582462, 2024-09-13T10:11:05.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016371101999999998, 2024-09-13T10:11:05.653 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.653 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.653 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.653 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002504+0000", 2024-09-13T10:11:05.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002504+0000", 2024-09-13T10:11:05.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002507+0000", 2024-09-13T10:11:05.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2699999999999999e-06 2024-09-13T10:11:05.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007079+0000", 2024-09-13T10:11:05.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045722200000000001 2024-09-13T10:11:05.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007114+0000", 2024-09-13T10:11:05.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5389000000000003e-05 2024-09-13T10:11:05.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007132+0000", 2024-09-13T10:11:05.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7489000000000001e-05 2024-09-13T10:11:05.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007266+0000", 2024-09-13T10:11:05.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013401900000000001 2024-09-13T10:11:05.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007299+0000", 2024-09-13T10:11:05.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3136000000000003e-05 2024-09-13T10:11:05.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.018741+0000", 2024-09-13T10:11:05.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011441957000000001 2024-09-13T10:11:05.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.018875+0000", 2024-09-13T10:11:05.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013462200000000001 2024-09-13T10:11:05.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.657 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.657 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:211 3.0 e38/30 3:0596651d:::smithi19530714-12 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:05.657 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.007453+0000", 2024-09-13T10:11:05.657 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.644633638, 2024-09-13T10:11:05.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018632359000000001, 2024-09-13T10:11:05.658 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.658 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.658 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.658 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007453+0000", 2024-09-13T10:11:05.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007453+0000", 2024-09-13T10:11:05.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007455+0000", 2024-09-13T10:11:05.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4030000000000001e-06 2024-09-13T10:11:05.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020746+0000", 2024-09-13T10:11:05.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013290219000000001 2024-09-13T10:11:05.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020788+0000", 2024-09-13T10:11:05.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2342000000000002e-05 2024-09-13T10:11:05.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020868+0000", 2024-09-13T10:11:05.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9519999999999998e-05 2024-09-13T10:11:05.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021080+0000", 2024-09-13T10:11:05.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021249600000000001 2024-09-13T10:11:05.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021113+0000", 2024-09-13T10:11:05.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2786000000000001e-05 2024-09-13T10:11:05.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.025974+0000", 2024-09-13T10:11:05.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048614929999999997 2024-09-13T10:11:05.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026085+0000", 2024-09-13T10:11:05.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001111 2024-09-13T10:11:05.662 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.662 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.662 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.662 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:205 3.1 e38/30 3:aec11a92:::smithi19530714-6 low_tier:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:05.662 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.014358+0000", 2024-09-13T10:11:05.662 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.637729014, 2024-09-13T10:11:05.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033491923999999999, 2024-09-13T10:11:05.662 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.662 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.662 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.663 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.014358+0000", 2024-09-13T10:11:05.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.014358+0000", 2024-09-13T10:11:05.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.014360+0000", 2024-09-13T10:11:05.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.057e-06 2024-09-13T10:11:05.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026358+0000", 2024-09-13T10:11:05.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011998688 2024-09-13T10:11:05.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026392+0000", 2024-09-13T10:11:05.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3575999999999998e-05 2024-09-13T10:11:05.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027340+0000", 2024-09-13T10:11:05.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00094826100000000005 2024-09-13T10:11:05.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027449+0000", 2024-09-13T10:11:05.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108797 2024-09-13T10:11:05.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027500+0000", 2024-09-13T10:11:05.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0757999999999998e-05 2024-09-13T10:11:05.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047748+0000", 2024-09-13T10:11:05.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020247884000000001 2024-09-13T10:11:05.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047850+0000", 2024-09-13T10:11:05.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101903 2024-09-13T10:11:05.666 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.666 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.666 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.667 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:212 3.0 e38/30 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:05.667 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.021018+0000", 2024-09-13T10:11:05.667 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.631068206, 2024-09-13T10:11:05.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044503290000000001, 2024-09-13T10:11:05.667 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.667 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.667 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.667 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021018+0000", 2024-09-13T10:11:05.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021018+0000", 2024-09-13T10:11:05.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021019+0000", 2024-09-13T10:11:05.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7400000000000002e-07 2024-09-13T10:11:05.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038659+0000", 2024-09-13T10:11:05.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017639555000000001 2024-09-13T10:11:05.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038678+0000", 2024-09-13T10:11:05.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8912000000000001e-05 2024-09-13T10:11:05.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038693+0000", 2024-09-13T10:11:05.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5393e-05 2024-09-13T10:11:05.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038829+0000", 2024-09-13T10:11:05.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000135912 2024-09-13T10:11:05.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038887+0000", 2024-09-13T10:11:05.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7858000000000001e-05 2024-09-13T10:11:05.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065395+0000", 2024-09-13T10:11:05.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02650773 2024-09-13T10:11:05.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065522+0000", 2024-09-13T10:11:05.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012705600000000001 2024-09-13T10:11:05.671 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.671 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.671 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.671 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:213 3.1 e38/30 3:accf4e56:::smithi19530714-14 low_tier:head v 38'3, mlcod=0'0)", 2024-09-13T10:11:05.672 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.027392+0000", 2024-09-13T10:11:05.672 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.624694697000001, 2024-09-13T10:11:05.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025423449000000001, 2024-09-13T10:11:05.672 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.672 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.672 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.672 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027392+0000", 2024-09-13T10:11:05.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027392+0000", 2024-09-13T10:11:05.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027394+0000", 2024-09-13T10:11:05.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7370000000000001e-06 2024-09-13T10:11:05.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032354+0000", 2024-09-13T10:11:05.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049601589999999996 2024-09-13T10:11:05.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032413+0000", 2024-09-13T10:11:05.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9095000000000003e-05 2024-09-13T10:11:05.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032426+0000", 2024-09-13T10:11:05.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3465999999999999e-05 2024-09-13T10:11:05.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032805+0000", 2024-09-13T10:11:05.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000378704 2024-09-13T10:11:05.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032862+0000", 2024-09-13T10:11:05.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6878e-05 2024-09-13T10:11:05.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.052688+0000", 2024-09-13T10:11:05.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019826308000000001 2024-09-13T10:11:05.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.052815+0000", 2024-09-13T10:11:05.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127102 2024-09-13T10:11:05.676 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.676 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.676 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.676 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:209 3.1 e38/30 3:9798d58d:::smithi19530714-10 low_tier:head v 38'4, mlcod=0'0)", 2024-09-13T10:11:05.676 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.032594+0000", 2024-09-13T10:11:05.676 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.619492262, 2024-09-13T10:11:05.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028120108000000001, 2024-09-13T10:11:05.676 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.676 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.676 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.677 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032594+0000", 2024-09-13T10:11:05.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032594+0000", 2024-09-13T10:11:05.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032596+0000", 2024-09-13T10:11:05.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2330000000000001e-06 2024-09-13T10:11:05.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.035217+0000", 2024-09-13T10:11:05.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026212679999999999 2024-09-13T10:11:05.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.035250+0000", 2024-09-13T10:11:05.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3065000000000003e-05 2024-09-13T10:11:05.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.035272+0000", 2024-09-13T10:11:05.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2163000000000001e-05 2024-09-13T10:11:05.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038562+0000", 2024-09-13T10:11:05.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003290322 2024-09-13T10:11:05.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038642+0000", 2024-09-13T10:11:05.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9457999999999997e-05 2024-09-13T10:11:05.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060605+0000", 2024-09-13T10:11:05.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021962751999999999 2024-09-13T10:11:05.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060715+0000", 2024-09-13T10:11:05.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010984699999999999 2024-09-13T10:11:05.680 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.680 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.680 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.681 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:207 3.0 e38/30 3:332cd44b:::smithi19530714-8 low_tier:head v 38'3, mlcod=0'0)", 2024-09-13T10:11:05.681 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.038735+0000", 2024-09-13T10:11:05.681 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.613351682999999, 2024-09-13T10:11:05.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.026872200999999998, 2024-09-13T10:11:05.681 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.681 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.681 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.681 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038735+0000", 2024-09-13T10:11:05.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038735+0000", 2024-09-13T10:11:05.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038737+0000", 2024-09-13T10:11:05.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5430000000000001e-06 2024-09-13T10:11:05.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046860+0000", 2024-09-13T10:11:05.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081236629999999997 2024-09-13T10:11:05.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046895+0000", 2024-09-13T10:11:05.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4989e-05 2024-09-13T10:11:05.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046915+0000", 2024-09-13T10:11:05.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0026000000000001e-05 2024-09-13T10:11:05.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047057+0000", 2024-09-13T10:11:05.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014151800000000001 2024-09-13T10:11:05.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047111+0000", 2024-09-13T10:11:05.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3885000000000003e-05 2024-09-13T10:11:05.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065539+0000", 2024-09-13T10:11:05.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018428225999999999 2024-09-13T10:11:05.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065607+0000", 2024-09-13T10:11:05.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8350999999999997e-05 2024-09-13T10:11:05.685 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.685 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.685 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.685 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:216 3.0 e38/30 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:11:05.685 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.046965+0000", 2024-09-13T10:11:05.685 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.605121220000001, 2024-09-13T10:11:05.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.022018617000000001, 2024-09-13T10:11:05.686 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.686 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.686 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.686 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046965+0000", 2024-09-13T10:11:05.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046965+0000", 2024-09-13T10:11:05.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046967+0000", 2024-09-13T10:11:05.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3689999999999999e-06 2024-09-13T10:11:05.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054950+0000", 2024-09-13T10:11:05.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0079834910000000005 2024-09-13T10:11:05.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054967+0000", 2024-09-13T10:11:05.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6382e-05 2024-09-13T10:11:05.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054981+0000", 2024-09-13T10:11:05.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3974999999999999e-05 2024-09-13T10:11:05.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055057+0000", 2024-09-13T10:11:05.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5964000000000007e-05 2024-09-13T10:11:05.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055100+0000", 2024-09-13T10:11:05.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3522000000000001e-05 2024-09-13T10:11:05.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.068892+0000", 2024-09-13T10:11:05.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01379175 2024-09-13T10:11:05.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.068984+0000", 2024-09-13T10:11:05.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2163999999999994e-05 2024-09-13T10:11:05.689 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.690 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.690 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.690 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:219 3.1 e38/30 3:87bfc7b1:::smithi19530714-20 low_tier:head v 38'5, mlcod=0'0)", 2024-09-13T10:11:05.690 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.048853+0000", 2024-09-13T10:11:05.690 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.603233747999999, 2024-09-13T10:11:05.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017128925999999999, 2024-09-13T10:11:05.691 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.691 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.691 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.691 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.048853+0000", 2024-09-13T10:11:05.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.048853+0000", 2024-09-13T10:11:05.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.048855+0000", 2024-09-13T10:11:05.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.877e-06 2024-09-13T10:11:05.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053357+0000", 2024-09-13T10:11:05.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045020149999999998 2024-09-13T10:11:05.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053378+0000", 2024-09-13T10:11:05.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1639000000000001e-05 2024-09-13T10:11:05.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053396+0000", 2024-09-13T10:11:05.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7135000000000001e-05 2024-09-13T10:11:05.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053526+0000", 2024-09-13T10:11:05.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013083999999999999 2024-09-13T10:11:05.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053590+0000", 2024-09-13T10:11:05.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3355000000000006e-05 2024-09-13T10:11:05.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065909+0000", 2024-09-13T10:11:05.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012319583 2024-09-13T10:11:05.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065982+0000", 2024-09-13T10:11:05.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2482000000000002e-05 2024-09-13T10:11:05.695 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.695 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.695 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.695 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:221 3.0 e38/30 3:18ae852b:::smithi19530714-22 low_tier:head v 38'5, mlcod=0'0)", 2024-09-13T10:11:05.695 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.058895+0000", 2024-09-13T10:11:05.695 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.593191533000001, 2024-09-13T10:11:05.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019544733000000002, 2024-09-13T10:11:05.695 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.695 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.695 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.695 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.058895+0000", 2024-09-13T10:11:05.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.058895+0000", 2024-09-13T10:11:05.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.058897+0000", 2024-09-13T10:11:05.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0420000000000001e-06 2024-09-13T10:11:05.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.061589+0000", 2024-09-13T10:11:05.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026919869999999999 2024-09-13T10:11:05.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.061614+0000", 2024-09-13T10:11:05.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4984999999999999e-05 2024-09-13T10:11:05.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.061629+0000", 2024-09-13T10:11:05.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4501999999999999e-05 2024-09-13T10:11:05.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065668+0000", 2024-09-13T10:11:05.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040393260000000002 2024-09-13T10:11:05.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065711+0000", 2024-09-13T10:11:05.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2673999999999998e-05 2024-09-13T10:11:05.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.078336+0000", 2024-09-13T10:11:05.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012625113 2024-09-13T10:11:05.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.078440+0000", 2024-09-13T10:11:05.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000104104 2024-09-13T10:11:05.702 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.702 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.702 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.703 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:228 3.0 e38/30 3:0168a995:::smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'7, mlcod=38'3)", 2024-09-13T10:11:05.703 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.075534+0000", 2024-09-13T10:11:05.703 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.576552835999999, 2024-09-13T10:11:05.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015201276999999999, 2024-09-13T10:11:05.703 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.703 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.703 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.703 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.075534+0000", 2024-09-13T10:11:05.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.075534+0000", 2024-09-13T10:11:05.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.075535+0000", 2024-09-13T10:11:05.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.406e-06 2024-09-13T10:11:05.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080156+0000", 2024-09-13T10:11:05.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0046210499999999998 2024-09-13T10:11:05.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080175+0000", 2024-09-13T10:11:05.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8732999999999998e-05 2024-09-13T10:11:05.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080186+0000", 2024-09-13T10:11:05.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1277000000000001e-05 2024-09-13T10:11:05.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080361+0000", 2024-09-13T10:11:05.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000174671 2024-09-13T10:11:05.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080400+0000", 2024-09-13T10:11:05.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8837000000000001e-05 2024-09-13T10:11:05.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090626+0000", 2024-09-13T10:11:05.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01022614 2024-09-13T10:11:05.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090735+0000", 2024-09-13T10:11:05.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000109163 2024-09-13T10:11:05.707 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.707 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.707 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.707 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:231 3.1 e38/30 3:9d247594:::smithi19530714-32 low_tier:head v 38'7, mlcod=38'4)", 2024-09-13T10:11:05.707 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.076614+0000", 2024-09-13T10:11:05.707 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.575472570000001, 2024-09-13T10:11:05.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017222986999999999, 2024-09-13T10:11:05.708 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.708 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.708 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.708 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.076614+0000", 2024-09-13T10:11:05.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.076614+0000", 2024-09-13T10:11:05.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.076616+0000", 2024-09-13T10:11:05.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5069999999999999e-06 2024-09-13T10:11:05.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082649+0000", 2024-09-13T10:11:05.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0060329499999999996 2024-09-13T10:11:05.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082674+0000", 2024-09-13T10:11:05.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5624000000000001e-05 2024-09-13T10:11:05.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082690+0000", 2024-09-13T10:11:05.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6152999999999999e-05 2024-09-13T10:11:05.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082812+0000", 2024-09-13T10:11:05.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012209499999999999 2024-09-13T10:11:05.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082863+0000", 2024-09-13T10:11:05.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0781000000000001e-05 2024-09-13T10:11:05.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093763+0000", 2024-09-13T10:11:05.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010899382000000001 2024-09-13T10:11:05.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093837+0000", 2024-09-13T10:11:05.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4494999999999997e-05 2024-09-13T10:11:05.711 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.711 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.712 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.712 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:234 3.1 e38/30 3:bdce9caa:::smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'8, mlcod=38'4)", 2024-09-13T10:11:05.712 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.086591+0000", 2024-09-13T10:11:05.712 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.565495457000001, 2024-09-13T10:11:05.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025699538000000001, 2024-09-13T10:11:05.712 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.712 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.712 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.712 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.086591+0000", 2024-09-13T10:11:05.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.086591+0000", 2024-09-13T10:11:05.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.086623+0000", 2024-09-13T10:11:05.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1335999999999999e-05 2024-09-13T10:11:05.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094695+0000", 2024-09-13T10:11:05.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080726919999999994 2024-09-13T10:11:05.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094716+0000", 2024-09-13T10:11:05.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1015000000000001e-05 2024-09-13T10:11:05.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094732+0000", 2024-09-13T10:11:05.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5532999999999998e-05 2024-09-13T10:11:05.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094807+0000", 2024-09-13T10:11:05.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4837000000000002e-05 2024-09-13T10:11:05.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094836+0000", 2024-09-13T10:11:05.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8912999999999999e-05 2024-09-13T10:11:05.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112239+0000", 2024-09-13T10:11:05.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017402995000000001 2024-09-13T10:11:05.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112291+0000", 2024-09-13T10:11:05.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2216999999999998e-05 2024-09-13T10:11:05.716 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.717 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.717 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.717 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:233 3.0 e38/30 3:24ead787:::smithi19530714-34 low_tier:head v 38'9, mlcod=38'4)", 2024-09-13T10:11:05.717 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.090071+0000", 2024-09-13T10:11:05.717 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.562015451000001, 2024-09-13T10:11:05.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.023396641999999999, 2024-09-13T10:11:05.717 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.717 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.717 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.717 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090071+0000", 2024-09-13T10:11:05.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090071+0000", 2024-09-13T10:11:05.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090073+0000", 2024-09-13T10:11:05.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0470000000000001e-06 2024-09-13T10:11:05.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097736+0000", 2024-09-13T10:11:05.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007662416 2024-09-13T10:11:05.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097761+0000", 2024-09-13T10:11:05.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4919e-05 2024-09-13T10:11:05.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097773+0000", 2024-09-13T10:11:05.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2182e-05 2024-09-13T10:11:05.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097877+0000", 2024-09-13T10:11:05.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010438900000000001 2024-09-13T10:11:05.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097910+0000", 2024-09-13T10:11:05.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2808999999999997e-05 2024-09-13T10:11:05.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113429+0000", 2024-09-13T10:11:05.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015518557000000001 2024-09-13T10:11:05.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113468+0000", 2024-09-13T10:11:05.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9323000000000003e-05 2024-09-13T10:11:05.721 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.721 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.721 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.722 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:223 3.1 e38/30 3:8759f4e0:::smithi19530714-24 low_tier:head v 38'9, mlcod=38'4)", 2024-09-13T10:11:05.722 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.094775+0000", 2024-09-13T10:11:05.722 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.557311933999999, 2024-09-13T10:11:05.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018476023000000001, 2024-09-13T10:11:05.722 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.722 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.722 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.722 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094775+0000", 2024-09-13T10:11:05.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094775+0000", 2024-09-13T10:11:05.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094776+0000", 2024-09-13T10:11:05.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.466e-06 2024-09-13T10:11:05.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097100+0000", 2024-09-13T10:11:05.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002323435 2024-09-13T10:11:05.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097123+0000", 2024-09-13T10:11:05.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3756000000000001e-05 2024-09-13T10:11:05.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097139+0000", 2024-09-13T10:11:05.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5424e-05 2024-09-13T10:11:05.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.098642+0000", 2024-09-13T10:11:05.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015034359999999999 2024-09-13T10:11:05.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.098683+0000", 2024-09-13T10:11:05.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0756000000000001e-05 2024-09-13T10:11:05.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113193+0000", 2024-09-13T10:11:05.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014509831000000001 2024-09-13T10:11:05.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113251+0000", 2024-09-13T10:11:05.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7918999999999999e-05 2024-09-13T10:11:05.726 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.726 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.726 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.726 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:239 3.0 e38/30 3:078df4f4:::smithi19530714-40 low_tier:head v 38'10, mlcod=38'7)", 2024-09-13T10:11:05.726 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.099908+0000", 2024-09-13T10:11:05.726 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.552179124, 2024-09-13T10:11:05.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014952633999999999, 2024-09-13T10:11:05.727 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.727 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.727 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.727 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099908+0000", 2024-09-13T10:11:05.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099908+0000", 2024-09-13T10:11:05.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099909+0000", 2024-09-13T10:11:05.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8360000000000001e-06 2024-09-13T10:11:05.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.102973+0000", 2024-09-13T10:11:05.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030636069999999999 2024-09-13T10:11:05.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.103005+0000", 2024-09-13T10:11:05.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1812e-05 2024-09-13T10:11:05.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.103027+0000", 2024-09-13T10:11:05.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2381000000000001e-05 2024-09-13T10:11:05.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.103219+0000", 2024-09-13T10:11:05.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019212999999999999 2024-09-13T10:11:05.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.103252+0000", 2024-09-13T10:11:05.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2957000000000003e-05 2024-09-13T10:11:05.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114799+0000", 2024-09-13T10:11:05.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011546242 2024-09-13T10:11:05.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114860+0000", 2024-09-13T10:11:05.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1668999999999995e-05 2024-09-13T10:11:05.731 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.731 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.731 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.731 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:248 3.1 e38/30 3:b0d90ce0:::smithi19530714-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'11, mlcod=38'8)", 2024-09-13T10:11:05.731 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.126401+0000", 2024-09-13T10:11:05.731 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.525686015, 2024-09-13T10:11:05.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014447702, 2024-09-13T10:11:05.731 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.731 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.731 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.732 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126401+0000", 2024-09-13T10:11:05.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126401+0000", 2024-09-13T10:11:05.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126401+0000", 2024-09-13T10:11:05.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4299999999999998e-07 2024-09-13T10:11:05.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.132397+0000", 2024-09-13T10:11:05.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.005995551 2024-09-13T10:11:05.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.132419+0000", 2024-09-13T10:11:05.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2716000000000001e-05 2024-09-13T10:11:05.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.132436+0000", 2024-09-13T10:11:05.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6708e-05 2024-09-13T10:11:05.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.132505+0000", 2024-09-13T10:11:05.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9115000000000005e-05 2024-09-13T10:11:05.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.132527+0000", 2024-09-13T10:11:05.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1733999999999999e-05 2024-09-13T10:11:05.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.140775+0000", 2024-09-13T10:11:05.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0082479140000000003 2024-09-13T10:11:05.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.140848+0000", 2024-09-13T10:11:05.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3521000000000003e-05 2024-09-13T10:11:05.735 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.736 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.736 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.736 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:247 3.0 e38/30 3:0b30fd85:::smithi19530714-48 low_tier:head v 38'11, mlcod=38'9)", 2024-09-13T10:11:05.736 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.132953+0000", 2024-09-13T10:11:05.736 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.519133309000001, 2024-09-13T10:11:05.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013144948, 2024-09-13T10:11:05.736 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.736 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.736 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.736 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.132953+0000", 2024-09-13T10:11:05.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.132953+0000", 2024-09-13T10:11:05.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.132955+0000", 2024-09-13T10:11:05.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.254e-06 2024-09-13T10:11:05.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.137837+0000", 2024-09-13T10:11:05.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048825129999999998 2024-09-13T10:11:05.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.137856+0000", 2024-09-13T10:11:05.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9001e-05 2024-09-13T10:11:05.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.137866+0000", 2024-09-13T10:11:05.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0042e-05 2024-09-13T10:11:05.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.137952+0000", 2024-09-13T10:11:05.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5927999999999995e-05 2024-09-13T10:11:05.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.137984+0000", 2024-09-13T10:11:05.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1667e-05 2024-09-13T10:11:05.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145873+0000", 2024-09-13T10:11:05.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0078890550000000007 2024-09-13T10:11:05.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.146098+0000", 2024-09-13T10:11:05.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022548800000000001 2024-09-13T10:11:05.740 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.741 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.741 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.741 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:241 3.0 e38/30 3:388ca573:::smithi19530714-42 low_tier:head v 38'12, mlcod=38'9)", 2024-09-13T10:11:05.741 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.137896+0000", 2024-09-13T10:11:05.741 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.514190280999999, 2024-09-13T10:11:05.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015462800000000001, 2024-09-13T10:11:05.741 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.741 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.741 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.741 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.137896+0000", 2024-09-13T10:11:05.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.137896+0000", 2024-09-13T10:11:05.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.137897+0000", 2024-09-13T10:11:05.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5099999999999998e-07 2024-09-13T10:11:05.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.142234+0000", 2024-09-13T10:11:05.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0043362160000000004 2024-09-13T10:11:05.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.142248+0000", 2024-09-13T10:11:05.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4854e-05 2024-09-13T10:11:05.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.142257+0000", 2024-09-13T10:11:05.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5820000000000002e-06 2024-09-13T10:11:05.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.142334+0000", 2024-09-13T10:11:05.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6518000000000002e-05 2024-09-13T10:11:05.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.142367+0000", 2024-09-13T10:11:05.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3667000000000001e-05 2024-09-13T10:11:05.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152035+0000", 2024-09-13T10:11:05.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0096680550000000001 2024-09-13T10:11:05.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.153359+0000", 2024-09-13T10:11:05.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001323957 2024-09-13T10:11:05.745 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.745 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.745 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.746 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:257 4.e 4:732b6721:::smithi19530714-8:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:05.746 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.215387+0000", 2024-09-13T10:11:05.746 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.436699996, 2024-09-13T10:11:05.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014191585, 2024-09-13T10:11:05.746 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:05.746 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:05.746 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:05.746 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:05.746 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:05.746 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:05.746 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 257 2024-09-13T10:11:05.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.747 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:05.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:05.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215387+0000", 2024-09-13T10:11:05.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:05.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215387+0000", 2024-09-13T10:11:05.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:05.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:05.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215389+0000", 2024-09-13T10:11:05.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7199999999999998e-06 2024-09-13T10:11:05.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:05.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215401+0000", 2024-09-13T10:11:05.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1559000000000001e-05 2024-09-13T10:11:05.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:05.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215426+0000", 2024-09-13T10:11:05.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4729e-05 2024-09-13T10:11:05.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:05.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215441+0000", 2024-09-13T10:11:05.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.523e-05 2024-09-13T10:11:05.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:05.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215565+0000", 2024-09-13T10:11:05.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000124432 2024-09-13T10:11:05.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:05.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215713+0000", 2024-09-13T10:11:05.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014722199999999999 2024-09-13T10:11:05.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 2,8", 2024-09-13T10:11:05.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.226360+0000", 2024-09-13T10:11:05.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010647428 2024-09-13T10:11:05.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:05.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.228344+0000", 2024-09-13T10:11:05.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0019844490000000001 2024-09-13T10:11:05.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:05.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.229110+0000", 2024-09-13T10:11:05.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00076580799999999998 2024-09-13T10:11:05.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:05.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.229377+0000", 2024-09-13T10:11:05.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00026719100000000001 2024-09-13T10:11:05.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:05.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.229477+0000", 2024-09-13T10:11:05.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9783000000000003e-05 2024-09-13T10:11:05.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:05.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:05.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:05.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.229578+0000", 2024-09-13T10:11:05.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101034 2024-09-13T10:11:05.752 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.752 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.752 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.752 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:05.752 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:05.752 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:05.753 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:05.649+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:05.754 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-13T10:11:05.755 INFO:tasks.rados.rados.0.smithi195.stdout:879: finishing write tid 1 to smithi19530714-49 2024-09-13T10:11:05.755 INFO:tasks.rados.rados.0.smithi195.stdout:879: finishing write tid 2 to smithi19530714-49 2024-09-13T10:11:05.755 INFO:tasks.rados.rados.0.smithi195.stdout:879: finishing write tid 3 to smithi19530714-49 2024-09-13T10:11:05.755 INFO:tasks.rados.rados.0.smithi195.stdout:882: expect (ObjNum 86 snap 0 seq_num 86) 2024-09-13T10:11:05.755 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:05.755 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:05.753+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:05.756 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 529833 2024-09-13T10:11:05.762 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-13T10:11:05.763 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-13T10:11:05.763 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-13T10:11:05.763 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-13T10:11:05.763 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-13T10:11:05.763 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-13T10:11:05.763 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-09-13T10:11:05.763 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-13T10:11:05.763 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-13T10:11:05.763 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-13T10:11:05.763 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-13T10:11:05.763 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-09-13T10:11:05.765 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T10:11:05.766 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-13T10:11:05.766 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 'trim stale osdmaps' 2024-09-13T10:11:05.809 INFO:tasks.rados.rados.0.smithi195.stdout:879: finishing write tid 4 to smithi19530714-49 2024-09-13T10:11:05.809 INFO:tasks.rados.rados.0.smithi195.stdout:879: oid 49 updating version 0 to 70 2024-09-13T10:11:05.809 INFO:tasks.rados.rados.0.smithi195.stdout:879: oid 49 updating version 70 to 71 2024-09-13T10:11:05.809 INFO:tasks.rados.rados.0.smithi195.stdout:879: oid 49 updating version 71 to 72 2024-09-13T10:11:05.809 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 72 (ObjNum 133 snap 0 seq_num 133) dirty exists 2024-09-13T10:11:05.809 INFO:tasks.rados.rados.0.smithi195.stdout:879: left oid 49 (ObjNum 133 snap 0 seq_num 133) 2024-09-13T10:11:05.809 INFO:tasks.rados.rados.0.smithi195.stdout:881: expect (ObjNum 95 snap 0 seq_num 95) 2024-09-13T10:11:05.856 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:05.855+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:05.873 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:05.873 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:11:05.873 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:05.886 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-13T10:11:05.957 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-13T10:11:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-13T10:11:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-13T10:11:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-13T10:11:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-13T10:11:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-13T10:11:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-09-13T10:11:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-13T10:11:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-13T10:11:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-13T10:11:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-13T10:11:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-09-13T10:11:05.959 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:05.955+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:05.960 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T10:11:05.960 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-13T10:11:05.961 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 'trim stale osdmaps' 2024-09-13T10:11:05.970 INFO:tasks.rados.rados.0.smithi195.stdout:862: done (15 left) 2024-09-13T10:11:05.970 INFO:tasks.rados.rados.0.smithi195.stdout:863: done (14 left) 2024-09-13T10:11:05.970 INFO:tasks.rados.rados.0.smithi195.stdout:864: done (13 left) 2024-09-13T10:11:05.970 INFO:tasks.rados.rados.0.smithi195.stdout:870: done (12 left) 2024-09-13T10:11:05.970 INFO:tasks.rados.rados.0.smithi195.stdout:871: done (11 left) 2024-09-13T10:11:05.970 INFO:tasks.rados.rados.0.smithi195.stdout:872: done (10 left) 2024-09-13T10:11:05.970 INFO:tasks.rados.rados.0.smithi195.stdout:873: done (9 left) 2024-09-13T10:11:05.970 INFO:tasks.rados.rados.0.smithi195.stdout:874: done (8 left) 2024-09-13T10:11:05.970 INFO:tasks.rados.rados.0.smithi195.stdout:875: done (7 left) 2024-09-13T10:11:05.970 INFO:tasks.rados.rados.0.smithi195.stdout:876: done (6 left) 2024-09-13T10:11:05.970 INFO:tasks.rados.rados.0.smithi195.stdout:877: done (5 left) 2024-09-13T10:11:05.971 INFO:tasks.rados.rados.0.smithi195.stdout:878: done (4 left) 2024-09-13T10:11:05.971 INFO:tasks.rados.rados.0.smithi195.stdout:879: done (3 left) 2024-09-13T10:11:05.971 INFO:tasks.rados.rados.0.smithi195.stdout:880: done (2 left) 2024-09-13T10:11:05.971 INFO:tasks.rados.rados.0.smithi195.stdout:881: done (1 left) 2024-09-13T10:11:05.971 INFO:tasks.rados.rados.0.smithi195.stdout:882: done (0 left) 2024-09-13T10:11:05.971 INFO:tasks.rados.rados.0.smithi195.stdout:883: write oid 50 current snap is 0 2024-09-13T10:11:05.971 INFO:tasks.rados.rados.0.smithi195.stdout:883: seq_num 134 ranges {652138=461367,1574940=677198} 2024-09-13T10:11:05.988 INFO:tasks.rados.rados.0.smithi195.stdout:883: writing smithi19530714-50 from 652138 to 1113505 tid 1 2024-09-13T10:11:05.998 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:05.998 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:05.998 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:11:05.998 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:11:05.998 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:06.007 INFO:tasks.rados.rados.0.smithi195.stdout:883: writing smithi19530714-50 from 1574940 to 2252138 tid 2 2024-09-13T10:11:06.007 INFO:tasks.rados.rados.0.smithi195.stdout:884: read oid 21 snap -1 2024-09-13T10:11:06.007 INFO:tasks.rados.rados.0.smithi195.stdout:884: expect (ObjNum 69 snap 0 seq_num 69) 2024-09-13T10:11:06.007 INFO:tasks.rados.rados.0.smithi195.stdout:885: read oid 12 snap -1 2024-09-13T10:11:06.007 INFO:tasks.rados.rados.0.smithi195.stdout:885: expect (ObjNum 85 snap 0 seq_num 85) 2024-09-13T10:11:06.007 INFO:tasks.rados.rados.0.smithi195.stdout:886: write (excl) oid 20 current snap is 0 2024-09-13T10:11:06.007 INFO:tasks.rados.rados.0.smithi195.stdout:886: seq_num 135 ranges {668945=536616,1633422=491363} 2024-09-13T10:11:06.011 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-13T10:11:06.025 INFO:tasks.rados.rados.0.smithi195.stdout:886: writing smithi19530714-20 from 668945 to 1205561 tid 1 2024-09-13T10:11:06.036 INFO:tasks.rados.rados.0.smithi195.stdout:886: writing smithi19530714-20 from 1633422 to 2124785 tid 2 2024-09-13T10:11:06.036 INFO:tasks.rados.rados.0.smithi195.stdout:887: write oid 43 current snap is 0 2024-09-13T10:11:06.036 INFO:tasks.rados.rados.0.smithi195.stdout:887: seq_num 136 ranges {685752=611865,1975551=310201} 2024-09-13T10:11:06.048 INFO:tasks.rados.rados.0.smithi195.stdout:887: writing smithi19530714-43 from 685752 to 1297617 tid 1 2024-09-13T10:11:06.057 INFO:tasks.rados.rados.0.smithi195.stdout:887: writing smithi19530714-43 from 1975551 to 2285752 tid 2 2024-09-13T10:11:06.057 INFO:tasks.rados.rados.0.smithi195.stdout:888: write (excl) oid 33 current snap is 0 2024-09-13T10:11:06.057 INFO:tasks.rados.rados.0.smithi195.stdout:888: seq_num 137 ranges {702559=403467,1750386=495032} 2024-09-13T10:11:06.058 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:06.056+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:06.065 INFO:tasks.rados.rados.0.smithi195.stdout:888: writing smithi19530714-33 from 702559 to 1106026 tid 1 2024-09-13T10:11:06.073 INFO:tasks.rados.rados.0.smithi195.stdout:888: writing smithi19530714-33 from 1750386 to 2245418 tid 2 2024-09-13T10:11:06.073 INFO:tasks.rados.rados.0.smithi195.stdout:889: read oid 38 snap -1 2024-09-13T10:11:06.073 INFO:tasks.rados.rados.0.smithi195.stdout:889: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:11:06.073 INFO:tasks.rados.rados.0.smithi195.stdout:890: read oid 26 snap -1 2024-09-13T10:11:06.073 INFO:tasks.rados.rados.0.smithi195.stdout:890: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:06.073 INFO:tasks.rados.rados.0.smithi195.stdout:891: set_chunk oid 48 target oid 2024-09-13T10:11:06.098 INFO:tasks.rados.rados.0.smithi195.stdout:891: set_chunk oid 48 offset: 92160 length: 868352 target oid 26cc86717aa4804df56de89cd1b263c166023ac79d73c34f078b5b73b68cfec3 offset: 0 snap -1 2024-09-13T10:11:06.098 INFO:tasks.rados.rados.0.smithi195.stdout:892: tier_flush oid 37 2024-09-13T10:11:06.099 INFO:tasks.rados.rados.0.smithi195.stdout:892: tier_flush oid 37 snap -1 2024-09-13T10:11:06.099 INFO:tasks.rados.rados.0.smithi195.stdout:884: expect (ObjNum 69 snap 0 seq_num 69) 2024-09-13T10:11:06.099 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:06.107 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-13T10:11:06.108 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-13T10:11:06.108 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-13T10:11:06.108 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-13T10:11:06.108 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-13T10:11:06.108 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-13T10:11:06.108 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-09-13T10:11:06.108 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-13T10:11:06.108 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-13T10:11:06.108 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-13T10:11:06.108 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-13T10:11:06.109 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-09-13T10:11:06.113 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T10:11:06.113 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-13T10:11:06.160 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:06.157+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.787163988000003, 2024-09-13T10:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034624456999999997, 2024-09-13T10:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 70 2024-09-13T10:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381723+0000", 2024-09-13T10:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.540000000000001e-07 2024-09-13T10:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381731+0000", 2024-09-13T10:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.486e-06 2024-09-13T10:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381742+0000", 2024-09-13T10:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1269e-05 2024-09-13T10:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381750+0000", 2024-09-13T10:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8830000000000006e-06 2024-09-13T10:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401255+0000", 2024-09-13T10:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019505110999999999 2024-09-13T10:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401388+0000", 2024-09-13T10:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013253200000000001 2024-09-13T10:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401609+0000", 2024-09-13T10:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000220981 2024-09-13T10:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407538+0000", 2024-09-13T10:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059291220000000002 2024-09-13T10:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407626+0000", 2024-09-13T10:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8361000000000003e-05 2024-09-13T10:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416154+0000", 2024-09-13T10:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0085281000000000003 2024-09-13T10:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416234+0000", 2024-09-13T10:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9594999999999999e-05 2024-09-13T10:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416347+0000", 2024-09-13T10:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113063 2024-09-13T10:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:71 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.787123405999999, 2024-09-13T10:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035004, 2024-09-13T10:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 71 2024-09-13T10:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381764+0000", 2024-09-13T10:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4600000000000004e-07 2024-09-13T10:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381769+0000", 2024-09-13T10:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.48e-06 2024-09-13T10:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381780+0000", 2024-09-13T10:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0893000000000001e-05 2024-09-13T10:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381788+0000", 2024-09-13T10:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6389999999999992e-06 2024-09-13T10:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407896+0000", 2024-09-13T10:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.026107807 2024-09-13T10:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.408029+0000", 2024-09-13T10:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013383899999999999 2024-09-13T10:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.408250+0000", 2024-09-13T10:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022105399999999999 2024-09-13T10:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416130+0000", 2024-09-13T10:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0078798500000000007 2024-09-13T10:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416488+0000", 2024-09-13T10:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00035819299999999998 2024-09-13T10:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416595+0000", 2024-09-13T10:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001064 2024-09-13T10:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416662+0000", 2024-09-13T10:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6775999999999999e-05 2024-09-13T10:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416767+0000", 2024-09-13T10:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000105423 2024-09-13T10:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:72 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.787086371999997, 2024-09-13T10:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039399762999999997, 2024-09-13T10:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 72 2024-09-13T10:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381801+0000", 2024-09-13T10:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7700000000000004e-07 2024-09-13T10:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381898+0000", 2024-09-13T10:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7066999999999997e-05 2024-09-13T10:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381908+0000", 2024-09-13T10:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0336000000000001e-05 2024-09-13T10:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381917+0000", 2024-09-13T10:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5169999999999999e-06 2024-09-13T10:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.411900+0000", 2024-09-13T10:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029983381999999999 2024-09-13T10:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.412028+0000", 2024-09-13T10:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127916 2024-09-13T10:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.412248+0000", 2024-09-13T10:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022014699999999999 2024-09-13T10:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416576+0000", 2024-09-13T10:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004327549 2024-09-13T10:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416860+0000", 2024-09-13T10:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028391000000000001 2024-09-13T10:11:06.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421014+0000", 2024-09-13T10:11:06.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041539330000000003 2024-09-13T10:11:06.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421093+0000", 2024-09-13T10:11:06.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9097999999999994e-05 2024-09-13T10:11:06.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421200+0000", 2024-09-13T10:11:06.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000107231 2024-09-13T10:11:06.190 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.190 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.190 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.190 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:73 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:06.190 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:11:06.190 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.786952659999997, 2024-09-13T10:11:06.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044034528000000003, 2024-09-13T10:11:06.191 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.191 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.191 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.191 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.191 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:06.191 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:06.191 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 73 2024-09-13T10:11:06.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.191 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:11:06.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:11:06.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381935+0000", 2024-09-13T10:11:06.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0999999999999997e-07 2024-09-13T10:11:06.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381940+0000", 2024-09-13T10:11:06.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7030000000000003e-06 2024-09-13T10:11:06.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381949+0000", 2024-09-13T10:11:06.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5359999999999992e-06 2024-09-13T10:11:06.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381961+0000", 2024-09-13T10:11:06.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2539e-05 2024-09-13T10:11:06.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416942+0000", 2024-09-13T10:11:06.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034980270000000001 2024-09-13T10:11:06.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.417074+0000", 2024-09-13T10:11:06.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013246199999999999 2024-09-13T10:11:06.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:06.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.417298+0000", 2024-09-13T10:11:06.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000223823 2024-09-13T10:11:06.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425459+0000", 2024-09-13T10:11:06.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081613750000000002 2024-09-13T10:11:06.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425575+0000", 2024-09-13T10:11:06.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115513 2024-09-13T10:11:06.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425774+0000", 2024-09-13T10:11:06.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019959399999999999 2024-09-13T10:11:06.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425854+0000", 2024-09-13T10:11:06.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9532999999999999e-05 2024-09-13T10:11:06.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425968+0000", 2024-09-13T10:11:06.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011437 2024-09-13T10:11:06.197 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.197 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.197 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.197 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:74 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:06.197 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:11:06.197 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.786909786000002, 2024-09-13T10:11:06.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052701496, 2024-09-13T10:11:06.197 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.197 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.197 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.198 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.198 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:06.198 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:06.198 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 74 2024-09-13T10:11:06.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.198 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:11:06.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:11:06.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:11:06.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7999999999999995e-07 2024-09-13T10:11:06.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381983+0000", 2024-09-13T10:11:06.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4430000000000002e-06 2024-09-13T10:11:06.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381991+0000", 2024-09-13T10:11:06.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8960000000000003e-06 2024-09-13T10:11:06.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381998+0000", 2024-09-13T10:11:06.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7729999999999999e-06 2024-09-13T10:11:06.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421291+0000", 2024-09-13T10:11:06.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039292753 2024-09-13T10:11:06.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421424+0000", 2024-09-13T10:11:06.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013291900000000001 2024-09-13T10:11:06.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:06.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421646+0000", 2024-09-13T10:11:06.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000221843 2024-09-13T10:11:06.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425667+0000", 2024-09-13T10:11:06.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040206779999999998 2024-09-13T10:11:06.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425755+0000", 2024-09-13T10:11:06.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8848e-05 2024-09-13T10:11:06.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434472+0000", 2024-09-13T10:11:06.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0087164879999999997 2024-09-13T10:11:06.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434544+0000", 2024-09-13T10:11:06.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2273000000000004e-05 2024-09-13T10:11:06.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434678+0000", 2024-09-13T10:11:06.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013390200000000001 2024-09-13T10:11:06.204 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.204 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.204 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.204 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:75 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:06.204 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:11:06.204 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.786872920999997, 2024-09-13T10:11:06.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052864585999999998, 2024-09-13T10:11:06.204 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.204 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.204 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.205 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.205 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:06.205 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:06.205 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 75 2024-09-13T10:11:06.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.205 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:11:06.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:11:06.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:11:06.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8299999999999996e-07 2024-09-13T10:11:06.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382120+0000", 2024-09-13T10:11:06.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010627400000000001 2024-09-13T10:11:06.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382131+0000", 2024-09-13T10:11:06.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0796e-05 2024-09-13T10:11:06.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382140+0000", 2024-09-13T10:11:06.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5560000000000008e-06 2024-09-13T10:11:06.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426014+0000", 2024-09-13T10:11:06.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.043873843000000003 2024-09-13T10:11:06.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426205+0000", 2024-09-13T10:11:06.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019154600000000001 2024-09-13T10:11:06.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:06.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426429+0000", 2024-09-13T10:11:06.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022344699999999999 2024-09-13T10:11:06.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434352+0000", 2024-09-13T10:11:06.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007923355 2024-09-13T10:11:06.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434452+0000", 2024-09-13T10:11:06.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9961000000000001e-05 2024-09-13T10:11:06.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434700+0000", 2024-09-13T10:11:06.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024790899999999999 2024-09-13T10:11:06.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434774+0000", 2024-09-13T10:11:06.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4457999999999998e-05 2024-09-13T10:11:06.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434878+0000", 2024-09-13T10:11:06.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010375800000000001 2024-09-13T10:11:06.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.211 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.212 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:76 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:06.212 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:11:06.212 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.786705941000001, 2024-09-13T10:11:06.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053104138000000002, 2024-09-13T10:11:06.212 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.212 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.212 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:06.212 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:06.212 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 76 2024-09-13T10:11:06.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.212 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:11:06.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:11:06.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:11:06.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4600000000000003e-07 2024-09-13T10:11:06.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382188+0000", 2024-09-13T10:11:06.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4869999999999999e-06 2024-09-13T10:11:06.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382197+0000", 2024-09-13T10:11:06.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5519999999999993e-06 2024-09-13T10:11:06.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382207+0000", 2024-09-13T10:11:06.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7240000000000004e-06 2024-09-13T10:11:06.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430285+0000", 2024-09-13T10:11:06.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.048078093000000002 2024-09-13T10:11:06.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430429+0000", 2024-09-13T10:11:06.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000143981 2024-09-13T10:11:06.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:06.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430717+0000", 2024-09-13T10:11:06.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028802 2024-09-13T10:11:06.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435015+0000", 2024-09-13T10:11:06.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042973150000000003 2024-09-13T10:11:06.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435083+0000", 2024-09-13T10:11:06.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8448999999999995e-05 2024-09-13T10:11:06.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435109+0000", 2024-09-13T10:11:06.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6064999999999999e-05 2024-09-13T10:11:06.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435176+0000", 2024-09-13T10:11:06.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7362999999999999e-05 2024-09-13T10:11:06.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435285+0000", 2024-09-13T10:11:06.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108243 2024-09-13T10:11:06.218 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.218 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.218 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.218 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:96 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:06.218 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:11:06.218 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.645074510999997, 2024-09-13T10:11:06.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032701419000000002, 2024-09-13T10:11:06.218 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.219 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.219 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.219 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.219 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:06.219 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:06.219 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 96 2024-09-13T10:11:06.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.219 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:11:06.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:11:06.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523813+0000", 2024-09-13T10:11:06.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.364e-06 2024-09-13T10:11:06.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523913+0000", 2024-09-13T10:11:06.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9827000000000005e-05 2024-09-13T10:11:06.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523927+0000", 2024-09-13T10:11:06.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4029000000000001e-05 2024-09-13T10:11:06.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523936+0000", 2024-09-13T10:11:06.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1030000000000008e-06 2024-09-13T10:11:06.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.544707+0000", 2024-09-13T10:11:06.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0207706 2024-09-13T10:11:06.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.544834+0000", 2024-09-13T10:11:06.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012683200000000001 2024-09-13T10:11:06.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:06.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.545057+0000", 2024-09-13T10:11:06.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022338100000000001 2024-09-13T10:11:06.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549279+0000", 2024-09-13T10:11:06.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042223080000000001 2024-09-13T10:11:06.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549556+0000", 2024-09-13T10:11:06.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000276307 2024-09-13T10:11:06.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556364+0000", 2024-09-13T10:11:06.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0068082510000000004 2024-09-13T10:11:06.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556445+0000", 2024-09-13T10:11:06.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1081000000000005e-05 2024-09-13T10:11:06.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556513+0000", 2024-09-13T10:11:06.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8336000000000004e-05 2024-09-13T10:11:06.224 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.224 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.224 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.225 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:97 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:06.225 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:11:06.225 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.644931626000002, 2024-09-13T10:11:06.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035410807000000002, 2024-09-13T10:11:06.225 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.225 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.225 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.225 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.225 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:06.225 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:06.225 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 97 2024-09-13T10:11:06.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.226 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:11:06.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:11:06.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523956+0000", 2024-09-13T10:11:06.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8599999999999997e-07 2024-09-13T10:11:06.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523962+0000", 2024-09-13T10:11:06.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1240000000000002e-06 2024-09-13T10:11:06.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523971+0000", 2024-09-13T10:11:06.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7320000000000004e-06 2024-09-13T10:11:06.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523978+0000", 2024-09-13T10:11:06.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0899999999999999e-06 2024-09-13T10:11:06.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549637+0000", 2024-09-13T10:11:06.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025658876000000001 2024-09-13T10:11:06.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549757+0000", 2024-09-13T10:11:06.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000120837 2024-09-13T10:11:06.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:06.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549971+0000", 2024-09-13T10:11:06.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021356000000000001 2024-09-13T10:11:06.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556335+0000", 2024-09-13T10:11:06.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0063639329999999996 2024-09-13T10:11:06.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556725+0000", 2024-09-13T10:11:06.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000390221 2024-09-13T10:11:06.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559219+0000", 2024-09-13T10:11:06.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0024943500000000002 2024-09-13T10:11:06.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559296+0000", 2024-09-13T10:11:06.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7065999999999997e-05 2024-09-13T10:11:06.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559366+0000", 2024-09-13T10:11:06.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9132000000000002e-05 2024-09-13T10:11:06.232 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.232 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.232 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.232 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:98 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:06.232 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:11:06.232 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.644897018000002, 2024-09-13T10:11:06.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035839604999999997, 2024-09-13T10:11:06.232 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.232 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.233 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.233 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.233 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:06.233 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:06.233 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 98 2024-09-13T10:11:06.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.233 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:11:06.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:11:06.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523990+0000", 2024-09-13T10:11:06.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6300000000000005e-07 2024-09-13T10:11:06.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523995+0000", 2024-09-13T10:11:06.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2669999999999997e-06 2024-09-13T10:11:06.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524007+0000", 2024-09-13T10:11:06.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2002e-05 2024-09-13T10:11:06.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524015+0000", 2024-09-13T10:11:06.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4869999999999996e-06 2024-09-13T10:11:06.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.553854+0000", 2024-09-13T10:11:06.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029839633000000001 2024-09-13T10:11:06.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.553985+0000", 2024-09-13T10:11:06.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013032899999999999 2024-09-13T10:11:06.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:06.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.554212+0000", 2024-09-13T10:11:06.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022723400000000001 2024-09-13T10:11:06.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559523+0000", 2024-09-13T10:11:06.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053113509999999997 2024-09-13T10:11:06.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559662+0000", 2024-09-13T10:11:06.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013833500000000001 2024-09-13T10:11:06.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559698+0000", 2024-09-13T10:11:06.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5982000000000002e-05 2024-09-13T10:11:06.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559764+0000", 2024-09-13T10:11:06.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6019000000000006e-05 2024-09-13T10:11:06.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559829+0000", 2024-09-13T10:11:06.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5302999999999995e-05 2024-09-13T10:11:06.239 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.239 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.239 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.239 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:99 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:06.239 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:11:06.239 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.644859324000002, 2024-09-13T10:11:06.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.037637084000000001, 2024-09-13T10:11:06.239 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.240 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.240 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.240 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.240 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:06.240 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:06.240 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 99 2024-09-13T10:11:06.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.240 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:11:06.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:11:06.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524028+0000", 2024-09-13T10:11:06.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7999999999999995e-07 2024-09-13T10:11:06.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524109+0000", 2024-09-13T10:11:06.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0834999999999994e-05 2024-09-13T10:11:06.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524119+0000", 2024-09-13T10:11:06.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0149999999999999e-05 2024-09-13T10:11:06.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524127+0000", 2024-09-13T10:11:06.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8660000000000006e-06 2024-09-13T10:11:06.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556868+0000", 2024-09-13T10:11:06.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.03274146 2024-09-13T10:11:06.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.557001+0000", 2024-09-13T10:11:06.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000133249 2024-09-13T10:11:06.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:06.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.557224+0000", 2024-09-13T10:11:06.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000222914 2024-09-13T10:11:06.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561387+0000", 2024-09-13T10:11:06.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041627019999999999 2024-09-13T10:11:06.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561419+0000", 2024-09-13T10:11:06.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1924999999999997e-05 2024-09-13T10:11:06.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561476+0000", 2024-09-13T10:11:06.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7346999999999998e-05 2024-09-13T10:11:06.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561552+0000", 2024-09-13T10:11:06.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5847999999999996e-05 2024-09-13T10:11:06.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561664+0000", 2024-09-13T10:11:06.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112108 2024-09-13T10:11:06.247 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.247 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.247 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.247 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:100 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:06.247 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:11:06.247 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.644683174000001, 2024-09-13T10:11:06.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.038825510000000001, 2024-09-13T10:11:06.247 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.247 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.247 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.247 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.248 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:06.248 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:06.248 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 100 2024-09-13T10:11:06.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.248 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:11:06.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:11:06.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524204+0000", 2024-09-13T10:11:06.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.54e-07 2024-09-13T10:11:06.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524211+0000", 2024-09-13T10:11:06.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6919999999999996e-06 2024-09-13T10:11:06.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524221+0000", 2024-09-13T10:11:06.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0223e-05 2024-09-13T10:11:06.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524229+0000", 2024-09-13T10:11:06.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9859999999999994e-06 2024-09-13T10:11:06.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559920+0000", 2024-09-13T10:11:06.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035690854000000001 2024-09-13T10:11:06.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.560052+0000", 2024-09-13T10:11:06.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013199899999999999 2024-09-13T10:11:06.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:06.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.560356+0000", 2024-09-13T10:11:06.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030450200000000001 2024-09-13T10:11:06.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562115+0000", 2024-09-13T10:11:06.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0017586430000000001 2024-09-13T10:11:06.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562512+0000", 2024-09-13T10:11:06.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00039728099999999998 2024-09-13T10:11:06.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562828+0000", 2024-09-13T10:11:06.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00031528799999999999 2024-09-13T10:11:06.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562913+0000", 2024-09-13T10:11:06.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5628000000000001e-05 2024-09-13T10:11:06.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.563029+0000", 2024-09-13T10:11:06.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011556 2024-09-13T10:11:06.253 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.253 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.253 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.254 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:121 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:06.254 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.640808+0000", 2024-09-13T10:11:06.254 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.528078694999998, 2024-09-13T10:11:06.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.027560819, 2024-09-13T10:11:06.254 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.254 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.254 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.254 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.254 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:06.254 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:06.254 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 121 2024-09-13T10:11:06.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.255 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640808+0000", 2024-09-13T10:11:06.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640808+0000", 2024-09-13T10:11:06.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640809+0000", 2024-09-13T10:11:06.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7e-06 2024-09-13T10:11:06.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640821+0000", 2024-09-13T10:11:06.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1666e-05 2024-09-13T10:11:06.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640839+0000", 2024-09-13T10:11:06.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8338999999999998e-05 2024-09-13T10:11:06.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640859+0000", 2024-09-13T10:11:06.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9972000000000001e-05 2024-09-13T10:11:06.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.660184+0000", 2024-09-13T10:11:06.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019324108999999999 2024-09-13T10:11:06.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.660413+0000", 2024-09-13T10:11:06.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000229163 2024-09-13T10:11:06.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:06.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.660777+0000", 2024-09-13T10:11:06.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00036470200000000001 2024-09-13T10:11:06.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.662947+0000", 2024-09-13T10:11:06.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021698059999999998 2024-09-13T10:11:06.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663078+0000", 2024-09-13T10:11:06.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013046800000000001 2024-09-13T10:11:06.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668230+0000", 2024-09-13T10:11:06.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0051526719999999996 2024-09-13T10:11:06.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668303+0000", 2024-09-13T10:11:06.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3102000000000006e-05 2024-09-13T10:11:06.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668369+0000", 2024-09-13T10:11:06.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.512e-05 2024-09-13T10:11:06.260 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.260 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.260 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.260 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:122 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:06.260 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:11:06.260 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.527998693000001, 2024-09-13T10:11:06.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.027900708, 2024-09-13T10:11:06.261 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.261 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.261 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.261 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.261 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:06.261 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:06.261 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 122 2024-09-13T10:11:06.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.261 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:11:06.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:11:06.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640889+0000", 2024-09-13T10:11:06.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4139999999999999e-06 2024-09-13T10:11:06.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640900+0000", 2024-09-13T10:11:06.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0559e-05 2024-09-13T10:11:06.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640916+0000", 2024-09-13T10:11:06.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6399e-05 2024-09-13T10:11:06.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640931+0000", 2024-09-13T10:11:06.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4659e-05 2024-09-13T10:11:06.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663602+0000", 2024-09-13T10:11:06.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.022671674999999999 2024-09-13T10:11:06.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663734+0000", 2024-09-13T10:11:06.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013189899999999999 2024-09-13T10:11:06.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:06.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663946+0000", 2024-09-13T10:11:06.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021198699999999999 2024-09-13T10:11:06.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668523+0000", 2024-09-13T10:11:06.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045765789999999999 2024-09-13T10:11:06.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668638+0000", 2024-09-13T10:11:06.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504 2024-09-13T10:11:06.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668657+0000", 2024-09-13T10:11:06.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9298999999999999e-05 2024-09-13T10:11:06.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668722+0000", 2024-09-13T10:11:06.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5126e-05 2024-09-13T10:11:06.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668788+0000", 2024-09-13T10:11:06.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6072e-05 2024-09-13T10:11:06.266 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.267 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.267 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.267 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:123 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:06.267 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:11:06.267 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.527932816000003, 2024-09-13T10:11:06.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028273795000000001, 2024-09-13T10:11:06.268 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.268 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.268 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.268 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.268 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:06.268 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:06.268 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 123 2024-09-13T10:11:06.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.268 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:11:06.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:11:06.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640955+0000", 2024-09-13T10:11:06.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.356e-06 2024-09-13T10:11:06.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641083+0000", 2024-09-13T10:11:06.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012763600000000001 2024-09-13T10:11:06.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641102+0000", 2024-09-13T10:11:06.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9055e-05 2024-09-13T10:11:06.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641117+0000", 2024-09-13T10:11:06.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5665999999999999e-05 2024-09-13T10:11:06.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.665747+0000", 2024-09-13T10:11:06.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.024629796999999998 2024-09-13T10:11:06.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.665876+0000", 2024-09-13T10:11:06.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012925600000000001 2024-09-13T10:11:06.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:06.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.666105+0000", 2024-09-13T10:11:06.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000228543 2024-09-13T10:11:06.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668924+0000", 2024-09-13T10:11:06.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028194309999999998 2024-09-13T10:11:06.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669070+0000", 2024-09-13T10:11:06.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014560199999999999 2024-09-13T10:11:06.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669104+0000", 2024-09-13T10:11:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3654e-05 2024-09-13T10:11:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669173+0000", 2024-09-13T10:11:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9819000000000004e-05 2024-09-13T10:11:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669227+0000", 2024-09-13T10:11:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3980000000000002e-05 2024-09-13T10:11:06.273 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.274 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.274 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.274 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:124 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:06.274 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:11:06.274 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.527739775000001, 2024-09-13T10:11:06.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030852801999999999, 2024-09-13T10:11:06.274 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.274 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.274 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.274 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.274 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:06.275 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:06.275 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 124 2024-09-13T10:11:06.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.275 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:11:06.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:11:06.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641148+0000", 2024-09-13T10:11:06.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4529999999999999e-06 2024-09-13T10:11:06.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641158+0000", 2024-09-13T10:11:06.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0003999999999999e-05 2024-09-13T10:11:06.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641199+0000", 2024-09-13T10:11:06.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0426999999999998e-05 2024-09-13T10:11:06.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641214+0000", 2024-09-13T10:11:06.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5716e-05 2024-09-13T10:11:06.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669267+0000", 2024-09-13T10:11:06.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028053112000000002 2024-09-13T10:11:06.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669394+0000", 2024-09-13T10:11:06.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012656000000000001 2024-09-13T10:11:06.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:06.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669614+0000", 2024-09-13T10:11:06.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021970400000000001 2024-09-13T10:11:06.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671267+0000", 2024-09-13T10:11:06.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001652976 2024-09-13T10:11:06.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671383+0000", 2024-09-13T10:11:06.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000116113 2024-09-13T10:11:06.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671828+0000", 2024-09-13T10:11:06.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000444871 2024-09-13T10:11:06.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671935+0000", 2024-09-13T10:11:06.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010703 2024-09-13T10:11:06.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671999+0000", 2024-09-13T10:11:06.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4836000000000001e-05 2024-09-13T10:11:06.280 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.280 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.280 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.280 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:201 3.3 e38/30 3:d4fb61be:::smithi19530714-2 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:11:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.155288882000001, 2024-09-13T10:11:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032127035999999998, 2024-09-13T10:11:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:11:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:11:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:11:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2600000000000001e-07 2024-09-13T10:11:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030449+0000", 2024-09-13T10:11:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016850477999999999 2024-09-13T10:11:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030499+0000", 2024-09-13T10:11:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0212e-05 2024-09-13T10:11:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030524+0000", 2024-09-13T10:11:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5222e-05 2024-09-13T10:11:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030739+0000", 2024-09-13T10:11:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000214396 2024-09-13T10:11:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030818+0000", 2024-09-13T10:11:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8941000000000002e-05 2024-09-13T10:11:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.045512+0000", 2024-09-13T10:11:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014694248 2024-09-13T10:11:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.045725+0000", 2024-09-13T10:11:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021261300000000001 2024-09-13T10:11:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:226 3.3 e38/30 3:fe5bfa3a:::smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'5, mlcod=38'3)", 2024-09-13T10:11:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:11:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.088428639, 2024-09-13T10:11:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030943633000000002, 2024-09-13T10:11:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:11:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:11:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080461+0000", 2024-09-13T10:11:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7049999999999999e-06 2024-09-13T10:11:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097571+0000", 2024-09-13T10:11:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017110658000000001 2024-09-13T10:11:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097615+0000", 2024-09-13T10:11:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3404e-05 2024-09-13T10:11:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097635+0000", 2024-09-13T10:11:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0279e-05 2024-09-13T10:11:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097995+0000", 2024-09-13T10:11:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00036038699999999999 2024-09-13T10:11:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.098029+0000", 2024-09-13T10:11:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4162000000000003e-05 2024-09-13T10:11:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111330+0000", 2024-09-13T10:11:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013300268000000001 2024-09-13T10:11:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111401+0000", 2024-09-13T10:11:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1769999999999999e-05 2024-09-13T10:11:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:237 3.3 e38/30 3:de40747e:::smithi19530714-38 low_tier:head v 38'7, mlcod=38'3)", 2024-09-13T10:11:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:11:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.054195847000001, 2024-09-13T10:11:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.037747504000000001, 2024-09-13T10:11:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:11:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:11:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:11:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7299999999999995e-07 2024-09-13T10:11:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145311+0000", 2024-09-13T10:11:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030619621 2024-09-13T10:11:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145336+0000", 2024-09-13T10:11:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5009999999999999e-05 2024-09-13T10:11:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145349+0000", 2024-09-13T10:11:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2724999999999999e-05 2024-09-13T10:11:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145445+0000", 2024-09-13T10:11:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6108999999999997e-05 2024-09-13T10:11:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145470+0000", 2024-09-13T10:11:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5514e-05 2024-09-13T10:11:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152389+0000", 2024-09-13T10:11:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069190739999999999 2024-09-13T10:11:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152438+0000", 2024-09-13T10:11:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8778e-05 2024-09-13T10:11:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:286 4.a 4:5ad8883b:::smithi19530714-37:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:11:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.92059922, 2024-09-13T10:11:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032980435000000002, 2024-09-13T10:11:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 286 2024-09-13T10:11:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:11:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:11:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248290+0000", 2024-09-13T10:11:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0170000000000001e-06 2024-09-13T10:11:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248302+0000", 2024-09-13T10:11:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1602e-05 2024-09-13T10:11:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248328+0000", 2024-09-13T10:11:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.622e-05 2024-09-13T10:11:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248344+0000", 2024-09-13T10:11:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6192999999999999e-05 2024-09-13T10:11:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248466+0000", 2024-09-13T10:11:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012186899999999999 2024-09-13T10:11:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248602+0000", 2024-09-13T10:11:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013623500000000001 2024-09-13T10:11:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 6,13", 2024-09-13T10:11:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274091+0000", 2024-09-13T10:11:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025489050999999999 2024-09-13T10:11:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275976+0000", 2024-09-13T10:11:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018847289999999999 2024-09-13T10:11:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276801+0000", 2024-09-13T10:11:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00082477899999999999 2024-09-13T10:11:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281064+0000", 2024-09-13T10:11:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042625950000000001 2024-09-13T10:11:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281173+0000", 2024-09-13T10:11:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000109391 2024-09-13T10:11:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281268+0000", 2024-09-13T10:11:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4753999999999994e-05 2024-09-13T10:11:06.302 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.302 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.302 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.302 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.302 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.302 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:06.303 INFO:tasks.rados.rados.0.smithi195.stdout:885: expect (ObjNum 85 snap 0 seq_num 85) 2024-09-13T10:11:06.304 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-13T10:11:06.305 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:06.301+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:06.352 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1474969 2024-09-13T10:11:06.389 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-13T10:11:06.404 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:06.401+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:06.429 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:06.429 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:06.429 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:11:06.429 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:06.435 INFO:tasks.rados.rados.0.smithi195.stdout:886: finishing write tid 1 to smithi19530714-20 2024-09-13T10:11:06.436 INFO:tasks.rados.rados.0.smithi195.stdout:886: finishing write tid 2 to smithi19530714-20 2024-09-13T10:11:06.436 INFO:tasks.rados.rados.0.smithi195.stdout:886: finishing write tid 3 to smithi19530714-20 2024-09-13T10:11:06.436 INFO:tasks.rados.rados.0.smithi195.stdout:886: finishing write tid 4 to smithi19530714-20 2024-09-13T10:11:06.436 INFO:tasks.rados.rados.0.smithi195.stdout:886: oid 20 updating version 0 to 34 2024-09-13T10:11:06.436 INFO:tasks.rados.rados.0.smithi195.stdout:886: oid 20 version 34 is already newer than 33 2024-09-13T10:11:06.436 INFO:tasks.rados.rados.0.smithi195.stdout:886: oid 20 updating version 34 to 35 2024-09-13T10:11:06.436 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 35 (ObjNum 135 snap 0 seq_num 135) dirty exists 2024-09-13T10:11:06.436 INFO:tasks.rados.rados.0.smithi195.stdout:886: left oid 20 (ObjNum 135 snap 0 seq_num 135) 2024-09-13T10:11:06.436 INFO:tasks.rados.rados.0.smithi195.stdout:887: finishing write tid 1 to smithi19530714-43 2024-09-13T10:11:06.436 INFO:tasks.rados.rados.0.smithi195.stdout:887: finishing write tid 2 to smithi19530714-43 2024-09-13T10:11:06.436 INFO:tasks.rados.rados.0.smithi195.stdout:887: finishing write tid 3 to smithi19530714-43 2024-09-13T10:11:06.436 INFO:tasks.rados.rados.0.smithi195.stdout:887: finishing write tid 4 to smithi19530714-43 2024-09-13T10:11:06.436 INFO:tasks.rados.rados.0.smithi195.stdout:887: oid 43 updating version 0 to 43 2024-09-13T10:11:06.437 INFO:tasks.rados.rados.0.smithi195.stdout:887: oid 43 version 43 is already newer than 41 2024-09-13T10:11:06.437 INFO:tasks.rados.rados.0.smithi195.stdout:887: oid 43 version 43 is already newer than 42 2024-09-13T10:11:06.437 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 43 (ObjNum 136 snap 0 seq_num 136) dirty exists 2024-09-13T10:11:06.437 INFO:tasks.rados.rados.0.smithi195.stdout:887: left oid 43 (ObjNum 136 snap 0 seq_num 136) 2024-09-13T10:11:06.437 INFO:tasks.rados.rados.0.smithi195.stdout:888: finishing write tid 1 to smithi19530714-33 2024-09-13T10:11:06.437 INFO:tasks.rados.rados.0.smithi195.stdout:888: finishing write tid 2 to smithi19530714-33 2024-09-13T10:11:06.437 INFO:tasks.rados.rados.0.smithi195.stdout:884: done (9 left) 2024-09-13T10:11:06.437 INFO:tasks.rados.rados.0.smithi195.stdout:885: done (8 left) 2024-09-13T10:11:06.437 INFO:tasks.rados.rados.0.smithi195.stdout:886: done (7 left) 2024-09-13T10:11:06.437 INFO:tasks.rados.rados.0.smithi195.stdout:887: done (6 left) 2024-09-13T10:11:06.437 INFO:tasks.rados.rados.0.smithi195.stdout:893: write oid 27 current snap is 0 2024-09-13T10:11:06.437 INFO:tasks.rados.rados.0.smithi195.stdout:888: finishing write tid 3 to smithi19530714-33 2024-09-13T10:11:06.437 INFO:tasks.rados.rados.0.smithi195.stdout:893: seq_num 138 ranges {719366=478716,1808868=510498} 2024-09-13T10:11:06.443 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-13T10:11:06.455 INFO:tasks.rados.rados.0.smithi195.stdout:893: writing smithi19530714-27 from 719366 to 1198082 tid 1 2024-09-13T10:11:06.473 INFO:tasks.rados.rados.0.smithi195.stdout:893: writing smithi19530714-27 from 1808868 to 2319366 tid 2 2024-09-13T10:11:06.473 INFO:tasks.rados.rados.0.smithi195.stdout:894: tier_evict oid 7 2024-09-13T10:11:06.473 INFO:tasks.rados.rados.0.smithi195.stdout:894: tier_evict oid 7 snap -1 2024-09-13T10:11:06.474 INFO:tasks.rados.rados.0.smithi195.stdout:888: finishing write tid 4 to smithi19530714-33 2024-09-13T10:11:06.474 INFO:tasks.rados.rados.0.smithi195.stdout:888: oid 33 updating version 0 to 30 2024-09-13T10:11:06.474 INFO:tasks.rados.rados.0.smithi195.stdout:888: oid 33 updating version 30 to 31 2024-09-13T10:11:06.474 INFO:tasks.rados.rados.0.smithi195.stdout:888: oid 33 updating version 31 to 32 2024-09-13T10:11:06.474 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 32 (ObjNum 137 snap 0 seq_num 137) dirty exists 2024-09-13T10:11:06.474 INFO:tasks.rados.rados.0.smithi195.stdout:888: left oid 33 (ObjNum 137 snap 0 seq_num 137) 2024-09-13T10:11:06.474 INFO:tasks.rados.rados.0.smithi195.stdout:890: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:06.474 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:06.505 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:06.504+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:06.535 INFO:tasks.rados.rados.0.smithi195.stdout:889: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:11:06.535 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:06.606 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:06.605+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:06.606 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:11:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:11:06.607 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:06.608 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2744743 2024-09-13T10:11:06.620 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-13T10:11:06.667 INFO:tasks.rados.rados.0.smithi195.stdout:891: finishing set_chunk to oid 48 2024-09-13T10:11:06.667 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 52 (ObjNum 123 snap 0 seq_num 123) dirty exists 2024-09-13T10:11:06.668 INFO:tasks.rados.rados.0.smithi195.stdout:892: got (0) Success 2024-09-13T10:11:06.668 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 0 (ObjNum 95 snap 0 seq_num 95) dirty exists 2024-09-13T10:11:06.668 INFO:tasks.rados.rados.0.smithi195.stdout:893: finishing write tid 1 to smithi19530714-27 2024-09-13T10:11:06.668 INFO:tasks.rados.rados.0.smithi195.stdout:894: got (22) Invalid argument 2024-09-13T10:11:06.668 INFO:tasks.rados.rados.0.smithi195.stdout:893: finishing write tid 2 to smithi19530714-27 2024-09-13T10:11:06.668 INFO:tasks.rados.rados.0.smithi195.stdout:893: finishing write tid 3 to smithi19530714-27 2024-09-13T10:11:06.668 INFO:tasks.rados.rados.0.smithi195.stdout:893: finishing write tid 4 to smithi19530714-27 2024-09-13T10:11:06.668 INFO:tasks.rados.rados.0.smithi195.stdout:893: oid 27 updating version 0 to 75 2024-09-13T10:11:06.668 INFO:tasks.rados.rados.0.smithi195.stdout:893: oid 27 version 75 is already newer than 74 2024-09-13T10:11:06.668 INFO:tasks.rados.rados.0.smithi195.stdout:893: oid 27 updating version 75 to 76 2024-09-13T10:11:06.668 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 76 (ObjNum 138 snap 0 seq_num 138) dirty exists 2024-09-13T10:11:06.668 INFO:tasks.rados.rados.0.smithi195.stdout:893: left oid 27 (ObjNum 138 snap 0 seq_num 138) 2024-09-13T10:11:06.668 INFO:tasks.rados.rados.0.smithi195.stdout:888: done (7 left) 2024-09-13T10:11:06.669 INFO:tasks.rados.rados.0.smithi195.stdout:889: done (6 left) 2024-09-13T10:11:06.669 INFO:tasks.rados.rados.0.smithi195.stdout:890: done (5 left) 2024-09-13T10:11:06.669 INFO:tasks.rados.rados.0.smithi195.stdout:891: done (4 left) 2024-09-13T10:11:06.669 INFO:tasks.rados.rados.0.smithi195.stdout:892: done (3 left) 2024-09-13T10:11:06.669 INFO:tasks.rados.rados.0.smithi195.stdout:893: done (2 left) 2024-09-13T10:11:06.669 INFO:tasks.rados.rados.0.smithi195.stdout:894: done (1 left) 2024-09-13T10:11:06.669 INFO:tasks.rados.rados.0.smithi195.stdout:895: write oid 22 current snap is 0 2024-09-13T10:11:06.669 INFO:tasks.rados.rados.0.smithi195.stdout:895: seq_num 139 ranges {736173=553965,1867350=468823} 2024-09-13T10:11:06.688 INFO:tasks.rados.rados.0.smithi195.stdout:895: writing smithi19530714-22 from 736173 to 1290138 tid 1 2024-09-13T10:11:06.705 INFO:tasks.rados.rados.0.smithi195.stdout:895: writing smithi19530714-22 from 1867350 to 2336173 tid 2 2024-09-13T10:11:06.705 INFO:tasks.rados.rados.0.smithi195.stdout:896: read oid 47 snap -1 2024-09-13T10:11:06.706 INFO:tasks.rados.rados.0.smithi195.stdout:896: expect deleted 2024-09-13T10:11:06.706 INFO:tasks.rados.rados.0.smithi195.stdout:897: read oid 8 snap -1 2024-09-13T10:11:06.706 INFO:tasks.rados.rados.0.smithi195.stdout:897: expect (ObjNum 104 snap 0 seq_num 104) 2024-09-13T10:11:06.706 INFO:tasks.rados.rados.0.smithi195.stdout:898: write oid 28 current snap is 0 2024-09-13T10:11:06.706 INFO:tasks.rados.rados.0.smithi195.stdout:898: seq_num 140 ranges {752980=629214,1809479=543501} 2024-09-13T10:11:06.707 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:06.703+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:06.725 INFO:tasks.rados.rados.0.smithi195.stdout:898: writing smithi19530714-28 from 752980 to 1382194 tid 1 2024-09-13T10:11:06.737 INFO:tasks.rados.rados.0.smithi195.stdout:898: writing smithi19530714-28 from 1809479 to 2352980 tid 2 2024-09-13T10:11:06.737 INFO:tasks.rados.rados.0.smithi195.stdout:899: read oid 31 snap -1 2024-09-13T10:11:06.737 INFO:tasks.rados.rados.0.smithi195.stdout:899: expect (ObjNum 93 snap 0 seq_num 93) 2024-09-13T10:11:06.737 INFO:tasks.rados.rados.0.smithi195.stdout:900: tier_promote oid 24 2024-09-13T10:11:06.737 INFO:tasks.rados.rados.0.smithi195.stdout:901: read oid 10 snap -1 2024-09-13T10:11:06.737 INFO:tasks.rados.rados.0.smithi195.stdout:901: expect (ObjNum 84 snap 0 seq_num 84) 2024-09-13T10:11:06.737 INFO:tasks.rados.rados.0.smithi195.stdout:902: read oid 37 snap -1 2024-09-13T10:11:06.738 INFO:tasks.rados.rados.0.smithi195.stdout:902: expect (ObjNum 95 snap 0 seq_num 95) 2024-09-13T10:11:06.738 INFO:tasks.rados.rados.0.smithi195.stdout:903: tier_flush oid 25 2024-09-13T10:11:06.738 INFO:tasks.rados.rados.0.smithi195.stdout:903: tier_flush oid 25 snap -1 2024-09-13T10:11:06.738 INFO:tasks.rados.rados.0.smithi195.stdout:904: read oid 36 snap -1 2024-09-13T10:11:06.738 INFO:tasks.rados.rados.0.smithi195.stdout:904: expect (ObjNum 80 snap 0 seq_num 80) 2024-09-13T10:11:06.738 INFO:tasks.rados.rados.0.smithi195.stdout:905: write (excl) oid 41 current snap is 0 2024-09-13T10:11:06.738 INFO:tasks.rados.rados.0.smithi195.stdout:905: seq_num 141 ranges {769787=704463,2267961=101826} 2024-09-13T10:11:06.752 INFO:tasks.rados.rados.0.smithi195.stdout:905: writing smithi19530714-41 from 769787 to 1474250 tid 1 2024-09-13T10:11:06.761 INFO:tasks.rados.rados.0.smithi195.stdout:905: writing smithi19530714-41 from 2267961 to 2369787 tid 2 2024-09-13T10:11:06.761 INFO:tasks.rados.rados.0.smithi195.stdout:906: read oid 30 snap -1 2024-09-13T10:11:06.761 INFO:tasks.rados.rados.0.smithi195.stdout:906: expect (ObjNum 102 snap 0 seq_num 102) 2024-09-13T10:11:06.761 INFO:tasks.rados.rados.0.smithi195.stdout:907: write (excl) oid 1 current snap is 0 2024-09-13T10:11:06.762 INFO:tasks.rados.rados.0.smithi195.stdout:907: seq_num 142 ranges {786594=779712,2326443=60151} 2024-09-13T10:11:06.778 INFO:tasks.rados.rados.0.smithi195.stdout:907: writing smithi19530714-1 from 786594 to 1566306 tid 1 2024-09-13T10:11:06.786 INFO:tasks.rados.rados.0.smithi195.stdout:907: writing smithi19530714-1 from 2326443 to 2386594 tid 2 2024-09-13T10:11:06.786 INFO:tasks.rados.rados.0.smithi195.stdout:908: read oid 26 snap -1 2024-09-13T10:11:06.786 INFO:tasks.rados.rados.0.smithi195.stdout:908: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:06.786 INFO:tasks.rados.rados.0.smithi195.stdout:909: tier_promote oid 9 2024-09-13T10:11:06.786 INFO:tasks.rados.rados.0.smithi195.stdout:895: finishing write tid 1 to smithi19530714-22 2024-09-13T10:11:06.786 INFO:tasks.rados.rados.0.smithi195.stdout:895: finishing write tid 2 to smithi19530714-22 2024-09-13T10:11:06.786 INFO:tasks.rados.rados.0.smithi195.stdout:895: finishing write tid 3 to smithi19530714-22 2024-09-13T10:11:06.786 INFO:tasks.rados.rados.0.smithi195.stdout:895: finishing write tid 4 to smithi19530714-22 2024-09-13T10:11:06.786 INFO:tasks.rados.rados.0.smithi195.stdout:895: oid 22 updating version 0 to 79 2024-09-13T10:11:06.786 INFO:tasks.rados.rados.0.smithi195.stdout:895: oid 22 version 79 is already newer than 77 2024-09-13T10:11:06.786 INFO:tasks.rados.rados.0.smithi195.stdout:895: oid 22 version 79 is already newer than 78 2024-09-13T10:11:06.787 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 79 (ObjNum 139 snap 0 seq_num 139) dirty exists 2024-09-13T10:11:06.787 INFO:tasks.rados.rados.0.smithi195.stdout:895: left oid 22 (ObjNum 139 snap 0 seq_num 139) 2024-09-13T10:11:06.787 INFO:tasks.rados.rados.0.smithi195.stdout:897: expect (ObjNum 104 snap 0 seq_num 104) 2024-09-13T10:11:06.793 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:06.793 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:11:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:11:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:11:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:53 4.c 4:3e6706e5:::smithi19530714-14:head [write 618491~694590,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:50.494212+0000", 2024-09-13T10:11:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 16.298160726999999, 2024-09-13T10:11:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0086614210000000007, 2024-09-13T10:11:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53 2024-09-13T10:11:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494212+0000", 2024-09-13T10:11:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494212+0000", 2024-09-13T10:11:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494213+0000", 2024-09-13T10:11:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.84e-06 2024-09-13T10:11:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494889+0000", 2024-09-13T10:11:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00067531799999999995 2024-09-13T10:11:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494907+0000", 2024-09-13T10:11:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7884999999999999e-05 2024-09-13T10:11:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494917+0000", 2024-09-13T10:11:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0043e-05 2024-09-13T10:11:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494971+0000", 2024-09-13T10:11:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4775000000000003e-05 2024-09-13T10:11:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.495081+0000", 2024-09-13T10:11:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110055 2024-09-13T10:11:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.495184+0000", 2024-09-13T10:11:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010221 2024-09-13T10:11:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.496618+0000", 2024-09-13T10:11:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014343590000000001 2024-09-13T10:11:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.498806+0000", 2024-09-13T10:11:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021880269999999999 2024-09-13T10:11:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.502814+0000", 2024-09-13T10:11:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040077890000000003 2024-09-13T10:11:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.502844+0000", 2024-09-13T10:11:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0462000000000001e-05 2024-09-13T10:11:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.502873+0000", 2024-09-13T10:11:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8657999999999999e-05 2024-09-13T10:11:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:161 4.d 4:b19b932a:::smithi19530714-41:head [write 672280~791725,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:50.852563+0000", 2024-09-13T10:11:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.93980876, 2024-09-13T10:11:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011425286999999999, 2024-09-13T10:11:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 161 2024-09-13T10:11:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.852563+0000", 2024-09-13T10:11:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.852563+0000", 2024-09-13T10:11:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.852567+0000", 2024-09-13T10:11:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1269999999999999e-06 2024-09-13T10:11:06.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853363+0000", 2024-09-13T10:11:06.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00079620899999999998 2024-09-13T10:11:06.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853394+0000", 2024-09-13T10:11:06.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1192000000000002e-05 2024-09-13T10:11:06.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.805 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.805 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853410+0000", 2024-09-13T10:11:06.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5896000000000002e-05 2024-09-13T10:11:06.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.805 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.805 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853574+0000", 2024-09-13T10:11:06.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000164358 2024-09-13T10:11:06.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.805 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.805 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853920+0000", 2024-09-13T10:11:06.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034612500000000002 2024-09-13T10:11:06.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:06.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.854305+0000", 2024-09-13T10:11:06.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038458900000000001 2024-09-13T10:11:06.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.858018+0000", 2024-09-13T10:11:06.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0037128619999999999 2024-09-13T10:11:06.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.858905+0000", 2024-09-13T10:11:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00088737899999999999 2024-09-13T10:11:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.863681+0000", 2024-09-13T10:11:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004775311 2024-09-13T10:11:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.863813+0000", 2024-09-13T10:11:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013254900000000001 2024-09-13T10:11:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.863989+0000", 2024-09-13T10:11:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017568999999999999 2024-09-13T10:11:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(osd.14.23:4 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:50.996997+0000", 2024-09-13T10:11:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.795375354000001, 2024-09-13T10:11:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0083262170000000003, 2024-09-13T10:11:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:11:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.996997+0000", 2024-09-13T10:11:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.996997+0000", 2024-09-13T10:11:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997000+0000", 2024-09-13T10:11:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3560000000000001e-06 2024-09-13T10:11:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997015+0000", 2024-09-13T10:11:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.523e-05 2024-09-13T10:11:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997048+0000", 2024-09-13T10:11:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2162000000000001e-05 2024-09-13T10:11:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997071+0000", 2024-09-13T10:11:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2918999999999999e-05 2024-09-13T10:11:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.001996+0000", 2024-09-13T10:11:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049256559999999996 2024-09-13T10:11:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002198+0000", 2024-09-13T10:11:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020196600000000001 2024-09-13T10:11:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.005323+0000", 2024-09-13T10:11:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0031249279999999999 2024-09-13T10:11:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:261 4.c 4:3d56d753:::smithi19530714-12:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.215641+0000", 2024-09-13T10:11:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.576731036, 2024-09-13T10:11:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010509116000000001, 2024-09-13T10:11:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 261 2024-09-13T10:11:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215641+0000", 2024-09-13T10:11:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215641+0000", 2024-09-13T10:11:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215646+0000", 2024-09-13T10:11:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9300000000000002e-06 2024-09-13T10:11:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215666+0000", 2024-09-13T10:11:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0095999999999999e-05 2024-09-13T10:11:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215713+0000", 2024-09-13T10:11:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7125e-05 2024-09-13T10:11:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215739+0000", 2024-09-13T10:11:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5488999999999999e-05 2024-09-13T10:11:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215918+0000", 2024-09-13T10:11:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017948399999999999 2024-09-13T10:11:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216165+0000", 2024-09-13T10:11:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024625200000000003 2024-09-13T10:11:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.223467+0000", 2024-09-13T10:11:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0073023840000000003 2024-09-13T10:11:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.225020+0000", 2024-09-13T10:11:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015529809999999999 2024-09-13T10:11:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.225406+0000", 2024-09-13T10:11:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038614699999999999 2024-09-13T10:11:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.225903+0000", 2024-09-13T10:11:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00049721900000000002 2024-09-13T10:11:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.226026+0000", 2024-09-13T10:11:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000123163 2024-09-13T10:11:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.226150+0000", 2024-09-13T10:11:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012384599999999999 2024-09-13T10:11:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:263 4.c 4:3e6706e5:::smithi19530714-14:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.215802+0000", 2024-09-13T10:11:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.576569828, 2024-09-13T10:11:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015708573999999999, 2024-09-13T10:11:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 263 2024-09-13T10:11:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215802+0000", 2024-09-13T10:11:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215802+0000", 2024-09-13T10:11:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215804+0000", 2024-09-13T10:11:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.624e-06 2024-09-13T10:11:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215813+0000", 2024-09-13T10:11:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4159999999999993e-06 2024-09-13T10:11:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215832+0000", 2024-09-13T10:11:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8768999999999999e-05 2024-09-13T10:11:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215843+0000", 2024-09-13T10:11:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0336999999999999e-05 2024-09-13T10:11:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216612+0000", 2024-09-13T10:11:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00076899399999999995 2024-09-13T10:11:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216767+0000", 2024-09-13T10:11:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015566399999999999 2024-09-13T10:11:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.227138+0000", 2024-09-13T10:11:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010370701 2024-09-13T10:11:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.229299+0000", 2024-09-13T10:11:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021611870000000002 2024-09-13T10:11:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.229896+0000", 2024-09-13T10:11:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00059659700000000003 2024-09-13T10:11:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231283+0000", 2024-09-13T10:11:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001387133 2024-09-13T10:11:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231392+0000", 2024-09-13T10:11:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010878999999999999 2024-09-13T10:11:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231511+0000", 2024-09-13T10:11:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119362 2024-09-13T10:11:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:264 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.215936+0000", 2024-09-13T10:11:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.576435848999999, 2024-09-13T10:11:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054954390999999998, 2024-09-13T10:11:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 264 2024-09-13T10:11:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215936+0000", 2024-09-13T10:11:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215936+0000", 2024-09-13T10:11:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215938+0000", 2024-09-13T10:11:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9860000000000001e-06 2024-09-13T10:11:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215951+0000", 2024-09-13T10:11:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2449999999999999e-05 2024-09-13T10:11:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215982+0000", 2024-09-13T10:11:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1133000000000002e-05 2024-09-13T10:11:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216009+0000", 2024-09-13T10:11:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6652000000000002e-05 2024-09-13T10:11:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.217072+0000", 2024-09-13T10:11:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010634240000000001 2024-09-13T10:11:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.217268+0000", 2024-09-13T10:11:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019641 2024-09-13T10:11:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.265415+0000", 2024-09-13T10:11:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.048146807999999999 2024-09-13T10:11:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.267067+0000", 2024-09-13T10:11:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00165211 2024-09-13T10:11:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.267319+0000", 2024-09-13T10:11:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025160500000000001 2024-09-13T10:11:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270591+0000", 2024-09-13T10:11:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0032720959999999999 2024-09-13T10:11:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270753+0000", 2024-09-13T10:11:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016215199999999999 2024-09-13T10:11:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270891+0000", 2024-09-13T10:11:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000137565 2024-09-13T10:11:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:278 4.c 4:39f17ee9:::smithi19530714-29:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.231988+0000", 2024-09-13T10:11:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.560383982999999, 2024-09-13T10:11:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011030108, 2024-09-13T10:11:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 278 2024-09-13T10:11:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231988+0000", 2024-09-13T10:11:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231988+0000", 2024-09-13T10:11:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231990+0000", 2024-09-13T10:11:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0499999999999999e-06 2024-09-13T10:11:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232002+0000", 2024-09-13T10:11:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2189e-05 2024-09-13T10:11:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232029+0000", 2024-09-13T10:11:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6673e-05 2024-09-13T10:11:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232043+0000", 2024-09-13T10:11:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4188999999999999e-05 2024-09-13T10:11:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232239+0000", 2024-09-13T10:11:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019557 2024-09-13T10:11:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232472+0000", 2024-09-13T10:11:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00023310900000000001 2024-09-13T10:11:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.237292+0000", 2024-09-13T10:11:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048195069999999998 2024-09-13T10:11:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.238595+0000", 2024-09-13T10:11:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013033260000000001 2024-09-13T10:11:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.239056+0000", 2024-09-13T10:11:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00046106800000000002 2024-09-13T10:11:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.242820+0000", 2024-09-13T10:11:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003763897 2024-09-13T10:11:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.242905+0000", 2024-09-13T10:11:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5580000000000004e-05 2024-09-13T10:11:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.243018+0000", 2024-09-13T10:11:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011294999999999999 2024-09-13T10:11:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:290 4.d 4:b19b932a:::smithi19530714-41:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.255335+0000", 2024-09-13T10:11:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.537037461000001, 2024-09-13T10:11:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021335703000000001, 2024-09-13T10:11:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 290 2024-09-13T10:11:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255335+0000", 2024-09-13T10:11:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255335+0000", 2024-09-13T10:11:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255338+0000", 2024-09-13T10:11:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6940000000000001e-06 2024-09-13T10:11:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255353+0000", 2024-09-13T10:11:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4136e-05 2024-09-13T10:11:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255390+0000", 2024-09-13T10:11:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7364999999999999e-05 2024-09-13T10:11:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255413+0000", 2024-09-13T10:11:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2690000000000001e-05 2024-09-13T10:11:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255615+0000", 2024-09-13T10:11:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000202306 2024-09-13T10:11:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255813+0000", 2024-09-13T10:11:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019769300000000001 2024-09-13T10:11:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274190+0000", 2024-09-13T10:11:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01837714 2024-09-13T10:11:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275400+0000", 2024-09-13T10:11:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001210275 2024-09-13T10:11:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276346+0000", 2024-09-13T10:11:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00094586699999999998 2024-09-13T10:11:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276497+0000", 2024-09-13T10:11:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015150299999999999 2024-09-13T10:11:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276597+0000", 2024-09-13T10:11:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9092999999999995e-05 2024-09-13T10:11:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276670+0000", 2024-09-13T10:11:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3941000000000002e-05 2024-09-13T10:11:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:295 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.261467+0000", 2024-09-13T10:11:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.530905321000001, 2024-09-13T10:11:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020013564000000001, 2024-09-13T10:11:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 295 2024-09-13T10:11:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261467+0000", 2024-09-13T10:11:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261467+0000", 2024-09-13T10:11:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261470+0000", 2024-09-13T10:11:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8200000000000001e-06 2024-09-13T10:11:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261487+0000", 2024-09-13T10:11:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7646e-05 2024-09-13T10:11:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261523+0000", 2024-09-13T10:11:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5299000000000002e-05 2024-09-13T10:11:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261545+0000", 2024-09-13T10:11:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2436e-05 2024-09-13T10:11:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261717+0000", 2024-09-13T10:11:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017231000000000001 2024-09-13T10:11:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261980+0000", 2024-09-13T10:11:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000262959 2024-09-13T10:11:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277104+0000", 2024-09-13T10:11:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015123983000000001 2024-09-13T10:11:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.279042+0000", 2024-09-13T10:11:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001937338 2024-09-13T10:11:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.279396+0000", 2024-09-13T10:11:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00035408199999999998 2024-09-13T10:11:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281302+0000", 2024-09-13T10:11:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001906346 2024-09-13T10:11:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281400+0000", 2024-09-13T10:11:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7528000000000005e-05 2024-09-13T10:11:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281480+0000", 2024-09-13T10:11:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0816999999999996e-05 2024-09-13T10:11:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:297 4.c 4:381e677b:::smithi19530714-48:head [assert-version v15,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.262493+0000", 2024-09-13T10:11:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.529879151999999, 2024-09-13T10:11:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012458446, 2024-09-13T10:11:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 297 2024-09-13T10:11:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262493+0000", 2024-09-13T10:11:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262493+0000", 2024-09-13T10:11:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262496+0000", 2024-09-13T10:11:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4729999999999999e-06 2024-09-13T10:11:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262510+0000", 2024-09-13T10:11:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4205999999999999e-05 2024-09-13T10:11:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262545+0000", 2024-09-13T10:11:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4789000000000002e-05 2024-09-13T10:11:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262566+0000", 2024-09-13T10:11:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1789000000000001e-05 2024-09-13T10:11:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262748+0000", 2024-09-13T10:11:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000181848 2024-09-13T10:11:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262986+0000", 2024-09-13T10:11:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00023747500000000001 2024-09-13T10:11:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.267794+0000", 2024-09-13T10:11:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048086479999999996 2024-09-13T10:11:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.269878+0000", 2024-09-13T10:11:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0020833900000000001 2024-09-13T10:11:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270928+0000", 2024-09-13T10:11:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001049901 2024-09-13T10:11:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274724+0000", 2024-09-13T10:11:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0037964880000000002 2024-09-13T10:11:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274862+0000", 2024-09-13T10:11:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000138286 2024-09-13T10:11:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274952+0000", 2024-09-13T10:11:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9153000000000005e-05 2024-09-13T10:11:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:313 4.c 4:3e6706e5:::smithi19530714-14:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.275533+0000", 2024-09-13T10:11:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.516838934999999, 2024-09-13T10:11:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019582932000000001, 2024-09-13T10:11:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 313 2024-09-13T10:11:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275533+0000", 2024-09-13T10:11:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275533+0000", 2024-09-13T10:11:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275536+0000", 2024-09-13T10:11:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4899999999999999e-06 2024-09-13T10:11:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275549+0000", 2024-09-13T10:11:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3699e-05 2024-09-13T10:11:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275574+0000", 2024-09-13T10:11:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4586000000000001e-05 2024-09-13T10:11:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275588+0000", 2024-09-13T10:11:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3746e-05 2024-09-13T10:11:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275705+0000", 2024-09-13T10:11:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000117352 2024-09-13T10:11:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275898+0000", 2024-09-13T10:11:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019293000000000001 2024-09-13T10:11:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286176+0000", 2024-09-13T10:11:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010277578000000001 2024-09-13T10:11:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.287526+0000", 2024-09-13T10:11:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013505349999999999 2024-09-13T10:11:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.288025+0000", 2024-09-13T10:11:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00049897500000000003 2024-09-13T10:11:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294958+0000", 2024-09-13T10:11:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069327980000000004 2024-09-13T10:11:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.295047+0000", 2024-09-13T10:11:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9154999999999996e-05 2024-09-13T10:11:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.295116+0000", 2024-09-13T10:11:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9088e-05 2024-09-13T10:11:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:314 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.276956+0000", 2024-09-13T10:11:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.515416256, 2024-09-13T10:11:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017940078000000002, 2024-09-13T10:11:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 314 2024-09-13T10:11:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276956+0000", 2024-09-13T10:11:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276956+0000", 2024-09-13T10:11:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276959+0000", 2024-09-13T10:11:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5730000000000002e-06 2024-09-13T10:11:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276971+0000", 2024-09-13T10:11:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2680000000000001e-05 2024-09-13T10:11:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276997+0000", 2024-09-13T10:11:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5579e-05 2024-09-13T10:11:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277018+0000", 2024-09-13T10:11:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1409000000000001e-05 2024-09-13T10:11:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277136+0000", 2024-09-13T10:11:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000117353 2024-09-13T10:11:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277237+0000", 2024-09-13T10:11:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101412 2024-09-13T10:11:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.285332+0000", 2024-09-13T10:11:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080951770000000003 2024-09-13T10:11:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.287120+0000", 2024-09-13T10:11:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001787828 2024-09-13T10:11:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.287200+0000", 2024-09-13T10:11:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0279999999999997e-05 2024-09-13T10:11:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294808+0000", 2024-09-13T10:11:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0076072400000000004 2024-09-13T10:11:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294853+0000", 2024-09-13T10:11:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5438000000000001e-05 2024-09-13T10:11:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294896+0000", 2024-09-13T10:11:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3108999999999997e-05 2024-09-13T10:11:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:340 4.d 4:b19b932a:::smithi19530714-41:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.284107+0000", 2024-09-13T10:11:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.508265026, 2024-09-13T10:11:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010293416, 2024-09-13T10:11:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 340 2024-09-13T10:11:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284107+0000", 2024-09-13T10:11:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284107+0000", 2024-09-13T10:11:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284109+0000", 2024-09-13T10:11:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.007e-06 2024-09-13T10:11:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284120+0000", 2024-09-13T10:11:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0346999999999999e-05 2024-09-13T10:11:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284144+0000", 2024-09-13T10:11:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3998999999999998e-05 2024-09-13T10:11:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284157+0000", 2024-09-13T10:11:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3045e-05 2024-09-13T10:11:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284244+0000", 2024-09-13T10:11:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7882999999999999e-05 2024-09-13T10:11:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284360+0000", 2024-09-13T10:11:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115467 2024-09-13T10:11:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.292368+0000", 2024-09-13T10:11:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080080380000000003 2024-09-13T10:11:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.293486+0000", 2024-09-13T10:11:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001117776 2024-09-13T10:11:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294261+0000", 2024-09-13T10:11:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00077481100000000003 2024-09-13T10:11:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294309+0000", 2024-09-13T10:11:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8086e-05 2024-09-13T10:11:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294358+0000", 2024-09-13T10:11:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8853999999999997e-05 2024-09-13T10:11:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294401+0000", 2024-09-13T10:11:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3102999999999997e-05 2024-09-13T10:11:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:345 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.286309+0000", 2024-09-13T10:11:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.506062847999999, 2024-09-13T10:11:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010478568000000001, 2024-09-13T10:11:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 345 2024-09-13T10:11:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286309+0000", 2024-09-13T10:11:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286309+0000", 2024-09-13T10:11:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286311+0000", 2024-09-13T10:11:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-13T10:11:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286320+0000", 2024-09-13T10:11:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8370000000000003e-06 2024-09-13T10:11:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286342+0000", 2024-09-13T10:11:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1446000000000001e-05 2024-09-13T10:11:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286385+0000", 2024-09-13T10:11:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3083000000000001e-05 2024-09-13T10:11:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286549+0000", 2024-09-13T10:11:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016463 2024-09-13T10:11:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286739+0000", 2024-09-13T10:11:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018965400000000001 2024-09-13T10:11:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294808+0000", 2024-09-13T10:11:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080692609999999994 2024-09-13T10:11:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.295388+0000", 2024-09-13T10:11:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00057994000000000003 2024-09-13T10:11:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296321+0000", 2024-09-13T10:11:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00093279099999999998 2024-09-13T10:11:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296622+0000", 2024-09-13T10:11:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030084300000000002 2024-09-13T10:11:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296702+0000", 2024-09-13T10:11:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0547e-05 2024-09-13T10:11:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296788+0000", 2024-09-13T10:11:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5612999999999995e-05 2024-09-13T10:11:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:364 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.297175+0000", 2024-09-13T10:11:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.495197672, 2024-09-13T10:11:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0085499329999999991, 2024-09-13T10:11:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 364 2024-09-13T10:11:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297175+0000", 2024-09-13T10:11:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297175+0000", 2024-09-13T10:11:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297176+0000", 2024-09-13T10:11:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9199999999999999e-07 2024-09-13T10:11:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297181+0000", 2024-09-13T10:11:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5149999999999997e-06 2024-09-13T10:11:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297191+0000", 2024-09-13T10:11:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8260000000000005e-06 2024-09-13T10:11:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297198+0000", 2024-09-13T10:11:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6950000000000003e-06 2024-09-13T10:11:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297244+0000", 2024-09-13T10:11:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6397999999999999e-05 2024-09-13T10:11:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297291+0000", 2024-09-13T10:11:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7342999999999997e-05 2024-09-13T10:11:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.303658+0000", 2024-09-13T10:11:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0063669 2024-09-13T10:11:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.304437+0000", 2024-09-13T10:11:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00077861399999999996 2024-09-13T10:11:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.304902+0000", 2024-09-13T10:11:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00046505899999999998 2024-09-13T10:11:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.305534+0000", 2024-09-13T10:11:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00063224199999999998 2024-09-13T10:11:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.305643+0000", 2024-09-13T10:11:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108384 2024-09-13T10:11:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.305725+0000", 2024-09-13T10:11:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1965000000000006e-05 2024-09-13T10:11:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:378 4.c 4:39f17ee9:::smithi19530714-29:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.320710+0000", 2024-09-13T10:11:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.471662301, 2024-09-13T10:11:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0079049020000000001, 2024-09-13T10:11:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 378 2024-09-13T10:11:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320710+0000", 2024-09-13T10:11:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320710+0000", 2024-09-13T10:11:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320712+0000", 2024-09-13T10:11:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4099999999999998e-06 2024-09-13T10:11:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320724+0000", 2024-09-13T10:11:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.134e-05 2024-09-13T10:11:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320746+0000", 2024-09-13T10:11:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2342999999999999e-05 2024-09-13T10:11:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320758+0000", 2024-09-13T10:11:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1557e-05 2024-09-13T10:11:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320840+0000", 2024-09-13T10:11:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2274000000000005e-05 2024-09-13T10:11:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320933+0000", 2024-09-13T10:11:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3375999999999995e-05 2024-09-13T10:11:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.326291+0000", 2024-09-13T10:11:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053575089999999999 2024-09-13T10:11:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.327254+0000", 2024-09-13T10:11:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00096337099999999998 2024-09-13T10:11:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.327902+0000", 2024-09-13T10:11:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00064762100000000003 2024-09-13T10:11:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328501+0000", 2024-09-13T10:11:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000599232 2024-09-13T10:11:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328547+0000", 2024-09-13T10:11:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5973000000000002e-05 2024-09-13T10:11:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328615+0000", 2024-09-13T10:11:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7896000000000002e-05 2024-09-13T10:11:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:395 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.322105+0000", 2024-09-13T10:11:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.470267228999999, 2024-09-13T10:11:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0084827089999999997, 2024-09-13T10:11:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 395 2024-09-13T10:11:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322105+0000", 2024-09-13T10:11:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322105+0000", 2024-09-13T10:11:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322106+0000", 2024-09-13T10:11:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2099999999999995e-07 2024-09-13T10:11:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322110+0000", 2024-09-13T10:11:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.827e-06 2024-09-13T10:11:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322120+0000", 2024-09-13T10:11:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0227999999999999e-05 2024-09-13T10:11:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322160+0000", 2024-09-13T10:11:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9941999999999998e-05 2024-09-13T10:11:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322232+0000", 2024-09-13T10:11:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2451000000000001e-05 2024-09-13T10:11:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322314+0000", 2024-09-13T10:11:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2089000000000006e-05 2024-09-13T10:11:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328374+0000", 2024-09-13T10:11:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0060591350000000002 2024-09-13T10:11:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.329439+0000", 2024-09-13T10:11:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010654830000000001 2024-09-13T10:11:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330219+0000", 2024-09-13T10:11:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00078003700000000003 2024-09-13T10:11:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330488+0000", 2024-09-13T10:11:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000269411 2024-09-13T10:11:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330544+0000", 2024-09-13T10:11:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5704000000000001e-05 2024-09-13T10:11:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330588+0000", 2024-09-13T10:11:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3581000000000002e-05 2024-09-13T10:11:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:595 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.370445+0000", 2024-09-13T10:11:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.421927195, 2024-09-13T10:11:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0084586149999999992, 2024-09-13T10:11:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 595 2024-09-13T10:11:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370445+0000", 2024-09-13T10:11:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370445+0000", 2024-09-13T10:11:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370447+0000", 2024-09-13T10:11:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.66e-06 2024-09-13T10:11:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370455+0000", 2024-09-13T10:11:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3059999999999998e-06 2024-09-13T10:11:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370472+0000", 2024-09-13T10:11:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7479999999999999e-05 2024-09-13T10:11:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370484+0000", 2024-09-13T10:11:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1882000000000001e-05 2024-09-13T10:11:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370562+0000", 2024-09-13T10:11:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7531e-05 2024-09-13T10:11:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370738+0000", 2024-09-13T10:11:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017587200000000001 2024-09-13T10:11:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.377463+0000", 2024-09-13T10:11:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0067247829999999998 2024-09-13T10:11:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378047+0000", 2024-09-13T10:11:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00058493100000000001 2024-09-13T10:11:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378730+0000", 2024-09-13T10:11:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068235799999999999 2024-09-13T10:11:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378840+0000", 2024-09-13T10:11:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110296 2024-09-13T10:11:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378875+0000", 2024-09-13T10:11:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4552999999999999e-05 2024-09-13T10:11:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378904+0000", 2024-09-13T10:11:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8963e-05 2024-09-13T10:11:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:782 4.c 4:39f17ee9:::smithi19530714-29:head [write 1543557~550004] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.601833+0000", 2024-09-13T10:11:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.190539320999999, 2024-09-13T10:11:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081663250000000003, 2024-09-13T10:11:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 782 2024-09-13T10:11:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.601833+0000", 2024-09-13T10:11:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.601833+0000", 2024-09-13T10:11:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.601835+0000", 2024-09-13T10:11:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.576e-06 2024-09-13T10:11:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602350+0000", 2024-09-13T10:11:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00051449700000000004 2024-09-13T10:11:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602379+0000", 2024-09-13T10:11:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8598999999999998e-05 2024-09-13T10:11:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602394+0000", 2024-09-13T10:11:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4952e-05 2024-09-13T10:11:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602534+0000", 2024-09-13T10:11:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014037899999999999 2024-09-13T10:11:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602796+0000", 2024-09-13T10:11:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000262415 2024-09-13T10:11:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.603214+0000", 2024-09-13T10:11:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00041785599999999998 2024-09-13T10:11:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.606389+0000", 2024-09-13T10:11:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0031751549999999998 2024-09-13T10:11:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.608306+0000", 2024-09-13T10:11:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0019163540000000001 2024-09-13T10:11:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.609577+0000", 2024-09-13T10:11:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001271466 2024-09-13T10:11:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.609872+0000", 2024-09-13T10:11:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00029501400000000002 2024-09-13T10:11:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.609999+0000", 2024-09-13T10:11:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127062 2024-09-13T10:11:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1085 4.c 4:3e6706e5:::smithi19530714-14:head [sparse-read 0~906036] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:56.648267+0000", 2024-09-13T10:11:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.144104798000001, 2024-09-13T10:11:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0083262089999999993, 2024-09-13T10:11:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:11:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1085 2024-09-13T10:11:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648267+0000", 2024-09-13T10:11:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648267+0000", 2024-09-13T10:11:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648269+0000", 2024-09-13T10:11:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1769999999999999e-06 2024-09-13T10:11:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648276+0000", 2024-09-13T10:11:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0530000000000001e-06 2024-09-13T10:11:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648292+0000", 2024-09-13T10:11:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6209e-05 2024-09-13T10:11:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648307+0000", 2024-09-13T10:11:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4647000000000001e-05 2024-09-13T10:11:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.655806+0000", 2024-09-13T10:11:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0074996500000000001 2024-09-13T10:11:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.656060+0000", 2024-09-13T10:11:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000253639 2024-09-13T10:11:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.656594+0000", 2024-09-13T10:11:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00053383399999999996 2024-09-13T10:11:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:06.928 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:06.928 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:06.932 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1738475 2024-09-13T10:11:06.933 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:06.933 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:06.927+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:06.934 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T10:11:06.960 INFO:tasks.rados.rados.0.smithi195.stdout:898: finishing write tid 1 to smithi19530714-28 2024-09-13T10:11:06.960 INFO:tasks.rados.rados.0.smithi195.stdout:903: got (2) No such file or directory 2024-09-13T10:11:06.960 INFO:tasks.rados.rados.0.smithi195.stdout:903: got expected ENOENT (src dne) 2024-09-13T10:11:06.960 INFO:tasks.rados.rados.0.smithi195.stdout:898: finishing write tid 2 to smithi19530714-28 2024-09-13T10:11:06.960 INFO:tasks.rados.rados.0.smithi195.stdout:898: finishing write tid 3 to smithi19530714-28 2024-09-13T10:11:06.960 INFO:tasks.rados.rados.0.smithi195.stdout:898: finishing write tid 4 to smithi19530714-28 2024-09-13T10:11:06.961 INFO:tasks.rados.rados.0.smithi195.stdout:898: oid 28 updating version 0 to 74 2024-09-13T10:11:06.961 INFO:tasks.rados.rados.0.smithi195.stdout:898: oid 28 updating version 74 to 75 2024-09-13T10:11:06.961 INFO:tasks.rados.rados.0.smithi195.stdout:898: oid 28 version 75 is already newer than 73 2024-09-13T10:11:06.961 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 75 (ObjNum 140 snap 0 seq_num 140) dirty exists 2024-09-13T10:11:06.961 INFO:tasks.rados.rados.0.smithi195.stdout:898: left oid 28 (ObjNum 140 snap 0 seq_num 140) 2024-09-13T10:11:06.961 INFO:tasks.rados.rados.0.smithi195.stdout:900: got (0) Success 2024-09-13T10:11:06.961 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 69 (ObjNum 121 snap 0 seq_num 121) dirty exists 2024-09-13T10:11:06.961 INFO:tasks.rados.rados.0.smithi195.stdout:901: expect (ObjNum 84 snap 0 seq_num 84) 2024-09-13T10:11:07.030 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:07.027+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:07.051 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:11:07.067 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:07.067 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:07.068 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:11:07.068 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:07.082 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T10:11:07.130 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:07.129+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:07.151 INFO:tasks.rados.rados.0.smithi195.stdout:902: expect (ObjNum 95 snap 0 seq_num 95) 2024-09-13T10:11:07.204 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:07.204 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:07.204 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:11:07.204 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:11:07.204 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:07.206 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:07.207 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:07.211 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:07.218 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T10:11:07.227 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:07.228 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:07.231 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:07.227+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:07.249 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:07.249 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:07.271 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:07.271 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:07.271 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:07.291 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:07.311 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:07.311 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:07.330 INFO:tasks.rados.rados.0.smithi195.stdout:904: expect (ObjNum 80 snap 0 seq_num 80) 2024-09-13T10:11:07.331 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:07.332 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:07.329+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:07.334 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:07.335 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:07.342 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:07.343 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:07.357 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T10:11:07.357 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:07.358 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:07.377 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:07.378 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:07.378 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2312135 2024-09-13T10:11:07.401 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:07.401 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:07.423 INFO:tasks.rados.rados.0.smithi195.stdout:905: finishing write tid 1 to smithi19530714-41 2024-09-13T10:11:07.423 INFO:tasks.rados.rados.0.smithi195.stdout:905: finishing write tid 2 to smithi19530714-41 2024-09-13T10:11:07.423 INFO:tasks.rados.rados.0.smithi195.stdout:905: finishing write tid 3 to smithi19530714-41 2024-09-13T10:11:07.424 INFO:tasks.rados.rados.0.smithi195.stdout:905: finishing write tid 4 to smithi19530714-41 2024-09-13T10:11:07.424 INFO:tasks.rados.rados.0.smithi195.stdout:905: oid 41 updating version 0 to 30 2024-09-13T10:11:07.424 INFO:tasks.rados.rados.0.smithi195.stdout:905: oid 41 updating version 30 to 32 2024-09-13T10:11:07.424 INFO:tasks.rados.rados.0.smithi195.stdout:905: oid 41 version 32 is already newer than 31 2024-09-13T10:11:07.424 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 32 (ObjNum 141 snap 0 seq_num 141) dirty exists 2024-09-13T10:11:07.424 INFO:tasks.rados.rados.0.smithi195.stdout:905: left oid 41 (ObjNum 141 snap 0 seq_num 141) 2024-09-13T10:11:07.424 INFO:tasks.rados.rados.0.smithi195.stdout:899: expect (ObjNum 93 snap 0 seq_num 93) 2024-09-13T10:11:07.424 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:07.425 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:07.429 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1150118 2024-09-13T10:11:07.432 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:07.429+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:07.445 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:07.445 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:07.453 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:07.463 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:07.463 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:07.482 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:07.482 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:07.488 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:07.488 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:07.488 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 84355 2024-09-13T10:11:07.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:07.489+0000 7fbaedd9b640 0 -- 172.21.15.73:0/686860680 >> v2:172.21.15.181:6804/1173379175 conn(0x7fbaf0221740 msgr2=0x7fbaf0223b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:11:07.496 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T10:11:07.523 INFO:tasks.rados.rados.0.smithi195.stdout:906: expect (ObjNum 102 snap 0 seq_num 102) 2024-09-13T10:11:07.533 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:07.532+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:07.632 INFO:tasks.rados.rados.0.smithi195.stdout:907: finishing write tid 1 to smithi19530714-1 2024-09-13T10:11:07.632 INFO:tasks.rados.rados.0.smithi195.stdout:907: finishing write tid 2 to smithi19530714-1 2024-09-13T10:11:07.632 INFO:tasks.rados.rados.0.smithi195.stdout:907: finishing write tid 3 to smithi19530714-1 2024-09-13T10:11:07.632 INFO:tasks.rados.rados.0.smithi195.stdout:907: finishing write tid 4 to smithi19530714-1 2024-09-13T10:11:07.633 INFO:tasks.rados.rados.0.smithi195.stdout:907: oid 1 updating version 0 to 44 2024-09-13T10:11:07.633 INFO:tasks.rados.rados.0.smithi195.stdout:907: oid 1 updating version 44 to 46 2024-09-13T10:11:07.633 INFO:tasks.rados.rados.0.smithi195.stdout:907: oid 1 version 46 is already newer than 45 2024-09-13T10:11:07.633 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 46 (ObjNum 142 snap 0 seq_num 142) dirty exists 2024-09-13T10:11:07.633 INFO:tasks.rados.rados.0.smithi195.stdout:907: left oid 1 (ObjNum 142 snap 0 seq_num 142) 2024-09-13T10:11:07.633 INFO:tasks.rados.rados.0.smithi195.stdout:908: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:07.635 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:07.632+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:07.653 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:07.662 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:07.662 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:07.674 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:07.676 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T10:11:07.697 INFO:tasks.rados.rados.0.smithi195.stdout:895: done (15 left) 2024-09-13T10:11:07.697 INFO:tasks.rados.rados.0.smithi195.stdout:896: done (14 left) 2024-09-13T10:11:07.697 INFO:tasks.rados.rados.0.smithi195.stdout:897: done (13 left) 2024-09-13T10:11:07.698 INFO:tasks.rados.rados.0.smithi195.stdout:898: done (12 left) 2024-09-13T10:11:07.698 INFO:tasks.rados.rados.0.smithi195.stdout:899: done (11 left) 2024-09-13T10:11:07.698 INFO:tasks.rados.rados.0.smithi195.stdout:900: done (10 left) 2024-09-13T10:11:07.698 INFO:tasks.rados.rados.0.smithi195.stdout:901: done (9 left) 2024-09-13T10:11:07.698 INFO:tasks.rados.rados.0.smithi195.stdout:902: done (8 left) 2024-09-13T10:11:07.698 INFO:tasks.rados.rados.0.smithi195.stdout:903: done (7 left) 2024-09-13T10:11:07.698 INFO:tasks.rados.rados.0.smithi195.stdout:904: done (6 left) 2024-09-13T10:11:07.698 INFO:tasks.rados.rados.0.smithi195.stdout:905: done (5 left) 2024-09-13T10:11:07.699 INFO:tasks.rados.rados.0.smithi195.stdout:906: done (4 left) 2024-09-13T10:11:07.699 INFO:tasks.rados.rados.0.smithi195.stdout:907: done (3 left) 2024-09-13T10:11:07.699 INFO:tasks.rados.rados.0.smithi195.stdout:908: done (2 left) 2024-09-13T10:11:07.699 INFO:tasks.rados.rados.0.smithi195.stdout:910: read oid 1 snap -1 2024-09-13T10:11:07.699 INFO:tasks.rados.rados.0.smithi195.stdout:910: expect (ObjNum 142 snap 0 seq_num 142) 2024-09-13T10:11:07.699 INFO:tasks.rados.rados.0.smithi195.stdout:911: read oid 43 snap -1 2024-09-13T10:11:07.699 INFO:tasks.rados.rados.0.smithi195.stdout:911: expect (ObjNum 136 snap 0 seq_num 136) 2024-09-13T10:11:07.699 INFO:tasks.rados.rados.0.smithi195.stdout:912: delete oid 28 current snap is 0 2024-09-13T10:11:07.705 INFO:tasks.rados.rados.0.smithi195.stdout:912: done (4 left) 2024-09-13T10:11:07.705 INFO:tasks.rados.rados.0.smithi195.stdout:913: set_chunk oid 29 target oid 2024-09-13T10:11:07.705 INFO:tasks.rados.rados.0.smithi195.stdout:913: set_chunk oid 29 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:07.705 INFO:tasks.rados.rados.0.smithi195.stdout:914: tier_promote oid 3 2024-09-13T10:11:07.705 INFO:tasks.rados.rados.0.smithi195.stdout:915: read oid 20 snap -1 2024-09-13T10:11:07.705 INFO:tasks.rados.rados.0.smithi195.stdout:915: expect (ObjNum 135 snap 0 seq_num 135) 2024-09-13T10:11:07.705 INFO:tasks.rados.rados.0.smithi195.stdout:916: read oid 26 snap -1 2024-09-13T10:11:07.705 INFO:tasks.rados.rados.0.smithi195.stdout:916: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:07.705 INFO:tasks.rados.rados.0.smithi195.stdout:917: read oid 24 snap -1 2024-09-13T10:11:07.705 INFO:tasks.rados.rados.0.smithi195.stdout:917: expect (ObjNum 121 snap 0 seq_num 121) 2024-09-13T10:11:07.706 INFO:tasks.rados.rados.0.smithi195.stdout:918: delete oid 21 current snap is 0 2024-09-13T10:11:07.708 INFO:tasks.rados.rados.0.smithi195.stdout:910: expect (ObjNum 142 snap 0 seq_num 142) 2024-09-13T10:11:07.708 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:07.724 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:07.725 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:07.736 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:07.733+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:07.755 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:07.755 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:07.769 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1531291 2024-09-13T10:11:07.796 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:07.799 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:07.799 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:07.814 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-13T10:11:07.836 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:07.835+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:07.840 INFO:tasks.rados.rados.0.smithi195.stdout:911: expect (ObjNum 136 snap 0 seq_num 136) 2024-09-13T10:11:07.862 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2172400 2024-09-13T10:11:07.905 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1154338 2024-09-13T10:11:07.923 INFO:tasks.rados.rados.0.smithi195.stdout:914: got (2) No such file or directory 2024-09-13T10:11:07.923 INFO:tasks.rados.rados.0.smithi195.stdout:914: got expected ENOENT (src dne) 2024-09-13T10:11:07.923 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 45 (ObjNum 257 snap 0 seq_num 643584448) clean dne 2024-09-13T10:11:07.923 INFO:tasks.rados.rados.0.smithi195.stdout:916: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:07.935 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:07.936 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:07.937 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:07.937+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:07.954 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-13T10:11:07.954 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1808284 2024-09-13T10:11:07.955 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 219118 2024-09-13T10:11:07.976 INFO:tasks.rados.rados.0.smithi195.stdout:909: got (0) Success 2024-09-13T10:11:07.976 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 61 (ObjNum 113 snap 0 seq_num 113) dirty exists 2024-09-13T10:11:07.976 INFO:tasks.rados.rados.0.smithi195.stdout:913: finishing set_chunk to oid 29 2024-09-13T10:11:07.976 INFO:tasks.rados.rados.0.smithi195.stdout:913: got expected ENOENT (src dne) 2024-09-13T10:11:07.977 INFO:tasks.rados.rados.0.smithi195.stdout:917: expect (ObjNum 121 snap 0 seq_num 121) 2024-09-13T10:11:08.038 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:08.037+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:08.050 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 822282 2024-09-13T10:11:08.076 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:08.077 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:08.077 INFO:tasks.rados.rados.0.smithi195.stdout:915: expect (ObjNum 135 snap 0 seq_num 135) 2024-09-13T10:11:08.077 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:08.093 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-13T10:11:08.139 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:08.137+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:08.162 INFO:tasks.rados.rados.0.smithi195.stdout:883: finishing write tid 1 to smithi19530714-50 2024-09-13T10:11:08.163 INFO:tasks.rados.rados.0.smithi195.stdout:883: finishing write tid 2 to smithi19530714-50 2024-09-13T10:11:08.163 INFO:tasks.rados.rados.0.smithi195.stdout:883: finishing write tid 3 to smithi19530714-50 2024-09-13T10:11:08.163 INFO:tasks.rados.rados.0.smithi195.stdout:883: finishing write tid 4 to smithi19530714-50 2024-09-13T10:11:08.163 INFO:tasks.rados.rados.0.smithi195.stdout:883: oid 50 updating version 0 to 25 2024-09-13T10:11:08.163 INFO:tasks.rados.rados.0.smithi195.stdout:883: oid 50 version 25 is already newer than 24 2024-09-13T10:11:08.163 INFO:tasks.rados.rados.0.smithi195.stdout:883: oid 50 updating version 25 to 26 2024-09-13T10:11:08.163 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 26 (ObjNum 134 snap 0 seq_num 134) dirty exists 2024-09-13T10:11:08.163 INFO:tasks.rados.rados.0.smithi195.stdout:883: left oid 50 (ObjNum 134 snap 0 seq_num 134) 2024-09-13T10:11:08.163 INFO:tasks.rados.rados.0.smithi195.stdout:883: done (9 left) 2024-09-13T10:11:08.163 INFO:tasks.rados.rados.0.smithi195.stdout:909: done (8 left) 2024-09-13T10:11:08.163 INFO:tasks.rados.rados.0.smithi195.stdout:910: done (7 left) 2024-09-13T10:11:08.164 INFO:tasks.rados.rados.0.smithi195.stdout:911: done (6 left) 2024-09-13T10:11:08.164 INFO:tasks.rados.rados.0.smithi195.stdout:913: done (5 left) 2024-09-13T10:11:08.164 INFO:tasks.rados.rados.0.smithi195.stdout:914: done (4 left) 2024-09-13T10:11:08.164 INFO:tasks.rados.rados.0.smithi195.stdout:915: done (3 left) 2024-09-13T10:11:08.164 INFO:tasks.rados.rados.0.smithi195.stdout:916: done (2 left) 2024-09-13T10:11:08.164 INFO:tasks.rados.rados.0.smithi195.stdout:917: done (1 left) 2024-09-13T10:11:08.164 INFO:tasks.rados.rados.0.smithi195.stdout:918: done (0 left) 2024-09-13T10:11:08.164 INFO:tasks.rados.rados.0.smithi195.stdout:919: write oid 37 current snap is 0 2024-09-13T10:11:08.164 INFO:tasks.rados.rados.0.smithi195.stdout:919: seq_num 145 ranges {437015=721812,1701889=509708} 2024-09-13T10:11:08.181 INFO:tasks.rados.rados.0.smithi195.stdout:919: writing smithi19530714-37 from 437015 to 1158827 tid 1 2024-09-13T10:11:08.199 INFO:tasks.rados.rados.0.smithi195.stdout:919: writing smithi19530714-37 from 1701889 to 2211597 tid 2 2024-09-13T10:11:08.199 INFO:tasks.rados.rados.0.smithi195.stdout:920: tier_evict oid 24 2024-09-13T10:11:08.200 INFO:tasks.rados.rados.0.smithi195.stdout:920: tier_evict oid 24 snap -1 2024-09-13T10:11:08.200 INFO:tasks.rados.rados.0.smithi195.stdout:919: finishing write tid 1 to smithi19530714-37 2024-09-13T10:11:08.200 INFO:tasks.rados.rados.0.smithi195.stdout:921: read oid 8 snap -1 2024-09-13T10:11:08.200 INFO:tasks.rados.rados.0.smithi195.stdout:921: expect (ObjNum 104 snap 0 seq_num 104) 2024-09-13T10:11:08.200 INFO:tasks.rados.rados.0.smithi195.stdout:922: read oid 15 snap -1 2024-09-13T10:11:08.200 INFO:tasks.rados.rados.0.smithi195.stdout:922: expect (ObjNum 89 snap 0 seq_num 89) 2024-09-13T10:11:08.200 INFO:tasks.rados.rados.0.smithi195.stdout:923: tier_evict oid 42 2024-09-13T10:11:08.200 INFO:tasks.rados.rados.0.smithi195.stdout:923: tier_evict oid 42 snap -1 2024-09-13T10:11:08.200 INFO:tasks.rados.rados.0.smithi195.stdout:924: set_chunk oid 34 target oid 2024-09-13T10:11:08.219 INFO:tasks.rados.rados.0.smithi195.stdout:924: set_chunk oid 34 offset: 486400 length: 834048 target oid 0db31492bb208c9a13420210767d73aad887e168b082d6694ba610b4b1e13756 offset: 0 snap -1 2024-09-13T10:11:08.219 INFO:tasks.rados.rados.0.smithi195.stdout:925: read oid 10 snap -1 2024-09-13T10:11:08.219 INFO:tasks.rados.rados.0.smithi195.stdout:925: expect (ObjNum 84 snap 0 seq_num 84) 2024-09-13T10:11:08.219 INFO:tasks.rados.rados.0.smithi195.stdout:926: write (excl) oid 50 current snap is 0 2024-09-13T10:11:08.219 INFO:tasks.rados.rados.0.smithi195.stdout:926: seq_num 146 ranges {453822=797061,1760371=653366} 2024-09-13T10:11:08.224 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:08.225 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:08.238 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-13T10:11:08.240 INFO:tasks.rados.rados.0.smithi195.stdout:926: writing smithi19530714-50 from 453822 to 1250883 tid 1 2024-09-13T10:11:08.240 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:08.239+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:08.257 INFO:tasks.rados.rados.0.smithi195.stdout:926: writing smithi19530714-50 from 1760371 to 2413737 tid 2 2024-09-13T10:11:08.258 INFO:tasks.rados.rados.0.smithi195.stdout:927: read oid 14 snap -1 2024-09-13T10:11:08.258 INFO:tasks.rados.rados.0.smithi195.stdout:927: expect (ObjNum 103 snap 0 seq_num 103) 2024-09-13T10:11:08.258 INFO:tasks.rados.rados.0.smithi195.stdout:928: set_chunk oid 28 target oid 2024-09-13T10:11:08.258 INFO:tasks.rados.rados.0.smithi195.stdout:928: set_chunk oid 28 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:08.258 INFO:tasks.rados.rados.0.smithi195.stdout:929: write (excl) oid 6 current snap is 0 2024-09-13T10:11:08.258 INFO:tasks.rados.rados.0.smithi195.stdout:920: got (22) Invalid argument 2024-09-13T10:11:08.258 INFO:tasks.rados.rados.0.smithi195.stdout:919: finishing write tid 2 to smithi19530714-37 2024-09-13T10:11:08.258 INFO:tasks.rados.rados.0.smithi195.stdout:919: finishing write tid 3 to smithi19530714-37 2024-09-13T10:11:08.258 INFO:tasks.rados.rados.0.smithi195.stdout:919: finishing write tid 4 to smithi19530714-37 2024-09-13T10:11:08.258 INFO:tasks.rados.rados.0.smithi195.stdout:919: oid 37 updating version 0 to 16 2024-09-13T10:11:08.258 INFO:tasks.rados.rados.0.smithi195.stdout:919: oid 37 updating version 16 to 18 2024-09-13T10:11:08.258 INFO:tasks.rados.rados.0.smithi195.stdout:919: oid 37 version 18 is already newer than 17 2024-09-13T10:11:08.259 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 18 (ObjNum 145 snap 0 seq_num 145) dirty exists 2024-09-13T10:11:08.259 INFO:tasks.rados.rados.0.smithi195.stdout:919: left oid 37 (ObjNum 145 snap 0 seq_num 145) 2024-09-13T10:11:08.259 INFO:tasks.rados.rados.0.smithi195.stdout:921: expect (ObjNum 104 snap 0 seq_num 104) 2024-09-13T10:11:08.259 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:08.342 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:08.340+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:08.403 INFO:tasks.rados.rados.0.smithi195.stdout:922: expect (ObjNum 89 snap 0 seq_num 89) 2024-09-13T10:11:08.403 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:08.403 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:08.413 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T10:11:08.418 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-13T10:11:08.449 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 167136 2024-09-13T10:11:08.457 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:08.457+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:08.495 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:08.539 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:08.540 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:08.544 INFO:tasks.rados.rados.0.smithi195.stdout:924: finishing set_chunk to oid 34 2024-09-13T10:11:08.544 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 34 v 22 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-09-13T10:11:08.544 INFO:tasks.rados.rados.0.smithi195.stdout:925: expect (ObjNum 84 snap 0 seq_num 84) 2024-09-13T10:11:08.544 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:08.541+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:08.555 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-13T10:11:08.602 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:08.645 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:08.643+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:08.659 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:08.716 INFO:tasks.rados.rados.0.smithi195.stdout:926: finishing write tid 1 to smithi19530714-50 2024-09-13T10:11:08.717 INFO:tasks.rados.rados.0.smithi195.stdout:926: finishing write tid 2 to smithi19530714-50 2024-09-13T10:11:08.717 INFO:tasks.rados.rados.0.smithi195.stdout:926: finishing write tid 3 to smithi19530714-50 2024-09-13T10:11:08.717 INFO:tasks.rados.rados.0.smithi195.stdout:926: finishing write tid 4 to smithi19530714-50 2024-09-13T10:11:08.717 INFO:tasks.rados.rados.0.smithi195.stdout:926: oid 50 updating version 0 to 27 2024-09-13T10:11:08.717 INFO:tasks.rados.rados.0.smithi195.stdout:926: oid 50 updating version 27 to 28 2024-09-13T10:11:08.717 INFO:tasks.rados.rados.0.smithi195.stdout:926: oid 50 updating version 28 to 29 2024-09-13T10:11:08.717 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 29 (ObjNum 146 snap 0 seq_num 146) dirty exists 2024-09-13T10:11:08.717 INFO:tasks.rados.rados.0.smithi195.stdout:926: left oid 50 (ObjNum 146 snap 0 seq_num 146) 2024-09-13T10:11:08.717 INFO:tasks.rados.rados.0.smithi195.stdout:928: finishing set_chunk to oid 28 2024-09-13T10:11:08.717 INFO:tasks.rados.rados.0.smithi195.stdout:928: got expected ENOENT (src dne) 2024-09-13T10:11:08.717 INFO:tasks.rados.rados.0.smithi195.stdout:927: expect (ObjNum 103 snap 0 seq_num 103) 2024-09-13T10:11:08.718 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:08.720 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:08.721 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:08.736 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T10:11:08.780 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:08.780+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:08.846 INFO:tasks.rados.rados.0.smithi195.stdout:923: got (22) Invalid argument 2024-09-13T10:11:08.846 INFO:tasks.rados.rados.0.smithi195.stdout:929: seq_num 147 ranges {470629=472310,1418853=513377} 2024-09-13T10:11:08.846 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:08.844+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:08.861 INFO:tasks.rados.rados.0.smithi195.stdout:929: writing smithi19530714-6 from 470629 to 942939 tid 1 2024-09-13T10:11:08.867 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:08.867 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:08.878 INFO:tasks.rados.rados.0.smithi195.stdout:929: writing smithi19530714-6 from 1418853 to 1932230 tid 2 2024-09-13T10:11:08.879 INFO:tasks.rados.rados.0.smithi195.stdout:919: done (10 left) 2024-09-13T10:11:08.879 INFO:tasks.rados.rados.0.smithi195.stdout:920: done (9 left) 2024-09-13T10:11:08.879 INFO:tasks.rados.rados.0.smithi195.stdout:921: done (8 left) 2024-09-13T10:11:08.879 INFO:tasks.rados.rados.0.smithi195.stdout:922: done (7 left) 2024-09-13T10:11:08.879 INFO:tasks.rados.rados.0.smithi195.stdout:923: done (6 left) 2024-09-13T10:11:08.879 INFO:tasks.rados.rados.0.smithi195.stdout:924: done (5 left) 2024-09-13T10:11:08.879 INFO:tasks.rados.rados.0.smithi195.stdout:925: done (4 left) 2024-09-13T10:11:08.879 INFO:tasks.rados.rados.0.smithi195.stdout:926: done (3 left) 2024-09-13T10:11:08.879 INFO:tasks.rados.rados.0.smithi195.stdout:927: done (2 left) 2024-09-13T10:11:08.879 INFO:tasks.rados.rados.0.smithi195.stdout:928: done (1 left) 2024-09-13T10:11:08.879 INFO:tasks.rados.rados.0.smithi195.stdout:930: tier_flush oid 24 2024-09-13T10:11:08.879 INFO:tasks.rados.rados.0.smithi195.stdout:930: tier_flush oid 24 snap -1 2024-09-13T10:11:08.879 INFO:tasks.rados.rados.0.smithi195.stdout:929: finishing write tid 1 to smithi19530714-6 2024-09-13T10:11:08.880 INFO:tasks.rados.rados.0.smithi195.stdout:931: tier_promote oid 49 2024-09-13T10:11:08.880 INFO:tasks.rados.rados.0.smithi195.stdout:932: write (excl) oid 34 current snap is 0 2024-09-13T10:11:08.880 INFO:tasks.rados.rados.0.smithi195.stdout:932: seq_num 148 ranges {487436=547559,1760982=657035} 2024-09-13T10:11:08.881 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T10:11:08.894 INFO:tasks.rados.rados.0.smithi195.stdout:932: writing smithi19530714-34 from 487436 to 1034995 tid 1 2024-09-13T10:11:08.913 INFO:tasks.rados.rados.0.smithi195.stdout:932: writing smithi19530714-34 from 1760982 to 2418017 tid 2 2024-09-13T10:11:08.914 INFO:tasks.rados.rados.0.smithi195.stdout:933: read oid 47 snap -1 2024-09-13T10:11:08.914 INFO:tasks.rados.rados.0.smithi195.stdout:933: expect deleted 2024-09-13T10:11:08.914 INFO:tasks.rados.rados.0.smithi195.stdout:934: read oid 22 snap -1 2024-09-13T10:11:08.914 INFO:tasks.rados.rados.0.smithi195.stdout:934: expect (ObjNum 139 snap 0 seq_num 139) 2024-09-13T10:11:08.914 INFO:tasks.rados.rados.0.smithi195.stdout:935: write (excl) oid 43 current snap is 0 2024-09-13T10:11:08.914 INFO:tasks.rados.rados.0.smithi195.stdout:935: seq_num 149 ranges {504243=622808,1819464=517046} 2024-09-13T10:11:08.924 INFO:tasks.rados.rados.0.smithi195.stdout:935: writing smithi19530714-43 from 504243 to 1127051 tid 1 2024-09-13T10:11:08.934 INFO:tasks.rados.rados.0.smithi195.stdout:935: writing smithi19530714-43 from 1819464 to 2336510 tid 2 2024-09-13T10:11:08.934 INFO:tasks.rados.rados.0.smithi195.stdout:936: set_chunk oid 2 target oid 2024-09-13T10:11:08.947 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:08.943+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:08.949 INFO:tasks.rados.rados.0.smithi195.stdout:936: set_chunk oid 2 offset: 325120 length: 896000 target oid 5dda4d3a7232b558d81b82089efa777a9df288c65ea7099fda11c3b5b6442ca9 offset: 0 snap -1 2024-09-13T10:11:08.949 INFO:tasks.rados.rados.0.smithi195.stdout:937: read oid 18 snap -1 2024-09-13T10:11:08.949 INFO:tasks.rados.rados.0.smithi195.stdout:937: expect (ObjNum 130 snap 0 seq_num 130) 2024-09-13T10:11:08.949 INFO:tasks.rados.rados.0.smithi195.stdout:929: finishing write tid 2 to smithi19530714-6 2024-09-13T10:11:08.949 INFO:tasks.rados.rados.0.smithi195.stdout:929: finishing write tid 3 to smithi19530714-6 2024-09-13T10:11:08.949 INFO:tasks.rados.rados.0.smithi195.stdout:929: finishing write tid 4 to smithi19530714-6 2024-09-13T10:11:08.949 INFO:tasks.rados.rados.0.smithi195.stdout:929: oid 6 updating version 0 to 46 2024-09-13T10:11:08.949 INFO:tasks.rados.rados.0.smithi195.stdout:929: oid 6 updating version 46 to 47 2024-09-13T10:11:08.950 INFO:tasks.rados.rados.0.smithi195.stdout:929: oid 6 version 47 is already newer than 45 2024-09-13T10:11:08.950 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 47 (ObjNum 147 snap 0 seq_num 147) dirty exists 2024-09-13T10:11:08.950 INFO:tasks.rados.rados.0.smithi195.stdout:929: left oid 6 (ObjNum 147 snap 0 seq_num 147) 2024-09-13T10:11:08.950 INFO:tasks.rados.rados.0.smithi195.stdout:931: got (0) Success 2024-09-13T10:11:08.950 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 72 (ObjNum 133 snap 0 seq_num 133) dirty exists 2024-09-13T10:11:08.950 INFO:tasks.rados.rados.0.smithi195.stdout:932: finishing write tid 1 to smithi19530714-34 2024-09-13T10:11:08.950 INFO:tasks.rados.rados.0.smithi195.stdout:938: read oid 12 snap -1 2024-09-13T10:11:08.950 INFO:tasks.rados.rados.0.smithi195.stdout:938: expect (ObjNum 85 snap 0 seq_num 85) 2024-09-13T10:11:08.950 INFO:tasks.rados.rados.0.smithi195.stdout:929: done (9 left) 2024-09-13T10:11:08.950 INFO:tasks.rados.rados.0.smithi195.stdout:931: done (8 left) 2024-09-13T10:11:08.950 INFO:tasks.rados.rados.0.smithi195.stdout:933: done (7 left) 2024-09-13T10:11:08.950 INFO:tasks.rados.rados.0.smithi195.stdout:939: set_chunk oid 16 target oid 2024-09-13T10:11:08.972 INFO:tasks.rados.rados.0.smithi195.stdout:939: set_chunk oid 16 offset: 995840 length: 1133056 target oid 82d6915d813d56160c7da3e2d2566b78db3ea210faa8d7246d4cd9a8e454a831 offset: 0 snap -1 2024-09-13T10:11:08.973 INFO:tasks.rados.rados.0.smithi195.stdout:940: read oid 7 snap -1 2024-09-13T10:11:08.973 INFO:tasks.rados.rados.0.smithi195.stdout:940: expect (ObjNum 132 snap 0 seq_num 132) 2024-09-13T10:11:08.973 INFO:tasks.rados.rados.0.smithi195.stdout:941: read oid 46 snap -1 2024-09-13T10:11:08.973 INFO:tasks.rados.rados.0.smithi195.stdout:941: expect deleted 2024-09-13T10:11:08.973 INFO:tasks.rados.rados.0.smithi195.stdout:942: set_chunk oid 32 target oid 2024-09-13T10:11:09.002 INFO:tasks.rados.rados.0.smithi195.stdout:942: set_chunk oid 32 offset: 1024 length: 1691136 target oid 6d11ab07b1d54313e1e6cdf8584784ea22a8354083b19a82a00aa3a441eede40 offset: 0 snap -1 2024-09-13T10:11:09.002 INFO:tasks.rados.rados.0.smithi195.stdout:943: write oid 47 current snap is 0 2024-09-13T10:11:09.003 INFO:tasks.rados.rados.0.smithi195.stdout:930: got (0) Success 2024-09-13T10:11:09.003 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 0 (ObjNum 121 snap 0 seq_num 121) dirty exists 2024-09-13T10:11:09.003 INFO:tasks.rados.rados.0.smithi195.stdout:932: finishing write tid 2 to smithi19530714-34 2024-09-13T10:11:09.003 INFO:tasks.rados.rados.0.smithi195.stdout:932: finishing write tid 3 to smithi19530714-34 2024-09-13T10:11:09.003 INFO:tasks.rados.rados.0.smithi195.stdout:943: seq_num 150 ranges {521050=698057,1877946=643104} 2024-09-13T10:11:09.022 INFO:tasks.rados.rados.0.smithi195.stdout:943: writing smithi19530714-47 from 521050 to 1219107 tid 1 2024-09-13T10:11:09.040 INFO:tasks.rados.rados.0.smithi195.stdout:943: writing smithi19530714-47 from 1877946 to 2521050 tid 2 2024-09-13T10:11:09.040 INFO:tasks.rados.rados.0.smithi195.stdout:930: done (11 left) 2024-09-13T10:11:09.040 INFO:tasks.rados.rados.0.smithi195.stdout:944: tier_promote oid 4 2024-09-13T10:11:09.040 INFO:tasks.rados.rados.0.smithi195.stdout:932: finishing write tid 4 to smithi19530714-34 2024-09-13T10:11:09.040 INFO:tasks.rados.rados.0.smithi195.stdout:932: oid 34 updating version 0 to 42 2024-09-13T10:11:09.041 INFO:tasks.rados.rados.0.smithi195.stdout:932: oid 34 version 42 is already newer than 41 2024-09-13T10:11:09.041 INFO:tasks.rados.rados.0.smithi195.stdout:932: oid 34 updating version 42 to 43 2024-09-13T10:11:09.041 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 34 v 43 (ObjNum 148 snap 0 seq_num 148) dirty exists 2024-09-13T10:11:09.041 INFO:tasks.rados.rados.0.smithi195.stdout:932: left oid 34 (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:09.041 INFO:tasks.rados.rados.0.smithi195.stdout:934: expect (ObjNum 139 snap 0 seq_num 139) 2024-09-13T10:11:09.043 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:09.043 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:09.047 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:09.045+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:09.058 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T10:11:09.148 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:09.148+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:09.181 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:09.181 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:09.194 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T10:11:09.229 INFO:tasks.rados.rados.0.smithi195.stdout:935: finishing write tid 1 to smithi19530714-43 2024-09-13T10:11:09.229 INFO:tasks.rados.rados.0.smithi195.stdout:935: finishing write tid 2 to smithi19530714-43 2024-09-13T10:11:09.229 INFO:tasks.rados.rados.0.smithi195.stdout:935: finishing write tid 3 to smithi19530714-43 2024-09-13T10:11:09.229 INFO:tasks.rados.rados.0.smithi195.stdout:935: finishing write tid 4 to smithi19530714-43 2024-09-13T10:11:09.229 INFO:tasks.rados.rados.0.smithi195.stdout:935: oid 43 updating version 0 to 49 2024-09-13T10:11:09.229 INFO:tasks.rados.rados.0.smithi195.stdout:935: oid 43 version 49 is already newer than 48 2024-09-13T10:11:09.229 INFO:tasks.rados.rados.0.smithi195.stdout:935: oid 43 version 49 is already newer than 47 2024-09-13T10:11:09.230 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 49 (ObjNum 149 snap 0 seq_num 149) dirty exists 2024-09-13T10:11:09.230 INFO:tasks.rados.rados.0.smithi195.stdout:935: left oid 43 (ObjNum 149 snap 0 seq_num 149) 2024-09-13T10:11:09.230 INFO:tasks.rados.rados.0.smithi195.stdout:936: finishing set_chunk to oid 2 2024-09-13T10:11:09.230 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 2 set_chunk returned error code -95 offset: 325120 length: 896000 2024-09-13T10:11:09.230 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 0 (ObjNum 68 snap 0 seq_num 68) dirty exists 2024-09-13T10:11:09.230 INFO:tasks.rados.rados.0.smithi195.stdout:938: expect (ObjNum 85 snap 0 seq_num 85) 2024-09-13T10:11:09.249 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:09.245+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:09.338 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1318120 2024-09-13T10:11:09.350 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:09.347+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:09.358 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:09.358 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:09.372 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T10:11:09.373 INFO:tasks.rados.rados.0.smithi195.stdout:937: expect (ObjNum 130 snap 0 seq_num 130) 2024-09-13T10:11:09.399 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2101836 2024-09-13T10:11:09.399 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:09.451 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:09.447+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:09.473 INFO:tasks.rados.rados.0.smithi195.stdout:939: finishing set_chunk to oid 16 2024-09-13T10:11:09.473 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 23 (ObjNum 128 snap 0 seq_num 128) dirty exists 2024-09-13T10:11:09.474 INFO:tasks.rados.rados.0.smithi195.stdout:940: expect (ObjNum 132 snap 0 seq_num 132) 2024-09-13T10:11:09.496 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:09.496 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:09.510 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T10:11:09.532 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:09.552 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:09.549+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:09.563 INFO:tasks.rados.rados.0.smithi195.stdout:942: finishing set_chunk to oid 32 2024-09-13T10:11:09.563 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 40 (ObjNum 131 snap 0 seq_num 131) dirty exists 2024-09-13T10:11:09.563 INFO:tasks.rados.rados.0.smithi195.stdout:943: finishing write tid 1 to smithi19530714-47 2024-09-13T10:11:09.563 INFO:tasks.rados.rados.0.smithi195.stdout:943: finishing write tid 2 to smithi19530714-47 2024-09-13T10:11:09.563 INFO:tasks.rados.rados.0.smithi195.stdout:943: finishing write tid 3 to smithi19530714-47 2024-09-13T10:11:09.564 INFO:tasks.rados.rados.0.smithi195.stdout:944: got (0) Success 2024-09-13T10:11:09.564 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 46 (ObjNum 115 snap 0 seq_num 115) dirty exists 2024-09-13T10:11:09.564 INFO:tasks.rados.rados.0.smithi195.stdout:943: finishing write tid 4 to smithi19530714-47 2024-09-13T10:11:09.564 INFO:tasks.rados.rados.0.smithi195.stdout:943: oid 47 updating version 0 to 35 2024-09-13T10:11:09.564 INFO:tasks.rados.rados.0.smithi195.stdout:943: oid 47 version 35 is already newer than 33 2024-09-13T10:11:09.564 INFO:tasks.rados.rados.0.smithi195.stdout:943: oid 47 version 35 is already newer than 34 2024-09-13T10:11:09.564 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 35 (ObjNum 150 snap 0 seq_num 150) dirty exists 2024-09-13T10:11:09.564 INFO:tasks.rados.rados.0.smithi195.stdout:943: left oid 47 (ObjNum 150 snap 0 seq_num 150) 2024-09-13T10:11:09.564 INFO:tasks.rados.rados.0.smithi195.stdout:932: done (11 left) 2024-09-13T10:11:09.564 INFO:tasks.rados.rados.0.smithi195.stdout:934: done (10 left) 2024-09-13T10:11:09.564 INFO:tasks.rados.rados.0.smithi195.stdout:935: done (9 left) 2024-09-13T10:11:09.564 INFO:tasks.rados.rados.0.smithi195.stdout:936: done (8 left) 2024-09-13T10:11:09.565 INFO:tasks.rados.rados.0.smithi195.stdout:937: done (7 left) 2024-09-13T10:11:09.565 INFO:tasks.rados.rados.0.smithi195.stdout:938: done (6 left) 2024-09-13T10:11:09.565 INFO:tasks.rados.rados.0.smithi195.stdout:939: done (5 left) 2024-09-13T10:11:09.565 INFO:tasks.rados.rados.0.smithi195.stdout:940: done (4 left) 2024-09-13T10:11:09.565 INFO:tasks.rados.rados.0.smithi195.stdout:941: done (3 left) 2024-09-13T10:11:09.565 INFO:tasks.rados.rados.0.smithi195.stdout:942: done (2 left) 2024-09-13T10:11:09.565 INFO:tasks.rados.rados.0.smithi195.stdout:943: done (1 left) 2024-09-13T10:11:09.565 INFO:tasks.rados.rados.0.smithi195.stdout:944: done (0 left) 2024-09-13T10:11:09.565 INFO:tasks.rados.rados.0.smithi195.stdout:945: write oid 21 current snap is 0 2024-09-13T10:11:09.565 INFO:tasks.rados.rados.0.smithi195.stdout:945: seq_num 151 ranges {537857=773306,1936428=520715} 2024-09-13T10:11:09.583 INFO:tasks.rados.rados.0.smithi195.stdout:945: writing smithi19530714-21 from 537857 to 1311163 tid 1 2024-09-13T10:11:09.602 INFO:tasks.rados.rados.0.smithi195.stdout:945: writing smithi19530714-21 from 1936428 to 2457143 tid 2 2024-09-13T10:11:09.602 INFO:tasks.rados.rados.0.smithi195.stdout:946: set_chunk oid 35 target oid 2024-09-13T10:11:09.632 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:09.632 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:09.645 INFO:tasks.rados.rados.0.smithi195.stdout:946: set_chunk oid 35 offset: 114176 length: 2637312 target oid b3f6c6b276ac77aba157e01e3316a117c79710df3e5ddd9249a93b4392d6c222 offset: 0 snap -1 2024-09-13T10:11:09.645 INFO:tasks.rados.rados.0.smithi195.stdout:947: write (excl) oid 18 current snap is 0 2024-09-13T10:11:09.645 INFO:tasks.rados.rados.0.smithi195.stdout:947: seq_num 152 ranges {554664=448555,1478557=664373} 2024-09-13T10:11:09.650 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-13T10:11:09.653 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:09.652+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:09.661 INFO:tasks.rados.rados.0.smithi195.stdout:947: writing smithi19530714-18 from 554664 to 1003219 tid 1 2024-09-13T10:11:09.679 INFO:tasks.rados.rados.0.smithi195.stdout:947: writing smithi19530714-18 from 1478557 to 2142930 tid 2 2024-09-13T10:11:09.679 INFO:tasks.rados.rados.0.smithi195.stdout:948: read oid 47 snap -1 2024-09-13T10:11:09.680 INFO:tasks.rados.rados.0.smithi195.stdout:948: expect (ObjNum 150 snap 0 seq_num 150) 2024-09-13T10:11:09.680 INFO:tasks.rados.rados.0.smithi195.stdout:949: write oid 17 current snap is 0 2024-09-13T10:11:09.680 INFO:tasks.rados.rados.0.smithi195.stdout:945: finishing write tid 1 to smithi19530714-21 2024-09-13T10:11:09.680 INFO:tasks.rados.rados.0.smithi195.stdout:945: finishing write tid 2 to smithi19530714-21 2024-09-13T10:11:09.680 INFO:tasks.rados.rados.0.smithi195.stdout:945: finishing write tid 3 to smithi19530714-21 2024-09-13T10:11:09.680 INFO:tasks.rados.rados.0.smithi195.stdout:949: seq_num 153 ranges {571471=640157,1653392=524384} 2024-09-13T10:11:09.695 INFO:tasks.rados.rados.0.smithi195.stdout:949: writing smithi19530714-17 from 571471 to 1211628 tid 1 2024-09-13T10:11:09.701 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-13T10:11:09.708 INFO:tasks.rados.rados.0.smithi195.stdout:949: writing smithi19530714-17 from 1653392 to 2177776 tid 2 2024-09-13T10:11:09.708 INFO:tasks.rados.rados.0.smithi195.stdout:950: read oid 27 snap -1 2024-09-13T10:11:09.708 INFO:tasks.rados.rados.0.smithi195.stdout:950: expect (ObjNum 138 snap 0 seq_num 138) 2024-09-13T10:11:09.708 INFO:tasks.rados.rados.0.smithi195.stdout:951: write (excl) oid 40 current snap is 0 2024-09-13T10:11:09.708 INFO:tasks.rados.rados.0.smithi195.stdout:945: finishing write tid 4 to smithi19530714-21 2024-09-13T10:11:09.708 INFO:tasks.rados.rados.0.smithi195.stdout:945: oid 21 updating version 0 to 85 2024-09-13T10:11:09.708 INFO:tasks.rados.rados.0.smithi195.stdout:945: oid 21 version 85 is already newer than 84 2024-09-13T10:11:09.708 INFO:tasks.rados.rados.0.smithi195.stdout:945: oid 21 version 85 is already newer than 83 2024-09-13T10:11:09.708 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 85 (ObjNum 151 snap 0 seq_num 151) dirty exists 2024-09-13T10:11:09.708 INFO:tasks.rados.rados.0.smithi195.stdout:945: left oid 21 (ObjNum 151 snap 0 seq_num 151) 2024-09-13T10:11:09.709 INFO:tasks.rados.rados.0.smithi195.stdout:946: finishing set_chunk to oid 35 2024-09-13T10:11:09.709 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 35 set_chunk returned error code -95 offset: 114176 length: 2637312 2024-09-13T10:11:09.709 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 0 (ObjNum 86 snap 0 seq_num 86) dirty exists 2024-09-13T10:11:09.709 INFO:tasks.rados.rados.0.smithi195.stdout:947: finishing write tid 1 to smithi19530714-18 2024-09-13T10:11:09.709 INFO:tasks.rados.rados.0.smithi195.stdout:951: seq_num 154 ranges {588278=715406,1711874=668042} 2024-09-13T10:11:09.723 INFO:tasks.rados.rados.0.smithi195.stdout:951: writing smithi19530714-40 from 588278 to 1303684 tid 1 2024-09-13T10:11:09.735 INFO:tasks.rados.rados.0.smithi195.stdout:951: writing smithi19530714-40 from 1711874 to 2379916 tid 2 2024-09-13T10:11:09.735 INFO:tasks.rados.rados.0.smithi195.stdout:945: done (6 left) 2024-09-13T10:11:09.735 INFO:tasks.rados.rados.0.smithi195.stdout:946: done (5 left) 2024-09-13T10:11:09.735 INFO:tasks.rados.rados.0.smithi195.stdout:952: write (excl) oid 25 current snap is 0 2024-09-13T10:11:09.735 INFO:tasks.rados.rados.0.smithi195.stdout:952: seq_num 155 ranges {605085=790655,2170356=434729} 2024-09-13T10:11:09.747 INFO:tasks.rados.rados.0.smithi195.stdout:952: writing smithi19530714-25 from 605085 to 1395740 tid 1 2024-09-13T10:11:09.754 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:09.751+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:09.757 INFO:tasks.rados.rados.0.smithi195.stdout:952: writing smithi19530714-25 from 2170356 to 2605085 tid 2 2024-09-13T10:11:09.757 INFO:tasks.rados.rados.0.smithi195.stdout:953: read oid 38 snap -1 2024-09-13T10:11:09.757 INFO:tasks.rados.rados.0.smithi195.stdout:953: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:11:09.757 INFO:tasks.rados.rados.0.smithi195.stdout:954: write (excl) oid 50 current snap is 0 2024-09-13T10:11:09.757 INFO:tasks.rados.rados.0.smithi195.stdout:954: seq_num 156 ranges {621892=465904,1712485=671711} 2024-09-13T10:11:09.764 INFO:tasks.rados.rados.0.smithi195.stdout:954: writing smithi19530714-50 from 621892 to 1087796 tid 1 2024-09-13T10:11:09.772 INFO:tasks.rados.rados.0.smithi195.stdout:954: writing smithi19530714-50 from 1712485 to 2384196 tid 2 2024-09-13T10:11:09.773 INFO:tasks.rados.rados.0.smithi195.stdout:955: tier_promote oid 21 2024-09-13T10:11:09.773 INFO:tasks.rados.rados.0.smithi195.stdout:948: expect (ObjNum 150 snap 0 seq_num 150) 2024-09-13T10:11:09.783 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:09.783 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:09.785 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:11:09.797 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-13T10:11:09.847 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 960757 2024-09-13T10:11:09.855 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:09.853+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:09.864 INFO:tasks.rados.rados.0.smithi195.stdout:947: finishing write tid 2 to smithi19530714-18 2024-09-13T10:11:09.865 INFO:tasks.rados.rados.0.smithi195.stdout:947: finishing write tid 3 to smithi19530714-18 2024-09-13T10:11:09.865 INFO:tasks.rados.rados.0.smithi195.stdout:947: finishing write tid 4 to smithi19530714-18 2024-09-13T10:11:09.865 INFO:tasks.rados.rados.0.smithi195.stdout:947: oid 18 updating version 0 to 59 2024-09-13T10:11:09.865 INFO:tasks.rados.rados.0.smithi195.stdout:947: oid 18 version 59 is already newer than 57 2024-09-13T10:11:09.865 INFO:tasks.rados.rados.0.smithi195.stdout:947: oid 18 version 59 is already newer than 58 2024-09-13T10:11:09.866 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 59 (ObjNum 152 snap 0 seq_num 152) dirty exists 2024-09-13T10:11:09.866 INFO:tasks.rados.rados.0.smithi195.stdout:947: left oid 18 (ObjNum 152 snap 0 seq_num 152) 2024-09-13T10:11:09.866 INFO:tasks.rados.rados.0.smithi195.stdout:949: finishing write tid 1 to smithi19530714-17 2024-09-13T10:11:09.866 INFO:tasks.rados.rados.0.smithi195.stdout:947: done (8 left) 2024-09-13T10:11:09.866 INFO:tasks.rados.rados.0.smithi195.stdout:948: done (7 left) 2024-09-13T10:11:09.866 INFO:tasks.rados.rados.0.smithi195.stdout:956: set_chunk oid 2 target oid 2024-09-13T10:11:09.877 INFO:tasks.rados.rados.0.smithi195.stdout:956: set_chunk oid 2 offset: 445952 length: 1115648 target oid 44b5e510be7066a66a302403cef87c58865d10248a80c02da59566dc7bd02031 offset: 0 snap -1 2024-09-13T10:11:09.878 INFO:tasks.rados.rados.0.smithi195.stdout:957: read oid 11 snap -1 2024-09-13T10:11:09.878 INFO:tasks.rados.rados.0.smithi195.stdout:957: expect (ObjNum 88 snap 0 seq_num 88) 2024-09-13T10:11:09.878 INFO:tasks.rados.rados.0.smithi195.stdout:958: write oid 6 current snap is 0 2024-09-13T10:11:09.878 INFO:tasks.rados.rados.0.smithi195.stdout:958: seq_num 157 ranges {638699=541153,1770967=531722} 2024-09-13T10:11:09.884 INFO:tasks.rados.rados.0.smithi195.stdout:958: writing smithi19530714-6 from 638699 to 1179852 tid 1 2024-09-13T10:11:09.891 INFO:tasks.rados.rados.0.smithi195.stdout:958: writing smithi19530714-6 from 1770967 to 2302689 tid 2 2024-09-13T10:11:09.891 INFO:tasks.rados.rados.0.smithi195.stdout:959: read oid 16 snap -1 2024-09-13T10:11:09.891 INFO:tasks.rados.rados.0.smithi195.stdout:959: expect (ObjNum 128 snap 0 seq_num 128) 2024-09-13T10:11:09.891 INFO:tasks.rados.rados.0.smithi195.stdout:960: write oid 45 current snap is 0 2024-09-13T10:11:09.891 INFO:tasks.rados.rados.0.smithi195.stdout:960: seq_num 158 ranges {655506=616402,1829449=675380} 2024-09-13T10:11:09.898 INFO:tasks.rados.rados.0.smithi195.stdout:960: writing smithi19530714-45 from 655506 to 1271908 tid 1 2024-09-13T10:11:09.905 INFO:tasks.rados.rados.0.smithi195.stdout:960: writing smithi19530714-45 from 1829449 to 2504829 tid 2 2024-09-13T10:11:09.905 INFO:tasks.rados.rados.0.smithi195.stdout:961: read oid 14 snap -1 2024-09-13T10:11:09.906 INFO:tasks.rados.rados.0.smithi195.stdout:961: expect (ObjNum 103 snap 0 seq_num 103) 2024-09-13T10:11:09.906 INFO:tasks.rados.rados.0.smithi195.stdout:962: read oid 33 snap -1 2024-09-13T10:11:09.906 INFO:tasks.rados.rados.0.smithi195.stdout:962: expect (ObjNum 137 snap 0 seq_num 137) 2024-09-13T10:11:09.906 INFO:tasks.rados.rados.0.smithi195.stdout:963: write (excl) oid 28 current snap is 0 2024-09-13T10:11:09.906 INFO:tasks.rados.rados.0.smithi195.stdout:949: finishing write tid 2 to smithi19530714-17 2024-09-13T10:11:09.906 INFO:tasks.rados.rados.0.smithi195.stdout:950: expect (ObjNum 138 snap 0 seq_num 138) 2024-09-13T10:11:09.923 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:09.923 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:09.927 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1668782 2024-09-13T10:11:09.938 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-13T10:11:09.943 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:09.943 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:09.964 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:09.964 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:09.982 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:09.982 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:09.983 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:09.984+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:09.994 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:10.002 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:10.002 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:10.019 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:10.020 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:10.039 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:10.039 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:10.040 INFO:tasks.rados.rados.0.smithi195.stdout:949: finishing write tid 3 to smithi19530714-17 2024-09-13T10:11:10.040 INFO:tasks.rados.rados.0.smithi195.stdout:949: finishing write tid 4 to smithi19530714-17 2024-09-13T10:11:10.040 INFO:tasks.rados.rados.0.smithi195.stdout:949: oid 17 updating version 0 to 43 2024-09-13T10:11:10.040 INFO:tasks.rados.rados.0.smithi195.stdout:949: oid 17 updating version 43 to 44 2024-09-13T10:11:10.040 INFO:tasks.rados.rados.0.smithi195.stdout:949: oid 17 version 44 is already newer than 42 2024-09-13T10:11:10.040 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 44 (ObjNum 153 snap 0 seq_num 153) dirty exists 2024-09-13T10:11:10.040 INFO:tasks.rados.rados.0.smithi195.stdout:949: left oid 17 (ObjNum 153 snap 0 seq_num 153) 2024-09-13T10:11:10.040 INFO:tasks.rados.rados.0.smithi195.stdout:951: finishing write tid 1 to smithi19530714-40 2024-09-13T10:11:10.040 INFO:tasks.rados.rados.0.smithi195.stdout:951: finishing write tid 2 to smithi19530714-40 2024-09-13T10:11:10.040 INFO:tasks.rados.rados.0.smithi195.stdout:951: finishing write tid 3 to smithi19530714-40 2024-09-13T10:11:10.040 INFO:tasks.rados.rados.0.smithi195.stdout:951: finishing write tid 4 to smithi19530714-40 2024-09-13T10:11:10.041 INFO:tasks.rados.rados.0.smithi195.stdout:951: oid 40 updating version 0 to 17 2024-09-13T10:11:10.041 INFO:tasks.rados.rados.0.smithi195.stdout:951: oid 40 version 17 is already newer than 16 2024-09-13T10:11:10.041 INFO:tasks.rados.rados.0.smithi195.stdout:951: oid 40 version 17 is already newer than 15 2024-09-13T10:11:10.041 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 17 (ObjNum 154 snap 0 seq_num 154) dirty exists 2024-09-13T10:11:10.041 INFO:tasks.rados.rados.0.smithi195.stdout:951: left oid 40 (ObjNum 154 snap 0 seq_num 154) 2024-09-13T10:11:10.041 INFO:tasks.rados.rados.0.smithi195.stdout:952: finishing write tid 1 to smithi19530714-25 2024-09-13T10:11:10.041 INFO:tasks.rados.rados.0.smithi195.stdout:952: finishing write tid 2 to smithi19530714-25 2024-09-13T10:11:10.041 INFO:tasks.rados.rados.0.smithi195.stdout:952: finishing write tid 3 to smithi19530714-25 2024-09-13T10:11:10.041 INFO:tasks.rados.rados.0.smithi195.stdout:952: finishing write tid 4 to smithi19530714-25 2024-09-13T10:11:10.041 INFO:tasks.rados.rados.0.smithi195.stdout:952: oid 25 updating version 0 to 80 2024-09-13T10:11:10.041 INFO:tasks.rados.rados.0.smithi195.stdout:952: oid 25 updating version 80 to 82 2024-09-13T10:11:10.041 INFO:tasks.rados.rados.0.smithi195.stdout:952: oid 25 version 82 is already newer than 81 2024-09-13T10:11:10.041 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 82 (ObjNum 155 snap 0 seq_num 155) dirty exists 2024-09-13T10:11:10.041 INFO:tasks.rados.rados.0.smithi195.stdout:952: left oid 25 (ObjNum 155 snap 0 seq_num 155) 2024-09-13T10:11:10.042 INFO:tasks.rados.rados.0.smithi195.stdout:953: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:11:10.055 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:10.056 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:10.056 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:10.053+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:10.057 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:10.057 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:10.070 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-13T10:11:10.114 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2921726 2024-09-13T10:11:10.124 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 263353 2024-09-13T10:11:10.157 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:10.155+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:10.170 INFO:tasks.rados.rados.0.smithi195.stdout:954: finishing write tid 1 to smithi19530714-50 2024-09-13T10:11:10.171 INFO:tasks.rados.rados.0.smithi195.stdout:955: got (0) Success 2024-09-13T10:11:10.171 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 85 (ObjNum 151 snap 0 seq_num 151) dirty exists 2024-09-13T10:11:10.171 INFO:tasks.rados.rados.0.smithi195.stdout:954: finishing write tid 2 to smithi19530714-50 2024-09-13T10:11:10.171 INFO:tasks.rados.rados.0.smithi195.stdout:954: finishing write tid 3 to smithi19530714-50 2024-09-13T10:11:10.171 INFO:tasks.rados.rados.0.smithi195.stdout:954: finishing write tid 4 to smithi19530714-50 2024-09-13T10:11:10.171 INFO:tasks.rados.rados.0.smithi195.stdout:954: oid 50 updating version 0 to 33 2024-09-13T10:11:10.171 INFO:tasks.rados.rados.0.smithi195.stdout:954: oid 50 version 33 is already newer than 31 2024-09-13T10:11:10.171 INFO:tasks.rados.rados.0.smithi195.stdout:954: oid 50 version 33 is already newer than 32 2024-09-13T10:11:10.171 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 33 (ObjNum 156 snap 0 seq_num 156) dirty exists 2024-09-13T10:11:10.171 INFO:tasks.rados.rados.0.smithi195.stdout:954: left oid 50 (ObjNum 156 snap 0 seq_num 156) 2024-09-13T10:11:10.172 INFO:tasks.rados.rados.0.smithi195.stdout:956: finishing set_chunk to oid 2 2024-09-13T10:11:10.172 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 2 set_chunk returned error code -95 offset: 445952 length: 1115648 2024-09-13T10:11:10.172 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 0 (ObjNum 68 snap 0 seq_num 68) dirty exists 2024-09-13T10:11:10.172 INFO:tasks.rados.rados.0.smithi195.stdout:957: expect (ObjNum 88 snap 0 seq_num 88) 2024-09-13T10:11:10.195 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2321830 2024-09-13T10:11:10.232 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:10.232 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:10.247 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-13T10:11:10.258 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:10.253+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:10.261 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:10.284 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:10.284 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:10.313 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:10.313 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:10.315 INFO:tasks.rados.rados.0.smithi195.stdout:958: finishing write tid 1 to smithi19530714-6 2024-09-13T10:11:10.315 INFO:tasks.rados.rados.0.smithi195.stdout:958: finishing write tid 2 to smithi19530714-6 2024-09-13T10:11:10.315 INFO:tasks.rados.rados.0.smithi195.stdout:958: finishing write tid 3 to smithi19530714-6 2024-09-13T10:11:10.315 INFO:tasks.rados.rados.0.smithi195.stdout:958: finishing write tid 4 to smithi19530714-6 2024-09-13T10:11:10.315 INFO:tasks.rados.rados.0.smithi195.stdout:958: oid 6 updating version 0 to 49 2024-09-13T10:11:10.315 INFO:tasks.rados.rados.0.smithi195.stdout:958: oid 6 updating version 49 to 50 2024-09-13T10:11:10.315 INFO:tasks.rados.rados.0.smithi195.stdout:958: oid 6 version 50 is already newer than 48 2024-09-13T10:11:10.315 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 50 (ObjNum 157 snap 0 seq_num 157) dirty exists 2024-09-13T10:11:10.315 INFO:tasks.rados.rados.0.smithi195.stdout:958: left oid 6 (ObjNum 157 snap 0 seq_num 157) 2024-09-13T10:11:10.315 INFO:tasks.rados.rados.0.smithi195.stdout:959: expect (ObjNum 128 snap 0 seq_num 128) 2024-09-13T10:11:10.315 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:10.346 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:10.347 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:10.358 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:10.357+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:10.368 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:10.368 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:10.375 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:10.376 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:10.378 INFO:tasks.rados.rados.0.smithi195.stdout:960: finishing write tid 1 to smithi19530714-45 2024-09-13T10:11:10.378 INFO:tasks.rados.rados.0.smithi195.stdout:962: expect (ObjNum 137 snap 0 seq_num 137) 2024-09-13T10:11:10.383 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-13T10:11:10.402 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:10.403 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:10.421 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:10.427 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:10.427 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:10.449 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:10.459 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:10.460+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:10.461 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:10.467 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:10.467 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:10.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:10.465+0000 7efe0922f640 0 -- 172.21.15.73:0/2100362530 >> v2:172.21.15.195:6805/848686366 conn(0x7efe04225ae0 msgr2=0x7efe04227ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:11:10.501 INFO:tasks.rados.rados.0.smithi195.stdout:960: finishing write tid 2 to smithi19530714-45 2024-09-13T10:11:10.501 INFO:tasks.rados.rados.0.smithi195.stdout:960: finishing write tid 3 to smithi19530714-45 2024-09-13T10:11:10.502 INFO:tasks.rados.rados.0.smithi195.stdout:960: finishing write tid 4 to smithi19530714-45 2024-09-13T10:11:10.502 INFO:tasks.rados.rados.0.smithi195.stdout:960: oid 45 updating version 0 to 84 2024-09-13T10:11:10.502 INFO:tasks.rados.rados.0.smithi195.stdout:960: oid 45 updating version 84 to 85 2024-09-13T10:11:10.502 INFO:tasks.rados.rados.0.smithi195.stdout:960: oid 45 version 85 is already newer than 83 2024-09-13T10:11:10.502 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 85 (ObjNum 158 snap 0 seq_num 158) dirty exists 2024-09-13T10:11:10.502 INFO:tasks.rados.rados.0.smithi195.stdout:960: left oid 45 (ObjNum 158 snap 0 seq_num 158) 2024-09-13T10:11:10.502 INFO:tasks.rados.rados.0.smithi195.stdout:961: expect (ObjNum 103 snap 0 seq_num 103) 2024-09-13T10:11:10.502 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:10.504 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:10.505 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:11:10.505 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:11:10.505 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:11:10.505 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.505 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:202 3.2 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.505 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.991310+0000", 2024-09-13T10:11:10.505 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.509396736999999, 2024-09-13T10:11:10.505 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.040945251000000002, 2024-09-13T10:11:10.505 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.505 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.505 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.506 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:10.506 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:10.506 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:10.506 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 202 2024-09-13T10:11:10.506 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.506 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.506 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.506 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.506 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991310+0000", 2024-09-13T10:11:10.506 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.506 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.506 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.506 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.507 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991310+0000", 2024-09-13T10:11:10.507 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.507 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.507 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.507 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991314+0000", 2024-09-13T10:11:10.507 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.7100000000000001e-06 2024-09-13T10:11:10.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.507 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.507 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.507 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991328+0000", 2024-09-13T10:11:10.507 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.413e-05 2024-09-13T10:11:10.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.508 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.508 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.508 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991361+0000", 2024-09-13T10:11:10.508 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2512999999999999e-05 2024-09-13T10:11:10.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.508 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.508 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.508 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991377+0000", 2024-09-13T10:11:10.508 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6478999999999999e-05 2024-09-13T10:11:10.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.508 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.508 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.509 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991546+0000", 2024-09-13T10:11:10.509 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016856600000000001 2024-09-13T10:11:10.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.509 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.509 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.509 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991850+0000", 2024-09-13T10:11:10.509 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00030478800000000001 2024-09-13T10:11:10.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.509 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.509 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.509 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.001416+0000", 2024-09-13T10:11:10.509 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0095659279999999996 2024-09-13T10:11:10.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.510 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.510 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.510 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031502+0000", 2024-09-13T10:11:10.510 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.030085526000000001 2024-09-13T10:11:10.510 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.510 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.510 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.510 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031585+0000", 2024-09-13T10:11:10.510 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.3175000000000002e-05 2024-09-13T10:11:10.510 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.510 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.510 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.510 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031775+0000", 2024-09-13T10:11:10.511 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000190161 2024-09-13T10:11:10.511 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.511 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.511 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.511 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031982+0000", 2024-09-13T10:11:10.511 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000207245 2024-09-13T10:11:10.511 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.511 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.511 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.511 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032255+0000", 2024-09-13T10:11:10.511 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00027303000000000001 2024-09-13T10:11:10.511 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.511 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.512 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.512 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.512 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.512 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:206 3.2 3:4cbdbe9e:::smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.512 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.991649+0000", 2024-09-13T10:11:10.512 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.509058409000001, 2024-09-13T10:11:10.512 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041234240999999998, 2024-09-13T10:11:10.512 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.512 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.512 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.512 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:10.512 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:10.512 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:10.513 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 206 2024-09-13T10:11:10.513 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.513 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.513 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.513 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.513 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991649+0000", 2024-09-13T10:11:10.513 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.513 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.513 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.513 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.513 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991649+0000", 2024-09-13T10:11:10.513 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.513 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.514 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.514 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.514 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991650+0000", 2024-09-13T10:11:10.514 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.483e-06 2024-09-13T10:11:10.514 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.514 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.514 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.514 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991657+0000", 2024-09-13T10:11:10.514 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.0899999999999999e-06 2024-09-13T10:11:10.514 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.514 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.514 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.514 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991672+0000", 2024-09-13T10:11:10.515 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4514e-05 2024-09-13T10:11:10.515 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.515 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.515 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.515 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991681+0000", 2024-09-13T10:11:10.515 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6360000000000006e-06 2024-09-13T10:11:10.515 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.515 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.515 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.515 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992159+0000", 2024-09-13T10:11:10.515 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00047820099999999998 2024-09-13T10:11:10.515 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.516 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.516 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.516 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992411+0000", 2024-09-13T10:11:10.516 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025199700000000001 2024-09-13T10:11:10.516 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.516 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.516 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.516 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.009775+0000", 2024-09-13T10:11:10.516 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017363841000000001 2024-09-13T10:11:10.516 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.516 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.516 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.516 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032446+0000", 2024-09-13T10:11:10.516 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022670490000000001 2024-09-13T10:11:10.517 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.517 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.517 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.517 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032482+0000", 2024-09-13T10:11:10.517 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5902000000000003e-05 2024-09-13T10:11:10.517 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.517 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.517 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.517 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032612+0000", 2024-09-13T10:11:10.517 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013022700000000001 2024-09-13T10:11:10.517 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.517 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.517 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.518 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032767+0000", 2024-09-13T10:11:10.518 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015477000000000001 2024-09-13T10:11:10.518 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.518 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.518 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.518 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032883+0000", 2024-09-13T10:11:10.518 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011608999999999999 2024-09-13T10:11:10.518 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.518 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.518 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.518 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.518 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.518 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:210 3.2 3:46ab5247:::smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.519 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.992337+0000", 2024-09-13T10:11:10.519 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.508370266, 2024-09-13T10:11:10.519 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.057985696000000003, 2024-09-13T10:11:10.519 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.519 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.519 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.519 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:10.519 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:10.519 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:10.519 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 210 2024-09-13T10:11:10.519 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.519 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.519 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.520 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.520 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992337+0000", 2024-09-13T10:11:10.520 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.520 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.520 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.520 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.520 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992337+0000", 2024-09-13T10:11:10.520 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.520 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.520 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.520 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.520 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992339+0000", 2024-09-13T10:11:10.520 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2400000000000002e-06 2024-09-13T10:11:10.521 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.521 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.521 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.521 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992350+0000", 2024-09-13T10:11:10.521 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1259e-05 2024-09-13T10:11:10.521 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.521 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.521 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.521 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992368+0000", 2024-09-13T10:11:10.521 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8133000000000001e-05 2024-09-13T10:11:10.521 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.521 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.521 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.522 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992382+0000", 2024-09-13T10:11:10.522 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3837e-05 2024-09-13T10:11:10.522 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.522 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.522 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.522 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992752+0000", 2024-09-13T10:11:10.522 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037008000000000001 2024-09-13T10:11:10.522 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.522 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.522 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.522 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992980+0000", 2024-09-13T10:11:10.522 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022799100000000001 2024-09-13T10:11:10.523 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.523 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.523 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.523 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027651+0000", 2024-09-13T10:11:10.523 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.034671027 2024-09-13T10:11:10.523 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.523 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.523 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.523 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.034868+0000", 2024-09-13T10:11:10.523 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0072169920000000002 2024-09-13T10:11:10.523 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.523 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.523 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.524 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038033+0000", 2024-09-13T10:11:10.524 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003164966 2024-09-13T10:11:10.524 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.524 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.524 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.524 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050187+0000", 2024-09-13T10:11:10.524 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012153815 2024-09-13T10:11:10.524 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.524 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.524 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.524 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050266+0000", 2024-09-13T10:11:10.524 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9359999999999999e-05 2024-09-13T10:11:10.524 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.525 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.525 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.525 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050322+0000", 2024-09-13T10:11:10.525 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5995999999999997e-05 2024-09-13T10:11:10.525 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.525 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.525 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.525 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.525 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.525 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:214 3.2 3:68ba95ba:::smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.525 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.992479+0000", 2024-09-13T10:11:10.525 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.508228193000001, 2024-09-13T10:11:10.525 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.045987867000000002, 2024-09-13T10:11:10.526 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.526 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.526 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.526 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:10.526 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:10.526 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:10.526 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 214 2024-09-13T10:11:10.526 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.526 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.526 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.526 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.526 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992479+0000", 2024-09-13T10:11:10.526 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.526 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.527 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.527 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.527 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992479+0000", 2024-09-13T10:11:10.527 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.527 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.527 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.527 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.527 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992481+0000", 2024-09-13T10:11:10.527 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2960000000000001e-06 2024-09-13T10:11:10.527 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.527 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.527 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.527 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992493+0000", 2024-09-13T10:11:10.527 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2077e-05 2024-09-13T10:11:10.528 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.528 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.528 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.528 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992519+0000", 2024-09-13T10:11:10.528 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6282e-05 2024-09-13T10:11:10.528 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.528 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.528 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.528 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992538+0000", 2024-09-13T10:11:10.528 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9083e-05 2024-09-13T10:11:10.528 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.528 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.528 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.529 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993393+0000", 2024-09-13T10:11:10.529 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00085445200000000001 2024-09-13T10:11:10.529 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.529 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.529 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.529 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993682+0000", 2024-09-13T10:11:10.529 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000288895 2024-09-13T10:11:10.529 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.529 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.529 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.529 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.019700+0000", 2024-09-13T10:11:10.529 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.02601794 2024-09-13T10:11:10.529 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.529 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.530 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.530 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032498+0000", 2024-09-13T10:11:10.530 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012798535999999999 2024-09-13T10:11:10.530 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.530 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.530 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.530 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033641+0000", 2024-09-13T10:11:10.530 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001142569 2024-09-13T10:11:10.530 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.530 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.530 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.530 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038259+0000", 2024-09-13T10:11:10.530 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0046180939999999997 2024-09-13T10:11:10.530 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.531 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.531 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.531 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038379+0000", 2024-09-13T10:11:10.531 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000119914 2024-09-13T10:11:10.531 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.531 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.531 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.531 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038467+0000", 2024-09-13T10:11:10.531 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7729e-05 2024-09-13T10:11:10.531 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.531 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.531 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.531 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.531 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.532 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:217 3.2 3:554c406d:::smithi19530714-18 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.532 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.027515+0000", 2024-09-13T10:11:10.532 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.473192358999999, 2024-09-13T10:11:10.532 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032787180999999999, 2024-09-13T10:11:10.532 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.532 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.532 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.532 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:10.532 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:10.532 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:10.532 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 217 2024-09-13T10:11:10.532 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.532 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.533 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.533 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.533 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027515+0000", 2024-09-13T10:11:10.533 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.533 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.533 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.533 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.533 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027515+0000", 2024-09-13T10:11:10.533 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.533 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.533 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.533 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.533 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027518+0000", 2024-09-13T10:11:10.534 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.4429999999999999e-06 2024-09-13T10:11:10.534 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.534 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.534 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.534 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027532+0000", 2024-09-13T10:11:10.534 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3517e-05 2024-09-13T10:11:10.534 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.534 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.534 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.534 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027563+0000", 2024-09-13T10:11:10.534 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1158999999999998e-05 2024-09-13T10:11:10.534 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.534 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.534 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.535 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027585+0000", 2024-09-13T10:11:10.535 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.228e-05 2024-09-13T10:11:10.535 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.535 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.535 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.535 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033014+0000", 2024-09-13T10:11:10.535 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0054288649999999997 2024-09-13T10:11:10.535 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.535 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.535 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.535 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033269+0000", 2024-09-13T10:11:10.535 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000255054 2024-09-13T10:11:10.535 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.536 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.536 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.536 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.044618+0000", 2024-09-13T10:11:10.536 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011349391 2024-09-13T10:11:10.536 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.536 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.536 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.536 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.054627+0000", 2024-09-13T10:11:10.536 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010008992 2024-09-13T10:11:10.536 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.536 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.536 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.536 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.058047+0000", 2024-09-13T10:11:10.536 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0034193000000000001 2024-09-13T10:11:10.537 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.537 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.537 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.537 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060139+0000", 2024-09-13T10:11:10.537 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020925240000000001 2024-09-13T10:11:10.537 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.537 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.537 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.537 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060222+0000", 2024-09-13T10:11:10.537 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.2917999999999995e-05 2024-09-13T10:11:10.537 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.537 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.537 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.538 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060302+0000", 2024-09-13T10:11:10.538 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9738000000000001e-05 2024-09-13T10:11:10.538 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.538 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.538 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.538 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.538 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.538 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:218 3.2 3:5915c927:::smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.538 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.032754+0000", 2024-09-13T10:11:10.538 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.467953090999998, 2024-09-13T10:11:10.538 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023630241, 2024-09-13T10:11:10.538 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.538 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.539 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.539 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:10.539 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:10.539 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:10.539 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 218 2024-09-13T10:11:10.539 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.539 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.539 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.539 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.539 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032754+0000", 2024-09-13T10:11:10.539 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.539 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.539 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.539 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.540 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032754+0000", 2024-09-13T10:11:10.540 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.540 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.540 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.540 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.540 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032757+0000", 2024-09-13T10:11:10.540 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6330000000000002e-06 2024-09-13T10:11:10.540 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.540 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.540 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.540 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032772+0000", 2024-09-13T10:11:10.540 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4636e-05 2024-09-13T10:11:10.540 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.541 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.541 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.541 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032796+0000", 2024-09-13T10:11:10.541 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3963000000000001e-05 2024-09-13T10:11:10.541 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.541 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.541 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.541 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032815+0000", 2024-09-13T10:11:10.541 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9159000000000001e-05 2024-09-13T10:11:10.541 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.541 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.541 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.541 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033742+0000", 2024-09-13T10:11:10.542 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00092672700000000002 2024-09-13T10:11:10.542 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.542 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.542 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.542 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.034124+0000", 2024-09-13T10:11:10.542 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00038220399999999997 2024-09-13T10:11:10.542 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.542 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.542 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.542 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.040963+0000", 2024-09-13T10:11:10.542 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0068392230000000002 2024-09-13T10:11:10.542 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.542 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.543 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.543 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050362+0000", 2024-09-13T10:11:10.543 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0093986060000000003 2024-09-13T10:11:10.543 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.543 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.543 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.543 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.054585+0000", 2024-09-13T10:11:10.543 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0042225429999999996 2024-09-13T10:11:10.543 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.543 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.543 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.543 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.056250+0000", 2024-09-13T10:11:10.543 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001665251 2024-09-13T10:11:10.543 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.544 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.544 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.544 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.056320+0000", 2024-09-13T10:11:10.544 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9913999999999995e-05 2024-09-13T10:11:10.544 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.544 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.544 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.544 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.056384+0000", 2024-09-13T10:11:10.544 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4381999999999994e-05 2024-09-13T10:11:10.544 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.544 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.544 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.544 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.545 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.545 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:220 3.2 3:753d5785:::smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.545 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.035716+0000", 2024-09-13T10:11:10.545 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.464990598, 2024-09-13T10:11:10.545 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.030171203000000001, 2024-09-13T10:11:10.545 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.545 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.545 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.546 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:10.546 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:10.546 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:10.546 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 220 2024-09-13T10:11:10.546 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.546 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.546 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.546 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.546 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035716+0000", 2024-09-13T10:11:10.546 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.546 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.546 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.546 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.547 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035716+0000", 2024-09-13T10:11:10.547 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.547 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.547 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.547 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.547 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035720+0000", 2024-09-13T10:11:10.547 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5310000000000002e-06 2024-09-13T10:11:10.547 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.547 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.548 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.548 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035735+0000", 2024-09-13T10:11:10.548 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5517000000000001e-05 2024-09-13T10:11:10.548 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.548 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.548 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.548 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035765+0000", 2024-09-13T10:11:10.548 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9771e-05 2024-09-13T10:11:10.548 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.548 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.548 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.548 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035786+0000", 2024-09-13T10:11:10.549 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0928e-05 2024-09-13T10:11:10.549 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.549 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.549 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.549 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035945+0000", 2024-09-13T10:11:10.549 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015871900000000001 2024-09-13T10:11:10.549 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.549 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.549 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.549 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.036337+0000", 2024-09-13T10:11:10.549 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039231299999999998 2024-09-13T10:11:10.549 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.550 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.550 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.550 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.051063+0000", 2024-09-13T10:11:10.550 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014725607 2024-09-13T10:11:10.550 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.550 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.550 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.550 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.055802+0000", 2024-09-13T10:11:10.550 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0047387929999999998 2024-09-13T10:11:10.550 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.550 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.550 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.550 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064353+0000", 2024-09-13T10:11:10.550 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0085516629999999993 2024-09-13T10:11:10.551 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.551 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.551 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.551 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065750+0000", 2024-09-13T10:11:10.551 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001396748 2024-09-13T10:11:10.551 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.551 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.551 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.551 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065811+0000", 2024-09-13T10:11:10.551 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.0770000000000003e-05 2024-09-13T10:11:10.551 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.551 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.551 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.552 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065888+0000", 2024-09-13T10:11:10.552 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6842999999999996e-05 2024-09-13T10:11:10.552 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.552 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.552 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.552 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.552 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.552 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:227 3.2 3:6ac7800c:::smithi19530714-28 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.552 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.060091+0000", 2024-09-13T10:11:10.552 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.440615532999999, 2024-09-13T10:11:10.552 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.034993362, 2024-09-13T10:11:10.552 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.552 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.553 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.553 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:10.553 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:10.553 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:10.553 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 227 2024-09-13T10:11:10.553 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.553 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.553 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.553 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.553 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060091+0000", 2024-09-13T10:11:10.553 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.554 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.554 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.554 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.554 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060091+0000", 2024-09-13T10:11:10.554 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.554 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.554 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.554 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.554 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060094+0000", 2024-09-13T10:11:10.554 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5330000000000001e-06 2024-09-13T10:11:10.554 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.554 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.554 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.554 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060103+0000", 2024-09-13T10:11:10.555 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7020000000000007e-06 2024-09-13T10:11:10.555 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.555 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.555 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.555 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060121+0000", 2024-09-13T10:11:10.555 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.791e-05 2024-09-13T10:11:10.555 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.555 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.555 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.555 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060135+0000", 2024-09-13T10:11:10.555 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4059999999999999e-05 2024-09-13T10:11:10.555 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.555 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.556 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.556 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060346+0000", 2024-09-13T10:11:10.556 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021175600000000001 2024-09-13T10:11:10.556 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.556 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.556 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.556 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060559+0000", 2024-09-13T10:11:10.556 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021239499999999999 2024-09-13T10:11:10.556 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.556 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.556 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.556 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.073003+0000", 2024-09-13T10:11:10.556 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012444255 2024-09-13T10:11:10.556 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.557 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.557 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.557 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.081388+0000", 2024-09-13T10:11:10.557 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0083852119999999995 2024-09-13T10:11:10.557 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.557 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.557 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.557 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094289+0000", 2024-09-13T10:11:10.557 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012900271 2024-09-13T10:11:10.557 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.557 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.557 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.557 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094913+0000", 2024-09-13T10:11:10.558 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00062429200000000003 2024-09-13T10:11:10.558 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.558 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.558 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.558 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094972+0000", 2024-09-13T10:11:10.558 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9457999999999999e-05 2024-09-13T10:11:10.558 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.558 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.558 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.558 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095085+0000", 2024-09-13T10:11:10.558 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000112518 2024-09-13T10:11:10.558 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.558 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.558 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.559 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.559 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.559 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:230 3.2 3:5d108080:::smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.559 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.065002+0000", 2024-09-13T10:11:10.559 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.435705088999999, 2024-09-13T10:11:10.559 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.049477431000000002, 2024-09-13T10:11:10.559 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.559 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.559 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.559 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:10.559 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:10.559 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:10.559 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 230 2024-09-13T10:11:10.559 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.560 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.560 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.560 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.560 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065002+0000", 2024-09-13T10:11:10.560 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.560 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.560 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.560 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.560 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065002+0000", 2024-09-13T10:11:10.560 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.560 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.560 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.560 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.561 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065004+0000", 2024-09-13T10:11:10.561 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4269999999999998e-06 2024-09-13T10:11:10.561 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.561 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.561 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.561 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065014+0000", 2024-09-13T10:11:10.561 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.9799999999999993e-06 2024-09-13T10:11:10.561 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.561 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.561 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.561 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065038+0000", 2024-09-13T10:11:10.561 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4229000000000001e-05 2024-09-13T10:11:10.561 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.561 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.562 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.562 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065054+0000", 2024-09-13T10:11:10.562 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5764999999999999e-05 2024-09-13T10:11:10.562 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.562 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.562 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.562 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065171+0000", 2024-09-13T10:11:10.562 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117032 2024-09-13T10:11:10.562 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.562 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.562 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.562 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065513+0000", 2024-09-13T10:11:10.562 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00034131 2024-09-13T10:11:10.562 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.563 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.563 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.563 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.087042+0000", 2024-09-13T10:11:10.563 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021529534999999999 2024-09-13T10:11:10.563 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.563 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.563 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.563 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.097903+0000", 2024-09-13T10:11:10.563 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010861282999999999 2024-09-13T10:11:10.563 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.563 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.563 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.563 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.099922+0000", 2024-09-13T10:11:10.564 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020186539999999999 2024-09-13T10:11:10.564 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.564 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.564 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.564 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114391+0000", 2024-09-13T10:11:10.564 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014468880999999999 2024-09-13T10:11:10.564 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.564 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.564 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.564 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114434+0000", 2024-09-13T10:11:10.564 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.3511999999999999e-05 2024-09-13T10:11:10.564 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.564 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.565 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.565 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114479+0000", 2024-09-13T10:11:10.565 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4823000000000001e-05 2024-09-13T10:11:10.565 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.565 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.565 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.565 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.565 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.565 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:232 3.2 3:6538713c:::smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.565 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.065891+0000", 2024-09-13T10:11:10.565 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.434816218000002, 2024-09-13T10:11:10.565 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048804489999999999, 2024-09-13T10:11:10.565 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.566 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.566 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.566 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:10.566 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:10.566 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:10.566 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 232 2024-09-13T10:11:10.566 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.566 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.566 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.566 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.566 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065891+0000", 2024-09-13T10:11:10.566 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.566 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.566 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.567 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.567 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065891+0000", 2024-09-13T10:11:10.567 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.567 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.567 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.567 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.567 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065892+0000", 2024-09-13T10:11:10.567 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6249999999999999e-06 2024-09-13T10:11:10.567 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.567 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.567 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.567 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065902+0000", 2024-09-13T10:11:10.568 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.4150000000000005e-06 2024-09-13T10:11:10.568 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.568 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.568 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.568 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065920+0000", 2024-09-13T10:11:10.568 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7907e-05 2024-09-13T10:11:10.568 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.568 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.568 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.568 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065932+0000", 2024-09-13T10:11:10.568 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2469999999999999e-05 2024-09-13T10:11:10.568 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.568 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.569 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.569 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.066072+0000", 2024-09-13T10:11:10.569 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013935999999999999 2024-09-13T10:11:10.569 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.569 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.569 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.569 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.066435+0000", 2024-09-13T10:11:10.569 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036323300000000002 2024-09-13T10:11:10.569 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.569 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.569 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.569 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091133+0000", 2024-09-13T10:11:10.569 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024698494000000001 2024-09-13T10:11:10.569 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.570 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.570 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.570 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.101407+0000", 2024-09-13T10:11:10.570 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010273835 2024-09-13T10:11:10.570 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.570 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.570 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.570 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114342+0000", 2024-09-13T10:11:10.570 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012934826 2024-09-13T10:11:10.570 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.570 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.570 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.570 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114548+0000", 2024-09-13T10:11:10.571 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000206243 2024-09-13T10:11:10.571 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.571 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.571 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.571 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114630+0000", 2024-09-13T10:11:10.571 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.2127999999999997e-05 2024-09-13T10:11:10.571 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.571 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.571 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.571 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114695+0000", 2024-09-13T10:11:10.571 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4954000000000002e-05 2024-09-13T10:11:10.571 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.571 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.571 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.572 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.572 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.572 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:236 3.2 3:73c2893b:::smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.572 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.076316+0000", 2024-09-13T10:11:10.572 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.424391097000001, 2024-09-13T10:11:10.572 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041217193999999999, 2024-09-13T10:11:10.572 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.572 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.572 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.572 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:10.572 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:10.572 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:10.572 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 236 2024-09-13T10:11:10.573 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.573 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.573 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.573 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.573 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076316+0000", 2024-09-13T10:11:10.573 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.573 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.573 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.573 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.573 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076316+0000", 2024-09-13T10:11:10.573 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.573 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.573 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.574 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.574 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076319+0000", 2024-09-13T10:11:10.574 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1750000000000001e-06 2024-09-13T10:11:10.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.574 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.574 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.574 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076333+0000", 2024-09-13T10:11:10.574 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3862e-05 2024-09-13T10:11:10.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.574 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.574 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.574 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076359+0000", 2024-09-13T10:11:10.574 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5641999999999999e-05 2024-09-13T10:11:10.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.575 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.575 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076378+0000", 2024-09-13T10:11:10.575 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9406999999999999e-05 2024-09-13T10:11:10.575 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.575 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.575 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.078831+0000", 2024-09-13T10:11:10.575 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0024532590000000002 2024-09-13T10:11:10.575 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.575 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.575 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079137+0000", 2024-09-13T10:11:10.575 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00030614500000000002 2024-09-13T10:11:10.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.576 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.576 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.576 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.102393+0000", 2024-09-13T10:11:10.576 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023255175999999999 2024-09-13T10:11:10.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.576 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.576 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.576 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114728+0000", 2024-09-13T10:11:10.576 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012335376 2024-09-13T10:11:10.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.576 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.576 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.577 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114735+0000", 2024-09-13T10:11:10.577 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2969999999999998e-06 2024-09-13T10:11:10.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.577 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.577 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.577 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117465+0000", 2024-09-13T10:11:10.577 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0027298019999999999 2024-09-13T10:11:10.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.577 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.577 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.577 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117501+0000", 2024-09-13T10:11:10.577 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5837999999999998e-05 2024-09-13T10:11:10.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.577 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.578 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.578 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117533+0000", 2024-09-13T10:11:10.578 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2215000000000002e-05 2024-09-13T10:11:10.578 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.578 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.578 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.578 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.578 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.578 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:240 3.2 3:7eee2f4a:::smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.578 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.091046+0000", 2024-09-13T10:11:10.578 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.409661256, 2024-09-13T10:11:10.578 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048376970999999998, 2024-09-13T10:11:10.578 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.578 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.579 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.579 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:10.579 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:10.579 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:10.579 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 240 2024-09-13T10:11:10.579 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.579 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.579 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.579 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.579 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091046+0000", 2024-09-13T10:11:10.579 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.579 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.579 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.579 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.579 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091046+0000", 2024-09-13T10:11:10.580 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.580 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.580 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.580 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.580 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091048+0000", 2024-09-13T10:11:10.580 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6299999999999998e-06 2024-09-13T10:11:10.580 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.580 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.580 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.580 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091060+0000", 2024-09-13T10:11:10.580 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1229999999999999e-05 2024-09-13T10:11:10.580 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.580 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.581 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.581 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091075+0000", 2024-09-13T10:11:10.581 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5313000000000001e-05 2024-09-13T10:11:10.581 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.581 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.581 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091085+0000", 2024-09-13T10:11:10.581 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0284e-05 2024-09-13T10:11:10.581 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.581 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.581 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094402+0000", 2024-09-13T10:11:10.581 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00331708 2024-09-13T10:11:10.581 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.582 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.582 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094649+0000", 2024-09-13T10:11:10.582 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024695300000000001 2024-09-13T10:11:10.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.582 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.582 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.107510+0000", 2024-09-13T10:11:10.582 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012860778999999999 2024-09-13T10:11:10.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.582 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.582 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114739+0000", 2024-09-13T10:11:10.582 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0072289700000000004 2024-09-13T10:11:10.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.583 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.583 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118693+0000", 2024-09-13T10:11:10.583 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0039540560000000001 2024-09-13T10:11:10.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.583 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.583 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139282+0000", 2024-09-13T10:11:10.583 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020589423999999999 2024-09-13T10:11:10.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.583 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.583 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139357+0000", 2024-09-13T10:11:10.583 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.4276999999999994e-05 2024-09-13T10:11:10.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.584 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.584 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.584 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139423+0000", 2024-09-13T10:11:10.584 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.5975000000000004e-05 2024-09-13T10:11:10.584 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.584 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.584 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.584 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.584 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.584 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:242 3.2 3:501d22a1:::smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.584 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.095846+0000", 2024-09-13T10:11:10.584 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.404861052000001, 2024-09-13T10:11:10.584 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.043813831999999997, 2024-09-13T10:11:10.584 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.585 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.585 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.585 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:10.585 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:10.585 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:10.585 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 242 2024-09-13T10:11:10.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.585 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.585 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.585 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.585 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095846+0000", 2024-09-13T10:11:10.585 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.586 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.586 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095846+0000", 2024-09-13T10:11:10.586 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.586 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.586 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095848+0000", 2024-09-13T10:11:10.586 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3760000000000002e-06 2024-09-13T10:11:10.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.586 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.586 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095859+0000", 2024-09-13T10:11:10.587 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0429e-05 2024-09-13T10:11:10.587 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.587 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.587 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.587 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095876+0000", 2024-09-13T10:11:10.587 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7504e-05 2024-09-13T10:11:10.587 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.587 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.587 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.587 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095886+0000", 2024-09-13T10:11:10.587 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0182e-05 2024-09-13T10:11:10.587 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.588 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.588 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.588 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.096010+0000", 2024-09-13T10:11:10.588 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012336700000000001 2024-09-13T10:11:10.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.588 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.588 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.588 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.096252+0000", 2024-09-13T10:11:10.588 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024189200000000001 2024-09-13T10:11:10.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.588 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.588 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.588 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111492+0000", 2024-09-13T10:11:10.588 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015239920000000001 2024-09-13T10:11:10.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.589 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.589 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115691+0000", 2024-09-13T10:11:10.589 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0041992410000000003 2024-09-13T10:11:10.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.589 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.589 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139492+0000", 2024-09-13T10:11:10.589 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023801495999999998 2024-09-13T10:11:10.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.589 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.589 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139560+0000", 2024-09-13T10:11:10.590 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.8163000000000005e-05 2024-09-13T10:11:10.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.590 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.590 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.590 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139619+0000", 2024-09-13T10:11:10.590 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8678999999999998e-05 2024-09-13T10:11:10.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.590 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.590 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.590 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139660+0000", 2024-09-13T10:11:10.590 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0583000000000001e-05 2024-09-13T10:11:10.590 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.590 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.590 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.591 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.591 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:244 3.2 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.591 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.112158+0000", 2024-09-13T10:11:10.591 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.388548588999999, 2024-09-13T10:11:10.591 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.030058202999999999, 2024-09-13T10:11:10.591 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.591 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.591 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.591 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:10.591 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:10.591 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:10.591 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 244 2024-09-13T10:11:10.591 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.591 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.591 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.592 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.592 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112158+0000", 2024-09-13T10:11:10.592 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.592 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.592 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112158+0000", 2024-09-13T10:11:10.592 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.592 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.592 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112161+0000", 2024-09-13T10:11:10.592 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.384e-06 2024-09-13T10:11:10.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.593 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.593 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112168+0000", 2024-09-13T10:11:10.593 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.7780000000000002e-06 2024-09-13T10:11:10.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.593 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.593 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.593 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112183+0000", 2024-09-13T10:11:10.593 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5917999999999999e-05 2024-09-13T10:11:10.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.593 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.593 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.593 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112195+0000", 2024-09-13T10:11:10.593 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1578e-05 2024-09-13T10:11:10.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.594 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.594 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.594 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114766+0000", 2024-09-13T10:11:10.594 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025710659999999999 2024-09-13T10:11:10.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.594 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.594 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.594 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114995+0000", 2024-09-13T10:11:10.594 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022844 2024-09-13T10:11:10.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.594 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.594 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.594 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.125677+0000", 2024-09-13T10:11:10.594 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010682677999999999 2024-09-13T10:11:10.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.595 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.595 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.595 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.140355+0000", 2024-09-13T10:11:10.595 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014677773999999999 2024-09-13T10:11:10.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.595 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.595 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.595 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.141368+0000", 2024-09-13T10:11:10.595 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0010134040000000001 2024-09-13T10:11:10.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.595 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.595 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.595 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142112+0000", 2024-09-13T10:11:10.595 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00074347599999999997 2024-09-13T10:11:10.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.596 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.596 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142172+0000", 2024-09-13T10:11:10.596 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9749e-05 2024-09-13T10:11:10.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.596 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.597 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142217+0000", 2024-09-13T10:11:10.597 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4957999999999999e-05 2024-09-13T10:11:10.597 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.597 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.597 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.597 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.597 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:245 3.2 3:7837e849:::smithi19530714-46 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.597 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.112398+0000", 2024-09-13T10:11:10.597 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.388308532, 2024-09-13T10:11:10.597 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.039535731999999997, 2024-09-13T10:11:10.597 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.597 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.598 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.598 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:10.598 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:10.598 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:10.598 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 245 2024-09-13T10:11:10.598 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.598 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.598 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.598 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.598 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112398+0000", 2024-09-13T10:11:10.598 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.598 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.598 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.599 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.599 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112398+0000", 2024-09-13T10:11:10.599 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.599 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.599 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.599 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112399+0000", 2024-09-13T10:11:10.599 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0470000000000001e-06 2024-09-13T10:11:10.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.599 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.599 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.599 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112405+0000", 2024-09-13T10:11:10.599 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.6899999999999997e-06 2024-09-13T10:11:10.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.599 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.600 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.600 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112417+0000", 2024-09-13T10:11:10.600 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2334000000000001e-05 2024-09-13T10:11:10.600 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.600 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.600 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.600 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112427+0000", 2024-09-13T10:11:10.600 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.0929999999999992e-06 2024-09-13T10:11:10.600 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.600 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.600 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.600 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115182+0000", 2024-09-13T10:11:10.600 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0027552739999999998 2024-09-13T10:11:10.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.601 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.601 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.601 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115305+0000", 2024-09-13T10:11:10.601 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012364200000000001 2024-09-13T10:11:10.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.601 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.601 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.601 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.136435+0000", 2024-09-13T10:11:10.601 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021129479999999999 2024-09-13T10:11:10.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.601 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.601 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.602 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143376+0000", 2024-09-13T10:11:10.602 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0069408600000000001 2024-09-13T10:11:10.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.602 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.602 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.602 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.150416+0000", 2024-09-13T10:11:10.602 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0070403669999999996 2024-09-13T10:11:10.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.602 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.602 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.602 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.151872+0000", 2024-09-13T10:11:10.602 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001456286 2024-09-13T10:11:10.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.603 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.603 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.603 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.151906+0000", 2024-09-13T10:11:10.603 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3167999999999998e-05 2024-09-13T10:11:10.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.603 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.603 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.603 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.151934+0000", 2024-09-13T10:11:10.603 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8490999999999999e-05 2024-09-13T10:11:10.603 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.603 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.603 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.604 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:250 3.2 3:78d40eb9:::smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.604 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.115099+0000", 2024-09-13T10:11:10.604 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.385608289, 2024-09-13T10:11:10.604 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.028750412999999999, 2024-09-13T10:11:10.604 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.604 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.604 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.604 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:10.604 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:10.604 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:10.604 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 250 2024-09-13T10:11:10.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.605 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.605 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.605 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.605 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115099+0000", 2024-09-13T10:11:10.605 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.605 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.605 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.605 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.605 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115099+0000", 2024-09-13T10:11:10.605 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.605 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.605 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.605 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.606 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115101+0000", 2024-09-13T10:11:10.606 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1789999999999998e-06 2024-09-13T10:11:10.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.606 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.606 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.606 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115132+0000", 2024-09-13T10:11:10.606 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1163e-05 2024-09-13T10:11:10.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.606 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.606 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.606 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115150+0000", 2024-09-13T10:11:10.606 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7864999999999999e-05 2024-09-13T10:11:10.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.607 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.607 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.607 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115159+0000", 2024-09-13T10:11:10.607 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1719999999999996e-06 2024-09-13T10:11:10.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.607 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.607 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.607 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115440+0000", 2024-09-13T10:11:10.607 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028068199999999998 2024-09-13T10:11:10.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.607 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.607 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.607 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115579+0000", 2024-09-13T10:11:10.608 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013910800000000001 2024-09-13T10:11:10.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.608 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.608 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.132849+0000", 2024-09-13T10:11:10.608 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017270127999999999 2024-09-13T10:11:10.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.608 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.608 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142332+0000", 2024-09-13T10:11:10.608 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0094831859999999994 2024-09-13T10:11:10.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.609 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.609 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143034+0000", 2024-09-13T10:11:10.609 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00070226499999999999 2024-09-13T10:11:10.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.609 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.609 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.609 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143793+0000", 2024-09-13T10:11:10.609 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00075876700000000004 2024-09-13T10:11:10.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.609 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.609 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.609 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143825+0000", 2024-09-13T10:11:10.609 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2017999999999997e-05 2024-09-13T10:11:10.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.610 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.610 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.610 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143849+0000", 2024-09-13T10:11:10.610 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3879999999999998e-05 2024-09-13T10:11:10.610 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.610 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.610 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.610 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.610 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.7.22:1 3.2 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.610 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.130657+0000", 2024-09-13T10:11:10.610 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.370049894000001, 2024-09-13T10:11:10.611 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021425745, 2024-09-13T10:11:10.611 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.611 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.611 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.611 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.611 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.611 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.611 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130657+0000", 2024-09-13T10:11:10.611 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.611 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.611 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.611 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.611 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130657+0000", 2024-09-13T10:11:10.611 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.612 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.612 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.612 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130660+0000", 2024-09-13T10:11:10.612 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.095e-06 2024-09-13T10:11:10.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.612 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.612 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.612 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130671+0000", 2024-09-13T10:11:10.612 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0455000000000001e-05 2024-09-13T10:11:10.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.612 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.612 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.612 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130684+0000", 2024-09-13T10:11:10.613 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2933e-05 2024-09-13T10:11:10.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.613 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.613 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130695+0000", 2024-09-13T10:11:10.613 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1138000000000001e-05 2024-09-13T10:11:10.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.613 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.613 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139699+0000", 2024-09-13T10:11:10.613 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0090041779999999998 2024-09-13T10:11:10.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.613 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.613 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139816+0000", 2024-09-13T10:11:10.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117484 2024-09-13T10:11:10.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.614 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.614 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.614 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.140032+0000", 2024-09-13T10:11:10.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021595699999999999 2024-09-13T10:11:10.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.614 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.614 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.614 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143538+0000", 2024-09-13T10:11:10.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0035057180000000001 2024-09-13T10:11:10.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.614 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.614 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.615 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.150610+0000", 2024-09-13T10:11:10.615 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0070718969999999997 2024-09-13T10:11:10.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.615 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.615 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.615 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152002+0000", 2024-09-13T10:11:10.615 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001391834 2024-09-13T10:11:10.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.615 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.615 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.615 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152049+0000", 2024-09-13T10:11:10.615 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.7097e-05 2024-09-13T10:11:10.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.615 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.616 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.616 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152083+0000", 2024-09-13T10:11:10.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3958999999999998e-05 2024-09-13T10:11:10.616 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.616 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.616 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.616 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.7.22:3 3.2 3:5915c927:::smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.616 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.216433+0000", 2024-09-13T10:11:10.616 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.284273911, 2024-09-13T10:11:10.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.037069469000000001, 2024-09-13T10:11:10.616 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.617 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.617 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.617 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.617 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216433+0000", 2024-09-13T10:11:10.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.617 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216433+0000", 2024-09-13T10:11:10.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.618 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.618 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.618 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216436+0000", 2024-09-13T10:11:10.618 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7800000000000001e-06 2024-09-13T10:11:10.618 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.618 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.618 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.618 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216473+0000", 2024-09-13T10:11:10.618 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6931999999999998e-05 2024-09-13T10:11:10.618 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.618 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.618 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.618 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216512+0000", 2024-09-13T10:11:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.8831999999999997e-05 2024-09-13T10:11:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216539+0000", 2024-09-13T10:11:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7705999999999999e-05 2024-09-13T10:11:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216732+0000", 2024-09-13T10:11:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019224500000000001 2024-09-13T10:11:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.619 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.620 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217000+0000", 2024-09-13T10:11:10.620 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026813100000000001 2024-09-13T10:11:10.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.620 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.620 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.620 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217638+0000", 2024-09-13T10:11:10.620 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00063874400000000001 2024-09-13T10:11:10.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.620 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.620 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.620 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.219940+0000", 2024-09-13T10:11:10.620 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023015890000000002 2024-09-13T10:11:10.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.620 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.620 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.621 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.220304+0000", 2024-09-13T10:11:10.621 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036399099999999998 2024-09-13T10:11:10.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.621 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.621 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.621 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253232+0000", 2024-09-13T10:11:10.621 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032927640000000001 2024-09-13T10:11:10.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.621 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.621 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.621 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253393+0000", 2024-09-13T10:11:10.621 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000161142 2024-09-13T10:11:10.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.622 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.622 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.622 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253503+0000", 2024-09-13T10:11:10.622 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000109737 2024-09-13T10:11:10.622 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.622 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.622 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.622 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.622 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.2.23:2 3.2 3:46ab5247:::smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.622 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.217995+0000", 2024-09-13T10:11:10.622 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.282712440000001, 2024-09-13T10:11:10.622 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036081690999999999, 2024-09-13T10:11:10.623 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.623 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.623 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.623 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.623 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.623 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.623 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217995+0000", 2024-09-13T10:11:10.623 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.623 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.623 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.623 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217995+0000", 2024-09-13T10:11:10.623 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.624 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.624 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217997+0000", 2024-09-13T10:11:10.624 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3429999999999998e-06 2024-09-13T10:11:10.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.624 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.624 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218007+0000", 2024-09-13T10:11:10.624 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.91e-06 2024-09-13T10:11:10.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.624 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.625 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218024+0000", 2024-09-13T10:11:10.625 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7549000000000001e-05 2024-09-13T10:11:10.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.625 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.625 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.625 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218038+0000", 2024-09-13T10:11:10.625 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3509999999999999e-05 2024-09-13T10:11:10.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.625 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.625 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.625 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218384+0000", 2024-09-13T10:11:10.625 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00034618499999999999 2024-09-13T10:11:10.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.626 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.626 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218605+0000", 2024-09-13T10:11:10.626 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022059400000000001 2024-09-13T10:11:10.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.626 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.626 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.219124+0000", 2024-09-13T10:11:10.626 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00051934699999999997 2024-09-13T10:11:10.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.626 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.627 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.222285+0000", 2024-09-13T10:11:10.627 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0031605790000000002 2024-09-13T10:11:10.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.627 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.627 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.627 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.224618+0000", 2024-09-13T10:11:10.627 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023338209999999998 2024-09-13T10:11:10.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.627 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.627 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.627 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253766+0000", 2024-09-13T10:11:10.627 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029147890999999999 2024-09-13T10:11:10.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.628 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.628 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.628 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253920+0000", 2024-09-13T10:11:10.628 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015327299999999999 2024-09-13T10:11:10.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.628 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.628 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.628 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254076+0000", 2024-09-13T10:11:10.628 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000156689 2024-09-13T10:11:10.628 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.628 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.628 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.629 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.8.23:1 3.2 3:73c2893b:::smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:10.629 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.252722+0000", 2024-09-13T10:11:10.629 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.247984946999999, 2024-09-13T10:11:10.629 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.019815335, 2024-09-13T10:11:10.629 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:10.629 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:10.629 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:10.629 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:10.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.629 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:10.629 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.252722+0000", 2024-09-13T10:11:10.629 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.630 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:10.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.252722+0000", 2024-09-13T10:11:10.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:10.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.630 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:10.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.252724+0000", 2024-09-13T10:11:10.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1490000000000001e-06 2024-09-13T10:11:10.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.630 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:10.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.252734+0000", 2024-09-13T10:11:10.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.3850000000000008e-06 2024-09-13T10:11:10.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.631 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:10.631 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.252752+0000", 2024-09-13T10:11:10.631 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8283000000000001e-05 2024-09-13T10:11:10.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.631 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:10.631 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.252766+0000", 2024-09-13T10:11:10.631 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4056999999999999e-05 2024-09-13T10:11:10.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.631 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:10.631 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262418+0000", 2024-09-13T10:11:10.631 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0096517010000000004 2024-09-13T10:11:10.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.632 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.632 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:10.632 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262653+0000", 2024-09-13T10:11:10.632 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023533500000000001 2024-09-13T10:11:10.632 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.632 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.632 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:10.632 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.263202+0000", 2024-09-13T10:11:10.632 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00054865399999999998 2024-09-13T10:11:10.632 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.632 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.632 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.632 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267428+0000", 2024-09-13T10:11:10.632 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.004226601 2024-09-13T10:11:10.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.633 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:10.633 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.268826+0000", 2024-09-13T10:11:10.633 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0013975389999999999 2024-09-13T10:11:10.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.633 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:10.633 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.272327+0000", 2024-09-13T10:11:10.633 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0035015689999999999 2024-09-13T10:11:10.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.633 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:10.633 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.272453+0000", 2024-09-13T10:11:10.633 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000125902 2024-09-13T10:11:10.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:10.634 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:10.634 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:10.634 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.272537+0000", 2024-09-13T10:11:10.634 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.4159999999999994e-05 2024-09-13T10:11:10.634 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.634 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.634 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.634 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:10.634 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:10.634 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:10.635 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:10.636 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T10:11:10.637 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:10.636+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:10.643 INFO:tasks.rados.rados.0.smithi195.stdout:963: seq_num 159 ranges {672313=691651,1887931=419038} 2024-09-13T10:11:10.654 INFO:tasks.rados.rados.0.smithi195.stdout:963: writing smithi19530714-28 from 672313 to 1363964 tid 1 2024-09-13T10:11:10.661 INFO:tasks.rados.rados.0.smithi195.stdout:963: writing smithi19530714-28 from 1887931 to 2306969 tid 2 2024-09-13T10:11:10.661 INFO:tasks.rados.rados.0.smithi195.stdout:949: done (14 left) 2024-09-13T10:11:10.661 INFO:tasks.rados.rados.0.smithi195.stdout:950: done (13 left) 2024-09-13T10:11:10.661 INFO:tasks.rados.rados.0.smithi195.stdout:951: done (12 left) 2024-09-13T10:11:10.661 INFO:tasks.rados.rados.0.smithi195.stdout:952: done (11 left) 2024-09-13T10:11:10.661 INFO:tasks.rados.rados.0.smithi195.stdout:953: done (10 left) 2024-09-13T10:11:10.662 INFO:tasks.rados.rados.0.smithi195.stdout:954: done (9 left) 2024-09-13T10:11:10.662 INFO:tasks.rados.rados.0.smithi195.stdout:955: done (8 left) 2024-09-13T10:11:10.662 INFO:tasks.rados.rados.0.smithi195.stdout:956: done (7 left) 2024-09-13T10:11:10.662 INFO:tasks.rados.rados.0.smithi195.stdout:957: done (6 left) 2024-09-13T10:11:10.662 INFO:tasks.rados.rados.0.smithi195.stdout:958: done (5 left) 2024-09-13T10:11:10.662 INFO:tasks.rados.rados.0.smithi195.stdout:959: done (4 left) 2024-09-13T10:11:10.662 INFO:tasks.rados.rados.0.smithi195.stdout:960: done (3 left) 2024-09-13T10:11:10.662 INFO:tasks.rados.rados.0.smithi195.stdout:961: done (2 left) 2024-09-13T10:11:10.662 INFO:tasks.rados.rados.0.smithi195.stdout:962: done (1 left) 2024-09-13T10:11:10.662 INFO:tasks.rados.rados.0.smithi195.stdout:964: set_chunk oid 8 target oid 2024-09-13T10:11:10.691 INFO:tasks.rados.rados.0.smithi195.stdout:964: set_chunk oid 8 offset: 241152 length: 2655744 target oid e22ed036f41f6a390924cf1e5e56fa98e0edea9e6c33080c396679f1537fbc7a offset: 0 snap -1 2024-09-13T10:11:10.691 INFO:tasks.rados.rados.0.smithi195.stdout:965: write oid 19 current snap is 0 2024-09-13T10:11:10.691 INFO:tasks.rados.rados.0.smithi195.stdout:963: finishing write tid 1 to smithi19530714-28 2024-09-13T10:11:10.691 INFO:tasks.rados.rados.0.smithi195.stdout:965: seq_num 160 ranges {689120=483253,1946413=679049} 2024-09-13T10:11:10.691 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:10.692 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:10.699 INFO:tasks.rados.rados.0.smithi195.stdout:965: writing smithi19530714-19 from 689120 to 1172373 tid 1 2024-09-13T10:11:10.710 INFO:tasks.rados.rados.0.smithi195.stdout:965: writing smithi19530714-19 from 1946413 to 2625462 tid 2 2024-09-13T10:11:10.710 INFO:tasks.rados.rados.0.smithi195.stdout:966: write (excl) oid 1 current snap is 0 2024-09-13T10:11:10.710 INFO:tasks.rados.rados.0.smithi195.stdout:963: finishing write tid 2 to smithi19530714-28 2024-09-13T10:11:10.710 INFO:tasks.rados.rados.0.smithi195.stdout:963: finishing write tid 3 to smithi19530714-28 2024-09-13T10:11:10.710 INFO:tasks.rados.rados.0.smithi195.stdout:966: seq_num 161 ranges {705927=558502,2004895=422707} 2024-09-13T10:11:10.720 INFO:tasks.rados.rados.0.smithi195.stdout:966: writing smithi19530714-1 from 705927 to 1264429 tid 1 2024-09-13T10:11:10.729 INFO:tasks.rados.rados.0.smithi195.stdout:966: writing smithi19530714-1 from 2004895 to 2427602 tid 2 2024-09-13T10:11:10.729 INFO:tasks.rados.rados.0.smithi195.stdout:967: read oid 50 snap -1 2024-09-13T10:11:10.729 INFO:tasks.rados.rados.0.smithi195.stdout:967: expect (ObjNum 156 snap 0 seq_num 156) 2024-09-13T10:11:10.729 INFO:tasks.rados.rados.0.smithi195.stdout:968: write oid 21 current snap is 0 2024-09-13T10:11:10.729 INFO:tasks.rados.rados.0.smithi195.stdout:968: seq_num 162 ranges {722734=633751,2063377=659357} 2024-09-13T10:11:10.736 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:10.735+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:10.738 INFO:tasks.rados.rados.0.smithi195.stdout:968: writing smithi19530714-21 from 722734 to 1356485 tid 1 2024-09-13T10:11:10.748 INFO:tasks.rados.rados.0.smithi195.stdout:968: writing smithi19530714-21 from 2063377 to 2722734 tid 2 2024-09-13T10:11:10.748 INFO:tasks.rados.rados.0.smithi195.stdout:969: tier_evict oid 2 2024-09-13T10:11:10.748 INFO:tasks.rados.rados.0.smithi195.stdout:969: tier_evict oid 2 snap -1 2024-09-13T10:11:10.748 INFO:tasks.rados.rados.0.smithi195.stdout:970: tier_flush oid 44 2024-09-13T10:11:10.748 INFO:tasks.rados.rados.0.smithi195.stdout:970: tier_flush oid 44 snap -1 2024-09-13T10:11:10.749 INFO:tasks.rados.rados.0.smithi195.stdout:963: finishing write tid 4 to smithi19530714-28 2024-09-13T10:11:10.749 INFO:tasks.rados.rados.0.smithi195.stdout:963: oid 28 updating version 0 to 88 2024-09-13T10:11:10.749 INFO:tasks.rados.rados.0.smithi195.stdout:963: oid 28 version 88 is already newer than 87 2024-09-13T10:11:10.749 INFO:tasks.rados.rados.0.smithi195.stdout:963: oid 28 version 88 is already newer than 86 2024-09-13T10:11:10.749 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 88 (ObjNum 159 snap 0 seq_num 159) dirty exists 2024-09-13T10:11:10.749 INFO:tasks.rados.rados.0.smithi195.stdout:963: left oid 28 (ObjNum 159 snap 0 seq_num 159) 2024-09-13T10:11:10.749 INFO:tasks.rados.rados.0.smithi195.stdout:963: done (7 left) 2024-09-13T10:11:10.749 INFO:tasks.rados.rados.0.smithi195.stdout:971: write oid 4 current snap is 0 2024-09-13T10:11:10.749 INFO:tasks.rados.rados.0.smithi195.stdout:971: seq_num 163 ranges {739541=709000,2121859=426376} 2024-09-13T10:11:10.757 INFO:tasks.rados.rados.0.smithi195.stdout:971: writing smithi19530714-4 from 739541 to 1448541 tid 1 2024-09-13T10:11:10.762 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:10.762 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:10.762 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:11:10.762 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:10.764 INFO:tasks.rados.rados.0.smithi195.stdout:971: writing smithi19530714-4 from 2121859 to 2548235 tid 2 2024-09-13T10:11:10.764 INFO:tasks.rados.rados.0.smithi195.stdout:972: read oid 42 snap -1 2024-09-13T10:11:10.764 INFO:tasks.rados.rados.0.smithi195.stdout:972: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-13T10:11:10.764 INFO:tasks.rados.rados.0.smithi195.stdout:973: read oid 14 snap -1 2024-09-13T10:11:10.764 INFO:tasks.rados.rados.0.smithi195.stdout:973: expect (ObjNum 103 snap 0 seq_num 103) 2024-09-13T10:11:10.764 INFO:tasks.rados.rados.0.smithi195.stdout:974: read oid 26 snap -1 2024-09-13T10:11:10.765 INFO:tasks.rados.rados.0.smithi195.stdout:974: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:10.765 INFO:tasks.rados.rados.0.smithi195.stdout:975: set_chunk oid 37 target oid 2024-09-13T10:11:10.778 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T10:11:10.788 INFO:tasks.rados.rados.0.smithi195.stdout:975: set_chunk oid 37 offset: 764928 length: 1436672 target oid 2e245f28c7be6cd897f9ca1085e42f40d08eff14d275d687f139c0a64a00b42a offset: 0 snap -1 2024-09-13T10:11:10.788 INFO:tasks.rados.rados.0.smithi195.stdout:976: read oid 5 snap -1 2024-09-13T10:11:10.788 INFO:tasks.rados.rados.0.smithi195.stdout:976: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-13T10:11:10.788 INFO:tasks.rados.rados.0.smithi195.stdout:977: read oid 33 snap -1 2024-09-13T10:11:10.788 INFO:tasks.rados.rados.0.smithi195.stdout:977: expect (ObjNum 137 snap 0 seq_num 137) 2024-09-13T10:11:10.788 INFO:tasks.rados.rados.0.smithi195.stdout:978: write (excl) oid 11 current snap is 0 2024-09-13T10:11:10.788 INFO:tasks.rados.rados.0.smithi195.stdout:964: finishing set_chunk to oid 8 2024-09-13T10:11:10.788 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 8 set_chunk returned error code -95 offset: 241152 length: 2655744 2024-09-13T10:11:10.788 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 0 (ObjNum 104 snap 0 seq_num 104) dirty exists 2024-09-13T10:11:10.788 INFO:tasks.rados.rados.0.smithi195.stdout:965: finishing write tid 1 to smithi19530714-19 2024-09-13T10:11:10.788 INFO:tasks.rados.rados.0.smithi195.stdout:965: finishing write tid 2 to smithi19530714-19 2024-09-13T10:11:10.789 INFO:tasks.rados.rados.0.smithi195.stdout:965: finishing write tid 3 to smithi19530714-19 2024-09-13T10:11:10.789 INFO:tasks.rados.rados.0.smithi195.stdout:965: finishing write tid 4 to smithi19530714-19 2024-09-13T10:11:10.789 INFO:tasks.rados.rados.0.smithi195.stdout:965: oid 19 updating version 0 to 49 2024-09-13T10:11:10.789 INFO:tasks.rados.rados.0.smithi195.stdout:965: oid 19 version 49 is already newer than 47 2024-09-13T10:11:10.789 INFO:tasks.rados.rados.0.smithi195.stdout:965: oid 19 version 49 is already newer than 48 2024-09-13T10:11:10.789 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 49 (ObjNum 160 snap 0 seq_num 160) dirty exists 2024-09-13T10:11:10.789 INFO:tasks.rados.rados.0.smithi195.stdout:965: left oid 19 (ObjNum 160 snap 0 seq_num 160) 2024-09-13T10:11:10.789 INFO:tasks.rados.rados.0.smithi195.stdout:966: finishing write tid 1 to smithi19530714-1 2024-09-13T10:11:10.789 INFO:tasks.rados.rados.0.smithi195.stdout:966: finishing write tid 2 to smithi19530714-1 2024-09-13T10:11:10.789 INFO:tasks.rados.rados.0.smithi195.stdout:966: finishing write tid 3 to smithi19530714-1 2024-09-13T10:11:10.789 INFO:tasks.rados.rados.0.smithi195.stdout:966: finishing write tid 4 to smithi19530714-1 2024-09-13T10:11:10.789 INFO:tasks.rados.rados.0.smithi195.stdout:966: oid 1 updating version 0 to 50 2024-09-13T10:11:10.789 INFO:tasks.rados.rados.0.smithi195.stdout:966: oid 1 updating version 50 to 51 2024-09-13T10:11:10.789 INFO:tasks.rados.rados.0.smithi195.stdout:966: oid 1 updating version 51 to 52 2024-09-13T10:11:10.789 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 52 (ObjNum 161 snap 0 seq_num 161) dirty exists 2024-09-13T10:11:10.790 INFO:tasks.rados.rados.0.smithi195.stdout:966: left oid 1 (ObjNum 161 snap 0 seq_num 161) 2024-09-13T10:11:10.790 INFO:tasks.rados.rados.0.smithi195.stdout:967: expect (ObjNum 156 snap 0 seq_num 156) 2024-09-13T10:11:10.838 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:10.836+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:10.866 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:10.897 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:11:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:11:10.898 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:10.909 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T10:11:10.939 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:10.936+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:10.992 INFO:tasks.rados.rados.0.smithi195.stdout:968: finishing write tid 1 to smithi19530714-21 2024-09-13T10:11:10.992 INFO:tasks.rados.rados.0.smithi195.stdout:969: got (22) Invalid argument 2024-09-13T10:11:10.992 INFO:tasks.rados.rados.0.smithi195.stdout:968: finishing write tid 2 to smithi19530714-21 2024-09-13T10:11:10.992 INFO:tasks.rados.rados.0.smithi195.stdout:968: finishing write tid 3 to smithi19530714-21 2024-09-13T10:11:10.993 INFO:tasks.rados.rados.0.smithi195.stdout:968: finishing write tid 4 to smithi19530714-21 2024-09-13T10:11:10.993 INFO:tasks.rados.rados.0.smithi195.stdout:968: oid 21 updating version 0 to 89 2024-09-13T10:11:10.993 INFO:tasks.rados.rados.0.smithi195.stdout:968: oid 21 version 89 is already newer than 88 2024-09-13T10:11:10.993 INFO:tasks.rados.rados.0.smithi195.stdout:968: oid 21 version 89 is already newer than 87 2024-09-13T10:11:10.993 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 89 (ObjNum 162 snap 0 seq_num 162) dirty exists 2024-09-13T10:11:10.993 INFO:tasks.rados.rados.0.smithi195.stdout:968: left oid 21 (ObjNum 162 snap 0 seq_num 162) 2024-09-13T10:11:10.993 INFO:tasks.rados.rados.0.smithi195.stdout:973: expect (ObjNum 103 snap 0 seq_num 103) 2024-09-13T10:11:11.030 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:11:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:11:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:11:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:207 3.0 3:332cd44b:::smithi19530714-8 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:11:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.037088402999998, 2024-09-13T10:11:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074034500000000003, 2024-09-13T10:11:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 207 2024-09-13T10:11:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:11:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:11:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991271+0000", 2024-09-13T10:11:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9289999999999998e-06 2024-09-13T10:11:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991284+0000", 2024-09-13T10:11:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2597e-05 2024-09-13T10:11:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991313+0000", 2024-09-13T10:11:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8982000000000001e-05 2024-09-13T10:11:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991334+0000", 2024-09-13T10:11:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1046000000000001e-05 2024-09-13T10:11:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991416+0000", 2024-09-13T10:11:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.153e-05 2024-09-13T10:11:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991504+0000", 2024-09-13T10:11:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8220999999999994e-05 2024-09-13T10:11:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017232+0000", 2024-09-13T10:11:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025728328000000002 2024-09-13T10:11:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.031588+0000", 2024-09-13T10:11:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014355557 2024-09-13T10:11:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034211+0000", 2024-09-13T10:11:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026232790000000001 2024-09-13T10:11:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065168+0000", 2024-09-13T10:11:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030956833999999999 2024-09-13T10:11:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065260+0000", 2024-09-13T10:11:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2044000000000002e-05 2024-09-13T10:11:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065303+0000", 2024-09-13T10:11:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3152999999999998e-05 2024-09-13T10:11:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:211 3.0 3:0596651d:::smithi19530714-12 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:11:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.036365024999998, 2024-09-13T10:11:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.042923122000000001, 2024-09-13T10:11:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 211 2024-09-13T10:11:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:11:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:11:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991995+0000", 2024-09-13T10:11:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7640000000000001e-06 2024-09-13T10:11:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992007+0000", 2024-09-13T10:11:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2058999999999999e-05 2024-09-13T10:11:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992040+0000", 2024-09-13T10:11:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3466000000000001e-05 2024-09-13T10:11:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992059+0000", 2024-09-13T10:11:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8485999999999999e-05 2024-09-13T10:11:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992136+0000", 2024-09-13T10:11:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6994000000000002e-05 2024-09-13T10:11:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992207+0000", 2024-09-13T10:11:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1357000000000001e-05 2024-09-13T10:11:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.006377+0000", 2024-09-13T10:11:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014170455 2024-09-13T10:11:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.022844+0000", 2024-09-13T10:11:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016466275999999998 2024-09-13T10:11:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023031+0000", 2024-09-13T10:11:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018686000000000001 2024-09-13T10:11:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034788+0000", 2024-09-13T10:11:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011756967 2024-09-13T10:11:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034857+0000", 2024-09-13T10:11:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9169000000000001e-05 2024-09-13T10:11:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034915+0000", 2024-09-13T10:11:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8269000000000001e-05 2024-09-13T10:11:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:212 3.0 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:11:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.036258692000001, 2024-09-13T10:11:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.073011809999999996, 2024-09-13T10:11:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 212 2024-09-13T10:11:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:11:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:11:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992099+0000", 2024-09-13T10:11:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4899999999999995e-07 2024-09-13T10:11:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992103+0000", 2024-09-13T10:11:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.33e-06 2024-09-13T10:11:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992110+0000", 2024-09-13T10:11:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8399999999999997e-06 2024-09-13T10:11:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992116+0000", 2024-09-13T10:11:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6790000000000002e-06 2024-09-13T10:11:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992292+0000", 2024-09-13T10:11:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017659199999999999 2024-09-13T10:11:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992385+0000", 2024-09-13T10:11:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2608000000000005e-05 2024-09-13T10:11:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011428+0000", 2024-09-13T10:11:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.019042631000000001 2024-09-13T10:11:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.022852+0000", 2024-09-13T10:11:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011424528999999999 2024-09-13T10:11:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.031660+0000", 2024-09-13T10:11:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088079899999999999 2024-09-13T10:11:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065032+0000", 2024-09-13T10:11:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033372247000000001 2024-09-13T10:11:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065073+0000", 2024-09-13T10:11:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0598e-05 2024-09-13T10:11:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065110+0000", 2024-09-13T10:11:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7116999999999997e-05 2024-09-13T10:11:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:202 3.2 e38/30 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:11:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.026768086000001, 2024-09-13T10:11:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017280298999999999, 2024-09-13T10:11:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:11:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:11:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001594+0000", 2024-09-13T10:11:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5550000000000002e-06 2024-09-13T10:11:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009486+0000", 2024-09-13T10:11:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078916799999999999 2024-09-13T10:11:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009519+0000", 2024-09-13T10:11:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2898999999999998e-05 2024-09-13T10:11:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009540+0000", 2024-09-13T10:11:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1559000000000001e-05 2024-09-13T10:11:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009655+0000", 2024-09-13T10:11:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114271 2024-09-13T10:11:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009683+0000", 2024-09-13T10:11:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8379000000000001e-05 2024-09-13T10:11:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.018766+0000", 2024-09-13T10:11:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0090829589999999998 2024-09-13T10:11:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.018869+0000", 2024-09-13T10:11:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102997 2024-09-13T10:11:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:216 3.0 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:11:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.023755550000001, 2024-09-13T10:11:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.063936142000000001, 2024-09-13T10:11:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 216 2024-09-13T10:11:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:11:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:11:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004603+0000", 2024-09-13T10:11:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5540000000000001e-06 2024-09-13T10:11:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004615+0000", 2024-09-13T10:11:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2597e-05 2024-09-13T10:11:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004637+0000", 2024-09-13T10:11:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1713e-05 2024-09-13T10:11:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004651+0000", 2024-09-13T10:11:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3878e-05 2024-09-13T10:11:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004985+0000", 2024-09-13T10:11:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033347299999999998 2024-09-13T10:11:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.005706+0000", 2024-09-13T10:11:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00072122199999999999 2024-09-13T10:11:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023801+0000", 2024-09-13T10:11:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018095466000000001 2024-09-13T10:11:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.032153+0000", 2024-09-13T10:11:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083516010000000002 2024-09-13T10:11:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.035250+0000", 2024-09-13T10:11:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030974409999999998 2024-09-13T10:11:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068439+0000", 2024-09-13T10:11:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033189042000000002 2024-09-13T10:11:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068499+0000", 2024-09-13T10:11:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9969000000000001e-05 2024-09-13T10:11:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068537+0000", 2024-09-13T10:11:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8186000000000003e-05 2024-09-13T10:11:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:221 3.0 3:18ae852b:::smithi19530714-22 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:11:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.989831042999999, 2024-09-13T10:11:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039809298, 2024-09-13T10:11:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 221 2024-09-13T10:11:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:11:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:11:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038528+0000", 2024-09-13T10:11:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8819999999999999e-06 2024-09-13T10:11:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038534+0000", 2024-09-13T10:11:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0730000000000001e-06 2024-09-13T10:11:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038546+0000", 2024-09-13T10:11:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.219e-05 2024-09-13T10:11:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038566+0000", 2024-09-13T10:11:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9867000000000001e-05 2024-09-13T10:11:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038608+0000", 2024-09-13T10:11:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1977999999999997e-05 2024-09-13T10:11:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038707+0000", 2024-09-13T10:11:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8708000000000004e-05 2024-09-13T10:11:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.057374+0000", 2024-09-13T10:11:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018667652999999999 2024-09-13T10:11:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061983+0000", 2024-09-13T10:11:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0046087089999999999 2024-09-13T10:11:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.063027+0000", 2024-09-13T10:11:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001044239 2024-09-13T10:11:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078233+0000", 2024-09-13T10:11:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015205639999999999 2024-09-13T10:11:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078298+0000", 2024-09-13T10:11:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5180999999999998e-05 2024-09-13T10:11:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078335+0000", 2024-09-13T10:11:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7178000000000002e-05 2024-09-13T10:11:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:225 3.0 3:26960251:::smithi19530714-26 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:11:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.969904768999999, 2024-09-13T10:11:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030704875999999999, 2024-09-13T10:11:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 225 2024-09-13T10:11:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:11:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:11:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058455+0000", 2024-09-13T10:11:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3220000000000001e-06 2024-09-13T10:11:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058467+0000", 2024-09-13T10:11:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1987999999999999e-05 2024-09-13T10:11:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058502+0000", 2024-09-13T10:11:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4873000000000003e-05 2024-09-13T10:11:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058520+0000", 2024-09-13T10:11:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7977000000000001e-05 2024-09-13T10:11:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060066+0000", 2024-09-13T10:11:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001545261 2024-09-13T10:11:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060680+0000", 2024-09-13T10:11:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00061431799999999998 2024-09-13T10:11:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070650+0000", 2024-09-13T10:11:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0099703099999999996 2024-09-13T10:11:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078192+0000", 2024-09-13T10:11:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0075419709999999997 2024-09-13T10:11:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084486+0000", 2024-09-13T10:11:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0062943779999999998 2024-09-13T10:11:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089029+0000", 2024-09-13T10:11:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045422379999999997 2024-09-13T10:11:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089092+0000", 2024-09-13T10:11:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3435999999999994e-05 2024-09-13T10:11:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089157+0000", 2024-09-13T10:11:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4803999999999998e-05 2024-09-13T10:11:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:228 3.0 3:0168a995:::smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:11:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.968103810999999, 2024-09-13T10:11:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030139446, 2024-09-13T10:11:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 228 2024-09-13T10:11:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:11:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:11:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060255+0000", 2024-09-13T10:11:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.787e-06 2024-09-13T10:11:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060262+0000", 2024-09-13T10:11:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8630000000000002e-06 2024-09-13T10:11:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060285+0000", 2024-09-13T10:11:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2935e-05 2024-09-13T10:11:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060294+0000", 2024-09-13T10:11:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3940000000000002e-06 2024-09-13T10:11:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061041+0000", 2024-09-13T10:11:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00074706700000000002 2024-09-13T10:11:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061148+0000", 2024-09-13T10:11:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107364 2024-09-13T10:11:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074199+0000", 2024-09-13T10:11:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013050447999999999 2024-09-13T10:11:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084506+0000", 2024-09-13T10:11:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010306789 2024-09-13T10:11:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089222+0000", 2024-09-13T10:11:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047163480000000004 2024-09-13T10:11:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090310+0000", 2024-09-13T10:11:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010880919999999999 2024-09-13T10:11:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090356+0000", 2024-09-13T10:11:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5522000000000002e-05 2024-09-13T10:11:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090393+0000", 2024-09-13T10:11:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6837e-05 2024-09-13T10:11:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:229 3.0 3:1e6cc58c:::smithi19530714-30 low_tier:head [copy-from ver 18] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:11:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.966797436, 2024-09-13T10:11:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033410479, 2024-09-13T10:11:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 229 2024-09-13T10:11:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:11:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:11:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061562+0000", 2024-09-13T10:11:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9730000000000002e-06 2024-09-13T10:11:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061577+0000", 2024-09-13T10:11:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5044e-05 2024-09-13T10:11:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061602+0000", 2024-09-13T10:11:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4494999999999998e-05 2024-09-13T10:11:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061615+0000", 2024-09-13T10:11:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3207000000000001e-05 2024-09-13T10:11:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061701+0000", 2024-09-13T10:11:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5694999999999999e-05 2024-09-13T10:11:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061871+0000", 2024-09-13T10:11:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017004100000000001 2024-09-13T10:11:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.079511+0000", 2024-09-13T10:11:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017639708 2024-09-13T10:11:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089295+0000", 2024-09-13T10:11:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0097846359999999993 2024-09-13T10:11:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089316+0000", 2024-09-13T10:11:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0288999999999998e-05 2024-09-13T10:11:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094875+0000", 2024-09-13T10:11:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0055598389999999996 2024-09-13T10:11:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094931+0000", 2024-09-13T10:11:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5556999999999997e-05 2024-09-13T10:11:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094970+0000", 2024-09-13T10:11:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8995000000000002e-05 2024-09-13T10:11:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:233 3.0 3:24ead787:::smithi19530714-34 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:11:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.962620932, 2024-09-13T10:11:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047200234000000001, 2024-09-13T10:11:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 233 2024-09-13T10:11:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:11:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:11:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065737+0000", 2024-09-13T10:11:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.283e-06 2024-09-13T10:11:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065745+0000", 2024-09-13T10:11:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3130000000000001e-06 2024-09-13T10:11:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065757+0000", 2024-09-13T10:11:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2627e-05 2024-09-13T10:11:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065765+0000", 2024-09-13T10:11:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4669999999999998e-06 2024-09-13T10:11:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065816+0000", 2024-09-13T10:11:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1545000000000002e-05 2024-09-13T10:11:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065940+0000", 2024-09-13T10:11:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012366699999999999 2024-09-13T10:11:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.086064+0000", 2024-09-13T10:11:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.020124487 2024-09-13T10:11:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090747+0000", 2024-09-13T10:11:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004682795 2024-09-13T10:11:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.091235+0000", 2024-09-13T10:11:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048779900000000002 2024-09-13T10:11:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112809+0000", 2024-09-13T10:11:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.021573967999999999 2024-09-13T10:11:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112878+0000", 2024-09-13T10:11:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9413999999999996e-05 2024-09-13T10:11:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112936+0000", 2024-09-13T10:11:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7868999999999998e-05 2024-09-13T10:11:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:227 3.2 e38/30 3:6ac7800c:::smithi19530714-28 low_tier:head v 38'8, mlcod=38'6)", 2024-09-13T10:11:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:11:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.954257868999999, 2024-09-13T10:11:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014433425999999999, 2024-09-13T10:11:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:11:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:11:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074101+0000", 2024-09-13T10:11:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.511e-06 2024-09-13T10:11:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083754+0000", 2024-09-13T10:11:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096533799999999996 2024-09-13T10:11:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083772+0000", 2024-09-13T10:11:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8241e-05 2024-09-13T10:11:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083782+0000", 2024-09-13T10:11:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.004e-05 2024-09-13T10:11:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083875+0000", 2024-09-13T10:11:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.268e-05 2024-09-13T10:11:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083918+0000", 2024-09-13T10:11:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-13T10:11:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.088456+0000", 2024-09-13T10:11:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045385540000000002 2024-09-13T10:11:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.088532+0000", 2024-09-13T10:11:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6069000000000006e-05 2024-09-13T10:11:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:239 3.0 3:078df4f4:::smithi19530714-40 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:11:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.939054357, 2024-09-13T10:11:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025192791999999999, 2024-09-13T10:11:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 239 2024-09-13T10:11:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:11:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:11:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089305+0000", 2024-09-13T10:11:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1710000000000001e-06 2024-09-13T10:11:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089312+0000", 2024-09-13T10:11:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6860000000000006e-06 2024-09-13T10:11:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089323+0000", 2024-09-13T10:11:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0467e-05 2024-09-13T10:11:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089330+0000", 2024-09-13T10:11:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3939999999999998e-06 2024-09-13T10:11:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089379+0000", 2024-09-13T10:11:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8990999999999999e-05 2024-09-13T10:11:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089543+0000", 2024-09-13T10:11:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016351700000000001 2024-09-13T10:11:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.098886+0000", 2024-09-13T10:11:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0093434980000000004 2024-09-13T10:11:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.105855+0000", 2024-09-13T10:11:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.006968279 2024-09-13T10:11:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.105868+0000", 2024-09-13T10:11:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3560999999999999e-05 2024-09-13T10:11:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114382+0000", 2024-09-13T10:11:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0085134730000000006 2024-09-13T10:11:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114458+0000", 2024-09-13T10:11:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6143999999999995e-05 2024-09-13T10:11:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114495+0000", 2024-09-13T10:11:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7611000000000003e-05 2024-09-13T10:11:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:232 3.2 e38/30 3:6538713c:::smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'10, mlcod=38'6)", 2024-09-13T10:11:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.093933+0000", 2024-09-13T10:11:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.934424322000002, 2024-09-13T10:11:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012304116, 2024-09-13T10:11:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.093933+0000", 2024-09-13T10:11:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.093933+0000", 2024-09-13T10:11:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.093933+0000", 2024-09-13T10:11:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.08e-07 2024-09-13T10:11:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.100900+0000", 2024-09-13T10:11:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0069670310000000003 2024-09-13T10:11:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.100928+0000", 2024-09-13T10:11:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8416999999999999e-05 2024-09-13T10:11:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.100946+0000", 2024-09-13T10:11:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7643e-05 2024-09-13T10:11:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.101024+0000", 2024-09-13T10:11:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8103999999999996e-05 2024-09-13T10:11:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.101050+0000", 2024-09-13T10:11:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5451000000000001e-05 2024-09-13T10:11:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.106184+0000", 2024-09-13T10:11:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0051339890000000003 2024-09-13T10:11:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.106237+0000", 2024-09-13T10:11:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3072999999999998e-05 2024-09-13T10:11:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:241 3.0 3:388ca573:::smithi19530714-42 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:11:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.933180634999999, 2024-09-13T10:11:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057595649999999998, 2024-09-13T10:11:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 241 2024-09-13T10:11:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:11:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:11:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095178+0000", 2024-09-13T10:11:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.345e-06 2024-09-13T10:11:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095184+0000", 2024-09-13T10:11:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4239999999999998e-06 2024-09-13T10:11:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095198+0000", 2024-09-13T10:11:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4037e-05 2024-09-13T10:11:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095206+0000", 2024-09-13T10:11:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1629999999999996e-06 2024-09-13T10:11:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095274+0000", 2024-09-13T10:11:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7948999999999996e-05 2024-09-13T10:11:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095485+0000", 2024-09-13T10:11:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021050100000000001 2024-09-13T10:11:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.134651+0000", 2024-09-13T10:11:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039166200999999998 2024-09-13T10:11:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.140536+0000", 2024-09-13T10:11:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058849719999999996 2024-09-13T10:11:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.140980+0000", 2024-09-13T10:11:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044394200000000001 2024-09-13T10:11:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152700+0000", 2024-09-13T10:11:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011720431999999999 2024-09-13T10:11:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152739+0000", 2024-09-13T10:11:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8448999999999997e-05 2024-09-13T10:11:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152772+0000", 2024-09-13T10:11:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3235000000000003e-05 2024-09-13T10:11:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:247 3.0 3:0b30fd85:::smithi19530714-48 low_tier:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:11:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.915280983999999, 2024-09-13T10:11:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032448926000000003, 2024-09-13T10:11:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 247 2024-09-13T10:11:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:11:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:11:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113078+0000", 2024-09-13T10:11:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8670000000000001e-06 2024-09-13T10:11:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113086+0000", 2024-09-13T10:11:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2260000000000005e-06 2024-09-13T10:11:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113104+0000", 2024-09-13T10:11:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7887e-05 2024-09-13T10:11:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113115+0000", 2024-09-13T10:11:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0763e-05 2024-09-13T10:11:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113163+0000", 2024-09-13T10:11:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8541999999999997e-05 2024-09-13T10:11:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113322+0000", 2024-09-13T10:11:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000159121 2024-09-13T10:11:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.125165+0000", 2024-09-13T10:11:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011842320999999999 2024-09-13T10:11:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.131189+0000", 2024-09-13T10:11:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.006024728 2024-09-13T10:11:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.131203+0000", 2024-09-13T10:11:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3551e-05 2024-09-13T10:11:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145435+0000", 2024-09-13T10:11:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014232273 2024-09-13T10:11:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145481+0000", 2024-09-13T10:11:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5918000000000003e-05 2024-09-13T10:11:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145525+0000", 2024-09-13T10:11:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3729000000000001e-05 2024-09-13T10:11:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:244 3.2 e38/30 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'14, mlcod=38'10)", 2024-09-13T10:11:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:11:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.901815181, 2024-09-13T10:11:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014967827, 2024-09-13T10:11:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:11:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:11:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126544+0000", 2024-09-13T10:11:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.03e-06 2024-09-13T10:11:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133374+0000", 2024-09-13T10:11:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068299119999999996 2024-09-13T10:11:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133398+0000", 2024-09-13T10:11:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4209000000000001e-05 2024-09-13T10:11:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133404+0000", 2024-09-13T10:11:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3570000000000003e-06 2024-09-13T10:11:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133457+0000", 2024-09-13T10:11:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2320000000000001e-05 2024-09-13T10:11:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133477+0000", 2024-09-13T10:11:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0721999999999999e-05 2024-09-13T10:11:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.141475+0000", 2024-09-13T10:11:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0079973649999999993 2024-09-13T10:11:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.141510+0000", 2024-09-13T10:11:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4912e-05 2024-09-13T10:11:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:254 4.8 4:1de9da87:::smithi19530714-5:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.214446+0000", 2024-09-13T10:11:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.813910842999999, 2024-09-13T10:11:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014408274, 2024-09-13T10:11:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 254 2024-09-13T10:11:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214446+0000", 2024-09-13T10:11:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214446+0000", 2024-09-13T10:11:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214449+0000", 2024-09-13T10:11:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7860000000000001e-06 2024-09-13T10:11:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214464+0000", 2024-09-13T10:11:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5290000000000001e-05 2024-09-13T10:11:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214499+0000", 2024-09-13T10:11:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5119e-05 2024-09-13T10:11:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214517+0000", 2024-09-13T10:11:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7563000000000001e-05 2024-09-13T10:11:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214680+0000", 2024-09-13T10:11:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016348799999999999 2024-09-13T10:11:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214937+0000", 2024-09-13T10:11:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025682599999999998 2024-09-13T10:11:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0,11", 2024-09-13T10:11:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225660+0000", 2024-09-13T10:11:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010722768000000001 2024-09-13T10:11:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.227197+0000", 2024-09-13T10:11:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015373749999999999 2024-09-13T10:11:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228428+0000", 2024-09-13T10:11:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001230863 2024-09-13T10:11:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228646+0000", 2024-09-13T10:11:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021769700000000001 2024-09-13T10:11:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228755+0000", 2024-09-13T10:11:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109551 2024-09-13T10:11:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228854+0000", 2024-09-13T10:11:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8948000000000002e-05 2024-09-13T10:11:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(osd.11.22:16 3.0 3:1e59f26f:::f36fb4795b66b358920b15346e41df644c7e2d9386f5f260380eec916b18bcae:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.841613+0000", 2024-09-13T10:11:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.186743568000001, 2024-09-13T10:11:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011923899, 2024-09-13T10:11:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841613+0000", 2024-09-13T10:11:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841613+0000", 2024-09-13T10:11:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841615+0000", 2024-09-13T10:11:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.593e-06 2024-09-13T10:11:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841739+0000", 2024-09-13T10:11:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012399000000000001 2024-09-13T10:11:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841762+0000", 2024-09-13T10:11:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3129000000000002e-05 2024-09-13T10:11:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841778+0000", 2024-09-13T10:11:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5982999999999999e-05 2024-09-13T10:11:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.848805+0000", 2024-09-13T10:11:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0070273339999999997 2024-09-13T10:11:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.848984+0000", 2024-09-13T10:11:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017825499999999999 2024-09-13T10:11:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.849268+0000", 2024-09-13T10:11:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028478200000000003 2024-09-13T10:11:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853103+0000", 2024-09-13T10:11:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0038350429999999998 2024-09-13T10:11:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853226+0000", 2024-09-13T10:11:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012296299999999999 2024-09-13T10:11:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853339+0000", 2024-09-13T10:11:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011251000000000001 2024-09-13T10:11:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853468+0000", 2024-09-13T10:11:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012881599999999999 2024-09-13T10:11:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853537+0000", 2024-09-13T10:11:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9500999999999997e-05 2024-09-13T10:11:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(osd.11.22:17 3.0 3:08307d25:::2b3cb797ae2d3abc24b2cae4a012f26325debe2fc38ccb08bd4503e949d305ac:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.841816+0000", 2024-09-13T10:11:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 19.186540496999999, 2024-09-13T10:11:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011960942, 2024-09-13T10:11:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841816+0000", 2024-09-13T10:11:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841816+0000", 2024-09-13T10:11:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841818+0000", 2024-09-13T10:11:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.353e-06 2024-09-13T10:11:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841888+0000", 2024-09-13T10:11:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9808000000000007e-05 2024-09-13T10:11:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841905+0000", 2024-09-13T10:11:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7686e-05 2024-09-13T10:11:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.841924+0000", 2024-09-13T10:11:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9176000000000001e-05 2024-09-13T10:11:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.850275+0000", 2024-09-13T10:11:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083507110000000002 2024-09-13T10:11:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.850536+0000", 2024-09-13T10:11:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026065000000000002 2024-09-13T10:11:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.851014+0000", 2024-09-13T10:11:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00047829799999999999 2024-09-13T10:11:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853304+0000", 2024-09-13T10:11:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002290233 2024-09-13T10:11:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853562+0000", 2024-09-13T10:11:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025763199999999999 2024-09-13T10:11:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853636+0000", 2024-09-13T10:11:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4196999999999995e-05 2024-09-13T10:11:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853710+0000", 2024-09-13T10:11:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4271999999999996e-05 2024-09-13T10:11:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.853777+0000", 2024-09-13T10:11:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6926000000000003e-05 2024-09-13T10:11:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1836 3.0 3:0cf17fa4:::e22ed036f41f6a390924cf1e5e56fa98e0edea9e6c33080c396679f1537fbc7a:head [write 0~2655744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-13T10:11:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:10.675934+0000", 2024-09-13T10:11:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 0.35242254699999997, 2024-09-13T10:11:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015481818, 2024-09-13T10:11:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1836 2024-09-13T10:11:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.675934+0000", 2024-09-13T10:11:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.675934+0000", 2024-09-13T10:11:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.675939+0000", 2024-09-13T10:11:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2830000000000003e-06 2024-09-13T10:11:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677769+0000", 2024-09-13T10:11:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0018299830000000001 2024-09-13T10:11:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677796+0000", 2024-09-13T10:11:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7858999999999999e-05 2024-09-13T10:11:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677809+0000", 2024-09-13T10:11:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2291e-05 2024-09-13T10:11:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677895+0000", 2024-09-13T10:11:11.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6581000000000003e-05 2024-09-13T10:11:11.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.678079+0000", 2024-09-13T10:11:11.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000183802 2024-09-13T10:11:11.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:11.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.678268+0000", 2024-09-13T10:11:11.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000188383 2024-09-13T10:11:11.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.683389+0000", 2024-09-13T10:11:11.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0051215840000000002 2024-09-13T10:11:11.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.684549+0000", 2024-09-13T10:11:11.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0011600169999999999 2024-09-13T10:11:11.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.691333+0000", 2024-09-13T10:11:11.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0067840510000000001 2024-09-13T10:11:11.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.691377+0000", 2024-09-13T10:11:11.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4313000000000001e-05 2024-09-13T10:11:11.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.691416+0000", 2024-09-13T10:11:11.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8671000000000003e-05 2024-09-13T10:11:11.152 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.152 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.152 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.152 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.152 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.153 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:11.153 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:11:11.153 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:11:11.153 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:11:11.153 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T10:11:11.154 INFO:tasks.thrashosds.thrasher:Setting osd 8 primary_affinity to 0.396252 2024-09-13T10:11:11.154 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 8 0.3962518832704345 2024-09-13T10:11:11.155 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:11.153+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:11.156 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T10:11:11.156 INFO:tasks.rados.rados.0.smithi195.stdout:971: finishing write tid 1 to smithi19530714-4 2024-09-13T10:11:11.156 INFO:tasks.rados.rados.0.smithi195.stdout:972: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-13T10:11:11.156 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:11.166 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2048002 2024-09-13T10:11:11.189 INFO:tasks.rados.rados.0.smithi195.stdout:971: finishing write tid 2 to smithi19530714-4 2024-09-13T10:11:11.189 INFO:tasks.rados.rados.0.smithi195.stdout:971: finishing write tid 3 to smithi19530714-4 2024-09-13T10:11:11.189 INFO:tasks.rados.rados.0.smithi195.stdout:971: finishing write tid 4 to smithi19530714-4 2024-09-13T10:11:11.189 INFO:tasks.rados.rados.0.smithi195.stdout:971: oid 4 updating version 0 to 62 2024-09-13T10:11:11.190 INFO:tasks.rados.rados.0.smithi195.stdout:971: oid 4 version 62 is already newer than 60 2024-09-13T10:11:11.190 INFO:tasks.rados.rados.0.smithi195.stdout:971: oid 4 version 62 is already newer than 61 2024-09-13T10:11:11.190 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 62 (ObjNum 163 snap 0 seq_num 163) dirty exists 2024-09-13T10:11:11.190 INFO:tasks.rados.rados.0.smithi195.stdout:971: left oid 4 (ObjNum 163 snap 0 seq_num 163) 2024-09-13T10:11:11.190 INFO:tasks.rados.rados.0.smithi195.stdout:976: expect (ObjNum 4 snap 0 seq_num 4) 2024-09-13T10:11:11.191 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1587176 2024-09-13T10:11:11.202 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:11.242 INFO:tasks.rados.rados.0.smithi195.stdout:977: expect (ObjNum 137 snap 0 seq_num 137) 2024-09-13T10:11:11.245 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 511166 2024-09-13T10:11:11.254 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:11.253+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:11.280 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:11.282 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:11.282 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:11.282 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:11:11.282 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:11.295 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T10:11:11.336 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1737210 2024-09-13T10:11:11.353 INFO:tasks.rados.rados.0.smithi195.stdout:970: got (0) Success 2024-09-13T10:11:11.353 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 0 (ObjNum 120 snap 0 seq_num 120) dirty exists 2024-09-13T10:11:11.353 INFO:tasks.rados.rados.0.smithi195.stdout:975: finishing set_chunk to oid 37 2024-09-13T10:11:11.354 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 18 (ObjNum 145 snap 0 seq_num 145) dirty exists 2024-09-13T10:11:11.354 INFO:tasks.rados.rados.0.smithi195.stdout:978: seq_num 164 ranges {756348=784249,2063988=686387} 2024-09-13T10:11:11.355 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:11.351+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:11.364 INFO:tasks.rados.rados.0.smithi195.stdout:978: writing smithi19530714-11 from 756348 to 1540597 tid 1 2024-09-13T10:11:11.374 INFO:tasks.rados.rados.0.smithi195.stdout:978: writing smithi19530714-11 from 2063988 to 2750375 tid 2 2024-09-13T10:11:11.374 INFO:tasks.rados.rados.0.smithi195.stdout:964: done (14 left) 2024-09-13T10:11:11.374 INFO:tasks.rados.rados.0.smithi195.stdout:965: done (13 left) 2024-09-13T10:11:11.374 INFO:tasks.rados.rados.0.smithi195.stdout:966: done (12 left) 2024-09-13T10:11:11.374 INFO:tasks.rados.rados.0.smithi195.stdout:967: done (11 left) 2024-09-13T10:11:11.374 INFO:tasks.rados.rados.0.smithi195.stdout:968: done (10 left) 2024-09-13T10:11:11.374 INFO:tasks.rados.rados.0.smithi195.stdout:969: done (9 left) 2024-09-13T10:11:11.374 INFO:tasks.rados.rados.0.smithi195.stdout:970: done (8 left) 2024-09-13T10:11:11.374 INFO:tasks.rados.rados.0.smithi195.stdout:971: done (7 left) 2024-09-13T10:11:11.375 INFO:tasks.rados.rados.0.smithi195.stdout:972: done (6 left) 2024-09-13T10:11:11.375 INFO:tasks.rados.rados.0.smithi195.stdout:973: done (5 left) 2024-09-13T10:11:11.375 INFO:tasks.rados.rados.0.smithi195.stdout:975: done (4 left) 2024-09-13T10:11:11.375 INFO:tasks.rados.rados.0.smithi195.stdout:976: done (3 left) 2024-09-13T10:11:11.375 INFO:tasks.rados.rados.0.smithi195.stdout:977: done (2 left) 2024-09-13T10:11:11.375 INFO:tasks.rados.rados.0.smithi195.stdout:979: write oid 44 current snap is 0 2024-09-13T10:11:11.375 INFO:tasks.rados.rados.0.smithi195.stdout:978: finishing write tid 1 to smithi19530714-11 2024-09-13T10:11:11.375 INFO:tasks.rados.rados.0.smithi195.stdout:979: seq_num 165 ranges {773155=459498,1722470=430045,2648676=124479} 2024-09-13T10:11:11.380 INFO:tasks.rados.rados.0.smithi195.stdout:979: writing smithi19530714-44 from 773155 to 1232653 tid 1 2024-09-13T10:11:11.385 INFO:tasks.rados.rados.0.smithi195.stdout:979: writing smithi19530714-44 from 1722470 to 2152515 tid 2 2024-09-13T10:11:11.390 INFO:tasks.rados.rados.0.smithi195.stdout:979: writing smithi19530714-44 from 2648676 to 2773155 tid 3 2024-09-13T10:11:11.391 INFO:tasks.rados.rados.0.smithi195.stdout:980: delete oid 32 current snap is 0 2024-09-13T10:11:11.391 INFO:tasks.rados.rados.0.smithi195.stdout:978: finishing write tid 2 to smithi19530714-11 2024-09-13T10:11:11.391 INFO:tasks.rados.rados.0.smithi195.stdout:978: finishing write tid 3 to smithi19530714-11 2024-09-13T10:11:11.391 INFO:tasks.rados.rados.0.smithi195.stdout:978: finishing write tid 4 to smithi19530714-11 2024-09-13T10:11:11.391 INFO:tasks.rados.rados.0.smithi195.stdout:978: oid 11 updating version 0 to 36 2024-09-13T10:11:11.391 INFO:tasks.rados.rados.0.smithi195.stdout:978: oid 11 updating version 36 to 38 2024-09-13T10:11:11.392 INFO:tasks.rados.rados.0.smithi195.stdout:978: oid 11 version 38 is already newer than 37 2024-09-13T10:11:11.392 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 38 (ObjNum 164 snap 0 seq_num 164) dirty exists 2024-09-13T10:11:11.392 INFO:tasks.rados.rados.0.smithi195.stdout:978: left oid 11 (ObjNum 164 snap 0 seq_num 164) 2024-09-13T10:11:11.392 INFO:tasks.rados.rados.0.smithi195.stdout:979: finishing write tid 1 to smithi19530714-44 2024-09-13T10:11:11.395 INFO:tasks.rados.rados.0.smithi195.stdout:978: done (3 left) 2024-09-13T10:11:11.396 INFO:tasks.rados.rados.0.smithi195.stdout:980: done (2 left) 2024-09-13T10:11:11.396 INFO:tasks.rados.rados.0.smithi195.stdout:981: write oid 28 current snap is 0 2024-09-13T10:11:11.396 INFO:tasks.rados.rados.0.smithi195.stdout:981: seq_num 167 ranges {406769=609996,1439434=433714,2303522=503247} 2024-09-13T10:11:11.402 INFO:tasks.rados.rados.0.smithi195.stdout:981: writing smithi19530714-28 from 406769 to 1016765 tid 1 2024-09-13T10:11:11.408 INFO:tasks.rados.rados.0.smithi195.stdout:981: writing smithi19530714-28 from 1439434 to 1873148 tid 2 2024-09-13T10:11:11.415 INFO:tasks.rados.rados.0.smithi195.stdout:981: writing smithi19530714-28 from 2303522 to 2806769 tid 3 2024-09-13T10:11:11.415 INFO:tasks.rados.rados.0.smithi195.stdout:982: read oid 23 snap -1 2024-09-13T10:11:11.415 INFO:tasks.rados.rados.0.smithi195.stdout:982: expect (ObjNum 90 snap 0 seq_num 90) 2024-09-13T10:11:11.416 INFO:tasks.rados.rados.0.smithi195.stdout:983: write oid 6 current snap is 0 2024-09-13T10:11:11.416 INFO:tasks.rados.rados.0.smithi195.stdout:979: finishing write tid 2 to smithi19530714-44 2024-09-13T10:11:11.416 INFO:tasks.rados.rados.0.smithi195.stdout:983: seq_num 168 ranges {423576=401598,1497916=693725} 2024-09-13T10:11:11.419 INFO:tasks.rados.rados.0.smithi195.stdout:983: writing smithi19530714-6 from 423576 to 825174 tid 1 2024-09-13T10:11:11.429 INFO:tasks.rados.rados.0.smithi195.stdout:983: writing smithi19530714-6 from 1497916 to 2191641 tid 2 2024-09-13T10:11:11.429 INFO:tasks.rados.rados.0.smithi195.stdout:984: write oid 20 current snap is 0 2024-09-13T10:11:11.429 INFO:tasks.rados.rados.0.smithi195.stdout:984: seq_num 169 ranges {440383=476847,1556398=437383,2474721=365662} 2024-09-13T10:11:11.435 INFO:tasks.rados.rados.0.smithi195.stdout:984: writing smithi19530714-20 from 440383 to 917230 tid 1 2024-09-13T10:11:11.442 INFO:tasks.rados.rados.0.smithi195.stdout:984: writing smithi19530714-20 from 1556398 to 1993781 tid 2 2024-09-13T10:11:11.447 INFO:tasks.rados.rados.0.smithi195.stdout:984: writing smithi19530714-20 from 2474721 to 2840383 tid 3 2024-09-13T10:11:11.447 INFO:tasks.rados.rados.0.smithi195.stdout:985: read oid 42 snap -1 2024-09-13T10:11:11.447 INFO:tasks.rados.rados.0.smithi195.stdout:985: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-13T10:11:11.447 INFO:tasks.rados.rados.0.smithi195.stdout:986: set_chunk oid 9 target oid 2024-09-13T10:11:11.456 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:11.453+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:11.472 INFO:tasks.rados.rados.0.smithi195.stdout:986: set_chunk oid 9 offset: 1464832 length: 1564160 target oid 94b535ebce1e4b876b60299ce2c898f21322eac22e53c870cc1a556b4f2522d9 offset: 0 snap -1 2024-09-13T10:11:11.472 INFO:tasks.rados.rados.0.smithi195.stdout:987: read oid 3 snap -1 2024-09-13T10:11:11.472 INFO:tasks.rados.rados.0.smithi195.stdout:987: expect deleted 2024-09-13T10:11:11.472 INFO:tasks.rados.rados.0.smithi195.stdout:988: read oid 31 snap -1 2024-09-13T10:11:11.472 INFO:tasks.rados.rados.0.smithi195.stdout:988: expect (ObjNum 93 snap 0 seq_num 93) 2024-09-13T10:11:11.472 INFO:tasks.rados.rados.0.smithi195.stdout:989: delete oid 12 current snap is 0 2024-09-13T10:11:11.472 INFO:tasks.rados.rados.0.smithi195.stdout:979: finishing write tid 3 to smithi19530714-44 2024-09-13T10:11:11.472 INFO:tasks.rados.rados.0.smithi195.stdout:979: finishing write tid 4 to smithi19530714-44 2024-09-13T10:11:11.472 INFO:tasks.rados.rados.0.smithi195.stdout:979: finishing write tid 5 to smithi19530714-44 2024-09-13T10:11:11.472 INFO:tasks.rados.rados.0.smithi195.stdout:979: oid 44 updating version 0 to 46 2024-09-13T10:11:11.473 INFO:tasks.rados.rados.0.smithi195.stdout:979: oid 44 updating version 46 to 48 2024-09-13T10:11:11.473 INFO:tasks.rados.rados.0.smithi195.stdout:979: oid 44 version 48 is already newer than 45 2024-09-13T10:11:11.473 INFO:tasks.rados.rados.0.smithi195.stdout:979: oid 44 version 48 is already newer than 47 2024-09-13T10:11:11.473 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 48 (ObjNum 165 snap 0 seq_num 165) dirty exists 2024-09-13T10:11:11.473 INFO:tasks.rados.rados.0.smithi195.stdout:979: left oid 44 (ObjNum 165 snap 0 seq_num 165) 2024-09-13T10:11:11.473 INFO:tasks.rados.rados.0.smithi195.stdout:981: finishing write tid 1 to smithi19530714-28 2024-09-13T10:11:11.473 INFO:tasks.rados.rados.0.smithi195.stdout:981: finishing write tid 2 to smithi19530714-28 2024-09-13T10:11:11.473 INFO:tasks.rados.rados.0.smithi195.stdout:982: expect (ObjNum 90 snap 0 seq_num 90) 2024-09-13T10:11:11.473 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:11.474 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:11.474 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:11:11.474 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:11:11.474 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:11.491 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T10:11:11.557 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:11.553+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:11.575 INFO:tasks.rados.rados.0.smithi195.stdout:981: finishing write tid 3 to smithi19530714-28 2024-09-13T10:11:11.575 INFO:tasks.rados.rados.0.smithi195.stdout:981: finishing write tid 4 to smithi19530714-28 2024-09-13T10:11:11.575 INFO:tasks.rados.rados.0.smithi195.stdout:981: finishing write tid 5 to smithi19530714-28 2024-09-13T10:11:11.576 INFO:tasks.rados.rados.0.smithi195.stdout:981: oid 28 updating version 0 to 89 2024-09-13T10:11:11.576 INFO:tasks.rados.rados.0.smithi195.stdout:981: oid 28 updating version 89 to 91 2024-09-13T10:11:11.576 INFO:tasks.rados.rados.0.smithi195.stdout:981: oid 28 updating version 91 to 92 2024-09-13T10:11:11.576 INFO:tasks.rados.rados.0.smithi195.stdout:981: oid 28 version 92 is already newer than 90 2024-09-13T10:11:11.576 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 92 (ObjNum 167 snap 0 seq_num 167) dirty exists 2024-09-13T10:11:11.576 INFO:tasks.rados.rados.0.smithi195.stdout:981: left oid 28 (ObjNum 167 snap 0 seq_num 167) 2024-09-13T10:11:11.576 INFO:tasks.rados.rados.0.smithi195.stdout:983: finishing write tid 1 to smithi19530714-6 2024-09-13T10:11:11.576 INFO:tasks.rados.rados.0.smithi195.stdout:983: finishing write tid 2 to smithi19530714-6 2024-09-13T10:11:11.576 INFO:tasks.rados.rados.0.smithi195.stdout:983: finishing write tid 3 to smithi19530714-6 2024-09-13T10:11:11.576 INFO:tasks.rados.rados.0.smithi195.stdout:983: finishing write tid 4 to smithi19530714-6 2024-09-13T10:11:11.576 INFO:tasks.rados.rados.0.smithi195.stdout:983: oid 6 updating version 0 to 52 2024-09-13T10:11:11.576 INFO:tasks.rados.rados.0.smithi195.stdout:983: oid 6 version 52 is already newer than 51 2024-09-13T10:11:11.576 INFO:tasks.rados.rados.0.smithi195.stdout:983: oid 6 updating version 52 to 53 2024-09-13T10:11:11.577 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 53 (ObjNum 168 snap 0 seq_num 168) dirty exists 2024-09-13T10:11:11.577 INFO:tasks.rados.rados.0.smithi195.stdout:983: left oid 6 (ObjNum 168 snap 0 seq_num 168) 2024-09-13T10:11:11.577 INFO:tasks.rados.rados.0.smithi195.stdout:984: finishing write tid 1 to smithi19530714-20 2024-09-13T10:11:11.577 INFO:tasks.rados.rados.0.smithi195.stdout:984: finishing write tid 2 to smithi19530714-20 2024-09-13T10:11:11.577 INFO:tasks.rados.rados.0.smithi195.stdout:984: finishing write tid 3 to smithi19530714-20 2024-09-13T10:11:11.577 INFO:tasks.rados.rados.0.smithi195.stdout:985: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-13T10:11:11.579 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1491798 2024-09-13T10:11:11.640 INFO:tasks.rados.rados.0.smithi195.stdout:984: finishing write tid 4 to smithi19530714-20 2024-09-13T10:11:11.640 INFO:tasks.rados.rados.0.smithi195.stdout:984: finishing write tid 5 to smithi19530714-20 2024-09-13T10:11:11.640 INFO:tasks.rados.rados.0.smithi195.stdout:984: oid 20 updating version 0 to 41 2024-09-13T10:11:11.640 INFO:tasks.rados.rados.0.smithi195.stdout:984: oid 20 version 41 is already newer than 40 2024-09-13T10:11:11.640 INFO:tasks.rados.rados.0.smithi195.stdout:984: oid 20 updating version 41 to 42 2024-09-13T10:11:11.640 INFO:tasks.rados.rados.0.smithi195.stdout:984: oid 20 version 42 is already newer than 39 2024-09-13T10:11:11.640 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 42 (ObjNum 169 snap 0 seq_num 169) dirty exists 2024-09-13T10:11:11.640 INFO:tasks.rados.rados.0.smithi195.stdout:984: left oid 20 (ObjNum 169 snap 0 seq_num 169) 2024-09-13T10:11:11.641 INFO:tasks.rados.rados.0.smithi195.stdout:986: finishing set_chunk to oid 9 2024-09-13T10:11:11.641 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 9 set_chunk returned error code -95 offset: 1464832 length: 1564160 2024-09-13T10:11:11.641 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 0 (ObjNum 113 snap 0 seq_num 113) dirty exists 2024-09-13T10:11:11.641 INFO:tasks.rados.rados.0.smithi195.stdout:988: expect (ObjNum 93 snap 0 seq_num 93) 2024-09-13T10:11:11.646 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:11.646 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:11:11.646 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:11:11.646 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:11:11.646 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.646 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:42 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'24, mlcod=25'19)", 2024-09-13T10:11:11.646 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.285251+0000", 2024-09-13T10:11:11.647 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.357866819999998, 2024-09-13T10:11:11.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.008704425, 2024-09-13T10:11:11.647 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.647 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.647 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.647 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.647 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285251+0000", 2024-09-13T10:11:11.647 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.647 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.647 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.647 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285251+0000", 2024-09-13T10:11:11.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285252+0000", 2024-09-13T10:11:11.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.33e-06 2024-09-13T10:11:11.648 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.648 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.648 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.648 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285294+0000", 2024-09-13T10:11:11.648 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1148e-05 2024-09-13T10:11:11.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285331+0000", 2024-09-13T10:11:11.649 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7478000000000003e-05 2024-09-13T10:11:11.649 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.649 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.649 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.649 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.285355+0000", 2024-09-13T10:11:11.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3524999999999999e-05 2024-09-13T10:11:11.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286264+0000", 2024-09-13T10:11:11.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090990400000000001 2024-09-13T10:11:11.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.650 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.650 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286300+0000", 2024-09-13T10:11:11.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5395000000000002e-05 2024-09-13T10:11:11.650 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.293876+0000", 2024-09-13T10:11:11.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0075758589999999999 2024-09-13T10:11:11.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.293955+0000", 2024-09-13T10:11:11.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9785999999999998e-05 2024-09-13T10:11:11.651 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.651 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.651 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.652 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:45 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'27, mlcod=25'19)", 2024-09-13T10:11:11.652 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.286596+0000", 2024-09-13T10:11:11.652 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.356522134999999, 2024-09-13T10:11:11.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0093831250000000008, 2024-09-13T10:11:11.652 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.652 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.652 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.652 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286596+0000", 2024-09-13T10:11:11.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286596+0000", 2024-09-13T10:11:11.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286598+0000", 2024-09-13T10:11:11.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8500000000000001e-06 2024-09-13T10:11:11.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286611+0000", 2024-09-13T10:11:11.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3456e-05 2024-09-13T10:11:11.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286635+0000", 2024-09-13T10:11:11.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4165e-05 2024-09-13T10:11:11.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286655+0000", 2024-09-13T10:11:11.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9383000000000001e-05 2024-09-13T10:11:11.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289627+0000", 2024-09-13T10:11:11.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029728900000000002 2024-09-13T10:11:11.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289685+0000", 2024-09-13T10:11:11.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7040000000000003e-05 2024-09-13T10:11:11.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.656 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.295871+0000", 2024-09-13T10:11:11.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061868629999999999 2024-09-13T10:11:11.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.295979+0000", 2024-09-13T10:11:11.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107478 2024-09-13T10:11:11.657 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.657 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.657 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.657 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:46 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'28, mlcod=25'19)", 2024-09-13T10:11:11.657 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.286979+0000", 2024-09-13T10:11:11.657 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.356139224000003, 2024-09-13T10:11:11.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0090737700000000001, 2024-09-13T10:11:11.658 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.658 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.658 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.658 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286979+0000", 2024-09-13T10:11:11.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286979+0000", 2024-09-13T10:11:11.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286981+0000", 2024-09-13T10:11:11.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0880000000000002e-06 2024-09-13T10:11:11.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286994+0000", 2024-09-13T10:11:11.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2884999999999999e-05 2024-09-13T10:11:11.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.662 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287017+0000", 2024-09-13T10:11:11.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3408e-05 2024-09-13T10:11:11.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287033+0000", 2024-09-13T10:11:11.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.641e-05 2024-09-13T10:11:11.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.290659+0000", 2024-09-13T10:11:11.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036255749999999998 2024-09-13T10:11:11.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.290692+0000", 2024-09-13T10:11:11.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3154999999999997e-05 2024-09-13T10:11:11.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.295986+0000", 2024-09-13T10:11:11.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0052934180000000003 2024-09-13T10:11:11.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296052+0000", 2024-09-13T10:11:11.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6830999999999998e-05 2024-09-13T10:11:11.666 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.666 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.666 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.667 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:47 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'29, mlcod=25'20)", 2024-09-13T10:11:11.667 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.287544+0000", 2024-09-13T10:11:11.667 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.355574271999998, 2024-09-13T10:11:11.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0085821309999999998, 2024-09-13T10:11:11.667 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.668 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.668 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.668 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287544+0000", 2024-09-13T10:11:11.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287544+0000", 2024-09-13T10:11:11.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.670 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.670 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287545+0000", 2024-09-13T10:11:11.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.646e-06 2024-09-13T10:11:11.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287727+0000", 2024-09-13T10:11:11.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000181528 2024-09-13T10:11:11.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287749+0000", 2024-09-13T10:11:11.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1744999999999999e-05 2024-09-13T10:11:11.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287765+0000", 2024-09-13T10:11:11.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.649e-05 2024-09-13T10:11:11.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.291452+0000", 2024-09-13T10:11:11.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036870370000000002 2024-09-13T10:11:11.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.291506+0000", 2024-09-13T10:11:11.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4323000000000001e-05 2024-09-13T10:11:11.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296064+0000", 2024-09-13T10:11:11.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045575340000000002 2024-09-13T10:11:11.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296126+0000", 2024-09-13T10:11:11.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1828000000000005e-05 2024-09-13T10:11:11.676 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.676 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.676 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.676 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:21 4.6 4:6150ba38:::smithi19530714-6:head [write 484035~776245,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.677 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.390657+0000", 2024-09-13T10:11:11.677 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.252460664000001, 2024-09-13T10:11:11.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088615770000000007, 2024-09-13T10:11:11.677 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.677 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.677 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.678 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.678 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.678 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.678 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 21 2024-09-13T10:11:11.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.679 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.390657+0000", 2024-09-13T10:11:11.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.390657+0000", 2024-09-13T10:11:11.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.390660+0000", 2024-09-13T10:11:11.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1970000000000001e-06 2024-09-13T10:11:11.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.391452+0000", 2024-09-13T10:11:11.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00079113899999999995 2024-09-13T10:11:11.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.391487+0000", 2024-09-13T10:11:11.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5116000000000001e-05 2024-09-13T10:11:11.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.391504+0000", 2024-09-13T10:11:11.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7010000000000001e-05 2024-09-13T10:11:11.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.391641+0000", 2024-09-13T10:11:11.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013700499999999999 2024-09-13T10:11:11.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.391867+0000", 2024-09-13T10:11:11.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000226677 2024-09-13T10:11:11.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:11.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.392145+0000", 2024-09-13T10:11:11.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000277368 2024-09-13T10:11:11.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.395418+0000", 2024-09-13T10:11:11.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032730530000000002 2024-09-13T10:11:11.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.396126+0000", 2024-09-13T10:11:11.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00070789900000000001 2024-09-13T10:11:11.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.399358+0000", 2024-09-13T10:11:11.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032325209999999999 2024-09-13T10:11:11.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.399436+0000", 2024-09-13T10:11:11.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7834000000000001e-05 2024-09-13T10:11:11.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.399519+0000", 2024-09-13T10:11:11.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2757999999999996e-05 2024-09-13T10:11:11.686 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.686 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.686 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.686 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:211 3.0 e38/30 3:0596651d:::smithi19530714-12 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:11.686 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:11:11.686 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.634798970999999, 2024-09-13T10:11:11.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013316957000000001, 2024-09-13T10:11:11.686 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.687 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.687 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.687 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:11:11.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:11:11.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008321+0000", 2024-09-13T10:11:11.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-13T10:11:11.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011069+0000", 2024-09-13T10:11:11.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002747893 2024-09-13T10:11:11.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011113+0000", 2024-09-13T10:11:11.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3921000000000002e-05 2024-09-13T10:11:11.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011140+0000", 2024-09-13T10:11:11.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7662999999999999e-05 2024-09-13T10:11:11.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011313+0000", 2024-09-13T10:11:11.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017232200000000001 2024-09-13T10:11:11.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011366+0000", 2024-09-13T10:11:11.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3888000000000003e-05 2024-09-13T10:11:11.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021577+0000", 2024-09-13T10:11:11.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010210472999999999 2024-09-13T10:11:11.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021636+0000", 2024-09-13T10:11:11.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8953999999999998e-05 2024-09-13T10:11:11.691 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.691 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.692 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.692 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:212 3.0 e38/30 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:11.692 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:11:11.692 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.631481865000001, 2024-09-13T10:11:11.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014217904, 2024-09-13T10:11:11.693 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.693 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.693 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.693 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:11:11.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:11:11.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011638+0000", 2024-09-13T10:11:11.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.672e-06 2024-09-13T10:11:11.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013289+0000", 2024-09-13T10:11:11.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001651511 2024-09-13T10:11:11.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013310+0000", 2024-09-13T10:11:11.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0917999999999999e-05 2024-09-13T10:11:11.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013322+0000", 2024-09-13T10:11:11.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2067e-05 2024-09-13T10:11:11.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.016882+0000", 2024-09-13T10:11:11.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035599130000000001 2024-09-13T10:11:11.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.016934+0000", 2024-09-13T10:11:11.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1715999999999997e-05 2024-09-13T10:11:11.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025787+0000", 2024-09-13T10:11:11.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088531809999999999 2024-09-13T10:11:11.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025854+0000", 2024-09-13T10:11:11.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6926000000000003e-05 2024-09-13T10:11:11.698 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.698 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.698 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.698 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:207 3.0 e38/30 3:332cd44b:::smithi19530714-8 low_tier:head v 38'3, mlcod=0'0)", 2024-09-13T10:11:11.699 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:11:11.699 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.625688424, 2024-09-13T10:11:11.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016660892, 2024-09-13T10:11:11.699 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.699 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.699 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.699 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:11:11.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:11:11.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017432+0000", 2024-09-13T10:11:11.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1490000000000001e-06 2024-09-13T10:11:11.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021310+0000", 2024-09-13T10:11:11.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003877985 2024-09-13T10:11:11.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021351+0000", 2024-09-13T10:11:11.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1105e-05 2024-09-13T10:11:11.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021376+0000", 2024-09-13T10:11:11.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5185999999999999e-05 2024-09-13T10:11:11.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021727+0000", 2024-09-13T10:11:11.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035131300000000002 2024-09-13T10:11:11.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021792+0000", 2024-09-13T10:11:11.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4368000000000004e-05 2024-09-13T10:11:11.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034027+0000", 2024-09-13T10:11:11.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012235582 2024-09-13T10:11:11.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034090+0000", 2024-09-13T10:11:11.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3204e-05 2024-09-13T10:11:11.703 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.703 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.703 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.704 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:216 3.0 e38/30 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:11:11.704 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:11:11.704 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.619122854, 2024-09-13T10:11:11.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011044523000000001, 2024-09-13T10:11:11.704 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.705 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.705 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.705 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:11:11.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:11:11.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023997+0000", 2024-09-13T10:11:11.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.533e-06 2024-09-13T10:11:11.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024872+0000", 2024-09-13T10:11:11.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087517699999999999 2024-09-13T10:11:11.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024894+0000", 2024-09-13T10:11:11.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2571000000000001e-05 2024-09-13T10:11:11.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024909+0000", 2024-09-13T10:11:11.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5158e-05 2024-09-13T10:11:11.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025979+0000", 2024-09-13T10:11:11.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010692329999999999 2024-09-13T10:11:11.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.026027+0000", 2024-09-13T10:11:11.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8158000000000002e-05 2024-09-13T10:11:11.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034992+0000", 2024-09-13T10:11:11.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0089650960000000005 2024-09-13T10:11:11.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.035040+0000", 2024-09-13T10:11:11.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7596999999999998e-05 2024-09-13T10:11:11.710 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.710 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.710 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.710 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:225 3.0 e38/30 3:26960251:::smithi19530714-26 low_tier:head v 38'6, mlcod=38'3)", 2024-09-13T10:11:11.710 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:11:11.710 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.572225777, 2024-09-13T10:11:11.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013205889, 2024-09-13T10:11:11.710 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.710 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.710 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.711 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:11:11.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:11:11.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070894+0000", 2024-09-13T10:11:11.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5e-06 2024-09-13T10:11:11.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072562+0000", 2024-09-13T10:11:11.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016683469999999999 2024-09-13T10:11:11.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072632+0000", 2024-09-13T10:11:11.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9665000000000005e-05 2024-09-13T10:11:11.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072661+0000", 2024-09-13T10:11:11.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9714999999999999e-05 2024-09-13T10:11:11.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072800+0000", 2024-09-13T10:11:11.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013886000000000001 2024-09-13T10:11:11.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072828+0000", 2024-09-13T10:11:11.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7886e-05 2024-09-13T10:11:11.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084041+0000", 2024-09-13T10:11:11.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011213064999999999 2024-09-13T10:11:11.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084098+0000", 2024-09-13T10:11:11.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6851000000000002e-05 2024-09-13T10:11:11.715 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.715 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.715 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.715 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:228 3.0 e38/30 3:0168a995:::smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'7, mlcod=38'3)", 2024-09-13T10:11:11.716 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:11:11.716 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.568442197, 2024-09-13T10:11:11.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094647819999999997, 2024-09-13T10:11:11.716 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.716 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.717 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.717 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:11:11.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:11:11.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074677+0000", 2024-09-13T10:11:11.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7099999999999999e-06 2024-09-13T10:11:11.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076599+0000", 2024-09-13T10:11:11.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019212459999999999 2024-09-13T10:11:11.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076616+0000", 2024-09-13T10:11:11.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7008999999999999e-05 2024-09-13T10:11:11.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.721 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076641+0000", 2024-09-13T10:11:11.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5380000000000001e-05 2024-09-13T10:11:11.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076772+0000", 2024-09-13T10:11:11.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013101599999999999 2024-09-13T10:11:11.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076867+0000", 2024-09-13T10:11:11.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5262999999999999e-05 2024-09-13T10:11:11.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084106+0000", 2024-09-13T10:11:11.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0072390859999999996 2024-09-13T10:11:11.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084140+0000", 2024-09-13T10:11:11.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4072000000000002e-05 2024-09-13T10:11:11.725 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.725 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.725 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.725 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:252 4.6 4:6ddb2b23:::smithi19530714-3:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.726 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:11:11.726 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.521054298999999, 2024-09-13T10:11:11.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.03110711, 2024-09-13T10:11:11.726 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.726 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.727 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.727 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.727 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.727 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.727 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 252 2024-09-13T10:11:11.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.728 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:11:11.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:11:11.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122066+0000", 2024-09-13T10:11:11.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4619999999999999e-06 2024-09-13T10:11:11.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122097+0000", 2024-09-13T10:11:11.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0488999999999999e-05 2024-09-13T10:11:11.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122120+0000", 2024-09-13T10:11:11.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3649999999999999e-05 2024-09-13T10:11:11.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122130+0000", 2024-09-13T10:11:11.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7950000000000002e-06 2024-09-13T10:11:11.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122212+0000", 2024-09-13T10:11:11.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2090999999999997e-05 2024-09-13T10:11:11.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122317+0000", 2024-09-13T10:11:11.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104954 2024-09-13T10:11:11.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:11.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.151952+0000", 2024-09-13T10:11:11.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.029634908000000001 2024-09-13T10:11:11.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152580+0000", 2024-09-13T10:11:11.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062796499999999997 2024-09-13T10:11:11.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152891+0000", 2024-09-13T10:11:11.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000310747 2024-09-13T10:11:11.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153077+0000", 2024-09-13T10:11:11.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018588200000000001 2024-09-13T10:11:11.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153130+0000", 2024-09-13T10:11:11.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3016000000000002e-05 2024-09-13T10:11:11.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153171+0000", 2024-09-13T10:11:11.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1151e-05 2024-09-13T10:11:11.739 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.739 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.739 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.740 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:255 4.6 4:6150ba38:::smithi19530714-6:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.740 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:11:11.740 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.428640094999999, 2024-09-13T10:11:11.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011708387000000001, 2024-09-13T10:11:11.740 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.741 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.741 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.741 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.741 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.741 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.741 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 255 2024-09-13T10:11:11.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.741 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:11:11.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:11:11.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214482+0000", 2024-09-13T10:11:11.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8139999999999998e-06 2024-09-13T10:11:11.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214498+0000", 2024-09-13T10:11:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6636000000000001e-05 2024-09-13T10:11:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214537+0000", 2024-09-13T10:11:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8668999999999999e-05 2024-09-13T10:11:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214562+0000", 2024-09-13T10:11:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5242999999999999e-05 2024-09-13T10:11:11.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214770+0000", 2024-09-13T10:11:11.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020771299999999999 2024-09-13T10:11:11.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214996+0000", 2024-09-13T10:11:11.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022615999999999999 2024-09-13T10:11:11.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:11.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.223455+0000", 2024-09-13T10:11:11.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0084591960000000004 2024-09-13T10:11:11.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225042+0000", 2024-09-13T10:11:11.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001586896 2024-09-13T10:11:11.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225719+0000", 2024-09-13T10:11:11.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00067701700000000003 2024-09-13T10:11:11.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225845+0000", 2024-09-13T10:11:11.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012626299999999999 2024-09-13T10:11:11.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226070+0000", 2024-09-13T10:11:11.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022411600000000001 2024-09-13T10:11:11.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226186+0000", 2024-09-13T10:11:11.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011666400000000001 2024-09-13T10:11:11.747 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.747 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.747 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.747 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:268 4.6 4:6fb28ed4:::smithi19530714-19:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.747 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:11:11.747 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.427721535, 2024-09-13T10:11:11.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041608707000000002, 2024-09-13T10:11:11.747 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.747 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.747 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.748 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.748 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.748 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.748 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 268 2024-09-13T10:11:11.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.748 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:11:11.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:11:11.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215399+0000", 2024-09-13T10:11:11.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8669999999999998e-06 2024-09-13T10:11:11.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215416+0000", 2024-09-13T10:11:11.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6792000000000001e-05 2024-09-13T10:11:11.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215480+0000", 2024-09-13T10:11:11.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3805000000000003e-05 2024-09-13T10:11:11.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215499+0000", 2024-09-13T10:11:11.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9349999999999999e-05 2024-09-13T10:11:11.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.753 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215600+0000", 2024-09-13T10:11:11.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010119 2024-09-13T10:11:11.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215715+0000", 2024-09-13T10:11:11.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011491 2024-09-13T10:11:11.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:11.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.253967+0000", 2024-09-13T10:11:11.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038251470000000003 2024-09-13T10:11:11.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.255748+0000", 2024-09-13T10:11:11.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017815089999999999 2024-09-13T10:11:11.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256188+0000", 2024-09-13T10:11:11.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00043959799999999997 2024-09-13T10:11:11.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256840+0000", 2024-09-13T10:11:11.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000652591 2024-09-13T10:11:11.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256932+0000", 2024-09-13T10:11:11.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1691e-05 2024-09-13T10:11:11.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.257005+0000", 2024-09-13T10:11:11.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2934000000000003e-05 2024-09-13T10:11:11.760 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.760 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.760 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.761 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:305 4.6 4:6150ba38:::smithi19530714-6:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.761 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:11:11.761 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.374490842, 2024-09-13T10:11:11.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010323297, 2024-09-13T10:11:11.761 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.762 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.762 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.762 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.762 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.762 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.763 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 305 2024-09-13T10:11:11.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.763 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:11:11.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:11:11.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268630+0000", 2024-09-13T10:11:11.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.548e-06 2024-09-13T10:11:11.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268644+0000", 2024-09-13T10:11:11.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.394e-05 2024-09-13T10:11:11.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268672+0000", 2024-09-13T10:11:11.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8242000000000002e-05 2024-09-13T10:11:11.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268689+0000", 2024-09-13T10:11:11.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7062e-05 2024-09-13T10:11:11.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268789+0000", 2024-09-13T10:11:11.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010026600000000001 2024-09-13T10:11:11.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268905+0000", 2024-09-13T10:11:11.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115908 2024-09-13T10:11:11.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:11.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.276776+0000", 2024-09-13T10:11:11.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078712000000000001 2024-09-13T10:11:11.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278412+0000", 2024-09-13T10:11:11.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016358670000000001 2024-09-13T10:11:11.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278664+0000", 2024-09-13T10:11:11.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025162600000000002 2024-09-13T10:11:11.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278785+0000", 2024-09-13T10:11:11.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012162100000000001 2024-09-13T10:11:11.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278895+0000", 2024-09-13T10:11:11.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109774 2024-09-13T10:11:11.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278950+0000", 2024-09-13T10:11:11.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5242999999999999e-05 2024-09-13T10:11:11.772 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.772 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.772 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.772 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:352 4.6 4:6ddb2b23:::smithi19530714-3:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:11.772 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.287738+0000", 2024-09-13T10:11:11.772 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.355380148999998, 2024-09-13T10:11:11.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0085772350000000008, 2024-09-13T10:11:11.773 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.773 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.773 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.773 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.773 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.773 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.773 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 352 2024-09-13T10:11:11.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.773 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.287738+0000", 2024-09-13T10:11:11.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.287738+0000", 2024-09-13T10:11:11.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.287740+0000", 2024-09-13T10:11:11.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.885e-06 2024-09-13T10:11:11.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.287750+0000", 2024-09-13T10:11:11.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0155e-05 2024-09-13T10:11:11.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.287775+0000", 2024-09-13T10:11:11.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5006000000000001e-05 2024-09-13T10:11:11.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.287790+0000", 2024-09-13T10:11:11.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4844999999999999e-05 2024-09-13T10:11:11.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.287946+0000", 2024-09-13T10:11:11.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000156601 2024-09-13T10:11:11.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.288081+0000", 2024-09-13T10:11:11.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000134719 2024-09-13T10:11:11.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:11.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294695+0000", 2024-09-13T10:11:11.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066140729999999998 2024-09-13T10:11:11.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.295777+0000", 2024-09-13T10:11:11.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001082153 2024-09-13T10:11:11.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.296096+0000", 2024-09-13T10:11:11.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031912200000000003 2024-09-13T10:11:11.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.296168+0000", 2024-09-13T10:11:11.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1231999999999998e-05 2024-09-13T10:11:11.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.296240+0000", 2024-09-13T10:11:11.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2112000000000003e-05 2024-09-13T10:11:11.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.296315+0000", 2024-09-13T10:11:11.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5332000000000003e-05 2024-09-13T10:11:11.778 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.779 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.779 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.779 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1023 4.16 4:6fb28ed4:::smithi19530714-19:head [write 1798066~477679] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:11.779 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.078200+0000", 2024-09-13T10:11:11.779 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.564917482, 2024-09-13T10:11:11.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0095557869999999996, 2024-09-13T10:11:11.779 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.779 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.779 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.779 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.779 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.779 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.780 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1023 2024-09-13T10:11:11.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.780 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078200+0000", 2024-09-13T10:11:11.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078200+0000", 2024-09-13T10:11:11.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078202+0000", 2024-09-13T10:11:11.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2529999999999999e-06 2024-09-13T10:11:11.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078505+0000", 2024-09-13T10:11:11.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000303228 2024-09-13T10:11:11.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078519+0000", 2024-09-13T10:11:11.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3918e-05 2024-09-13T10:11:11.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078526+0000", 2024-09-13T10:11:11.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.311e-06 2024-09-13T10:11:11.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078577+0000", 2024-09-13T10:11:11.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1125999999999998e-05 2024-09-13T10:11:11.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078682+0000", 2024-09-13T10:11:11.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104671 2024-09-13T10:11:11.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:11.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078864+0000", 2024-09-13T10:11:11.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018166999999999999 2024-09-13T10:11:11.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.080583+0000", 2024-09-13T10:11:11.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001719138 2024-09-13T10:11:11.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.085104+0000", 2024-09-13T10:11:11.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045216950000000001 2024-09-13T10:11:11.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.087686+0000", 2024-09-13T10:11:11.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025817499999999998 2024-09-13T10:11:11.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.087721+0000", 2024-09-13T10:11:11.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5009999999999999e-05 2024-09-13T10:11:11.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.087756+0000", 2024-09-13T10:11:11.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5017e-05 2024-09-13T10:11:11.785 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.785 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.785 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.785 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1040 4.6 4:61ec6a5a:::smithi19530714-35:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:11.785 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.095747+0000", 2024-09-13T10:11:11.785 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.547370505, 2024-09-13T10:11:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036815318999999999, 2024-09-13T10:11:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1040 2024-09-13T10:11:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095747+0000", 2024-09-13T10:11:11.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095747+0000", 2024-09-13T10:11:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095749+0000", 2024-09-13T10:11:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9039999999999999e-06 2024-09-13T10:11:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095757+0000", 2024-09-13T10:11:11.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6660000000000008e-06 2024-09-13T10:11:11.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095784+0000", 2024-09-13T10:11:11.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6860999999999999e-05 2024-09-13T10:11:11.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095794+0000", 2024-09-13T10:11:11.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7149999999999993e-06 2024-09-13T10:11:11.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095860+0000", 2024-09-13T10:11:11.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6699999999999995e-05 2024-09-13T10:11:11.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095984+0000", 2024-09-13T10:11:11.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123676 2024-09-13T10:11:11.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.132563+0000", 2024-09-13T10:11:11.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036578797000000003 2024-09-13T10:11:11.790 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.790 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.790 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.790 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1045 4.6 4:6150ba38:::smithi19530714-6:head [write 527753~708201,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:11.790 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.313424+0000", 2024-09-13T10:11:11.790 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.329693571, 2024-09-13T10:11:11.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011098433, 2024-09-13T10:11:11.790 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.790 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.790 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.790 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:11.791 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:11.791 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:11.791 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1045 2024-09-13T10:11:11.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.791 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313424+0000", 2024-09-13T10:11:11.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313424+0000", 2024-09-13T10:11:11.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313427+0000", 2024-09-13T10:11:11.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0230000000000001e-06 2024-09-13T10:11:11.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313756+0000", 2024-09-13T10:11:11.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032905500000000003 2024-09-13T10:11:11.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313789+0000", 2024-09-13T10:11:11.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2359e-05 2024-09-13T10:11:11.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313807+0000", 2024-09-13T10:11:11.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8130000000000001e-05 2024-09-13T10:11:11.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313953+0000", 2024-09-13T10:11:11.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014562200000000001 2024-09-13T10:11:11.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.314185+0000", 2024-09-13T10:11:11.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023246399999999999 2024-09-13T10:11:11.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:11.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.314442+0000", 2024-09-13T10:11:11.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000257023 2024-09-13T10:11:11.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:11.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.317258+0000", 2024-09-13T10:11:11.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0028160149999999998 2024-09-13T10:11:11.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.319440+0000", 2024-09-13T10:11:11.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021817149999999999 2024-09-13T10:11:11.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:11.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.324214+0000", 2024-09-13T10:11:11.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047739189999999997 2024-09-13T10:11:11.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.324440+0000", 2024-09-13T10:11:11.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000225944 2024-09-13T10:11:11.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.324523+0000", 2024-09-13T10:11:11.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3164000000000005e-05 2024-09-13T10:11:11.796 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.796 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.796 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.797 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:1665 4.d e49/44 4:b19b932a:::smithi19530714-41:head v 49'30, mlcod=46'28)", 2024-09-13T10:11:11.797 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.757457+0000", 2024-09-13T10:11:11.797 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.8856612249999998, 2024-09-13T10:11:11.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0087798230000000008, 2024-09-13T10:11:11.797 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:11.797 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:11.797 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:11.797 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:11.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:11.797 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.757457+0000", 2024-09-13T10:11:11.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:11.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.757457+0000", 2024-09-13T10:11:11.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:11.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:11.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.757458+0000", 2024-09-13T10:11:11.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.206e-06 2024-09-13T10:11:11.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:11.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.758475+0000", 2024-09-13T10:11:11.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010167749999999999 2024-09-13T10:11:11.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:11.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.758505+0000", 2024-09-13T10:11:11.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0209000000000002e-05 2024-09-13T10:11:11.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:11.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.758530+0000", 2024-09-13T10:11:11.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4660000000000001e-05 2024-09-13T10:11:11.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:11.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.758704+0000", 2024-09-13T10:11:11.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017476300000000001 2024-09-13T10:11:11.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:11.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.758760+0000", 2024-09-13T10:11:11.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6051999999999998e-05 2024-09-13T10:11:11.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:11.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766191+0000", 2024-09-13T10:11:11.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0074307260000000003 2024-09-13T10:11:11.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:11.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:11.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:11.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766236+0000", 2024-09-13T10:11:11.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5432000000000001e-05 2024-09-13T10:11:11.801 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.801 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.801 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.801 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:11.801 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:11.801 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:11.802 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T10:11:11.803 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:11.801+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:11.804 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.8 primary-affinity to 0.396252 (8259682) 2024-09-13T10:11:11.805 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-13T10:11:11.806 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1787462 2024-09-13T10:11:11.806 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 431842 2024-09-13T10:11:11.806 INFO:tasks.rados.rados.0.smithi195.stdout:979: done (10 left) 2024-09-13T10:11:11.806 INFO:tasks.rados.rados.0.smithi195.stdout:981: done (9 left) 2024-09-13T10:11:11.806 INFO:tasks.rados.rados.0.smithi195.stdout:982: done (8 left) 2024-09-13T10:11:11.806 INFO:tasks.rados.rados.0.smithi195.stdout:983: done (7 left) 2024-09-13T10:11:11.806 INFO:tasks.rados.rados.0.smithi195.stdout:984: done (6 left) 2024-09-13T10:11:11.806 INFO:tasks.rados.rados.0.smithi195.stdout:985: done (5 left) 2024-09-13T10:11:11.806 INFO:tasks.rados.rados.0.smithi195.stdout:986: done (4 left) 2024-09-13T10:11:11.807 INFO:tasks.rados.rados.0.smithi195.stdout:987: done (3 left) 2024-09-13T10:11:11.807 INFO:tasks.rados.rados.0.smithi195.stdout:988: done (2 left) 2024-09-13T10:11:11.807 INFO:tasks.rados.rados.0.smithi195.stdout:989: done (1 left) 2024-09-13T10:11:11.807 INFO:tasks.rados.rados.0.smithi195.stdout:990: set_chunk oid 40 target oid 2024-09-13T10:11:11.807 INFO:tasks.rados.rados.0.smithi195.stdout:990: set_chunk oid 40 offset: 1655296 length: 513024 target oid e09a9a5fcf49ca9ef41b08a9b457aa3d4fbbd0d612e91215d6f5e4571aa4df4b offset: 0 snap -1 2024-09-13T10:11:11.807 INFO:tasks.rados.rados.0.smithi195.stdout:991: read oid 41 snap -1 2024-09-13T10:11:11.807 INFO:tasks.rados.rados.0.smithi195.stdout:991: expect (ObjNum 141 snap 0 seq_num 141) 2024-09-13T10:11:11.807 INFO:tasks.rados.rados.0.smithi195.stdout:992: read oid 49 snap -1 2024-09-13T10:11:11.807 INFO:tasks.rados.rados.0.smithi195.stdout:992: expect (ObjNum 133 snap 0 seq_num 133) 2024-09-13T10:11:11.807 INFO:tasks.rados.rados.0.smithi195.stdout:993: read oid 4 snap -1 2024-09-13T10:11:11.807 INFO:tasks.rados.rados.0.smithi195.stdout:993: expect (ObjNum 163 snap 0 seq_num 163) 2024-09-13T10:11:11.807 INFO:tasks.rados.rados.0.smithi195.stdout:994: read oid 13 snap -1 2024-09-13T10:11:11.807 INFO:tasks.rados.rados.0.smithi195.stdout:994: expect (ObjNum 56 snap 0 seq_num 56) 2024-09-13T10:11:11.807 INFO:tasks.rados.rados.0.smithi195.stdout:995: tier_flush oid 22 2024-09-13T10:11:11.808 INFO:tasks.rados.rados.0.smithi195.stdout:995: tier_flush oid 22 snap -1 2024-09-13T10:11:11.808 INFO:tasks.rados.rados.0.smithi195.stdout:996: read oid 29 snap -1 2024-09-13T10:11:11.808 INFO:tasks.rados.rados.0.smithi195.stdout:996: expect deleted 2024-09-13T10:11:11.808 INFO:tasks.rados.rados.0.smithi195.stdout:997: write (excl) oid 32 current snap is 0 2024-09-13T10:11:11.808 INFO:tasks.rados.rados.0.smithi195.stdout:997: seq_num 171 ranges {473997=627345,1673362=441052,2529567=344430} 2024-09-13T10:11:11.808 INFO:tasks.rados.rados.0.smithi195.stdout:997: writing smithi19530714-32 from 473997 to 1101342 tid 1 2024-09-13T10:11:11.808 INFO:tasks.rados.rados.0.smithi195.stdout:997: writing smithi19530714-32 from 1673362 to 2114414 tid 2 2024-09-13T10:11:11.809 INFO:tasks.rados.rados.0.smithi195.stdout:997: writing smithi19530714-32 from 2529567 to 2873997 tid 3 2024-09-13T10:11:11.809 INFO:tasks.rados.rados.0.smithi195.stdout:998: write (excl) oid 20 current snap is 0 2024-09-13T10:11:11.809 INFO:tasks.rados.rados.0.smithi195.stdout:998: seq_num 172 ranges {490804=702594,1615491=584710,2724284=166520} 2024-09-13T10:11:11.821 INFO:tasks.rados.rados.0.smithi195.stdout:998: writing smithi19530714-20 from 490804 to 1193398 tid 1 2024-09-13T10:11:11.831 INFO:tasks.rados.rados.0.smithi195.stdout:998: writing smithi19530714-20 from 1615491 to 2200201 tid 2 2024-09-13T10:11:11.837 INFO:tasks.rados.rados.0.smithi195.stdout:998: writing smithi19530714-20 from 2724284 to 2890804 tid 3 2024-09-13T10:11:11.837 INFO:tasks.rados.rados.0.smithi195.stdout:999: read oid 31 snap -1 2024-09-13T10:11:11.837 INFO:tasks.rados.rados.0.smithi195.stdout:999: expect (ObjNum 93 snap 0 seq_num 93) 2024-09-13T10:11:11.837 INFO:tasks.rados.rados.0.smithi195.stdout:1000: write (excl) oid 5 current snap is 0 2024-09-13T10:11:11.837 INFO:tasks.rados.rados.0.smithi195.stdout:1000: seq_num 173 ranges {507611=777843,2073973=444721} 2024-09-13T10:11:11.849 INFO:tasks.rados.rados.0.smithi195.stdout:1000: writing smithi19530714-5 from 507611 to 1285454 tid 1 2024-09-13T10:11:11.858 INFO:tasks.rados.rados.0.smithi195.stdout:1000: writing smithi19530714-5 from 2073973 to 2518694 tid 2 2024-09-13T10:11:11.859 INFO:tasks.rados.rados.0.smithi195.stdout:1001: read oid 16 snap -1 2024-09-13T10:11:11.859 INFO:tasks.rados.rados.0.smithi195.stdout:1001: expect (ObjNum 128 snap 0 seq_num 128) 2024-09-13T10:11:11.859 INFO:tasks.rados.rados.0.smithi195.stdout:1002: read oid 3 snap -1 2024-09-13T10:11:11.859 INFO:tasks.rados.rados.0.smithi195.stdout:1002: expect deleted 2024-09-13T10:11:11.859 INFO:tasks.rados.rados.0.smithi195.stdout:1003: write (excl) oid 39 current snap is 0 2024-09-13T10:11:11.859 INFO:tasks.rados.rados.0.smithi195.stdout:1003: seq_num 174 ranges {524418=453092,1732455=588379,2779130=145288} 2024-09-13T10:11:11.864 INFO:tasks.rados.rados.0.smithi195.stdout:1003: writing smithi19530714-39 from 524418 to 977510 tid 1 2024-09-13T10:11:11.872 INFO:tasks.rados.rados.0.smithi195.stdout:1003: writing smithi19530714-39 from 1732455 to 2320834 tid 2 2024-09-13T10:11:11.876 INFO:tasks.rados.rados.0.smithi195.stdout:1003: writing smithi19530714-39 from 2779130 to 2924418 tid 3 2024-09-13T10:11:11.876 INFO:tasks.rados.rados.0.smithi195.stdout:1004: tier_evict oid 35 2024-09-13T10:11:11.877 INFO:tasks.rados.rados.0.smithi195.stdout:1004: tier_evict oid 35 snap -1 2024-09-13T10:11:11.877 INFO:tasks.rados.rados.0.smithi195.stdout:991: expect (ObjNum 141 snap 0 seq_num 141) 2024-09-13T10:11:11.877 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:11.903 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:11.903+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:11.937 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:11.937 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:11.937 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:11:11.937 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:11.951 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-13T10:11:11.953 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1405629 2024-09-13T10:11:12.004 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:12.001+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:12.023 INFO:tasks.rados.rados.0.smithi195.stdout:990: finishing set_chunk to oid 40 2024-09-13T10:11:12.023 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 17 (ObjNum 154 snap 0 seq_num 154) dirty exists 2024-09-13T10:11:12.023 INFO:tasks.rados.rados.0.smithi195.stdout:974: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:12.023 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:12.062 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:12.062 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:12.062 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:11:12.062 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:11:12.062 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:12.069 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 116628 2024-09-13T10:11:12.074 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-13T10:11:12.092 INFO:tasks.rados.rados.0.smithi195.stdout:992: expect (ObjNum 133 snap 0 seq_num 133) 2024-09-13T10:11:12.101 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1463226 2024-09-13T10:11:12.105 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:12.105+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:12.160 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2041665 2024-09-13T10:11:12.179 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1846673 2024-09-13T10:11:12.196 INFO:tasks.rados.rados.0.smithi195.stdout:994: expect (ObjNum 56 snap 0 seq_num 56) 2024-09-13T10:11:12.197 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:12.202 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:12.202 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:11:12.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:11:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:11:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:89 4.9 4:96fcf081:::smithi19530714-23:head [write 769754~688184,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.809155+0000", 2024-09-13T10:11:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.390936843999999, 2024-09-13T10:11:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011562566, 2024-09-13T10:11:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.203 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 89 2024-09-13T10:11:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.809155+0000", 2024-09-13T10:11:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.809155+0000", 2024-09-13T10:11:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.809158+0000", 2024-09-13T10:11:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5770000000000001e-06 2024-09-13T10:11:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810210+0000", 2024-09-13T10:11:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010519889999999999 2024-09-13T10:11:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810238+0000", 2024-09-13T10:11:12.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8449000000000002e-05 2024-09-13T10:11:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810265+0000", 2024-09-13T10:11:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6832e-05 2024-09-13T10:11:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810374+0000", 2024-09-13T10:11:12.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109274 2024-09-13T10:11:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810519+0000", 2024-09-13T10:11:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014449299999999999 2024-09-13T10:11:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:12.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810684+0000", 2024-09-13T10:11:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000165744 2024-09-13T10:11:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816442+0000", 2024-09-13T10:11:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057578990000000004 2024-09-13T10:11:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816685+0000", 2024-09-13T10:11:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024254999999999999 2024-09-13T10:11:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820614+0000", 2024-09-13T10:11:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0039290970000000003 2024-09-13T10:11:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820668+0000", 2024-09-13T10:11:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3567999999999999e-05 2024-09-13T10:11:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820718+0000", 2024-09-13T10:11:12.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0093999999999998e-05 2024-09-13T10:11:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:90 4.9 4:96fcf081:::smithi19530714-23:head [write 2021192~348562] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810322+0000", 2024-09-13T10:11:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.389769608999998, 2024-09-13T10:11:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01166355, 2024-09-13T10:11:12.211 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 90 2024-09-13T10:11:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810322+0000", 2024-09-13T10:11:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810322+0000", 2024-09-13T10:11:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810324+0000", 2024-09-13T10:11:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.097e-06 2024-09-13T10:11:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810494+0000", 2024-09-13T10:11:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016961499999999999 2024-09-13T10:11:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810515+0000", 2024-09-13T10:11:12.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0939999999999999e-05 2024-09-13T10:11:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810533+0000", 2024-09-13T10:11:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7909000000000001e-05 2024-09-13T10:11:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.814429+0000", 2024-09-13T10:11:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0038957409999999999 2024-09-13T10:11:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.814603+0000", 2024-09-13T10:11:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174294 2024-09-13T10:11:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.814804+0000", 2024-09-13T10:11:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020149799999999999 2024-09-13T10:11:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820569+0000", 2024-09-13T10:11:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057648389999999999 2024-09-13T10:11:12.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820761+0000", 2024-09-13T10:11:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019123 2024-09-13T10:11:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821893+0000", 2024-09-13T10:11:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001132051 2024-09-13T10:11:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821949+0000", 2024-09-13T10:11:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6832000000000001e-05 2024-09-13T10:11:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821986+0000", 2024-09-13T10:11:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6504000000000002e-05 2024-09-13T10:11:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.219 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:91 4.9 4:96fcf081:::smithi19530714-23:head [setxattr _header (56) in=7b,truncate 2369754] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810564+0000", 2024-09-13T10:11:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.389528399, 2024-09-13T10:11:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01178625, 2024-09-13T10:11:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.220 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 91 2024-09-13T10:11:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810564+0000", 2024-09-13T10:11:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810564+0000", 2024-09-13T10:11:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810565+0000", 2024-09-13T10:11:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3960000000000001e-06 2024-09-13T10:11:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810574+0000", 2024-09-13T10:11:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9339999999999995e-06 2024-09-13T10:11:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810590+0000", 2024-09-13T10:11:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5858999999999998e-05 2024-09-13T10:11:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810604+0000", 2024-09-13T10:11:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4533e-05 2024-09-13T10:11:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816017+0000", 2024-09-13T10:11:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0054122390000000001 2024-09-13T10:11:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816092+0000", 2024-09-13T10:11:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5537999999999994e-05 2024-09-13T10:11:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816206+0000", 2024-09-13T10:11:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114298 2024-09-13T10:11:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820805+0000", 2024-09-13T10:11:12.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004598347 2024-09-13T10:11:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821735+0000", 2024-09-13T10:11:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000930348 2024-09-13T10:11:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822118+0000", 2024-09-13T10:11:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038271099999999999 2024-09-13T10:11:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822208+0000", 2024-09-13T10:11:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0271000000000004e-05 2024-09-13T10:11:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822350+0000", 2024-09-13T10:11:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014177600000000001 2024-09-13T10:11:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:92 4.9 4:96fcf081:::smithi19530714-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:12.226 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810624+0000", 2024-09-13T10:11:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.389467973999999, 2024-09-13T10:11:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012449517, 2024-09-13T10:11:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 92 2024-09-13T10:11:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810624+0000", 2024-09-13T10:11:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.228 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810624+0000", 2024-09-13T10:11:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810625+0000", 2024-09-13T10:11:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3459999999999999e-06 2024-09-13T10:11:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810638+0000", 2024-09-13T10:11:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2853e-05 2024-09-13T10:11:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810655+0000", 2024-09-13T10:11:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6370000000000001e-05 2024-09-13T10:11:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810668+0000", 2024-09-13T10:11:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3906e-05 2024-09-13T10:11:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816513+0000", 2024-09-13T10:11:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058443369999999998 2024-09-13T10:11:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816657+0000", 2024-09-13T10:11:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014419199999999999 2024-09-13T10:11:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.232 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822492+0000", 2024-09-13T10:11:12.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.005835107 2024-09-13T10:11:12.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.232 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822740+0000", 2024-09-13T10:11:12.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024784600000000003 2024-09-13T10:11:12.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823074+0000", 2024-09-13T10:11:12.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033356 2024-09-13T10:11:12.233 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.233 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.233 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.233 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:129 4.9 4:998d9bf1:::smithi19530714-33:head [write 537824~473380,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:12.234 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810689+0000", 2024-09-13T10:11:12.234 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.389403199, 2024-09-13T10:11:12.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01301894, 2024-09-13T10:11:12.234 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.234 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.234 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.234 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.234 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.234 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.234 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 129 2024-09-13T10:11:12.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.234 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810689+0000", 2024-09-13T10:11:12.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810689+0000", 2024-09-13T10:11:12.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.236 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810690+0000", 2024-09-13T10:11:12.236 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2330000000000001e-06 2024-09-13T10:11:12.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811247+0000", 2024-09-13T10:11:12.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055654999999999995 2024-09-13T10:11:12.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811264+0000", 2024-09-13T10:11:12.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7813e-05 2024-09-13T10:11:12.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811280+0000", 2024-09-13T10:11:12.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5676e-05 2024-09-13T10:11:12.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816755+0000", 2024-09-13T10:11:12.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0054754139999999996 2024-09-13T10:11:12.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816940+0000", 2024-09-13T10:11:12.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184091 2024-09-13T10:11:12.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.239 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.239 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:12.239 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.817141+0000", 2024-09-13T10:11:12.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020179099999999999 2024-09-13T10:11:12.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.239 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.239 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:12.239 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821746+0000", 2024-09-13T10:11:12.240 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004604543 2024-09-13T10:11:12.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823211+0000", 2024-09-13T10:11:12.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001464824 2024-09-13T10:11:12.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823536+0000", 2024-09-13T10:11:12.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032508599999999998 2024-09-13T10:11:12.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823620+0000", 2024-09-13T10:11:12.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4524999999999995e-05 2024-09-13T10:11:12.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823708+0000", 2024-09-13T10:11:12.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7393999999999997e-05 2024-09-13T10:11:12.242 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.242 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.242 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.243 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:130 4.9 4:998d9bf1:::smithi19530714-33:head [write 1806012~625998] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:12.243 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.811308+0000", 2024-09-13T10:11:12.243 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.388784234999999, 2024-09-13T10:11:12.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012775038000000001, 2024-09-13T10:11:12.243 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.243 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.243 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.243 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.243 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.243 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.243 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 130 2024-09-13T10:11:12.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.243 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.244 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811308+0000", 2024-09-13T10:11:12.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.244 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811308+0000", 2024-09-13T10:11:12.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.244 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811310+0000", 2024-09-13T10:11:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9400000000000001e-06 2024-09-13T10:11:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811989+0000", 2024-09-13T10:11:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00067975999999999996 2024-09-13T10:11:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.245 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.246 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.246 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812006+0000", 2024-09-13T10:11:12.246 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6175999999999999e-05 2024-09-13T10:11:12.246 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.246 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.246 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.246 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812024+0000", 2024-09-13T10:11:12.246 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8813000000000001e-05 2024-09-13T10:11:12.246 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.247 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.247 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.247 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.818639+0000", 2024-09-13T10:11:12.247 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066148639999999998 2024-09-13T10:11:12.247 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.247 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.247 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.247 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.818763+0000", 2024-09-13T10:11:12.247 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012372300000000001 2024-09-13T10:11:12.247 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.247 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.247 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:12.248 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.818947+0000", 2024-09-13T10:11:12.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184394 2024-09-13T10:11:12.248 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.248 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.248 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:12.248 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822370+0000", 2024-09-13T10:11:12.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034224989999999999 2024-09-13T10:11:12.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.249 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823413+0000", 2024-09-13T10:11:12.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001042739 2024-09-13T10:11:12.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.249 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823829+0000", 2024-09-13T10:11:12.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000416477 2024-09-13T10:11:12.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823982+0000", 2024-09-13T10:11:12.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015238599999999999 2024-09-13T10:11:12.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.250 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.250 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824083+0000", 2024-09-13T10:11:12.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000101267 2024-09-13T10:11:12.250 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.250 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.250 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.251 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.251 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:131 4.9 4:998d9bf1:::smithi19530714-33:head [setxattr _header (56) in=7b,truncate 2537824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:12.251 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.812049+0000", 2024-09-13T10:11:12.251 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.388042476999999, 2024-09-13T10:11:12.251 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012575461, 2024-09-13T10:11:12.251 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.251 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.251 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.251 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.251 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.251 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.251 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 131 2024-09-13T10:11:12.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.252 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.252 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812049+0000", 2024-09-13T10:11:12.252 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.252 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812049+0000", 2024-09-13T10:11:12.252 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812051+0000", 2024-09-13T10:11:12.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-13T10:11:12.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.253 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812060+0000", 2024-09-13T10:11:12.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9930000000000001e-06 2024-09-13T10:11:12.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.254 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812078+0000", 2024-09-13T10:11:12.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8266000000000001e-05 2024-09-13T10:11:12.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.254 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812093+0000", 2024-09-13T10:11:12.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5153e-05 2024-09-13T10:11:12.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.254 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820907+0000", 2024-09-13T10:11:12.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088138650000000006 2024-09-13T10:11:12.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.255 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821046+0000", 2024-09-13T10:11:12.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013850900000000001 2024-09-13T10:11:12.255 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:12.255 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821338+0000", 2024-09-13T10:11:12.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029241800000000002 2024-09-13T10:11:12.255 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:12.255 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823249+0000", 2024-09-13T10:11:12.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019105610000000001 2024-09-13T10:11:12.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824157+0000", 2024-09-13T10:11:12.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090782899999999997 2024-09-13T10:11:12.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824422+0000", 2024-09-13T10:11:12.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026587900000000001 2024-09-13T10:11:12.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824497+0000", 2024-09-13T10:11:12.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4443999999999994e-05 2024-09-13T10:11:12.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824625+0000", 2024-09-13T10:11:12.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012811699999999999 2024-09-13T10:11:12.257 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.257 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.257 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.258 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:132 4.9 4:998d9bf1:::smithi19530714-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:12.258 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.812113+0000", 2024-09-13T10:11:12.258 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.387978465, 2024-09-13T10:11:12.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01281326, 2024-09-13T10:11:12.258 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.258 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.258 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:12.258 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.258 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.258 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.258 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 132 2024-09-13T10:11:12.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.259 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812113+0000", 2024-09-13T10:11:12.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812113+0000", 2024-09-13T10:11:12.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812115+0000", 2024-09-13T10:11:12.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.826e-06 2024-09-13T10:11:12.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812121+0000", 2024-09-13T10:11:12.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9540000000000001e-06 2024-09-13T10:11:12.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812158+0000", 2024-09-13T10:11:12.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6619000000000003e-05 2024-09-13T10:11:12.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812175+0000", 2024-09-13T10:11:12.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6662000000000001e-05 2024-09-13T10:11:12.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821791+0000", 2024-09-13T10:11:12.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.009616187 2024-09-13T10:11:12.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:12.263 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821855+0000", 2024-09-13T10:11:12.263 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4183999999999994e-05 2024-09-13T10:11:12.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.263 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.263 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824680+0000", 2024-09-13T10:11:12.263 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0028252749999999999 2024-09-13T10:11:12.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.263 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.263 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824784+0000", 2024-09-13T10:11:12.263 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010365 2024-09-13T10:11:12.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.264 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824927+0000", 2024-09-13T10:11:12.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000142903 2024-09-13T10:11:12.264 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.264 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.264 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.264 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:272 4.9 4:96fcf081:::smithi19530714-23:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:12.264 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.226803+0000", 2024-09-13T10:11:12.264 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.973288751999998, 2024-09-13T10:11:12.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0087344910000000005, 2024-09-13T10:11:12.265 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.265 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.265 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.265 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.265 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.265 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.265 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 272 2024-09-13T10:11:12.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.265 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.265 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.265 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226803+0000", 2024-09-13T10:11:12.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226803+0000", 2024-09-13T10:11:12.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226807+0000", 2024-09-13T10:11:12.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8829999999999999e-06 2024-09-13T10:11:12.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226823+0000", 2024-09-13T10:11:12.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5979999999999999e-05 2024-09-13T10:11:12.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226859+0000", 2024-09-13T10:11:12.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6331999999999997e-05 2024-09-13T10:11:12.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226882+0000", 2024-09-13T10:11:12.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2452000000000001e-05 2024-09-13T10:11:12.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.227037+0000", 2024-09-13T10:11:12.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000155563 2024-09-13T10:11:12.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.227232+0000", 2024-09-13T10:11:12.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019476500000000001 2024-09-13T10:11:12.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:12.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.233676+0000", 2024-09-13T10:11:12.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0064435719999999998 2024-09-13T10:11:12.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235081+0000", 2024-09-13T10:11:12.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001404953 2024-09-13T10:11:12.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:12.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235114+0000", 2024-09-13T10:11:12.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2996000000000001e-05 2024-09-13T10:11:12.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235286+0000", 2024-09-13T10:11:12.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171965 2024-09-13T10:11:12.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235431+0000", 2024-09-13T10:11:12.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000145205 2024-09-13T10:11:12.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235538+0000", 2024-09-13T10:11:12.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010682499999999999 2024-09-13T10:11:12.271 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.271 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.271 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.271 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:282 4.9 4:998d9bf1:::smithi19530714-33:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:12.271 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.240459+0000", 2024-09-13T10:11:12.272 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.959633097000001, 2024-09-13T10:11:12.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.022877255999999999, 2024-09-13T10:11:12.272 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.272 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.272 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.272 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.272 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.272 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.272 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 282 2024-09-13T10:11:12.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.272 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240459+0000", 2024-09-13T10:11:12.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240459+0000", 2024-09-13T10:11:12.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240462+0000", 2024-09-13T10:11:12.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.971e-06 2024-09-13T10:11:12.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240479+0000", 2024-09-13T10:11:12.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6937000000000001e-05 2024-09-13T10:11:12.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240512+0000", 2024-09-13T10:11:12.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3012999999999997e-05 2024-09-13T10:11:12.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.275 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240533+0000", 2024-09-13T10:11:12.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1138999999999999e-05 2024-09-13T10:11:12.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.275 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240687+0000", 2024-09-13T10:11:12.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000154275 2024-09-13T10:11:12.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240829+0000", 2024-09-13T10:11:12.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000141619 2024-09-13T10:11:12.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:12.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.260691+0000", 2024-09-13T10:11:12.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.019861706 2024-09-13T10:11:12.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:12.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.262189+0000", 2024-09-13T10:11:12.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0014983069999999999 2024-09-13T10:11:12.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.277 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.277 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.262975+0000", 2024-09-13T10:11:12.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00078655900000000004 2024-09-13T10:11:12.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.277 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.277 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.263100+0000", 2024-09-13T10:11:12.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012431899999999999 2024-09-13T10:11:12.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.263232+0000", 2024-09-13T10:11:12.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013227399999999999 2024-09-13T10:11:12.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.263336+0000", 2024-09-13T10:11:12.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104137 2024-09-13T10:11:12.278 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.278 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.278 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.279 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:296 4.9 4:9bec18ef:::smithi19530714-47:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:12.279 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.261575+0000", 2024-09-13T10:11:12.279 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.938517155, 2024-09-13T10:11:12.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0097761170000000008, 2024-09-13T10:11:12.279 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.279 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.279 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.279 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.279 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.279 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.279 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 296 2024-09-13T10:11:12.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.280 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261575+0000", 2024-09-13T10:11:12.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261575+0000", 2024-09-13T10:11:12.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261579+0000", 2024-09-13T10:11:12.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7740000000000002e-06 2024-09-13T10:11:12.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261594+0000", 2024-09-13T10:11:12.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4973999999999999e-05 2024-09-13T10:11:12.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261625+0000", 2024-09-13T10:11:12.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1519000000000001e-05 2024-09-13T10:11:12.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261645+0000", 2024-09-13T10:11:12.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9568e-05 2024-09-13T10:11:12.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261758+0000", 2024-09-13T10:11:12.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011301700000000001 2024-09-13T10:11:12.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261902+0000", 2024-09-13T10:11:12.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014435300000000001 2024-09-13T10:11:12.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:12.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.269548+0000", 2024-09-13T10:11:12.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0076456850000000002 2024-09-13T10:11:12.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.270867+0000", 2024-09-13T10:11:12.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013193930000000001 2024-09-13T10:11:12.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:12.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.270884+0000", 2024-09-13T10:11:12.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7116999999999999e-05 2024-09-13T10:11:12.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.271218+0000", 2024-09-13T10:11:12.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033386800000000002 2024-09-13T10:11:12.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.271305+0000", 2024-09-13T10:11:12.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6738999999999997e-05 2024-09-13T10:11:12.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.285 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.271351+0000", 2024-09-13T10:11:12.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6109999999999997e-05 2024-09-13T10:11:12.285 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.285 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.285 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.286 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:322 4.9 4:96fcf081:::smithi19530714-23:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:12.286 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.279243+0000", 2024-09-13T10:11:12.286 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.920849423, 2024-09-13T10:11:12.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0076374249999999998, 2024-09-13T10:11:12.286 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.286 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.286 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.286 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.286 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.286 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.286 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 322 2024-09-13T10:11:12.286 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.287 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.279243+0000", 2024-09-13T10:11:12.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.279243+0000", 2024-09-13T10:11:12.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.279244+0000", 2024-09-13T10:11:12.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9539999999999998e-06 2024-09-13T10:11:12.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.279253+0000", 2024-09-13T10:11:12.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0040000000000005e-06 2024-09-13T10:11:12.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.279275+0000", 2024-09-13T10:11:12.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1637e-05 2024-09-13T10:11:12.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.279286+0000", 2024-09-13T10:11:12.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0794e-05 2024-09-13T10:11:12.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.279409+0000", 2024-09-13T10:11:12.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012336500000000001 2024-09-13T10:11:12.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.279591+0000", 2024-09-13T10:11:12.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018197299999999999 2024-09-13T10:11:12.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:12.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.284745+0000", 2024-09-13T10:11:12.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0051533200000000003 2024-09-13T10:11:12.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285975+0000", 2024-09-13T10:11:12.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001230128 2024-09-13T10:11:12.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:12.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286521+0000", 2024-09-13T10:11:12.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00054675999999999998 2024-09-13T10:11:12.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286643+0000", 2024-09-13T10:11:12.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000121088 2024-09-13T10:11:12.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286779+0000", 2024-09-13T10:11:12.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000136933 2024-09-13T10:11:12.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286880+0000", 2024-09-13T10:11:12.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000100469 2024-09-13T10:11:12.292 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.292 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.292 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.293 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:346 4.9 4:9bec18ef:::smithi19530714-47:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:12.293 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.285952+0000", 2024-09-13T10:11:12.293 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.914140306, 2024-09-13T10:11:12.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0081934079999999992, 2024-09-13T10:11:12.293 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.293 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.293 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.293 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.293 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.293 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.293 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 346 2024-09-13T10:11:12.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.294 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285952+0000", 2024-09-13T10:11:12.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285952+0000", 2024-09-13T10:11:12.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.295 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.295 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285954+0000", 2024-09-13T10:11:12.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3530000000000002e-06 2024-09-13T10:11:12.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285965+0000", 2024-09-13T10:11:12.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0718e-05 2024-09-13T10:11:12.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285988+0000", 2024-09-13T10:11:12.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3095000000000002e-05 2024-09-13T10:11:12.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286005+0000", 2024-09-13T10:11:12.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7147999999999999e-05 2024-09-13T10:11:12.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286119+0000", 2024-09-13T10:11:12.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000113999 2024-09-13T10:11:12.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286281+0000", 2024-09-13T10:11:12.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000161948 2024-09-13T10:11:12.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:12.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.291713+0000", 2024-09-13T10:11:12.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0054324810000000003 2024-09-13T10:11:12.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293743+0000", 2024-09-13T10:11:12.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020298439999999998 2024-09-13T10:11:12.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293904+0000", 2024-09-13T10:11:12.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000160853 2024-09-13T10:11:12.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:12.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293952+0000", 2024-09-13T10:11:12.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7991999999999997e-05 2024-09-13T10:11:12.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294052+0000", 2024-09-13T10:11:12.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9893000000000001e-05 2024-09-13T10:11:12.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.300 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.300 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294145+0000", 2024-09-13T10:11:12.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3084000000000005e-05 2024-09-13T10:11:12.300 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.300 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.300 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.300 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:801 4.9 4:96fcf081:::smithi19530714-23:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:12.300 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.798334+0000", 2024-09-13T10:11:12.300 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 20.401757948, 2024-09-13T10:11:12.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056549887, 2024-09-13T10:11:12.300 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.301 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.301 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:12.301 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.302 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.302 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.302 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 801 2024-09-13T10:11:12.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.302 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.302 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798334+0000", 2024-09-13T10:11:12.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.302 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798334+0000", 2024-09-13T10:11:12.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798338+0000", 2024-09-13T10:11:12.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3239999999999996e-06 2024-09-13T10:11:12.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798355+0000", 2024-09-13T10:11:12.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7149000000000001e-05 2024-09-13T10:11:12.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798403+0000", 2024-09-13T10:11:12.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7132000000000002e-05 2024-09-13T10:11:12.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798431+0000", 2024-09-13T10:11:12.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8365999999999999e-05 2024-09-13T10:11:12.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798534+0000", 2024-09-13T10:11:12.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102545 2024-09-13T10:11:12.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798715+0000", 2024-09-13T10:11:12.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018161800000000001 2024-09-13T10:11:12.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.854884+0000", 2024-09-13T10:11:12.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056168753000000002 2024-09-13T10:11:12.306 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.306 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.306 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.306 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1065 4.19 4:998d9bf1:::smithi19530714-33:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:12.306 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.618527+0000", 2024-09-13T10:11:12.307 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.581565231999999, 2024-09-13T10:11:12.307 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010820649999999999, 2024-09-13T10:11:12.307 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.307 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.307 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:12.307 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.307 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.307 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.307 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1065 2024-09-13T10:11:12.307 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.308 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618527+0000", 2024-09-13T10:11:12.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618527+0000", 2024-09-13T10:11:12.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618531+0000", 2024-09-13T10:11:12.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9899999999999999e-06 2024-09-13T10:11:12.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618546+0000", 2024-09-13T10:11:12.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5733e-05 2024-09-13T10:11:12.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618603+0000", 2024-09-13T10:11:12.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6075000000000001e-05 2024-09-13T10:11:12.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618635+0000", 2024-09-13T10:11:12.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2053e-05 2024-09-13T10:11:12.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618731+0000", 2024-09-13T10:11:12.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6514000000000005e-05 2024-09-13T10:11:12.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618887+0000", 2024-09-13T10:11:12.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000155415 2024-09-13T10:11:12.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.311 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.311 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.311 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.629347+0000", 2024-09-13T10:11:12.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010460870000000001 2024-09-13T10:11:12.312 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.312 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.312 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.312 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(osd.2.0:47 4.3 e46/41 4:c5b5ff75:::smithi19530714-20:head v 46'28, mlcod=43'26)", 2024-09-13T10:11:12.312 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:01.500294+0000", 2024-09-13T10:11:12.312 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.69979807, 2024-09-13T10:11:12.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010309196, 2024-09-13T10:11:12.312 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.312 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.312 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.313 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.500294+0000", 2024-09-13T10:11:12.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.500294+0000", 2024-09-13T10:11:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.500296+0000", 2024-09-13T10:11:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2800000000000002e-06 2024-09-13T10:11:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505222+0000", 2024-09-13T10:11:12.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0049261269999999998 2024-09-13T10:11:12.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505252+0000", 2024-09-13T10:11:12.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9272999999999999e-05 2024-09-13T10:11:12.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505266+0000", 2024-09-13T10:11:12.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4459e-05 2024-09-13T10:11:12.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505445+0000", 2024-09-13T10:11:12.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017880899999999999 2024-09-13T10:11:12.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505501+0000", 2024-09-13T10:11:12.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5926e-05 2024-09-13T10:11:12.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.510523+0000", 2024-09-13T10:11:12.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0050222440000000004 2024-09-13T10:11:12.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.510603+0000", 2024-09-13T10:11:12.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0078000000000001e-05 2024-09-13T10:11:12.317 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.317 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.317 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.317 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1440 4.19 4:9bec18ef:::smithi19530714-47:head [read 0~2773122,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-09-13T10:11:12.318 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:02.134868+0000", 2024-09-13T10:11:12.318 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.065224329999999, 2024-09-13T10:11:12.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0076280769999999996, 2024-09-13T10:11:12.318 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.318 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.318 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:12.318 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.318 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.318 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.319 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1440 2024-09-13T10:11:12.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.319 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:02.134868+0000", 2024-09-13T10:11:12.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:02.134868+0000", 2024-09-13T10:11:12.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:02.134869+0000", 2024-09-13T10:11:12.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1629999999999999e-06 2024-09-13T10:11:12.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:02.134874+0000", 2024-09-13T10:11:12.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3870000000000003e-06 2024-09-13T10:11:12.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:02.134886+0000", 2024-09-13T10:11:12.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1463e-05 2024-09-13T10:11:12.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:02.134895+0000", 2024-09-13T10:11:12.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1970000000000002e-06 2024-09-13T10:11:12.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:02.139008+0000", 2024-09-13T10:11:12.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0041131800000000001 2024-09-13T10:11:12.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:02.139144+0000", 2024-09-13T10:11:12.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013626 2024-09-13T10:11:12.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:02.142496+0000", 2024-09-13T10:11:12.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033514270000000001 2024-09-13T10:11:12.322 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.322 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.323 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.323 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1619 4.19 4:998d9bf1:::smithi19530714-33:head [write 1750386~495032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:12.323 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.073959+0000", 2024-09-13T10:11:12.323 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.1261330630000002, 2024-09-13T10:11:12.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0081975420000000004, 2024-09-13T10:11:12.323 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.323 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.323 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.323 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.323 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.324 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.324 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1619 2024-09-13T10:11:12.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.324 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.073959+0000", 2024-09-13T10:11:12.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.073959+0000", 2024-09-13T10:11:12.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.073961+0000", 2024-09-13T10:11:12.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3829999999999999e-06 2024-09-13T10:11:12.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.325 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.325 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074180+0000", 2024-09-13T10:11:12.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000218352 2024-09-13T10:11:12.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074208+0000", 2024-09-13T10:11:12.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7880000000000001e-05 2024-09-13T10:11:12.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074222+0000", 2024-09-13T10:11:12.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4250000000000001e-05 2024-09-13T10:11:12.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074360+0000", 2024-09-13T10:11:12.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000138023 2024-09-13T10:11:12.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074587+0000", 2024-09-13T10:11:12.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000226789 2024-09-13T10:11:12.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:12.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074984+0000", 2024-09-13T10:11:12.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000397031 2024-09-13T10:11:12.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:12.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.078275+0000", 2024-09-13T10:11:12.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032917250000000001 2024-09-13T10:11:12.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.080241+0000", 2024-09-13T10:11:12.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019660910000000001 2024-09-13T10:11:12.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.081869+0000", 2024-09-13T10:11:12.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001627325 2024-09-13T10:11:12.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082041+0000", 2024-09-13T10:11:12.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017206599999999999 2024-09-13T10:11:12.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082156+0000", 2024-09-13T10:11:12.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115627 2024-09-13T10:11:12.330 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.330 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.331 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.331 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1620 4.19 4:998d9bf1:::smithi19530714-33:head [setxattr _header (56) in=7b,truncate 2302559] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:12.331 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.074286+0000", 2024-09-13T10:11:12.331 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.1258056969999997, 2024-09-13T10:11:12.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0086728220000000002, 2024-09-13T10:11:12.331 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.331 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.331 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.332 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1620 2024-09-13T10:11:12.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.332 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.332 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074286+0000", 2024-09-13T10:11:12.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074286+0000", 2024-09-13T10:11:12.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074288+0000", 2024-09-13T10:11:12.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.677e-06 2024-09-13T10:11:12.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074298+0000", 2024-09-13T10:11:12.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0020000000000001e-05 2024-09-13T10:11:12.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074316+0000", 2024-09-13T10:11:12.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7546000000000001e-05 2024-09-13T10:11:12.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074330+0000", 2024-09-13T10:11:12.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4399999999999999e-05 2024-09-13T10:11:12.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.076843+0000", 2024-09-13T10:11:12.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025133289999999999 2024-09-13T10:11:12.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.076976+0000", 2024-09-13T10:11:12.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013269000000000001 2024-09-13T10:11:12.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:12.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.077195+0000", 2024-09-13T10:11:12.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000219278 2024-09-13T10:11:12.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:12.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.078998+0000", 2024-09-13T10:11:12.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0018027379999999999 2024-09-13T10:11:12.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082240+0000", 2024-09-13T10:11:12.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032419929999999999 2024-09-13T10:11:12.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082653+0000", 2024-09-13T10:11:12.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041340499999999999 2024-09-13T10:11:12.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082806+0000", 2024-09-13T10:11:12.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015261300000000001 2024-09-13T10:11:12.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082959+0000", 2024-09-13T10:11:12.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015313300000000001 2024-09-13T10:11:12.338 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.338 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.338 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.338 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1621 4.19 4:998d9bf1:::smithi19530714-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:12.338 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.074359+0000", 2024-09-13T10:11:12.338 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.1257329179999997, 2024-09-13T10:11:12.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0092381359999999992, 2024-09-13T10:11:12.338 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.338 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.338 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:12.338 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.339 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.339 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.339 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1621 2024-09-13T10:11:12.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.339 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074359+0000", 2024-09-13T10:11:12.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074359+0000", 2024-09-13T10:11:12.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074361+0000", 2024-09-13T10:11:12.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4699999999999999e-06 2024-09-13T10:11:12.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074387+0000", 2024-09-13T10:11:12.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6472e-05 2024-09-13T10:11:12.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074402+0000", 2024-09-13T10:11:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5437000000000001e-05 2024-09-13T10:11:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074411+0000", 2024-09-13T10:11:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8699999999999998e-06 2024-09-13T10:11:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.078356+0000", 2024-09-13T10:11:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003944894 2024-09-13T10:11:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.078565+0000", 2024-09-13T10:11:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020851800000000001 2024-09-13T10:11:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.083093+0000", 2024-09-13T10:11:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045284660000000001 2024-09-13T10:11:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.083319+0000", 2024-09-13T10:11:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022566200000000001 2024-09-13T10:11:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.083597+0000", 2024-09-13T10:11:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027834700000000002 2024-09-13T10:11:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.343 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.344 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:12.345 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-13T10:11:12.346 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:12.346 INFO:tasks.rados.rados.0.smithi195.stdout:993: expect (ObjNum 163 snap 0 seq_num 163) 2024-09-13T10:11:12.346 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:12.343+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:12.446 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:12.443+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:12.476 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:12.517 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:12.517 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:12.517 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:11:12.517 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:12.531 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-13T10:11:12.549 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:12.548+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:12.550 INFO:tasks.rados.rados.0.smithi195.stdout:997: finishing write tid 1 to smithi19530714-32 2024-09-13T10:11:12.550 INFO:tasks.rados.rados.0.smithi195.stdout:995: got (0) Success 2024-09-13T10:11:12.551 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 0 (ObjNum 139 snap 0 seq_num 139) dirty exists 2024-09-13T10:11:12.551 INFO:tasks.rados.rados.0.smithi195.stdout:997: finishing write tid 2 to smithi19530714-32 2024-09-13T10:11:12.551 INFO:tasks.rados.rados.0.smithi195.stdout:997: finishing write tid 3 to smithi19530714-32 2024-09-13T10:11:12.551 INFO:tasks.rados.rados.0.smithi195.stdout:997: finishing write tid 4 to smithi19530714-32 2024-09-13T10:11:12.551 INFO:tasks.rados.rados.0.smithi195.stdout:997: finishing write tid 5 to smithi19530714-32 2024-09-13T10:11:12.551 INFO:tasks.rados.rados.0.smithi195.stdout:997: oid 32 updating version 0 to 47 2024-09-13T10:11:12.551 INFO:tasks.rados.rados.0.smithi195.stdout:997: oid 32 updating version 47 to 49 2024-09-13T10:11:12.551 INFO:tasks.rados.rados.0.smithi195.stdout:997: oid 32 version 49 is already newer than 48 2024-09-13T10:11:12.551 INFO:tasks.rados.rados.0.smithi195.stdout:997: oid 32 version 49 is already newer than 46 2024-09-13T10:11:12.551 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 49 (ObjNum 171 snap 0 seq_num 171) dirty exists 2024-09-13T10:11:12.551 INFO:tasks.rados.rados.0.smithi195.stdout:997: left oid 32 (ObjNum 171 snap 0 seq_num 171) 2024-09-13T10:11:12.551 INFO:tasks.rados.rados.0.smithi195.stdout:998: finishing write tid 1 to smithi19530714-20 2024-09-13T10:11:12.551 INFO:tasks.rados.rados.0.smithi195.stdout:998: finishing write tid 2 to smithi19530714-20 2024-09-13T10:11:12.552 INFO:tasks.rados.rados.0.smithi195.stdout:974: done (15 left) 2024-09-13T10:11:12.552 INFO:tasks.rados.rados.0.smithi195.stdout:990: done (14 left) 2024-09-13T10:11:12.552 INFO:tasks.rados.rados.0.smithi195.stdout:991: done (13 left) 2024-09-13T10:11:12.552 INFO:tasks.rados.rados.0.smithi195.stdout:992: done (12 left) 2024-09-13T10:11:12.552 INFO:tasks.rados.rados.0.smithi195.stdout:993: done (11 left) 2024-09-13T10:11:12.552 INFO:tasks.rados.rados.0.smithi195.stdout:994: done (10 left) 2024-09-13T10:11:12.552 INFO:tasks.rados.rados.0.smithi195.stdout:995: done (9 left) 2024-09-13T10:11:12.552 INFO:tasks.rados.rados.0.smithi195.stdout:996: done (8 left) 2024-09-13T10:11:12.552 INFO:tasks.rados.rados.0.smithi195.stdout:997: done (7 left) 2024-09-13T10:11:12.552 INFO:tasks.rados.rados.0.smithi195.stdout:1005: set_chunk oid 22 target oid 2024-09-13T10:11:12.559 INFO:tasks.rados.rados.0.smithi195.stdout:1005: set_chunk oid 22 offset: 1526784 length: 363008 target oid 9005be416544dcc1056fe0056f27ea4e392f95bc6cdd1ef4d9686f141b96d0b6 offset: 0 snap -1 2024-09-13T10:11:12.559 INFO:tasks.rados.rados.0.smithi195.stdout:1006: read oid 15 snap -1 2024-09-13T10:11:12.559 INFO:tasks.rados.rados.0.smithi195.stdout:1006: expect (ObjNum 89 snap 0 seq_num 89) 2024-09-13T10:11:12.559 INFO:tasks.rados.rados.0.smithi195.stdout:1007: read oid 26 snap -1 2024-09-13T10:11:12.559 INFO:tasks.rados.rados.0.smithi195.stdout:1007: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:12.560 INFO:tasks.rados.rados.0.smithi195.stdout:1008: read oid 21 snap -1 2024-09-13T10:11:12.560 INFO:tasks.rados.rados.0.smithi195.stdout:1008: expect (ObjNum 162 snap 0 seq_num 162) 2024-09-13T10:11:12.560 INFO:tasks.rados.rados.0.smithi195.stdout:1009: tier_evict oid 45 2024-09-13T10:11:12.560 INFO:tasks.rados.rados.0.smithi195.stdout:1009: tier_evict oid 45 snap -1 2024-09-13T10:11:12.560 INFO:tasks.rados.rados.0.smithi195.stdout:998: finishing write tid 3 to smithi19530714-20 2024-09-13T10:11:12.560 INFO:tasks.rados.rados.0.smithi195.stdout:998: finishing write tid 4 to smithi19530714-20 2024-09-13T10:11:12.560 INFO:tasks.rados.rados.0.smithi195.stdout:998: finishing write tid 5 to smithi19530714-20 2024-09-13T10:11:12.560 INFO:tasks.rados.rados.0.smithi195.stdout:998: oid 20 updating version 0 to 43 2024-09-13T10:11:12.560 INFO:tasks.rados.rados.0.smithi195.stdout:998: oid 20 updating version 43 to 46 2024-09-13T10:11:12.560 INFO:tasks.rados.rados.0.smithi195.stdout:998: oid 20 version 46 is already newer than 44 2024-09-13T10:11:12.560 INFO:tasks.rados.rados.0.smithi195.stdout:998: oid 20 version 46 is already newer than 45 2024-09-13T10:11:12.560 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 46 (ObjNum 172 snap 0 seq_num 172) dirty exists 2024-09-13T10:11:12.560 INFO:tasks.rados.rados.0.smithi195.stdout:998: left oid 20 (ObjNum 172 snap 0 seq_num 172) 2024-09-13T10:11:12.561 INFO:tasks.rados.rados.0.smithi195.stdout:999: expect (ObjNum 93 snap 0 seq_num 93) 2024-09-13T10:11:12.561 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:12.596 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:12.633 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:12.649 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:12.647+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:12.667 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:12.667 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:12.667 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:11:12.667 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:11:12.667 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:12.668 INFO:tasks.rados.rados.0.smithi195.stdout:1000: finishing write tid 1 to smithi19530714-5 2024-09-13T10:11:12.669 INFO:tasks.rados.rados.0.smithi195.stdout:1000: finishing write tid 2 to smithi19530714-5 2024-09-13T10:11:12.669 INFO:tasks.rados.rados.0.smithi195.stdout:1000: finishing write tid 3 to smithi19530714-5 2024-09-13T10:11:12.669 INFO:tasks.rados.rados.0.smithi195.stdout:1000: finishing write tid 4 to smithi19530714-5 2024-09-13T10:11:12.669 INFO:tasks.rados.rados.0.smithi195.stdout:1000: oid 5 updating version 0 to 6 2024-09-13T10:11:12.669 INFO:tasks.rados.rados.0.smithi195.stdout:1000: oid 5 updating version 6 to 7 2024-09-13T10:11:12.669 INFO:tasks.rados.rados.0.smithi195.stdout:1000: oid 5 updating version 7 to 8 2024-09-13T10:11:12.669 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 8 (ObjNum 173 snap 0 seq_num 173) dirty exists 2024-09-13T10:11:12.669 INFO:tasks.rados.rados.0.smithi195.stdout:1000: left oid 5 (ObjNum 173 snap 0 seq_num 173) 2024-09-13T10:11:12.669 INFO:tasks.rados.rados.0.smithi195.stdout:1001: expect (ObjNum 128 snap 0 seq_num 128) 2024-09-13T10:11:12.680 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-13T10:11:12.688 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-13T10:11:12.691 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:12.720 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:11:12.739 INFO:tasks.rados.rados.0.smithi195.stdout:1004: got (22) Invalid argument 2024-09-13T10:11:12.739 INFO:tasks.rados.rados.0.smithi195.stdout:1005: finishing set_chunk to oid 22 2024-09-13T10:11:12.739 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 22 set_chunk returned error code -95 offset: 1526784 length: 363008 2024-09-13T10:11:12.739 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 0 (ObjNum 139 snap 0 seq_num 139) dirty exists 2024-09-13T10:11:12.739 INFO:tasks.rados.rados.0.smithi195.stdout:1007: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:12.740 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:12.751 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:12.748+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:12.799 INFO:tasks.rados.rados.0.smithi195.stdout:1008: expect (ObjNum 162 snap 0 seq_num 162) 2024-09-13T10:11:12.810 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:11:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:11:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:11:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:161 4.d e38/31 4:b19b932a:::smithi19530714-41:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:11:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.953290264, 2024-09-13T10:11:12.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056705289999999997, 2024-09-13T10:11:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:11:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:11:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:11:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0999999999999998e-07 2024-09-13T10:11:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856051+0000", 2024-09-13T10:11:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020382130000000001 2024-09-13T10:11:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856079+0000", 2024-09-13T10:11:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8345999999999999e-05 2024-09-13T10:11:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856093+0000", 2024-09-13T10:11:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4353e-05 2024-09-13T10:11:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856232+0000", 2024-09-13T10:11:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013922199999999999 2024-09-13T10:11:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856267+0000", 2024-09-13T10:11:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4823000000000002e-05 2024-09-13T10:11:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.859505+0000", 2024-09-13T10:11:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032373269999999999 2024-09-13T10:11:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.859682+0000", 2024-09-13T10:11:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017753499999999999 2024-09-13T10:11:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.815 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:162 4.d e38/31 4:b19b932a:::smithi19530714-41:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.861006+0000", 2024-09-13T10:11:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.946295987999999, 2024-09-13T10:11:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003007591, 2024-09-13T10:11:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861006+0000", 2024-09-13T10:11:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861006+0000", 2024-09-13T10:11:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861008+0000", 2024-09-13T10:11:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9860000000000001e-06 2024-09-13T10:11:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861797+0000", 2024-09-13T10:11:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00078932099999999995 2024-09-13T10:11:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861821+0000", 2024-09-13T10:11:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4165e-05 2024-09-13T10:11:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861841+0000", 2024-09-13T10:11:12.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9383000000000001e-05 2024-09-13T10:11:12.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861960+0000", 2024-09-13T10:11:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000118997 2024-09-13T10:11:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.821 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861996+0000", 2024-09-13T10:11:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6099000000000001e-05 2024-09-13T10:11:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.863918+0000", 2024-09-13T10:11:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019220489999999999 2024-09-13T10:11:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.864013+0000", 2024-09-13T10:11:12.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5591e-05 2024-09-13T10:11:12.824 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.824 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.824 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.825 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:181 4.d e38/31 4:ba7d2bc5:::smithi19530714-46:head v 38'4, mlcod=38'2)", 2024-09-13T10:11:12.825 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.926727+0000", 2024-09-13T10:11:12.825 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.880575255, 2024-09-13T10:11:12.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0042825909999999997, 2024-09-13T10:11:12.825 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.825 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.825 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.826 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.926727+0000", 2024-09-13T10:11:12.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.926727+0000", 2024-09-13T10:11:12.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.926729+0000", 2024-09-13T10:11:12.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.244e-06 2024-09-13T10:11:12.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.927979+0000", 2024-09-13T10:11:12.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0012504230000000001 2024-09-13T10:11:12.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.928010+0000", 2024-09-13T10:11:12.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1115000000000003e-05 2024-09-13T10:11:12.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.928030+0000", 2024-09-13T10:11:12.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0063000000000001e-05 2024-09-13T10:11:12.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.928185+0000", 2024-09-13T10:11:12.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015451899999999999 2024-09-13T10:11:12.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.928248+0000", 2024-09-13T10:11:12.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3391000000000003e-05 2024-09-13T10:11:12.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.930963+0000", 2024-09-13T10:11:12.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027147170000000002 2024-09-13T10:11:12.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.931009+0000", 2024-09-13T10:11:12.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6118999999999997e-05 2024-09-13T10:11:12.830 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.830 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.830 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.830 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:182 4.d e38/31 4:ba7d2bc5:::smithi19530714-46:head v 38'5, mlcod=38'3)", 2024-09-13T10:11:12.830 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.933744+0000", 2024-09-13T10:11:12.830 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.873557426000001, 2024-09-13T10:11:12.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0031909019999999998, 2024-09-13T10:11:12.831 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.831 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.831 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.831 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.933744+0000", 2024-09-13T10:11:12.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.933744+0000", 2024-09-13T10:11:12.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.933746+0000", 2024-09-13T10:11:12.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.903e-06 2024-09-13T10:11:12.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.934786+0000", 2024-09-13T10:11:12.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001039683 2024-09-13T10:11:12.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.934812+0000", 2024-09-13T10:11:12.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5902e-05 2024-09-13T10:11:12.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.934826+0000", 2024-09-13T10:11:12.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3696000000000001e-05 2024-09-13T10:11:12.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.934896+0000", 2024-09-13T10:11:12.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0117999999999995e-05 2024-09-13T10:11:12.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.934915+0000", 2024-09-13T10:11:12.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9505e-05 2024-09-13T10:11:12.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.936894+0000", 2024-09-13T10:11:12.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019791990000000001 2024-09-13T10:11:12.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.936935+0000", 2024-09-13T10:11:12.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0896000000000003e-05 2024-09-13T10:11:12.836 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.836 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.836 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.837 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:183 4.d e38/31 4:ba7d2bc5:::smithi19530714-46:head v 38'6, mlcod=38'3)", 2024-09-13T10:11:12.837 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.935517+0000", 2024-09-13T10:11:12.837 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.871784712, 2024-09-13T10:11:12.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001882075, 2024-09-13T10:11:12.837 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.837 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.837 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.837 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.935517+0000", 2024-09-13T10:11:12.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.935517+0000", 2024-09-13T10:11:12.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.935519+0000", 2024-09-13T10:11:12.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8199999999999999e-06 2024-09-13T10:11:12.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.935530+0000", 2024-09-13T10:11:12.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1286e-05 2024-09-13T10:11:12.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.935554+0000", 2024-09-13T10:11:12.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4068999999999999e-05 2024-09-13T10:11:12.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.935574+0000", 2024-09-13T10:11:12.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9986000000000001e-05 2024-09-13T10:11:12.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.935844+0000", 2024-09-13T10:11:12.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026950900000000002 2024-09-13T10:11:12.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.935920+0000", 2024-09-13T10:11:12.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6440999999999995e-05 2024-09-13T10:11:12.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.937367+0000", 2024-09-13T10:11:12.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001446941 2024-09-13T10:11:12.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.937399+0000", 2024-09-13T10:11:12.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2023000000000002e-05 2024-09-13T10:11:12.842 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.842 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.842 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.843 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:786 4.d e39/31 4:b19b932a:::smithi19530714-41:head v 39'15, mlcod=39'13)", 2024-09-13T10:11:12.843 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.620217+0000", 2024-09-13T10:11:12.843 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.187084451, 2024-09-13T10:11:12.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033766159999999998, 2024-09-13T10:11:12.843 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.843 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.843 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.843 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.620217+0000", 2024-09-13T10:11:12.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.620217+0000", 2024-09-13T10:11:12.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.620220+0000", 2024-09-13T10:11:12.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6230000000000001e-06 2024-09-13T10:11:12.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.621490+0000", 2024-09-13T10:11:12.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0012697019999999999 2024-09-13T10:11:12.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.621531+0000", 2024-09-13T10:11:12.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1205999999999998e-05 2024-09-13T10:11:12.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.621552+0000", 2024-09-13T10:11:12.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0948999999999999e-05 2024-09-13T10:11:12.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.621651+0000", 2024-09-13T10:11:12.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9482999999999996e-05 2024-09-13T10:11:12.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.621676+0000", 2024-09-13T10:11:12.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4570999999999999e-05 2024-09-13T10:11:12.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.623532+0000", 2024-09-13T10:11:12.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001856218 2024-09-13T10:11:12.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.623594+0000", 2024-09-13T10:11:12.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1864000000000003e-05 2024-09-13T10:11:12.847 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.847 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.847 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.848 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:787 4.d e39/31 4:b19b932a:::smithi19530714-41:head v 39'16, mlcod=39'14)", 2024-09-13T10:11:12.848 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.628023+0000", 2024-09-13T10:11:12.848 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.179278960000001, 2024-09-13T10:11:12.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002858612, 2024-09-13T10:11:12.848 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.848 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.849 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.849 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.628023+0000", 2024-09-13T10:11:12.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.628023+0000", 2024-09-13T10:11:12.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.628025+0000", 2024-09-13T10:11:12.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9800000000000001e-06 2024-09-13T10:11:12.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.628754+0000", 2024-09-13T10:11:12.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00072891500000000001 2024-09-13T10:11:12.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.628773+0000", 2024-09-13T10:11:12.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8719000000000002e-05 2024-09-13T10:11:12.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.628782+0000", 2024-09-13T10:11:12.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6860000000000001e-06 2024-09-13T10:11:12.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.628902+0000", 2024-09-13T10:11:12.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120214 2024-09-13T10:11:12.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.628966+0000", 2024-09-13T10:11:12.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3561999999999999e-05 2024-09-13T10:11:12.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.630810+0000", 2024-09-13T10:11:12.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0018439050000000001 2024-09-13T10:11:12.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.630882+0000", 2024-09-13T10:11:12.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1631000000000006e-05 2024-09-13T10:11:12.855 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.855 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.855 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.856 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:788 4.d e39/31 4:b19b932a:::smithi19530714-41:head v 39'17, mlcod=39'15)", 2024-09-13T10:11:12.856 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.634939+0000", 2024-09-13T10:11:12.856 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.172363162, 2024-09-13T10:11:12.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027623980000000001, 2024-09-13T10:11:12.856 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.856 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.856 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.856 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.634939+0000", 2024-09-13T10:11:12.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.634939+0000", 2024-09-13T10:11:12.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.634955+0000", 2024-09-13T10:11:12.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6209999999999999e-05 2024-09-13T10:11:12.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.635351+0000", 2024-09-13T10:11:12.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000396225 2024-09-13T10:11:12.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.635378+0000", 2024-09-13T10:11:12.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6562000000000001e-05 2024-09-13T10:11:12.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.635397+0000", 2024-09-13T10:11:12.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8836999999999999e-05 2024-09-13T10:11:12.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.635476+0000", 2024-09-13T10:11:12.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9122999999999994e-05 2024-09-13T10:11:12.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.635492+0000", 2024-09-13T10:11:12.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6158e-05 2024-09-13T10:11:12.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.637653+0000", 2024-09-13T10:11:12.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021612160000000001 2024-09-13T10:11:12.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.637701+0000", 2024-09-13T10:11:12.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8066999999999999e-05 2024-09-13T10:11:12.862 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.862 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.862 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.863 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:789 4.d e39/31 4:b19b932a:::smithi19530714-41:head v 39'18, mlcod=39'15)", 2024-09-13T10:11:12.863 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.636020+0000", 2024-09-13T10:11:12.863 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 21.171282010999999, 2024-09-13T10:11:12.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0022949960000000001, 2024-09-13T10:11:12.863 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.863 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.863 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.863 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.636020+0000", 2024-09-13T10:11:12.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.636020+0000", 2024-09-13T10:11:12.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.636021+0000", 2024-09-13T10:11:12.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9599999999999998e-07 2024-09-13T10:11:12.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.636028+0000", 2024-09-13T10:11:12.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3329999999999999e-06 2024-09-13T10:11:12.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.636038+0000", 2024-09-13T10:11:12.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0141e-05 2024-09-13T10:11:12.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.636046+0000", 2024-09-13T10:11:12.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3119999999999996e-06 2024-09-13T10:11:12.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.636731+0000", 2024-09-13T10:11:12.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000685168 2024-09-13T10:11:12.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.636784+0000", 2024-09-13T10:11:12.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3600999999999997e-05 2024-09-13T10:11:12.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.638211+0000", 2024-09-13T10:11:12.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001426681 2024-09-13T10:11:12.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.638315+0000", 2024-09-13T10:11:12.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010386399999999999 2024-09-13T10:11:12.870 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.870 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.871 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.871 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1337 4.d 4:b19b932a:::smithi19530714-41:head [cache-evict] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e46)", 2024-09-13T10:11:12.871 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:00.391725+0000", 2024-09-13T10:11:12.871 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 12.415576429, 2024-09-13T10:11:12.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036187670000000002, 2024-09-13T10:11:12.872 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.872 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.872 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:12.873 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.873 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.873 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.873 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1337 2024-09-13T10:11:12.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.873 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.391725+0000", 2024-09-13T10:11:12.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.391725+0000", 2024-09-13T10:11:12.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.391728+0000", 2024-09-13T10:11:12.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7829999999999999e-06 2024-09-13T10:11:12.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.391741+0000", 2024-09-13T10:11:12.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3166000000000001e-05 2024-09-13T10:11:12.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.391772+0000", 2024-09-13T10:11:12.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1069999999999999e-05 2024-09-13T10:11:12.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.391790+0000", 2024-09-13T10:11:12.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7062e-05 2024-09-13T10:11:12.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.391910+0000", 2024-09-13T10:11:12.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120703 2024-09-13T10:11:12.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op must wait for map", 2024-09-13T10:11:12.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.391918+0000", 2024-09-13T10:11:12.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2570000000000007e-06 2024-09-13T10:11:12.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.395125+0000", 2024-09-13T10:11:12.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032068610000000001 2024-09-13T10:11:12.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.395250+0000", 2024-09-13T10:11:12.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012494700000000001 2024-09-13T10:11:12.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.395344+0000", 2024-09-13T10:11:12.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3918000000000004e-05 2024-09-13T10:11:12.878 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.878 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.878 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.878 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1367 4.d 4:ba7d2bc5:::smithi19530714-46:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T10:11:12.878 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:01.023318+0000", 2024-09-13T10:11:12.879 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.783983709999999, 2024-09-13T10:11:12.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034398089999999998, 2024-09-13T10:11:12.879 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.879 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.879 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:12.879 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.879 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.879 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.879 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1367 2024-09-13T10:11:12.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.880 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.023318+0000", 2024-09-13T10:11:12.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.023318+0000", 2024-09-13T10:11:12.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.023321+0000", 2024-09-13T10:11:12.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9170000000000002e-06 2024-09-13T10:11:12.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.023334+0000", 2024-09-13T10:11:12.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3263e-05 2024-09-13T10:11:12.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.023365+0000", 2024-09-13T10:11:12.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0945999999999998e-05 2024-09-13T10:11:12.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.023383+0000", 2024-09-13T10:11:12.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7546000000000001e-05 2024-09-13T10:11:12.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.023464+0000", 2024-09-13T10:11:12.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1623000000000001e-05 2024-09-13T10:11:12.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.023596+0000", 2024-09-13T10:11:12.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013198100000000001 2024-09-13T10:11:12.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.026758+0000", 2024-09-13T10:11:12.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0031615329999999998 2024-09-13T10:11:12.886 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.886 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.886 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.886 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1665 4.d 4:b19b932a:::smithi19530714-41:head [write 769787~704463,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-13T10:11:12.886 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.752922+0000", 2024-09-13T10:11:12.887 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.0543798449999997, 2024-09-13T10:11:12.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014152090000000001, 2024-09-13T10:11:12.887 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.887 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.887 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.887 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.887 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.888 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.888 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1665 2024-09-13T10:11:12.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.888 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.752922+0000", 2024-09-13T10:11:12.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.752922+0000", 2024-09-13T10:11:12.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.889 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.752927+0000", 2024-09-13T10:11:12.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7430000000000002e-06 2024-09-13T10:11:12.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.753904+0000", 2024-09-13T10:11:12.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00097707799999999993 2024-09-13T10:11:12.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.753959+0000", 2024-09-13T10:11:12.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5099000000000002e-05 2024-09-13T10:11:12.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.753987+0000", 2024-09-13T10:11:12.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7736999999999999e-05 2024-09-13T10:11:12.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.754289+0000", 2024-09-13T10:11:12.893 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000301926 2024-09-13T10:11:12.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.755253+0000", 2024-09-13T10:11:12.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00096452500000000004 2024-09-13T10:11:12.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:12.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.756279+0000", 2024-09-13T10:11:12.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010256919999999999 2024-09-13T10:11:12.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:12.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.765950+0000", 2024-09-13T10:11:12.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096707410000000001 2024-09-13T10:11:12.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766011+0000", 2024-09-13T10:11:12.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1826000000000001e-05 2024-09-13T10:11:12.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766431+0000", 2024-09-13T10:11:12.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041941400000000001 2024-09-13T10:11:12.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.767009+0000", 2024-09-13T10:11:12.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00057826500000000001 2024-09-13T10:11:12.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.898 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.767074+0000", 2024-09-13T10:11:12.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5043999999999996e-05 2024-09-13T10:11:12.898 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.898 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.898 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.898 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1666 4.d 4:b19b932a:::smithi19530714-41:head [write 2267961~101826] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-13T10:11:12.899 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.762495+0000", 2024-09-13T10:11:12.899 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.0448073139999998, 2024-09-13T10:11:12.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061889800000000002, 2024-09-13T10:11:12.899 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.899 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.899 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.899 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.899 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.899 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.899 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1666 2024-09-13T10:11:12.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.900 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762495+0000", 2024-09-13T10:11:12.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762495+0000", 2024-09-13T10:11:12.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762542+0000", 2024-09-13T10:11:12.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7382999999999997e-05 2024-09-13T10:11:12.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762850+0000", 2024-09-13T10:11:12.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030803999999999998 2024-09-13T10:11:12.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762888+0000", 2024-09-13T10:11:12.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7645999999999998e-05 2024-09-13T10:11:12.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762919+0000", 2024-09-13T10:11:12.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0879e-05 2024-09-13T10:11:12.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763057+0000", 2024-09-13T10:11:12.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000138672 2024-09-13T10:11:12.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763221+0000", 2024-09-13T10:11:12.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016337200000000001 2024-09-13T10:11:12.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:12.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763457+0000", 2024-09-13T10:11:12.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023669399999999999 2024-09-13T10:11:12.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:12.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766139+0000", 2024-09-13T10:11:12.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026813819999999999 2024-09-13T10:11:12.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.767138+0000", 2024-09-13T10:11:12.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0009992200000000001 2024-09-13T10:11:12.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768189+0000", 2024-09-13T10:11:12.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001051614 2024-09-13T10:11:12.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768616+0000", 2024-09-13T10:11:12.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042670000000000002 2024-09-13T10:11:12.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768684+0000", 2024-09-13T10:11:12.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7378000000000004e-05 2024-09-13T10:11:12.906 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.906 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.906 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.906 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1667 4.d 4:b19b932a:::smithi19530714-41:head [setxattr _header (56) in=7b,truncate 2369787] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-13T10:11:12.907 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.762978+0000", 2024-09-13T10:11:12.907 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.0443237510000003, 2024-09-13T10:11:12.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0077634310000000003, 2024-09-13T10:11:12.907 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.907 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.907 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:12.907 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.907 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.907 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.907 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1667 2024-09-13T10:11:12.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.907 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762978+0000", 2024-09-13T10:11:12.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762978+0000", 2024-09-13T10:11:12.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.909 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762980+0000", 2024-09-13T10:11:12.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.654e-06 2024-09-13T10:11:12.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.909 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762990+0000", 2024-09-13T10:11:12.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9629999999999993e-06 2024-09-13T10:11:12.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763008+0000", 2024-09-13T10:11:12.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8122e-05 2024-09-13T10:11:12.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763026+0000", 2024-09-13T10:11:12.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7747000000000001e-05 2024-09-13T10:11:12.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.764724+0000", 2024-09-13T10:11:12.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001698726 2024-09-13T10:11:12.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.764905+0000", 2024-09-13T10:11:12.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018053200000000001 2024-09-13T10:11:12.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:12.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.765167+0000", 2024-09-13T10:11:12.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026248100000000001 2024-09-13T10:11:12.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:12.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.767098+0000", 2024-09-13T10:11:12.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019303949999999999 2024-09-13T10:11:12.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768108+0000", 2024-09-13T10:11:12.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010101820000000001 2024-09-13T10:11:12.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.913 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:12.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768766+0000", 2024-09-13T10:11:12.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00065837299999999997 2024-09-13T10:11:12.913 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.913 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.913 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:12.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770530+0000", 2024-09-13T10:11:12.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00176384 2024-09-13T10:11:12.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.914 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.914 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770742+0000", 2024-09-13T10:11:12.914 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000211416 2024-09-13T10:11:12.914 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.914 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.914 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.914 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.914 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1668 4.d 4:b19b932a:::smithi19530714-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-09-13T10:11:12.915 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.763054+0000", 2024-09-13T10:11:12.915 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.0442476530000002, 2024-09-13T10:11:12.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0080519860000000006, 2024-09-13T10:11:12.915 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.915 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.915 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:12.915 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.915 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.915 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.915 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1668 2024-09-13T10:11:12.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.915 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763054+0000", 2024-09-13T10:11:12.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763054+0000", 2024-09-13T10:11:12.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763056+0000", 2024-09-13T10:11:12.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.404e-06 2024-09-13T10:11:12.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763062+0000", 2024-09-13T10:11:12.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1260000000000003e-06 2024-09-13T10:11:12.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763080+0000", 2024-09-13T10:11:12.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7864e-05 2024-09-13T10:11:12.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763093+0000", 2024-09-13T10:11:12.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3702e-05 2024-09-13T10:11:12.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766050+0000", 2024-09-13T10:11:12.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029562400000000002 2024-09-13T10:11:12.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.919 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:12.919 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766127+0000", 2024-09-13T10:11:12.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7469e-05 2024-09-13T10:11:12.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.919 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.919 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770802+0000", 2024-09-13T10:11:12.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0046747309999999997 2024-09-13T10:11:12.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.920 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.920 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770954+0000", 2024-09-13T10:11:12.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015184000000000001 2024-09-13T10:11:12.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.920 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.920 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.771106+0000", 2024-09-13T10:11:12.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015260999999999999 2024-09-13T10:11:12.920 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.921 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.921 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.921 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.921 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(client.4607.0:1665 4.d e49/44 ondisk, result = 0)", 2024-09-13T10:11:12.921 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.763155+0000", 2024-09-13T10:11:12.921 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.0441465279999997, 2024-09-13T10:11:12.921 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002865163, 2024-09-13T10:11:12.921 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.921 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.921 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:12.922 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763155+0000", 2024-09-13T10:11:12.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763155+0000", 2024-09-13T10:11:12.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763157+0000", 2024-09-13T10:11:12.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4300000000000001e-06 2024-09-13T10:11:12.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763165+0000", 2024-09-13T10:11:12.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1200000000000002e-06 2024-09-13T10:11:12.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763184+0000", 2024-09-13T10:11:12.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9253e-05 2024-09-13T10:11:12.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763198+0000", 2024-09-13T10:11:12.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3533000000000001e-05 2024-09-13T10:11:12.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.765999+0000", 2024-09-13T10:11:12.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0028012850000000001 2024-09-13T10:11:12.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766006+0000", 2024-09-13T10:11:12.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2189999999999998e-06 2024-09-13T10:11:12.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766021+0000", 2024-09-13T10:11:12.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4323e-05 2024-09-13T10:11:12.925 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.926 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.926 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.926 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(client.4607.0:1667 4.d e49/44 ondisk, result = 0)", 2024-09-13T10:11:12.926 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.768119+0000", 2024-09-13T10:11:12.926 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 6.0391824700000001, 2024-09-13T10:11:12.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026328739999999999, 2024-09-13T10:11:12.926 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.926 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.926 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:12.927 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768119+0000", 2024-09-13T10:11:12.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768119+0000", 2024-09-13T10:11:12.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768120+0000", 2024-09-13T10:11:12.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.32e-07 2024-09-13T10:11:12.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768123+0000", 2024-09-13T10:11:12.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4939999999999999e-06 2024-09-13T10:11:12.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768132+0000", 2024-09-13T10:11:12.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3119999999999994e-06 2024-09-13T10:11:12.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768139+0000", 2024-09-13T10:11:12.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1400000000000002e-06 2024-09-13T10:11:12.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768747+0000", 2024-09-13T10:11:12.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00060837599999999999 2024-09-13T10:11:12.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768758+0000", 2024-09-13T10:11:12.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1116e-05 2024-09-13T10:11:12.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770752+0000", 2024-09-13T10:11:12.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019940040000000002 2024-09-13T10:11:12.931 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.931 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.931 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.931 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1920 4.d 4:b19b932a:::smithi19530714-41:head [read 0~2369787,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:12.931 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:11.758355+0000", 2024-09-13T10:11:12.931 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.0489464420000001, 2024-09-13T10:11:12.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0052833139999999999, 2024-09-13T10:11:12.931 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.932 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.932 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:12.932 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.932 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.932 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.932 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1920 2024-09-13T10:11:12.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.932 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.932 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.932 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758355+0000", 2024-09-13T10:11:12.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758355+0000", 2024-09-13T10:11:12.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758358+0000", 2024-09-13T10:11:12.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7130000000000001e-06 2024-09-13T10:11:12.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758369+0000", 2024-09-13T10:11:12.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0974e-05 2024-09-13T10:11:12.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758405+0000", 2024-09-13T10:11:12.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6272000000000001e-05 2024-09-13T10:11:12.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758422+0000", 2024-09-13T10:11:12.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6322999999999999e-05 2024-09-13T10:11:12.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758592+0000", 2024-09-13T10:11:12.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000170147 2024-09-13T10:11:12.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758876+0000", 2024-09-13T10:11:12.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028450099999999999 2024-09-13T10:11:12.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.763639+0000", 2024-09-13T10:11:12.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047623839999999997 2024-09-13T10:11:12.936 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.936 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.937 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.937 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1921 4.d 4:b19b932a:::smithi19530714-41:head [read 1405629~964158,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:12.937 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:11.758460+0000", 2024-09-13T10:11:12.937 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.048842297, 2024-09-13T10:11:12.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068984509999999999, 2024-09-13T10:11:12.937 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.937 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.937 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:12.938 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.938 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.938 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.938 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1921 2024-09-13T10:11:12.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.938 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758460+0000", 2024-09-13T10:11:12.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758460+0000", 2024-09-13T10:11:12.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758461+0000", 2024-09-13T10:11:12.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-13T10:11:12.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758469+0000", 2024-09-13T10:11:12.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8029999999999997e-06 2024-09-13T10:11:12.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.940 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758490+0000", 2024-09-13T10:11:12.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1365999999999998e-05 2024-09-13T10:11:12.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.940 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758503+0000", 2024-09-13T10:11:12.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3200000000000001e-05 2024-09-13T10:11:12.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.763779+0000", 2024-09-13T10:11:12.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0052761079999999998 2024-09-13T10:11:12.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.764101+0000", 2024-09-13T10:11:12.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032119700000000001 2024-09-13T10:11:12.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.765358+0000", 2024-09-13T10:11:12.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001257467 2024-09-13T10:11:12.942 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.942 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.942 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.943 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1922 4.d 4:b19b932a:::smithi19530714-41:head [sparse-read 1834913~534874] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:12.943 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:11.758578+0000", 2024-09-13T10:11:12.943 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.048723673, 2024-09-13T10:11:12.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007449954, 2024-09-13T10:11:12.943 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:12.943 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:12.943 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:12.943 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:12.943 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:12.943 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:12.943 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1922 2024-09-13T10:11:12.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.944 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:12.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:12.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758578+0000", 2024-09-13T10:11:12.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:12.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758578+0000", 2024-09-13T10:11:12.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:12.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:12.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758580+0000", 2024-09-13T10:11:12.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.79e-06 2024-09-13T10:11:12.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:12.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758587+0000", 2024-09-13T10:11:12.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5599999999999999e-06 2024-09-13T10:11:12.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:12.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758620+0000", 2024-09-13T10:11:12.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3649000000000003e-05 2024-09-13T10:11:12.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:12.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758628+0000", 2024-09-13T10:11:12.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2400000000000007e-06 2024-09-13T10:11:12.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:12.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.765429+0000", 2024-09-13T10:11:12.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068003600000000001 2024-09-13T10:11:12.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:12.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.765651+0000", 2024-09-13T10:11:12.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022221900000000001 2024-09-13T10:11:12.947 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:12.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:12.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:12.948 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.766028+0000", 2024-09-13T10:11:12.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00037713599999999998 2024-09-13T10:11:12.948 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.948 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.948 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.948 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:12.948 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:12.948 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:12.950 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:12.950 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2326655 2024-09-13T10:11:12.950 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:12.951 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:12.951 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:12.952 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T10:11:12.953 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:12.953 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1867731 2024-09-13T10:11:12.953 INFO:tasks.rados.rados.0.smithi195.stdout:1009: got (22) Invalid argument 2024-09-13T10:11:12.953 INFO:tasks.rados.rados.0.smithi195.stdout:1006: expect (ObjNum 89 snap 0 seq_num 89) 2024-09-13T10:11:12.953 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:12.954 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:12.954 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:12.947+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:12.954 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:12.955 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:12.955 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:12.955 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:12.967 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:12.967 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:12.977 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:13.005 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:13.005 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:13.026 INFO:tasks.rados.rados.0.smithi195.stdout:998: done (11 left) 2024-09-13T10:11:13.026 INFO:tasks.rados.rados.0.smithi195.stdout:999: done (10 left) 2024-09-13T10:11:13.026 INFO:tasks.rados.rados.0.smithi195.stdout:1000: done (9 left) 2024-09-13T10:11:13.026 INFO:tasks.rados.rados.0.smithi195.stdout:1001: done (8 left) 2024-09-13T10:11:13.026 INFO:tasks.rados.rados.0.smithi195.stdout:1002: done (7 left) 2024-09-13T10:11:13.026 INFO:tasks.rados.rados.0.smithi195.stdout:1004: done (6 left) 2024-09-13T10:11:13.027 INFO:tasks.rados.rados.0.smithi195.stdout:1005: done (5 left) 2024-09-13T10:11:13.027 INFO:tasks.rados.rados.0.smithi195.stdout:1006: done (4 left) 2024-09-13T10:11:13.027 INFO:tasks.rados.rados.0.smithi195.stdout:1007: done (3 left) 2024-09-13T10:11:13.027 INFO:tasks.rados.rados.0.smithi195.stdout:1008: done (2 left) 2024-09-13T10:11:13.027 INFO:tasks.rados.rados.0.smithi195.stdout:1009: done (1 left) 2024-09-13T10:11:13.027 INFO:tasks.rados.rados.0.smithi195.stdout:1010: read oid 46 snap -1 2024-09-13T10:11:13.027 INFO:tasks.rados.rados.0.smithi195.stdout:1010: expect deleted 2024-09-13T10:11:13.027 INFO:tasks.rados.rados.0.smithi195.stdout:1011: read oid 38 snap -1 2024-09-13T10:11:13.027 INFO:tasks.rados.rados.0.smithi195.stdout:1011: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:11:13.027 INFO:tasks.rados.rados.0.smithi195.stdout:1012: delete oid 32 current snap is 0 2024-09-13T10:11:13.031 INFO:tasks.rados.rados.0.smithi195.stdout:1010: done (3 left) 2024-09-13T10:11:13.031 INFO:tasks.rados.rados.0.smithi195.stdout:1012: done (2 left) 2024-09-13T10:11:13.031 INFO:tasks.rados.rados.0.smithi195.stdout:1013: write oid 2 current snap is 0 2024-09-13T10:11:13.031 INFO:tasks.rados.rados.0.smithi195.stdout:1013: seq_num 176 ranges {558032=719943,1849419=592048} 2024-09-13T10:11:13.040 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:13.040 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:13.051 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:13.048+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:13.055 INFO:tasks.rados.rados.0.smithi195.stdout:1013: writing smithi19530714-2 from 558032 to 1277975 tid 1 2024-09-13T10:11:13.057 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:13.057 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:13.074 INFO:tasks.rados.rados.0.smithi195.stdout:1013: writing smithi19530714-2 from 1849419 to 2441467 tid 2 2024-09-13T10:11:13.074 INFO:tasks.rados.rados.0.smithi195.stdout:1014: write oid 33 current snap is 0 2024-09-13T10:11:13.074 INFO:tasks.rados.rados.0.smithi195.stdout:1014: seq_num 177 ranges {574839=795192,1907901=452059} 2024-09-13T10:11:13.078 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:13.078 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:13.093 INFO:tasks.rados.rados.0.smithi195.stdout:1014: writing smithi19530714-33 from 574839 to 1370031 tid 1 2024-09-13T10:11:13.097 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:13.097 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:13.099 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:13.099 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:13.103 INFO:tasks.rados.rados.0.smithi195.stdout:1014: writing smithi19530714-33 from 1907901 to 2359960 tid 2 2024-09-13T10:11:13.104 INFO:tasks.rados.rados.0.smithi195.stdout:1015: delete oid 31 current snap is 0 2024-09-13T10:11:13.104 INFO:tasks.rados.rados.0.smithi195.stdout:1011: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:11:13.112 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T10:11:13.120 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:13.121 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:13.138 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:13.138 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:13.152 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:13.149+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:13.159 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:13.159 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:13.159 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:13.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:13.177+0000 7ff69e7fc640 0 -- 172.21.15.73:0/1844464415 >> v2:172.21.15.181:6804/1173379175 conn(0x7ff6a821fc30 msgr2=0x7ff6a8222080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:11:13.184 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:13.205 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:13.206 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:13.243 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:13.244 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:13.251 INFO:tasks.rados.rados.0.smithi195.stdout:1013: finishing write tid 1 to smithi19530714-2 2024-09-13T10:11:13.251 INFO:tasks.rados.rados.0.smithi195.stdout:1013: finishing write tid 2 to smithi19530714-2 2024-09-13T10:11:13.251 INFO:tasks.rados.rados.0.smithi195.stdout:1013: finishing write tid 3 to smithi19530714-2 2024-09-13T10:11:13.251 INFO:tasks.rados.rados.0.smithi195.stdout:1013: finishing write tid 4 to smithi19530714-2 2024-09-13T10:11:13.251 INFO:tasks.rados.rados.0.smithi195.stdout:1013: oid 2 updating version 0 to 91 2024-09-13T10:11:13.252 INFO:tasks.rados.rados.0.smithi195.stdout:1013: oid 2 updating version 91 to 93 2024-09-13T10:11:13.252 INFO:tasks.rados.rados.0.smithi195.stdout:1013: oid 2 version 93 is already newer than 92 2024-09-13T10:11:13.252 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 93 (ObjNum 176 snap 0 seq_num 176) dirty exists 2024-09-13T10:11:13.252 INFO:tasks.rados.rados.0.smithi195.stdout:1013: left oid 2 (ObjNum 176 snap 0 seq_num 176) 2024-09-13T10:11:13.252 INFO:tasks.rados.rados.0.smithi195.stdout:1014: finishing write tid 1 to smithi19530714-33 2024-09-13T10:11:13.252 INFO:tasks.rados.rados.0.smithi195.stdout:1014: finishing write tid 2 to smithi19530714-33 2024-09-13T10:11:13.252 INFO:tasks.rados.rados.0.smithi195.stdout:1014: finishing write tid 3 to smithi19530714-33 2024-09-13T10:11:13.252 INFO:tasks.rados.rados.0.smithi195.stdout:1014: finishing write tid 4 to smithi19530714-33 2024-09-13T10:11:13.252 INFO:tasks.rados.rados.0.smithi195.stdout:1014: oid 33 updating version 0 to 37 2024-09-13T10:11:13.252 INFO:tasks.rados.rados.0.smithi195.stdout:1014: oid 33 updating version 37 to 38 2024-09-13T10:11:13.253 INFO:tasks.rados.rados.0.smithi195.stdout:1014: oid 33 version 38 is already newer than 36 2024-09-13T10:11:13.253 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 38 (ObjNum 177 snap 0 seq_num 177) dirty exists 2024-09-13T10:11:13.253 INFO:tasks.rados.rados.0.smithi195.stdout:1014: left oid 33 (ObjNum 177 snap 0 seq_num 177) 2024-09-13T10:11:13.253 INFO:tasks.rados.rados.0.smithi195.stdout:1011: done (4 left) 2024-09-13T10:11:13.253 INFO:tasks.rados.rados.0.smithi195.stdout:1013: done (3 left) 2024-09-13T10:11:13.253 INFO:tasks.rados.rados.0.smithi195.stdout:1014: done (2 left) 2024-09-13T10:11:13.253 INFO:tasks.rados.rados.0.smithi195.stdout:1015: done (1 left) 2024-09-13T10:11:13.253 INFO:tasks.rados.rados.0.smithi195.stdout:1016: read oid 26 snap -1 2024-09-13T10:11:13.253 INFO:tasks.rados.rados.0.smithi195.stdout:1016: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:13.253 INFO:tasks.rados.rados.0.smithi195.stdout:1017: read oid 35 snap -1 2024-09-13T10:11:13.253 INFO:tasks.rados.rados.0.smithi195.stdout:1017: expect (ObjNum 86 snap 0 seq_num 86) 2024-09-13T10:11:13.253 INFO:tasks.rados.rados.0.smithi195.stdout:1018: read oid 50 snap -1 2024-09-13T10:11:13.253 INFO:tasks.rados.rados.0.smithi195.stdout:1018: expect (ObjNum 156 snap 0 seq_num 156) 2024-09-13T10:11:13.253 INFO:tasks.rados.rados.0.smithi195.stdout:1019: write oid 33 current snap is 0 2024-09-13T10:11:13.254 INFO:tasks.rados.rados.0.smithi195.stdout:1019: seq_num 179 ranges {608453=545690,1624865=455728,2632598=375855} 2024-09-13T10:11:13.255 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:13.253+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:13.258 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T10:11:13.264 INFO:tasks.rados.rados.0.smithi195.stdout:1019: writing smithi19530714-33 from 608453 to 1154143 tid 1 2024-09-13T10:11:13.275 INFO:tasks.rados.rados.0.smithi195.stdout:1019: writing smithi19530714-33 from 1624865 to 2080593 tid 2 2024-09-13T10:11:13.285 INFO:tasks.rados.rados.0.smithi195.stdout:1019: writing smithi19530714-33 from 2632598 to 3008453 tid 3 2024-09-13T10:11:13.285 INFO:tasks.rados.rados.0.smithi195.stdout:1020: read oid 42 snap -1 2024-09-13T10:11:13.285 INFO:tasks.rados.rados.0.smithi195.stdout:1020: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-13T10:11:13.286 INFO:tasks.rados.rados.0.smithi195.stdout:1021: read oid 49 snap -1 2024-09-13T10:11:13.286 INFO:tasks.rados.rados.0.smithi195.stdout:1021: expect (ObjNum 133 snap 0 seq_num 133) 2024-09-13T10:11:13.286 INFO:tasks.rados.rados.0.smithi195.stdout:1022: tier_flush oid 14 2024-09-13T10:11:13.286 INFO:tasks.rados.rados.0.smithi195.stdout:1022: tier_flush oid 14 snap -1 2024-09-13T10:11:13.286 INFO:tasks.rados.rados.0.smithi195.stdout:1017: expect (ObjNum 86 snap 0 seq_num 86) 2024-09-13T10:11:13.356 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:13.356+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:13.365 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 692462 2024-09-13T10:11:13.426 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:13.426 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:13.440 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T10:11:13.461 INFO:tasks.rados.rados.0.smithi195.stdout:1016: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:13.490 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:13.489+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:13.493 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1834487 2024-09-13T10:11:13.508 INFO:tasks.rados.rados.0.smithi195.stdout:1018: expect (ObjNum 156 snap 0 seq_num 156) 2024-09-13T10:11:13.508 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:13.558 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:13.556+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:13.563 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:13.563 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:13.577 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T10:11:13.659 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:13.655+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:13.678 INFO:tasks.rados.rados.0.smithi195.stdout:1019: finishing write tid 1 to smithi19530714-33 2024-09-13T10:11:13.678 INFO:tasks.rados.rados.0.smithi195.stdout:1019: finishing write tid 2 to smithi19530714-33 2024-09-13T10:11:13.678 INFO:tasks.rados.rados.0.smithi195.stdout:1019: finishing write tid 3 to smithi19530714-33 2024-09-13T10:11:13.678 INFO:tasks.rados.rados.0.smithi195.stdout:1019: finishing write tid 4 to smithi19530714-33 2024-09-13T10:11:13.678 INFO:tasks.rados.rados.0.smithi195.stdout:1019: finishing write tid 5 to smithi19530714-33 2024-09-13T10:11:13.678 INFO:tasks.rados.rados.0.smithi195.stdout:1019: oid 33 updating version 0 to 40 2024-09-13T10:11:13.678 INFO:tasks.rados.rados.0.smithi195.stdout:1019: oid 33 updating version 40 to 42 2024-09-13T10:11:13.678 INFO:tasks.rados.rados.0.smithi195.stdout:1019: oid 33 version 42 is already newer than 39 2024-09-13T10:11:13.678 INFO:tasks.rados.rados.0.smithi195.stdout:1019: oid 33 version 42 is already newer than 41 2024-09-13T10:11:13.678 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 42 (ObjNum 179 snap 0 seq_num 179) dirty exists 2024-09-13T10:11:13.679 INFO:tasks.rados.rados.0.smithi195.stdout:1019: left oid 33 (ObjNum 179 snap 0 seq_num 179) 2024-09-13T10:11:13.679 INFO:tasks.rados.rados.0.smithi195.stdout:1021: expect (ObjNum 133 snap 0 seq_num 133) 2024-09-13T10:11:13.679 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 203116 2024-09-13T10:11:13.749 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:13.749 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:13.760 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:13.757+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:13.763 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T10:11:13.772 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1778649 2024-09-13T10:11:13.790 INFO:tasks.rados.rados.0.smithi195.stdout:1020: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-13T10:11:13.808 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 936758 2024-09-13T10:11:13.825 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:13.852 INFO:tasks.rados.rados.0.smithi195.stdout:1022: got (0) Success 2024-09-13T10:11:13.852 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 0 (ObjNum 103 snap 0 seq_num 103) dirty exists 2024-09-13T10:11:13.852 INFO:tasks.rados.rados.0.smithi195.stdout:1016: done (7 left) 2024-09-13T10:11:13.852 INFO:tasks.rados.rados.0.smithi195.stdout:1017: done (6 left) 2024-09-13T10:11:13.853 INFO:tasks.rados.rados.0.smithi195.stdout:1018: done (5 left) 2024-09-13T10:11:13.853 INFO:tasks.rados.rados.0.smithi195.stdout:1019: done (4 left) 2024-09-13T10:11:13.853 INFO:tasks.rados.rados.0.smithi195.stdout:1020: done (3 left) 2024-09-13T10:11:13.853 INFO:tasks.rados.rados.0.smithi195.stdout:1021: done (2 left) 2024-09-13T10:11:13.853 INFO:tasks.rados.rados.0.smithi195.stdout:1022: done (1 left) 2024-09-13T10:11:13.853 INFO:tasks.rados.rados.0.smithi195.stdout:1023: read oid 40 snap -1 2024-09-13T10:11:13.853 INFO:tasks.rados.rados.0.smithi195.stdout:1023: expect (ObjNum 154 snap 0 seq_num 154) 2024-09-13T10:11:13.853 INFO:tasks.rados.rados.0.smithi195.stdout:1024: read oid 1 snap -1 2024-09-13T10:11:13.853 INFO:tasks.rados.rados.0.smithi195.stdout:1024: expect (ObjNum 161 snap 0 seq_num 161) 2024-09-13T10:11:13.853 INFO:tasks.rados.rados.0.smithi195.stdout:1025: read oid 15 snap -1 2024-09-13T10:11:13.853 INFO:tasks.rados.rados.0.smithi195.stdout:1025: expect (ObjNum 89 snap 0 seq_num 89) 2024-09-13T10:11:13.853 INFO:tasks.rados.rados.0.smithi195.stdout:1026: write (excl) oid 28 current snap is 0 2024-09-13T10:11:13.853 INFO:tasks.rados.rados.0.smithi195.stdout:1026: seq_num 180 ranges {625260=620939,1683347=599386,2943668=81592} 2024-09-13T10:11:13.861 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:13.859+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:13.874 INFO:tasks.rados.rados.0.smithi195.stdout:1026: writing smithi19530714-28 from 625260 to 1246199 tid 1 2024-09-13T10:11:13.890 INFO:tasks.rados.rados.0.smithi195.stdout:1026: writing smithi19530714-28 from 1683347 to 2282733 tid 2 2024-09-13T10:11:13.902 INFO:tasks.rados.rados.0.smithi195.stdout:1026: writing smithi19530714-28 from 2943668 to 3025260 tid 3 2024-09-13T10:11:13.902 INFO:tasks.rados.rados.0.smithi195.stdout:1027: set_chunk oid 31 target oid 2024-09-13T10:11:13.902 INFO:tasks.rados.rados.0.smithi195.stdout:1027: set_chunk oid 31 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:13.903 INFO:tasks.rados.rados.0.smithi195.stdout:1028: read oid 10 snap -1 2024-09-13T10:11:13.903 INFO:tasks.rados.rados.0.smithi195.stdout:1028: expect (ObjNum 84 snap 0 seq_num 84) 2024-09-13T10:11:13.903 INFO:tasks.rados.rados.0.smithi195.stdout:1029: read oid 49 snap -1 2024-09-13T10:11:13.903 INFO:tasks.rados.rados.0.smithi195.stdout:1029: expect (ObjNum 133 snap 0 seq_num 133) 2024-09-13T10:11:13.903 INFO:tasks.rados.rados.0.smithi195.stdout:1025: expect (ObjNum 89 snap 0 seq_num 89) 2024-09-13T10:11:13.927 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:13.927 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:13.929 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2752149 2024-09-13T10:11:13.941 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-13T10:11:13.989 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:13.989+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:14.014 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3336006 2024-09-13T10:11:14.015 INFO:tasks.rados.rados.0.smithi195.stdout:1023: expect (ObjNum 154 snap 0 seq_num 154) 2024-09-13T10:11:14.015 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:14.063 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:14.060+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:14.064 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:14.064 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:14.067 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1540062 2024-09-13T10:11:14.078 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-13T10:11:14.088 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:14.125 INFO:tasks.rados.rados.0.smithi195.stdout:1024: expect (ObjNum 161 snap 0 seq_num 161) 2024-09-13T10:11:14.126 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:14.164 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:14.164+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:14.234 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:14.245 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:14.246 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:14.260 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-13T10:11:14.265 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:14.264+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:14.289 INFO:tasks.rados.rados.0.smithi195.stdout:1026: finishing write tid 1 to smithi19530714-28 2024-09-13T10:11:14.289 INFO:tasks.rados.rados.0.smithi195.stdout:1026: finishing write tid 2 to smithi19530714-28 2024-09-13T10:11:14.289 INFO:tasks.rados.rados.0.smithi195.stdout:1027: finishing set_chunk to oid 31 2024-09-13T10:11:14.289 INFO:tasks.rados.rados.0.smithi195.stdout:1027: got expected ENOENT (src dne) 2024-09-13T10:11:14.289 INFO:tasks.rados.rados.0.smithi195.stdout:1029: expect (ObjNum 133 snap 0 seq_num 133) 2024-09-13T10:11:14.319 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2228804 2024-09-13T10:11:14.366 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:14.365+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:14.382 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:14.382 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:14.397 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-13T10:11:14.397 INFO:tasks.rados.rados.0.smithi195.stdout:1026: finishing write tid 3 to smithi19530714-28 2024-09-13T10:11:14.398 INFO:tasks.rados.rados.0.smithi195.stdout:1026: finishing write tid 4 to smithi19530714-28 2024-09-13T10:11:14.398 INFO:tasks.rados.rados.0.smithi195.stdout:1026: finishing write tid 5 to smithi19530714-28 2024-09-13T10:11:14.398 INFO:tasks.rados.rados.0.smithi195.stdout:1026: oid 28 updating version 0 to 94 2024-09-13T10:11:14.398 INFO:tasks.rados.rados.0.smithi195.stdout:1026: oid 28 updating version 94 to 95 2024-09-13T10:11:14.398 INFO:tasks.rados.rados.0.smithi195.stdout:1026: oid 28 version 95 is already newer than 93 2024-09-13T10:11:14.398 INFO:tasks.rados.rados.0.smithi195.stdout:1026: oid 28 updating version 95 to 96 2024-09-13T10:11:14.398 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 96 (ObjNum 180 snap 0 seq_num 180) dirty exists 2024-09-13T10:11:14.398 INFO:tasks.rados.rados.0.smithi195.stdout:1026: left oid 28 (ObjNum 180 snap 0 seq_num 180) 2024-09-13T10:11:14.398 INFO:tasks.rados.rados.0.smithi195.stdout:1028: expect (ObjNum 84 snap 0 seq_num 84) 2024-09-13T10:11:14.436 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2910126 2024-09-13T10:11:14.440 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:14.467 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:14.465+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:14.508 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1746428 2024-09-13T10:11:14.515 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:14.515 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:14.529 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-13T10:11:14.547 INFO:tasks.rados.rados.0.smithi195.stdout:1003: finishing write tid 1 to smithi19530714-39 2024-09-13T10:11:14.547 INFO:tasks.rados.rados.0.smithi195.stdout:1003: finishing write tid 2 to smithi19530714-39 2024-09-13T10:11:14.547 INFO:tasks.rados.rados.0.smithi195.stdout:1003: finishing write tid 3 to smithi19530714-39 2024-09-13T10:11:14.548 INFO:tasks.rados.rados.0.smithi195.stdout:1003: finishing write tid 4 to smithi19530714-39 2024-09-13T10:11:14.548 INFO:tasks.rados.rados.0.smithi195.stdout:1003: finishing write tid 5 to smithi19530714-39 2024-09-13T10:11:14.548 INFO:tasks.rados.rados.0.smithi195.stdout:1003: oid 39 updating version 0 to 22 2024-09-13T10:11:14.548 INFO:tasks.rados.rados.0.smithi195.stdout:1003: oid 39 version 22 is already newer than 21 2024-09-13T10:11:14.548 INFO:tasks.rados.rados.0.smithi195.stdout:1003: oid 39 updating version 22 to 24 2024-09-13T10:11:14.548 INFO:tasks.rados.rados.0.smithi195.stdout:1003: oid 39 version 24 is already newer than 23 2024-09-13T10:11:14.548 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 24 (ObjNum 174 snap 0 seq_num 174) dirty exists 2024-09-13T10:11:14.548 INFO:tasks.rados.rados.0.smithi195.stdout:1003: left oid 39 (ObjNum 174 snap 0 seq_num 174) 2024-09-13T10:11:14.548 INFO:tasks.rados.rados.0.smithi195.stdout:1030: read oid 19 snap -1 2024-09-13T10:11:14.548 INFO:tasks.rados.rados.0.smithi195.stdout:1030: expect (ObjNum 160 snap 0 seq_num 160) 2024-09-13T10:11:14.548 INFO:tasks.rados.rados.0.smithi195.stdout:1003: done (8 left) 2024-09-13T10:11:14.548 INFO:tasks.rados.rados.0.smithi195.stdout:1023: done (7 left) 2024-09-13T10:11:14.549 INFO:tasks.rados.rados.0.smithi195.stdout:1024: done (6 left) 2024-09-13T10:11:14.549 INFO:tasks.rados.rados.0.smithi195.stdout:1025: done (5 left) 2024-09-13T10:11:14.549 INFO:tasks.rados.rados.0.smithi195.stdout:1026: done (4 left) 2024-09-13T10:11:14.549 INFO:tasks.rados.rados.0.smithi195.stdout:1027: done (3 left) 2024-09-13T10:11:14.549 INFO:tasks.rados.rados.0.smithi195.stdout:1028: done (2 left) 2024-09-13T10:11:14.549 INFO:tasks.rados.rados.0.smithi195.stdout:1029: done (1 left) 2024-09-13T10:11:14.549 INFO:tasks.rados.rados.0.smithi195.stdout:1031: tier_evict oid 20 2024-09-13T10:11:14.549 INFO:tasks.rados.rados.0.smithi195.stdout:1031: tier_evict oid 20 snap -1 2024-09-13T10:11:14.549 INFO:tasks.rados.rados.0.smithi195.stdout:1032: write oid 49 current snap is 0 2024-09-13T10:11:14.549 INFO:tasks.rados.rados.0.smithi195.stdout:1032: seq_num 181 ranges {642067=696188,2025476=459397,2971091=70976} 2024-09-13T10:11:14.568 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:14.567+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:14.568 INFO:tasks.rados.rados.0.smithi195.stdout:1032: writing smithi19530714-49 from 642067 to 1338255 tid 1 2024-09-13T10:11:14.587 INFO:tasks.rados.rados.0.smithi195.stdout:1032: writing smithi19530714-49 from 2025476 to 2484873 tid 2 2024-09-13T10:11:14.596 INFO:tasks.rados.rados.0.smithi195.stdout:1032: writing smithi19530714-49 from 2971091 to 3042067 tid 3 2024-09-13T10:11:14.596 INFO:tasks.rados.rados.0.smithi195.stdout:1033: write oid 47 current snap is 0 2024-09-13T10:11:14.596 INFO:tasks.rados.rados.0.smithi195.stdout:1033: seq_num 182 ranges {658874=771437,2083958=603055} 2024-09-13T10:11:14.613 INFO:tasks.rados.rados.0.smithi195.stdout:1033: writing smithi19530714-47 from 658874 to 1430311 tid 1 2024-09-13T10:11:14.626 INFO:tasks.rados.rados.0.smithi195.stdout:1033: writing smithi19530714-47 from 2083958 to 2687013 tid 2 2024-09-13T10:11:14.626 INFO:tasks.rados.rados.0.smithi195.stdout:1034: write oid 7 current snap is 0 2024-09-13T10:11:14.626 INFO:tasks.rados.rados.0.smithi195.stdout:1034: seq_num 183 ranges {675681=563039,1858793=746713,3025937=49744} 2024-09-13T10:11:14.635 INFO:tasks.rados.rados.0.smithi195.stdout:1034: writing smithi19530714-7 from 675681 to 1238720 tid 1 2024-09-13T10:11:14.645 INFO:tasks.rados.rados.0.smithi195.stdout:1034: writing smithi19530714-7 from 1858793 to 2605506 tid 2 2024-09-13T10:11:14.646 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:14.646 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:14.649 INFO:tasks.rados.rados.0.smithi195.stdout:1034: writing smithi19530714-7 from 3025937 to 3075681 tid 3 2024-09-13T10:11:14.649 INFO:tasks.rados.rados.0.smithi195.stdout:1035: write oid 1 current snap is 0 2024-09-13T10:11:14.649 INFO:tasks.rados.rados.0.smithi195.stdout:1035: seq_num 184 ranges {692488=638288,1917275=606724} 2024-09-13T10:11:14.657 INFO:tasks.rados.rados.0.smithi195.stdout:1035: writing smithi19530714-1 from 692488 to 1330776 tid 1 2024-09-13T10:11:14.661 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-13T10:11:14.664 INFO:tasks.rados.rados.0.smithi195.stdout:1035: writing smithi19530714-1 from 1917275 to 2523999 tid 2 2024-09-13T10:11:14.664 INFO:tasks.rados.rados.0.smithi195.stdout:1036: read oid 5 snap -1 2024-09-13T10:11:14.665 INFO:tasks.rados.rados.0.smithi195.stdout:1036: expect (ObjNum 173 snap 0 seq_num 173) 2024-09-13T10:11:14.665 INFO:tasks.rados.rados.0.smithi195.stdout:1037: read oid 40 snap -1 2024-09-13T10:11:14.665 INFO:tasks.rados.rados.0.smithi195.stdout:1037: expect (ObjNum 154 snap 0 seq_num 154) 2024-09-13T10:11:14.665 INFO:tasks.rados.rados.0.smithi195.stdout:1038: write oid 48 current snap is 0 2024-09-13T10:11:14.665 INFO:tasks.rados.rados.0.smithi195.stdout:1038: seq_num 185 ranges {709295=713537,1859404=750382} 2024-09-13T10:11:14.668 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:14.665+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:14.674 INFO:tasks.rados.rados.0.smithi195.stdout:1038: writing smithi19530714-48 from 709295 to 1422832 tid 1 2024-09-13T10:11:14.681 INFO:tasks.rados.rados.0.smithi195.stdout:1038: writing smithi19530714-48 from 1859404 to 2609786 tid 2 2024-09-13T10:11:14.681 INFO:tasks.rados.rados.0.smithi195.stdout:1039: read oid 2 snap -1 2024-09-13T10:11:14.682 INFO:tasks.rados.rados.0.smithi195.stdout:1039: expect (ObjNum 176 snap 0 seq_num 176) 2024-09-13T10:11:14.682 INFO:tasks.rados.rados.0.smithi195.stdout:1040: write (excl) oid 11 current snap is 0 2024-09-13T10:11:14.682 INFO:tasks.rados.rados.0.smithi195.stdout:1040: seq_num 186 ranges {726102=788786,1917886=610393,3108206=17896} 2024-09-13T10:11:14.690 INFO:tasks.rados.rados.0.smithi195.stdout:1040: writing smithi19530714-11 from 726102 to 1514888 tid 1 2024-09-13T10:11:14.696 INFO:tasks.rados.rados.0.smithi195.stdout:1040: writing smithi19530714-11 from 1917886 to 2528279 tid 2 2024-09-13T10:11:14.700 INFO:tasks.rados.rados.0.smithi195.stdout:1040: writing smithi19530714-11 from 3108206 to 3126102 tid 3 2024-09-13T10:11:14.700 INFO:tasks.rados.rados.0.smithi195.stdout:1041: read oid 16 snap -1 2024-09-13T10:11:14.700 INFO:tasks.rados.rados.0.smithi195.stdout:1041: expect (ObjNum 128 snap 0 seq_num 128) 2024-09-13T10:11:14.700 INFO:tasks.rados.rados.0.smithi195.stdout:1042: set_chunk oid 36 target oid 2024-09-13T10:11:14.700 INFO:tasks.rados.rados.0.smithi195.stdout:1031: got (22) Invalid argument 2024-09-13T10:11:14.718 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T10:11:14.721 INFO:tasks.rados.rados.0.smithi195.stdout:1042: set_chunk oid 36 offset: 315904 length: 1813504 target oid b035afc457fc47bb3ab256a1f3198bd7247e4f47c97973848faac1fb5b5faaef offset: 0 snap -1 2024-09-13T10:11:14.721 INFO:tasks.rados.rados.0.smithi195.stdout:1031: done (12 left) 2024-09-13T10:11:14.721 INFO:tasks.rados.rados.0.smithi195.stdout:1043: set_chunk oid 23 target oid 2024-09-13T10:11:14.750 INFO:tasks.rados.rados.0.smithi195.stdout:1043: set_chunk oid 23 offset: 242176 length: 2176000 target oid 8e6a7cdce023cc11013be96d8e090980c9e532d4721c77aad6e5966e3e626b45 offset: 0 snap -1 2024-09-13T10:11:14.750 INFO:tasks.rados.rados.0.smithi195.stdout:1044: set_chunk oid 15 target oid 2024-09-13T10:11:14.765 INFO:tasks.rados.rados.0.smithi195.stdout:1044: set_chunk oid 15 offset: 1842688 length: 1078784 target oid 90eeb4b6588ac95580e585263e8503ce8518dc44d7820da473db4ae943687f20 offset: 0 snap -1 2024-09-13T10:11:14.765 INFO:tasks.rados.rados.0.smithi195.stdout:1045: write oid 13 current snap is 0 2024-09-13T10:11:14.765 INFO:tasks.rados.rados.0.smithi195.stdout:1045: seq_num 187 ranges {742909=464035,1976368=754051} 2024-09-13T10:11:14.769 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:14.765+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:14.777 INFO:tasks.rados.rados.0.smithi195.stdout:1045: writing smithi19530714-13 from 742909 to 1206944 tid 1 2024-09-13T10:11:14.782 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:14.782 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:14.792 INFO:tasks.rados.rados.0.smithi195.stdout:1045: writing smithi19530714-13 from 1976368 to 2730419 tid 2 2024-09-13T10:11:14.793 INFO:tasks.rados.rados.0.smithi195.stdout:1046: read oid 34 snap -1 2024-09-13T10:11:14.793 INFO:tasks.rados.rados.0.smithi195.stdout:1046: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:14.793 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:14.793 INFO:tasks.rados.rados.0.smithi195.stdout:1030: expect (ObjNum 160 snap 0 seq_num 160) 2024-09-13T10:11:14.796 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T10:11:14.870 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:14.867+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:14.897 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2637995 2024-09-13T10:11:14.902 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 208636 2024-09-13T10:11:14.929 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:14.929 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:14.943 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T10:11:14.971 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:14.972 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:14.972+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:15.041 INFO:tasks.rados.rados.0.smithi195.stdout:1032: finishing write tid 1 to smithi19530714-49 2024-09-13T10:11:15.041 INFO:tasks.rados.rados.0.smithi195.stdout:1032: finishing write tid 2 to smithi19530714-49 2024-09-13T10:11:15.041 INFO:tasks.rados.rados.0.smithi195.stdout:1032: finishing write tid 3 to smithi19530714-49 2024-09-13T10:11:15.041 INFO:tasks.rados.rados.0.smithi195.stdout:1032: finishing write tid 4 to smithi19530714-49 2024-09-13T10:11:15.041 INFO:tasks.rados.rados.0.smithi195.stdout:1032: finishing write tid 5 to smithi19530714-49 2024-09-13T10:11:15.042 INFO:tasks.rados.rados.0.smithi195.stdout:1032: oid 49 updating version 0 to 97 2024-09-13T10:11:15.042 INFO:tasks.rados.rados.0.smithi195.stdout:1032: oid 49 updating version 97 to 99 2024-09-13T10:11:15.042 INFO:tasks.rados.rados.0.smithi195.stdout:1032: oid 49 updating version 99 to 100 2024-09-13T10:11:15.042 INFO:tasks.rados.rados.0.smithi195.stdout:1032: oid 49 version 100 is already newer than 98 2024-09-13T10:11:15.042 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 100 (ObjNum 181 snap 0 seq_num 181) dirty exists 2024-09-13T10:11:15.042 INFO:tasks.rados.rados.0.smithi195.stdout:1032: left oid 49 (ObjNum 181 snap 0 seq_num 181) 2024-09-13T10:11:15.042 INFO:tasks.rados.rados.0.smithi195.stdout:1033: finishing write tid 1 to smithi19530714-47 2024-09-13T10:11:15.042 INFO:tasks.rados.rados.0.smithi195.stdout:1033: finishing write tid 2 to smithi19530714-47 2024-09-13T10:11:15.042 INFO:tasks.rados.rados.0.smithi195.stdout:1033: finishing write tid 3 to smithi19530714-47 2024-09-13T10:11:15.042 INFO:tasks.rados.rados.0.smithi195.stdout:1033: finishing write tid 4 to smithi19530714-47 2024-09-13T10:11:15.042 INFO:tasks.rados.rados.0.smithi195.stdout:1033: oid 47 updating version 0 to 45 2024-09-13T10:11:15.042 INFO:tasks.rados.rados.0.smithi195.stdout:1033: oid 47 version 45 is already newer than 43 2024-09-13T10:11:15.042 INFO:tasks.rados.rados.0.smithi195.stdout:1033: oid 47 version 45 is already newer than 44 2024-09-13T10:11:15.043 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 45 (ObjNum 182 snap 0 seq_num 182) dirty exists 2024-09-13T10:11:15.043 INFO:tasks.rados.rados.0.smithi195.stdout:1033: left oid 47 (ObjNum 182 snap 0 seq_num 182) 2024-09-13T10:11:15.043 INFO:tasks.rados.rados.0.smithi195.stdout:1034: finishing write tid 1 to smithi19530714-7 2024-09-13T10:11:15.043 INFO:tasks.rados.rados.0.smithi195.stdout:1034: finishing write tid 2 to smithi19530714-7 2024-09-13T10:11:15.043 INFO:tasks.rados.rados.0.smithi195.stdout:1034: finishing write tid 3 to smithi19530714-7 2024-09-13T10:11:15.043 INFO:tasks.rados.rados.0.smithi195.stdout:1034: finishing write tid 4 to smithi19530714-7 2024-09-13T10:11:15.043 INFO:tasks.rados.rados.0.smithi195.stdout:1034: finishing write tid 5 to smithi19530714-7 2024-09-13T10:11:15.043 INFO:tasks.rados.rados.0.smithi195.stdout:1034: oid 7 updating version 0 to 66 2024-09-13T10:11:15.043 INFO:tasks.rados.rados.0.smithi195.stdout:1034: oid 7 version 66 is already newer than 65 2024-09-13T10:11:15.043 INFO:tasks.rados.rados.0.smithi195.stdout:1034: oid 7 version 66 is already newer than 64 2024-09-13T10:11:15.043 INFO:tasks.rados.rados.0.smithi195.stdout:1034: oid 7 version 66 is already newer than 63 2024-09-13T10:11:15.043 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 66 (ObjNum 183 snap 0 seq_num 183) dirty exists 2024-09-13T10:11:15.043 INFO:tasks.rados.rados.0.smithi195.stdout:1034: left oid 7 (ObjNum 183 snap 0 seq_num 183) 2024-09-13T10:11:15.044 INFO:tasks.rados.rados.0.smithi195.stdout:1035: finishing write tid 1 to smithi19530714-1 2024-09-13T10:11:15.044 INFO:tasks.rados.rados.0.smithi195.stdout:1035: finishing write tid 2 to smithi19530714-1 2024-09-13T10:11:15.044 INFO:tasks.rados.rados.0.smithi195.stdout:1036: expect (ObjNum 173 snap 0 seq_num 173) 2024-09-13T10:11:15.066 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:15.066 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:15.073 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:15.069+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:15.080 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T10:11:15.081 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:15.159 INFO:tasks.rados.rados.0.smithi195.stdout:1035: finishing write tid 3 to smithi19530714-1 2024-09-13T10:11:15.159 INFO:tasks.rados.rados.0.smithi195.stdout:1035: finishing write tid 4 to smithi19530714-1 2024-09-13T10:11:15.159 INFO:tasks.rados.rados.0.smithi195.stdout:1035: oid 1 updating version 0 to 54 2024-09-13T10:11:15.159 INFO:tasks.rados.rados.0.smithi195.stdout:1035: oid 1 updating version 54 to 56 2024-09-13T10:11:15.159 INFO:tasks.rados.rados.0.smithi195.stdout:1035: oid 1 version 56 is already newer than 55 2024-09-13T10:11:15.160 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 56 (ObjNum 184 snap 0 seq_num 184) dirty exists 2024-09-13T10:11:15.160 INFO:tasks.rados.rados.0.smithi195.stdout:1035: left oid 1 (ObjNum 184 snap 0 seq_num 184) 2024-09-13T10:11:15.160 INFO:tasks.rados.rados.0.smithi195.stdout:1037: expect (ObjNum 154 snap 0 seq_num 154) 2024-09-13T10:11:15.173 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:15.169+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:15.205 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:15.205 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:15.219 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T10:11:15.224 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:15.225 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:11:15.264 INFO:tasks.rados.rados.0.smithi195.stdout:1038: finishing write tid 1 to smithi19530714-48 2024-09-13T10:11:15.264 INFO:tasks.rados.rados.0.smithi195.stdout:1038: finishing write tid 2 to smithi19530714-48 2024-09-13T10:11:15.264 INFO:tasks.rados.rados.0.smithi195.stdout:1038: finishing write tid 3 to smithi19530714-48 2024-09-13T10:11:15.264 INFO:tasks.rados.rados.0.smithi195.stdout:1038: finishing write tid 4 to smithi19530714-48 2024-09-13T10:11:15.264 INFO:tasks.rados.rados.0.smithi195.stdout:1038: oid 48 updating version 0 to 58 2024-09-13T10:11:15.264 INFO:tasks.rados.rados.0.smithi195.stdout:1038: oid 48 version 58 is already newer than 57 2024-09-13T10:11:15.264 INFO:tasks.rados.rados.0.smithi195.stdout:1038: oid 48 version 58 is already newer than 56 2024-09-13T10:11:15.264 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 58 (ObjNum 185 snap 0 seq_num 185) dirty exists 2024-09-13T10:11:15.265 INFO:tasks.rados.rados.0.smithi195.stdout:1038: left oid 48 (ObjNum 185 snap 0 seq_num 185) 2024-09-13T10:11:15.265 INFO:tasks.rados.rados.0.smithi195.stdout:1039: expect (ObjNum 176 snap 0 seq_num 176) 2024-09-13T10:11:15.274 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:15.271+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:15.346 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2912253 2024-09-13T10:11:15.375 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:15.373+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:15.378 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:15.378 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:15.387 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:15.387 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:15.401 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T10:11:15.405 INFO:tasks.rados.rados.0.smithi195.stdout:1040: finishing write tid 1 to smithi19530714-11 2024-09-13T10:11:15.405 INFO:tasks.rados.rados.0.smithi195.stdout:1040: finishing write tid 2 to smithi19530714-11 2024-09-13T10:11:15.406 INFO:tasks.rados.rados.0.smithi195.stdout:1040: finishing write tid 3 to smithi19530714-11 2024-09-13T10:11:15.406 INFO:tasks.rados.rados.0.smithi195.stdout:1040: finishing write tid 4 to smithi19530714-11 2024-09-13T10:11:15.406 INFO:tasks.rados.rados.0.smithi195.stdout:1040: finishing write tid 5 to smithi19530714-11 2024-09-13T10:11:15.406 INFO:tasks.rados.rados.0.smithi195.stdout:1040: oid 11 updating version 0 to 48 2024-09-13T10:11:15.406 INFO:tasks.rados.rados.0.smithi195.stdout:1040: oid 11 updating version 48 to 49 2024-09-13T10:11:15.406 INFO:tasks.rados.rados.0.smithi195.stdout:1040: oid 11 version 49 is already newer than 47 2024-09-13T10:11:15.406 INFO:tasks.rados.rados.0.smithi195.stdout:1040: oid 11 updating version 49 to 50 2024-09-13T10:11:15.406 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 50 (ObjNum 186 snap 0 seq_num 186) dirty exists 2024-09-13T10:11:15.406 INFO:tasks.rados.rados.0.smithi195.stdout:1040: left oid 11 (ObjNum 186 snap 0 seq_num 186) 2024-09-13T10:11:15.406 INFO:tasks.rados.rados.0.smithi195.stdout:1041: expect (ObjNum 128 snap 0 seq_num 128) 2024-09-13T10:11:15.423 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:15.424 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:15.449 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:15.449 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:15.451 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:15.475 INFO:tasks.rados.rados.0.smithi195.stdout:1042: finishing set_chunk to oid 36 2024-09-13T10:11:15.475 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 17 (ObjNum 80 snap 0 seq_num 80) dirty exists 2024-09-13T10:11:15.476 INFO:tasks.rados.rados.0.smithi195.stdout:1043: finishing set_chunk to oid 23 2024-09-13T10:11:15.476 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 23 set_chunk returned error code -95 offset: 242176 length: 2176000 2024-09-13T10:11:15.476 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 0 (ObjNum 90 snap 0 seq_num 90) dirty exists 2024-09-13T10:11:15.476 INFO:tasks.rados.rados.0.smithi195.stdout:1044: finishing set_chunk to oid 15 2024-09-13T10:11:15.476 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 45 (ObjNum 89 snap 0 seq_num 89) dirty exists 2024-09-13T10:11:15.476 INFO:tasks.rados.rados.0.smithi195.stdout:1045: finishing write tid 1 to smithi19530714-13 2024-09-13T10:11:15.476 INFO:tasks.rados.rados.0.smithi195.stdout:1030: done (15 left) 2024-09-13T10:11:15.476 INFO:tasks.rados.rados.0.smithi195.stdout:1032: done (14 left) 2024-09-13T10:11:15.476 INFO:tasks.rados.rados.0.smithi195.stdout:1033: done (13 left) 2024-09-13T10:11:15.476 INFO:tasks.rados.rados.0.smithi195.stdout:1034: done (12 left) 2024-09-13T10:11:15.477 INFO:tasks.rados.rados.0.smithi195.stdout:1035: done (11 left) 2024-09-13T10:11:15.477 INFO:tasks.rados.rados.0.smithi195.stdout:1036: done (10 left) 2024-09-13T10:11:15.477 INFO:tasks.rados.rados.0.smithi195.stdout:1037: done (9 left) 2024-09-13T10:11:15.477 INFO:tasks.rados.rados.0.smithi195.stdout:1038: done (8 left) 2024-09-13T10:11:15.477 INFO:tasks.rados.rados.0.smithi195.stdout:1039: done (7 left) 2024-09-13T10:11:15.477 INFO:tasks.rados.rados.0.smithi195.stdout:1040: done (6 left) 2024-09-13T10:11:15.477 INFO:tasks.rados.rados.0.smithi195.stdout:1041: done (5 left) 2024-09-13T10:11:15.477 INFO:tasks.rados.rados.0.smithi195.stdout:1042: done (4 left) 2024-09-13T10:11:15.477 INFO:tasks.rados.rados.0.smithi195.stdout:1043: done (3 left) 2024-09-13T10:11:15.477 INFO:tasks.rados.rados.0.smithi195.stdout:1044: done (2 left) 2024-09-13T10:11:15.477 INFO:tasks.rados.rados.0.smithi195.stdout:1047: delete oid 36 current snap is 0 2024-09-13T10:11:15.477 INFO:tasks.rados.rados.0.smithi195.stdout:1045: finishing write tid 2 to smithi19530714-13 2024-09-13T10:11:15.478 INFO:tasks.rados.rados.0.smithi195.stdout:1045: finishing write tid 3 to smithi19530714-13 2024-09-13T10:11:15.478 INFO:tasks.rados.rados.0.smithi195.stdout:1045: finishing write tid 4 to smithi19530714-13 2024-09-13T10:11:15.478 INFO:tasks.rados.rados.0.smithi195.stdout:1045: oid 13 updating version 0 to 51 2024-09-13T10:11:15.478 INFO:tasks.rados.rados.0.smithi195.stdout:1045: oid 13 version 51 is already newer than 49 2024-09-13T10:11:15.478 INFO:tasks.rados.rados.0.smithi195.stdout:1045: oid 13 version 51 is already newer than 50 2024-09-13T10:11:15.478 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 51 (ObjNum 187 snap 0 seq_num 187) dirty exists 2024-09-13T10:11:15.478 INFO:tasks.rados.rados.0.smithi195.stdout:1045: left oid 13 (ObjNum 187 snap 0 seq_num 187) 2024-09-13T10:11:15.478 INFO:tasks.rados.rados.0.smithi195.stdout:1046: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:15.478 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:15.475+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:15.479 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:15.479 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:15.500 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:15.500 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:15.501 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2251223 2024-09-13T10:11:15.525 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:15.526 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:15.550 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:15.551 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:15.567 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:15.567 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:15.567 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:11:15.567 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:11:15.567 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:15.575 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:15.575 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:15.576 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:15.575+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:15.581 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T10:11:15.587 INFO:tasks.rados.rados.0.smithi195.stdout:1045: done (2 left) 2024-09-13T10:11:15.588 INFO:tasks.rados.rados.0.smithi195.stdout:1046: done (1 left) 2024-09-13T10:11:15.588 INFO:tasks.rados.rados.0.smithi195.stdout:1047: done (0 left) 2024-09-13T10:11:15.588 INFO:tasks.rados.rados.0.smithi195.stdout:1048: read oid 39 snap -1 2024-09-13T10:11:15.588 INFO:tasks.rados.rados.0.smithi195.stdout:1048: expect (ObjNum 174 snap 0 seq_num 174) 2024-09-13T10:11:15.588 INFO:tasks.rados.rados.0.smithi195.stdout:1049: write (excl) oid 18 current snap is 0 2024-09-13T10:11:15.588 INFO:tasks.rados.rados.0.smithi195.stdout:1049: seq_num 189 ranges {776523=614533,1976979=757720} 2024-09-13T10:11:15.599 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:15.599 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:15.610 INFO:tasks.rados.rados.0.smithi195.stdout:1049: writing smithi19530714-18 from 776523 to 1391056 tid 1 2024-09-13T10:11:15.621 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:15.621 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:15.632 INFO:tasks.rados.rados.0.smithi195.stdout:1049: writing smithi19530714-18 from 1976979 to 2734699 tid 2 2024-09-13T10:11:15.632 INFO:tasks.rados.rados.0.smithi195.stdout:1050: set_chunk oid 25 target oid 2024-09-13T10:11:15.632 INFO:tasks.rados.rados.0.smithi195.stdout:1049: finishing write tid 1 to smithi19530714-18 2024-09-13T10:11:15.641 INFO:tasks.rados.rados.0.smithi195.stdout:1050: set_chunk oid 25 offset: 969216 length: 73728 target oid 1a20085fdcd707c61675d04dfc2afaf96f8db9808c764e767319fc21d011467f offset: 0 snap -1 2024-09-13T10:11:15.641 INFO:tasks.rados.rados.0.smithi195.stdout:1051: tier_evict oid 40 2024-09-13T10:11:15.641 INFO:tasks.rados.rados.0.smithi195.stdout:1051: tier_evict oid 40 snap -1 2024-09-13T10:11:15.641 INFO:tasks.rados.rados.0.smithi195.stdout:1052: write (excl) oid 36 current snap is 0 2024-09-13T10:11:15.641 INFO:tasks.rados.rados.0.smithi195.stdout:1052: seq_num 190 ranges {793330=689782,2035461=617731,3101545=91785} 2024-09-13T10:11:15.646 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:15.646 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:15.666 INFO:tasks.rados.rados.0.smithi195.stdout:1052: writing smithi19530714-36 from 793330 to 1483112 tid 1 2024-09-13T10:11:15.672 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:15.672 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:15.677 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:15.675+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:15.682 INFO:tasks.rados.rados.0.smithi195.stdout:1052: writing smithi19530714-36 from 2035461 to 2653192 tid 2 2024-09-13T10:11:15.688 INFO:tasks.rados.rados.0.smithi195.stdout:1052: writing smithi19530714-36 from 3101545 to 3193330 tid 3 2024-09-13T10:11:15.688 INFO:tasks.rados.rados.0.smithi195.stdout:1053: write oid 4 current snap is 0 2024-09-13T10:11:15.688 INFO:tasks.rados.rados.0.smithi195.stdout:1053: seq_num 191 ranges {410137=481384,1410296=761389,2728968=481169} 2024-09-13T10:11:15.694 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:15.694 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:15.697 INFO:tasks.rados.rados.0.smithi195.stdout:1053: writing smithi19530714-4 from 410137 to 891521 tid 1 2024-09-13T10:11:15.708 INFO:tasks.rados.rados.0.smithi195.stdout:1053: writing smithi19530714-4 from 1410296 to 2171685 tid 2 2024-09-13T10:11:15.715 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:15.715 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:11:15.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:11:15.716 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:11:15.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.716 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:61 4.0 4:0b4c105e:::smithi19530714-16:head [write 652105~445088,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:15.716 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.720236+0000", 2024-09-13T10:11:15.716 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.991084674, 2024-09-13T10:11:15.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086548600000000003, 2024-09-13T10:11:15.716 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.716 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.716 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:15.716 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.717 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 61 2024-09-13T10:11:15.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.717 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720236+0000", 2024-09-13T10:11:15.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720236+0000", 2024-09-13T10:11:15.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720240+0000", 2024-09-13T10:11:15.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3629999999999998e-06 2024-09-13T10:11:15.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720942+0000", 2024-09-13T10:11:15.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070250300000000005 2024-09-13T10:11:15.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720968+0000", 2024-09-13T10:11:15.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5245999999999999e-05 2024-09-13T10:11:15.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720984+0000", 2024-09-13T10:11:15.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6821999999999999e-05 2024-09-13T10:11:15.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721070+0000", 2024-09-13T10:11:15.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5660000000000003e-05 2024-09-13T10:11:15.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721221+0000", 2024-09-13T10:11:15.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015077499999999999 2024-09-13T10:11:15.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:15.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721364+0000", 2024-09-13T10:11:15.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014364299999999999 2024-09-13T10:11:15.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728585+0000", 2024-09-13T10:11:15.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072205680000000001 2024-09-13T10:11:15.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:15.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728595+0000", 2024-09-13T10:11:15.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0108e-05 2024-09-13T10:11:15.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728688+0000", 2024-09-13T10:11:15.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2405999999999996e-05 2024-09-13T10:11:15.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:15.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728769+0000", 2024-09-13T10:11:15.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1174000000000006e-05 2024-09-13T10:11:15.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728891+0000", 2024-09-13T10:11:15.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012259200000000001 2024-09-13T10:11:15.723 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.723 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.724 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.724 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:62 4.0 4:0b4c105e:::smithi19530714-16:head [write 1728171~452988] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:15.724 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.721021+0000", 2024-09-13T10:11:15.724 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.990299644, 2024-09-13T10:11:15.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096364669999999993, 2024-09-13T10:11:15.724 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.724 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.725 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:15.725 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.725 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.725 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.725 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 62 2024-09-13T10:11:15.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.725 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721021+0000", 2024-09-13T10:11:15.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721021+0000", 2024-09-13T10:11:15.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721023+0000", 2024-09-13T10:11:15.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-13T10:11:15.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721744+0000", 2024-09-13T10:11:15.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000721092 2024-09-13T10:11:15.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721766+0000", 2024-09-13T10:11:15.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2391999999999998e-05 2024-09-13T10:11:15.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721779+0000", 2024-09-13T10:11:15.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2864000000000001e-05 2024-09-13T10:11:15.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.724492+0000", 2024-09-13T10:11:15.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027128759999999999 2024-09-13T10:11:15.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.724746+0000", 2024-09-13T10:11:15.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000253914 2024-09-13T10:11:15.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:15.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.725111+0000", 2024-09-13T10:11:15.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036536400000000002 2024-09-13T10:11:15.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728978+0000", 2024-09-13T10:11:15.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038665969999999998 2024-09-13T10:11:15.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:15.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728991+0000", 2024-09-13T10:11:15.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3711e-05 2024-09-13T10:11:15.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.730564+0000", 2024-09-13T10:11:15.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001572564 2024-09-13T10:11:15.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:15.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.730617+0000", 2024-09-13T10:11:15.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2744999999999997e-05 2024-09-13T10:11:15.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.730658+0000", 2024-09-13T10:11:15.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1291999999999997e-05 2024-09-13T10:11:15.730 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.730 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.730 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.730 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:63 4.0 4:0b4c105e:::smithi19530714-16:head [setxattr _header (56) in=7b,truncate 2252105] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:15.731 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.721807+0000", 2024-09-13T10:11:15.731 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.989514101000001, 2024-09-13T10:11:15.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093328749999999992, 2024-09-13T10:11:15.731 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.731 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.731 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:15.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.731 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 63 2024-09-13T10:11:15.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.731 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721807+0000", 2024-09-13T10:11:15.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721807+0000", 2024-09-13T10:11:15.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721809+0000", 2024-09-13T10:11:15.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6559999999999999e-06 2024-09-13T10:11:15.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721818+0000", 2024-09-13T10:11:15.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1530000000000003e-06 2024-09-13T10:11:15.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721836+0000", 2024-09-13T10:11:15.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8240000000000002e-05 2024-09-13T10:11:15.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721854+0000", 2024-09-13T10:11:15.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7532000000000001e-05 2024-09-13T10:11:15.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.727557+0000", 2024-09-13T10:11:15.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057033259999999999 2024-09-13T10:11:15.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.727772+0000", 2024-09-13T10:11:15.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021469600000000001 2024-09-13T10:11:15.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:15.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728048+0000", 2024-09-13T10:11:15.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027671299999999999 2024-09-13T10:11:15.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.729819+0000", 2024-09-13T10:11:15.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017708489999999999 2024-09-13T10:11:15.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:15.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.730310+0000", 2024-09-13T10:11:15.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049081999999999999 2024-09-13T10:11:15.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731047+0000", 2024-09-13T10:11:15.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073742899999999995 2024-09-13T10:11:15.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:15.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731096+0000", 2024-09-13T10:11:15.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8573999999999999e-05 2024-09-13T10:11:15.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731140+0000", 2024-09-13T10:11:15.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3887000000000002e-05 2024-09-13T10:11:15.737 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.737 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.737 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.737 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:64 4.0 4:0b4c105e:::smithi19530714-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:15.737 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.721878+0000", 2024-09-13T10:11:15.737 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.989442961000002, 2024-09-13T10:11:15.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094623740000000008, 2024-09-13T10:11:15.737 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.737 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.737 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:15.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.738 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 64 2024-09-13T10:11:15.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.738 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721878+0000", 2024-09-13T10:11:15.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721878+0000", 2024-09-13T10:11:15.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721880+0000", 2024-09-13T10:11:15.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3939999999999999e-06 2024-09-13T10:11:15.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721886+0000", 2024-09-13T10:11:15.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0390000000000001e-06 2024-09-13T10:11:15.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721902+0000", 2024-09-13T10:11:15.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6520000000000001e-05 2024-09-13T10:11:15.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721917+0000", 2024-09-13T10:11:15.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4959e-05 2024-09-13T10:11:15.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.729034+0000", 2024-09-13T10:11:15.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071171100000000003 2024-09-13T10:11:15.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:15.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.729161+0000", 2024-09-13T10:11:15.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012695600000000001 2024-09-13T10:11:15.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731172+0000", 2024-09-13T10:11:15.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002010651 2024-09-13T10:11:15.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731241+0000", 2024-09-13T10:11:15.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9646000000000004e-05 2024-09-13T10:11:15.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731341+0000", 2024-09-13T10:11:15.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9098999999999994e-05 2024-09-13T10:11:15.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.742 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:150 4.4 e38/31 4:24275569:::smithi19530714-38:head v 38'11, mlcod=38'9)", 2024-09-13T10:11:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.815678+0000", 2024-09-13T10:11:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.895643515, 2024-09-13T10:11:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066202199999999996, 2024-09-13T10:11:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815678+0000", 2024-09-13T10:11:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815678+0000", 2024-09-13T10:11:15.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815689+0000", 2024-09-13T10:11:15.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1334e-05 2024-09-13T10:11:15.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816641+0000", 2024-09-13T10:11:15.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000952307 2024-09-13T10:11:15.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816667+0000", 2024-09-13T10:11:15.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5636e-05 2024-09-13T10:11:15.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816679+0000", 2024-09-13T10:11:15.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2373e-05 2024-09-13T10:11:15.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816864+0000", 2024-09-13T10:11:15.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018461399999999999 2024-09-13T10:11:15.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816918+0000", 2024-09-13T10:11:15.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4413999999999998e-05 2024-09-13T10:11:15.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:15.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822241+0000", 2024-09-13T10:11:15.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053230430000000004 2024-09-13T10:11:15.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822298+0000", 2024-09-13T10:11:15.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6499000000000002e-05 2024-09-13T10:11:15.747 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.747 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.747 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.747 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:157 4.0 4:03fab808:::smithi19530714-40:head [write 655473~716476,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:15.747 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.836742+0000", 2024-09-13T10:11:15.747 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.874578619000001, 2024-09-13T10:11:15.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0076813209999999996, 2024-09-13T10:11:15.748 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.748 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.748 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:15.748 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.748 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.748 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.748 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 157 2024-09-13T10:11:15.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.748 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.836742+0000", 2024-09-13T10:11:15.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.836742+0000", 2024-09-13T10:11:15.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.836746+0000", 2024-09-13T10:11:15.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1930000000000002e-06 2024-09-13T10:11:15.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837565+0000", 2024-09-13T10:11:15.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081902899999999998 2024-09-13T10:11:15.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837613+0000", 2024-09-13T10:11:15.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8640999999999997e-05 2024-09-13T10:11:15.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837628+0000", 2024-09-13T10:11:15.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4867e-05 2024-09-13T10:11:15.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837701+0000", 2024-09-13T10:11:15.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2862999999999997e-05 2024-09-13T10:11:15.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837831+0000", 2024-09-13T10:11:15.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012971599999999999 2024-09-13T10:11:15.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:15.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837974+0000", 2024-09-13T10:11:15.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014289899999999999 2024-09-13T10:11:15.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:15.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.839710+0000", 2024-09-13T10:11:15.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00173613 2024-09-13T10:11:15.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.842419+0000", 2024-09-13T10:11:15.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027087539999999998 2024-09-13T10:11:15.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844336+0000", 2024-09-13T10:11:15.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001916947 2024-09-13T10:11:15.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:15.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844381+0000", 2024-09-13T10:11:15.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5377000000000003e-05 2024-09-13T10:11:15.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844424+0000", 2024-09-13T10:11:15.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2904999999999997e-05 2024-09-13T10:11:15.755 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.755 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.755 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.756 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:158 4.0 4:03fab808:::smithi19530714-40:head [write 2099033~556440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:15.756 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.843944+0000", 2024-09-13T10:11:15.756 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.867376725, 2024-09-13T10:11:15.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006981821, 2024-09-13T10:11:15.756 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.756 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.756 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:15.756 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.756 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.756 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.756 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 158 2024-09-13T10:11:15.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.757 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.843944+0000", 2024-09-13T10:11:15.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.843944+0000", 2024-09-13T10:11:15.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.843946+0000", 2024-09-13T10:11:15.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.75e-06 2024-09-13T10:11:15.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844474+0000", 2024-09-13T10:11:15.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052765500000000003 2024-09-13T10:11:15.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844496+0000", 2024-09-13T10:11:15.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2552e-05 2024-09-13T10:11:15.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844509+0000", 2024-09-13T10:11:15.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2387e-05 2024-09-13T10:11:15.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844578+0000", 2024-09-13T10:11:15.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9664000000000003e-05 2024-09-13T10:11:15.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844657+0000", 2024-09-13T10:11:15.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8417000000000005e-05 2024-09-13T10:11:15.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:15.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844784+0000", 2024-09-13T10:11:15.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012675800000000001 2024-09-13T10:11:15.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:15.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.846495+0000", 2024-09-13T10:11:15.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017114350000000001 2024-09-13T10:11:15.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.848918+0000", 2024-09-13T10:11:15.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002422783 2024-09-13T10:11:15.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.850771+0000", 2024-09-13T10:11:15.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001853616 2024-09-13T10:11:15.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:15.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.850859+0000", 2024-09-13T10:11:15.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7786000000000003e-05 2024-09-13T10:11:15.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.850926+0000", 2024-09-13T10:11:15.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7018000000000001e-05 2024-09-13T10:11:15.762 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.762 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.762 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.762 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:159 4.0 4:03fab808:::smithi19530714-40:head [setxattr _header (56) in=7b,truncate 2655473] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:15.762 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.844538+0000", 2024-09-13T10:11:15.763 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.866783259000002, 2024-09-13T10:11:15.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067064480000000003, 2024-09-13T10:11:15.763 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.763 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.763 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:15.763 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.763 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.763 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.763 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 159 2024-09-13T10:11:15.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.763 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844538+0000", 2024-09-13T10:11:15.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844538+0000", 2024-09-13T10:11:15.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844539+0000", 2024-09-13T10:11:15.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.223e-06 2024-09-13T10:11:15.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844545+0000", 2024-09-13T10:11:15.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8100000000000003e-06 2024-09-13T10:11:15.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844555+0000", 2024-09-13T10:11:15.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6490000000000003e-06 2024-09-13T10:11:15.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844563+0000", 2024-09-13T10:11:15.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4370000000000007e-06 2024-09-13T10:11:15.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.845602+0000", 2024-09-13T10:11:15.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00103863 2024-09-13T10:11:15.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.845695+0000", 2024-09-13T10:11:15.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2913999999999998e-05 2024-09-13T10:11:15.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:15.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.845812+0000", 2024-09-13T10:11:15.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011715899999999999 2024-09-13T10:11:15.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:15.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.846904+0000", 2024-09-13T10:11:15.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001092733 2024-09-13T10:11:15.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.849487+0000", 2024-09-13T10:11:15.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025827459999999999 2024-09-13T10:11:15.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851147+0000", 2024-09-13T10:11:15.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001659878 2024-09-13T10:11:15.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:15.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851184+0000", 2024-09-13T10:11:15.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7435999999999999e-05 2024-09-13T10:11:15.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851244+0000", 2024-09-13T10:11:15.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9833000000000001e-05 2024-09-13T10:11:15.769 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.769 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.769 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.769 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:160 4.0 4:03fab808:::smithi19530714-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:15.769 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.844579+0000", 2024-09-13T10:11:15.769 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.866741671, 2024-09-13T10:11:15.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006862506, 2024-09-13T10:11:15.769 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.769 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.769 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:15.769 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.770 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.770 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.770 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 160 2024-09-13T10:11:15.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.770 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844579+0000", 2024-09-13T10:11:15.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844579+0000", 2024-09-13T10:11:15.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844580+0000", 2024-09-13T10:11:15.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1699999999999997e-07 2024-09-13T10:11:15.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844585+0000", 2024-09-13T10:11:15.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6360000000000002e-06 2024-09-13T10:11:15.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844595+0000", 2024-09-13T10:11:15.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9739999999999997e-06 2024-09-13T10:11:15.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844604+0000", 2024-09-13T10:11:15.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7560000000000005e-06 2024-09-13T10:11:15.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.846062+0000", 2024-09-13T10:11:15.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001458446 2024-09-13T10:11:15.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:15.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.846121+0000", 2024-09-13T10:11:15.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8697999999999999e-05 2024-09-13T10:11:15.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851297+0000", 2024-09-13T10:11:15.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051764810000000001 2024-09-13T10:11:15.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851355+0000", 2024-09-13T10:11:15.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7794999999999998e-05 2024-09-13T10:11:15.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851442+0000", 2024-09-13T10:11:15.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6903000000000005e-05 2024-09-13T10:11:15.774 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.774 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.774 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.774 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:198 4.0 4:0bd0428c:::smithi19530714-50:head [write 1767500~799008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:15.774 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991140+0000", 2024-09-13T10:11:15.775 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.720180681999999, 2024-09-13T10:11:15.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079451699999999997, 2024-09-13T10:11:15.775 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.775 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.775 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:15.775 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.775 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.775 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.775 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 198 2024-09-13T10:11:15.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.775 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991140+0000", 2024-09-13T10:11:15.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991140+0000", 2024-09-13T10:11:15.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991144+0000", 2024-09-13T10:11:15.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2499999999999998e-06 2024-09-13T10:11:15.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992036+0000", 2024-09-13T10:11:15.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000891931 2024-09-13T10:11:15.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992052+0000", 2024-09-13T10:11:15.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6070999999999999e-05 2024-09-13T10:11:15.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992060+0000", 2024-09-13T10:11:15.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9719999999999992e-06 2024-09-13T10:11:15.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992128+0000", 2024-09-13T10:11:15.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7858e-05 2024-09-13T10:11:15.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992208+0000", 2024-09-13T10:11:15.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0383000000000006e-05 2024-09-13T10:11:15.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:15.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992324+0000", 2024-09-13T10:11:15.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011587 2024-09-13T10:11:15.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:15.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997567+0000", 2024-09-13T10:11:15.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052430419999999998 2024-09-13T10:11:15.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.998107+0000", 2024-09-13T10:11:15.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054063099999999997 2024-09-13T10:11:15.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.998940+0000", 2024-09-13T10:11:15.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083243599999999998 2024-09-13T10:11:15.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:15.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.999021+0000", 2024-09-13T10:11:15.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1335999999999995e-05 2024-09-13T10:11:15.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.999086+0000", 2024-09-13T10:11:15.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4389999999999998e-05 2024-09-13T10:11:15.781 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.781 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.781 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.781 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:199 4.0 4:0bd0428c:::smithi19530714-50:head [setxattr _header (56) in=7b,truncate 2823543] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:15.781 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.992081+0000", 2024-09-13T10:11:15.781 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.719240181, 2024-09-13T10:11:15.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011866829000000001, 2024-09-13T10:11:15.781 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.781 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.782 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:15.782 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.782 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.782 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.782 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 199 2024-09-13T10:11:15.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.782 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992081+0000", 2024-09-13T10:11:15.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992081+0000", 2024-09-13T10:11:15.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992082+0000", 2024-09-13T10:11:15.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.004e-06 2024-09-13T10:11:15.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992090+0000", 2024-09-13T10:11:15.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7440000000000007e-06 2024-09-13T10:11:15.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:11:15.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3650000000000004e-06 2024-09-13T10:11:15.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992107+0000", 2024-09-13T10:11:15.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9230000000000008e-06 2024-09-13T10:11:15.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.995938+0000", 2024-09-13T10:11:15.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003830773 2024-09-13T10:11:15.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.996229+0000", 2024-09-13T10:11:15.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029133300000000001 2024-09-13T10:11:15.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:15.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.996692+0000", 2024-09-13T10:11:15.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046326700000000001 2024-09-13T10:11:15.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:15.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997974+0000", 2024-09-13T10:11:15.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001282016 2024-09-13T10:11:15.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.999243+0000", 2024-09-13T10:11:15.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012682360000000001 2024-09-13T10:11:15.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.003767+0000", 2024-09-13T10:11:15.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045243970000000003 2024-09-13T10:11:15.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:15.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.003851+0000", 2024-09-13T10:11:15.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4195000000000003e-05 2024-09-13T10:11:15.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.003948+0000", 2024-09-13T10:11:15.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6576000000000005e-05 2024-09-13T10:11:15.788 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.788 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.788 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.788 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:200 4.0 4:0bd0428c:::smithi19530714-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:15.788 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.992119+0000", 2024-09-13T10:11:15.788 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.719202021000001, 2024-09-13T10:11:15.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012149748, 2024-09-13T10:11:15.788 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.788 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.788 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:15.788 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.789 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.789 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.789 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 200 2024-09-13T10:11:15.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.789 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992119+0000", 2024-09-13T10:11:15.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992119+0000", 2024-09-13T10:11:15.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992120+0000", 2024-09-13T10:11:15.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9999999999999997e-07 2024-09-13T10:11:15.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992123+0000", 2024-09-13T10:11:15.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9569999999999998e-06 2024-09-13T10:11:15.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992131+0000", 2024-09-13T10:11:15.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2024-09-13T10:11:15.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992140+0000", 2024-09-13T10:11:15.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7900000000000005e-06 2024-09-13T10:11:15.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997418+0000", 2024-09-13T10:11:15.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052784349999999997 2024-09-13T10:11:15.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:15.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997546+0000", 2024-09-13T10:11:15.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012788900000000001 2024-09-13T10:11:15.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.004004+0000", 2024-09-13T10:11:15.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064580549999999999 2024-09-13T10:11:15.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.004128+0000", 2024-09-13T10:11:15.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123628 2024-09-13T10:11:15.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.004269+0000", 2024-09-13T10:11:15.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000141292 2024-09-13T10:11:15.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.793 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.793 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:265 4.0 4:0b4c105e:::smithi19530714-16:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:15.793 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.215372+0000", 2024-09-13T10:11:15.793 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.495948988999999, 2024-09-13T10:11:15.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013030243, 2024-09-13T10:11:15.794 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.794 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.794 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:15.794 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.794 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.794 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.794 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 265 2024-09-13T10:11:15.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.794 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215372+0000", 2024-09-13T10:11:15.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215372+0000", 2024-09-13T10:11:15.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215376+0000", 2024-09-13T10:11:15.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.393e-06 2024-09-13T10:11:15.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215387+0000", 2024-09-13T10:11:15.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1986e-05 2024-09-13T10:11:15.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215421+0000", 2024-09-13T10:11:15.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3355999999999997e-05 2024-09-13T10:11:15.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215438+0000", 2024-09-13T10:11:15.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7059e-05 2024-09-13T10:11:15.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215529+0000", 2024-09-13T10:11:15.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1571999999999997e-05 2024-09-13T10:11:15.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215621+0000", 2024-09-13T10:11:15.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1094999999999994e-05 2024-09-13T10:11:15.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:15.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.223688+0000", 2024-09-13T10:11:15.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080669859999999999 2024-09-13T10:11:15.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:15.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.225291+0000", 2024-09-13T10:11:15.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001603548 2024-09-13T10:11:15.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.225833+0000", 2024-09-13T10:11:15.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054185500000000005 2024-09-13T10:11:15.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228151+0000", 2024-09-13T10:11:15.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023176640000000001 2024-09-13T10:11:15.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:15.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228300+0000", 2024-09-13T10:11:15.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000149152 2024-09-13T10:11:15.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228402+0000", 2024-09-13T10:11:15.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102577 2024-09-13T10:11:15.800 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.800 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.800 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.800 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:289 4.0 4:03fab808:::smithi19530714-40:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:15.800 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.250617+0000", 2024-09-13T10:11:15.800 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.460703703, 2024-09-13T10:11:15.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010740764999999999, 2024-09-13T10:11:15.800 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.801 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.801 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:15.801 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.801 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.801 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.801 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 289 2024-09-13T10:11:15.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.801 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250617+0000", 2024-09-13T10:11:15.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250617+0000", 2024-09-13T10:11:15.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250621+0000", 2024-09-13T10:11:15.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4029999999999999e-06 2024-09-13T10:11:15.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250637+0000", 2024-09-13T10:11:15.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5950000000000001e-05 2024-09-13T10:11:15.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250673+0000", 2024-09-13T10:11:15.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6559e-05 2024-09-13T10:11:15.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250699+0000", 2024-09-13T10:11:15.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5479000000000001e-05 2024-09-13T10:11:15.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250825+0000", 2024-09-13T10:11:15.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012585600000000001 2024-09-13T10:11:15.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250971+0000", 2024-09-13T10:11:15.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000146002 2024-09-13T10:11:15.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:15.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257715+0000", 2024-09-13T10:11:15.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067440360000000001 2024-09-13T10:11:15.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:15.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258695+0000", 2024-09-13T10:11:15.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0009798509999999999 2024-09-13T10:11:15.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.260555+0000", 2024-09-13T10:11:15.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018600439999999999 2024-09-13T10:11:15.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261249+0000", 2024-09-13T10:11:15.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00069435899999999997 2024-09-13T10:11:15.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:15.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261307+0000", 2024-09-13T10:11:15.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7630000000000002e-05 2024-09-13T10:11:15.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261358+0000", 2024-09-13T10:11:15.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1595999999999999e-05 2024-09-13T10:11:15.806 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.806 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.806 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.807 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:299 4.0 4:0bd0428c:::smithi19530714-50:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:15.807 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.264107+0000", 2024-09-13T10:11:15.807 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.447214314, 2024-09-13T10:11:15.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020961186999999999, 2024-09-13T10:11:15.807 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.807 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.807 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:15.807 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.807 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.807 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.807 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 299 2024-09-13T10:11:15.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.808 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264107+0000", 2024-09-13T10:11:15.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264107+0000", 2024-09-13T10:11:15.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264111+0000", 2024-09-13T10:11:15.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0330000000000002e-06 2024-09-13T10:11:15.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264121+0000", 2024-09-13T10:11:15.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0543999999999999e-05 2024-09-13T10:11:15.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264150+0000", 2024-09-13T10:11:15.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8581999999999998e-05 2024-09-13T10:11:15.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264171+0000", 2024-09-13T10:11:15.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-13T10:11:15.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264369+0000", 2024-09-13T10:11:15.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019837000000000001 2024-09-13T10:11:15.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264535+0000", 2024-09-13T10:11:15.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000165324 2024-09-13T10:11:15.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:15.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.276997+0000", 2024-09-13T10:11:15.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012461842000000001 2024-09-13T10:11:15.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:15.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.277946+0000", 2024-09-13T10:11:15.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00094919299999999998 2024-09-13T10:11:15.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.281540+0000", 2024-09-13T10:11:15.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035946910000000001 2024-09-13T10:11:15.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.284932+0000", 2024-09-13T10:11:15.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033913799999999998 2024-09-13T10:11:15.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:15.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285010+0000", 2024-09-13T10:11:15.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8059000000000006e-05 2024-09-13T10:11:15.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285068+0000", 2024-09-13T10:11:15.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8072999999999998e-05 2024-09-13T10:11:15.813 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.813 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.813 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.814 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:489 4.0 4:03fab808:::smithi19530714-40:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:15.814 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.350324+0000", 2024-09-13T10:11:15.814 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.360997297000001, 2024-09-13T10:11:15.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014394344, 2024-09-13T10:11:15.814 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.814 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.814 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:15.814 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.814 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.814 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.814 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 489 2024-09-13T10:11:15.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.814 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350324+0000", 2024-09-13T10:11:15.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350324+0000", 2024-09-13T10:11:15.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350325+0000", 2024-09-13T10:11:15.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6270000000000001e-06 2024-09-13T10:11:15.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350333+0000", 2024-09-13T10:11:15.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3719999999999999e-06 2024-09-13T10:11:15.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350349+0000", 2024-09-13T10:11:15.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6004000000000001e-05 2024-09-13T10:11:15.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350359+0000", 2024-09-13T10:11:15.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.73e-06 2024-09-13T10:11:15.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350455+0000", 2024-09-13T10:11:15.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6260000000000003e-05 2024-09-13T10:11:15.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350526+0000", 2024-09-13T10:11:15.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0946999999999997e-05 2024-09-13T10:11:15.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:15.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.357046+0000", 2024-09-13T10:11:15.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065198510000000001 2024-09-13T10:11:15.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:15.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.357616+0000", 2024-09-13T10:11:15.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056994299999999995 2024-09-13T10:11:15.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.358085+0000", 2024-09-13T10:11:15.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046959399999999999 2024-09-13T10:11:15.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.364636+0000", 2024-09-13T10:11:15.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065509560000000001 2024-09-13T10:11:15.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:15.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.364680+0000", 2024-09-13T10:11:15.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3859999999999997e-05 2024-09-13T10:11:15.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.364718+0000", 2024-09-13T10:11:15.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.82e-05 2024-09-13T10:11:15.820 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.820 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.820 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.820 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.6.0:77 4.4 e39/31 4:244eab67:::smithi19530714-24:head v 39'35, mlcod=39'33)", 2024-09-13T10:11:15.820 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.572069+0000", 2024-09-13T10:11:15.820 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.13925193, 2024-09-13T10:11:15.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072872479999999996, 2024-09-13T10:11:15.820 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.820 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.820 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:15.821 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.572069+0000", 2024-09-13T10:11:15.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.572069+0000", 2024-09-13T10:11:15.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.572072+0000", 2024-09-13T10:11:15.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.909e-06 2024-09-13T10:11:15.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.574286+0000", 2024-09-13T10:11:15.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022137580000000001 2024-09-13T10:11:15.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.574318+0000", 2024-09-13T10:11:15.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2582000000000001e-05 2024-09-13T10:11:15.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.574335+0000", 2024-09-13T10:11:15.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6906999999999999e-05 2024-09-13T10:11:15.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.574487+0000", 2024-09-13T10:11:15.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015193499999999999 2024-09-13T10:11:15.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.574552+0000", 2024-09-13T10:11:15.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4721000000000006e-05 2024-09-13T10:11:15.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:15.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.579275+0000", 2024-09-13T10:11:15.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004723222 2024-09-13T10:11:15.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.579356+0000", 2024-09-13T10:11:15.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1213999999999999e-05 2024-09-13T10:11:15.825 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.825 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.825 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.825 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1001 4.10 4:0bd0428c:::smithi19530714-50:head [read 2021403~802140,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:15.825 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:55.653905+0000", 2024-09-13T10:11:15.825 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.057416189000001, 2024-09-13T10:11:15.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006442316, 2024-09-13T10:11:15.825 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.825 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.825 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:15.825 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.826 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.826 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.826 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1001 2024-09-13T10:11:15.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.826 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:55.653905+0000", 2024-09-13T10:11:15.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:55.653905+0000", 2024-09-13T10:11:15.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:55.653906+0000", 2024-09-13T10:11:15.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5370000000000001e-06 2024-09-13T10:11:15.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:55.653920+0000", 2024-09-13T10:11:15.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.328e-05 2024-09-13T10:11:15.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:55.653940+0000", 2024-09-13T10:11:15.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9848999999999999e-05 2024-09-13T10:11:15.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:55.653956+0000", 2024-09-13T10:11:15.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5979999999999999e-05 2024-09-13T10:11:15.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:55.657972+0000", 2024-09-13T10:11:15.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040162039999999998 2024-09-13T10:11:15.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:55.658204+0000", 2024-09-13T10:11:15.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000232506 2024-09-13T10:11:15.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:55.660347+0000", 2024-09-13T10:11:15.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021429600000000002 2024-09-13T10:11:15.829 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.829 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.829 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.830 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:1587 4.4 e47/41 4:24da8222:::smithi19530714-49:head v 47'70, mlcod=47'68)", 2024-09-13T10:11:15.830 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:05.127296+0000", 2024-09-13T10:11:15.830 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.584024876000001, 2024-09-13T10:11:15.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066656880000000003, 2024-09-13T10:11:15.830 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.830 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.830 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:15.830 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.127296+0000", 2024-09-13T10:11:15.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.127296+0000", 2024-09-13T10:11:15.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.127298+0000", 2024-09-13T10:11:15.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0030000000000001e-06 2024-09-13T10:11:15.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.128661+0000", 2024-09-13T10:11:15.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001362858 2024-09-13T10:11:15.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.128690+0000", 2024-09-13T10:11:15.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9076000000000001e-05 2024-09-13T10:11:15.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.128705+0000", 2024-09-13T10:11:15.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4459e-05 2024-09-13T10:11:15.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.128856+0000", 2024-09-13T10:11:15.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000151273 2024-09-13T10:11:15.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.128918+0000", 2024-09-13T10:11:15.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2001999999999994e-05 2024-09-13T10:11:15.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:15.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.133766+0000", 2024-09-13T10:11:15.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048483850000000002 2024-09-13T10:11:15.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.133962+0000", 2024-09-13T10:11:15.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000195632 2024-09-13T10:11:15.834 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.834 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.834 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.834 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1919 4.0 4:03fab808:::smithi19530714-40:head [assert-version v17,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:15.835 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:11.758517+0000", 2024-09-13T10:11:15.835 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.952803882, 2024-09-13T10:11:15.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082282680000000004, 2024-09-13T10:11:15.835 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:15.835 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:15.835 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:15.835 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:15.835 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:15.835 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:15.835 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1919 2024-09-13T10:11:15.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.835 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:15.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:15.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.758517+0000", 2024-09-13T10:11:15.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:15.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.758517+0000", 2024-09-13T10:11:15.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:15.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:15.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.758520+0000", 2024-09-13T10:11:15.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8499999999999998e-06 2024-09-13T10:11:15.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:15.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.758534+0000", 2024-09-13T10:11:15.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3920000000000001e-05 2024-09-13T10:11:15.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:15.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.758566+0000", 2024-09-13T10:11:15.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2475000000000003e-05 2024-09-13T10:11:15.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:15.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.758582+0000", 2024-09-13T10:11:15.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6033999999999999e-05 2024-09-13T10:11:15.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:15.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.758717+0000", 2024-09-13T10:11:15.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013453299999999999 2024-09-13T10:11:15.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:15.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.758944+0000", 2024-09-13T10:11:15.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000227252 2024-09-13T10:11:15.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:15.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.763328+0000", 2024-09-13T10:11:15.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043840880000000004 2024-09-13T10:11:15.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.765136+0000", 2024-09-13T10:11:15.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018076850000000001 2024-09-13T10:11:15.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:15.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.765180+0000", 2024-09-13T10:11:15.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4119000000000002e-05 2024-09-13T10:11:15.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:15.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.766547+0000", 2024-09-13T10:11:15.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001367006 2024-09-13T10:11:15.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:15.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.766637+0000", 2024-09-13T10:11:15.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0241999999999994e-05 2024-09-13T10:11:15.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:15.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:15.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:15.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.766745+0000", 2024-09-13T10:11:15.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108064 2024-09-13T10:11:15.841 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.841 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.841 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.841 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:15.841 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:15.841 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:15.842 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:15.843 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:15.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:15.717+0000 7fc1c94a3640 0 -- 172.21.15.73:0/905811448 >> v2:172.21.15.181:6804/1173379175 conn(0x7fc1c4221aa0 msgr2=0x7fc1c4223ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:11:15.844 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-13T10:11:15.844 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:15.845 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:15.845 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-13T10:11:15.847 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:15.850 INFO:tasks.rados.rados.0.smithi195.stdout:1053: writing smithi19530714-4 from 2728968 to 3210137 tid 3 2024-09-13T10:11:15.850 INFO:tasks.rados.rados.0.smithi195.stdout:1054: read oid 50 snap -1 2024-09-13T10:11:15.850 INFO:tasks.rados.rados.0.smithi195.stdout:1054: expect (ObjNum 156 snap 0 seq_num 156) 2024-09-13T10:11:15.850 INFO:tasks.rados.rados.0.smithi195.stdout:1055: read oid 48 snap -1 2024-09-13T10:11:15.850 INFO:tasks.rados.rados.0.smithi195.stdout:1055: expect (ObjNum 185 snap 0 seq_num 185) 2024-09-13T10:11:15.850 INFO:tasks.rados.rados.0.smithi195.stdout:1056: set_chunk oid 46 target oid 2024-09-13T10:11:15.850 INFO:tasks.rados.rados.0.smithi195.stdout:1056: set_chunk oid 46 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:15.850 INFO:tasks.rados.rados.0.smithi195.stdout:1057: set_chunk oid 10 target oid 2024-09-13T10:11:15.850 INFO:tasks.rados.rados.0.smithi195.stdout:1057: set_chunk oid 10 offset: 785920 length: 921600 target oid 7c7772d7eaaf44938e11bd78f2f73c9488a2106aefd56816382a81c5ab60323c offset: 0 snap -1 2024-09-13T10:11:15.850 INFO:tasks.rados.rados.0.smithi195.stdout:1058: read oid 38 snap -1 2024-09-13T10:11:15.850 INFO:tasks.rados.rados.0.smithi195.stdout:1058: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:11:15.850 INFO:tasks.rados.rados.0.smithi195.stdout:1059: read oid 6 snap -1 2024-09-13T10:11:15.851 INFO:tasks.rados.rados.0.smithi195.stdout:1059: expect (ObjNum 168 snap 0 seq_num 168) 2024-09-13T10:11:15.851 INFO:tasks.rados.rados.0.smithi195.stdout:1060: tier_evict oid 2 2024-09-13T10:11:15.851 INFO:tasks.rados.rados.0.smithi195.stdout:1060: tier_evict oid 2 snap -1 2024-09-13T10:11:15.851 INFO:tasks.rados.rados.0.smithi195.stdout:1051: got (22) Invalid argument 2024-09-13T10:11:15.851 INFO:tasks.rados.rados.0.smithi195.stdout:1051: done (12 left) 2024-09-13T10:11:15.851 INFO:tasks.rados.rados.0.smithi195.stdout:1061: read oid 26 snap -1 2024-09-13T10:11:15.851 INFO:tasks.rados.rados.0.smithi195.stdout:1061: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:15.851 INFO:tasks.rados.rados.0.smithi195.stdout:1062: read oid 19 snap -1 2024-09-13T10:11:15.851 INFO:tasks.rados.rados.0.smithi195.stdout:1062: expect (ObjNum 160 snap 0 seq_num 160) 2024-09-13T10:11:15.851 INFO:tasks.rados.rados.0.smithi195.stdout:1063: tier_flush oid 34 2024-09-13T10:11:15.851 INFO:tasks.rados.rados.0.smithi195.stdout:1063: tier_flush oid 34 snap -1 2024-09-13T10:11:15.851 INFO:tasks.rados.rados.0.smithi195.stdout:1049: finishing write tid 2 to smithi19530714-18 2024-09-13T10:11:15.852 INFO:tasks.rados.rados.0.smithi195.stdout:1049: finishing write tid 3 to smithi19530714-18 2024-09-13T10:11:15.852 INFO:tasks.rados.rados.0.smithi195.stdout:1064: set_chunk oid 5 target oid 2024-09-13T10:11:15.852 INFO:tasks.rados.rados.0.smithi195.stdout:1064: set_chunk oid 5 offset: 273408 length: 270848 target oid 559ecb83ad603d96f038ee44085d828ac522158d87108a1b346bcacbe90a65e1 offset: 0 snap -1 2024-09-13T10:11:15.852 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:15.852 INFO:tasks.rados.rados.0.smithi195.stdout:1049: finishing write tid 4 to smithi19530714-18 2024-09-13T10:11:15.852 INFO:tasks.rados.rados.0.smithi195.stdout:1049: oid 18 updating version 0 to 68 2024-09-13T10:11:15.852 INFO:tasks.rados.rados.0.smithi195.stdout:1049: oid 18 version 68 is already newer than 67 2024-09-13T10:11:15.852 INFO:tasks.rados.rados.0.smithi195.stdout:1049: oid 18 updating version 68 to 69 2024-09-13T10:11:15.852 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 69 (ObjNum 189 snap 0 seq_num 189) dirty exists 2024-09-13T10:11:15.852 INFO:tasks.rados.rados.0.smithi195.stdout:1049: left oid 18 (ObjNum 189 snap 0 seq_num 189) 2024-09-13T10:11:15.852 INFO:tasks.rados.rados.0.smithi195.stdout:1050: finishing set_chunk to oid 25 2024-09-13T10:11:15.852 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 82 (ObjNum 155 snap 0 seq_num 155) dirty exists 2024-09-13T10:11:15.852 INFO:tasks.rados.rados.0.smithi195.stdout:1052: finishing write tid 1 to smithi19530714-36 2024-09-13T10:11:15.853 INFO:tasks.rados.rados.0.smithi195.stdout:1052: finishing write tid 2 to smithi19530714-36 2024-09-13T10:11:15.853 INFO:tasks.rados.rados.0.smithi195.stdout:1052: finishing write tid 3 to smithi19530714-36 2024-09-13T10:11:15.853 INFO:tasks.rados.rados.0.smithi195.stdout:1049: done (15 left) 2024-09-13T10:11:15.853 INFO:tasks.rados.rados.0.smithi195.stdout:1050: done (14 left) 2024-09-13T10:11:15.853 INFO:tasks.rados.rados.0.smithi195.stdout:1065: read oid 32 snap -1 2024-09-13T10:11:15.853 INFO:tasks.rados.rados.0.smithi195.stdout:1065: expect deleted 2024-09-13T10:11:15.853 INFO:tasks.rados.rados.0.smithi195.stdout:1052: finishing write tid 4 to smithi19530714-36 2024-09-13T10:11:15.853 INFO:tasks.rados.rados.0.smithi195.stdout:1052: finishing write tid 5 to smithi19530714-36 2024-09-13T10:11:15.854 INFO:tasks.rados.rados.0.smithi195.stdout:1052: oid 36 updating version 0 to 26 2024-09-13T10:11:15.854 INFO:tasks.rados.rados.0.smithi195.stdout:1052: oid 36 version 26 is already newer than 23 2024-09-13T10:11:15.854 INFO:tasks.rados.rados.0.smithi195.stdout:1052: oid 36 version 26 is already newer than 24 2024-09-13T10:11:15.854 INFO:tasks.rados.rados.0.smithi195.stdout:1052: oid 36 version 26 is already newer than 25 2024-09-13T10:11:15.854 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 26 (ObjNum 190 snap 0 seq_num 190) dirty exists 2024-09-13T10:11:15.854 INFO:tasks.rados.rados.0.smithi195.stdout:1052: left oid 36 (ObjNum 190 snap 0 seq_num 190) 2024-09-13T10:11:15.854 INFO:tasks.rados.rados.0.smithi195.stdout:1053: finishing write tid 1 to smithi19530714-4 2024-09-13T10:11:15.854 INFO:tasks.rados.rados.0.smithi195.stdout:1056: finishing set_chunk to oid 46 2024-09-13T10:11:15.854 INFO:tasks.rados.rados.0.smithi195.stdout:1056: got expected ENOENT (src dne) 2024-09-13T10:11:15.854 INFO:tasks.rados.rados.0.smithi195.stdout:1053: finishing write tid 2 to smithi19530714-4 2024-09-13T10:11:15.854 INFO:tasks.rados.rados.0.smithi195.stdout:1053: finishing write tid 3 to smithi19530714-4 2024-09-13T10:11:15.854 INFO:tasks.rados.rados.0.smithi195.stdout:1053: finishing write tid 4 to smithi19530714-4 2024-09-13T10:11:15.854 INFO:tasks.rados.rados.0.smithi195.stdout:1053: finishing write tid 5 to smithi19530714-4 2024-09-13T10:11:15.855 INFO:tasks.rados.rados.0.smithi195.stdout:1053: oid 4 updating version 0 to 73 2024-09-13T10:11:15.855 INFO:tasks.rados.rados.0.smithi195.stdout:1053: oid 4 version 73 is already newer than 71 2024-09-13T10:11:15.855 INFO:tasks.rados.rados.0.smithi195.stdout:1053: oid 4 version 73 is already newer than 70 2024-09-13T10:11:15.855 INFO:tasks.rados.rados.0.smithi195.stdout:1053: oid 4 version 73 is already newer than 72 2024-09-13T10:11:15.855 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 73 (ObjNum 191 snap 0 seq_num 191) dirty exists 2024-09-13T10:11:15.855 INFO:tasks.rados.rados.0.smithi195.stdout:1053: left oid 4 (ObjNum 191 snap 0 seq_num 191) 2024-09-13T10:11:15.855 INFO:tasks.rados.rados.0.smithi195.stdout:1054: expect (ObjNum 156 snap 0 seq_num 156) 2024-09-13T10:11:15.855 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:15.839+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:15.855 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1736629 2024-09-13T10:11:15.938 INFO:tasks.rados.rados.0.smithi195.stdout:1066: read oid 44 snap -1 2024-09-13T10:11:15.939 INFO:tasks.rados.rados.0.smithi195.stdout:1066: expect (ObjNum 165 snap 0 seq_num 165) 2024-09-13T10:11:15.939 INFO:tasks.rados.rados.0.smithi195.stdout:1052: done (15 left) 2024-09-13T10:11:15.939 INFO:tasks.rados.rados.0.smithi195.stdout:1053: done (14 left) 2024-09-13T10:11:15.939 INFO:tasks.rados.rados.0.smithi195.stdout:1054: done (13 left) 2024-09-13T10:11:15.939 INFO:tasks.rados.rados.0.smithi195.stdout:1056: done (12 left) 2024-09-13T10:11:15.939 INFO:tasks.rados.rados.0.smithi195.stdout:1067: read oid 43 snap -1 2024-09-13T10:11:15.939 INFO:tasks.rados.rados.0.smithi195.stdout:1067: expect (ObjNum 149 snap 0 seq_num 149) 2024-09-13T10:11:15.939 INFO:tasks.rados.rados.0.smithi195.stdout:1068: read oid 45 snap -1 2024-09-13T10:11:15.939 INFO:tasks.rados.rados.0.smithi195.stdout:1068: expect (ObjNum 158 snap 0 seq_num 158) 2024-09-13T10:11:15.939 INFO:tasks.rados.rados.0.smithi195.stdout:1069: set_chunk oid 31 target oid 2024-09-13T10:11:15.939 INFO:tasks.rados.rados.0.smithi195.stdout:1069: set_chunk oid 31 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:15.939 INFO:tasks.rados.rados.0.smithi195.stdout:1070: read oid 13 snap -1 2024-09-13T10:11:15.940 INFO:tasks.rados.rados.0.smithi195.stdout:1070: expect (ObjNum 187 snap 0 seq_num 187) 2024-09-13T10:11:15.940 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:15.940 INFO:tasks.rados.rados.0.smithi195.stdout:1057: finishing set_chunk to oid 10 2024-09-13T10:11:15.940 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 10 set_chunk returned error code -95 offset: 785920 length: 921600 2024-09-13T10:11:15.940 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 0 (ObjNum 84 snap 0 seq_num 84) dirty exists 2024-09-13T10:11:15.940 INFO:tasks.rados.rados.0.smithi195.stdout:1055: expect (ObjNum 185 snap 0 seq_num 185) 2024-09-13T10:11:15.940 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:15.943 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:15.941+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:15.979 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:15.979 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:15.979 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:11:15.979 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:15.985 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:15.993 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-13T10:11:16.045 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:16.041+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:16.077 INFO:tasks.rados.rados.0.smithi195.stdout:1060: got (22) Invalid argument 2024-09-13T10:11:16.077 INFO:tasks.rados.rados.0.smithi195.stdout:1059: expect (ObjNum 168 snap 0 seq_num 168) 2024-09-13T10:11:16.078 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 413567 2024-09-13T10:11:16.115 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:16.115 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:16.115 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:11:16.115 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:11:16.115 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:16.131 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-13T10:11:16.146 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:16.145+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:16.157 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1374224 2024-09-13T10:11:16.246 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:16.243+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:16.276 INFO:tasks.rados.rados.0.smithi195.stdout:1062: expect (ObjNum 160 snap 0 seq_num 160) 2024-09-13T10:11:16.276 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:16.299 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:16.299 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:11:16.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:11:16.299 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:11:16.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.299 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4365.0:5 1.2 e26/24 1:5cea7035:::rbd_info:head v 26'1, mlcod=0'0)", 2024-09-13T10:11:16.299 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:31.142893+0000", 2024-09-13T10:11:16.300 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.154039820999998, 2024-09-13T10:11:16.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024212330000000001, 2024-09-13T10:11:16.300 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.300 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.300 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.300 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142893+0000", 2024-09-13T10:11:16.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142893+0000", 2024-09-13T10:11:16.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142893+0000", 2024-09-13T10:11:16.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1699999999999998e-07 2024-09-13T10:11:16.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142914+0000", 2024-09-13T10:11:16.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0378999999999999e-05 2024-09-13T10:11:16.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142951+0000", 2024-09-13T10:11:16.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7471000000000001e-05 2024-09-13T10:11:16.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142983+0000", 2024-09-13T10:11:16.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2422000000000002e-05 2024-09-13T10:11:16.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.143202+0000", 2024-09-13T10:11:16.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021875799999999999 2024-09-13T10:11:16.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.143308+0000", 2024-09-13T10:11:16.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105487 2024-09-13T10:11:16.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.145207+0000", 2024-09-13T10:11:16.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001899848 2024-09-13T10:11:16.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.145314+0000", 2024-09-13T10:11:16.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106351 2024-09-13T10:11:16.304 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.304 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.304 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.304 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4365.0:7 1.2 e27/24 1:5cea7035:::rbd_info:head v 27'2, mlcod=0'0)", 2024-09-13T10:11:16.304 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:32.145197+0000", 2024-09-13T10:11:16.304 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.151735832999996, 2024-09-13T10:11:16.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020358360000000001, 2024-09-13T10:11:16.304 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.305 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.305 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.305 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145197+0000", 2024-09-13T10:11:16.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145197+0000", 2024-09-13T10:11:16.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145197+0000", 2024-09-13T10:11:16.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5000000000000002e-07 2024-09-13T10:11:16.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145212+0000", 2024-09-13T10:11:16.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4619e-05 2024-09-13T10:11:16.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145241+0000", 2024-09-13T10:11:16.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8676000000000001e-05 2024-09-13T10:11:16.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145257+0000", 2024-09-13T10:11:16.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6155e-05 2024-09-13T10:11:16.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145397+0000", 2024-09-13T10:11:16.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014000199999999999 2024-09-13T10:11:16.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.145437+0000", 2024-09-13T10:11:16.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0615000000000003e-05 2024-09-13T10:11:16.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.147104+0000", 2024-09-13T10:11:16.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016668259999999999 2024-09-13T10:11:16.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:32.147232+0000", 2024-09-13T10:11:16.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000128293 2024-09-13T10:11:16.309 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.309 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.309 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.309 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:89 4.9 e38/31 4:96fcf081:::smithi19530714-23:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:16.309 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.811705+0000", 2024-09-13T10:11:16.310 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.485227576, 2024-09-13T10:11:16.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053533510000000001, 2024-09-13T10:11:16.310 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.310 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.310 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.310 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.811705+0000", 2024-09-13T10:11:16.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.811705+0000", 2024-09-13T10:11:16.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.811706+0000", 2024-09-13T10:11:16.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7700000000000003e-07 2024-09-13T10:11:16.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813717+0000", 2024-09-13T10:11:16.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002011494 2024-09-13T10:11:16.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813743+0000", 2024-09-13T10:11:16.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5528999999999999e-05 2024-09-13T10:11:16.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813759+0000", 2024-09-13T10:11:16.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6178999999999999e-05 2024-09-13T10:11:16.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813873+0000", 2024-09-13T10:11:16.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113863 2024-09-13T10:11:16.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813922+0000", 2024-09-13T10:11:16.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9558999999999998e-05 2024-09-13T10:11:16.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816994+0000", 2024-09-13T10:11:16.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003071299 2024-09-13T10:11:16.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817058+0000", 2024-09-13T10:11:16.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4550999999999999e-05 2024-09-13T10:11:16.314 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.314 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.314 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.314 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:90 4.9 e38/31 4:96fcf081:::smithi19530714-23:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:16.315 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.815985+0000", 2024-09-13T10:11:16.315 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.480947240999999, 2024-09-13T10:11:16.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00489822, 2024-09-13T10:11:16.315 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.315 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.315 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.315 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815985+0000", 2024-09-13T10:11:16.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815985+0000", 2024-09-13T10:11:16.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815987+0000", 2024-09-13T10:11:16.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.277e-06 2024-09-13T10:11:16.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817028+0000", 2024-09-13T10:11:16.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001040664 2024-09-13T10:11:16.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817055+0000", 2024-09-13T10:11:16.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6835999999999999e-05 2024-09-13T10:11:16.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817074+0000", 2024-09-13T10:11:16.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8629000000000001e-05 2024-09-13T10:11:16.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817153+0000", 2024-09-13T10:11:16.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9794000000000002e-05 2024-09-13T10:11:16.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817183+0000", 2024-09-13T10:11:16.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9479e-05 2024-09-13T10:11:16.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820783+0000", 2024-09-13T10:11:16.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035997720000000002 2024-09-13T10:11:16.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820883+0000", 2024-09-13T10:11:16.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100769 2024-09-13T10:11:16.319 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.319 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.319 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.319 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:91 4.9 e38/31 4:96fcf081:::smithi19530714-23:head v 38'3, mlcod=0'0)", 2024-09-13T10:11:16.319 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.817117+0000", 2024-09-13T10:11:16.320 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.479815036000002, 2024-09-13T10:11:16.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038490590000000002, 2024-09-13T10:11:16.320 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.320 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.320 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.320 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817117+0000", 2024-09-13T10:11:16.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817117+0000", 2024-09-13T10:11:16.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817119+0000", 2024-09-13T10:11:16.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1999999999999999e-06 2024-09-13T10:11:16.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817130+0000", 2024-09-13T10:11:16.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1313e-05 2024-09-13T10:11:16.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817145+0000", 2024-09-13T10:11:16.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4786e-05 2024-09-13T10:11:16.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817157+0000", 2024-09-13T10:11:16.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2775999999999999e-05 2024-09-13T10:11:16.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818394+0000", 2024-09-13T10:11:16.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012367999999999999 2024-09-13T10:11:16.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818496+0000", 2024-09-13T10:11:16.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102098 2024-09-13T10:11:16.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820895+0000", 2024-09-13T10:11:16.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023989189999999998 2024-09-13T10:11:16.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820966+0000", 2024-09-13T10:11:16.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1167000000000005e-05 2024-09-13T10:11:16.324 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.324 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.324 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.324 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:129 4.9 e38/31 4:998d9bf1:::smithi19530714-33:head v 38'4, mlcod=0'0)", 2024-09-13T10:11:16.324 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.817768+0000", 2024-09-13T10:11:16.324 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.479164231999999, 2024-09-13T10:11:16.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047213589999999996, 2024-09-13T10:11:16.324 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.324 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.325 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.325 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817768+0000", 2024-09-13T10:11:16.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817768+0000", 2024-09-13T10:11:16.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817769+0000", 2024-09-13T10:11:16.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.184e-06 2024-09-13T10:11:16.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818638+0000", 2024-09-13T10:11:16.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086842199999999999 2024-09-13T10:11:16.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818665+0000", 2024-09-13T10:11:16.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6979e-05 2024-09-13T10:11:16.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818685+0000", 2024-09-13T10:11:16.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0063000000000001e-05 2024-09-13T10:11:16.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818994+0000", 2024-09-13T10:11:16.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030959500000000002 2024-09-13T10:11:16.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819019+0000", 2024-09-13T10:11:16.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4508999999999999e-05 2024-09-13T10:11:16.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822389+0000", 2024-09-13T10:11:16.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033701970000000001 2024-09-13T10:11:16.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822490+0000", 2024-09-13T10:11:16.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010041 2024-09-13T10:11:16.328 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.328 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.329 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.329 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:130 4.9 e38/31 4:998d9bf1:::smithi19530714-33:head v 38'5, mlcod=0'0)", 2024-09-13T10:11:16.329 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.819657+0000", 2024-09-13T10:11:16.329 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.477275268, 2024-09-13T10:11:16.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032383609999999999, 2024-09-13T10:11:16.329 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.329 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.329 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.329 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819657+0000", 2024-09-13T10:11:16.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819657+0000", 2024-09-13T10:11:16.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819659+0000", 2024-09-13T10:11:16.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.624e-06 2024-09-13T10:11:16.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820879+0000", 2024-09-13T10:11:16.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001220184 2024-09-13T10:11:16.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820906+0000", 2024-09-13T10:11:16.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6536000000000001e-05 2024-09-13T10:11:16.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820930+0000", 2024-09-13T10:11:16.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4552000000000001e-05 2024-09-13T10:11:16.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821017+0000", 2024-09-13T10:11:16.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6539999999999995e-05 2024-09-13T10:11:16.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821043+0000", 2024-09-13T10:11:16.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6018999999999999e-05 2024-09-13T10:11:16.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822848+0000", 2024-09-13T10:11:16.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001805673 2024-09-13T10:11:16.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822896+0000", 2024-09-13T10:11:16.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7233e-05 2024-09-13T10:11:16.333 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.333 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.333 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.334 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:185 4.9 e38/31 4:9bec18ef:::smithi19530714-47:head v 38'7, mlcod=38'5)", 2024-09-13T10:11:16.334 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.941317+0000", 2024-09-13T10:11:16.334 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.355615638, 2024-09-13T10:11:16.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002782858, 2024-09-13T10:11:16.334 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.334 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.334 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.334 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.941317+0000", 2024-09-13T10:11:16.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.941317+0000", 2024-09-13T10:11:16.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.941319+0000", 2024-09-13T10:11:16.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9860000000000001e-06 2024-09-13T10:11:16.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942196+0000", 2024-09-13T10:11:16.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087739999999999997 2024-09-13T10:11:16.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942230+0000", 2024-09-13T10:11:16.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4118000000000001e-05 2024-09-13T10:11:16.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942252+0000", 2024-09-13T10:11:16.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2030000000000001e-05 2024-09-13T10:11:16.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942365+0000", 2024-09-13T10:11:16.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112689 2024-09-13T10:11:16.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942398+0000", 2024-09-13T10:11:16.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2897e-05 2024-09-13T10:11:16.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.944044+0000", 2024-09-13T10:11:16.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016460800000000001 2024-09-13T10:11:16.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.944100+0000", 2024-09-13T10:11:16.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5658000000000001e-05 2024-09-13T10:11:16.338 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.338 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.338 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.338 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:186 4.9 e38/31 4:9bec18ef:::smithi19530714-47:head v 38'8, mlcod=38'6)", 2024-09-13T10:11:16.338 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.948439+0000", 2024-09-13T10:11:16.338 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.348493564999998, 2024-09-13T10:11:16.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002770018, 2024-09-13T10:11:16.339 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.339 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.339 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.339 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.948439+0000", 2024-09-13T10:11:16.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.948439+0000", 2024-09-13T10:11:16.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.948440+0000", 2024-09-13T10:11:16.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4440000000000001e-06 2024-09-13T10:11:16.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949519+0000", 2024-09-13T10:11:16.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010790039999999999 2024-09-13T10:11:16.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949537+0000", 2024-09-13T10:11:16.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7312e-05 2024-09-13T10:11:16.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949545+0000", 2024-09-13T10:11:16.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2570000000000007e-06 2024-09-13T10:11:16.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949615+0000", 2024-09-13T10:11:16.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9614000000000001e-05 2024-09-13T10:11:16.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949633+0000", 2024-09-13T10:11:16.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8640000000000001e-05 2024-09-13T10:11:16.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.951146+0000", 2024-09-13T10:11:16.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015129570000000001 2024-09-13T10:11:16.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.951209+0000", 2024-09-13T10:11:16.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.279e-05 2024-09-13T10:11:16.343 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.343 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.343 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.343 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.11.0:22 4.19 e42/41 4:998d9bf1:::smithi19530714-33:head v 42'23, mlcod=0'0)", 2024-09-13T10:11:16.343 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.624507+0000", 2024-09-13T10:11:16.343 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.672425034, 2024-09-13T10:11:16.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039005810000000002, 2024-09-13T10:11:16.343 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.344 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.344 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.344 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.624507+0000", 2024-09-13T10:11:16.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.624507+0000", 2024-09-13T10:11:16.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.624510+0000", 2024-09-13T10:11:16.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4770000000000002e-06 2024-09-13T10:11:16.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.625777+0000", 2024-09-13T10:11:16.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001267589 2024-09-13T10:11:16.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.625809+0000", 2024-09-13T10:11:16.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1288999999999998e-05 2024-09-13T10:11:16.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.625827+0000", 2024-09-13T10:11:16.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8312999999999999e-05 2024-09-13T10:11:16.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626008+0000", 2024-09-13T10:11:16.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000181421 2024-09-13T10:11:16.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626077+0000", 2024-09-13T10:11:16.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8471000000000002e-05 2024-09-13T10:11:16.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.628292+0000", 2024-09-13T10:11:16.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002214728 2024-09-13T10:11:16.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.628408+0000", 2024-09-13T10:11:16.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011629300000000001 2024-09-13T10:11:16.348 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.348 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.348 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.348 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "pg_update_log_missing(4.19 epoch 42/41 rep_tid 23 entries 42'24 (0'0) error 4:998d9bf1:::smithi19530714-33:head by client.4607.0:1065 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 40'21 roll_forward_to 0'0)", 2024-09-13T10:11:16.348 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.626821+0000", 2024-09-13T10:11:16.348 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.670111156000001, 2024-09-13T10:11:16.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022279040000000002, 2024-09-13T10:11:16.348 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.348 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.348 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:11:16.348 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626821+0000", 2024-09-13T10:11:16.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626821+0000", 2024-09-13T10:11:16.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626823+0000", 2024-09-13T10:11:16.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.77e-06 2024-09-13T10:11:16.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626840+0000", 2024-09-13T10:11:16.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7272999999999999e-05 2024-09-13T10:11:16.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626882+0000", 2024-09-13T10:11:16.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1936e-05 2024-09-13T10:11:16.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626930+0000", 2024-09-13T10:11:16.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7284000000000003e-05 2024-09-13T10:11:16.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.627490+0000", 2024-09-13T10:11:16.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056048399999999998 2024-09-13T10:11:16.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.629049+0000", 2024-09-13T10:11:16.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001559157 2024-09-13T10:11:16.352 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.352 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.352 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.352 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:1276 4.19 e45/41 4:998d9bf1:::smithi19530714-33:head v 45'25, mlcod=42'23)", 2024-09-13T10:11:16.352 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:59.884039+0000", 2024-09-13T10:11:16.352 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.412893446999998, 2024-09-13T10:11:16.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043044900000000002, 2024-09-13T10:11:16.352 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.352 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.352 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.352 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.884039+0000", 2024-09-13T10:11:16.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.884039+0000", 2024-09-13T10:11:16.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.884039+0000", 2024-09-13T10:11:16.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6199999999999999e-07 2024-09-13T10:11:16.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.885292+0000", 2024-09-13T10:11:16.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012528999999999999 2024-09-13T10:11:16.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.885310+0000", 2024-09-13T10:11:16.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8121000000000001e-05 2024-09-13T10:11:16.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.885323+0000", 2024-09-13T10:11:16.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2672e-05 2024-09-13T10:11:16.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.885441+0000", 2024-09-13T10:11:16.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011766599999999999 2024-09-13T10:11:16.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.885479+0000", 2024-09-13T10:11:16.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8124999999999998e-05 2024-09-13T10:11:16.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.888240+0000", 2024-09-13T10:11:16.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027611440000000001 2024-09-13T10:11:16.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.888343+0000", 2024-09-13T10:11:16.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010349999999999999 2024-09-13T10:11:16.356 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.357 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.357 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.357 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:1277 4.19 e45/41 4:998d9bf1:::smithi19530714-33:head v 45'26, mlcod=42'24)", 2024-09-13T10:11:16.357 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:59.891710+0000", 2024-09-13T10:11:16.357 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.405222593000001, 2024-09-13T10:11:16.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003460627, 2024-09-13T10:11:16.357 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.357 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.357 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.357 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.891710+0000", 2024-09-13T10:11:16.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.891710+0000", 2024-09-13T10:11:16.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.891711+0000", 2024-09-13T10:11:16.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3820000000000001e-06 2024-09-13T10:11:16.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.892660+0000", 2024-09-13T10:11:16.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00094837899999999995 2024-09-13T10:11:16.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.892683+0000", 2024-09-13T10:11:16.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3767000000000001e-05 2024-09-13T10:11:16.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.892697+0000", 2024-09-13T10:11:16.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3619e-05 2024-09-13T10:11:16.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.892795+0000", 2024-09-13T10:11:16.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7553000000000006e-05 2024-09-13T10:11:16.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.892831+0000", 2024-09-13T10:11:16.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5976000000000002e-05 2024-09-13T10:11:16.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.895127+0000", 2024-09-13T10:11:16.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022967830000000002 2024-09-13T10:11:16.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.895170+0000", 2024-09-13T10:11:16.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3167999999999997e-05 2024-09-13T10:11:16.361 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.361 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.361 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.362 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1600 4.10 4.31420bd0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:16.362 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:05.990716+0000", 2024-09-13T10:11:16.362 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.30621648, 2024-09-13T10:11:16.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.412517299, 2024-09-13T10:11:16.362 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.362 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.362 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:11:16.362 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.362 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.362 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.362 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1600 2024-09-13T10:11:16.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.362 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.990716+0000", 2024-09-13T10:11:16.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.990716+0000", 2024-09-13T10:11:16.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.990720+0000", 2024-09-13T10:11:16.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9689999999999996e-06 2024-09-13T10:11:16.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991825+0000", 2024-09-13T10:11:16.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001104764 2024-09-13T10:11:16.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991856+0000", 2024-09-13T10:11:16.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1473000000000002e-05 2024-09-13T10:11:16.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991872+0000", 2024-09-13T10:11:16.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6212999999999999e-05 2024-09-13T10:11:16.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991974+0000", 2024-09-13T10:11:16.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010160799999999999 2024-09-13T10:11:16.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:16.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991988+0000", 2024-09-13T10:11:16.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4106e-05 2024-09-13T10:11:16.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403179+0000", 2024-09-13T10:11:16.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41119127100000002 2024-09-13T10:11:16.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403233+0000", 2024-09-13T10:11:16.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3894999999999998e-05 2024-09-13T10:11:16.367 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.367 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.367 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.367 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1601 4.10 4.31420bd0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:16.367 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.008020+0000", 2024-09-13T10:11:16.367 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.288912304, 2024-09-13T10:11:16.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39533983, 2024-09-13T10:11:16.367 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.368 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.368 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:11:16.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.368 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.368 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1601 2024-09-13T10:11:16.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.368 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.008020+0000", 2024-09-13T10:11:16.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.008020+0000", 2024-09-13T10:11:16.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.008023+0000", 2024-09-13T10:11:16.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.807e-06 2024-09-13T10:11:16.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009268+0000", 2024-09-13T10:11:16.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001245257 2024-09-13T10:11:16.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009298+0000", 2024-09-13T10:11:16.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0171999999999998e-05 2024-09-13T10:11:16.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009313+0000", 2024-09-13T10:11:16.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.471e-05 2024-09-13T10:11:16.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009469+0000", 2024-09-13T10:11:16.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015595600000000001 2024-09-13T10:11:16.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:16.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009481+0000", 2024-09-13T10:11:16.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2454e-05 2024-09-13T10:11:16.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403334+0000", 2024-09-13T10:11:16.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39385227499999997 2024-09-13T10:11:16.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403360+0000", 2024-09-13T10:11:16.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6199000000000001e-05 2024-09-13T10:11:16.372 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.372 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.372 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.372 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1602 4.10 4.31420bd0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:16.372 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.009344+0000", 2024-09-13T10:11:16.372 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.287588471999999, 2024-09-13T10:11:16.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39420750999999998, 2024-09-13T10:11:16.373 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.373 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.373 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:11:16.373 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.373 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.373 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.373 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1602 2024-09-13T10:11:16.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.373 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009344+0000", 2024-09-13T10:11:16.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009344+0000", 2024-09-13T10:11:16.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009345+0000", 2024-09-13T10:11:16.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1930000000000001e-06 2024-09-13T10:11:16.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009351+0000", 2024-09-13T10:11:16.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1399999999999997e-06 2024-09-13T10:11:16.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009361+0000", 2024-09-13T10:11:16.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9569999999999997e-06 2024-09-13T10:11:16.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009370+0000", 2024-09-13T10:11:16.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.619e-06 2024-09-13T10:11:16.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009570+0000", 2024-09-13T10:11:16.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019990599999999999 2024-09-13T10:11:16.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:16.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009583+0000", 2024-09-13T10:11:16.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2886e-05 2024-09-13T10:11:16.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403522+0000", 2024-09-13T10:11:16.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39393976000000003 2024-09-13T10:11:16.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403551+0000", 2024-09-13T10:11:16.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9048999999999999e-05 2024-09-13T10:11:16.377 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.377 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.377 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.377 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1603 4.10 4.31420bd0 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:16.377 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.009383+0000", 2024-09-13T10:11:16.378 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.28754958, 2024-09-13T10:11:16.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39425412900000001, 2024-09-13T10:11:16.378 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.378 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.378 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:11:16.378 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.378 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.378 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.378 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1603 2024-09-13T10:11:16.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.378 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009383+0000", 2024-09-13T10:11:16.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009383+0000", 2024-09-13T10:11:16.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009384+0000", 2024-09-13T10:11:16.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8700000000000005e-07 2024-09-13T10:11:16.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009389+0000", 2024-09-13T10:11:16.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1669999999999998e-06 2024-09-13T10:11:16.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009398+0000", 2024-09-13T10:11:16.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6260000000000007e-06 2024-09-13T10:11:16.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009407+0000", 2024-09-13T10:11:16.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1960000000000008e-06 2024-09-13T10:11:16.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009623+0000", 2024-09-13T10:11:16.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000216842 2024-09-13T10:11:16.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:16.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009632+0000", 2024-09-13T10:11:16.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3529999999999995e-06 2024-09-13T10:11:16.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403609+0000", 2024-09-13T10:11:16.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39397721600000002 2024-09-13T10:11:16.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403637+0000", 2024-09-13T10:11:16.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7942000000000001e-05 2024-09-13T10:11:16.382 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.382 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.382 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.383 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:1618 4.19 e48/41 4:998d9bf1:::smithi19530714-33:head v 48'30, mlcod=45'28)", 2024-09-13T10:11:16.383 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.067938+0000", 2024-09-13T10:11:16.383 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.228994184999999, 2024-09-13T10:11:16.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040350539999999997, 2024-09-13T10:11:16.383 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.383 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.383 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.383 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.067938+0000", 2024-09-13T10:11:16.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.067938+0000", 2024-09-13T10:11:16.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.067939+0000", 2024-09-13T10:11:16.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.33e-07 2024-09-13T10:11:16.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.069175+0000", 2024-09-13T10:11:16.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001235667 2024-09-13T10:11:16.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.069203+0000", 2024-09-13T10:11:16.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7742999999999998e-05 2024-09-13T10:11:16.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.069219+0000", 2024-09-13T10:11:16.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6195999999999999e-05 2024-09-13T10:11:16.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.069355+0000", 2024-09-13T10:11:16.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013634300000000001 2024-09-13T10:11:16.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.069396+0000", 2024-09-13T10:11:16.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1099000000000001e-05 2024-09-13T10:11:16.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.071866+0000", 2024-09-13T10:11:16.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024698720000000001 2024-09-13T10:11:16.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.071973+0000", 2024-09-13T10:11:16.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010720100000000001 2024-09-13T10:11:16.387 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.387 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.387 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.387 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:1619 4.19 e48/41 4:998d9bf1:::smithi19530714-33:head v 48'31, mlcod=47'29)", 2024-09-13T10:11:16.387 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.076123+0000", 2024-09-13T10:11:16.388 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.220809795999999, 2024-09-13T10:11:16.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057688829999999998, 2024-09-13T10:11:16.388 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.388 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.388 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.388 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.076123+0000", 2024-09-13T10:11:16.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.076123+0000", 2024-09-13T10:11:16.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.076125+0000", 2024-09-13T10:11:16.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.633e-06 2024-09-13T10:11:16.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077122+0000", 2024-09-13T10:11:16.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00099714000000000009 2024-09-13T10:11:16.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077157+0000", 2024-09-13T10:11:16.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4869000000000001e-05 2024-09-13T10:11:16.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077177+0000", 2024-09-13T10:11:16.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9245999999999998e-05 2024-09-13T10:11:16.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077335+0000", 2024-09-13T10:11:16.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015893999999999999 2024-09-13T10:11:16.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077389+0000", 2024-09-13T10:11:16.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3871e-05 2024-09-13T10:11:16.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.081809+0000", 2024-09-13T10:11:16.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044200189999999999 2024-09-13T10:11:16.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.081892+0000", 2024-09-13T10:11:16.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2164999999999997e-05 2024-09-13T10:11:16.392 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.392 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.392 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.392 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:1620 4.19 e48/41 4:998d9bf1:::smithi19530714-33:head v 48'32, mlcod=47'29)", 2024-09-13T10:11:16.392 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.077802+0000", 2024-09-13T10:11:16.392 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.219130165999999, 2024-09-13T10:11:16.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049265200000000002, 2024-09-13T10:11:16.392 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.393 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.393 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.393 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077802+0000", 2024-09-13T10:11:16.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077802+0000", 2024-09-13T10:11:16.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077804+0000", 2024-09-13T10:11:16.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.933e-06 2024-09-13T10:11:16.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077820+0000", 2024-09-13T10:11:16.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5899000000000001e-05 2024-09-13T10:11:16.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077841+0000", 2024-09-13T10:11:16.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0409999999999999e-05 2024-09-13T10:11:16.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077858+0000", 2024-09-13T10:11:16.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7903000000000001e-05 2024-09-13T10:11:16.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.079658+0000", 2024-09-13T10:11:16.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00179999 2024-09-13T10:11:16.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.079710+0000", 2024-09-13T10:11:16.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1969000000000003e-05 2024-09-13T10:11:16.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.082655+0000", 2024-09-13T10:11:16.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029449340000000002 2024-09-13T10:11:16.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.082729+0000", 2024-09-13T10:11:16.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3481999999999999e-05 2024-09-13T10:11:16.397 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.397 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.397 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.397 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.397 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.397 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:16.398 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:16.399 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-13T10:11:16.399 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:16.397+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:16.472 INFO:tasks.rados.rados.0.smithi195.stdout:1061: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:16.477 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 776829 2024-09-13T10:11:16.499 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:16.497+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:16.514 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 765431 2024-09-13T10:11:16.526 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:16.526 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:16.526 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:11:16.526 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:16.526 INFO:tasks.rados.rados.0.smithi195.stdout:1064: finishing set_chunk to oid 5 2024-09-13T10:11:16.527 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 8 (ObjNum 173 snap 0 seq_num 173) dirty exists 2024-09-13T10:11:16.527 INFO:tasks.rados.rados.0.smithi195.stdout:1058: expect (ObjNum 71 snap 0 seq_num 71) 2024-09-13T10:11:16.540 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-13T10:11:16.600 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:16.599+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:16.611 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1775823 2024-09-13T10:11:16.641 INFO:tasks.rados.rados.0.smithi195.stdout:1048: expect (ObjNum 174 snap 0 seq_num 174) 2024-09-13T10:11:16.641 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:16.663 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:16.664 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:16.664 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:11:16.664 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:11:16.664 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:16.678 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-13T10:11:16.704 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1718522 2024-09-13T10:11:16.722 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:16.721+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:16.724 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:16.771 INFO:tasks.rados.rados.0.smithi195.stdout:1063: got (0) Success 2024-09-13T10:11:16.771 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 34 v 0 (ObjNum 148 snap 0 seq_num 148) dirty exists 2024-09-13T10:11:16.771 INFO:tasks.rados.rados.0.smithi195.stdout:1048: done (15 left) 2024-09-13T10:11:16.771 INFO:tasks.rados.rados.0.smithi195.stdout:1055: done (14 left) 2024-09-13T10:11:16.771 INFO:tasks.rados.rados.0.smithi195.stdout:1057: done (13 left) 2024-09-13T10:11:16.771 INFO:tasks.rados.rados.0.smithi195.stdout:1058: done (12 left) 2024-09-13T10:11:16.771 INFO:tasks.rados.rados.0.smithi195.stdout:1059: done (11 left) 2024-09-13T10:11:16.771 INFO:tasks.rados.rados.0.smithi195.stdout:1060: done (10 left) 2024-09-13T10:11:16.771 INFO:tasks.rados.rados.0.smithi195.stdout:1061: done (9 left) 2024-09-13T10:11:16.772 INFO:tasks.rados.rados.0.smithi195.stdout:1062: done (8 left) 2024-09-13T10:11:16.772 INFO:tasks.rados.rados.0.smithi195.stdout:1063: done (7 left) 2024-09-13T10:11:16.772 INFO:tasks.rados.rados.0.smithi195.stdout:1064: done (6 left) 2024-09-13T10:11:16.772 INFO:tasks.rados.rados.0.smithi195.stdout:1065: done (5 left) 2024-09-13T10:11:16.772 INFO:tasks.rados.rados.0.smithi195.stdout:1071: set_chunk oid 1 target oid 2024-09-13T10:11:16.784 INFO:tasks.rados.rados.0.smithi195.stdout:1071: set_chunk oid 1 offset: 357376 length: 779264 target oid 580090fb1beadc090315c147901b1fc9fdef143b1e46ca097185779f6a171f1e offset: 0 snap -1 2024-09-13T10:11:16.784 INFO:tasks.rados.rados.0.smithi195.stdout:1072: write (excl) oid 20 current snap is 0 2024-09-13T10:11:16.784 INFO:tasks.rados.rados.0.smithi195.stdout:1067: expect (ObjNum 149 snap 0 seq_num 149) 2024-09-13T10:11:16.801 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:16.801 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:11:16.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:11:16.801 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:11:16.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.801 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4365.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:16.801 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:30.119900+0000", 2024-09-13T10:11:16.801 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 46.677680889999998, 2024-09-13T10:11:16.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020673093999999999, 2024-09-13T10:11:16.801 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.802 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.802 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.802 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.802 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4365", 2024-09-13T10:11:16.802 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.73:0/186187721", 2024-09-13T10:11:16.802 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-13T10:11:16.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.802 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119900+0000", 2024-09-13T10:11:16.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119900+0000", 2024-09-13T10:11:16.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119903+0000", 2024-09-13T10:11:16.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0929999999999999e-06 2024-09-13T10:11:16.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119919+0000", 2024-09-13T10:11:16.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6082999999999998e-05 2024-09-13T10:11:16.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119954+0000", 2024-09-13T10:11:16.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4944000000000003e-05 2024-09-13T10:11:16.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119983+0000", 2024-09-13T10:11:16.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8932e-05 2024-09-13T10:11:16.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.120157+0000", 2024-09-13T10:11:16.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017421 2024-09-13T10:11:16.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-13T10:11:16.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.120180+0000", 2024-09-13T10:11:16.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3039000000000001e-05 2024-09-13T10:11:16.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.121820+0000", 2024-09-13T10:11:16.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016397479999999999 2024-09-13T10:11:16.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:16.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.121842+0000", 2024-09-13T10:11:16.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2612e-05 2024-09-13T10:11:16.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.125236+0000", 2024-09-13T10:11:16.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003394181 2024-09-13T10:11:16.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.125597+0000", 2024-09-13T10:11:16.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036106199999999999 2024-09-13T10:11:16.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:16.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.125908+0000", 2024-09-13T10:11:16.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031034200000000002 2024-09-13T10:11:16.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.139886+0000", 2024-09-13T10:11:16.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013977824 2024-09-13T10:11:16.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140020+0000", 2024-09-13T10:11:16.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013476199999999999 2024-09-13T10:11:16.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140357+0000", 2024-09-13T10:11:16.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033711 2024-09-13T10:11:16.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140464+0000", 2024-09-13T10:11:16.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106028 2024-09-13T10:11:16.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140573+0000", 2024-09-13T10:11:16.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010912399999999999 2024-09-13T10:11:16.809 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.809 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.809 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.810 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:203 3.1 3:b033e801:::smithi19530714-4 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:16.810 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:11:16.810 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.806116847999998, 2024-09-13T10:11:16.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035547764000000003, 2024-09-13T10:11:16.810 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.810 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.810 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.810 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 203 2024-09-13T10:11:16.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.810 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:11:16.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:11:16.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991467+0000", 2024-09-13T10:11:16.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5030000000000002e-06 2024-09-13T10:11:16.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991483+0000", 2024-09-13T10:11:16.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5503e-05 2024-09-13T10:11:16.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991522+0000", 2024-09-13T10:11:16.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9399e-05 2024-09-13T10:11:16.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991544+0000", 2024-09-13T10:11:16.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2408999999999998e-05 2024-09-13T10:11:16.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991646+0000", 2024-09-13T10:11:16.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101798 2024-09-13T10:11:16.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991818+0000", 2024-09-13T10:11:16.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017204300000000001 2024-09-13T10:11:16.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:16.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.001618+0000", 2024-09-13T10:11:16.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097993490000000006 2024-09-13T10:11:16.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026785+0000", 2024-09-13T10:11:16.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025167446 2024-09-13T10:11:16.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026856+0000", 2024-09-13T10:11:16.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0575999999999996e-05 2024-09-13T10:11:16.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026915+0000", 2024-09-13T10:11:16.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9258000000000001e-05 2024-09-13T10:11:16.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026964+0000", 2024-09-13T10:11:16.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000003e-05 2024-09-13T10:11:16.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027011+0000", 2024-09-13T10:11:16.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7200000000000002e-05 2024-09-13T10:11:16.816 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.816 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.816 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.816 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:204 3.1 3:babcb876:::smithi19530714-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:16.816 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:11:16.816 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.805992331999999, 2024-09-13T10:11:16.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.22268644700000001, 2024-09-13T10:11:16.817 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.817 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.817 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.817 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.817 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.817 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.817 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 204 2024-09-13T10:11:16.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.817 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:11:16.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:11:16.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991589+0000", 2024-09-13T10:11:16.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3859999999999999e-06 2024-09-13T10:11:16.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991598+0000", 2024-09-13T10:11:16.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7669999999999992e-06 2024-09-13T10:11:16.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991616+0000", 2024-09-13T10:11:16.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7589e-05 2024-09-13T10:11:16.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991631+0000", 2024-09-13T10:11:16.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.47e-05 2024-09-13T10:11:16.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992050+0000", 2024-09-13T10:11:16.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041913399999999999 2024-09-13T10:11:16.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992229+0000", 2024-09-13T10:11:16.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017956499999999999 2024-09-13T10:11:16.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:16.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.203594+0000", 2024-09-13T10:11:16.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.211364303 2024-09-13T10:11:16.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.208526+0000", 2024-09-13T10:11:16.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049325020000000001 2024-09-13T10:11:16.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214019+0000", 2024-09-13T10:11:16.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054933270000000001 2024-09-13T10:11:16.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214108+0000", 2024-09-13T10:11:16.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8517000000000006e-05 2024-09-13T10:11:16.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214202+0000", 2024-09-13T10:11:16.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3763000000000003e-05 2024-09-13T10:11:16.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214275+0000", 2024-09-13T10:11:16.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2893999999999997e-05 2024-09-13T10:11:16.823 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.823 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.823 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.823 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:205 3.1 3:aec11a92:::smithi19530714-6 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:16.823 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:11:16.823 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.805916677999999, 2024-09-13T10:11:16.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056950259000000003, 2024-09-13T10:11:16.823 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.823 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.823 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.823 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.824 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.824 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.824 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 205 2024-09-13T10:11:16.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.824 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:11:16.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:11:16.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991665+0000", 2024-09-13T10:11:16.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1400000000000001e-06 2024-09-13T10:11:16.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991673+0000", 2024-09-13T10:11:16.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5730000000000008e-06 2024-09-13T10:11:16.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991689+0000", 2024-09-13T10:11:16.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5956000000000001e-05 2024-09-13T10:11:16.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991705+0000", 2024-09-13T10:11:16.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5539000000000001e-05 2024-09-13T10:11:16.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992393+0000", 2024-09-13T10:11:16.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068836600000000004 2024-09-13T10:11:16.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992509+0000", 2024-09-13T10:11:16.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115563 2024-09-13T10:11:16.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:16.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.011894+0000", 2024-09-13T10:11:16.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019385594999999999 2024-09-13T10:11:16.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027029+0000", 2024-09-13T10:11:16.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015134664000000001 2024-09-13T10:11:16.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.031277+0000", 2024-09-13T10:11:16.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004247804 2024-09-13T10:11:16.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048542+0000", 2024-09-13T10:11:16.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017265401999999999 2024-09-13T10:11:16.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048579+0000", 2024-09-13T10:11:16.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6238000000000001e-05 2024-09-13T10:11:16.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048614+0000", 2024-09-13T10:11:16.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5419000000000001e-05 2024-09-13T10:11:16.829 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.829 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.829 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.829 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:209 3.1 3:9798d58d:::smithi19530714-10 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:16.830 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:11:16.830 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.805788462999999, 2024-09-13T10:11:16.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.069824676000000002, 2024-09-13T10:11:16.830 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.830 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.830 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.830 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.830 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.830 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.830 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 209 2024-09-13T10:11:16.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.830 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:11:16.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:11:16.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991793+0000", 2024-09-13T10:11:16.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.057e-06 2024-09-13T10:11:16.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991803+0000", 2024-09-13T10:11:16.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7860000000000008e-06 2024-09-13T10:11:16.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991817+0000", 2024-09-13T10:11:16.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4066000000000001e-05 2024-09-13T10:11:16.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991828+0000", 2024-09-13T10:11:16.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1097000000000001e-05 2024-09-13T10:11:16.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992666+0000", 2024-09-13T10:11:16.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083755799999999997 2024-09-13T10:11:16.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992792+0000", 2024-09-13T10:11:16.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000126334 2024-09-13T10:11:16.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:16.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027675+0000", 2024-09-13T10:11:16.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034883299 2024-09-13T10:11:16.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.035648+0000", 2024-09-13T10:11:16.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007972972 2024-09-13T10:11:16.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.045764+0000", 2024-09-13T10:11:16.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010115832 2024-09-13T10:11:16.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061460+0000", 2024-09-13T10:11:16.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015695563999999999 2024-09-13T10:11:16.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061544+0000", 2024-09-13T10:11:16.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4866999999999999e-05 2024-09-13T10:11:16.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061617+0000", 2024-09-13T10:11:16.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2243999999999995e-05 2024-09-13T10:11:16.836 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.836 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.836 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.836 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:213 3.1 3:accf4e56:::smithi19530714-14 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:16.836 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:11:16.836 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.80501452, 2024-09-13T10:11:16.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.060593045999999998, 2024-09-13T10:11:16.836 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.836 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.837 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.837 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.837 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.837 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.837 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 213 2024-09-13T10:11:16.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.837 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:11:16.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:11:16.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992568+0000", 2024-09-13T10:11:16.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3640000000000002e-06 2024-09-13T10:11:16.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992580+0000", 2024-09-13T10:11:16.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1796e-05 2024-09-13T10:11:16.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992662+0000", 2024-09-13T10:11:16.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1814e-05 2024-09-13T10:11:16.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992683+0000", 2024-09-13T10:11:16.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0746000000000001e-05 2024-09-13T10:11:16.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.993024+0000", 2024-09-13T10:11:16.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034141299999999999 2024-09-13T10:11:16.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.993182+0000", 2024-09-13T10:11:16.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015817699999999999 2024-09-13T10:11:16.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:16.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.019852+0000", 2024-09-13T10:11:16.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026669470000000001 2024-09-13T10:11:16.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.032551+0000", 2024-09-13T10:11:16.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012699729999999999 2024-09-13T10:11:16.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.034351+0000", 2024-09-13T10:11:16.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017998269999999999 2024-09-13T10:11:16.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053106+0000", 2024-09-13T10:11:16.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018755236000000002 2024-09-13T10:11:16.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053131+0000", 2024-09-13T10:11:16.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4729e-05 2024-09-13T10:11:16.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053159+0000", 2024-09-13T10:11:16.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7744000000000001e-05 2024-09-13T10:11:16.842 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.842 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.843 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.843 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:219 3.1 3:87bfc7b1:::smithi19530714-20 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:16.843 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:11:16.843 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.764565446999999, 2024-09-13T10:11:16.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.037494069999999997, 2024-09-13T10:11:16.843 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.843 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.843 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.843 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.843 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.843 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.843 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 219 2024-09-13T10:11:16.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.844 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:11:16.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:11:16.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033019+0000", 2024-09-13T10:11:16.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.726e-06 2024-09-13T10:11:16.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033036+0000", 2024-09-13T10:11:16.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7329999999999998e-05 2024-09-13T10:11:16.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033060+0000", 2024-09-13T10:11:16.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4286000000000001e-05 2024-09-13T10:11:16.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033079+0000", 2024-09-13T10:11:16.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8872000000000002e-05 2024-09-13T10:11:16.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033155+0000", 2024-09-13T10:11:16.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6118999999999994e-05 2024-09-13T10:11:16.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033297+0000", 2024-09-13T10:11:16.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014154700000000001 2024-09-13T10:11:16.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:16.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.041666+0000", 2024-09-13T10:11:16.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083696180000000005 2024-09-13T10:11:16.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046621+0000", 2024-09-13T10:11:16.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049545759999999996 2024-09-13T10:11:16.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.052333+0000", 2024-09-13T10:11:16.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057120160000000003 2024-09-13T10:11:16.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070340+0000", 2024-09-13T10:11:16.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018006859 2024-09-13T10:11:16.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070447+0000", 2024-09-13T10:11:16.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106637 2024-09-13T10:11:16.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070509+0000", 2024-09-13T10:11:16.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2483999999999993e-05 2024-09-13T10:11:16.849 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.849 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.849 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.849 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:222 3.1 3:bdac4e61:::smithi19530714-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:16.850 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:11:16.850 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.750877472999999, 2024-09-13T10:11:16.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041434034000000002, 2024-09-13T10:11:16.850 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.850 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.850 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.850 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.850 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.850 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.850 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 222 2024-09-13T10:11:16.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.850 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:11:16.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:11:16.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046706+0000", 2024-09-13T10:11:16.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0089999999999999e-06 2024-09-13T10:11:16.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046719+0000", 2024-09-13T10:11:16.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2565000000000001e-05 2024-09-13T10:11:16.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046740+0000", 2024-09-13T10:11:16.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0979e-05 2024-09-13T10:11:16.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046752+0000", 2024-09-13T10:11:16.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2917e-05 2024-09-13T10:11:16.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046803+0000", 2024-09-13T10:11:16.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0417000000000002e-05 2024-09-13T10:11:16.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046908+0000", 2024-09-13T10:11:16.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105039 2024-09-13T10:11:16.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:16.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.066097+0000", 2024-09-13T10:11:16.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019189277000000001 2024-09-13T10:11:16.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070969+0000", 2024-09-13T10:11:16.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048721429999999998 2024-09-13T10:11:16.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.087852+0000", 2024-09-13T10:11:16.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016882489000000001 2024-09-13T10:11:16.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.087942+0000", 2024-09-13T10:11:16.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9738e-05 2024-09-13T10:11:16.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088076+0000", 2024-09-13T10:11:16.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013400700000000001 2024-09-13T10:11:16.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088137+0000", 2024-09-13T10:11:16.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1453999999999998e-05 2024-09-13T10:11:16.857 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.857 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.857 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.857 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:223 3.1 3:8759f4e0:::smithi19530714-24 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:16.857 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:11:16.858 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.738200603999999, 2024-09-13T10:11:16.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053691092000000003, 2024-09-13T10:11:16.858 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.858 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.858 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.858 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.858 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.858 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.858 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 223 2024-09-13T10:11:16.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.858 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.858 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.858 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:11:16.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:11:16.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059383+0000", 2024-09-13T10:11:16.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.847e-06 2024-09-13T10:11:16.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059399+0000", 2024-09-13T10:11:16.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6739e-05 2024-09-13T10:11:16.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059433+0000", 2024-09-13T10:11:16.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3946000000000003e-05 2024-09-13T10:11:16.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059456+0000", 2024-09-13T10:11:16.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2163000000000001e-05 2024-09-13T10:11:16.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059621+0000", 2024-09-13T10:11:16.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016546700000000001 2024-09-13T10:11:16.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.060003+0000", 2024-09-13T10:11:16.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000381982 2024-09-13T10:11:16.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:16.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.085339+0000", 2024-09-13T10:11:16.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025335610000000001 2024-09-13T10:11:16.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.089896+0000", 2024-09-13T10:11:16.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045575930000000004 2024-09-13T10:11:16.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.102515+0000", 2024-09-13T10:11:16.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012618664999999999 2024-09-13T10:11:16.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113009+0000", 2024-09-13T10:11:16.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010493784000000001 2024-09-13T10:11:16.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113044+0000", 2024-09-13T10:11:16.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5543999999999997e-05 2024-09-13T10:11:16.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113071+0000", 2024-09-13T10:11:16.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6752000000000001e-05 2024-09-13T10:11:16.864 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.864 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.864 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.864 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:231 3.1 3:9d247594:::smithi19530714-32 low_tier:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:16.864 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.065612+0000", 2024-09-13T10:11:16.864 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.731967946000001, 2024-09-13T10:11:16.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031259105000000002, 2024-09-13T10:11:16.864 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.864 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.864 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.865 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.865 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.865 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.865 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 231 2024-09-13T10:11:16.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.865 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.865 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065612+0000", 2024-09-13T10:11:16.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065612+0000", 2024-09-13T10:11:16.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065614+0000", 2024-09-13T10:11:16.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7370000000000001e-06 2024-09-13T10:11:16.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065622+0000", 2024-09-13T10:11:16.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1110000000000008e-06 2024-09-13T10:11:16.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065640+0000", 2024-09-13T10:11:16.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7499999999999998e-05 2024-09-13T10:11:16.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065655+0000", 2024-09-13T10:11:16.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5021999999999999e-05 2024-09-13T10:11:16.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069879+0000", 2024-09-13T10:11:16.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042246360000000004 2024-09-13T10:11:16.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070080+0000", 2024-09-13T10:11:16.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020099199999999999 2024-09-13T10:11:16.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:16.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.075971+0000", 2024-09-13T10:11:16.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058906030000000003 2024-09-13T10:11:16.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088156+0000", 2024-09-13T10:11:16.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012184868999999999 2024-09-13T10:11:16.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.095237+0000", 2024-09-13T10:11:16.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0070814279999999999 2024-09-13T10:11:16.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.869 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.869 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.096766+0000", 2024-09-13T10:11:16.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001529007 2024-09-13T10:11:16.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.096823+0000", 2024-09-13T10:11:16.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6789999999999997e-05 2024-09-13T10:11:16.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.096872+0000", 2024-09-13T10:11:16.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8409999999999999e-05 2024-09-13T10:11:16.870 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.870 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.870 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.871 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:234 3.1 3:bdce9caa:::smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:16.871 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:11:16.871 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.728481342999999, 2024-09-13T10:11:16.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043032263000000001, 2024-09-13T10:11:16.871 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.871 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.871 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.871 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.871 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.871 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.871 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 234 2024-09-13T10:11:16.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.872 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:11:16.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:11:16.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069102+0000", 2024-09-13T10:11:16.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.543e-06 2024-09-13T10:11:16.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069112+0000", 2024-09-13T10:11:16.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0091e-05 2024-09-13T10:11:16.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069133+0000", 2024-09-13T10:11:16.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1188000000000001e-05 2024-09-13T10:11:16.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069143+0000", 2024-09-13T10:11:16.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0456e-05 2024-09-13T10:11:16.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.874 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070588+0000", 2024-09-13T10:11:16.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001445046 2024-09-13T10:11:16.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.874 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070801+0000", 2024-09-13T10:11:16.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000212151 2024-09-13T10:11:16.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:16.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.082362+0000", 2024-09-13T10:11:16.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561781 2024-09-13T10:11:16.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088161+0000", 2024-09-13T10:11:16.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057986119999999999 2024-09-13T10:11:16.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.101972+0000", 2024-09-13T10:11:16.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013811498 2024-09-13T10:11:16.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112028+0000", 2024-09-13T10:11:16.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010055530999999999 2024-09-13T10:11:16.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112082+0000", 2024-09-13T10:11:16.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3974999999999997e-05 2024-09-13T10:11:16.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112131+0000", 2024-09-13T10:11:16.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9391000000000002e-05 2024-09-13T10:11:16.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.877 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.877 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.877 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:243 3.1 3:a4dd4282:::smithi19530714-44 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:16.877 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:11:16.877 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.700273637999999, 2024-09-13T10:11:16.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036759694000000002, 2024-09-13T10:11:16.878 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.878 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.878 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.878 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.878 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.878 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.878 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 243 2024-09-13T10:11:16.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.878 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:11:16.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:11:16.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097308+0000", 2024-09-13T10:11:16.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2099999999999995e-07 2024-09-13T10:11:16.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097314+0000", 2024-09-13T10:11:16.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9989999999999997e-06 2024-09-13T10:11:16.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097326+0000", 2024-09-13T10:11:16.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1787000000000001e-05 2024-09-13T10:11:16.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097332+0000", 2024-09-13T10:11:16.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6950000000000003e-06 2024-09-13T10:11:16.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097378+0000", 2024-09-13T10:11:16.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5744000000000001e-05 2024-09-13T10:11:16.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097489+0000", 2024-09-13T10:11:16.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111352 2024-09-13T10:11:16.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:16.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.118571+0000", 2024-09-13T10:11:16.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021081249999999999 2024-09-13T10:11:16.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.128494+0000", 2024-09-13T10:11:16.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099237270000000002 2024-09-13T10:11:16.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.128775+0000", 2024-09-13T10:11:16.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028027299999999999 2024-09-13T10:11:16.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.133963+0000", 2024-09-13T10:11:16.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005188191 2024-09-13T10:11:16.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134015+0000", 2024-09-13T10:11:16.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2438000000000002e-05 2024-09-13T10:11:16.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134066+0000", 2024-09-13T10:11:16.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1316999999999997e-05 2024-09-13T10:11:16.883 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.884 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.884 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.884 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:248 3.1 3:b0d90ce0:::smithi19530714-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 18] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:16.884 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.114744+0000", 2024-09-13T10:11:16.884 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.682836736999999, 2024-09-13T10:11:16.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028886754000000001, 2024-09-13T10:11:16.884 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.884 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.884 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.884 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.885 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.885 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.885 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 248 2024-09-13T10:11:16.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.885 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114744+0000", 2024-09-13T10:11:16.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.885 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.885 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.885 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.885 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114744+0000", 2024-09-13T10:11:16.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114745+0000", 2024-09-13T10:11:16.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.55e-06 2024-09-13T10:11:16.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.886 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114752+0000", 2024-09-13T10:11:16.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1670000000000002e-06 2024-09-13T10:11:16.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.886 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114768+0000", 2024-09-13T10:11:16.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5471000000000001e-05 2024-09-13T10:11:16.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114775+0000", 2024-09-13T10:11:16.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3799999999999996e-06 2024-09-13T10:11:16.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114826+0000", 2024-09-13T10:11:16.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0909999999999999e-05 2024-09-13T10:11:16.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114981+0000", 2024-09-13T10:11:16.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015455699999999999 2024-09-13T10:11:16.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:16.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.125745+0000", 2024-09-13T10:11:16.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010764404 2024-09-13T10:11:16.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.129548+0000", 2024-09-13T10:11:16.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038027019999999998 2024-09-13T10:11:16.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.140602+0000", 2024-09-13T10:11:16.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011054082 2024-09-13T10:11:16.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.143523+0000", 2024-09-13T10:11:16.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002920962 2024-09-13T10:11:16.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.143586+0000", 2024-09-13T10:11:16.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2866000000000004e-05 2024-09-13T10:11:16.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.143630+0000", 2024-09-13T10:11:16.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4703000000000002e-05 2024-09-13T10:11:16.890 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.890 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.890 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.891 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:280 4.f 4:fe60987c:::smithi19530714-31:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:16.891 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.236614+0000", 2024-09-13T10:11:16.891 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.560966849, 2024-09-13T10:11:16.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021117787999999998, 2024-09-13T10:11:16.891 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.891 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.891 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.891 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.891 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.891 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.891 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280 2024-09-13T10:11:16.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.891 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236614+0000", 2024-09-13T10:11:16.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236614+0000", 2024-09-13T10:11:16.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236616+0000", 2024-09-13T10:11:16.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4700000000000001e-06 2024-09-13T10:11:16.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236627+0000", 2024-09-13T10:11:16.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1226000000000001e-05 2024-09-13T10:11:16.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236654+0000", 2024-09-13T10:11:16.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6876000000000002e-05 2024-09-13T10:11:16.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236675+0000", 2024-09-13T10:11:16.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1199000000000002e-05 2024-09-13T10:11:16.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236772+0000", 2024-09-13T10:11:16.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6359000000000004e-05 2024-09-13T10:11:16.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.894 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.894 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236873+0000", 2024-09-13T10:11:16.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010113199999999999 2024-09-13T10:11:16.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2,7", 2024-09-13T10:11:16.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.254958+0000", 2024-09-13T10:11:16.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018085127999999999 2024-09-13T10:11:16.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.256855+0000", 2024-09-13T10:11:16.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001896888 2024-09-13T10:11:16.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257456+0000", 2024-09-13T10:11:16.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060094800000000002 2024-09-13T10:11:16.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257633+0000", 2024-09-13T10:11:16.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017736499999999999 2024-09-13T10:11:16.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257684+0000", 2024-09-13T10:11:16.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0812000000000001e-05 2024-09-13T10:11:16.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257731+0000", 2024-09-13T10:11:16.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7385000000000001e-05 2024-09-13T10:11:16.897 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.897 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.897 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.897 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:292 4.f 4:f6a0e8ce:::smithi19530714-43:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:16.897 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.257863+0000", 2024-09-13T10:11:16.897 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.539717517, 2024-09-13T10:11:16.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022747474, 2024-09-13T10:11:16.898 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.898 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.898 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.898 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.898 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.898 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.898 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 292 2024-09-13T10:11:16.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.898 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257863+0000", 2024-09-13T10:11:16.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257863+0000", 2024-09-13T10:11:16.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257865+0000", 2024-09-13T10:11:16.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.883e-06 2024-09-13T10:11:16.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257874+0000", 2024-09-13T10:11:16.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0699999999999996e-06 2024-09-13T10:11:16.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257894+0000", 2024-09-13T10:11:16.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0349000000000001e-05 2024-09-13T10:11:16.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257925+0000", 2024-09-13T10:11:16.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1092e-05 2024-09-13T10:11:16.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258040+0000", 2024-09-13T10:11:16.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114474 2024-09-13T10:11:16.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258126+0000", 2024-09-13T10:11:16.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6479999999999999e-05 2024-09-13T10:11:16.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2,7", 2024-09-13T10:11:16.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.273691+0000", 2024-09-13T10:11:16.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015564626999999999 2024-09-13T10:11:16.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.275091+0000", 2024-09-13T10:11:16.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001400079 2024-09-13T10:11:16.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.902 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.276990+0000", 2024-09-13T10:11:16.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018994179999999999 2024-09-13T10:11:16.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.280333+0000", 2024-09-13T10:11:16.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033424320000000002 2024-09-13T10:11:16.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.280486+0000", 2024-09-13T10:11:16.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015310500000000001 2024-09-13T10:11:16.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.280610+0000", 2024-09-13T10:11:16.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012446500000000001 2024-09-13T10:11:16.903 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.903 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.904 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.904 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:300 4.f 4:f7ad6ef6:::smithi19530714-1:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:16.904 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.264825+0000", 2024-09-13T10:11:16.904 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.532755215000002, 2024-09-13T10:11:16.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018261785999999999, 2024-09-13T10:11:16.904 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.904 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.904 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.904 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.905 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.905 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 300 2024-09-13T10:11:16.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.905 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264825+0000", 2024-09-13T10:11:16.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264825+0000", 2024-09-13T10:11:16.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264827+0000", 2024-09-13T10:11:16.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.204e-06 2024-09-13T10:11:16.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264838+0000", 2024-09-13T10:11:16.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0212e-05 2024-09-13T10:11:16.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264859+0000", 2024-09-13T10:11:16.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1613000000000001e-05 2024-09-13T10:11:16.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264873+0000", 2024-09-13T10:11:16.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3893000000000001e-05 2024-09-13T10:11:16.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264941+0000", 2024-09-13T10:11:16.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8355000000000005e-05 2024-09-13T10:11:16.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.265012+0000", 2024-09-13T10:11:16.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0759999999999993e-05 2024-09-13T10:11:16.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2,7", 2024-09-13T10:11:16.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.276117+0000", 2024-09-13T10:11:16.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011104283 2024-09-13T10:11:16.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.278041+0000", 2024-09-13T10:11:16.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001924427 2024-09-13T10:11:16.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.278343+0000", 2024-09-13T10:11:16.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030249900000000002 2024-09-13T10:11:16.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.909 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.283022+0000", 2024-09-13T10:11:16.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046781840000000002 2024-09-13T10:11:16.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.909 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.909 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.283053+0000", 2024-09-13T10:11:16.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1696999999999998e-05 2024-09-13T10:11:16.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.283087+0000", 2024-09-13T10:11:16.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3658999999999997e-05 2024-09-13T10:11:16.910 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.910 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.910 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.910 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.1.22:23 3.1 3:969e51ee:::93c498e92137985868c9dcdff823d39ad7f943e0cf15bbbe4a7d3098e4bc2da6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:16.911 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.128192+0000", 2024-09-13T10:11:16.911 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.669388784999999, 2024-09-13T10:11:16.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016280833000000001, 2024-09-13T10:11:16.911 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.911 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.911 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.911 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128192+0000", 2024-09-13T10:11:16.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128192+0000", 2024-09-13T10:11:16.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128193+0000", 2024-09-13T10:11:16.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0139999999999999e-06 2024-09-13T10:11:16.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128359+0000", 2024-09-13T10:11:16.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000166455 2024-09-13T10:11:16.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128372+0000", 2024-09-13T10:11:16.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3029e-05 2024-09-13T10:11:16.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128382+0000", 2024-09-13T10:11:16.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.55e-06 2024-09-13T10:11:16.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.137523+0000", 2024-09-13T10:11:16.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091413320000000003 2024-09-13T10:11:16.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.137721+0000", 2024-09-13T10:11:16.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000197777 2024-09-13T10:11:16.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:16.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.138223+0000", 2024-09-13T10:11:16.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050221499999999997 2024-09-13T10:11:16.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.144219+0000", 2024-09-13T10:11:16.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059962310000000003 2024-09-13T10:11:16.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.144284+0000", 2024-09-13T10:11:16.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4734999999999996e-05 2024-09-13T10:11:16.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.915 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.144319+0000", 2024-09-13T10:11:16.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5271999999999997e-05 2024-09-13T10:11:16.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.144403+0000", 2024-09-13T10:11:16.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3792e-05 2024-09-13T10:11:16.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.144472+0000", 2024-09-13T10:11:16.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9431000000000007e-05 2024-09-13T10:11:16.916 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.916 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.916 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.916 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.1.22:26 3.1 3:81602715:::0cd1941c9b5b3a2b092b213a045ec3831dc9f094ae77b0454d92fa84833c395e:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:16.917 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.128673+0000", 2024-09-13T10:11:16.917 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.668907015999999, 2024-09-13T10:11:16.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017705240000000001, 2024-09-13T10:11:16.917 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.917 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.917 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.917 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128673+0000", 2024-09-13T10:11:16.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128673+0000", 2024-09-13T10:11:16.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128676+0000", 2024-09-13T10:11:16.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9129999999999999e-06 2024-09-13T10:11:16.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.129361+0000", 2024-09-13T10:11:16.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068506999999999997 2024-09-13T10:11:16.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.129403+0000", 2024-09-13T10:11:16.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1776000000000001e-05 2024-09-13T10:11:16.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.129420+0000", 2024-09-13T10:11:16.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7125e-05 2024-09-13T10:11:16.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.141718+0000", 2024-09-13T10:11:16.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012297439 2024-09-13T10:11:16.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.141913+0000", 2024-09-13T10:11:16.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019493000000000001 2024-09-13T10:11:16.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:16.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.142410+0000", 2024-09-13T10:11:16.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049744199999999996 2024-09-13T10:11:16.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.145111+0000", 2024-09-13T10:11:16.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027010329999999998 2024-09-13T10:11:16.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.145797+0000", 2024-09-13T10:11:16.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068605300000000005 2024-09-13T10:11:16.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.146186+0000", 2024-09-13T10:11:16.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038835200000000001 2024-09-13T10:11:16.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.146290+0000", 2024-09-13T10:11:16.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010416000000000001 2024-09-13T10:11:16.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.146379+0000", 2024-09-13T10:11:16.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8947000000000001e-05 2024-09-13T10:11:16.922 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.922 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.923 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.923 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1272 4.f 4:f7ad6ef6:::smithi19530714-1:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e45)", 2024-09-13T10:11:16.923 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:11:16.923 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.922045463, 2024-09-13T10:11:16.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.050041173000000001, 2024-09-13T10:11:16.923 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.923 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.923 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:16.923 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:16.923 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:16.923 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:16.924 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1272 2024-09-13T10:11:16.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.924 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:11:16.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:11:16.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875537+0000", 2024-09-13T10:11:16.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2689999999999998e-06 2024-09-13T10:11:16.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875548+0000", 2024-09-13T10:11:16.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0291e-05 2024-09-13T10:11:16.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875567+0000", 2024-09-13T10:11:16.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9284999999999999e-05 2024-09-13T10:11:16.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875578+0000", 2024-09-13T10:11:16.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1256e-05 2024-09-13T10:11:16.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875763+0000", 2024-09-13T10:11:16.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000185396 2024-09-13T10:11:16.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875908+0000", 2024-09-13T10:11:16.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014412299999999999 2024-09-13T10:11:16.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.925576+0000", 2024-09-13T10:11:16.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049668552999999997 2024-09-13T10:11:16.927 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.927 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.927 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.928 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.8.23:18 3.1 3:b8525f43:::de5f2c8a0f038bec78c01b38757160286dcdb61c7c0f452da0dab0bc4f36d9ec:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:16.928 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.116176+0000", 2024-09-13T10:11:16.928 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.681404288, 2024-09-13T10:11:16.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014979068, 2024-09-13T10:11:16.928 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:16.928 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:16.928 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:16.928 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:16.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:16.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.116176+0000", 2024-09-13T10:11:16.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:16.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.116176+0000", 2024-09-13T10:11:16.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:16.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:16.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.116178+0000", 2024-09-13T10:11:16.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5069999999999999e-06 2024-09-13T10:11:16.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:16.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.116912+0000", 2024-09-13T10:11:16.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073388900000000005 2024-09-13T10:11:16.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:16.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.116936+0000", 2024-09-13T10:11:16.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4179e-05 2024-09-13T10:11:16.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:16.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.116956+0000", 2024-09-13T10:11:16.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0115999999999998e-05 2024-09-13T10:11:16.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:16.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.121763+0000", 2024-09-13T10:11:16.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048070400000000003 2024-09-13T10:11:16.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:16.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.121993+0000", 2024-09-13T10:11:16.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000230394 2024-09-13T10:11:16.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:16.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.122441+0000", 2024-09-13T10:11:16.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044773000000000002 2024-09-13T10:11:16.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:16.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.126110+0000", 2024-09-13T10:11:16.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036694259999999999 2024-09-13T10:11:16.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.126444+0000", 2024-09-13T10:11:16.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033349900000000001 2024-09-13T10:11:16.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:16.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.130994+0000", 2024-09-13T10:11:16.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045496479999999999 2024-09-13T10:11:16.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:16.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.131079+0000", 2024-09-13T10:11:16.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5345000000000004e-05 2024-09-13T10:11:16.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:16.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:16.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:16.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.131155+0000", 2024-09-13T10:11:16.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6295000000000001e-05 2024-09-13T10:11:16.933 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.934 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.934 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.934 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:16.934 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:16.934 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:16.934 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:11:16.934 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:11:16.934 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:11:16.935 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T10:11:16.935 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T10:11:16.936 INFO:tasks.rados.rados.0.smithi195.stdout:1069: finishing set_chunk to oid 31 2024-09-13T10:11:16.936 INFO:tasks.rados.rados.0.smithi195.stdout:1069: got expected ENOENT (src dne) 2024-09-13T10:11:16.936 INFO:tasks.rados.rados.0.smithi195.stdout:1066: expect (ObjNum 165 snap 0 seq_num 165) 2024-09-13T10:11:16.936 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 267575 2024-09-13T10:11:16.937 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:16.935+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:16.937 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T10:11:17.029 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:17.036 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:17.035+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:17.070 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:17.070 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:17.070 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:11:17.070 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:17.084 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T10:11:17.088 INFO:tasks.rados.rados.0.smithi195.stdout:1068: expect (ObjNum 158 snap 0 seq_num 158) 2024-09-13T10:11:17.088 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:17.138 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:17.133+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:17.139 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:17.202 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:17.202 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:17.202 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:11:17.202 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:11:17.202 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:17.216 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T10:11:17.238 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:17.237+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:17.240 INFO:tasks.rados.rados.0.smithi195.stdout:1070: expect (ObjNum 187 snap 0 seq_num 187) 2024-09-13T10:11:17.240 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:11:17.255 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: "epoch": 59, 2024-09-13T10:11:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: "fsid": "c1e91d47-9492-4461-a043-8698fd74ccf3", 2024-09-13T10:11:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "created": "2024-09-13T10:10:14.019912+0000", 2024-09-13T10:11:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "modified": "2024-09-13T10:11:15.734826+0000", 2024-09-13T10:11:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "last_up_change": "2024-09-13T10:10:28.089885+0000", 2024-09-13T10:11:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "last_in_change": "2024-09-13T10:10:18.489546+0000", 2024-09-13T10:11:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T10:11:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_num": 5799936, 2024-09-13T10:11:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_set": [ 2024-09-13T10:11:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "pglog_hardlimit", 2024-09-13T10:11:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "purged_snapdirs", 2024-09-13T10:11:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_deletes", 2024-09-13T10:11:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "sortbitwise" 2024-09-13T10:11:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_version": 6, 2024-09-13T10:11:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T10:11:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T10:11:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T10:11:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_snapshot": "", 2024-09-13T10:11:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_max": 4, 2024-09-13T10:11:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd": 16, 2024-09-13T10:11:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "require_min_compat_client": "luminous", 2024-09-13T10:11:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "min_compat_client": "jewel", 2024-09-13T10:11:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "require_osd_release": "squid", 2024-09-13T10:11:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_crimson": false, 2024-09-13T10:11:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "pools": [ 2024-09-13T10:11:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 1, 2024-09-13T10:11:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "rbd", 2024-09-13T10:11:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:28.924860+0000", 2024-09-13T10:11:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-13T10:11:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T10:11:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:11:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:11:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:11:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:11:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:11:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:11:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:11:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:11:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:11:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:11:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:11:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 8, 2024-09-13T10:11:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 8, 2024-09-13T10:11:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 8, 2024-09-13T10:11:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 8, 2024-09-13T10:11:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 8, 2024-09-13T10:11:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:11:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:11:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:11:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:11:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:11:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:11:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:11:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "27", 2024-09-13T10:11:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:11:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:11:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:11:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:11:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:11:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 2, 2024-09-13T10:11:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 27, 2024-09-13T10:11:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:11:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:11:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:11:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:11:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:11:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:11:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:11:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:11:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:11:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:11:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:11:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:11:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:11:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:11:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:11:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:11:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:11:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:11:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:11:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:11:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:11:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:11:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:11:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:11:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:11:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:11:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:11:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:11:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:11:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:11:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-13T10:11:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-13T10:11:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-13T10:11:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:11:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:11:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 3.7699999809265137, 2024-09-13T10:11:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 3.7699999809265137, 2024-09-13T10:11:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.73000001907348633, 2024-09-13T10:11:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2.75, 2024-09-13T10:11:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2.75, 2024-09-13T10:11:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8997650146484375, 2024-09-13T10:11:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:11:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-13T10:11:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-13T10:11:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": ".mgr", 2024-09-13T10:11:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:28.972519+0000", 2024-09-13T10:11:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-13T10:11:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-13T10:11:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:11:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:11:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:11:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:11:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:11:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:11:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:11:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:11:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:11:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:11:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:11:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 1, 2024-09-13T10:11:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 1, 2024-09-13T10:11:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 1, 2024-09-13T10:11:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 1, 2024-09-13T10:11:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 1, 2024-09-13T10:11:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:11:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:11:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:11:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:11:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:11:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:11:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:11:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "26", 2024-09-13T10:11:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:11:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:11:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:11:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:11:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:11:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-13T10:11:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-13T10:11:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:11:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:11:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:11:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:11:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:11:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:11:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:11:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:11:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:11:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:11:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:11:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:11:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:11:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:11:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:11:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:11:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:11:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:11:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:11:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:11:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:11:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:11:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:11:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:11:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:11:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:11:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:11:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:11:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:11:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:11:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-13T10:11:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_max": 32, 2024-09-13T10:11:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_min": 1 2024-09-13T10:11:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-13T10:11:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": {} 2024-09-13T10:11:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:11:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:11:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 15, 2024-09-13T10:11:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 15, 2024-09-13T10:11:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.20000000298023224, 2024-09-13T10:11:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 3, 2024-09-13T10:11:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 3, 2024-09-13T10:11:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8997650146484375, 2024-09-13T10:11:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:11:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-13T10:11:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-13T10:11:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "low_tier", 2024-09-13T10:11:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:41.320719+0000", 2024-09-13T10:11:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-13T10:11:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-13T10:11:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:11:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:11:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:11:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:11:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:11:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:11:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:11:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:11:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:11:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:11:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:11:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 4, 2024-09-13T10:11:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 4, 2024-09-13T10:11:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 4, 2024-09-13T10:11:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 4, 2024-09-13T10:11:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 4, 2024-09-13T10:11:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:11:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:11:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:11:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:11:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:11:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:11:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:11:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "32", 2024-09-13T10:11:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:11:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:11:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:11:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:11:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:11:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-13T10:11:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-13T10:11:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:11:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:11:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:11:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:11:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:11:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:11:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:11:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:11:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:11:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:11:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:11:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:11:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:11:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:11:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:11:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:11:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:11:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:11:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:11:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:11:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:11:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:11:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:11:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:11:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:11:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:11:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:11:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:11:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:11:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:11:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-13T10:11:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": {}, 2024-09-13T10:11:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:11:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:11:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 3.75, 2024-09-13T10:11:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 3.75, 2024-09-13T10:11:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.60000002384185791, 2024-09-13T10:11:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2.25, 2024-09-13T10:11:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2.25, 2024-09-13T10:11:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8997650146484375, 2024-09-13T10:11:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:11:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-13T10:11:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 4, 2024-09-13T10:11:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "unique_pool_0", 2024-09-13T10:11:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:42.575395+0000", 2024-09-13T10:11:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-13T10:11:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-13T10:11:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:11:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:11:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:11:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:11:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:11:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:11:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:11:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:11:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:11:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:11:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:11:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 26, 2024-09-13T10:11:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 21, 2024-09-13T10:11:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 26, 2024-09-13T10:11:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 26, 2024-09-13T10:11:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 26, 2024-09-13T10:11:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:11:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:11:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:11:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:11:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:11:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:11:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:11:17.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.280 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "57", 2024-09-13T10:11:17.280 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:11:17.280 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:11:17.280 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "41", 2024-09-13T10:11:17.280 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:11:17.280 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:11:17.280 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-13T10:11:17.280 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-13T10:11:17.280 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:11:17.280 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:11:17.280 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:11:17.280 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:11:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:11:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:11:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:11:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:11:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:11:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:11:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:11:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:11:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:11:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:11:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:11:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:11:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:11:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:11:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:11:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:11:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:11:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:11:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:11:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:11:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:11:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:11:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:11:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:11:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:11:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:11:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-13T10:11:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "dedup_cdc_chunk_size": 131072, 2024-09-13T10:11:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "dedup_chunk_algorithm": "fastcdc", 2024-09-13T10:11:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "dedup_tier": 3, 2024-09-13T10:11:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "fingerprint_algorithm": "sha256" 2024-09-13T10:11:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-13T10:11:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "rados": {} 2024-09-13T10:11:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:11:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:11:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 4.0300002098083496, 2024-09-13T10:11:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 3.4500000476837158, 2024-09-13T10:11:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 1.0700000524520874, 2024-09-13T10:11:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 4.309999942779541, 2024-09-13T10:11:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 3.690000057220459, 2024-09-13T10:11:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8997650146484375, 2024-09-13T10:11:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:11:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-13T10:11:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-13T10:11:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "f7d92188-c27c-4744-a330-4b7105e0f2d6", 2024-09-13T10:11:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 0, 2024-09-13T10:11:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6812", 2024-09-13T10:11:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:11:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6813", 2024-09-13T10:11:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:11:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6815", 2024-09-13T10:11:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:11:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6814", 2024-09-13T10:11:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:11:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6812/1977392321", 2024-09-13T10:11:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6813/1977392321", 2024-09-13T10:11:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6815/1977392321", 2024-09-13T10:11:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6814/1977392321", 2024-09-13T10:11:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-13T10:11:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e900b771-7c96-4b5d-93d7-35a8ea6c026a", 2024-09-13T10:11:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 41, 2024-09-13T10:11:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6808", 2024-09-13T10:11:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:11:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6809", 2024-09-13T10:11:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:11:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6811", 2024-09-13T10:11:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:11:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6810", 2024-09-13T10:11:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:11:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6808/1228102864", 2024-09-13T10:11:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6809/1228102864", 2024-09-13T10:11:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6811/1228102864", 2024-09-13T10:11:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6810/1228102864", 2024-09-13T10:11:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-13T10:11:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "28b7aa59-33b2-4e24-8f3d-758f90e25f68", 2024-09-13T10:11:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 57, 2024-09-13T10:11:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6800", 2024-09-13T10:11:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:11:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6801", 2024-09-13T10:11:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:11:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6803", 2024-09-13T10:11:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:11:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6802", 2024-09-13T10:11:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:11:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6800/3314820936", 2024-09-13T10:11:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6801/3314820936", 2024-09-13T10:11:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6803/3314820936", 2024-09-13T10:11:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6802/3314820936", 2024-09-13T10:11:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-13T10:11:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "a93d2df5-781e-49e1-a507-1718b6f954ed", 2024-09-13T10:11:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 55, 2024-09-13T10:11:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6801", 2024-09-13T10:11:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:11:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6802", 2024-09-13T10:11:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:11:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6804", 2024-09-13T10:11:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:11:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6803", 2024-09-13T10:11:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:11:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6801/2558565460", 2024-09-13T10:11:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6802/2558565460", 2024-09-13T10:11:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6804/2558565460", 2024-09-13T10:11:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6803/2558565460", 2024-09-13T10:11:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-13T10:11:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e69053fd-f6d9-437e-a9d0-8896de6b927f", 2024-09-13T10:11:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 58, 2024-09-13T10:11:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6800", 2024-09-13T10:11:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:11:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6801", 2024-09-13T10:11:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:11:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6803", 2024-09-13T10:11:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:11:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6802", 2024-09-13T10:11:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:11:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6800/858330493", 2024-09-13T10:11:17.307 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6801/858330493", 2024-09-13T10:11:17.307 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6803/858330493", 2024-09-13T10:11:17.307 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6802/858330493", 2024-09-13T10:11:17.307 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:17.307 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:17.307 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:17.307 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.307 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.307 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-13T10:11:17.307 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "3667ec10-561e-4a23-8197-6357db4af315", 2024-09-13T10:11:17.307 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:17.307 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:17.307 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:17.308 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:17.308 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:17.308 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:17.308 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:17.308 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 58, 2024-09-13T10:11:17.308 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:17.308 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:17.308 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:17.308 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.308 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.308 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.308 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6804", 2024-09-13T10:11:17.308 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:11:17.309 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.309 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.309 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:17.309 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.309 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.309 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.309 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6805", 2024-09-13T10:11:17.309 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:11:17.309 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.309 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.310 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:17.310 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.310 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.310 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.310 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6807", 2024-09-13T10:11:17.310 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:11:17.310 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.310 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.310 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:17.310 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.310 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.310 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.310 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6806", 2024-09-13T10:11:17.311 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:11:17.311 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.311 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.311 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6804/984313481", 2024-09-13T10:11:17.311 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6805/984313481", 2024-09-13T10:11:17.311 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6807/984313481", 2024-09-13T10:11:17.311 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6806/984313481", 2024-09-13T10:11:17.311 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:17.311 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:17.311 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:17.311 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.312 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.312 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-13T10:11:17.312 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "30d4d63c-f04f-4423-91a6-816a7cbc0287", 2024-09-13T10:11:17.312 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:17.312 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:17.312 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:17.312 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:17.312 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:17.312 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:17.312 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:17.312 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 55, 2024-09-13T10:11:17.312 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:17.312 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:17.313 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:17.313 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.313 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.313 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.313 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6812", 2024-09-13T10:11:17.313 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:11:17.313 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.313 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.313 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:17.313 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.313 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.313 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.314 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6813", 2024-09-13T10:11:17.314 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:11:17.314 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.314 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.314 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:17.314 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.314 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.314 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.314 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6815", 2024-09-13T10:11:17.314 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:11:17.314 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.315 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.315 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:17.315 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.315 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.315 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.315 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6814", 2024-09-13T10:11:17.315 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:11:17.315 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.315 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.315 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6812/1505939978", 2024-09-13T10:11:17.315 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6813/1505939978", 2024-09-13T10:11:17.315 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6815/1505939978", 2024-09-13T10:11:17.316 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6814/1505939978", 2024-09-13T10:11:17.316 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:17.316 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:17.316 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:17.316 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.316 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.316 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-13T10:11:17.316 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "48812f05-82f1-4263-8ec5-4861a6e91b61", 2024-09-13T10:11:17.316 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:17.316 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:17.316 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:17.317 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:17.317 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:17.317 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:17.317 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:17.317 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 41, 2024-09-13T10:11:17.317 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:17.317 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:17.317 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:17.317 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.317 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.317 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.317 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6809", 2024-09-13T10:11:17.317 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:11:17.317 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.318 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.318 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.318 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:17.318 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.318 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.318 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.318 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6810", 2024-09-13T10:11:17.318 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:11:17.318 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.318 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.318 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.318 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:17.318 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.319 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.319 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6812", 2024-09-13T10:11:17.319 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:11:17.319 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.319 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.319 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:17.319 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.319 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.319 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6811", 2024-09-13T10:11:17.319 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:11:17.320 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.320 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.320 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6809/3679292757", 2024-09-13T10:11:17.320 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6810/3679292757", 2024-09-13T10:11:17.320 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6812/3679292757", 2024-09-13T10:11:17.320 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6811/3679292757", 2024-09-13T10:11:17.320 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:17.320 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:17.320 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:17.320 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.321 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 8, 2024-09-13T10:11:17.321 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "72056e61-d04b-4fd3-94bf-57ee3df4b466", 2024-09-13T10:11:17.321 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:17.321 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:17.321 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:17.321 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0.396240234375, 2024-09-13T10:11:17.321 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:17.321 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:17.321 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:17.321 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 31, 2024-09-13T10:11:17.321 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:17.321 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:17.321 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:17.322 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.322 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.322 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.322 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6804", 2024-09-13T10:11:17.322 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:11:17.322 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.322 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.322 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:17.322 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.322 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.322 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.322 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6805", 2024-09-13T10:11:17.323 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:11:17.323 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.323 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.323 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:17.323 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.323 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.323 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.323 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6807", 2024-09-13T10:11:17.323 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:11:17.323 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.323 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.323 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:17.324 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.324 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.324 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.324 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6806", 2024-09-13T10:11:17.324 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:11:17.324 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.324 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.324 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6804/4052106885", 2024-09-13T10:11:17.324 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6805/4052106885", 2024-09-13T10:11:17.324 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6807/4052106885", 2024-09-13T10:11:17.324 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6806/4052106885", 2024-09-13T10:11:17.324 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:17.325 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:17.325 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:17.325 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.325 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.325 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 9, 2024-09-13T10:11:17.325 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "20e7f118-7148-4c27-9ec4-a33954709515", 2024-09-13T10:11:17.325 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:17.325 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:17.325 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:17.325 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:17.325 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:17.325 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:17.326 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 21, 2024-09-13T10:11:17.326 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 48, 2024-09-13T10:11:17.326 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:17.326 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:17.326 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:17.326 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.326 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.326 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.326 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6800", 2024-09-13T10:11:17.326 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:11:17.326 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.326 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.326 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.327 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:17.327 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.327 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.327 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6801", 2024-09-13T10:11:17.327 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:11:17.327 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.327 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.327 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:17.327 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.327 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.328 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6803", 2024-09-13T10:11:17.328 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:11:17.328 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.328 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.328 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:17.328 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.328 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.328 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.328 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6802", 2024-09-13T10:11:17.328 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:11:17.328 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.328 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.329 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6800/4266571820", 2024-09-13T10:11:17.329 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6801/4266571820", 2024-09-13T10:11:17.329 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6803/4266571820", 2024-09-13T10:11:17.329 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6802/4266571820", 2024-09-13T10:11:17.329 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:17.329 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:17.329 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:17.329 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.329 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.329 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 10, 2024-09-13T10:11:17.329 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "012409e8-51c2-4584-aba7-95996f0d72ad", 2024-09-13T10:11:17.330 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:17.330 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:17.330 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:17.330 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:17.330 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:17.330 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:17.330 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:17.330 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 30, 2024-09-13T10:11:17.330 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:17.330 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:17.330 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:17.330 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.330 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.331 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.331 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6808", 2024-09-13T10:11:17.331 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:11:17.331 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.331 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.331 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:17.331 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.331 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.331 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.331 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6809", 2024-09-13T10:11:17.331 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:11:17.332 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.332 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.332 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:17.332 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.332 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.332 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.332 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6811", 2024-09-13T10:11:17.332 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:11:17.332 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.332 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.332 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:17.333 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.333 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.333 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.333 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6810", 2024-09-13T10:11:17.333 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:11:17.333 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.333 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.333 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6808/958255259", 2024-09-13T10:11:17.333 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6809/958255259", 2024-09-13T10:11:17.333 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6811/958255259", 2024-09-13T10:11:17.333 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6810/958255259", 2024-09-13T10:11:17.334 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:17.334 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:17.334 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:17.334 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.334 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.334 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.334 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 11, 2024-09-13T10:11:17.334 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "88b137b1-7d30-45f7-ac5c-ba026704ef24", 2024-09-13T10:11:17.334 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:17.334 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:17.334 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:17.334 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:17.335 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:17.335 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:17.335 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:17.335 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 57, 2024-09-13T10:11:17.335 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:17.335 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:17.335 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:17.335 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.335 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.335 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.335 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6813", 2024-09-13T10:11:17.335 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:11:17.335 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.336 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.336 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:17.336 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.336 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.336 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.336 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6814", 2024-09-13T10:11:17.336 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:11:17.336 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.336 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.336 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:17.336 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.337 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.337 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.337 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6816", 2024-09-13T10:11:17.337 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:11:17.337 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.337 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.337 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:17.337 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.337 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.337 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.337 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6815", 2024-09-13T10:11:17.338 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:11:17.338 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.338 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.338 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6813/281884267", 2024-09-13T10:11:17.338 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6814/281884267", 2024-09-13T10:11:17.338 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6816/281884267", 2024-09-13T10:11:17.338 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6815/281884267", 2024-09-13T10:11:17.338 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:17.338 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:17.338 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:17.338 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.339 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.339 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 12, 2024-09-13T10:11:17.339 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "b54732a2-9dff-42e1-b4ef-281cbca3ad7c", 2024-09-13T10:11:17.339 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:17.339 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:17.339 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:17.339 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0, 2024-09-13T10:11:17.339 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:17.339 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:17.339 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:17.339 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 31, 2024-09-13T10:11:17.339 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:17.339 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:17.339 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:17.340 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.340 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.340 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.340 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6808", 2024-09-13T10:11:17.340 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:11:17.340 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.340 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.340 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:17.340 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.340 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.340 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.340 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6809", 2024-09-13T10:11:17.341 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:11:17.341 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.341 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.341 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:17.341 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.341 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.341 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.341 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6811", 2024-09-13T10:11:17.341 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:11:17.341 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.341 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.341 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:17.342 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.342 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.342 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.342 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6810", 2024-09-13T10:11:17.342 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:11:17.342 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.342 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.342 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.342 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6808/3532043806", 2024-09-13T10:11:17.342 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6809/3532043806", 2024-09-13T10:11:17.342 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6811/3532043806", 2024-09-13T10:11:17.342 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6810/3532043806", 2024-09-13T10:11:17.342 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:17.343 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:17.343 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:17.343 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.343 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.343 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 13, 2024-09-13T10:11:17.343 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "20b48864-44dc-4732-8dff-d0d21cc08032", 2024-09-13T10:11:17.343 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:17.343 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:17.343 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:17.343 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:17.343 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:17.343 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:17.343 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 21, 2024-09-13T10:11:17.344 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 53, 2024-09-13T10:11:17.344 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:17.344 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:17.344 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:17.344 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.344 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.344 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.344 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6812", 2024-09-13T10:11:17.344 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:11:17.344 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.344 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.344 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.344 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:17.345 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.345 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.345 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.345 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6813", 2024-09-13T10:11:17.345 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:11:17.345 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.345 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.345 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:17.345 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.345 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.345 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.345 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6815", 2024-09-13T10:11:17.346 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:11:17.346 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.346 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.346 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:17.346 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.346 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.346 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.346 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6814", 2024-09-13T10:11:17.346 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:11:17.346 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.346 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.347 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6812/4083982131", 2024-09-13T10:11:17.347 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6813/4083982131", 2024-09-13T10:11:17.347 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6815/4083982131", 2024-09-13T10:11:17.347 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6814/4083982131", 2024-09-13T10:11:17.347 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:17.347 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:17.347 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:17.347 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.347 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.347 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 14, 2024-09-13T10:11:17.347 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "9fac3ef4-e9be-4e8b-9a21-0de35b543d65", 2024-09-13T10:11:17.347 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:17.348 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:17.348 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:17.348 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:17.348 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:17.348 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:17.348 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:17.348 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 31, 2024-09-13T10:11:17.348 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:17.348 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:17.348 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:17.348 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.348 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.348 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.349 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6804", 2024-09-13T10:11:17.349 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:11:17.349 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.349 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.349 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.349 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:17.349 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.349 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.349 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.349 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6805", 2024-09-13T10:11:17.349 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:11:17.349 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.349 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.349 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.350 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:17.350 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.350 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.350 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6807", 2024-09-13T10:11:17.350 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:11:17.350 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.350 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.350 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:17.350 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.350 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.350 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6806", 2024-09-13T10:11:17.351 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:11:17.351 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.351 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.351 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6804/1173379175", 2024-09-13T10:11:17.351 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6805/1173379175", 2024-09-13T10:11:17.351 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6807/1173379175", 2024-09-13T10:11:17.351 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6806/1173379175", 2024-09-13T10:11:17.351 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:17.351 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:17.351 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:17.351 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.352 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.352 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 15, 2024-09-13T10:11:17.352 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "16b34f90-3148-4d41-a403-ba0f900991e1", 2024-09-13T10:11:17.352 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:17.352 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:17.352 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:17.352 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:17.352 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:17.352 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:17.352 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:17.352 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 44, 2024-09-13T10:11:17.353 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:17.353 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:17.353 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:17.353 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.353 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.353 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.353 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6805", 2024-09-13T10:11:17.353 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:11:17.353 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.353 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.353 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:17.353 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.354 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.354 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6806", 2024-09-13T10:11:17.354 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:11:17.354 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.354 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.354 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:17.354 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.354 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.354 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6808", 2024-09-13T10:11:17.355 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:11:17.355 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.355 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.355 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:17.355 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:17.355 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.355 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:17.355 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6807", 2024-09-13T10:11:17.355 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:11:17.355 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.355 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.356 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6805/848686366", 2024-09-13T10:11:17.356 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6806/848686366", 2024-09-13T10:11:17.356 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6808/848686366", 2024-09-13T10:11:17.356 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6807/848686366", 2024-09-13T10:11:17.356 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:17.356 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:17.356 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:17.356 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.356 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.356 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:17.356 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_xinfo": [ 2024-09-13T10:11:17.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.357 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-13T10:11:17.357 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:17.357 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:17.357 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:17.357 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:17.357 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:17.357 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.372908+0000", 2024-09-13T10:11:17.357 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:17.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.357 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.357 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-13T10:11:17.357 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:17.358 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:17.358 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:17.358 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:17.358 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:17.358 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.641263+0000", 2024-09-13T10:11:17.358 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:17.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.358 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-13T10:11:17.358 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:17.358 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:17.358 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:17.358 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:17.359 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:17.359 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.237394+0000", 2024-09-13T10:11:17.359 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:17.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.359 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-13T10:11:17.359 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:17.359 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:17.359 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:17.359 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:17.359 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:17.359 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:24.991965+0000", 2024-09-13T10:11:17.359 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:17.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.360 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.360 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-13T10:11:17.360 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:17.360 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:17.360 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:17.360 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:17.360 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:17.360 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.066350+0000", 2024-09-13T10:11:17.360 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:17.360 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.360 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.360 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-13T10:11:17.361 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:17.361 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:17.361 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:17.361 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:17.361 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:17.361 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.003317+0000", 2024-09-13T10:11:17.361 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:17.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.361 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-13T10:11:17.361 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:17.361 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:17.361 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:17.362 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:17.362 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:17.362 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.353521+0000", 2024-09-13T10:11:17.362 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:17.362 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.362 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.362 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-13T10:11:17.362 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:17.362 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:17.362 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:17.362 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:17.362 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:17.362 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.263941+0000", 2024-09-13T10:11:17.363 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:17.363 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.363 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.363 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 8, 2024-09-13T10:11:17.363 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:17.363 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:17.363 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:17.363 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:17.363 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:17.363 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:24.922976+0000", 2024-09-13T10:11:17.363 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:17.363 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.363 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.364 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 9, 2024-09-13T10:11:17.364 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:17.364 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:17.364 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:17.364 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:17.364 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:17.364 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:24.743520+0000", 2024-09-13T10:11:17.364 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:17.364 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.364 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.364 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 10, 2024-09-13T10:11:17.364 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:17.364 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:17.365 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:17.365 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:17.365 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:17.365 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.407426+0000", 2024-09-13T10:11:17.365 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:17.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.365 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.365 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 11, 2024-09-13T10:11:17.365 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:17.365 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:17.365 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:17.365 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:17.365 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:17.366 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.542266+0000", 2024-09-13T10:11:17.366 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:17.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.366 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.366 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 12, 2024-09-13T10:11:17.366 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:17.366 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:17.366 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:17.366 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:17.366 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:17.366 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.457347+0000", 2024-09-13T10:11:17.366 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:17.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.366 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.367 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 13, 2024-09-13T10:11:17.367 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:17.367 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:17.367 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:17.367 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:17.367 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:17.367 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:24.923863+0000", 2024-09-13T10:11:17.367 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:17.367 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.367 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.367 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 14, 2024-09-13T10:11:17.367 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:17.367 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:17.368 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:17.368 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:17.368 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:17.368 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:24.980077+0000", 2024-09-13T10:11:17.368 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:17.368 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.368 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.368 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 15, 2024-09-13T10:11:17.368 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:17.368 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:17.368 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:17.368 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:17.368 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:17.369 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.070162+0000", 2024-09-13T10:11:17.369 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:17.369 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.369 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:17.369 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap": [], 2024-09-13T10:11:17.369 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_items": [], 2024-09-13T10:11:17.369 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_primaries": [], 2024-09-13T10:11:17.369 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_temp": [ 2024-09-13T10:11:17.369 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.369 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.10", 2024-09-13T10:11:17.369 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:17.369 INFO:teuthology.orchestra.run.smithi073.stdout: 4, 2024-09-13T10:11:17.369 INFO:teuthology.orchestra.run.smithi073.stdout: 5, 2024-09-13T10:11:17.370 INFO:teuthology.orchestra.run.smithi073.stdout: 6 2024-09-13T10:11:17.370 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.370 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.370 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.11", 2024-09-13T10:11:17.370 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:17.370 INFO:teuthology.orchestra.run.smithi073.stdout: 2, 2024-09-13T10:11:17.370 INFO:teuthology.orchestra.run.smithi073.stdout: 12, 2024-09-13T10:11:17.370 INFO:teuthology.orchestra.run.smithi073.stdout: 8 2024-09-13T10:11:17.370 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.370 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.370 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.12", 2024-09-13T10:11:17.372 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:17.372 INFO:teuthology.orchestra.run.smithi073.stdout: 6, 2024-09-13T10:11:17.372 INFO:teuthology.orchestra.run.smithi073.stdout: 8, 2024-09-13T10:11:17.372 INFO:teuthology.orchestra.run.smithi073.stdout: 14 2024-09-13T10:11:17.372 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.372 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.372 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:17.372 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.14", 2024-09-13T10:11:17.373 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:17.373 INFO:teuthology.orchestra.run.smithi073.stdout: 5, 2024-09-13T10:11:17.373 INFO:teuthology.orchestra.run.smithi073.stdout: 6, 2024-09-13T10:11:17.373 INFO:teuthology.orchestra.run.smithi073.stdout: 10 2024-09-13T10:11:17.373 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:17.373 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.373 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:17.373 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_temp": [], 2024-09-13T10:11:17.373 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklist": {}, 2024-09-13T10:11:17.373 INFO:teuthology.orchestra.run.smithi073.stdout: "range_blocklist": {}, 2024-09-13T10:11:17.373 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profiles": { 2024-09-13T10:11:17.373 INFO:teuthology.orchestra.run.smithi073.stdout: "default": { 2024-09-13T10:11:17.373 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-failure-domain": "osd", 2024-09-13T10:11:17.374 INFO:teuthology.orchestra.run.smithi073.stdout: "k": "2", 2024-09-13T10:11:17.374 INFO:teuthology.orchestra.run.smithi073.stdout: "m": "1", 2024-09-13T10:11:17.374 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin": "jerasure", 2024-09-13T10:11:17.374 INFO:teuthology.orchestra.run.smithi073.stdout: "technique": "reed_sol_van" 2024-09-13T10:11:17.374 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.374 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:17.374 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps_queue": [], 2024-09-13T10:11:17.374 INFO:teuthology.orchestra.run.smithi073.stdout: "new_removed_snaps": [], 2024-09-13T10:11:17.374 INFO:teuthology.orchestra.run.smithi073.stdout: "new_purged_snaps": [], 2024-09-13T10:11:17.374 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_node_flags": {}, 2024-09-13T10:11:17.374 INFO:teuthology.orchestra.run.smithi073.stdout: "device_class_flags": {}, 2024-09-13T10:11:17.374 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode": { 2024-09-13T10:11:17.374 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_enabled": false, 2024-09-13T10:11:17.374 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_bucket_count": 0, 2024-09-13T10:11:17.375 INFO:teuthology.orchestra.run.smithi073.stdout: "degraded_stretch_mode": 0, 2024-09-13T10:11:17.375 INFO:teuthology.orchestra.run.smithi073.stdout: "recovering_stretch_mode": 0, 2024-09-13T10:11:17.375 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_bucket": 0 2024-09-13T10:11:17.375 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:17.375 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:17.376 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:17.378 INFO:tasks.thrashosds.thrasher:j is {'epoch': 59, 'fsid': 'c1e91d47-9492-4461-a043-8698fd74ccf3', 'created': '2024-09-13T10:10:14.019912+0000', 'modified': '2024-09-13T10:11:15.734826+0000', 'last_up_change': '2024-09-13T10:10:28.089885+0000', 'last_in_change': '2024-09-13T10:10:18.489546+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T10:10:28.924860+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 27, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.7699999809265137, 'score_stable': 3.7699999809265137, 'optimal_score': 0.7300000190734863, 'raw_score_acting': 2.75, 'raw_score_stable': 2.75, 'primary_affinity_weighted': 0.8997650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T10:10:28.972519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 15, 'score_stable': 15, 'optimal_score': 0.20000000298023224, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.8997650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 3, 'pool_name': 'low_tier', 'create_time': '2024-09-13T10:10:41.320719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.75, 'score_stable': 3.75, 'optimal_score': 0.6000000238418579, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8997650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 4, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-13T10:10:42.575395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 21, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '41', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'dedup_cdc_chunk_size': 131072, 'dedup_chunk_algorithm': 'fastcdc', 'dedup_tier': 3, 'fingerprint_algorithm': 'sha256'}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.03000020980835, 'score_stable': 3.450000047683716, 'optimal_score': 1.0700000524520874, 'raw_score_acting': 4.309999942779541, 'raw_score_stable': 3.690000057220459, 'primary_affinity_weighted': 0.8997650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8999999761581421}}], 'osds': [{'osd': 0, 'uuid': 'f7d92188-c27c-4744-a330-4b7105e0f2d6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6812', 'nonce': 1977392321}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6813', 'nonce': 1977392321}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6815', 'nonce': 1977392321}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6814', 'nonce': 1977392321}]}, 'public_addr': '172.21.15.73:6812/1977392321', 'cluster_addr': '172.21.15.73:6813/1977392321', 'heartbeat_back_addr': '172.21.15.73:6815/1977392321', 'heartbeat_front_addr': '172.21.15.73:6814/1977392321', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e900b771-7c96-4b5d-93d7-35a8ea6c026a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6808', 'nonce': 1228102864}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6809', 'nonce': 1228102864}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6811', 'nonce': 1228102864}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6810', 'nonce': 1228102864}]}, 'public_addr': '172.21.15.136:6808/1228102864', 'cluster_addr': '172.21.15.136:6809/1228102864', 'heartbeat_back_addr': '172.21.15.136:6811/1228102864', 'heartbeat_front_addr': '172.21.15.136:6810/1228102864', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '28b7aa59-33b2-4e24-8f3d-758f90e25f68', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6800', 'nonce': 3314820936}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6801', 'nonce': 3314820936}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6803', 'nonce': 3314820936}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6802', 'nonce': 3314820936}]}, 'public_addr': '172.21.15.181:6800/3314820936', 'cluster_addr': '172.21.15.181:6801/3314820936', 'heartbeat_back_addr': '172.21.15.181:6803/3314820936', 'heartbeat_front_addr': '172.21.15.181:6802/3314820936', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a93d2df5-781e-49e1-a507-1718b6f954ed', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6801', 'nonce': 2558565460}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6802', 'nonce': 2558565460}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6804', 'nonce': 2558565460}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6803', 'nonce': 2558565460}]}, 'public_addr': '172.21.15.195:6801/2558565460', 'cluster_addr': '172.21.15.195:6802/2558565460', 'heartbeat_back_addr': '172.21.15.195:6804/2558565460', 'heartbeat_front_addr': '172.21.15.195:6803/2558565460', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e69053fd-f6d9-437e-a9d0-8896de6b927f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6800', 'nonce': 858330493}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6801', 'nonce': 858330493}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6803', 'nonce': 858330493}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6802', 'nonce': 858330493}]}, 'public_addr': '172.21.15.73:6800/858330493', 'cluster_addr': '172.21.15.73:6801/858330493', 'heartbeat_back_addr': '172.21.15.73:6803/858330493', 'heartbeat_front_addr': '172.21.15.73:6802/858330493', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '3667ec10-561e-4a23-8197-6357db4af315', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6804', 'nonce': 984313481}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6805', 'nonce': 984313481}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6807', 'nonce': 984313481}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6806', 'nonce': 984313481}]}, 'public_addr': '172.21.15.136:6804/984313481', 'cluster_addr': '172.21.15.136:6805/984313481', 'heartbeat_back_addr': '172.21.15.136:6807/984313481', 'heartbeat_front_addr': '172.21.15.136:6806/984313481', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '30d4d63c-f04f-4423-91a6-816a7cbc0287', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6812', 'nonce': 1505939978}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6813', 'nonce': 1505939978}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6815', 'nonce': 1505939978}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6814', 'nonce': 1505939978}]}, 'public_addr': '172.21.15.181:6812/1505939978', 'cluster_addr': '172.21.15.181:6813/1505939978', 'heartbeat_back_addr': '172.21.15.181:6815/1505939978', 'heartbeat_front_addr': '172.21.15.181:6814/1505939978', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '48812f05-82f1-4263-8ec5-4861a6e91b61', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6809', 'nonce': 3679292757}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6810', 'nonce': 3679292757}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6812', 'nonce': 3679292757}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6811', 'nonce': 3679292757}]}, 'public_addr': '172.21.15.195:6809/3679292757', 'cluster_addr': '172.21.15.195:6810/3679292757', 'heartbeat_back_addr': '172.21.15.195:6812/3679292757', 'heartbeat_front_addr': '172.21.15.195:6811/3679292757', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '72056e61-d04b-4fd3-94bf-57ee3df4b466', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.396240234375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6804', 'nonce': 4052106885}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6805', 'nonce': 4052106885}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6807', 'nonce': 4052106885}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6806', 'nonce': 4052106885}]}, 'public_addr': '172.21.15.73:6804/4052106885', 'cluster_addr': '172.21.15.73:6805/4052106885', 'heartbeat_back_addr': '172.21.15.73:6807/4052106885', 'heartbeat_front_addr': '172.21.15.73:6806/4052106885', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '20e7f118-7148-4c27-9ec4-a33954709515', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6800', 'nonce': 4266571820}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6801', 'nonce': 4266571820}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6803', 'nonce': 4266571820}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6802', 'nonce': 4266571820}]}, 'public_addr': '172.21.15.136:6800/4266571820', 'cluster_addr': '172.21.15.136:6801/4266571820', 'heartbeat_back_addr': '172.21.15.136:6803/4266571820', 'heartbeat_front_addr': '172.21.15.136:6802/4266571820', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '012409e8-51c2-4584-aba7-95996f0d72ad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6808', 'nonce': 958255259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6809', 'nonce': 958255259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6811', 'nonce': 958255259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6810', 'nonce': 958255259}]}, 'public_addr': '172.21.15.181:6808/958255259', 'cluster_addr': '172.21.15.181:6809/958255259', 'heartbeat_back_addr': '172.21.15.181:6811/958255259', 'heartbeat_front_addr': '172.21.15.181:6810/958255259', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '88b137b1-7d30-45f7-ac5c-ba026704ef24', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6813', 'nonce': 281884267}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6814', 'nonce': 281884267}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6816', 'nonce': 281884267}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6815', 'nonce': 281884267}]}, 'public_addr': '172.21.15.195:6813/281884267', 'cluster_addr': '172.21.15.195:6814/281884267', 'heartbeat_back_addr': '172.21.15.195:6816/281884267', 'heartbeat_front_addr': '172.21.15.195:6815/281884267', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': 'b54732a2-9dff-42e1-b4ef-281cbca3ad7c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6808', 'nonce': 3532043806}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6809', 'nonce': 3532043806}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6811', 'nonce': 3532043806}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6810', 'nonce': 3532043806}]}, 'public_addr': '172.21.15.73:6808/3532043806', 'cluster_addr': '172.21.15.73:6809/3532043806', 'heartbeat_back_addr': '172.21.15.73:6811/3532043806', 'heartbeat_front_addr': '172.21.15.73:6810/3532043806', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '20b48864-44dc-4732-8dff-d0d21cc08032', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6812', 'nonce': 4083982131}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6813', 'nonce': 4083982131}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6815', 'nonce': 4083982131}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6814', 'nonce': 4083982131}]}, 'public_addr': '172.21.15.136:6812/4083982131', 'cluster_addr': '172.21.15.136:6813/4083982131', 'heartbeat_back_addr': '172.21.15.136:6815/4083982131', 'heartbeat_front_addr': '172.21.15.136:6814/4083982131', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '9fac3ef4-e9be-4e8b-9a21-0de35b543d65', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6804', 'nonce': 1173379175}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6805', 'nonce': 1173379175}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6807', 'nonce': 1173379175}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6806', 'nonce': 1173379175}]}, 'public_addr': '172.21.15.181:6804/1173379175', 'cluster_addr': '172.21.15.181:6805/1173379175', 'heartbeat_back_addr': '172.21.15.181:6807/1173379175', 'heartbeat_front_addr': '172.21.15.181:6806/1173379175', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '16b34f90-3148-4d41-a403-ba0f900991e1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6805', 'nonce': 848686366}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6806', 'nonce': 848686366}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6808', 'nonce': 848686366}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6807', 'nonce': 848686366}]}, 'public_addr': '172.21.15.195:6805/848686366', 'cluster_addr': '172.21.15.195:6806/848686366', 'heartbeat_back_addr': '172.21.15.195:6808/848686366', 'heartbeat_front_addr': '172.21.15.195:6807/848686366', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.372908+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.641263+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.237394+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:24.991965+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.066350+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.003317+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.353521+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.263941+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:24.922976+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:24.743520+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.407426+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.542266+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.457347+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:24.923863+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:24.980077+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.070162+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '4.10', 'osds': [4, 5, 6]}, {'pgid': '4.11', 'osds': [2, 12, 8]}, {'pgid': '4.12', 'osds': [6, 8, 14]}, {'pgid': '4.14', 'osds': [5, 6, 10]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T10:11:17.379 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:11:17.379 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:17.380 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:11:17.380 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:11:17.380 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:11:17.380 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.380 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4365.0:5 1.2 1:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e26)", 2024-09-13T10:11:17.380 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:31.135075+0000", 2024-09-13T10:11:17.380 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 46.198734985999998, 2024-09-13T10:11:17.380 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010617898000000001, 2024-09-13T10:11:17.380 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.380 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.380 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:17.380 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.380 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4365", 2024-09-13T10:11:17.381 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.73:0/186187721", 2024-09-13T10:11:17.381 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 5 2024-09-13T10:11:17.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.381 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.381 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.381 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.381 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135075+0000", 2024-09-13T10:11:17.381 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.381 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.381 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.381 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135075+0000", 2024-09-13T10:11:17.381 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.382 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.382 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.382 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.382 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135089+0000", 2024-09-13T10:11:17.382 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3512e-05 2024-09-13T10:11:17.382 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.382 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.382 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.382 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135113+0000", 2024-09-13T10:11:17.382 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4006e-05 2024-09-13T10:11:17.382 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.382 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.382 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.383 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135150+0000", 2024-09-13T10:11:17.383 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6838999999999997e-05 2024-09-13T10:11:17.383 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.383 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.383 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.383 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135169+0000", 2024-09-13T10:11:17.383 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9499000000000001e-05 2024-09-13T10:11:17.383 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.383 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.383 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.383 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135393+0000", 2024-09-13T10:11:17.383 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000224081 2024-09-13T10:11:17.383 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.384 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.384 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op must wait for map", 2024-09-13T10:11:17.384 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.135412+0000", 2024-09-13T10:11:17.384 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8989000000000001e-05 2024-09-13T10:11:17.384 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.384 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.384 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.384 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.138668+0000", 2024-09-13T10:11:17.384 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0032558909999999999 2024-09-13T10:11:17.384 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.384 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.384 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.384 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.138896+0000", 2024-09-13T10:11:17.385 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022737699999999999 2024-09-13T10:11:17.385 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.385 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.385 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 6,9", 2024-09-13T10:11:17.385 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.139159+0000", 2024-09-13T10:11:17.385 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026282399999999999 2024-09-13T10:11:17.385 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.385 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.385 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:17.385 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.143646+0000", 2024-09-13T10:11:17.385 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044877249999999997 2024-09-13T10:11:17.385 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.385 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.386 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.386 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.144479+0000", 2024-09-13T10:11:17.386 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00083277000000000004 2024-09-13T10:11:17.386 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.386 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.386 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.386 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145487+0000", 2024-09-13T10:11:17.386 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001008246 2024-09-13T10:11:17.386 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.386 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.386 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:17.386 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145571+0000", 2024-09-13T10:11:17.386 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.4040000000000002e-05 2024-09-13T10:11:17.387 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.387 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.387 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.387 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:31.145693+0000", 2024-09-13T10:11:17.387 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000122099 2024-09-13T10:11:17.387 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.387 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.387 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.387 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.387 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.387 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:37 4.1 4:85f1e656:::smithi19530714-10:head [write 551263~793594,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:17.387 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.445403+0000", 2024-09-13T10:11:17.387 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.888407014999999, 2024-09-13T10:11:17.388 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010589464, 2024-09-13T10:11:17.388 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.388 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.388 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:17.388 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.388 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.388 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.388 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 37 2024-09-13T10:11:17.388 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.388 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.388 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.388 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.388 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.445403+0000", 2024-09-13T10:11:17.389 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.389 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.389 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.389 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.389 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.445403+0000", 2024-09-13T10:11:17.389 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.389 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.389 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.389 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.389 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.445406+0000", 2024-09-13T10:11:17.389 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.993e-06 2024-09-13T10:11:17.389 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.389 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.390 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.390 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.446815+0000", 2024-09-13T10:11:17.390 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001408687 2024-09-13T10:11:17.390 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.390 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.390 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.390 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.446840+0000", 2024-09-13T10:11:17.390 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4426e-05 2024-09-13T10:11:17.390 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.390 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.390 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.390 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.446856+0000", 2024-09-13T10:11:17.390 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6073e-05 2024-09-13T10:11:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.446986+0000", 2024-09-13T10:11:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000130746 2024-09-13T10:11:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.447206+0000", 2024-09-13T10:11:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021936199999999999 2024-09-13T10:11:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:17.391 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.447450+0000", 2024-09-13T10:11:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024457999999999999 2024-09-13T10:11:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.455772+0000", 2024-09-13T10:11:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.008322009 2024-09-13T10:11:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.455796+0000", 2024-09-13T10:11:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4110000000000001e-05 2024-09-13T10:11:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.392 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.455846+0000", 2024-09-13T10:11:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.9672000000000001e-05 2024-09-13T10:11:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.455927+0000", 2024-09-13T10:11:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.0927000000000006e-05 2024-09-13T10:11:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.455993+0000", 2024-09-13T10:11:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.5878999999999997e-05 2024-09-13T10:11:17.393 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:39 4.1 4:85f1e656:::smithi19530714-10:head [setxattr _header (56) in=7b,truncate 2151263] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.449473+0000", 2024-09-13T10:11:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.884337943999999, 2024-09-13T10:11:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011271674000000001, 2024-09-13T10:11:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.394 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 39 2024-09-13T10:11:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449473+0000", 2024-09-13T10:11:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449473+0000", 2024-09-13T10:11:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.395 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449474+0000", 2024-09-13T10:11:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5740000000000001e-06 2024-09-13T10:11:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449486+0000", 2024-09-13T10:11:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1545999999999999e-05 2024-09-13T10:11:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.396 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449504+0000", 2024-09-13T10:11:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8502999999999999e-05 2024-09-13T10:11:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449517+0000", 2024-09-13T10:11:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2753e-05 2024-09-13T10:11:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.454948+0000", 2024-09-13T10:11:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0054313540000000002 2024-09-13T10:11:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.397 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.455084+0000", 2024-09-13T10:11:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013585 2024-09-13T10:11:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.455296+0000", 2024-09-13T10:11:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021145100000000001 2024-09-13T10:11:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.456431+0000", 2024-09-13T10:11:17.398 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011352210000000001 2024-09-13T10:11:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.456490+0000", 2024-09-13T10:11:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9457999999999999e-05 2024-09-13T10:11:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.460547+0000", 2024-09-13T10:11:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0040571690000000002 2024-09-13T10:11:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:17.399 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.460663+0000", 2024-09-13T10:11:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000115691 2024-09-13T10:11:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.460744+0000", 2024-09-13T10:11:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.1104000000000001e-05 2024-09-13T10:11:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:40 4.1 4:85f1e656:::smithi19530714-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:17.400 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.449537+0000", 2024-09-13T10:11:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.884273298, 2024-09-13T10:11:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011651083, 2024-09-13T10:11:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 40 2024-09-13T10:11:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.401 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449537+0000", 2024-09-13T10:11:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449537+0000", 2024-09-13T10:11:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449538+0000", 2024-09-13T10:11:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0070000000000001e-06 2024-09-13T10:11:17.402 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449543+0000", 2024-09-13T10:11:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.3560000000000003e-06 2024-09-13T10:11:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449554+0000", 2024-09-13T10:11:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1263e-05 2024-09-13T10:11:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.403 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449566+0000", 2024-09-13T10:11:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2383e-05 2024-09-13T10:11:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.456239+0000", 2024-09-13T10:11:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0066730080000000002 2024-09-13T10:11:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.404 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.456350+0000", 2024-09-13T10:11:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000111214 2024-09-13T10:11:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.460834+0000", 2024-09-13T10:11:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044834080000000004 2024-09-13T10:11:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.460956+0000", 2024-09-13T10:11:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000121934 2024-09-13T10:11:17.405 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.461188+0000", 2024-09-13T10:11:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023251000000000001 2024-09-13T10:11:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:259 4.1 4:85f1e656:::smithi19530714-10:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:11:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.118643879, 2024-09-13T10:11:17.406 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014693513, 2024-09-13T10:11:17.407 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.407 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.407 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:17.407 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.407 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.407 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.407 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 259 2024-09-13T10:11:17.407 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.407 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.407 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.407 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.407 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:11:17.407 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.407 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.408 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.408 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.408 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:11:17.408 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.408 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.408 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.408 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.408 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215171+0000", 2024-09-13T10:11:17.408 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.3329999999999998e-06 2024-09-13T10:11:17.408 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.408 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.408 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.408 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215190+0000", 2024-09-13T10:11:17.409 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8672999999999999e-05 2024-09-13T10:11:17.409 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.409 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.409 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.409 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215229+0000", 2024-09-13T10:11:17.409 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9076000000000003e-05 2024-09-13T10:11:17.409 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.409 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.409 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.409 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215287+0000", 2024-09-13T10:11:17.409 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8142e-05 2024-09-13T10:11:17.409 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.409 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.410 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.410 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215435+0000", 2024-09-13T10:11:17.410 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014800600000000001 2024-09-13T10:11:17.410 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.410 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.410 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.410 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215593+0000", 2024-09-13T10:11:17.410 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015864300000000001 2024-09-13T10:11:17.410 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.410 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.410 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:17.410 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226037+0000", 2024-09-13T10:11:17.410 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010443948 2024-09-13T10:11:17.411 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.411 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.411 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:17.411 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228312+0000", 2024-09-13T10:11:17.411 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022743009999999998 2024-09-13T10:11:17.411 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.411 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.411 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.411 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228708+0000", 2024-09-13T10:11:17.411 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039651999999999998 2024-09-13T10:11:17.411 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.411 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.411 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.411 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229625+0000", 2024-09-13T10:11:17.412 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00091692300000000004 2024-09-13T10:11:17.412 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.412 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.412 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:17.412 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229750+0000", 2024-09-13T10:11:17.412 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012532099999999999 2024-09-13T10:11:17.412 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.412 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.412 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.412 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229860+0000", 2024-09-13T10:11:17.412 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000109627 2024-09-13T10:11:17.412 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.412 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.413 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.413 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.413 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.413 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:260 4.3 4:c426503f:::smithi19530714-11:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:17.413 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:11:17.413 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.118475920000002, 2024-09-13T10:11:17.413 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041961871999999997, 2024-09-13T10:11:17.413 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.413 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.413 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:17.413 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.413 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.413 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.414 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 260 2024-09-13T10:11:17.414 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.414 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.414 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.414 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.414 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:11:17.414 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.414 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.414 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.414 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.414 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:11:17.414 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.414 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.415 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.415 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.415 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215336+0000", 2024-09-13T10:11:17.415 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9069999999999999e-06 2024-09-13T10:11:17.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.415 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.415 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.415 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215345+0000", 2024-09-13T10:11:17.415 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.6659999999999997e-06 2024-09-13T10:11:17.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.415 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.415 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.415 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215363+0000", 2024-09-13T10:11:17.415 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8153000000000001e-05 2024-09-13T10:11:17.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.416 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.416 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.416 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215388+0000", 2024-09-13T10:11:17.416 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4556e-05 2024-09-13T10:11:17.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.416 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.416 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.416 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215542+0000", 2024-09-13T10:11:17.416 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000154637 2024-09-13T10:11:17.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.416 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.417 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.417 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215825+0000", 2024-09-13T10:11:17.417 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028208600000000002 2024-09-13T10:11:17.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.417 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.417 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:17.417 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254572+0000", 2024-09-13T10:11:17.417 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.038747464000000002 2024-09-13T10:11:17.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.417 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.417 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:17.417 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.255694+0000", 2024-09-13T10:11:17.417 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011222929999999999 2024-09-13T10:11:17.418 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.418 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.418 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.418 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.256931+0000", 2024-09-13T10:11:17.418 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0012370980000000001 2024-09-13T10:11:17.418 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.418 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.418 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.418 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257080+0000", 2024-09-13T10:11:17.418 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014874800000000001 2024-09-13T10:11:17.418 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.418 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.418 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:17.418 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257211+0000", 2024-09-13T10:11:17.419 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000130674 2024-09-13T10:11:17.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.419 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.419 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.419 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257296+0000", 2024-09-13T10:11:17.419 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5589999999999999e-05 2024-09-13T10:11:17.419 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.419 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.419 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.419 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.419 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:269 4.3 4:c5b5ff75:::smithi19530714-20:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:17.419 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:11:17.420 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.117971682, 2024-09-13T10:11:17.420 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014988243, 2024-09-13T10:11:17.420 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.420 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.420 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:17.420 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.420 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.420 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.420 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 269 2024-09-13T10:11:17.420 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.420 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.420 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.420 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.421 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:11:17.421 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.421 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.421 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.421 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.421 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:11:17.421 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.421 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.421 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.421 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.421 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215841+0000", 2024-09-13T10:11:17.421 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0829999999999998e-06 2024-09-13T10:11:17.421 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.422 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.422 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.422 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215854+0000", 2024-09-13T10:11:17.422 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2976000000000001e-05 2024-09-13T10:11:17.422 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.422 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.422 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.422 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215875+0000", 2024-09-13T10:11:17.422 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1100000000000001e-05 2024-09-13T10:11:17.422 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.422 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.422 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.422 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215892+0000", 2024-09-13T10:11:17.423 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7388999999999999e-05 2024-09-13T10:11:17.423 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.423 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.423 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.423 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216353+0000", 2024-09-13T10:11:17.423 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00046100900000000002 2024-09-13T10:11:17.423 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.423 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.423 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.423 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216611+0000", 2024-09-13T10:11:17.423 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025716299999999998 2024-09-13T10:11:17.423 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.424 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.424 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:17.424 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.227177+0000", 2024-09-13T10:11:17.424 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010566676000000001 2024-09-13T10:11:17.424 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.424 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.424 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:17.424 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229574+0000", 2024-09-13T10:11:17.424 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023964469999999999 2024-09-13T10:11:17.424 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.424 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.424 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.424 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230619+0000", 2024-09-13T10:11:17.425 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0010455950000000001 2024-09-13T10:11:17.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.425 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.425 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.425 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230697+0000", 2024-09-13T10:11:17.425 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8021000000000004e-05 2024-09-13T10:11:17.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.425 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.425 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:17.425 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230767+0000", 2024-09-13T10:11:17.425 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9544999999999999e-05 2024-09-13T10:11:17.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.425 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.426 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.426 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230827+0000", 2024-09-13T10:11:17.426 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.0238999999999997e-05 2024-09-13T10:11:17.426 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.426 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.426 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.426 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.426 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.426 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:288 4.3 4:cceea2a7:::smithi19530714-39:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:17.426 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.249445+0000", 2024-09-13T10:11:17.426 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.084365390999999, 2024-09-13T10:11:17.426 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012024603, 2024-09-13T10:11:17.426 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.427 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.427 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:17.427 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.427 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.427 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.427 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 288 2024-09-13T10:11:17.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.427 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.427 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.427 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.427 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249445+0000", 2024-09-13T10:11:17.427 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.428 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.428 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.428 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249445+0000", 2024-09-13T10:11:17.428 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.428 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.428 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.428 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249448+0000", 2024-09-13T10:11:17.428 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0350000000000002e-06 2024-09-13T10:11:17.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.428 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.428 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.428 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249460+0000", 2024-09-13T10:11:17.429 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2214e-05 2024-09-13T10:11:17.429 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.429 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.429 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.429 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249490+0000", 2024-09-13T10:11:17.429 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0130000000000001e-05 2024-09-13T10:11:17.429 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.429 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.429 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.429 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249510+0000", 2024-09-13T10:11:17.429 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9757000000000001e-05 2024-09-13T10:11:17.429 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.429 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.430 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.430 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249684+0000", 2024-09-13T10:11:17.430 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000174086 2024-09-13T10:11:17.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.430 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.430 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.430 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249913+0000", 2024-09-13T10:11:17.430 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022833400000000001 2024-09-13T10:11:17.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.430 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.430 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:17.430 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257665+0000", 2024-09-13T10:11:17.430 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0077526330000000001 2024-09-13T10:11:17.431 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.431 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.431 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.431 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.259621+0000", 2024-09-13T10:11:17.431 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019558610000000001 2024-09-13T10:11:17.431 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.431 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.431 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.431 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260048+0000", 2024-09-13T10:11:17.431 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00042668799999999999 2024-09-13T10:11:17.431 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.431 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.431 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:17.432 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.261264+0000", 2024-09-13T10:11:17.432 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001216445 2024-09-13T10:11:17.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.432 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.432 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:17.432 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.261374+0000", 2024-09-13T10:11:17.432 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110117 2024-09-13T10:11:17.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.432 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.432 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.432 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.261470+0000", 2024-09-13T10:11:17.432 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5303000000000006e-05 2024-09-13T10:11:17.432 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.433 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.433 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.433 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.433 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:291 4.1 4:8e413aa3:::smithi19530714-42:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:17.433 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.257868+0000", 2024-09-13T10:11:17.433 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.075942613999999, 2024-09-13T10:11:17.433 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010494850999999999, 2024-09-13T10:11:17.433 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.433 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.433 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:17.433 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.433 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.434 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.434 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 291 2024-09-13T10:11:17.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.434 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.434 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.434 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.434 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257868+0000", 2024-09-13T10:11:17.434 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.434 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.434 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.434 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257868+0000", 2024-09-13T10:11:17.434 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.435 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.435 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.435 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257870+0000", 2024-09-13T10:11:17.435 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3800000000000001e-06 2024-09-13T10:11:17.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.435 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.435 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.435 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257882+0000", 2024-09-13T10:11:17.435 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1522000000000001e-05 2024-09-13T10:11:17.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.435 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.435 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.436 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257901+0000", 2024-09-13T10:11:17.436 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9672000000000001e-05 2024-09-13T10:11:17.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.436 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.436 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.436 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257918+0000", 2024-09-13T10:11:17.436 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6407000000000001e-05 2024-09-13T10:11:17.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.436 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.436 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.436 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.258034+0000", 2024-09-13T10:11:17.436 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011616399999999999 2024-09-13T10:11:17.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.437 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.437 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.437 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.258174+0000", 2024-09-13T10:11:17.437 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014034 2024-09-13T10:11:17.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.437 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.437 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:17.437 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.265104+0000", 2024-09-13T10:11:17.437 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0069297889999999996 2024-09-13T10:11:17.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.437 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.437 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:17.438 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.266805+0000", 2024-09-13T10:11:17.438 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001700685 2024-09-13T10:11:17.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.438 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.438 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.438 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267613+0000", 2024-09-13T10:11:17.438 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00080784300000000004 2024-09-13T10:11:17.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.438 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.438 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.438 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.268164+0000", 2024-09-13T10:11:17.438 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00055089600000000005 2024-09-13T10:11:17.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.439 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.439 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:17.439 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.268274+0000", 2024-09-13T10:11:17.439 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110927 2024-09-13T10:11:17.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.439 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.439 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.439 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.268363+0000", 2024-09-13T10:11:17.439 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8226000000000005e-05 2024-09-13T10:11:17.439 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.439 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.439 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.439 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.440 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:310 4.3 4:c426503f:::smithi19530714-11:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:17.440 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:11:17.440 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.060675609, 2024-09-13T10:11:17.440 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013782279999999999, 2024-09-13T10:11:17.440 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.440 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.440 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:17.440 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.440 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.440 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.440 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 310 2024-09-13T10:11:17.440 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.441 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.441 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.441 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.441 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:11:17.441 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.441 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.441 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.441 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.441 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:11:17.441 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.441 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.441 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.441 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.441 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273138+0000", 2024-09-13T10:11:17.442 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8459999999999999e-06 2024-09-13T10:11:17.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.442 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.442 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.442 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273150+0000", 2024-09-13T10:11:17.442 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2133999999999999e-05 2024-09-13T10:11:17.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.442 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.442 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.442 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273175+0000", 2024-09-13T10:11:17.442 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5321999999999999e-05 2024-09-13T10:11:17.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.442 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.443 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.443 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273192+0000", 2024-09-13T10:11:17.443 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.681e-05 2024-09-13T10:11:17.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.443 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.443 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.443 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273314+0000", 2024-09-13T10:11:17.443 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012228 2024-09-13T10:11:17.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.443 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.443 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.443 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273474+0000", 2024-09-13T10:11:17.443 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016019799999999999 2024-09-13T10:11:17.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.444 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.444 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:17.444 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.284136+0000", 2024-09-13T10:11:17.444 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010661761000000001 2024-09-13T10:11:17.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.444 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.444 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.444 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286085+0000", 2024-09-13T10:11:17.444 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019484929999999999 2024-09-13T10:11:17.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.444 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.444 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:17.445 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286111+0000", 2024-09-13T10:11:17.445 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5834e-05 2024-09-13T10:11:17.445 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.445 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.445 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.445 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286721+0000", 2024-09-13T10:11:17.445 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00061038599999999998 2024-09-13T10:11:17.445 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.445 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.445 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:17.445 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286838+0000", 2024-09-13T10:11:17.445 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117064 2024-09-13T10:11:17.445 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.446 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.446 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.446 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286917+0000", 2024-09-13T10:11:17.446 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9152000000000004e-05 2024-09-13T10:11:17.446 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.446 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.446 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.446 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.446 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.446 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:360 4.3 4:c426503f:::smithi19530714-11:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:17.446 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.294403+0000", 2024-09-13T10:11:17.446 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.039407884999999, 2024-09-13T10:11:17.446 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010710917, 2024-09-13T10:11:17.446 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.447 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.447 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:17.447 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.447 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.447 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.447 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 360 2024-09-13T10:11:17.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.447 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.447 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.447 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.447 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294403+0000", 2024-09-13T10:11:17.447 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.447 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.448 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.448 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294403+0000", 2024-09-13T10:11:17.448 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.448 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.448 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.448 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294404+0000", 2024-09-13T10:11:17.448 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.257e-06 2024-09-13T10:11:17.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.448 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.448 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.448 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294413+0000", 2024-09-13T10:11:17.448 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7830000000000004e-06 2024-09-13T10:11:17.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.449 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.449 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.449 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294436+0000", 2024-09-13T10:11:17.449 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3468e-05 2024-09-13T10:11:17.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.449 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.449 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.449 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294447+0000", 2024-09-13T10:11:17.449 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0908e-05 2024-09-13T10:11:17.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.449 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.449 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.450 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294521+0000", 2024-09-13T10:11:17.450 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.4468999999999995e-05 2024-09-13T10:11:17.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.450 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.450 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op must wait for map", 2024-09-13T10:11:17.450 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294528+0000", 2024-09-13T10:11:17.450 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.2750000000000001e-06 2024-09-13T10:11:17.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.450 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.450 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.450 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.297057+0000", 2024-09-13T10:11:17.450 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025294499999999999 2024-09-13T10:11:17.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.451 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.451 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.451 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.297196+0000", 2024-09-13T10:11:17.451 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000138646 2024-09-13T10:11:17.451 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.451 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.451 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:17.451 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.303520+0000", 2024-09-13T10:11:17.451 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00632381 2024-09-13T10:11:17.451 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.451 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.451 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:17.451 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.304331+0000", 2024-09-13T10:11:17.451 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00081096399999999996 2024-09-13T10:11:17.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.452 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.452 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.452 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.304707+0000", 2024-09-13T10:11:17.452 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037599399999999999 2024-09-13T10:11:17.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.452 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.452 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.452 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.305024+0000", 2024-09-13T10:11:17.452 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00031715999999999998 2024-09-13T10:11:17.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.452 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.452 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:17.453 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.305073+0000", 2024-09-13T10:11:17.453 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.9635000000000002e-05 2024-09-13T10:11:17.453 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.453 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.453 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.453 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.305114+0000", 2024-09-13T10:11:17.453 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0098000000000001e-05 2024-09-13T10:11:17.453 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.453 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.453 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.453 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.453 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.453 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:391 4.1 4:8e413aa3:::smithi19530714-42:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:17.454 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.321223+0000", 2024-09-13T10:11:17.454 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.012587602, 2024-09-13T10:11:17.454 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011654998999999999, 2024-09-13T10:11:17.454 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.454 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.454 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:17.454 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.454 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.454 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.454 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 391 2024-09-13T10:11:17.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.454 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.454 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.454 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.455 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321223+0000", 2024-09-13T10:11:17.455 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.455 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.455 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.455 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321223+0000", 2024-09-13T10:11:17.455 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.455 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.455 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.455 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321225+0000", 2024-09-13T10:11:17.455 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0360000000000001e-06 2024-09-13T10:11:17.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.456 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.456 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.456 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321236+0000", 2024-09-13T10:11:17.456 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1419e-05 2024-09-13T10:11:17.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.456 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.456 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.456 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321262+0000", 2024-09-13T10:11:17.456 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6035000000000001e-05 2024-09-13T10:11:17.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.456 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.456 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.457 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321280+0000", 2024-09-13T10:11:17.457 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8022000000000001e-05 2024-09-13T10:11:17.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.457 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.457 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.457 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321388+0000", 2024-09-13T10:11:17.457 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010752900000000001 2024-09-13T10:11:17.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.457 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.457 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.457 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321518+0000", 2024-09-13T10:11:17.457 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000129901 2024-09-13T10:11:17.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.457 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.458 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:17.458 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.329954+0000", 2024-09-13T10:11:17.458 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0084364639999999994 2024-09-13T10:11:17.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.458 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.458 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.458 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.331173+0000", 2024-09-13T10:11:17.458 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001218968 2024-09-13T10:11:17.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.458 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.458 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:17.458 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.331209+0000", 2024-09-13T10:11:17.458 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6018e-05 2024-09-13T10:11:17.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.459 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.459 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.459 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.332764+0000", 2024-09-13T10:11:17.459 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001554466 2024-09-13T10:11:17.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.459 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.459 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:17.459 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.332838+0000", 2024-09-13T10:11:17.459 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.3779e-05 2024-09-13T10:11:17.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.459 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.459 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.332878+0000", 2024-09-13T10:11:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0361999999999997e-05 2024-09-13T10:11:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:993 4.3 4:c5b5ff75:::smithi19530714-20:head [read 0~2319333,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.181393+0000", 2024-09-13T10:11:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 22.152417716999999, 2024-09-13T10:11:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0095355430000000005, 2024-09-13T10:11:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.460 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.461 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:17.461 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.461 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.461 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.461 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 993 2024-09-13T10:11:17.461 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.461 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.461 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.461 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.461 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.181393+0000", 2024-09-13T10:11:17.461 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.461 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.462 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.462 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.462 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.181393+0000", 2024-09-13T10:11:17.462 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.462 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.462 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.462 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.462 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.181394+0000", 2024-09-13T10:11:17.462 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4300000000000001e-06 2024-09-13T10:11:17.462 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.462 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.462 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.462 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.181403+0000", 2024-09-13T10:11:17.462 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1460000000000002e-06 2024-09-13T10:11:17.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.463 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.463 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.463 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.181422+0000", 2024-09-13T10:11:17.463 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8916999999999999e-05 2024-09-13T10:11:17.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.463 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.463 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.463 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.181438+0000", 2024-09-13T10:11:17.463 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6146000000000001e-05 2024-09-13T10:11:17.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.463 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.463 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.464 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.187420+0000", 2024-09-13T10:11:17.464 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0059811339999999999 2024-09-13T10:11:17.464 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.464 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.464 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.464 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.187640+0000", 2024-09-13T10:11:17.464 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022068500000000001 2024-09-13T10:11:17.464 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.464 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.464 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.464 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.190928+0000", 2024-09-13T10:11:17.464 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0032880850000000001 2024-09-13T10:11:17.464 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.465 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.465 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.465 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.465 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.465 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1006 4.3 4:c5b5ff75:::smithi19530714-20:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:17.465 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:11:17.465 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 21.679762361000002, 2024-09-13T10:11:17.465 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.047764635, 2024-09-13T10:11:17.465 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.465 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.465 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:17.465 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.465 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.466 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.466 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1006 2024-09-13T10:11:17.466 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.466 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.466 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.466 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.466 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:11:17.466 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.466 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.466 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.466 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.466 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:11:17.466 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.467 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.467 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.467 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.467 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654049+0000", 2024-09-13T10:11:17.467 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5600000000000005e-07 2024-09-13T10:11:17.467 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.467 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.467 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.467 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654058+0000", 2024-09-13T10:11:17.467 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8200000000000003e-06 2024-09-13T10:11:17.467 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.467 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.467 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.467 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654067+0000", 2024-09-13T10:11:17.468 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.4599999999999992e-06 2024-09-13T10:11:17.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.468 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.468 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.468 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654075+0000", 2024-09-13T10:11:17.468 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.1070000000000005e-06 2024-09-13T10:11:17.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.468 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.468 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.468 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654188+0000", 2024-09-13T10:11:17.468 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000112734 2024-09-13T10:11:17.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.468 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.469 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.469 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654395+0000", 2024-09-13T10:11:17.469 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000207235 2024-09-13T10:11:17.469 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.469 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.469 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.469 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.701813+0000", 2024-09-13T10:11:17.469 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.047417523000000003 2024-09-13T10:11:17.469 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.469 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.469 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.469 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.470 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.470 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1013 4.3 4:c426503f:::smithi19530714-11:head [write 477332~766101] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:17.470 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:11:17.470 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 21.672441107000001, 2024-09-13T10:11:17.470 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027363838000000001, 2024-09-13T10:11:17.470 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.470 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.470 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:17.470 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.470 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.470 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.470 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1013 2024-09-13T10:11:17.470 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.470 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.471 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.471 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.471 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:11:17.471 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.471 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.471 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.471 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.471 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:11:17.471 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.471 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.471 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.471 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.471 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661374+0000", 2024-09-13T10:11:17.472 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.0930000000000002e-06 2024-09-13T10:11:17.472 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.472 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.472 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.472 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665828+0000", 2024-09-13T10:11:17.472 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044532119999999998 2024-09-13T10:11:17.472 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.472 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.472 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.472 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665864+0000", 2024-09-13T10:11:17.472 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5979000000000002e-05 2024-09-13T10:11:17.472 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.472 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.473 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.473 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665881+0000", 2024-09-13T10:11:17.473 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7836e-05 2024-09-13T10:11:17.473 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.473 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.473 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.473 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.678976+0000", 2024-09-13T10:11:17.473 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.01309448 2024-09-13T10:11:17.473 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.473 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.473 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.473 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.679188+0000", 2024-09-13T10:11:17.474 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000211945 2024-09-13T10:11:17.474 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.474 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.474 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:17.474 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.679468+0000", 2024-09-13T10:11:17.474 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028034000000000002 2024-09-13T10:11:17.474 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.474 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.474 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:17.474 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688233+0000", 2024-09-13T10:11:17.474 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0087645369999999993 2024-09-13T10:11:17.474 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.474 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.475 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.475 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688388+0000", 2024-09-13T10:11:17.475 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015488000000000001 2024-09-13T10:11:17.475 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.475 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.475 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.475 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688505+0000", 2024-09-13T10:11:17.475 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117114 2024-09-13T10:11:17.475 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.475 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.475 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:17.475 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688612+0000", 2024-09-13T10:11:17.475 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010741100000000001 2024-09-13T10:11:17.476 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.476 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.476 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.476 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688733+0000", 2024-09-13T10:11:17.476 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000121011 2024-09-13T10:11:17.476 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.476 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.476 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.476 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.476 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.476 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1014 4.3 4:c426503f:::smithi19530714-11:head [write 1681102~474010] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:17.476 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:11:17.476 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 21.667167720999998, 2024-09-13T10:11:17.477 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022694374, 2024-09-13T10:11:17.477 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.477 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.477 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:17.477 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.477 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.477 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.477 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1014 2024-09-13T10:11:17.477 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.477 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.477 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.477 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.477 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:11:17.478 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.478 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.478 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.478 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.478 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:11:17.478 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.478 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.478 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.478 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.478 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666645+0000", 2024-09-13T10:11:17.478 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2970000000000002e-06 2024-09-13T10:11:17.478 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.478 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.478 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.479 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667326+0000", 2024-09-13T10:11:17.479 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00068073199999999997 2024-09-13T10:11:17.479 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.479 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.479 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.479 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667346+0000", 2024-09-13T10:11:17.479 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0188999999999999e-05 2024-09-13T10:11:17.479 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.479 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.479 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.479 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667361+0000", 2024-09-13T10:11:17.479 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5299999999999999e-05 2024-09-13T10:11:17.479 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.479 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.480 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.480 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.683468+0000", 2024-09-13T10:11:17.480 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016106944000000002 2024-09-13T10:11:17.480 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.480 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.480 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.480 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.683714+0000", 2024-09-13T10:11:17.480 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024595699999999999 2024-09-13T10:11:17.480 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.480 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.480 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:17.480 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.684132+0000", 2024-09-13T10:11:17.481 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00041775800000000001 2024-09-13T10:11:17.481 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.481 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.481 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:17.481 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688771+0000", 2024-09-13T10:11:17.481 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0046394540000000003 2024-09-13T10:11:17.481 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.481 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.481 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.481 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688920+0000", 2024-09-13T10:11:17.481 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014886000000000001 2024-09-13T10:11:17.481 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.481 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.482 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.482 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689085+0000", 2024-09-13T10:11:17.482 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000164539 2024-09-13T10:11:17.482 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.482 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.482 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:17.482 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689230+0000", 2024-09-13T10:11:17.482 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000145394 2024-09-13T10:11:17.482 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.482 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.482 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.482 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689337+0000", 2024-09-13T10:11:17.482 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010695 2024-09-13T10:11:17.483 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.483 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.483 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.483 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.483 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.483 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1015 4.3 4:c426503f:::smithi19530714-11:head [write 2687912~589420] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:17.483 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:11:17.483 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 21.660851044000001, 2024-09-13T10:11:17.483 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.018489873, 2024-09-13T10:11:17.483 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.484 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.484 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:17.484 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.484 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.484 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.484 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1015 2024-09-13T10:11:17.484 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.484 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.484 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.484 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.484 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:11:17.485 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.485 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.485 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.485 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.485 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:11:17.485 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.485 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.485 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.485 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.485 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672962+0000", 2024-09-13T10:11:17.485 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.277e-06 2024-09-13T10:11:17.485 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.486 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.486 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.486 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673758+0000", 2024-09-13T10:11:17.486 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00079599100000000002 2024-09-13T10:11:17.486 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.486 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.486 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.486 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673787+0000", 2024-09-13T10:11:17.486 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9280000000000001e-05 2024-09-13T10:11:17.486 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.487 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.487 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.487 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673801+0000", 2024-09-13T10:11:17.487 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4493e-05 2024-09-13T10:11:17.487 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.487 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.487 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.487 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.685885+0000", 2024-09-13T10:11:17.487 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012083692 2024-09-13T10:11:17.487 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.487 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.488 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.488 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.686057+0000", 2024-09-13T10:11:17.488 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017133900000000001 2024-09-13T10:11:17.488 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.488 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.488 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:17.488 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.686396+0000", 2024-09-13T10:11:17.488 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000339809 2024-09-13T10:11:17.488 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.488 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.489 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.489 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691040+0000", 2024-09-13T10:11:17.489 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.004643215 2024-09-13T10:11:17.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.489 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.489 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:17.489 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691082+0000", 2024-09-13T10:11:17.489 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.2499000000000001e-05 2024-09-13T10:11:17.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.489 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.489 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.490 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691179+0000", 2024-09-13T10:11:17.490 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7044000000000001e-05 2024-09-13T10:11:17.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.490 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.490 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:17.490 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691324+0000", 2024-09-13T10:11:17.490 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014532299999999999 2024-09-13T10:11:17.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.490 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.490 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.490 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691449+0000", 2024-09-13T10:11:17.491 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012491100000000001 2024-09-13T10:11:17.491 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.491 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.491 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.491 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.491 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1016 4.3 4:c426503f:::smithi19530714-11:head [setxattr _header (56) in=7b,truncate 3277332] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:17.491 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:11:17.491 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 21.659982086999999, 2024-09-13T10:11:17.491 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.019820562999999999, 2024-09-13T10:11:17.491 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.492 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.492 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:17.492 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.492 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.492 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.492 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1016 2024-09-13T10:11:17.492 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.492 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.492 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.492 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.492 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:11:17.492 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.493 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.493 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.493 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.493 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:11:17.493 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.493 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.493 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.493 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.493 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673830+0000", 2024-09-13T10:11:17.493 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1170000000000001e-06 2024-09-13T10:11:17.493 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.494 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.494 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.494 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673835+0000", 2024-09-13T10:11:17.494 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5559999999999998e-06 2024-09-13T10:11:17.494 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.494 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.494 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.494 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673845+0000", 2024-09-13T10:11:17.494 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6330000000000008e-06 2024-09-13T10:11:17.494 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.494 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.495 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.495 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673856+0000", 2024-09-13T10:11:17.495 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0963e-05 2024-09-13T10:11:17.495 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.495 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.495 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.495 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689480+0000", 2024-09-13T10:11:17.495 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015624605 2024-09-13T10:11:17.495 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.495 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.495 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.495 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689739+0000", 2024-09-13T10:11:17.496 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025878700000000001 2024-09-13T10:11:17.496 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.496 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.496 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:17.496 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.690154+0000", 2024-09-13T10:11:17.496 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00041533800000000001 2024-09-13T10:11:17.496 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.496 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.496 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.496 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691858+0000", 2024-09-13T10:11:17.496 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001703811 2024-09-13T10:11:17.496 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.497 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.497 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:17.497 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691880+0000", 2024-09-13T10:11:17.497 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1997e-05 2024-09-13T10:11:17.497 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.497 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.497 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:17.497 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693384+0000", 2024-09-13T10:11:17.497 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0015040190000000001 2024-09-13T10:11:17.497 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.497 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.497 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:17.498 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693541+0000", 2024-09-13T10:11:17.498 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015723899999999999 2024-09-13T10:11:17.498 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.498 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.498 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.498 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693649+0000", 2024-09-13T10:11:17.498 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010749800000000001 2024-09-13T10:11:17.498 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.498 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.498 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.498 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.498 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.498 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1017 4.3 4:c426503f:::smithi19530714-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:17.499 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:11:17.499 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 21.659942075, 2024-09-13T10:11:17.499 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020166488999999999, 2024-09-13T10:11:17.499 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.499 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.499 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:17.499 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.499 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.499 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.499 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1017 2024-09-13T10:11:17.499 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.499 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.500 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.500 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.500 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:11:17.500 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.500 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.500 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.500 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.500 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:11:17.500 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.500 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.500 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.500 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.501 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673869+0000", 2024-09-13T10:11:17.501 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9699999999999995e-07 2024-09-13T10:11:17.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.501 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.501 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.501 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673873+0000", 2024-09-13T10:11:17.501 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.4970000000000001e-06 2024-09-13T10:11:17.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.501 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.501 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.501 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673885+0000", 2024-09-13T10:11:17.502 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2398999999999999e-05 2024-09-13T10:11:17.502 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.502 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.502 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.502 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673893+0000", 2024-09-13T10:11:17.502 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7530000000000001e-06 2024-09-13T10:11:17.502 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.502 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.502 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.502 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691638+0000", 2024-09-13T10:11:17.503 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017745007 2024-09-13T10:11:17.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.503 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.503 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:17.503 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691789+0000", 2024-09-13T10:11:17.503 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015133400000000001 2024-09-13T10:11:17.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.503 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.503 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.503 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693731+0000", 2024-09-13T10:11:17.504 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019422459999999999 2024-09-13T10:11:17.504 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.504 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.504 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.504 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693870+0000", 2024-09-13T10:11:17.504 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013820299999999999 2024-09-13T10:11:17.504 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.504 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.504 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.504 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.694035+0000", 2024-09-13T10:11:17.505 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000165253 2024-09-13T10:11:17.505 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.505 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.505 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.505 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.505 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.505 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1405 4.3 4:c5b5ff75:::smithi19530714-20:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T10:11:17.505 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:01.492787+0000", 2024-09-13T10:11:17.505 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 15.841023431, 2024-09-13T10:11:17.505 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021225698000000001, 2024-09-13T10:11:17.505 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:17.505 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:17.506 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:17.506 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:17.506 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:17.506 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:17.506 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1405 2024-09-13T10:11:17.506 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.506 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:17.506 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.506 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:17.506 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492787+0000", 2024-09-13T10:11:17.506 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.507 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.507 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:17.507 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492787+0000", 2024-09-13T10:11:17.507 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:17.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.507 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.507 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:17.507 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492791+0000", 2024-09-13T10:11:17.507 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.89e-06 2024-09-13T10:11:17.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.507 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.508 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:17.508 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492805+0000", 2024-09-13T10:11:17.508 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4083000000000001e-05 2024-09-13T10:11:17.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.508 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.508 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:17.508 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492846+0000", 2024-09-13T10:11:17.508 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0849000000000001e-05 2024-09-13T10:11:17.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.508 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.508 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:17.508 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492867+0000", 2024-09-13T10:11:17.509 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1058999999999999e-05 2024-09-13T10:11:17.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.509 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.509 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:17.509 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492973+0000", 2024-09-13T10:11:17.509 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000106084 2024-09-13T10:11:17.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.509 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.509 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:17.509 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.493105+0000", 2024-09-13T10:11:17.509 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013215899999999999 2024-09-13T10:11:17.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:17.510 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:17.510 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:17.510 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.514013+0000", 2024-09-13T10:11:17.510 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020907574000000002 2024-09-13T10:11:17.510 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.510 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.510 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.510 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:17.510 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:17.510 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:17.511 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T10:11:17.511 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:17.375+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:17.514 INFO:tasks.rados.rados.0.smithi195.stdout:1071: finishing set_chunk to oid 1 2024-09-13T10:11:17.514 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 1 set_chunk returned error code -95 offset: 357376 length: 779264 2024-09-13T10:11:17.515 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 0 (ObjNum 184 snap 0 seq_num 184) dirty exists 2024-09-13T10:11:17.515 INFO:tasks.rados.rados.0.smithi195.stdout:1072: seq_num 192 ranges {426944=556633,1468778=621400,2872744=354200} 2024-09-13T10:11:17.515 INFO:tasks.rados.rados.0.smithi195.stdout:1072: writing smithi19530714-20 from 426944 to 983577 tid 1 2024-09-13T10:11:17.515 INFO:tasks.rados.rados.0.smithi195.stdout:1072: writing smithi19530714-20 from 1468778 to 2090178 tid 2 2024-09-13T10:11:17.515 INFO:tasks.rados.rados.0.smithi195.stdout:1072: writing smithi19530714-20 from 2872744 to 3226944 tid 3 2024-09-13T10:11:17.515 INFO:tasks.rados.rados.0.smithi195.stdout:1066: done (6 left) 2024-09-13T10:11:17.515 INFO:tasks.rados.rados.0.smithi195.stdout:1067: done (5 left) 2024-09-13T10:11:17.515 INFO:tasks.rados.rados.0.smithi195.stdout:1068: done (4 left) 2024-09-13T10:11:17.515 INFO:tasks.rados.rados.0.smithi195.stdout:1069: done (3 left) 2024-09-13T10:11:17.515 INFO:tasks.rados.rados.0.smithi195.stdout:1070: done (2 left) 2024-09-13T10:11:17.515 INFO:tasks.rados.rados.0.smithi195.stdout:1071: done (1 left) 2024-09-13T10:11:17.515 INFO:tasks.rados.rados.0.smithi195.stdout:1073: read oid 2 snap -1 2024-09-13T10:11:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1073: expect (ObjNum 176 snap 0 seq_num 176) 2024-09-13T10:11:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1074: read oid 47 snap -1 2024-09-13T10:11:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1074: expect (ObjNum 182 snap 0 seq_num 182) 2024-09-13T10:11:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1075: tier_promote oid 40 2024-09-13T10:11:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1072: finishing write tid 1 to smithi19530714-20 2024-09-13T10:11:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1076: read oid 10 snap -1 2024-09-13T10:11:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1076: expect (ObjNum 84 snap 0 seq_num 84) 2024-09-13T10:11:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1077: write (excl) oid 38 current snap is 0 2024-09-13T10:11:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1077: seq_num 193 ranges {443751=631882,1810907=765058,3067461=176290} 2024-09-13T10:11:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1077: writing smithi19530714-38 from 443751 to 1075633 tid 1 2024-09-13T10:11:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1077: writing smithi19530714-38 from 1810907 to 2575965 tid 2 2024-09-13T10:11:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1077: writing smithi19530714-38 from 3067461 to 3243751 tid 3 2024-09-13T10:11:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1078: read oid 41 snap -1 2024-09-13T10:11:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1078: expect (ObjNum 141 snap 0 seq_num 141) 2024-09-13T10:11:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1079: set_chunk oid 19 target oid 2024-09-13T10:11:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1079: set_chunk oid 19 offset: 584704 length: 1572864 target oid f58c9c6d1a885b65eaea1deae881bf55e7a380cf519520eb88a42b500e3fb4ba offset: 0 snap -1 2024-09-13T10:11:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1080: tier_flush oid 25 2024-09-13T10:11:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1080: tier_flush oid 25 snap -1 2024-09-13T10:11:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1073: expect (ObjNum 176 snap 0 seq_num 176) 2024-09-13T10:11:17.517 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:17.511+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:17.591 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2484685 2024-09-13T10:11:17.599 INFO:tasks.rados.rados.0.smithi195.stdout:1074: expect (ObjNum 182 snap 0 seq_num 182) 2024-09-13T10:11:17.605 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1559966 2024-09-13T10:11:17.614 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:17.611+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:17.642 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:17.642 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:17.642 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:11:17.642 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:17.656 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T10:11:17.721 INFO:tasks.rados.rados.0.smithi195.stdout:1072: finishing write tid 2 to smithi19530714-20 2024-09-13T10:11:17.721 INFO:tasks.rados.rados.0.smithi195.stdout:1075: got (0) Success 2024-09-13T10:11:17.721 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 17 (ObjNum 154 snap 0 seq_num 154) dirty exists 2024-09-13T10:11:17.721 INFO:tasks.rados.rados.0.smithi195.stdout:1076: expect (ObjNum 84 snap 0 seq_num 84) 2024-09-13T10:11:17.721 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:17.744 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:17.743+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:17.816 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:17.813+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:17.822 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:17.822 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:17.823 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:11:17.823 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:11:17.823 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:17.833 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:11:17.833 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:11:17.837 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T10:11:17.840 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1131899 2024-09-13T10:11:17.848 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T10:11:17.849 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:11:17.851 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-13T10:11:16.938779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":187226378,"num_objects":158,"num_object_clones":0,"num_object_copies":474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":1669,"num_read_kb":862572,"num_write":1386,"num_write_kb":312683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":25,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105,"ondisk_log_size":105,"up":117,"acting":117,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":852196,"kb_used_data":394216,"kb_used_omap":24,"kb_used_meta":452903,"kb_avail":1509097244,"statfs":{"total":1546188226560,"available":1545315577856,"internally_reserved":0,"allocated":403677184,"data_stored":397847896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":463772751},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11554125,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":436,"num_read_kb":304590,"num_write":232,"num_write_kb":50516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.247269"},"pg_stats":[{"pgid":"4.19","version":"52'35","reported_seq":148,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.706837+0000","last_change":"2024-09-13T10:11:00.388311+0000","last_active":"2024-09-13T10:11:12.706837+0000","last_peered":"2024-09-13T10:11:12.706837+0000","last_clean":"2024-09-13T10:11:12.706837+0000","last_became_active":"2024-09-13T10:10:54.324793+0000","last_became_peered":"2024-09-13T10:10:54.324793+0000","last_unstale":"2024-09-13T10:11:12.706837+0000","last_undegraded":"2024-09-13T10:11:12.706837+0000","last_fullsized":"2024-09-13T10:11:12.706837+0000","mapping_epoch":41,"log_start":"48'32","ondisk_log_start":"48'32","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'28","last_scrub_stamp":"2024-09-13T10:11:00.388132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.388132+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":32,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:17.033000+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4823609,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":35932,"num_write":9,"num_write_kb":2190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,9],"acting":[11,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.18","version":"55'8","reported_seq":106,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.693872+0000","last_change":"2024-09-13T10:11:01.783685+0000","last_active":"2024-09-13T10:11:12.693872+0000","last_peered":"2024-09-13T10:11:12.693872+0000","last_clean":"2024-09-13T10:11:12.693872+0000","last_became_active":"2024-09-13T10:10:54.319034+0000","last_became_peered":"2024-09-13T10:10:54.319034+0000","last_unstale":"2024-09-13T10:11:12.693872+0000","last_undegraded":"2024-09-13T10:11:12.693872+0000","last_fullsized":"2024-09-13T10:11:12.693872+0000","mapping_epoch":41,"log_start":"39'5","ondisk_log_start":"39'5","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'5","last_scrub_stamp":"2024-09-13T10:11:01.783496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.783496+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":5,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:07.769513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2907611,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":11422,"num_write":4,"num_write_kb":1195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,0],"acting":[3,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"54'62","reported_seq":194,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T10:11:11.766947+0000","last_change":"2024-09-13T10:11:00.442471+0000","last_active":"2024-09-13T10:11:11.766947+0000","last_peered":"2024-09-13T10:11:11.766947+0000","last_clean":"2024-09-13T10:11:11.766947+0000","last_became_active":"2024-09-13T10:10:54.318781+0000","last_became_peered":"2024-09-13T10:10:54.318781+0000","last_unstale":"2024-09-13T10:11:11.766947+0000","last_undegraded":"2024-09-13T10:11:11.766947+0000","last_fullsized":"2024-09-13T10:11:11.766947+0000","mapping_epoch":41,"log_start":"52'59","ondisk_log_start":"52'59","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'34","last_scrub_stamp":"2024-09-13T10:11:00.442413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.442413+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":59,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:25.413102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7512729,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":20998,"num_write":46,"num_write_kb":11933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,10],"acting":[1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"53'49","reported_seq":166,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.691295+0000","last_change":"2024-09-13T10:11:00.456606+0000","last_active":"2024-09-13T10:11:12.691295+0000","last_peered":"2024-09-13T10:11:12.691295+0000","last_clean":"2024-09-13T10:11:12.691295+0000","last_became_active":"2024-09-13T10:10:54.324059+0000","last_became_peered":"2024-09-13T10:10:54.324059+0000","last_unstale":"2024-09-13T10:11:12.691295+0000","last_undegraded":"2024-09-13T10:11:12.691295+0000","last_fullsized":"2024-09-13T10:11:12.691295+0000","mapping_epoch":41,"log_start":"51'46","ondisk_log_start":"51'46","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"46'44","last_scrub_stamp":"2024-09-13T10:11:00.456457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.456457+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":43,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:11.544372+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2689120,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":17457,"num_write":5,"num_write_kb":1136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,13,1],"acting":[7,13,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"45'36","reported_seq":150,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.699761+0000","last_change":"2024-09-13T10:11:05.611718+0000","last_active":"2024-09-13T10:11:12.699761+0000","last_peered":"2024-09-13T10:11:12.699761+0000","last_clean":"2024-09-13T10:11:12.699761+0000","last_became_active":"2024-09-13T10:10:54.326181+0000","last_became_peered":"2024-09-13T10:10:54.326181+0000","last_unstale":"2024-09-13T10:11:12.699761+0000","last_undegraded":"2024-09-13T10:11:12.699761+0000","last_fullsized":"2024-09-13T10:11:12.699761+0000","mapping_epoch":41,"log_start":"45'33","ondisk_log_start":"45'33","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'36","last_scrub_stamp":"2024-09-13T10:11:05.611551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.611551+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":31,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:29.302418+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714314,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":4993,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,0],"acting":[6,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"39'40","reported_seq":149,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.699511+0000","last_change":"2024-09-13T10:11:04.595393+0000","last_active":"2024-09-13T10:11:12.699511+0000","last_peered":"2024-09-13T10:11:12.699511+0000","last_clean":"2024-09-13T10:11:12.699511+0000","last_became_active":"2024-09-13T10:10:54.326555+0000","last_became_peered":"2024-09-13T10:10:54.326555+0000","last_unstale":"2024-09-13T10:11:12.699511+0000","last_undegraded":"2024-09-13T10:11:12.699511+0000","last_fullsized":"2024-09-13T10:11:12.699511+0000","mapping_epoch":41,"log_start":"39'37","ondisk_log_start":"39'37","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'40","last_scrub_stamp":"2024-09-13T10:11:04.595221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.595221+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":37,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:12.683252+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10],"acting":[6,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"47'20","reported_seq":128,"reported_epoch":57,"state":"activating+remapped","last_fresh":"2024-09-13T10:11:13.735272+0000","last_change":"2024-09-13T10:11:13.699344+0000","last_active":"2024-09-13T10:11:12.701963+0000","last_peered":"2024-09-13T10:11:10.721188+0000","last_clean":"2024-09-13T10:11:10.721188+0000","last_became_active":"2024-09-13T10:10:54.325296+0000","last_became_peered":"2024-09-13T10:10:54.325296+0000","last_unstale":"2024-09-13T10:11:13.735272+0000","last_undegraded":"2024-09-13T10:11:13.735272+0000","last_fullsized":"2024-09-13T10:11:13.735272+0000","mapping_epoch":56,"log_start":"39'15","ondisk_log_start":"39'15","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'17","last_scrub_stamp":"2024-09-13T10:11:03.220846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:03.220846+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":15,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2117682,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[2,8,11],"avail_no_missing":["2","8","11"],"object_location_counts":[{"shards":"2,8,11","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"47'20","reported_seq":161,"reported_epoch":56,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:11:13.254415+0000","last_change":"2024-09-13T10:11:11.763857+0000","last_active":"2024-09-13T10:11:13.254415+0000","last_peered":"2024-09-13T10:11:13.254415+0000","last_clean":"2024-09-13T10:11:08.694521+0000","last_became_active":"2024-09-13T10:11:11.753624+0000","last_became_peered":"2024-09-13T10:11:11.753624+0000","last_unstale":"2024-09-13T10:11:13.254415+0000","last_undegraded":"2024-09-13T10:11:13.254415+0000","last_fullsized":"2024-09-13T10:11:13.254415+0000","mapping_epoch":54,"log_start":"42'17","ondisk_log_start":"42'17","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'17","last_scrub_stamp":"2024-09-13T10:11:02.582829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:02.582829+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":17,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5395014,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":49262,"num_write":4,"num_write_kb":1159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,9],"acting":[6,8,14],"avail_no_missing":["6","8","14"],"object_location_counts":[{"shards":"6,8,14","objects":2}],"blocked_by":[],"up_primary":13,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"40'13","reported_seq":124,"reported_epoch":57,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:11:13.697918+0000","last_change":"2024-09-13T10:11:08.740093+0000","last_active":"2024-09-13T10:11:13.697918+0000","last_peered":"2024-09-13T10:11:13.697918+0000","last_clean":"2024-09-13T10:11:06.437508+0000","last_became_active":"2024-09-13T10:11:08.737198+0000","last_became_peered":"2024-09-13T10:11:08.737198+0000","last_unstale":"2024-09-13T10:11:13.697918+0000","last_undegraded":"2024-09-13T10:11:13.697918+0000","last_fullsized":"2024-09-13T10:11:13.697918+0000","mapping_epoch":51,"log_start":"39'10","ondisk_log_start":"39'10","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"40'13","last_scrub_stamp":"2024-09-13T10:11:04.242723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.242723+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":10,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2689087,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":26,"num_read_kb":19408,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,12],"acting":[2,12,8],"avail_no_missing":["2","8","12"],"object_location_counts":[{"shards":"2,8,12","objects":1}],"blocked_by":[],"up_primary":13,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"53'33","reported_seq":158,"reported_epoch":56,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:11:12.697911+0000","last_change":"2024-09-13T10:11:07.791128+0000","last_active":"2024-09-13T10:11:12.697911+0000","last_peered":"2024-09-13T10:11:12.697911+0000","last_clean":"2024-09-13T10:11:04.408213+0000","last_became_active":"2024-09-13T10:11:07.782322+0000","last_became_peered":"2024-09-13T10:11:07.782322+0000","last_unstale":"2024-09-13T10:11:12.697911+0000","last_undegraded":"2024-09-13T10:11:12.697911+0000","last_fullsized":"2024-09-13T10:11:12.697911+0000","mapping_epoch":49,"log_start":"52'30","ondisk_log_start":"52'30","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'16","last_scrub_stamp":"2024-09-13T10:11:00.706874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.706874+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":28,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4773188,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":30892,"num_write":21,"num_write_kb":5962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,14,3],"acting":[4,5,6],"avail_no_missing":["4","5","6"],"object_location_counts":[{"shards":"4,5,6","objects":2}],"blocked_by":[],"up_primary":9,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.745091+0000","last_change":"2024-09-13T10:11:12.744453+0000","last_active":"2024-09-13T10:11:12.745091+0000","last_peered":"2024-09-13T10:11:12.745091+0000","last_clean":"2024-09-13T10:11:12.745091+0000","last_became_active":"2024-09-13T10:11:12.744090+0000","last_became_peered":"2024-09-13T10:11:12.744090+0000","last_unstale":"2024-09-13T10:11:12.745091+0000","last_undegraded":"2024-09-13T10:11:12.745091+0000","last_fullsized":"2024-09-13T10:11:12.745091+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:57.001325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,12],"acting":[3,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"39'12","reported_seq":110,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.700551+0000","last_change":"2024-09-13T10:11:01.537080+0000","last_active":"2024-09-13T10:11:12.700551+0000","last_peered":"2024-09-13T10:11:12.700551+0000","last_clean":"2024-09-13T10:11:12.700551+0000","last_became_active":"2024-09-13T10:10:54.318519+0000","last_became_peered":"2024-09-13T10:10:54.318519+0000","last_unstale":"2024-09-13T10:11:12.700551+0000","last_undegraded":"2024-09-13T10:11:12.700551+0000","last_fullsized":"2024-09-13T10:11:12.700551+0000","mapping_epoch":31,"log_start":"38'9","ondisk_log_start":"38'9","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'12","last_scrub_stamp":"2024-09-13T10:11:01.536963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.536963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":9,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:27.498398+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,14],"acting":[6,8,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.693183+0000","last_change":"2024-09-13T10:10:32.932079+0000","last_active":"2024-09-13T10:11:12.693183+0000","last_peered":"2024-09-13T10:11:12.693183+0000","last_clean":"2024-09-13T10:11:12.693183+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:11:12.693183+0000","last_undegraded":"2024-09-13T10:11:12.693183+0000","last_fullsized":"2024-09-13T10:11:12.693183+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:57.533673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"55'46","reported_seq":172,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T10:11:13.703353+0000","last_change":"2024-09-13T10:11:01.219554+0000","last_active":"2024-09-13T10:11:13.703353+0000","last_peered":"2024-09-13T10:11:13.703353+0000","last_clean":"2024-09-13T10:11:13.703353+0000","last_became_active":"2024-09-13T10:10:54.314545+0000","last_became_peered":"2024-09-13T10:10:54.314545+0000","last_unstale":"2024-09-13T10:11:13.703353+0000","last_undegraded":"2024-09-13T10:11:13.703353+0000","last_fullsized":"2024-09-13T10:11:13.703353+0000","mapping_epoch":31,"log_start":"55'43","ondisk_log_start":"55'43","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'27","last_scrub_stamp":"2024-09-13T10:11:01.219486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.219486+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":42,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:06.525021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5647152,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":27064,"num_write":30,"num_write_kb":8545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,8],"acting":[2,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":77,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.693803+0000","last_change":"2024-09-13T10:10:32.177488+0000","last_active":"2024-09-13T10:11:12.693803+0000","last_peered":"2024-09-13T10:11:12.693803+0000","last_clean":"2024-09-13T10:11:12.693803+0000","last_became_active":"2024-09-13T10:10:30.128658+0000","last_became_peered":"2024-09-13T10:10:30.128658+0000","last_unstale":"2024-09-13T10:11:12.693803+0000","last_undegraded":"2024-09-13T10:11:12.693803+0000","last_fullsized":"2024-09-13T10:11:12.693803+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:50.278429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.0","version":"55'18","reported_seq":130,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.745802+0000","last_change":"2024-09-13T10:11:05.681812+0000","last_active":"2024-09-13T10:11:12.745802+0000","last_peered":"2024-09-13T10:11:12.745802+0000","last_clean":"2024-09-13T10:11:12.745802+0000","last_became_active":"2024-09-13T10:10:54.320014+0000","last_became_peered":"2024-09-13T10:10:54.320014+0000","last_unstale":"2024-09-13T10:11:12.745802+0000","last_undegraded":"2024-09-13T10:11:12.745802+0000","last_fullsized":"2024-09-13T10:11:12.745802+0000","mapping_epoch":31,"log_start":"53'15","ondisk_log_start":"53'15","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'14","last_scrub_stamp":"2024-09-13T10:11:05.681653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.681653+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":14,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:19.662203+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2588278,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":502,"num_write":4,"num_write_kb":1352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T10:11:13.698709+0000","last_change":"2024-09-13T10:10:32.701581+0000","last_active":"2024-09-13T10:11:13.698709+0000","last_peered":"2024-09-13T10:11:13.698709+0000","last_clean":"2024-09-13T10:11:13.698709+0000","last_became_active":"2024-09-13T10:10:30.126588+0000","last_became_peered":"2024-09-13T10:10:30.126588+0000","last_unstale":"2024-09-13T10:11:13.698709+0000","last_undegraded":"2024-09-13T10:11:13.698709+0000","last_fullsized":"2024-09-13T10:11:13.698709+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:40.095103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089379399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.1","version":"44'18","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T10:11:13.910316+0000","last_change":"2024-09-13T10:11:02.190704+0000","last_active":"2024-09-13T10:11:13.910316+0000","last_peered":"2024-09-13T10:11:13.910316+0000","last_clean":"2024-09-13T10:11:13.910316+0000","last_became_active":"2024-09-13T10:10:54.317600+0000","last_became_peered":"2024-09-13T10:10:54.317600+0000","last_unstale":"2024-09-13T10:11:13.910316+0000","last_undegraded":"2024-09-13T10:11:13.910316+0000","last_fullsized":"2024-09-13T10:11:13.910316+0000","mapping_epoch":31,"log_start":"43'14","ondisk_log_start":"43'14","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'18","last_scrub_stamp":"2024-09-13T10:11:02.190506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:02.190506+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":14,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:28.399688+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3411788,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":32502,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,12],"acting":[2,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"59'151","reported_seq":229,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T10:11:15.775224+0000","last_change":"2024-09-13T10:10:43.231291+0000","last_active":"2024-09-13T10:11:15.775224+0000","last_peered":"2024-09-13T10:11:15.775224+0000","last_clean":"2024-09-13T10:11:15.775224+0000","last_became_active":"2024-09-13T10:10:43.230866+0000","last_became_peered":"2024-09-13T10:10:43.230866+0000","last_unstale":"2024-09-13T10:11:15.775224+0000","last_undegraded":"2024-09-13T10:11:15.775224+0000","last_fullsized":"2024-09-13T10:11:15.775224+0000","mapping_epoch":30,"log_start":"58'146","ondisk_log_start":"58'146","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":161,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:54.564637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15925502,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":141,"num_read_kb":2263,"num_write":198,"num_write_kb":43825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,4,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":160,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.700378+0000","last_change":"2024-09-13T10:11:12.699917+0000","last_active":"2024-09-13T10:11:12.700378+0000","last_peered":"2024-09-13T10:11:12.700378+0000","last_clean":"2024-09-13T10:11:12.700378+0000","last_became_active":"2024-09-13T10:11:12.699043+0000","last_became_peered":"2024-09-13T10:11:12.699043+0000","last_unstale":"2024-09-13T10:11:12.700378+0000","last_undegraded":"2024-09-13T10:11:12.700378+0000","last_fullsized":"2024-09-13T10:11:12.700378+0000","mapping_epoch":55,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:41.120070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,7],"acting":[3,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":86,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T10:11:15.769949+0000","last_change":"2024-09-13T10:10:32.702214+0000","last_active":"2024-09-13T10:11:15.769949+0000","last_peered":"2024-09-13T10:11:15.769949+0000","last_clean":"2024-09-13T10:11:15.769949+0000","last_became_active":"2024-09-13T10:10:30.124759+0000","last_became_peered":"2024-09-13T10:10:30.124759+0000","last_unstale":"2024-09-13T10:11:15.769949+0000","last_undegraded":"2024-09-13T10:11:15.769949+0000","last_fullsized":"2024-09-13T10:11:15.769949+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:45.386731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090474700000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.6","version":"54'53","reported_seq":174,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.691529+0000","last_change":"2024-09-13T10:11:05.397056+0000","last_active":"2024-09-13T10:11:12.691529+0000","last_peered":"2024-09-13T10:11:12.691529+0000","last_clean":"2024-09-13T10:11:12.691529+0000","last_became_active":"2024-09-13T10:10:54.319456+0000","last_became_peered":"2024-09-13T10:10:54.319456+0000","last_unstale":"2024-09-13T10:11:12.691529+0000","last_undegraded":"2024-09-13T10:11:12.691529+0000","last_fullsized":"2024-09-13T10:11:12.691529+0000","mapping_epoch":31,"log_start":"53'50","ondisk_log_start":"53'50","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'44","last_scrub_stamp":"2024-09-13T10:11:05.396838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.396838+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":50,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:30.295795+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6268978,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2579,"num_write":12,"num_write_kb":3084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,13,1],"acting":[7,13,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"55'162","reported_seq":228,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.693767+0000","last_change":"2024-09-13T10:10:43.220235+0000","last_active":"2024-09-13T10:11:12.693767+0000","last_peered":"2024-09-13T10:11:12.693767+0000","last_clean":"2024-09-13T10:11:12.693767+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:11:12.693767+0000","last_undegraded":"2024-09-13T10:11:12.693767+0000","last_fullsized":"2024-09-13T10:11:12.693767+0000","mapping_epoch":30,"log_start":"54'156","ondisk_log_start":"54'156","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":168,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:55.166722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18216975,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":142,"num_read_kb":2568,"num_write":211,"num_write_kb":53217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":83,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T10:11:13.697946+0000","last_change":"2024-09-13T10:10:32.146042+0000","last_active":"2024-09-13T10:11:13.697946+0000","last_peered":"2024-09-13T10:11:13.697946+0000","last_clean":"2024-09-13T10:11:13.697946+0000","last_became_active":"2024-09-13T10:10:30.127232+0000","last_became_peered":"2024-09-13T10:10:30.127232+0000","last_unstale":"2024-09-13T10:11:13.697946+0000","last_undegraded":"2024-09-13T10:11:13.697946+0000","last_fullsized":"2024-09-13T10:11:13.697946+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:29.955802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034622780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"55'49","reported_seq":176,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T10:11:11.815075+0000","last_change":"2024-09-13T10:11:05.436449+0000","last_active":"2024-09-13T10:11:11.815075+0000","last_peered":"2024-09-13T10:11:11.815075+0000","last_clean":"2024-09-13T10:11:11.815075+0000","last_became_active":"2024-09-13T10:10:54.314465+0000","last_became_peered":"2024-09-13T10:10:54.314465+0000","last_unstale":"2024-09-13T10:11:11.815075+0000","last_undegraded":"2024-09-13T10:11:11.815075+0000","last_fullsized":"2024-09-13T10:11:11.815075+0000","mapping_epoch":31,"log_start":"55'46","ondisk_log_start":"55'46","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'40","last_scrub_stamp":"2024-09-13T10:11:05.436272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.436272+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":43,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:15.243458+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5445468,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":1654,"num_write":10,"num_write_kb":2520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,10],"acting":[1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"55'129","reported_seq":193,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T10:11:11.749125+0000","last_change":"2024-09-13T10:10:43.220955+0000","last_active":"2024-09-13T10:11:11.749125+0000","last_peered":"2024-09-13T10:11:11.749125+0000","last_clean":"2024-09-13T10:11:11.749125+0000","last_became_active":"2024-09-13T10:10:43.220074+0000","last_became_peered":"2024-09-13T10:10:43.220074+0000","last_unstale":"2024-09-13T10:11:11.749125+0000","last_undegraded":"2024-09-13T10:11:11.749125+0000","last_fullsized":"2024-09-13T10:11:11.749125+0000","mapping_epoch":30,"log_start":"54'126","ondisk_log_start":"54'126","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":138,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:54.562129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18303268,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":112,"num_read_kb":2643,"num_write":161,"num_write_kb":42583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.691376+0000","last_change":"2024-09-13T10:10:33.223611+0000","last_active":"2024-09-13T10:11:12.691376+0000","last_peered":"2024-09-13T10:11:12.691376+0000","last_clean":"2024-09-13T10:11:12.691376+0000","last_became_active":"2024-09-13T10:10:30.128310+0000","last_became_peered":"2024-09-13T10:10:30.128310+0000","last_unstale":"2024-09-13T10:11:12.691376+0000","last_undegraded":"2024-09-13T10:11:12.691376+0000","last_fullsized":"2024-09-13T10:11:12.691376+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:46.008600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081844899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"54'92","reported_seq":288,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:13.294436+0000","last_change":"2024-09-13T10:11:03.633738+0000","last_active":"2024-09-13T10:11:13.294436+0000","last_peered":"2024-09-13T10:11:13.294436+0000","last_clean":"2024-09-13T10:11:13.294436+0000","last_became_active":"2024-09-13T10:10:54.316518+0000","last_became_peered":"2024-09-13T10:10:54.316518+0000","last_unstale":"2024-09-13T10:11:13.294436+0000","last_undegraded":"2024-09-13T10:11:13.294436+0000","last_fullsized":"2024-09-13T10:11:13.294436+0000","mapping_epoch":31,"log_start":"54'89","ondisk_log_start":"54'89","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'66","last_scrub_stamp":"2024-09-13T10:11:03.633499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:03.633499+0000","objects_scrubbed":5,"log_size":3,"log_dups_size":89,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:03.835715+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15529635,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":98,"num_read_kb":77140,"num_write":30,"num_write_kb":8293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10],"acting":[6,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'191","reported_seq":265,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T10:11:13.698355+0000","last_change":"2024-09-13T10:10:43.230767+0000","last_active":"2024-09-13T10:11:13.698355+0000","last_peered":"2024-09-13T10:11:13.698355+0000","last_clean":"2024-09-13T10:11:13.698355+0000","last_became_active":"2024-09-13T10:10:43.229961+0000","last_became_peered":"2024-09-13T10:10:43.229961+0000","last_unstale":"2024-09-13T10:11:13.698355+0000","last_undegraded":"2024-09-13T10:11:13.698355+0000","last_fullsized":"2024-09-13T10:11:13.698355+0000","mapping_epoch":30,"log_start":"56'188","ondisk_log_start":"56'188","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":197,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:53.856674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9281018,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":178,"num_read_kb":2853,"num_write":231,"num_write_kb":49860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.699431+0000","last_change":"2024-09-13T10:10:32.149042+0000","last_active":"2024-09-13T10:11:12.699431+0000","last_peered":"2024-09-13T10:11:12.699431+0000","last_clean":"2024-09-13T10:11:12.699431+0000","last_became_active":"2024-09-13T10:10:30.272007+0000","last_became_peered":"2024-09-13T10:10:30.272007+0000","last_unstale":"2024-09-13T10:11:12.699431+0000","last_undegraded":"2024-09-13T10:11:12.699431+0000","last_fullsized":"2024-09-13T10:11:12.699431+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:49.552306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000987604,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"56'93","reported_seq":270,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:13.084159+0000","last_change":"2024-09-13T10:11:00.541038+0000","last_active":"2024-09-13T10:11:13.084159+0000","last_peered":"2024-09-13T10:11:13.084159+0000","last_clean":"2024-09-13T10:11:13.084159+0000","last_became_active":"2024-09-13T10:10:54.315196+0000","last_became_peered":"2024-09-13T10:10:54.315196+0000","last_unstale":"2024-09-13T10:11:13.084159+0000","last_undegraded":"2024-09-13T10:11:13.084159+0000","last_fullsized":"2024-09-13T10:11:13.084159+0000","mapping_epoch":31,"log_start":"55'90","ondisk_log_start":"55'90","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'48","last_scrub_stamp":"2024-09-13T10:11:00.540905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.540905+0000","objects_scrubbed":5,"log_size":3,"log_dups_size":89,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:05.282698+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14235496,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":58,"num_read_kb":33807,"num_write":54,"num_write_kb":15402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,0],"acting":[6,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"39'5","reported_seq":92,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.693579+0000","last_change":"2024-09-13T10:11:00.786261+0000","last_active":"2024-09-13T10:11:12.693579+0000","last_peered":"2024-09-13T10:11:12.693579+0000","last_clean":"2024-09-13T10:11:12.693579+0000","last_became_active":"2024-09-13T10:10:54.314935+0000","last_became_peered":"2024-09-13T10:10:54.314935+0000","last_unstale":"2024-09-13T10:11:12.693579+0000","last_undegraded":"2024-09-13T10:11:12.693579+0000","last_fullsized":"2024-09-13T10:11:12.693579+0000","mapping_epoch":31,"log_start":"38'2","ondisk_log_start":"38'2","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'5","last_scrub_stamp":"2024-09-13T10:11:00.786110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.786110+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:22.428968+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,0],"acting":[3,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"46'29","reported_seq":134,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.706829+0000","last_change":"2024-09-13T10:11:04.446852+0000","last_active":"2024-09-13T10:11:12.706829+0000","last_peered":"2024-09-13T10:11:12.706829+0000","last_clean":"2024-09-13T10:11:12.706829+0000","last_became_active":"2024-09-13T10:10:54.316057+0000","last_became_peered":"2024-09-13T10:10:54.316057+0000","last_unstale":"2024-09-13T10:11:12.706829+0000","last_undegraded":"2024-09-13T10:11:12.706829+0000","last_fullsized":"2024-09-13T10:11:12.706829+0000","mapping_epoch":31,"log_start":"44'26","ondisk_log_start":"44'26","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'29","last_scrub_stamp":"2024-09-13T10:11:04.446717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.446717+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":23,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:06.702582+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3512630,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":11550,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,9],"acting":[11,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.a","version":"54'19","reported_seq":112,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.709355+0000","last_change":"2024-09-13T10:11:12.708135+0000","last_active":"2024-09-13T10:11:12.709355+0000","last_peered":"2024-09-13T10:11:12.709355+0000","last_clean":"2024-09-13T10:11:12.709355+0000","last_became_active":"2024-09-13T10:11:12.707306+0000","last_became_peered":"2024-09-13T10:11:12.707306+0000","last_unstale":"2024-09-13T10:11:12.709355+0000","last_undegraded":"2024-09-13T10:11:12.709355+0000","last_fullsized":"2024-09-13T10:11:12.709355+0000","mapping_epoch":55,"log_start":"51'16","ondisk_log_start":"51'16","created":31,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":16,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:02.338860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2437015,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":34314,"num_write":18,"num_write_kb":5695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,13],"acting":[6,8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"54'48","reported_seq":168,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-13T10:11:13.698850+0000","last_change":"2024-09-13T10:10:44.227804+0000","last_active":"2024-09-13T10:11:13.698850+0000","last_peered":"2024-09-13T10:11:13.698850+0000","last_clean":"2024-09-13T10:11:13.698850+0000","last_became_active":"2024-09-13T10:10:44.226956+0000","last_became_peered":"2024-09-13T10:10:44.226956+0000","last_unstale":"2024-09-13T10:11:13.698850+0000","last_undegraded":"2024-09-13T10:11:13.698850+0000","last_fullsized":"2024-09-13T10:11:13.698850+0000","mapping_epoch":31,"log_start":"54'45","ondisk_log_start":"54'45","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":45,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:44.906351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8201783,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":89848,"num_write":45,"num_write_kb":12537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,4],"acting":[14,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.c","version":"56'55","reported_seq":210,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.699892+0000","last_change":"2024-09-13T10:10:59.373229+0000","last_active":"2024-09-13T10:11:12.699892+0000","last_peered":"2024-09-13T10:11:12.699892+0000","last_clean":"2024-09-13T10:11:12.699892+0000","last_became_active":"2024-09-13T10:10:59.366341+0000","last_became_peered":"2024-09-13T10:10:59.366341+0000","last_unstale":"2024-09-13T10:11:12.699892+0000","last_undegraded":"2024-09-13T10:11:12.699892+0000","last_fullsized":"2024-09-13T10:11:12.699892+0000","mapping_epoch":44,"log_start":"47'51","ondisk_log_start":"47'51","created":31,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":51,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:02.624274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9294205,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":182,"num_read_kb":170109,"num_write":47,"num_write_kb":12078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,12,3],"acting":[6,12,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"49'32","reported_seq":129,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.693696+0000","last_change":"2024-09-13T10:10:59.365358+0000","last_active":"2024-09-13T10:11:12.693696+0000","last_peered":"2024-09-13T10:11:12.693696+0000","last_clean":"2024-09-13T10:11:12.693696+0000","last_became_active":"2024-09-13T10:10:59.364443+0000","last_became_peered":"2024-09-13T10:10:59.364443+0000","last_unstale":"2024-09-13T10:11:12.693696+0000","last_undegraded":"2024-09-13T10:11:12.693696+0000","last_fullsized":"2024-09-13T10:11:12.693696+0000","mapping_epoch":44,"log_start":"47'29","ondisk_log_start":"47'29","created":31,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":29,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:54.931956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2369787,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":47,"num_read_kb":41459,"num_write":27,"num_write_kb":7702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,12,7],"acting":[15,12,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.e","version":"46'10","reported_seq":97,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T10:11:12.698915+0000","last_change":"2024-09-13T10:10:44.218427+0000","last_active":"2024-09-13T10:11:12.698915+0000","last_peered":"2024-09-13T10:11:12.698915+0000","last_clean":"2024-09-13T10:11:12.698915+0000","last_became_active":"2024-09-13T10:10:44.217784+0000","last_became_peered":"2024-09-13T10:10:44.217784+0000","last_unstale":"2024-09-13T10:11:12.698915+0000","last_undegraded":"2024-09-13T10:11:12.698915+0000","last_fullsized":"2024-09-13T10:11:12.698915+0000","mapping_epoch":31,"log_start":"46'7","ondisk_log_start":"46'7","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":7,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:01.647136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3747928,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":33,"num_read_kb":29838,"num_write":9,"num_write_kb":2294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,8],"acting":[4,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"57'56","reported_seq":180,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T10:11:15.942202+0000","last_change":"2024-09-13T10:10:44.224347+0000","last_active":"2024-09-13T10:11:15.942202+0000","last_peered":"2024-09-13T10:11:15.942202+0000","last_clean":"2024-09-13T10:11:15.942202+0000","last_became_active":"2024-09-13T10:10:44.223700+0000","last_became_peered":"2024-09-13T10:10:44.223700+0000","last_unstale":"2024-09-13T10:11:15.942202+0000","last_undegraded":"2024-09-13T10:11:15.942202+0000","last_fullsized":"2024-09-13T10:11:15.942202+0000","mapping_epoch":31,"log_start":"56'53","ondisk_log_start":"56'53","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":53,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:03.389875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5596731,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":109441,"num_write":59,"num_write_kb":16433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,7],"acting":[13,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":26,"stat_sum":{"num_bytes":124909228,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":1010,"num_read_kb":852172,"num_write":438,"num_write_kb":120524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":25,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225148928,"data_stored":224311708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":78,"acting":78,"num_store_stats":16},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":61726763,"num_objects":109,"num_object_clones":0,"num_object_copies":327,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":573,"num_read_kb":10327,"num_write":801,"num_write_kb":189485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166014976,"data_stored":165447798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":11}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247820,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60532,"kb_used_data":33508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94311308,"statfs":{"total":96636764160,"available":96574779392,"internally_reserved":0,"allocated":34312192,"data_stored":33910365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247822,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37204,"kb_used_data":10180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334636,"statfs":{"total":96636764160,"available":96598667264,"internally_reserved":0,"allocated":10424320,"data_stored":10122054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247822,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65820,"kb_used_data":34636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306020,"statfs":{"total":96636764160,"available":96569364480,"internally_reserved":0,"allocated":35467264,"data_stored":35092927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313229,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":4820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":4935680,"data_stored":4649610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,5,6,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313229,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54692,"kb_used_data":27668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94317148,"statfs":{"total":96636764160,"available":96580759552,"internally_reserved":0,"allocated":28332032,"data_stored":27991470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,3,4,6,7,8,9,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247820,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49836,"kb_used_data":16892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94322004,"statfs":{"total":96636764160,"available":96585732096,"internally_reserved":0,"allocated":17297408,"data_stored":16922918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280524,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56488,"kb_used_data":24276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94315352,"statfs":{"total":96636764160,"available":96578920448,"internally_reserved":0,"allocated":24858624,"data_stored":24465986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,6,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280524,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60892,"kb_used_data":33868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94310948,"statfs":{"total":96636764160,"available":96574410752,"internally_reserved":0,"allocated":34680832,"data_stored":34294820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,6,7,8,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280525,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70516,"kb_used_data":43492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94301324,"statfs":{"total":96636764160,"available":96564555776,"internally_reserved":0,"allocated":44535808,"data_stored":44139144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,3,5,6,8,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280523,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46512,"kb_used_data":19488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94325328,"statfs":{"total":96636764160,"available":96589135872,"internally_reserved":0,"allocated":19955712,"data_stored":19644452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[3,4,6,7,9,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280524,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58276,"kb_used_data":31252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94313564,"statfs":{"total":96636764160,"available":96577089536,"internally_reserved":0,"allocated":32002048,"data_stored":31651513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280524,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65596,"kb_used_data":38568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94306244,"statfs":{"total":96636764160,"available":96569593856,"internally_reserved":0,"allocated":39493632,"data_stored":39011770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280524,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43372,"kb_used_data":12188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328468,"statfs":{"total":96636764160,"available":96592351232,"internally_reserved":0,"allocated":12480512,"data_stored":12184120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,5,7,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280524,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29288,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342552,"statfs":{"total":96636764160,"available":96606773248,"internally_reserved":0,"allocated":2318336,"data_stored":2043250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247820,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46300,"kb_used_data":19276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94325540,"statfs":{"total":96636764160,"available":96589352960,"internally_reserved":0,"allocated":19738624,"data_stored":19411182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247819,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75028,"kb_used_data":41840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94296812,"statfs":{"total":96636764160,"available":96559935488,"internally_reserved":0,"allocated":42844160,"data_stored":42312315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5545984,"data_stored":5509114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10375168,"data_stored":10329380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":27144192,"data_stored":27056260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":33738752,"data_stored":33646686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":18272256,"data_stored":18216975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":18673664,"data_stored":18615486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":10375168,"data_stored":10329380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":15982592,"data_stored":15925502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":25907200,"data_stored":25819015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11591680,"data_stored":11522560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13885440,"data_stored":13811879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19124224,"data_stored":19078186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6332416,"data_stored":6315215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":10563584,"data_stored":10530739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":19357696,"data_stored":19319725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40988672,"data_stored":40748374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12537856,"data_stored":12519443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":15577088,"data_stored":15545486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4345856,"data_stored":4333449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":28520448,"data_stored":28357663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":11882496,"data_stored":11859393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":9814016,"data_stored":9789077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":11112448,"data_stored":11103623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":7794688,"data_stored":7758373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":1720320,"data_stored":1718523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:11:17.854 INFO:tasks.thrashosds.thrasher:Setting 4.e to [8, 3, 15] 2024-09-13T10:11:17.854 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '4.e', '8', '3', '15'] 2024-09-13T10:11:17.854 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 4.e 8 3 15 2024-09-13T10:11:17.881 INFO:tasks.rados.rados.0.smithi195.stdout:1072: finishing write tid 3 to smithi19530714-20 2024-09-13T10:11:17.881 INFO:tasks.rados.rados.0.smithi195.stdout:1072: finishing write tid 4 to smithi19530714-20 2024-09-13T10:11:17.881 INFO:tasks.rados.rados.0.smithi195.stdout:1072: finishing write tid 5 to smithi19530714-20 2024-09-13T10:11:17.881 INFO:tasks.rados.rados.0.smithi195.stdout:1072: oid 20 updating version 0 to 51 2024-09-13T10:11:17.881 INFO:tasks.rados.rados.0.smithi195.stdout:1072: oid 20 updating version 51 to 52 2024-09-13T10:11:17.881 INFO:tasks.rados.rados.0.smithi195.stdout:1072: oid 20 updating version 52 to 53 2024-09-13T10:11:17.881 INFO:tasks.rados.rados.0.smithi195.stdout:1072: oid 20 updating version 53 to 54 2024-09-13T10:11:17.881 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 54 (ObjNum 192 snap 0 seq_num 192) dirty exists 2024-09-13T10:11:17.881 INFO:tasks.rados.rados.0.smithi195.stdout:1072: left oid 20 (ObjNum 192 snap 0 seq_num 192) 2024-09-13T10:11:17.881 INFO:tasks.rados.rados.0.smithi195.stdout:1077: finishing write tid 1 to smithi19530714-38 2024-09-13T10:11:17.882 INFO:tasks.rados.rados.0.smithi195.stdout:1078: expect (ObjNum 141 snap 0 seq_num 141) 2024-09-13T10:11:17.882 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:17.937 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:17.937+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:17.984 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 843561 2024-09-13T10:11:17.991 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:17.991 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:18.002 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:18.002 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:11:18.002 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:11:18.002 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:11:18.002 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.002 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:251 4.5 4:a71ca386:::smithi19530714-2:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:18.002 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:11:18.003 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.883090023000001, 2024-09-13T10:11:18.003 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041813326999999997, 2024-09-13T10:11:18.003 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.003 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.003 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:18.003 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.003 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.003 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.003 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 251 2024-09-13T10:11:18.003 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.003 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.003 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.004 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.004 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:11:18.004 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.004 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.004 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.004 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.004 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:11:18.004 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.004 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.004 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.004 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.004 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117990+0000", 2024-09-13T10:11:18.005 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.203e-06 2024-09-13T10:11:18.005 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.005 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.005 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.005 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117997+0000", 2024-09-13T10:11:18.005 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9750000000000001e-06 2024-09-13T10:11:18.005 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.005 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.005 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.005 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118009+0000", 2024-09-13T10:11:18.005 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2478e-05 2024-09-13T10:11:18.005 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.006 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.006 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.006 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118017+0000", 2024-09-13T10:11:18.006 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6909999999999997e-06 2024-09-13T10:11:18.006 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.006 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.006 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.006 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118106+0000", 2024-09-13T10:11:18.006 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.9331000000000002e-05 2024-09-13T10:11:18.006 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.006 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.006 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.006 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118184+0000", 2024-09-13T10:11:18.007 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7448999999999997e-05 2024-09-13T10:11:18.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.007 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.007 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,10", 2024-09-13T10:11:18.007 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158604+0000", 2024-09-13T10:11:18.007 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.040419954000000001 2024-09-13T10:11:18.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.007 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.007 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:18.007 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159306+0000", 2024-09-13T10:11:18.007 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00070229299999999999 2024-09-13T10:11:18.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.007 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.008 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.008 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159539+0000", 2024-09-13T10:11:18.008 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023353600000000001 2024-09-13T10:11:18.008 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.008 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.008 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.008 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159696+0000", 2024-09-13T10:11:18.008 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015616 2024-09-13T10:11:18.008 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.008 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.008 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:18.008 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159754+0000", 2024-09-13T10:11:18.008 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8513e-05 2024-09-13T10:11:18.009 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.009 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.009 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.009 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159802+0000", 2024-09-13T10:11:18.009 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.7744000000000002e-05 2024-09-13T10:11:18.009 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.009 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.009 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.009 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.009 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.009 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:270 4.5 4:a59e88f7:::smithi19530714-21:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:18.009 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:11:18.010 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.775043897, 2024-09-13T10:11:18.010 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041472594000000002, 2024-09-13T10:11:18.010 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.010 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.010 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:18.010 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.010 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.010 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.010 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 270 2024-09-13T10:11:18.010 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.010 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.010 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.010 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.011 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:11:18.011 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.011 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.011 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.011 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.011 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:11:18.011 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.011 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.011 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.011 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.011 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226037+0000", 2024-09-13T10:11:18.011 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6560000000000001e-06 2024-09-13T10:11:18.011 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.012 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.012 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.012 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226048+0000", 2024-09-13T10:11:18.012 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0893000000000001e-05 2024-09-13T10:11:18.012 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.012 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.012 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.012 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226079+0000", 2024-09-13T10:11:18.012 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0412999999999998e-05 2024-09-13T10:11:18.012 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.012 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.012 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.012 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226106+0000", 2024-09-13T10:11:18.013 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7160000000000001e-05 2024-09-13T10:11:18.013 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.013 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.013 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.013 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226234+0000", 2024-09-13T10:11:18.013 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000128393 2024-09-13T10:11:18.013 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.013 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.013 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.013 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226388+0000", 2024-09-13T10:11:18.013 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015424300000000001 2024-09-13T10:11:18.013 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.013 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.014 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,10", 2024-09-13T10:11:18.014 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262647+0000", 2024-09-13T10:11:18.014 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036258460999999999 2024-09-13T10:11:18.014 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.014 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.014 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.014 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264690+0000", 2024-09-13T10:11:18.014 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020429889999999998 2024-09-13T10:11:18.014 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.014 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.014 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:18.014 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.266720+0000", 2024-09-13T10:11:18.014 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002029669 2024-09-13T10:11:18.015 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.015 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.015 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.015 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267350+0000", 2024-09-13T10:11:18.015 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00063076699999999996 2024-09-13T10:11:18.015 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.015 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.015 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:18.015 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267409+0000", 2024-09-13T10:11:18.015 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8366000000000003e-05 2024-09-13T10:11:18.015 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.015 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.015 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.016 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267507+0000", 2024-09-13T10:11:18.016 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.8584000000000003e-05 2024-09-13T10:11:18.016 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.016 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.016 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.016 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.016 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.016 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:275 4.2 4:4ac406d9:::smithi19530714-26:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:18.016 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:11:18.016 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.771296834000001, 2024-09-13T10:11:18.016 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024603505000000001, 2024-09-13T10:11:18.016 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.016 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.017 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:18.017 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.017 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.017 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.017 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 275 2024-09-13T10:11:18.017 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.017 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.017 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.017 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.017 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:11:18.017 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.017 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.017 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.018 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.018 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:11:18.018 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.018 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.018 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.018 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.018 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229783+0000", 2024-09-13T10:11:18.018 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.44e-06 2024-09-13T10:11:18.018 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.018 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.018 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.018 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229793+0000", 2024-09-13T10:11:18.018 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0163000000000001e-05 2024-09-13T10:11:18.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.019 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.019 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.019 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229814+0000", 2024-09-13T10:11:18.019 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0564000000000002e-05 2024-09-13T10:11:18.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.019 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.019 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.019 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229831+0000", 2024-09-13T10:11:18.019 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7142e-05 2024-09-13T10:11:18.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.019 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.019 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.020 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229949+0000", 2024-09-13T10:11:18.020 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000118014 2024-09-13T10:11:18.020 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.020 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.020 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.020 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230081+0000", 2024-09-13T10:11:18.020 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013181299999999999 2024-09-13T10:11:18.020 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.020 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.020 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,14", 2024-09-13T10:11:18.020 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.236541+0000", 2024-09-13T10:11:18.020 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0064597680000000003 2024-09-13T10:11:18.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.021 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.021 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.021 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.238773+0000", 2024-09-13T10:11:18.021 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022326680000000002 2024-09-13T10:11:18.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.021 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.021 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:18.021 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.241978+0000", 2024-09-13T10:11:18.021 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00320455 2024-09-13T10:11:18.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.021 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.021 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.022 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254159+0000", 2024-09-13T10:11:18.022 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012181149 2024-09-13T10:11:18.022 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.022 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.022 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:18.022 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254259+0000", 2024-09-13T10:11:18.022 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000100184 2024-09-13T10:11:18.022 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.022 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.022 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.022 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254385+0000", 2024-09-13T10:11:18.022 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012605 2024-09-13T10:11:18.022 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.023 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.023 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.023 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:270 4.5 4:a59e88f7:::smithi19530714-21:head [assert-version v9,set-chunk] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:18.023 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:11:18.023 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.767642404, 2024-09-13T10:11:18.023 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033985986000000003, 2024-09-13T10:11:18.023 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.023 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.023 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "waiting for ondisk", 2024-09-13T10:11:18.023 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.023 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.023 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.024 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 270 2024-09-13T10:11:18.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.024 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.024 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.024 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.024 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:11:18.024 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.024 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.024 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.024 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:11:18.024 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.025 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.025 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.025 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233438+0000", 2024-09-13T10:11:18.025 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.187e-06 2024-09-13T10:11:18.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.025 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.025 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.025 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233449+0000", 2024-09-13T10:11:18.025 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0926e-05 2024-09-13T10:11:18.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.025 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.025 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.026 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233477+0000", 2024-09-13T10:11:18.026 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7515999999999999e-05 2024-09-13T10:11:18.026 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.026 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.026 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.026 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233499+0000", 2024-09-13T10:11:18.026 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1719999999999999e-05 2024-09-13T10:11:18.026 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.026 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.026 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.026 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233670+0000", 2024-09-13T10:11:18.026 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000171146 2024-09-13T10:11:18.027 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.027 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.027 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for blocked object", 2024-09-13T10:11:18.027 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233823+0000", 2024-09-13T10:11:18.027 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015309300000000001 2024-09-13T10:11:18.027 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.027 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.027 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.027 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.263191+0000", 2024-09-13T10:11:18.027 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029368157999999998 2024-09-13T10:11:18.027 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.027 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.027 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for ondisk", 2024-09-13T10:11:18.028 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.263301+0000", 2024-09-13T10:11:18.028 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110214 2024-09-13T10:11:18.028 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.028 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.028 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.028 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267422+0000", 2024-09-13T10:11:18.028 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0041210259999999999 2024-09-13T10:11:18.028 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.029 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.029 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.029 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.029 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:275 4.2 4:4ac406d9:::smithi19530714-26:head [assert-version v3,set-chunk] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:18.029 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.233895+0000", 2024-09-13T10:11:18.029 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.767184018000002, 2024-09-13T10:11:18.029 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020388462, 2024-09-13T10:11:18.029 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.029 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.029 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "waiting for ondisk", 2024-09-13T10:11:18.029 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.030 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.030 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.030 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 275 2024-09-13T10:11:18.030 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.030 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.030 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.030 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.030 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233895+0000", 2024-09-13T10:11:18.030 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.030 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.030 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.030 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.030 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233895+0000", 2024-09-13T10:11:18.031 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.031 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.031 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.031 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233896+0000", 2024-09-13T10:11:18.031 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3230000000000001e-06 2024-09-13T10:11:18.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.031 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.031 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.031 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233904+0000", 2024-09-13T10:11:18.031 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6930000000000007e-06 2024-09-13T10:11:18.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.032 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.032 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.032 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233921+0000", 2024-09-13T10:11:18.032 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7739000000000001e-05 2024-09-13T10:11:18.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.032 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.032 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.032 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233937+0000", 2024-09-13T10:11:18.032 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5444e-05 2024-09-13T10:11:18.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.032 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.032 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.033 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234067+0000", 2024-09-13T10:11:18.033 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013029 2024-09-13T10:11:18.033 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.033 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.033 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for blocked object", 2024-09-13T10:11:18.033 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234173+0000", 2024-09-13T10:11:18.033 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010631 2024-09-13T10:11:18.033 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.033 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.033 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.033 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.237380+0000", 2024-09-13T10:11:18.033 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0032067369999999999 2024-09-13T10:11:18.034 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.034 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.034 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for ondisk", 2024-09-13T10:11:18.034 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.237533+0000", 2024-09-13T10:11:18.034 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000152748 2024-09-13T10:11:18.034 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.034 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.034 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.034 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254283+0000", 2024-09-13T10:11:18.034 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016750178000000001 2024-09-13T10:11:18.034 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.034 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.034 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.035 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.035 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.035 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:277 4.4 4:20c7201f:::smithi19530714-28:head [assert-version v9,set-chunk] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:18.035 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:11:18.035 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.767036318999999, 2024-09-13T10:11:18.035 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.042430906999999997, 2024-09-13T10:11:18.035 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.035 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.035 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:18.035 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.035 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.035 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.036 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 277 2024-09-13T10:11:18.036 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.036 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.036 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.036 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.036 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:11:18.036 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.036 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.036 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.036 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.036 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:11:18.036 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.037 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.037 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.037 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.037 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234043+0000", 2024-09-13T10:11:18.037 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.13e-06 2024-09-13T10:11:18.037 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.037 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.037 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.037 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234050+0000", 2024-09-13T10:11:18.037 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.263e-06 2024-09-13T10:11:18.037 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.037 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.038 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.038 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234065+0000", 2024-09-13T10:11:18.038 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5207e-05 2024-09-13T10:11:18.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.038 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.038 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.038 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234078+0000", 2024-09-13T10:11:18.038 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3643e-05 2024-09-13T10:11:18.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.038 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.038 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.038 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.235383+0000", 2024-09-13T10:11:18.038 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0013048269999999999 2024-09-13T10:11:18.039 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.039 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.039 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.039 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.235604+0000", 2024-09-13T10:11:18.039 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022116900000000001 2024-09-13T10:11:18.039 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.039 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.039 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:18.039 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.271455+0000", 2024-09-13T10:11:18.039 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035850173999999999 2024-09-13T10:11:18.039 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.039 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.039 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.040 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273083+0000", 2024-09-13T10:11:18.040 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0016287649999999999 2024-09-13T10:11:18.040 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.040 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.040 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:18.040 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.275756+0000", 2024-09-13T10:11:18.040 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0026721140000000002 2024-09-13T10:11:18.040 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.040 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.040 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.040 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276255+0000", 2024-09-13T10:11:18.040 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00049961600000000004 2024-09-13T10:11:18.040 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.041 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.041 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:18.041 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276401+0000", 2024-09-13T10:11:18.041 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014584099999999999 2024-09-13T10:11:18.041 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.041 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.041 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.041 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276473+0000", 2024-09-13T10:11:18.041 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2157999999999996e-05 2024-09-13T10:11:18.041 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.041 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.041 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.041 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.042 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.042 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:294 4.4 4:25c904b0:::smithi19530714-45:head [assert-version v15,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:18.042 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.260461+0000", 2024-09-13T10:11:18.042 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.740617490999998, 2024-09-13T10:11:18.042 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020509151, 2024-09-13T10:11:18.042 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.042 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.042 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:18.042 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.042 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.042 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.042 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 294 2024-09-13T10:11:18.042 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.043 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.043 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.043 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.043 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260461+0000", 2024-09-13T10:11:18.043 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.043 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.043 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.043 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.043 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260461+0000", 2024-09-13T10:11:18.043 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.043 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.043 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.044 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.044 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260464+0000", 2024-09-13T10:11:18.044 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4279999999999999e-06 2024-09-13T10:11:18.044 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.044 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.044 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.044 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260474+0000", 2024-09-13T10:11:18.044 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.031e-05 2024-09-13T10:11:18.044 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.044 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.044 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.045 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260494+0000", 2024-09-13T10:11:18.045 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9894e-05 2024-09-13T10:11:18.045 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.045 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.045 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.045 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260509+0000", 2024-09-13T10:11:18.045 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5135e-05 2024-09-13T10:11:18.045 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.045 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.045 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.045 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260654+0000", 2024-09-13T10:11:18.045 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014543100000000001 2024-09-13T10:11:18.046 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.046 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.046 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.046 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260980+0000", 2024-09-13T10:11:18.046 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00032611599999999999 2024-09-13T10:11:18.046 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.046 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.046 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:18.046 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.274012+0000", 2024-09-13T10:11:18.046 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013031205000000001 2024-09-13T10:11:18.046 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.046 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.047 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.047 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.275836+0000", 2024-09-13T10:11:18.047 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0018243630000000001 2024-09-13T10:11:18.047 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.047 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.047 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:18.047 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280160+0000", 2024-09-13T10:11:18.047 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0043236020000000002 2024-09-13T10:11:18.047 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.047 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.047 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.047 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280818+0000", 2024-09-13T10:11:18.048 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00065824600000000003 2024-09-13T10:11:18.048 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.048 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.048 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:18.048 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280897+0000", 2024-09-13T10:11:18.048 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9623000000000007e-05 2024-09-13T10:11:18.048 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.048 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.048 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.048 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280970+0000", 2024-09-13T10:11:18.048 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2798000000000003e-05 2024-09-13T10:11:18.048 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.049 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.049 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.049 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.049 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.049 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:298 4.4 4:24da8222:::smithi19530714-49:head [assert-version v18,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:18.049 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.262102+0000", 2024-09-13T10:11:18.049 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.738976935, 2024-09-13T10:11:18.049 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023401737999999998, 2024-09-13T10:11:18.049 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.049 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.049 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:18.049 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.050 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.050 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.050 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 298 2024-09-13T10:11:18.050 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.050 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.050 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.050 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.050 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262102+0000", 2024-09-13T10:11:18.050 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.050 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.050 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.051 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.051 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262102+0000", 2024-09-13T10:11:18.051 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.051 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.051 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.051 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.051 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262103+0000", 2024-09-13T10:11:18.051 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7719999999999999e-06 2024-09-13T10:11:18.051 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.051 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.051 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.051 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262112+0000", 2024-09-13T10:11:18.051 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.242e-06 2024-09-13T10:11:18.052 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.052 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.052 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.052 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262130+0000", 2024-09-13T10:11:18.052 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8057999999999999e-05 2024-09-13T10:11:18.052 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.052 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.052 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.052 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262141+0000", 2024-09-13T10:11:18.052 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0893000000000001e-05 2024-09-13T10:11:18.052 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.052 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.053 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.053 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262226+0000", 2024-09-13T10:11:18.053 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5694999999999999e-05 2024-09-13T10:11:18.053 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.053 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.053 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.053 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262321+0000", 2024-09-13T10:11:18.053 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.4394999999999993e-05 2024-09-13T10:11:18.053 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.053 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.053 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:18.053 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.274901+0000", 2024-09-13T10:11:18.053 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012580071999999999 2024-09-13T10:11:18.054 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.054 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.054 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.054 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276773+0000", 2024-09-13T10:11:18.054 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0018726579999999999 2024-09-13T10:11:18.054 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.054 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.054 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:18.054 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283725+0000", 2024-09-13T10:11:18.054 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0069512460000000003 2024-09-13T10:11:18.054 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.054 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.055 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.055 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.285294+0000", 2024-09-13T10:11:18.055 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0015694859999999999 2024-09-13T10:11:18.055 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.055 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.055 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:18.055 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.285420+0000", 2024-09-13T10:11:18.055 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012607699999999999 2024-09-13T10:11:18.055 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.055 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.055 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.055 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.285503+0000", 2024-09-13T10:11:18.056 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.3144000000000002e-05 2024-09-13T10:11:18.056 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.056 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.056 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.056 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.056 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.056 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:324 4.4 4:26b855f3:::smithi19530714-25:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:18.056 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:11:18.056 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.720374072999999, 2024-09-13T10:11:18.056 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033888489000000001, 2024-09-13T10:11:18.056 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.056 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.056 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:18.057 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.057 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.057 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.057 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 324 2024-09-13T10:11:18.057 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.057 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.057 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.057 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.057 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:11:18.057 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.057 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.058 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.058 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.058 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:11:18.058 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.058 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.058 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.058 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.058 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280706+0000", 2024-09-13T10:11:18.058 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6479999999999999e-06 2024-09-13T10:11:18.058 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.058 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.058 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.058 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280715+0000", 2024-09-13T10:11:18.059 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1500000000000005e-06 2024-09-13T10:11:18.059 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.059 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.059 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.059 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280733+0000", 2024-09-13T10:11:18.059 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8168e-05 2024-09-13T10:11:18.059 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.059 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.059 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.059 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280747+0000", 2024-09-13T10:11:18.059 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3661e-05 2024-09-13T10:11:18.059 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.059 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.060 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.060 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281013+0000", 2024-09-13T10:11:18.060 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026565099999999999 2024-09-13T10:11:18.060 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.060 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.060 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.060 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281124+0000", 2024-09-13T10:11:18.060 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000111715 2024-09-13T10:11:18.060 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.060 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.061 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:18.061 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286502+0000", 2024-09-13T10:11:18.061 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0053775259999999997 2024-09-13T10:11:18.061 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.061 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.061 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:18.061 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.289334+0000", 2024-09-13T10:11:18.061 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0028316779999999998 2024-09-13T10:11:18.061 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.061 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.062 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.062 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.290331+0000", 2024-09-13T10:11:18.062 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0009974929999999999 2024-09-13T10:11:18.062 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.062 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.062 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.062 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314453+0000", 2024-09-13T10:11:18.062 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024121417999999999 2024-09-13T10:11:18.062 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.062 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.062 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:18.062 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314531+0000", 2024-09-13T10:11:18.062 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8841e-05 2024-09-13T10:11:18.063 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.063 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.063 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.063 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314593+0000", 2024-09-13T10:11:18.063 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.1539999999999997e-05 2024-09-13T10:11:18.063 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.063 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.063 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.063 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.063 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.063 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:327 4.4 4:20c7201f:::smithi19530714-28:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:18.063 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:11:18.064 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.719906244000001, 2024-09-13T10:11:18.064 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035855512999999999, 2024-09-13T10:11:18.064 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.064 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.064 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:18.064 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.064 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.064 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.064 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 327 2024-09-13T10:11:18.064 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.064 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.064 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.065 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.065 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:11:18.065 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.065 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.065 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.065 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.065 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:11:18.065 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.065 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.065 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.065 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.065 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281173+0000", 2024-09-13T10:11:18.066 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0020000000000001e-06 2024-09-13T10:11:18.066 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.066 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.066 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.066 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281180+0000", 2024-09-13T10:11:18.066 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.8260000000000004e-06 2024-09-13T10:11:18.066 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.066 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.066 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.066 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281192+0000", 2024-09-13T10:11:18.066 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1725e-05 2024-09-13T10:11:18.066 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.066 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.067 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.067 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281202+0000", 2024-09-13T10:11:18.067 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0492000000000001e-05 2024-09-13T10:11:18.067 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.067 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.067 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.067 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281396+0000", 2024-09-13T10:11:18.067 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019404200000000001 2024-09-13T10:11:18.067 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.067 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.067 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.067 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281533+0000", 2024-09-13T10:11:18.067 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000136267 2024-09-13T10:11:18.068 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.068 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.068 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:18.068 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.313834+0000", 2024-09-13T10:11:18.068 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032301491000000002 2024-09-13T10:11:18.068 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.068 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.068 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.068 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314993+0000", 2024-09-13T10:11:18.068 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001158585 2024-09-13T10:11:18.068 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.068 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.068 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:18.069 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.315097+0000", 2024-09-13T10:11:18.069 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000104685 2024-09-13T10:11:18.069 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.069 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.069 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.069 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.316808+0000", 2024-09-13T10:11:18.069 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0017110529999999999 2024-09-13T10:11:18.069 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.069 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.069 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:18.069 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.316946+0000", 2024-09-13T10:11:18.069 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000137737 2024-09-13T10:11:18.070 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.070 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.070 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.070 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.317028+0000", 2024-09-13T10:11:18.070 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.1607999999999995e-05 2024-09-13T10:11:18.070 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.070 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.070 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.070 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.070 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.070 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:335 4.2 4:4a24ecac:::smithi19530714-36:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:18.070 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:11:18.070 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.718587498000002, 2024-09-13T10:11:18.071 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032260373000000002, 2024-09-13T10:11:18.071 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.071 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.071 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:18.071 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.071 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.071 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.071 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 335 2024-09-13T10:11:18.071 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.071 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.071 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.071 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.071 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:11:18.072 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.072 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.072 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.072 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.072 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:11:18.072 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.072 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.072 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.072 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.072 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282494+0000", 2024-09-13T10:11:18.072 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5390000000000002e-06 2024-09-13T10:11:18.072 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.072 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.073 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.073 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282506+0000", 2024-09-13T10:11:18.073 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2646000000000001e-05 2024-09-13T10:11:18.073 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.073 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.073 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.073 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282539+0000", 2024-09-13T10:11:18.073 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3176000000000002e-05 2024-09-13T10:11:18.073 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.073 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.073 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.073 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282558+0000", 2024-09-13T10:11:18.073 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9065000000000001e-05 2024-09-13T10:11:18.074 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.074 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.074 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.074 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282697+0000", 2024-09-13T10:11:18.074 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013857999999999999 2024-09-13T10:11:18.074 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.074 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.074 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.074 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282874+0000", 2024-09-13T10:11:18.074 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017656900000000001 2024-09-13T10:11:18.074 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.074 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.074 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,14", 2024-09-13T10:11:18.074 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.287911+0000", 2024-09-13T10:11:18.075 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0050369220000000001 2024-09-13T10:11:18.075 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.075 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.075 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.075 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.290491+0000", 2024-09-13T10:11:18.075 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025802500000000001 2024-09-13T10:11:18.075 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.075 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.075 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:18.075 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.291374+0000", 2024-09-13T10:11:18.075 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00088367800000000005 2024-09-13T10:11:18.075 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.075 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.076 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.076 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314557+0000", 2024-09-13T10:11:18.076 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023182094 2024-09-13T10:11:18.076 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.076 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.076 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:18.076 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314664+0000", 2024-09-13T10:11:18.076 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010751099999999999 2024-09-13T10:11:18.076 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.076 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.076 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.076 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314751+0000", 2024-09-13T10:11:18.076 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7342999999999994e-05 2024-09-13T10:11:18.077 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.077 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.077 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.077 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.077 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.077 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:337 4.4 4:24275569:::smithi19530714-38:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:18.077 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:11:18.077 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.717930946999999, 2024-09-13T10:11:18.077 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.031660147, 2024-09-13T10:11:18.077 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.077 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.077 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:18.077 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.078 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.078 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.078 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 337 2024-09-13T10:11:18.078 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.078 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.078 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.078 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.078 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:11:18.078 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.078 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.078 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.079 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.079 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:11:18.079 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.079 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.079 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.079 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.079 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283149+0000", 2024-09-13T10:11:18.079 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.655e-06 2024-09-13T10:11:18.079 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.079 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.080 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.080 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283159+0000", 2024-09-13T10:11:18.080 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.9529999999999994e-06 2024-09-13T10:11:18.080 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.080 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.080 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.080 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283177+0000", 2024-09-13T10:11:18.080 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8168e-05 2024-09-13T10:11:18.080 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.080 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.080 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.080 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283193+0000", 2024-09-13T10:11:18.081 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5563e-05 2024-09-13T10:11:18.081 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.081 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.081 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.081 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283317+0000", 2024-09-13T10:11:18.081 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012421700000000001 2024-09-13T10:11:18.081 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.081 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.081 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.081 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283478+0000", 2024-09-13T10:11:18.081 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000160378 2024-09-13T10:11:18.081 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.081 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.082 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:18.082 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.289593+0000", 2024-09-13T10:11:18.082 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0061152200000000002 2024-09-13T10:11:18.082 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.082 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.082 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.082 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.291636+0000", 2024-09-13T10:11:18.082 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020435430000000001 2024-09-13T10:11:18.082 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.082 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.082 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:18.082 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294186+0000", 2024-09-13T10:11:18.082 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025494760000000002 2024-09-13T10:11:18.083 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.083 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.083 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:18.083 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314672+0000", 2024-09-13T10:11:18.083 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020486392999999999 2024-09-13T10:11:18.083 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.083 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.083 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:18.083 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314741+0000", 2024-09-13T10:11:18.083 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9003000000000003e-05 2024-09-13T10:11:18.083 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.083 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.083 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.084 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314808+0000", 2024-09-13T10:11:18.084 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.6577999999999999e-05 2024-09-13T10:11:18.084 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.084 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.084 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.084 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.084 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.084 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1089 4.5 4:a7549961:::smithi19530714-27:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:18.084 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:56.657202+0000", 2024-09-13T10:11:18.084 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 21.343876391999999, 2024-09-13T10:11:18.084 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020240790000000002, 2024-09-13T10:11:18.084 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.084 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.084 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:18.085 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.085 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.085 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.085 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1089 2024-09-13T10:11:18.085 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.085 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.085 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.085 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.086 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657202+0000", 2024-09-13T10:11:18.086 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.086 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.086 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.086 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.086 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657202+0000", 2024-09-13T10:11:18.086 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.086 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.086 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.086 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.086 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657205+0000", 2024-09-13T10:11:18.086 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.373e-06 2024-09-13T10:11:18.087 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.087 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.087 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.087 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657215+0000", 2024-09-13T10:11:18.087 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0563e-05 2024-09-13T10:11:18.087 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.087 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.087 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.087 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657243+0000", 2024-09-13T10:11:18.087 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8065999999999999e-05 2024-09-13T10:11:18.087 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.087 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.087 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.088 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657259+0000", 2024-09-13T10:11:18.088 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5415999999999999e-05 2024-09-13T10:11:18.088 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.088 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.088 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.088 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657404+0000", 2024-09-13T10:11:18.088 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014504000000000001 2024-09-13T10:11:18.088 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.088 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.088 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.088 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657619+0000", 2024-09-13T10:11:18.088 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021532499999999999 2024-09-13T10:11:18.089 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.089 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.089 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.089 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.677443+0000", 2024-09-13T10:11:18.089 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.019824007000000001 2024-09-13T10:11:18.089 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.089 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.089 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.089 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.089 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.089 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1578 4.5 4:a59e88f7:::smithi19530714-21:head [read 826538~2333145,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:11:18.089 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:04.717141+0000", 2024-09-13T10:11:18.089 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 13.283938015, 2024-09-13T10:11:18.090 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016160997, 2024-09-13T10:11:18.090 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.090 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.090 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:18.090 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.090 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.090 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.090 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1578 2024-09-13T10:11:18.090 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.090 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.090 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.090 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.090 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717141+0000", 2024-09-13T10:11:18.091 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.091 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.091 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.091 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.091 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717141+0000", 2024-09-13T10:11:18.091 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.091 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.091 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.092 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.092 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717142+0000", 2024-09-13T10:11:18.092 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.516e-06 2024-09-13T10:11:18.092 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.092 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.092 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.092 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717152+0000", 2024-09-13T10:11:18.092 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5999999999999996e-06 2024-09-13T10:11:18.092 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.092 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.092 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.092 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717172+0000", 2024-09-13T10:11:18.092 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0157e-05 2024-09-13T10:11:18.093 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.093 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.093 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.093 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717188+0000", 2024-09-13T10:11:18.093 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6022e-05 2024-09-13T10:11:18.093 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.093 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.093 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.093 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.721146+0000", 2024-09-13T10:11:18.093 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0039581699999999996 2024-09-13T10:11:18.093 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.093 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.093 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.094 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.721384+0000", 2024-09-13T10:11:18.094 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023784199999999999 2024-09-13T10:11:18.094 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.094 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.094 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.094 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.733302+0000", 2024-09-13T10:11:18.094 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.01191769 2024-09-13T10:11:18.094 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.094 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.094 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.094 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.094 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.095 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1579 4.5 4:a59e88f7:::smithi19530714-21:head [sparse-read 1584598~750371] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:11:18.095 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:04.717221+0000", 2024-09-13T10:11:18.095 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 13.283857386999999, 2024-09-13T10:11:18.095 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016269826000000001, 2024-09-13T10:11:18.095 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.095 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.095 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:18.095 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.095 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.095 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.095 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1579 2024-09-13T10:11:18.095 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.096 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.096 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.096 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.096 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717221+0000", 2024-09-13T10:11:18.096 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.096 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.096 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.096 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.096 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717221+0000", 2024-09-13T10:11:18.097 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.097 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.097 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.097 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.097 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717223+0000", 2024-09-13T10:11:18.097 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6339999999999999e-06 2024-09-13T10:11:18.097 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.097 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.097 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.097 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717229+0000", 2024-09-13T10:11:18.097 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.2330000000000003e-06 2024-09-13T10:11:18.097 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.097 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.098 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.098 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717240+0000", 2024-09-13T10:11:18.098 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0606e-05 2024-09-13T10:11:18.098 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.098 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.098 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.098 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717253+0000", 2024-09-13T10:11:18.098 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3743e-05 2024-09-13T10:11:18.098 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.099 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.099 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.099 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.732724+0000", 2024-09-13T10:11:18.099 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.01547045 2024-09-13T10:11:18.099 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.099 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.099 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.099 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.732946+0000", 2024-09-13T10:11:18.099 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000221676 2024-09-13T10:11:18.099 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.099 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.099 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.099 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.733491+0000", 2024-09-13T10:11:18.100 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00054548400000000005 2024-09-13T10:11:18.100 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.100 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.100 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.100 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.100 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.100 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1865 4.12 4:4ac406d9:::smithi19530714-26:head [read 0~2050454,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:18.100 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.728406+0000", 2024-09-13T10:11:18.100 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 6.2726723719999997, 2024-09-13T10:11:18.100 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029384652000000001, 2024-09-13T10:11:18.100 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.100 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.101 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:18.101 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.101 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.101 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.101 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1865 2024-09-13T10:11:18.101 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.101 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.101 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.101 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.101 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728406+0000", 2024-09-13T10:11:18.101 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.101 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.101 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.102 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.102 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728406+0000", 2024-09-13T10:11:18.102 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.102 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.102 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.102 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.102 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728410+0000", 2024-09-13T10:11:18.102 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9129999999999996e-06 2024-09-13T10:11:18.102 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.102 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.102 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.102 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728423+0000", 2024-09-13T10:11:18.102 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2425999999999999e-05 2024-09-13T10:11:18.103 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.103 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.103 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.103 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728453+0000", 2024-09-13T10:11:18.103 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0673999999999998e-05 2024-09-13T10:11:18.103 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.103 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.103 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.103 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728478+0000", 2024-09-13T10:11:18.103 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5099000000000001e-05 2024-09-13T10:11:18.103 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.103 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.103 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.103 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730729+0000", 2024-09-13T10:11:18.104 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002251009 2024-09-13T10:11:18.104 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.104 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.104 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:11:18.104 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730758+0000", 2024-09-13T10:11:18.104 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8623e-05 2024-09-13T10:11:18.104 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.104 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.104 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.104 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.754517+0000", 2024-09-13T10:11:18.104 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023759199000000002 2024-09-13T10:11:18.104 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.104 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.105 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.105 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.754761+0000", 2024-09-13T10:11:18.105 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000243923 2024-09-13T10:11:18.105 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.105 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.105 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.105 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.757791+0000", 2024-09-13T10:11:18.105 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003029786 2024-09-13T10:11:18.105 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.105 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.105 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.105 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.106 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.106 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1866 4.12 4:4ac406d9:::smithi19530714-26:head [sparse-read 0~1216897] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:18.106 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.728515+0000", 2024-09-13T10:11:18.106 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 6.2725633170000004, 2024-09-13T10:11:18.106 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.031206358999999999, 2024-09-13T10:11:18.106 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.106 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.106 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:18.106 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.106 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.106 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.107 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1866 2024-09-13T10:11:18.107 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.107 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.107 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.107 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.107 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728515+0000", 2024-09-13T10:11:18.107 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.107 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.107 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.107 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.107 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728515+0000", 2024-09-13T10:11:18.107 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.107 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.108 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.108 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.108 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728516+0000", 2024-09-13T10:11:18.108 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1429999999999999e-06 2024-09-13T10:11:18.108 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.108 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.108 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.108 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728522+0000", 2024-09-13T10:11:18.108 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5740000000000003e-06 2024-09-13T10:11:18.108 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.108 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.108 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.108 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728534+0000", 2024-09-13T10:11:18.108 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1539000000000001e-05 2024-09-13T10:11:18.109 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.109 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.109 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.109 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728543+0000", 2024-09-13T10:11:18.109 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6059999999999992e-06 2024-09-13T10:11:18.109 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.109 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.109 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.109 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730840+0000", 2024-09-13T10:11:18.109 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022966789999999998 2024-09-13T10:11:18.109 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.109 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.109 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:11:18.110 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730859+0000", 2024-09-13T10:11:18.110 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9613e-05 2024-09-13T10:11:18.110 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.110 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.110 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.110 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.758405+0000", 2024-09-13T10:11:18.110 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027545656000000002 2024-09-13T10:11:18.110 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.110 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.110 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.110 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.758574+0000", 2024-09-13T10:11:18.110 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000168808 2024-09-13T10:11:18.110 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.111 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.111 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.111 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759722+0000", 2024-09-13T10:11:18.111 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011477410000000001 2024-09-13T10:11:18.111 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.111 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.111 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.111 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.111 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.111 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1867 4.12 4:4ac406d9:::smithi19530714-26:head [read 116628~1933826,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:18.111 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.728557+0000", 2024-09-13T10:11:18.111 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 6.272521835, 2024-09-13T10:11:18.112 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033656772000000001, 2024-09-13T10:11:18.112 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.112 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.112 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:18.112 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.112 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.112 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.112 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1867 2024-09-13T10:11:18.112 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.112 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.112 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.112 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.113 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728557+0000", 2024-09-13T10:11:18.113 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.113 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.113 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.113 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.113 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728557+0000", 2024-09-13T10:11:18.113 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.113 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.113 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.113 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.113 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728558+0000", 2024-09-13T10:11:18.113 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8400000000000003e-07 2024-09-13T10:11:18.113 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.114 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.114 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.114 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728562+0000", 2024-09-13T10:11:18.114 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.8029999999999996e-06 2024-09-13T10:11:18.114 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.114 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.114 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.114 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728589+0000", 2024-09-13T10:11:18.114 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6536000000000001e-05 2024-09-13T10:11:18.114 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.114 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.114 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.114 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728603+0000", 2024-09-13T10:11:18.115 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4483e-05 2024-09-13T10:11:18.115 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.115 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.115 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.115 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730933+0000", 2024-09-13T10:11:18.115 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023295939999999999 2024-09-13T10:11:18.115 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.115 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.115 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:11:18.115 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730952+0000", 2024-09-13T10:11:18.115 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8614000000000002e-05 2024-09-13T10:11:18.115 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.115 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.116 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.116 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759881+0000", 2024-09-13T10:11:18.116 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.02892962 2024-09-13T10:11:18.116 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.116 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.116 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.116 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.760061+0000", 2024-09-13T10:11:18.116 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00018033 2024-09-13T10:11:18.116 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.116 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.116 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.116 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.762214+0000", 2024-09-13T10:11:18.117 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002152008 2024-09-13T10:11:18.117 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.117 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.117 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.117 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.117 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.117 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1932 4.5 4:a3098556:::smithi19530714-22:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:18.117 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.759351+0000", 2024-09-13T10:11:18.117 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 6.2417274220000003, 2024-09-13T10:11:18.117 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.043979880999999998, 2024-09-13T10:11:18.117 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.117 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.117 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:18.118 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.118 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.118 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.118 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1932 2024-09-13T10:11:18.118 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.118 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.118 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.118 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.118 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759351+0000", 2024-09-13T10:11:18.118 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.118 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.118 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.118 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.119 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759351+0000", 2024-09-13T10:11:18.119 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.119 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.119 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.119 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.119 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759352+0000", 2024-09-13T10:11:18.119 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.28e-06 2024-09-13T10:11:18.119 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.119 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.119 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.119 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759358+0000", 2024-09-13T10:11:18.119 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8699999999999997e-06 2024-09-13T10:11:18.120 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.120 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.120 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.120 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759376+0000", 2024-09-13T10:11:18.120 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7532000000000001e-05 2024-09-13T10:11:18.120 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.120 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.120 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.120 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759391+0000", 2024-09-13T10:11:18.120 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5590000000000002e-05 2024-09-13T10:11:18.120 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.120 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.121 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.121 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759607+0000", 2024-09-13T10:11:18.121 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021536999999999999 2024-09-13T10:11:18.121 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.121 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.121 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.121 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759831+0000", 2024-09-13T10:11:18.121 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000224243 2024-09-13T10:11:18.121 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.121 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.121 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.121 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.803331+0000", 2024-09-13T10:11:18.121 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.043499995999999999 2024-09-13T10:11:18.122 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.122 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.122 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.122 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.122 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.122 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2189 4.4 4:26b855f3:::smithi19530714-25:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:11:18.122 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:17.462885+0000", 2024-09-13T10:11:18.122 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 0.53819371599999999, 2024-09-13T10:11:18.122 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041082475, 2024-09-13T10:11:18.122 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:18.122 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:18.122 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:18.122 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:18.122 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:18.123 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:18.123 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2189 2024-09-13T10:11:18.123 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.123 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:18.123 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.123 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:18.123 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462885+0000", 2024-09-13T10:11:18.123 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.123 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.123 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.123 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:18.123 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462885+0000", 2024-09-13T10:11:18.123 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:18.124 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.124 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.124 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:18.124 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462888+0000", 2024-09-13T10:11:18.124 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.74e-06 2024-09-13T10:11:18.124 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.124 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.124 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:18.124 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462898+0000", 2024-09-13T10:11:18.124 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0618999999999999e-05 2024-09-13T10:11:18.124 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.124 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.124 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:18.125 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462928+0000", 2024-09-13T10:11:18.125 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9685000000000001e-05 2024-09-13T10:11:18.125 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.125 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.125 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:18.125 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462944+0000", 2024-09-13T10:11:18.125 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6175999999999999e-05 2024-09-13T10:11:18.125 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.125 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.125 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:18.125 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.463063+0000", 2024-09-13T10:11:18.125 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000119393 2024-09-13T10:11:18.125 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.126 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.126 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:18.126 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.463204+0000", 2024-09-13T10:11:18.126 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014020600000000001 2024-09-13T10:11:18.126 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:18.126 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:18.126 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:18.126 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.503967+0000", 2024-09-13T10:11:18.126 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.040763656000000002 2024-09-13T10:11:18.126 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.126 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.126 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.127 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:18.127 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:18.127 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:18.128 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:18.129 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:18.130 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:18.130 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:18.131 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-13T10:11:18.132 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:18.132 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:18.133 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:18.133 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:18.134 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:18.134 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:18.135 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:18.135 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:18.136 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:18.125+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:18.137 INFO:tasks.rados.rados.0.smithi195.stdout:1077: finishing write tid 2 to smithi19530714-38 2024-09-13T10:11:18.137 INFO:tasks.rados.rados.0.smithi195.stdout:1077: finishing write tid 3 to smithi19530714-38 2024-09-13T10:11:18.137 INFO:tasks.rados.rados.0.smithi195.stdout:1077: finishing write tid 4 to smithi19530714-38 2024-09-13T10:11:18.137 INFO:tasks.rados.rados.0.smithi195.stdout:1077: finishing write tid 5 to smithi19530714-38 2024-09-13T10:11:18.137 INFO:tasks.rados.rados.0.smithi195.stdout:1077: oid 38 updating version 0 to 102 2024-09-13T10:11:18.137 INFO:tasks.rados.rados.0.smithi195.stdout:1077: oid 38 updating version 102 to 103 2024-09-13T10:11:18.137 INFO:tasks.rados.rados.0.smithi195.stdout:1077: oid 38 updating version 103 to 104 2024-09-13T10:11:18.137 INFO:tasks.rados.rados.0.smithi195.stdout:1077: oid 38 updating version 104 to 105 2024-09-13T10:11:18.137 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 105 (ObjNum 193 snap 0 seq_num 193) dirty exists 2024-09-13T10:11:18.137 INFO:tasks.rados.rados.0.smithi195.stdout:1077: left oid 38 (ObjNum 193 snap 0 seq_num 193) 2024-09-13T10:11:18.137 INFO:tasks.rados.rados.0.smithi195.stdout:1079: finishing set_chunk to oid 19 2024-09-13T10:11:18.138 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 19 set_chunk returned error code -95 offset: 584704 length: 1572864 2024-09-13T10:11:18.138 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 0 (ObjNum 160 snap 0 seq_num 160) dirty exists 2024-09-13T10:11:18.138 INFO:tasks.rados.rados.0.smithi195.stdout:1080: got (0) Success 2024-09-13T10:11:18.138 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 0 (ObjNum 155 snap 0 seq_num 155) dirty exists 2024-09-13T10:11:18.138 INFO:tasks.rados.rados.0.smithi195.stdout:1072: done (8 left) 2024-09-13T10:11:18.138 INFO:tasks.rados.rados.0.smithi195.stdout:1073: done (7 left) 2024-09-13T10:11:18.138 INFO:tasks.rados.rados.0.smithi195.stdout:1074: done (6 left) 2024-09-13T10:11:18.138 INFO:tasks.rados.rados.0.smithi195.stdout:1075: done (5 left) 2024-09-13T10:11:18.138 INFO:tasks.rados.rados.0.smithi195.stdout:1076: done (4 left) 2024-09-13T10:11:18.138 INFO:tasks.rados.rados.0.smithi195.stdout:1077: done (3 left) 2024-09-13T10:11:18.138 INFO:tasks.rados.rados.0.smithi195.stdout:1078: done (2 left) 2024-09-13T10:11:18.138 INFO:tasks.rados.rados.0.smithi195.stdout:1079: done (1 left) 2024-09-13T10:11:18.138 INFO:tasks.rados.rados.0.smithi195.stdout:1080: done (0 left) 2024-09-13T10:11:18.139 INFO:tasks.rados.rados.0.smithi195.stdout:1081: read oid 38 snap -1 2024-09-13T10:11:18.139 INFO:tasks.rados.rados.0.smithi195.stdout:1081: expect (ObjNum 193 snap 0 seq_num 193) 2024-09-13T10:11:18.139 INFO:tasks.rados.rados.0.smithi195.stdout:1082: write oid 20 current snap is 0 2024-09-13T10:11:18.139 INFO:tasks.rados.rados.0.smithi195.stdout:1082: seq_num 194 ranges {460558=707131,1869389=508716,3094884=165674} 2024-09-13T10:11:18.139 INFO:tasks.rados.rados.0.smithi195.stdout:1082: writing smithi19530714-20 from 460558 to 1167689 tid 1 2024-09-13T10:11:18.139 INFO:tasks.rados.rados.0.smithi195.stdout:1082: writing smithi19530714-20 from 1869389 to 2378105 tid 2 2024-09-13T10:11:18.139 INFO:tasks.rados.rados.0.smithi195.stdout:1082: writing smithi19530714-20 from 3094884 to 3260558 tid 3 2024-09-13T10:11:18.139 INFO:tasks.rados.rados.0.smithi195.stdout:1083: read oid 4 snap -1 2024-09-13T10:11:18.139 INFO:tasks.rados.rados.0.smithi195.stdout:1083: expect (ObjNum 191 snap 0 seq_num 191) 2024-09-13T10:11:18.139 INFO:tasks.rados.rados.0.smithi195.stdout:1084: read oid 32 snap -1 2024-09-13T10:11:18.139 INFO:tasks.rados.rados.0.smithi195.stdout:1084: expect deleted 2024-09-13T10:11:18.139 INFO:tasks.rados.rados.0.smithi195.stdout:1085: write oid 11 current snap is 0 2024-09-13T10:11:18.140 INFO:tasks.rados.rados.0.smithi195.stdout:1085: seq_num 195 ranges {477365=782380,1927871=768727,3122307=155058} 2024-09-13T10:11:18.140 INFO:tasks.rados.rados.0.smithi195.stdout:1085: writing smithi19530714-11 from 477365 to 1259745 tid 1 2024-09-13T10:11:18.140 INFO:tasks.rados.rados.0.smithi195.stdout:1085: writing smithi19530714-11 from 1927871 to 2696598 tid 2 2024-09-13T10:11:18.140 INFO:tasks.rados.rados.0.smithi195.stdout:1085: writing smithi19530714-11 from 3122307 to 3277365 tid 3 2024-09-13T10:11:18.140 INFO:tasks.rados.rados.0.smithi195.stdout:1086: write oid 3 current snap is 0 2024-09-13T10:11:18.140 INFO:tasks.rados.rados.0.smithi195.stdout:1086: seq_num 196 ranges {494172=457629,1586353=512385,2749730=544442} 2024-09-13T10:11:18.140 INFO:tasks.rados.rados.0.smithi195.stdout:1086: writing smithi19530714-3 from 494172 to 951801 tid 1 2024-09-13T10:11:18.140 INFO:tasks.rados.rados.0.smithi195.stdout:1086: writing smithi19530714-3 from 1586353 to 2098738 tid 2 2024-09-13T10:11:18.140 INFO:tasks.rados.rados.0.smithi195.stdout:1086: writing smithi19530714-3 from 2749730 to 3294172 tid 3 2024-09-13T10:11:18.140 INFO:tasks.rados.rados.0.smithi195.stdout:1087: tier_flush oid 10 2024-09-13T10:11:18.140 INFO:tasks.rados.rados.0.smithi195.stdout:1087: tier_flush oid 10 snap -1 2024-09-13T10:11:18.140 INFO:tasks.rados.rados.0.smithi195.stdout:1088: read oid 34 snap -1 2024-09-13T10:11:18.140 INFO:tasks.rados.rados.0.smithi195.stdout:1088: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:18.141 INFO:tasks.rados.rados.0.smithi195.stdout:1089: read oid 43 snap -1 2024-09-13T10:11:18.141 INFO:tasks.rados.rados.0.smithi195.stdout:1089: expect (ObjNum 149 snap 0 seq_num 149) 2024-09-13T10:11:18.141 INFO:tasks.rados.rados.0.smithi195.stdout:1090: read oid 36 snap -1 2024-09-13T10:11:18.141 INFO:tasks.rados.rados.0.smithi195.stdout:1090: expect (ObjNum 190 snap 0 seq_num 190) 2024-09-13T10:11:18.141 INFO:tasks.rados.rados.0.smithi195.stdout:1091: read oid 18 snap -1 2024-09-13T10:11:18.141 INFO:tasks.rados.rados.0.smithi195.stdout:1091: expect (ObjNum 189 snap 0 seq_num 189) 2024-09-13T10:11:18.141 INFO:tasks.rados.rados.0.smithi195.stdout:1092: write oid 21 current snap is 0 2024-09-13T10:11:18.141 INFO:tasks.rados.rados.0.smithi195.stdout:1092: seq_num 197 ranges {510979=532878,1528482=772396,3060800=250179} 2024-09-13T10:11:18.141 INFO:tasks.rados.rados.0.smithi195.stdout:1092: writing smithi19530714-21 from 510979 to 1043857 tid 1 2024-09-13T10:11:18.141 INFO:tasks.rados.rados.0.smithi195.stdout:1092: writing smithi19530714-21 from 1528482 to 2300878 tid 2 2024-09-13T10:11:18.142 INFO:tasks.rados.rados.0.smithi195.stdout:1092: writing smithi19530714-21 from 3060800 to 3310979 tid 3 2024-09-13T10:11:18.143 INFO:tasks.rados.rados.0.smithi195.stdout:1093: read oid 45 snap -1 2024-09-13T10:11:18.143 INFO:tasks.rados.rados.0.smithi195.stdout:1093: expect (ObjNum 158 snap 0 seq_num 158) 2024-09-13T10:11:18.143 INFO:tasks.rados.rados.0.smithi195.stdout:1094: tier_flush oid 30 2024-09-13T10:11:18.143 INFO:tasks.rados.rados.0.smithi195.stdout:1094: tier_flush oid 30 snap -1 2024-09-13T10:11:18.143 INFO:tasks.rados.rados.0.smithi195.stdout:1081: expect (ObjNum 193 snap 0 seq_num 193) 2024-09-13T10:11:18.143 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:18.167 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:18.167 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:18.185 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:18.185 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:18.208 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:18.209 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:18.227 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:18.227 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:18.229 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:18.227+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:18.242 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1043945 2024-09-13T10:11:18.247 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:18.247 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:18.263 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:18.263 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:18.264 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:18.265 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:18.279 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-13T10:11:18.284 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:18.284 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:18.305 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:18.305 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:18.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:18.305+0000 7f993d105640 0 -- 172.21.15.73:0/111688174 >> v2:172.21.15.195:6805/848686366 conn(0x7f9938225d80 msgr2=0x7f9938228170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:11:18.329 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:18.329+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:18.349 INFO:tasks.rados.rados.0.smithi195.stdout:1082: finishing write tid 1 to smithi19530714-20 2024-09-13T10:11:18.349 INFO:tasks.rados.rados.0.smithi195.stdout:1082: finishing write tid 2 to smithi19530714-20 2024-09-13T10:11:18.349 INFO:tasks.rados.rados.0.smithi195.stdout:1081: done (13 left) 2024-09-13T10:11:18.349 INFO:tasks.rados.rados.0.smithi195.stdout:1095: read oid 39 snap -1 2024-09-13T10:11:18.349 INFO:tasks.rados.rados.0.smithi195.stdout:1095: expect (ObjNum 174 snap 0 seq_num 174) 2024-09-13T10:11:18.349 INFO:tasks.rados.rados.0.smithi195.stdout:1096: set_chunk oid 23 target oid 2024-09-13T10:11:18.357 INFO:tasks.rados.rados.0.smithi195.stdout:1096: set_chunk oid 23 offset: 680448 length: 558080 target oid 271808fc44c9bb143bb83931e904cedc0261cdc1b34bc4fd81a5954afb610855 offset: 0 snap -1 2024-09-13T10:11:18.357 INFO:tasks.rados.rados.0.smithi195.stdout:1082: finishing write tid 3 to smithi19530714-20 2024-09-13T10:11:18.357 INFO:tasks.rados.rados.0.smithi195.stdout:1082: finishing write tid 4 to smithi19530714-20 2024-09-13T10:11:18.357 INFO:tasks.rados.rados.0.smithi195.stdout:1082: finishing write tid 5 to smithi19530714-20 2024-09-13T10:11:18.357 INFO:tasks.rados.rados.0.smithi195.stdout:1082: oid 20 updating version 0 to 55 2024-09-13T10:11:18.357 INFO:tasks.rados.rados.0.smithi195.stdout:1082: oid 20 updating version 55 to 57 2024-09-13T10:11:18.357 INFO:tasks.rados.rados.0.smithi195.stdout:1082: oid 20 updating version 57 to 58 2024-09-13T10:11:18.357 INFO:tasks.rados.rados.0.smithi195.stdout:1082: oid 20 version 58 is already newer than 56 2024-09-13T10:11:18.357 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 58 (ObjNum 194 snap 0 seq_num 194) dirty exists 2024-09-13T10:11:18.357 INFO:tasks.rados.rados.0.smithi195.stdout:1082: left oid 20 (ObjNum 194 snap 0 seq_num 194) 2024-09-13T10:11:18.357 INFO:tasks.rados.rados.0.smithi195.stdout:1083: expect (ObjNum 191 snap 0 seq_num 191) 2024-09-13T10:11:18.358 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:18.415 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:18.431 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:18.427+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:18.441 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:18.441 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:18.456 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-13T10:11:18.531 INFO:tasks.rados.rados.0.smithi195.stdout:1085: finishing write tid 1 to smithi19530714-11 2024-09-13T10:11:18.531 INFO:tasks.rados.rados.0.smithi195.stdout:1097: read oid 2 snap -1 2024-09-13T10:11:18.531 INFO:tasks.rados.rados.0.smithi195.stdout:1097: expect (ObjNum 176 snap 0 seq_num 176) 2024-09-13T10:11:18.531 INFO:tasks.rados.rados.0.smithi195.stdout:1082: done (15 left) 2024-09-13T10:11:18.531 INFO:tasks.rados.rados.0.smithi195.stdout:1083: done (14 left) 2024-09-13T10:11:18.531 INFO:tasks.rados.rados.0.smithi195.stdout:1084: done (13 left) 2024-09-13T10:11:18.531 INFO:tasks.rados.rados.0.smithi195.stdout:1098: set_chunk oid 25 target oid 2024-09-13T10:11:18.532 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:18.527+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:18.532 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:18.533 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:18.546 INFO:tasks.rados.rados.0.smithi195.stdout:1098: set_chunk oid 25 offset: 1081856 length: 810496 target oid 7252c04ab1cec53e2550a4714ace830ec1bd464567cf22388bb4b34ee15b9d4e offset: 0 snap -1 2024-09-13T10:11:18.546 INFO:tasks.rados.rados.0.smithi195.stdout:1099: read oid 42 snap -1 2024-09-13T10:11:18.546 INFO:tasks.rados.rados.0.smithi195.stdout:1099: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-13T10:11:18.546 INFO:tasks.rados.rados.0.smithi195.stdout:1100: delete oid 9 current snap is 0 2024-09-13T10:11:18.546 INFO:tasks.rados.rados.0.smithi195.stdout:1085: finishing write tid 2 to smithi19530714-11 2024-09-13T10:11:18.546 INFO:tasks.rados.rados.0.smithi195.stdout:1085: finishing write tid 3 to smithi19530714-11 2024-09-13T10:11:18.546 INFO:tasks.rados.rados.0.smithi195.stdout:1085: finishing write tid 4 to smithi19530714-11 2024-09-13T10:11:18.546 INFO:tasks.rados.rados.0.smithi195.stdout:1085: finishing write tid 5 to smithi19530714-11 2024-09-13T10:11:18.546 INFO:tasks.rados.rados.0.smithi195.stdout:1085: oid 11 updating version 0 to 59 2024-09-13T10:11:18.546 INFO:tasks.rados.rados.0.smithi195.stdout:1085: oid 11 updating version 59 to 60 2024-09-13T10:11:18.547 INFO:tasks.rados.rados.0.smithi195.stdout:1085: oid 11 updating version 60 to 62 2024-09-13T10:11:18.547 INFO:tasks.rados.rados.0.smithi195.stdout:1085: oid 11 version 62 is already newer than 61 2024-09-13T10:11:18.547 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 62 (ObjNum 195 snap 0 seq_num 195) dirty exists 2024-09-13T10:11:18.547 INFO:tasks.rados.rados.0.smithi195.stdout:1085: left oid 11 (ObjNum 195 snap 0 seq_num 195) 2024-09-13T10:11:18.547 INFO:tasks.rados.rados.0.smithi195.stdout:1086: finishing write tid 1 to smithi19530714-3 2024-09-13T10:11:18.547 INFO:tasks.rados.rados.0.smithi195.stdout:1086: finishing write tid 2 to smithi19530714-3 2024-09-13T10:11:18.547 INFO:tasks.rados.rados.0.smithi195.stdout:1087: got (0) Success 2024-09-13T10:11:18.547 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 17 (ObjNum 84 snap 0 seq_num 84) dirty exists 2024-09-13T10:11:18.547 INFO:tasks.rados.rados.0.smithi195.stdout:1086: finishing write tid 3 to smithi19530714-3 2024-09-13T10:11:18.547 INFO:tasks.rados.rados.0.smithi195.stdout:1086: finishing write tid 4 to smithi19530714-3 2024-09-13T10:11:18.547 INFO:tasks.rados.rados.0.smithi195.stdout:1086: finishing write tid 5 to smithi19530714-3 2024-09-13T10:11:18.547 INFO:tasks.rados.rados.0.smithi195.stdout:1086: oid 3 updating version 0 to 50 2024-09-13T10:11:18.548 INFO:tasks.rados.rados.0.smithi195.stdout:1086: oid 3 updating version 50 to 53 2024-09-13T10:11:18.548 INFO:tasks.rados.rados.0.smithi195.stdout:1086: oid 3 version 53 is already newer than 51 2024-09-13T10:11:18.548 INFO:tasks.rados.rados.0.smithi195.stdout:1086: oid 3 version 53 is already newer than 52 2024-09-13T10:11:18.548 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 53 (ObjNum 196 snap 0 seq_num 196) dirty exists 2024-09-13T10:11:18.548 INFO:tasks.rados.rados.0.smithi195.stdout:1086: left oid 3 (ObjNum 196 snap 0 seq_num 196) 2024-09-13T10:11:18.548 INFO:tasks.rados.rados.0.smithi195.stdout:1088: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:18.583 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:18.584 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:18.598 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-13T10:11:18.632 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:18.629+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:18.650 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1670123 2024-09-13T10:11:18.668 INFO:tasks.rados.rados.0.smithi195.stdout:1089: expect (ObjNum 149 snap 0 seq_num 149) 2024-09-13T10:11:18.733 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:18.731+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:18.748 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-13T10:11:18.749 INFO:teuthology.orchestra.run.smithi073.stderr:set 4.e pg_upmap mapping to [8,3,15] 2024-09-13T10:11:18.761 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:18.767 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:18.767 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:18.782 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-13T10:11:18.811 INFO:tasks.rados.rados.0.smithi195.stdout:1090: expect (ObjNum 190 snap 0 seq_num 190) 2024-09-13T10:11:18.811 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:18.833 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:18.831+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:18.843 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 442243 2024-09-13T10:11:18.872 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:18.905 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:18.905 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:18.905 INFO:tasks.rados.rados.0.smithi195.stdout:1092: finishing write tid 1 to smithi19530714-21 2024-09-13T10:11:18.905 INFO:tasks.rados.rados.0.smithi195.stdout:1092: finishing write tid 2 to smithi19530714-21 2024-09-13T10:11:18.905 INFO:tasks.rados.rados.0.smithi195.stdout:1091: expect (ObjNum 189 snap 0 seq_num 189) 2024-09-13T10:11:18.905 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:18.918 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-13T10:11:18.935 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:18.933+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:18.959 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 488387 2024-09-13T10:11:19.035 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:19.033+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:19.056 INFO:tasks.rados.rados.0.smithi195.stdout:1092: finishing write tid 3 to smithi19530714-21 2024-09-13T10:11:19.056 INFO:tasks.rados.rados.0.smithi195.stdout:1092: finishing write tid 4 to smithi19530714-21 2024-09-13T10:11:19.056 INFO:tasks.rados.rados.0.smithi195.stdout:1085: done (15 left) 2024-09-13T10:11:19.056 INFO:tasks.rados.rados.0.smithi195.stdout:1086: done (14 left) 2024-09-13T10:11:19.056 INFO:tasks.rados.rados.0.smithi195.stdout:1087: done (13 left) 2024-09-13T10:11:19.056 INFO:tasks.rados.rados.0.smithi195.stdout:1088: done (12 left) 2024-09-13T10:11:19.057 INFO:tasks.rados.rados.0.smithi195.stdout:1089: done (11 left) 2024-09-13T10:11:19.057 INFO:tasks.rados.rados.0.smithi195.stdout:1090: done (10 left) 2024-09-13T10:11:19.057 INFO:tasks.rados.rados.0.smithi195.stdout:1091: done (9 left) 2024-09-13T10:11:19.057 INFO:tasks.rados.rados.0.smithi195.stdout:1100: done (8 left) 2024-09-13T10:11:19.057 INFO:tasks.rados.rados.0.smithi195.stdout:1101: delete oid 44 current snap is 0 2024-09-13T10:11:19.057 INFO:tasks.rados.rados.0.smithi195.stdout:1092: finishing write tid 5 to smithi19530714-21 2024-09-13T10:11:19.057 INFO:tasks.rados.rados.0.smithi195.stdout:1092: oid 21 updating version 0 to 94 2024-09-13T10:11:19.057 INFO:tasks.rados.rados.0.smithi195.stdout:1092: oid 21 updating version 94 to 95 2024-09-13T10:11:19.057 INFO:tasks.rados.rados.0.smithi195.stdout:1092: oid 21 updating version 95 to 97 2024-09-13T10:11:19.057 INFO:tasks.rados.rados.0.smithi195.stdout:1092: oid 21 version 97 is already newer than 96 2024-09-13T10:11:19.057 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 97 (ObjNum 197 snap 0 seq_num 197) dirty exists 2024-09-13T10:11:19.057 INFO:tasks.rados.rados.0.smithi195.stdout:1092: left oid 21 (ObjNum 197 snap 0 seq_num 197) 2024-09-13T10:11:19.057 INFO:tasks.rados.rados.0.smithi195.stdout:1093: expect (ObjNum 158 snap 0 seq_num 158) 2024-09-13T10:11:19.058 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:19.089 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:19.089 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:19.103 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T10:11:19.148 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:19.147+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:19.151 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 25170 2024-09-13T10:11:19.202 INFO:tasks.rados.rados.0.smithi195.stdout:1094: got (0) Success 2024-09-13T10:11:19.202 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 0 (ObjNum 102 snap 0 seq_num 102) dirty exists 2024-09-13T10:11:19.202 INFO:tasks.rados.rados.0.smithi195.stdout:1095: expect (ObjNum 174 snap 0 seq_num 174) 2024-09-13T10:11:19.226 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:19.226 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:19.237 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:19.235+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:19.239 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T10:11:19.251 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:19.338 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:19.337+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:19.346 INFO:tasks.rados.rados.0.smithi195.stdout:1096: finishing set_chunk to oid 23 2024-09-13T10:11:19.346 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 28 (ObjNum 90 snap 0 seq_num 90) dirty exists 2024-09-13T10:11:19.346 INFO:tasks.rados.rados.0.smithi195.stdout:1092: done (8 left) 2024-09-13T10:11:19.346 INFO:tasks.rados.rados.0.smithi195.stdout:1093: done (7 left) 2024-09-13T10:11:19.346 INFO:tasks.rados.rados.0.smithi195.stdout:1094: done (6 left) 2024-09-13T10:11:19.346 INFO:tasks.rados.rados.0.smithi195.stdout:1095: done (5 left) 2024-09-13T10:11:19.346 INFO:tasks.rados.rados.0.smithi195.stdout:1096: done (4 left) 2024-09-13T10:11:19.346 INFO:tasks.rados.rados.0.smithi195.stdout:1101: done (3 left) 2024-09-13T10:11:19.346 INFO:tasks.rados.rados.0.smithi195.stdout:1102: write oid 11 current snap is 0 2024-09-13T10:11:19.346 INFO:tasks.rados.rados.0.smithi195.stdout:1097: expect (ObjNum 176 snap 0 seq_num 176) 2024-09-13T10:11:19.368 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:19.369 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:19.382 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T10:11:19.439 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:19.437+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:19.490 INFO:tasks.rados.rados.0.smithi195.stdout:1098: finishing set_chunk to oid 25 2024-09-13T10:11:19.490 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 25 set_chunk returned error code -95 offset: 1081856 length: 810496 2024-09-13T10:11:19.490 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 0 (ObjNum 155 snap 0 seq_num 155) dirty exists 2024-09-13T10:11:19.490 INFO:tasks.rados.rados.0.smithi195.stdout:1102: seq_num 200 ranges {561400=474978,1820281=519723,2859422=457579} 2024-09-13T10:11:19.495 INFO:tasks.rados.rados.0.smithi195.stdout:1102: writing smithi19530714-11 from 561400 to 1036378 tid 1 2024-09-13T10:11:19.497 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:19.497 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:19.502 INFO:tasks.rados.rados.0.smithi195.stdout:1102: writing smithi19530714-11 from 1820281 to 2340004 tid 2 2024-09-13T10:11:19.508 INFO:tasks.rados.rados.0.smithi195.stdout:1102: writing smithi19530714-11 from 2859422 to 3317001 tid 3 2024-09-13T10:11:19.508 INFO:tasks.rados.rados.0.smithi195.stdout:1097: done (3 left) 2024-09-13T10:11:19.508 INFO:tasks.rados.rados.0.smithi195.stdout:1098: done (2 left) 2024-09-13T10:11:19.508 INFO:tasks.rados.rados.0.smithi195.stdout:1103: write oid 43 current snap is 0 2024-09-13T10:11:19.508 INFO:tasks.rados.rados.0.smithi195.stdout:1103: seq_num 201 ranges {578207=550227,1762410=779734,3286845=91362} 2024-09-13T10:11:19.511 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T10:11:19.514 INFO:tasks.rados.rados.0.smithi195.stdout:1103: writing smithi19530714-43 from 578207 to 1128434 tid 1 2024-09-13T10:11:19.523 INFO:tasks.rados.rados.0.smithi195.stdout:1103: writing smithi19530714-43 from 1762410 to 2542144 tid 2 2024-09-13T10:11:19.530 INFO:tasks.rados.rados.0.smithi195.stdout:1103: writing smithi19530714-43 from 3286845 to 3378207 tid 3 2024-09-13T10:11:19.530 INFO:tasks.rados.rados.0.smithi195.stdout:1104: read oid 1 snap -1 2024-09-13T10:11:19.530 INFO:tasks.rados.rados.0.smithi195.stdout:1104: expect (ObjNum 184 snap 0 seq_num 184) 2024-09-13T10:11:19.530 INFO:tasks.rados.rados.0.smithi195.stdout:1105: write oid 45 current snap is 0 2024-09-13T10:11:19.530 INFO:tasks.rados.rados.0.smithi195.stdout:1099: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-13T10:11:19.533 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1284545 2024-09-13T10:11:19.541 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:19.539+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:19.605 INFO:tasks.rados.rados.0.smithi195.stdout:1102: finishing write tid 1 to smithi19530714-11 2024-09-13T10:11:19.605 INFO:tasks.rados.rados.0.smithi195.stdout:1102: finishing write tid 2 to smithi19530714-11 2024-09-13T10:11:19.605 INFO:tasks.rados.rados.0.smithi195.stdout:1102: finishing write tid 3 to smithi19530714-11 2024-09-13T10:11:19.605 INFO:tasks.rados.rados.0.smithi195.stdout:1105: seq_num 202 ranges {595014=625476,1820892=523392,2797915=596476} 2024-09-13T10:11:19.610 INFO:tasks.rados.rados.0.smithi195.stdout:1105: writing smithi19530714-45 from 595014 to 1220490 tid 1 2024-09-13T10:11:19.616 INFO:tasks.rados.rados.0.smithi195.stdout:1105: writing smithi19530714-45 from 1820892 to 2344284 tid 2 2024-09-13T10:11:19.622 INFO:tasks.rados.rados.0.smithi195.stdout:1105: writing smithi19530714-45 from 2797915 to 3394391 tid 3 2024-09-13T10:11:19.623 INFO:tasks.rados.rados.0.smithi195.stdout:1099: done (4 left) 2024-09-13T10:11:19.623 INFO:tasks.rados.rados.0.smithi195.stdout:1106: read oid 12 snap -1 2024-09-13T10:11:19.623 INFO:tasks.rados.rados.0.smithi195.stdout:1106: expect deleted 2024-09-13T10:11:19.623 INFO:tasks.rados.rados.0.smithi195.stdout:1107: write oid 40 current snap is 0 2024-09-13T10:11:19.623 INFO:tasks.rados.rados.0.smithi195.stdout:1102: finishing write tid 4 to smithi19530714-11 2024-09-13T10:11:19.623 INFO:tasks.rados.rados.0.smithi195.stdout:1102: finishing write tid 5 to smithi19530714-11 2024-09-13T10:11:19.623 INFO:tasks.rados.rados.0.smithi195.stdout:1102: oid 11 updating version 0 to 65 2024-09-13T10:11:19.623 INFO:tasks.rados.rados.0.smithi195.stdout:1102: oid 11 version 65 is already newer than 64 2024-09-13T10:11:19.623 INFO:tasks.rados.rados.0.smithi195.stdout:1102: oid 11 updating version 65 to 66 2024-09-13T10:11:19.624 INFO:tasks.rados.rados.0.smithi195.stdout:1102: oid 11 version 66 is already newer than 63 2024-09-13T10:11:19.624 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 66 (ObjNum 200 snap 0 seq_num 200) dirty exists 2024-09-13T10:11:19.624 INFO:tasks.rados.rados.0.smithi195.stdout:1102: left oid 11 (ObjNum 200 snap 0 seq_num 200) 2024-09-13T10:11:19.624 INFO:tasks.rados.rados.0.smithi195.stdout:1103: finishing write tid 1 to smithi19530714-43 2024-09-13T10:11:19.624 INFO:tasks.rados.rados.0.smithi195.stdout:1103: finishing write tid 2 to smithi19530714-43 2024-09-13T10:11:19.624 INFO:tasks.rados.rados.0.smithi195.stdout:1103: finishing write tid 3 to smithi19530714-43 2024-09-13T10:11:19.624 INFO:tasks.rados.rados.0.smithi195.stdout:1103: finishing write tid 4 to smithi19530714-43 2024-09-13T10:11:19.624 INFO:tasks.rados.rados.0.smithi195.stdout:1107: seq_num 203 ranges {611821=700725,1879374=783403,3341691=70130} 2024-09-13T10:11:19.631 INFO:tasks.rados.rados.0.smithi195.stdout:1107: writing smithi19530714-40 from 611821 to 1312546 tid 1 2024-09-13T10:11:19.637 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:19.637 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:19.640 INFO:tasks.rados.rados.0.smithi195.stdout:1107: writing smithi19530714-40 from 1879374 to 2662777 tid 2 2024-09-13T10:11:19.641 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:19.637+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:19.645 INFO:tasks.rados.rados.0.smithi195.stdout:1107: writing smithi19530714-40 from 3341691 to 3411821 tid 3 2024-09-13T10:11:19.645 INFO:tasks.rados.rados.0.smithi195.stdout:1102: done (5 left) 2024-09-13T10:11:19.645 INFO:tasks.rados.rados.0.smithi195.stdout:1108: tier_evict oid 32 2024-09-13T10:11:19.645 INFO:tasks.rados.rados.0.smithi195.stdout:1108: tier_evict oid 32 snap -1 2024-09-13T10:11:19.646 INFO:tasks.rados.rados.0.smithi195.stdout:1103: finishing write tid 5 to smithi19530714-43 2024-09-13T10:11:19.646 INFO:tasks.rados.rados.0.smithi195.stdout:1103: oid 43 updating version 0 to 58 2024-09-13T10:11:19.646 INFO:tasks.rados.rados.0.smithi195.stdout:1103: oid 43 updating version 58 to 60 2024-09-13T10:11:19.646 INFO:tasks.rados.rados.0.smithi195.stdout:1103: oid 43 version 60 is already newer than 57 2024-09-13T10:11:19.646 INFO:tasks.rados.rados.0.smithi195.stdout:1103: oid 43 version 60 is already newer than 59 2024-09-13T10:11:19.646 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 60 (ObjNum 201 snap 0 seq_num 201) dirty exists 2024-09-13T10:11:19.646 INFO:tasks.rados.rados.0.smithi195.stdout:1103: left oid 43 (ObjNum 201 snap 0 seq_num 201) 2024-09-13T10:11:19.646 INFO:tasks.rados.rados.0.smithi195.stdout:1104: expect (ObjNum 184 snap 0 seq_num 184) 2024-09-13T10:11:19.651 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T10:11:19.721 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:19.741 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:19.739+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:19.816 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:19.816 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:19.829 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T10:11:19.851 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2720273 2024-09-13T10:11:19.852 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:19.847+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:19.862 INFO:tasks.rados.rados.0.smithi195.stdout:1105: finishing write tid 1 to smithi19530714-45 2024-09-13T10:11:19.862 INFO:tasks.rados.rados.0.smithi195.stdout:1105: finishing write tid 2 to smithi19530714-45 2024-09-13T10:11:19.862 INFO:tasks.rados.rados.0.smithi195.stdout:1105: finishing write tid 3 to smithi19530714-45 2024-09-13T10:11:19.862 INFO:tasks.rados.rados.0.smithi195.stdout:1105: finishing write tid 4 to smithi19530714-45 2024-09-13T10:11:19.862 INFO:tasks.rados.rados.0.smithi195.stdout:1105: finishing write tid 5 to smithi19530714-45 2024-09-13T10:11:19.862 INFO:tasks.rados.rados.0.smithi195.stdout:1105: oid 45 updating version 0 to 109 2024-09-13T10:11:19.862 INFO:tasks.rados.rados.0.smithi195.stdout:1105: oid 45 version 109 is already newer than 108 2024-09-13T10:11:19.863 INFO:tasks.rados.rados.0.smithi195.stdout:1105: oid 45 updating version 109 to 110 2024-09-13T10:11:19.863 INFO:tasks.rados.rados.0.smithi195.stdout:1105: oid 45 version 110 is already newer than 107 2024-09-13T10:11:19.863 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 110 (ObjNum 202 snap 0 seq_num 202) dirty exists 2024-09-13T10:11:19.863 INFO:tasks.rados.rados.0.smithi195.stdout:1105: left oid 45 (ObjNum 202 snap 0 seq_num 202) 2024-09-13T10:11:19.863 INFO:tasks.rados.rados.0.smithi195.stdout:1107: finishing write tid 1 to smithi19530714-40 2024-09-13T10:11:19.863 INFO:tasks.rados.rados.0.smithi195.stdout:1108: got (22) Invalid argument 2024-09-13T10:11:19.863 INFO:tasks.rados.rados.0.smithi195.stdout:1107: finishing write tid 2 to smithi19530714-40 2024-09-13T10:11:19.863 INFO:tasks.rados.rados.0.smithi195.stdout:1103: done (5 left) 2024-09-13T10:11:19.863 INFO:tasks.rados.rados.0.smithi195.stdout:1104: done (4 left) 2024-09-13T10:11:19.863 INFO:tasks.rados.rados.0.smithi195.stdout:1105: done (3 left) 2024-09-13T10:11:19.863 INFO:tasks.rados.rados.0.smithi195.stdout:1106: done (2 left) 2024-09-13T10:11:19.863 INFO:tasks.rados.rados.0.smithi195.stdout:1108: done (1 left) 2024-09-13T10:11:19.863 INFO:tasks.rados.rados.0.smithi195.stdout:1109: tier_promote oid 19 2024-09-13T10:11:19.864 INFO:tasks.rados.rados.0.smithi195.stdout:1107: finishing write tid 3 to smithi19530714-40 2024-09-13T10:11:19.864 INFO:tasks.rados.rados.0.smithi195.stdout:1107: finishing write tid 4 to smithi19530714-40 2024-09-13T10:11:19.864 INFO:tasks.rados.rados.0.smithi195.stdout:1107: finishing write tid 5 to smithi19530714-40 2024-09-13T10:11:19.864 INFO:tasks.rados.rados.0.smithi195.stdout:1107: oid 40 updating version 0 to 21 2024-09-13T10:11:19.864 INFO:tasks.rados.rados.0.smithi195.stdout:1107: oid 40 updating version 21 to 24 2024-09-13T10:11:19.864 INFO:tasks.rados.rados.0.smithi195.stdout:1107: oid 40 version 24 is already newer than 23 2024-09-13T10:11:19.864 INFO:tasks.rados.rados.0.smithi195.stdout:1107: oid 40 version 24 is already newer than 22 2024-09-13T10:11:19.864 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 24 (ObjNum 203 snap 0 seq_num 203) dirty exists 2024-09-13T10:11:19.864 INFO:tasks.rados.rados.0.smithi195.stdout:1107: left oid 40 (ObjNum 203 snap 0 seq_num 203) 2024-09-13T10:11:19.864 INFO:tasks.rados.rados.0.smithi195.stdout:1107: done (1 left) 2024-09-13T10:11:19.864 INFO:tasks.rados.rados.0.smithi195.stdout:1110: write oid 15 current snap is 0 2024-09-13T10:11:19.864 INFO:tasks.rados.rados.0.smithi195.stdout:1110: seq_num 204 ranges {628628=775974,1937856=527061,3252761=175867} 2024-09-13T10:11:19.870 INFO:tasks.rados.rados.0.smithi195.stdout:1110: writing smithi19530714-15 from 628628 to 1404602 tid 1 2024-09-13T10:11:19.878 INFO:tasks.rados.rados.0.smithi195.stdout:1110: writing smithi19530714-15 from 1937856 to 2464917 tid 2 2024-09-13T10:11:19.885 INFO:tasks.rados.rados.0.smithi195.stdout:1110: writing smithi19530714-15 from 3252761 to 3428628 tid 3 2024-09-13T10:11:19.886 INFO:tasks.rados.rados.0.smithi195.stdout:1111: write oid 44 current snap is 0 2024-09-13T10:11:19.886 INFO:tasks.rados.rados.0.smithi195.stdout:1109: got (0) Success 2024-09-13T10:11:19.886 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 49 (ObjNum 160 snap 0 seq_num 160) dirty exists 2024-09-13T10:11:19.886 INFO:tasks.rados.rados.0.smithi195.stdout:1110: finishing write tid 1 to smithi19530714-15 2024-09-13T10:11:19.886 INFO:tasks.rados.rados.0.smithi195.stdout:1110: finishing write tid 2 to smithi19530714-15 2024-09-13T10:11:19.886 INFO:tasks.rados.rados.0.smithi195.stdout:1111: seq_num 205 ranges {645435=451223,1879985=787072,3280184=165251} 2024-09-13T10:11:19.893 INFO:tasks.rados.rados.0.smithi195.stdout:1111: writing smithi19530714-44 from 645435 to 1096658 tid 1 2024-09-13T10:11:19.904 INFO:tasks.rados.rados.0.smithi195.stdout:1111: writing smithi19530714-44 from 1879985 to 2667057 tid 2 2024-09-13T10:11:19.910 INFO:tasks.rados.rados.0.smithi195.stdout:1111: writing smithi19530714-44 from 3280184 to 3445435 tid 3 2024-09-13T10:11:19.910 INFO:tasks.rados.rados.0.smithi195.stdout:1109: done (2 left) 2024-09-13T10:11:19.910 INFO:tasks.rados.rados.0.smithi195.stdout:1112: read oid 9 snap -1 2024-09-13T10:11:19.910 INFO:tasks.rados.rados.0.smithi195.stdout:1112: expect deleted 2024-09-13T10:11:19.910 INFO:tasks.rados.rados.0.smithi195.stdout:1113: tier_flush oid 13 2024-09-13T10:11:19.910 INFO:tasks.rados.rados.0.smithi195.stdout:1113: tier_flush oid 13 snap -1 2024-09-13T10:11:19.910 INFO:tasks.rados.rados.0.smithi195.stdout:1110: finishing write tid 3 to smithi19530714-15 2024-09-13T10:11:19.910 INFO:tasks.rados.rados.0.smithi195.stdout:1114: write oid 23 current snap is 0 2024-09-13T10:11:19.910 INFO:tasks.rados.rados.0.smithi195.stdout:1110: finishing write tid 4 to smithi19530714-15 2024-09-13T10:11:19.910 INFO:tasks.rados.rados.0.smithi195.stdout:1110: finishing write tid 5 to smithi19530714-15 2024-09-13T10:11:19.911 INFO:tasks.rados.rados.0.smithi195.stdout:1110: oid 15 updating version 0 to 61 2024-09-13T10:11:19.911 INFO:tasks.rados.rados.0.smithi195.stdout:1110: oid 15 updating version 61 to 62 2024-09-13T10:11:19.911 INFO:tasks.rados.rados.0.smithi195.stdout:1110: oid 15 updating version 62 to 63 2024-09-13T10:11:19.911 INFO:tasks.rados.rados.0.smithi195.stdout:1110: oid 15 version 63 is already newer than 60 2024-09-13T10:11:19.911 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 63 (ObjNum 204 snap 0 seq_num 204) dirty exists 2024-09-13T10:11:19.911 INFO:tasks.rados.rados.0.smithi195.stdout:1110: left oid 15 (ObjNum 204 snap 0 seq_num 204) 2024-09-13T10:11:19.911 INFO:tasks.rados.rados.0.smithi195.stdout:1111: finishing write tid 1 to smithi19530714-44 2024-09-13T10:11:19.911 INFO:tasks.rados.rados.0.smithi195.stdout:1114: seq_num 206 ranges {662242=642825,2054820=530730,3307607=154635} 2024-09-13T10:11:19.917 INFO:tasks.rados.rados.0.smithi195.stdout:1114: writing smithi19530714-23 from 662242 to 1305067 tid 1 2024-09-13T10:11:19.924 INFO:tasks.rados.rados.0.smithi195.stdout:1114: writing smithi19530714-23 from 2054820 to 2585550 tid 2 2024-09-13T10:11:19.930 INFO:tasks.rados.rados.0.smithi195.stdout:1114: writing smithi19530714-23 from 3307607 to 3462242 tid 3 2024-09-13T10:11:19.930 INFO:tasks.rados.rados.0.smithi195.stdout:1110: done (4 left) 2024-09-13T10:11:19.930 INFO:tasks.rados.rados.0.smithi195.stdout:1115: read oid 30 snap -1 2024-09-13T10:11:19.930 INFO:tasks.rados.rados.0.smithi195.stdout:1115: expect (ObjNum 102 snap 0 seq_num 102) 2024-09-13T10:11:19.930 INFO:tasks.rados.rados.0.smithi195.stdout:1116: read oid 18 snap -1 2024-09-13T10:11:19.930 INFO:tasks.rados.rados.0.smithi195.stdout:1116: expect (ObjNum 189 snap 0 seq_num 189) 2024-09-13T10:11:19.930 INFO:tasks.rados.rados.0.smithi195.stdout:1117: set_chunk oid 6 target oid 2024-09-13T10:11:19.940 INFO:tasks.rados.rados.0.smithi195.stdout:1117: set_chunk oid 6 offset: 689152 length: 619008 target oid 569f0035bbe8460b2c185e8a8245faab1f9c169600eef854292c259d64863ff0 offset: 0 snap -1 2024-09-13T10:11:19.940 INFO:tasks.rados.rados.0.smithi195.stdout:1118: read oid 48 snap -1 2024-09-13T10:11:19.940 INFO:tasks.rados.rados.0.smithi195.stdout:1118: expect (ObjNum 185 snap 0 seq_num 185) 2024-09-13T10:11:19.940 INFO:tasks.rados.rados.0.smithi195.stdout:1119: read oid 31 snap -1 2024-09-13T10:11:19.941 INFO:tasks.rados.rados.0.smithi195.stdout:1119: expect deleted 2024-09-13T10:11:19.941 INFO:tasks.rados.rados.0.smithi195.stdout:1120: set_chunk oid 15 target oid 2024-09-13T10:11:19.943 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:19.941+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:19.978 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:19.978 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:19.985 INFO:tasks.rados.rados.0.smithi195.stdout:1120: set_chunk oid 15 offset: 71168 length: 3092992 target oid 48ed550cad4a2f7ef270ccceb0b237d14120033f8b9c01b7bc0522b0f5e92062 offset: 0 snap -1 2024-09-13T10:11:19.985 INFO:tasks.rados.rados.0.smithi195.stdout:1121: set_chunk oid 33 target oid 2024-09-13T10:11:19.991 INFO:tasks.rados.rados.0.smithi195.stdout:1121: set_chunk oid 33 offset: 1019392 length: 299008 target oid 1d370b717e9c595923ae338ffb086624804c18fae62c32f74eb81d34cfb975e4 offset: 0 snap -1 2024-09-13T10:11:19.991 INFO:tasks.rados.rados.0.smithi195.stdout:1122: read oid 12 snap -1 2024-09-13T10:11:19.991 INFO:tasks.rados.rados.0.smithi195.stdout:1122: expect deleted 2024-09-13T10:11:19.991 INFO:tasks.rados.rados.0.smithi195.stdout:1123: write oid 10 current snap is 0 2024-09-13T10:11:19.991 INFO:tasks.rados.rados.0.smithi195.stdout:1123: seq_num 207 ranges {679049=718074,2113302=674388,3335030=144019} 2024-09-13T10:11:19.992 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-13T10:11:19.999 INFO:tasks.rados.rados.0.smithi195.stdout:1123: writing smithi19530714-10 from 679049 to 1397123 tid 1 2024-09-13T10:11:20.006 INFO:tasks.rados.rados.0.smithi195.stdout:1123: writing smithi19530714-10 from 2113302 to 2787690 tid 2 2024-09-13T10:11:20.011 INFO:tasks.rados.rados.0.smithi195.stdout:1123: writing smithi19530714-10 from 3335030 to 3479049 tid 3 2024-09-13T10:11:20.011 INFO:tasks.rados.rados.0.smithi195.stdout:1124: tier_flush oid 19 2024-09-13T10:11:20.011 INFO:tasks.rados.rados.0.smithi195.stdout:1124: tier_flush oid 19 snap -1 2024-09-13T10:11:20.011 INFO:tasks.rados.rados.0.smithi195.stdout:1112: done (13 left) 2024-09-13T10:11:20.012 INFO:tasks.rados.rados.0.smithi195.stdout:1125: tier_promote oid 9 2024-09-13T10:11:20.012 INFO:tasks.rados.rados.0.smithi195.stdout:1111: finishing write tid 2 to smithi19530714-44 2024-09-13T10:11:20.012 INFO:tasks.rados.rados.0.smithi195.stdout:1111: finishing write tid 3 to smithi19530714-44 2024-09-13T10:11:20.012 INFO:tasks.rados.rados.0.smithi195.stdout:1114: finishing write tid 1 to smithi19530714-23 2024-09-13T10:11:20.012 INFO:tasks.rados.rados.0.smithi195.stdout:1114: finishing write tid 2 to smithi19530714-23 2024-09-13T10:11:20.012 INFO:tasks.rados.rados.0.smithi195.stdout:1114: finishing write tid 3 to smithi19530714-23 2024-09-13T10:11:20.012 INFO:tasks.rados.rados.0.smithi195.stdout:1114: finishing write tid 4 to smithi19530714-23 2024-09-13T10:11:20.012 INFO:tasks.rados.rados.0.smithi195.stdout:1114: finishing write tid 5 to smithi19530714-23 2024-09-13T10:11:20.012 INFO:tasks.rados.rados.0.smithi195.stdout:1114: oid 23 updating version 0 to 33 2024-09-13T10:11:20.012 INFO:tasks.rados.rados.0.smithi195.stdout:1114: oid 23 version 33 is already newer than 31 2024-09-13T10:11:20.012 INFO:tasks.rados.rados.0.smithi195.stdout:1114: oid 23 updating version 33 to 34 2024-09-13T10:11:20.012 INFO:tasks.rados.rados.0.smithi195.stdout:1114: oid 23 version 34 is already newer than 32 2024-09-13T10:11:20.013 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 34 (ObjNum 206 snap 0 seq_num 206) dirty exists 2024-09-13T10:11:20.013 INFO:tasks.rados.rados.0.smithi195.stdout:1114: left oid 23 (ObjNum 206 snap 0 seq_num 206) 2024-09-13T10:11:20.013 INFO:tasks.rados.rados.0.smithi195.stdout:1114: done (13 left) 2024-09-13T10:11:20.013 INFO:tasks.rados.rados.0.smithi195.stdout:1126: read oid 2 snap -1 2024-09-13T10:11:20.013 INFO:tasks.rados.rados.0.smithi195.stdout:1126: expect (ObjNum 176 snap 0 seq_num 176) 2024-09-13T10:11:20.013 INFO:tasks.rados.rados.0.smithi195.stdout:1127: write oid 35 current snap is 0 2024-09-13T10:11:20.013 INFO:tasks.rados.rados.0.smithi195.stdout:1127: seq_num 208 ranges {695856=793323,2171784=534399,3362453=133403} 2024-09-13T10:11:20.019 INFO:tasks.rados.rados.0.smithi195.stdout:1127: writing smithi19530714-35 from 695856 to 1489179 tid 1 2024-09-13T10:11:20.027 INFO:tasks.rados.rados.0.smithi195.stdout:1127: writing smithi19530714-35 from 2171784 to 2706183 tid 2 2024-09-13T10:11:20.032 INFO:tasks.rados.rados.0.smithi195.stdout:1127: writing smithi19530714-35 from 3362453 to 3495856 tid 3 2024-09-13T10:11:20.032 INFO:tasks.rados.rados.0.smithi195.stdout:1128: read oid 50 snap -1 2024-09-13T10:11:20.032 INFO:tasks.rados.rados.0.smithi195.stdout:1128: expect (ObjNum 156 snap 0 seq_num 156) 2024-09-13T10:11:20.032 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:20.032 INFO:tasks.rados.rados.0.smithi195.stdout:1116: expect (ObjNum 189 snap 0 seq_num 189) 2024-09-13T10:11:20.032 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:20.044 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:20.041+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:20.097 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:20.122 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:20.123 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:20.136 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-13T10:11:20.145 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:20.143+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:20.154 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:20.211 INFO:tasks.rados.rados.0.smithi195.stdout:1117: finishing set_chunk to oid 6 2024-09-13T10:11:20.211 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 53 (ObjNum 168 snap 0 seq_num 168) dirty exists 2024-09-13T10:11:20.211 INFO:tasks.rados.rados.0.smithi195.stdout:1118: expect (ObjNum 185 snap 0 seq_num 185) 2024-09-13T10:11:20.246 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:20.245+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:20.268 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:20.268 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:20.283 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-13T10:11:20.296 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:20.340 INFO:tasks.rados.rados.0.smithi195.stdout:1111: finishing write tid 4 to smithi19530714-44 2024-09-13T10:11:20.341 INFO:tasks.rados.rados.0.smithi195.stdout:1111: finishing write tid 5 to smithi19530714-44 2024-09-13T10:11:20.341 INFO:tasks.rados.rados.0.smithi195.stdout:1111: oid 44 updating version 0 to 56 2024-09-13T10:11:20.341 INFO:tasks.rados.rados.0.smithi195.stdout:1111: oid 44 updating version 56 to 57 2024-09-13T10:11:20.341 INFO:tasks.rados.rados.0.smithi195.stdout:1111: oid 44 version 57 is already newer than 55 2024-09-13T10:11:20.341 INFO:tasks.rados.rados.0.smithi195.stdout:1111: oid 44 version 57 is already newer than 54 2024-09-13T10:11:20.341 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 57 (ObjNum 205 snap 0 seq_num 205) dirty exists 2024-09-13T10:11:20.341 INFO:tasks.rados.rados.0.smithi195.stdout:1111: left oid 44 (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:11:20.341 INFO:tasks.rados.rados.0.smithi195.stdout:1113: got (0) Success 2024-09-13T10:11:20.341 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 0 (ObjNum 187 snap 0 seq_num 187) dirty exists 2024-09-13T10:11:20.341 INFO:tasks.rados.rados.0.smithi195.stdout:1111: done (15 left) 2024-09-13T10:11:20.341 INFO:tasks.rados.rados.0.smithi195.stdout:1113: done (14 left) 2024-09-13T10:11:20.341 INFO:tasks.rados.rados.0.smithi195.stdout:1116: done (13 left) 2024-09-13T10:11:20.341 INFO:tasks.rados.rados.0.smithi195.stdout:1117: done (12 left) 2024-09-13T10:11:20.341 INFO:tasks.rados.rados.0.smithi195.stdout:1118: done (11 left) 2024-09-13T10:11:20.342 INFO:tasks.rados.rados.0.smithi195.stdout:1119: done (10 left) 2024-09-13T10:11:20.342 INFO:tasks.rados.rados.0.smithi195.stdout:1129: read oid 26 snap -1 2024-09-13T10:11:20.342 INFO:tasks.rados.rados.0.smithi195.stdout:1129: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:20.342 INFO:tasks.rados.rados.0.smithi195.stdout:1130: set_chunk oid 22 target oid 2024-09-13T10:11:20.348 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:20.343+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:20.351 INFO:tasks.rados.rados.0.smithi195.stdout:1130: set_chunk oid 22 offset: 1484288 length: 456704 target oid 20fd6aea0a676fea09bafdcb507c929fd67c96fa5f16c57f361d98dbf2a9644a offset: 0 snap -1 2024-09-13T10:11:20.351 INFO:tasks.rados.rados.0.smithi195.stdout:1131: read oid 29 snap -1 2024-09-13T10:11:20.351 INFO:tasks.rados.rados.0.smithi195.stdout:1131: expect deleted 2024-09-13T10:11:20.351 INFO:tasks.rados.rados.0.smithi195.stdout:1132: tier_evict oid 43 2024-09-13T10:11:20.351 INFO:tasks.rados.rados.0.smithi195.stdout:1132: tier_evict oid 43 snap -1 2024-09-13T10:11:20.351 INFO:tasks.rados.rados.0.smithi195.stdout:1120: finishing set_chunk to oid 15 2024-09-13T10:11:20.351 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 15 set_chunk returned error code -95 offset: 71168 length: 3092992 2024-09-13T10:11:20.351 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 0 (ObjNum 204 snap 0 seq_num 204) dirty exists 2024-09-13T10:11:20.352 INFO:tasks.rados.rados.0.smithi195.stdout:1121: finishing set_chunk to oid 33 2024-09-13T10:11:20.352 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 42 (ObjNum 179 snap 0 seq_num 179) dirty exists 2024-09-13T10:11:20.352 INFO:tasks.rados.rados.0.smithi195.stdout:1123: finishing write tid 1 to smithi19530714-10 2024-09-13T10:11:20.352 INFO:tasks.rados.rados.0.smithi195.stdout:1123: finishing write tid 2 to smithi19530714-10 2024-09-13T10:11:20.352 INFO:tasks.rados.rados.0.smithi195.stdout:1123: finishing write tid 3 to smithi19530714-10 2024-09-13T10:11:20.352 INFO:tasks.rados.rados.0.smithi195.stdout:1123: finishing write tid 4 to smithi19530714-10 2024-09-13T10:11:20.352 INFO:tasks.rados.rados.0.smithi195.stdout:1123: finishing write tid 5 to smithi19530714-10 2024-09-13T10:11:20.352 INFO:tasks.rados.rados.0.smithi195.stdout:1123: oid 10 updating version 0 to 20 2024-09-13T10:11:20.352 INFO:tasks.rados.rados.0.smithi195.stdout:1123: oid 10 updating version 20 to 21 2024-09-13T10:11:20.352 INFO:tasks.rados.rados.0.smithi195.stdout:1123: oid 10 version 21 is already newer than 19 2024-09-13T10:11:20.352 INFO:tasks.rados.rados.0.smithi195.stdout:1123: oid 10 updating version 21 to 22 2024-09-13T10:11:20.352 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 22 (ObjNum 207 snap 0 seq_num 207) dirty exists 2024-09-13T10:11:20.353 INFO:tasks.rados.rados.0.smithi195.stdout:1123: left oid 10 (ObjNum 207 snap 0 seq_num 207) 2024-09-13T10:11:20.353 INFO:tasks.rados.rados.0.smithi195.stdout:1126: expect (ObjNum 176 snap 0 seq_num 176) 2024-09-13T10:11:20.378 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1890561 2024-09-13T10:11:20.399 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 694510 2024-09-13T10:11:20.402 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:20.403 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:20.416 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-13T10:11:20.447 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 891615 2024-09-13T10:11:20.448 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:20.445+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:20.487 INFO:tasks.rados.rados.0.smithi195.stdout:1125: got (2) No such file or directory 2024-09-13T10:11:20.487 INFO:tasks.rados.rados.0.smithi195.stdout:1125: got expected ENOENT (src dne) 2024-09-13T10:11:20.487 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 98 (ObjNum 129 snap 0 seq_num 643458416) clean dne 2024-09-13T10:11:20.487 INFO:tasks.rados.rados.0.smithi195.stdout:1127: finishing write tid 1 to smithi19530714-35 2024-09-13T10:11:20.487 INFO:tasks.rados.rados.0.smithi195.stdout:1120: done (13 left) 2024-09-13T10:11:20.487 INFO:tasks.rados.rados.0.smithi195.stdout:1121: done (12 left) 2024-09-13T10:11:20.487 INFO:tasks.rados.rados.0.smithi195.stdout:1122: done (11 left) 2024-09-13T10:11:20.488 INFO:tasks.rados.rados.0.smithi195.stdout:1123: done (10 left) 2024-09-13T10:11:20.488 INFO:tasks.rados.rados.0.smithi195.stdout:1125: done (9 left) 2024-09-13T10:11:20.488 INFO:tasks.rados.rados.0.smithi195.stdout:1126: done (8 left) 2024-09-13T10:11:20.488 INFO:tasks.rados.rados.0.smithi195.stdout:1133: tier_evict oid 48 2024-09-13T10:11:20.488 INFO:tasks.rados.rados.0.smithi195.stdout:1133: tier_evict oid 48 snap -1 2024-09-13T10:11:20.488 INFO:tasks.rados.rados.0.smithi195.stdout:1127: finishing write tid 2 to smithi19530714-35 2024-09-13T10:11:20.488 INFO:tasks.rados.rados.0.smithi195.stdout:1127: finishing write tid 3 to smithi19530714-35 2024-09-13T10:11:20.488 INFO:tasks.rados.rados.0.smithi195.stdout:1124: got (0) Success 2024-09-13T10:11:20.488 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 0 (ObjNum 160 snap 0 seq_num 160) dirty exists 2024-09-13T10:11:20.488 INFO:tasks.rados.rados.0.smithi195.stdout:1127: finishing write tid 4 to smithi19530714-35 2024-09-13T10:11:20.488 INFO:tasks.rados.rados.0.smithi195.stdout:1127: finishing write tid 5 to smithi19530714-35 2024-09-13T10:11:20.488 INFO:tasks.rados.rados.0.smithi195.stdout:1127: oid 35 updating version 0 to 55 2024-09-13T10:11:20.488 INFO:tasks.rados.rados.0.smithi195.stdout:1127: oid 35 updating version 55 to 58 2024-09-13T10:11:20.488 INFO:tasks.rados.rados.0.smithi195.stdout:1127: oid 35 version 58 is already newer than 57 2024-09-13T10:11:20.489 INFO:tasks.rados.rados.0.smithi195.stdout:1127: oid 35 version 58 is already newer than 56 2024-09-13T10:11:20.489 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 58 (ObjNum 208 snap 0 seq_num 208) dirty exists 2024-09-13T10:11:20.489 INFO:tasks.rados.rados.0.smithi195.stdout:1127: left oid 35 (ObjNum 208 snap 0 seq_num 208) 2024-09-13T10:11:20.489 INFO:tasks.rados.rados.0.smithi195.stdout:1128: expect (ObjNum 156 snap 0 seq_num 156) 2024-09-13T10:11:20.545 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:20.549 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:20.545+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:20.558 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:11:20.559 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:20.559 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:20.571 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-13T10:11:20.604 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 212421 2024-09-13T10:11:20.650 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:20.645+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:20.662 INFO:tasks.rados.rados.0.smithi195.stdout:1129: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:20.680 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:20.713 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1668103 2024-09-13T10:11:20.714 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:20.715 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:20.717 INFO:tasks.rados.rados.0.smithi195.stdout:1132: got (22) Invalid argument 2024-09-13T10:11:20.717 INFO:tasks.rados.rados.0.smithi195.stdout:1130: finishing set_chunk to oid 22 2024-09-13T10:11:20.717 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 22 set_chunk returned error code -95 offset: 1484288 length: 456704 2024-09-13T10:11:20.717 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 0 (ObjNum 139 snap 0 seq_num 139) dirty exists 2024-09-13T10:11:20.717 INFO:tasks.rados.rados.0.smithi195.stdout:1133: got (22) Invalid argument 2024-09-13T10:11:20.717 INFO:tasks.rados.rados.0.smithi195.stdout:1124: done (8 left) 2024-09-13T10:11:20.717 INFO:tasks.rados.rados.0.smithi195.stdout:1127: done (7 left) 2024-09-13T10:11:20.717 INFO:tasks.rados.rados.0.smithi195.stdout:1128: done (6 left) 2024-09-13T10:11:20.717 INFO:tasks.rados.rados.0.smithi195.stdout:1129: done (5 left) 2024-09-13T10:11:20.717 INFO:tasks.rados.rados.0.smithi195.stdout:1130: done (4 left) 2024-09-13T10:11:20.717 INFO:tasks.rados.rados.0.smithi195.stdout:1131: done (3 left) 2024-09-13T10:11:20.718 INFO:tasks.rados.rados.0.smithi195.stdout:1132: done (2 left) 2024-09-13T10:11:20.718 INFO:tasks.rados.rados.0.smithi195.stdout:1133: done (1 left) 2024-09-13T10:11:20.718 INFO:tasks.rados.rados.0.smithi195.stdout:1134: read oid 40 snap -1 2024-09-13T10:11:20.718 INFO:tasks.rados.rados.0.smithi195.stdout:1134: expect (ObjNum 203 snap 0 seq_num 203) 2024-09-13T10:11:20.718 INFO:tasks.rados.rados.0.smithi195.stdout:1135: tier_promote oid 44 2024-09-13T10:11:20.718 INFO:tasks.rados.rados.0.smithi195.stdout:1136: read oid 41 snap -1 2024-09-13T10:11:20.718 INFO:tasks.rados.rados.0.smithi195.stdout:1136: expect (ObjNum 141 snap 0 seq_num 141) 2024-09-13T10:11:20.718 INFO:tasks.rados.rados.0.smithi195.stdout:1137: read oid 5 snap -1 2024-09-13T10:11:20.718 INFO:tasks.rados.rados.0.smithi195.stdout:1137: expect (ObjNum 173 snap 0 seq_num 173) 2024-09-13T10:11:20.718 INFO:tasks.rados.rados.0.smithi195.stdout:1138: write oid 14 current snap is 0 2024-09-13T10:11:20.718 INFO:tasks.rados.rados.0.smithi195.stdout:1138: seq_num 209 ranges {712663=468572,1713913=678057,2873523=591733} 2024-09-13T10:11:20.725 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:20.725 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:20.736 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:20.736 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:20.736 INFO:tasks.rados.rados.0.smithi195.stdout:1138: writing smithi19530714-14 from 712663 to 1181235 tid 1 2024-09-13T10:11:20.742 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-13T10:11:20.751 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:20.747+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:20.755 INFO:tasks.rados.rados.0.smithi195.stdout:1138: writing smithi19530714-14 from 1713913 to 2391970 tid 2 2024-09-13T10:11:20.759 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:20.759 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:20.771 INFO:tasks.rados.rados.0.smithi195.stdout:1138: writing smithi19530714-14 from 2873523 to 3465256 tid 3 2024-09-13T10:11:20.772 INFO:tasks.rados.rados.0.smithi195.stdout:1139: read oid 28 snap -1 2024-09-13T10:11:20.772 INFO:tasks.rados.rados.0.smithi195.stdout:1139: expect (ObjNum 180 snap 0 seq_num 180) 2024-09-13T10:11:20.772 INFO:tasks.rados.rados.0.smithi195.stdout:1140: write oid 25 current snap is 0 2024-09-13T10:11:20.772 INFO:tasks.rados.rados.0.smithi195.stdout:1140: seq_num 210 ranges {729470=543821,1772395=538068,2900946=403005} 2024-09-13T10:11:20.776 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T10:11:20.777 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:20.777 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:20.785 INFO:tasks.rados.rados.0.smithi195.stdout:1140: writing smithi19530714-25 from 729470 to 1273291 tid 1 2024-09-13T10:11:20.795 INFO:tasks.rados.rados.0.smithi195.stdout:1140: writing smithi19530714-25 from 1772395 to 2310463 tid 2 2024-09-13T10:11:20.799 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:20.799 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:20.803 INFO:tasks.rados.rados.0.smithi195.stdout:1140: writing smithi19530714-25 from 2900946 to 3303951 tid 3 2024-09-13T10:11:20.804 INFO:tasks.rados.rados.0.smithi195.stdout:1141: read oid 1 snap -1 2024-09-13T10:11:20.804 INFO:tasks.rados.rados.0.smithi195.stdout:1141: expect (ObjNum 184 snap 0 seq_num 184) 2024-09-13T10:11:20.804 INFO:tasks.rados.rados.0.smithi195.stdout:1142: read oid 12 snap -1 2024-09-13T10:11:20.804 INFO:tasks.rados.rados.0.smithi195.stdout:1142: expect deleted 2024-09-13T10:11:20.804 INFO:tasks.rados.rados.0.smithi195.stdout:1143: read oid 34 snap -1 2024-09-13T10:11:20.804 INFO:tasks.rados.rados.0.smithi195.stdout:1143: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:20.804 INFO:tasks.rados.rados.0.smithi195.stdout:1144: read oid 10 snap -1 2024-09-13T10:11:20.805 INFO:tasks.rados.rados.0.smithi195.stdout:1144: expect (ObjNum 207 snap 0 seq_num 207) 2024-09-13T10:11:20.805 INFO:tasks.rados.rados.0.smithi195.stdout:1145: set_chunk oid 20 target oid 2024-09-13T10:11:20.821 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:20.821 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:20.826 INFO:tasks.rados.rados.0.smithi195.stdout:1145: set_chunk oid 20 offset: 1102336 length: 1097216 target oid 1eae3f6817254e3244c987c537eb0bb6ce40331352cec1388afa4293175e1df9 offset: 0 snap -1 2024-09-13T10:11:20.827 INFO:tasks.rados.rados.0.smithi195.stdout:1146: read oid 2 snap -1 2024-09-13T10:11:20.827 INFO:tasks.rados.rados.0.smithi195.stdout:1146: expect (ObjNum 176 snap 0 seq_num 176) 2024-09-13T10:11:20.827 INFO:tasks.rados.rados.0.smithi195.stdout:1147: read oid 32 snap -1 2024-09-13T10:11:20.827 INFO:tasks.rados.rados.0.smithi195.stdout:1147: expect deleted 2024-09-13T10:11:20.827 INFO:tasks.rados.rados.0.smithi195.stdout:1148: set_chunk oid 42 target oid 2024-09-13T10:11:20.843 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:20.844 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:20.847 INFO:tasks.rados.rados.0.smithi195.stdout:1148: set_chunk oid 42 offset: 291328 length: 1497600 target oid 72bf9b78e6447e6c7df03f9dd6f93367f618ff88837f95b5291906b17e2aa878 offset: 0 snap -1 2024-09-13T10:11:20.847 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:20.847 INFO:tasks.rados.rados.0.smithi195.stdout:1135: got (0) Success 2024-09-13T10:11:20.847 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 57 (ObjNum 205 snap 0 seq_num 205) dirty exists 2024-09-13T10:11:20.848 INFO:tasks.rados.rados.0.smithi195.stdout:1136: expect (ObjNum 141 snap 0 seq_num 141) 2024-09-13T10:11:20.852 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:20.849+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:20.859 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1483645 2024-09-13T10:11:20.859 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:20.860 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:20.879 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:20.879 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:20.886 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:20.886 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:20.899 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:20.900 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:20.903 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T10:11:20.923 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:20.923 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:20.943 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:20.944 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:20.953 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:20.951+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:20.977 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1919083 2024-09-13T10:11:20.989 INFO:tasks.rados.rados.0.smithi195.stdout:1137: expect (ObjNum 173 snap 0 seq_num 173) 2024-09-13T10:11:21.019 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:21.019 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:11:21.019 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:21.033 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T10:11:21.055 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:21.051+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:21.072 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:21.116 INFO:tasks.rados.rados.0.smithi195.stdout:1134: expect (ObjNum 203 snap 0 seq_num 203) 2024-09-13T10:11:21.116 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:21.156 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:21.155+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:21.164 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:21.164 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:21.185 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:21.185 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:21.202 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:21.202 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:21.203 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:11:21.203 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:11:21.203 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:21.212 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:21.212 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:21.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:21.213+0000 7fd03ad06640 0 -- 172.21.15.73:0/3556342670 >> v2:172.21.15.195:6805/848686366 conn(0x7fd034227950 msgr2=0x7fd034229d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:11:21.218 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T10:11:21.234 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:21.257 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:21.255+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:21.292 INFO:tasks.rados.rados.0.smithi195.stdout:1138: finishing write tid 1 to smithi19530714-14 2024-09-13T10:11:21.292 INFO:tasks.rados.rados.0.smithi195.stdout:1138: finishing write tid 2 to smithi19530714-14 2024-09-13T10:11:21.293 INFO:tasks.rados.rados.0.smithi195.stdout:1138: finishing write tid 3 to smithi19530714-14 2024-09-13T10:11:21.293 INFO:tasks.rados.rados.0.smithi195.stdout:1138: finishing write tid 4 to smithi19530714-14 2024-09-13T10:11:21.293 INFO:tasks.rados.rados.0.smithi195.stdout:1138: finishing write tid 5 to smithi19530714-14 2024-09-13T10:11:21.293 INFO:tasks.rados.rados.0.smithi195.stdout:1138: oid 14 updating version 0 to 64 2024-09-13T10:11:21.293 INFO:tasks.rados.rados.0.smithi195.stdout:1138: oid 14 updating version 64 to 65 2024-09-13T10:11:21.293 INFO:tasks.rados.rados.0.smithi195.stdout:1138: oid 14 updating version 65 to 66 2024-09-13T10:11:21.293 INFO:tasks.rados.rados.0.smithi195.stdout:1138: oid 14 updating version 66 to 67 2024-09-13T10:11:21.293 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 67 (ObjNum 209 snap 0 seq_num 209) dirty exists 2024-09-13T10:11:21.293 INFO:tasks.rados.rados.0.smithi195.stdout:1138: left oid 14 (ObjNum 209 snap 0 seq_num 209) 2024-09-13T10:11:21.293 INFO:tasks.rados.rados.0.smithi195.stdout:1139: expect (ObjNum 180 snap 0 seq_num 180) 2024-09-13T10:11:21.293 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:21.338 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:11:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:11:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:11:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:202 3.2 e38/30 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:11:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.333908313999999, 2024-09-13T10:11:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016741669000000001, 2024-09-13T10:11:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:11:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:11:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002368+0000", 2024-09-13T10:11:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-13T10:11:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010047+0000", 2024-09-13T10:11:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007678748 2024-09-13T10:11:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010081+0000", 2024-09-13T10:11:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4011999999999999e-05 2024-09-13T10:11:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010098+0000", 2024-09-13T10:11:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7125999999999998e-05 2024-09-13T10:11:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010253+0000", 2024-09-13T10:11:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000154881 2024-09-13T10:11:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010294+0000", 2024-09-13T10:11:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1746000000000003e-05 2024-09-13T10:11:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.018922+0000", 2024-09-13T10:11:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0086279619999999994 2024-09-13T10:11:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.019109+0000", 2024-09-13T10:11:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000186057 2024-09-13T10:11:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:206 3.2 e38/30 3:4cbdbe9e:::smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:11:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.325523582999999, 2024-09-13T10:11:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011224615, 2024-09-13T10:11:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:11:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:11:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010754+0000", 2024-09-13T10:11:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9999999999999999e-06 2024-09-13T10:11:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016750+0000", 2024-09-13T10:11:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059966480000000003 2024-09-13T10:11:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016782+0000", 2024-09-13T10:11:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2218999999999998e-05 2024-09-13T10:11:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016799+0000", 2024-09-13T10:11:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6685999999999999e-05 2024-09-13T10:11:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016923+0000", 2024-09-13T10:11:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000124122 2024-09-13T10:11:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016962+0000", 2024-09-13T10:11:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8748999999999998e-05 2024-09-13T10:11:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021869+0000", 2024-09-13T10:11:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049073249999999997 2024-09-13T10:11:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021976+0000", 2024-09-13T10:11:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106866 2024-09-13T10:11:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:214 3.2 e38/30 3:68ba95ba:::smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'3, mlcod=0'0)", 2024-09-13T10:11:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:11:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.315820927000001, 2024-09-13T10:11:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011518149, 2024-09-13T10:11:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:11:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:11:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020456+0000", 2024-09-13T10:11:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0430000000000002e-06 2024-09-13T10:11:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026998+0000", 2024-09-13T10:11:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065420169999999998 2024-09-13T10:11:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027030+0000", 2024-09-13T10:11:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1976e-05 2024-09-13T10:11:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027048+0000", 2024-09-13T10:11:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7615e-05 2024-09-13T10:11:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027206+0000", 2024-09-13T10:11:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015788900000000001 2024-09-13T10:11:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027283+0000", 2024-09-13T10:11:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6942999999999998e-05 2024-09-13T10:11:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.031808+0000", 2024-09-13T10:11:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045256940000000002 2024-09-13T10:11:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.031972+0000", 2024-09-13T10:11:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016397199999999999 2024-09-13T10:11:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:210 3.2 e38/30 3:46ab5247:::smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:11:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:11:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.307955208999999, 2024-09-13T10:11:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015681245, 2024-09-13T10:11:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:11:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:11:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028322+0000", 2024-09-13T10:11:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0200000000000001e-06 2024-09-13T10:11:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037903+0000", 2024-09-13T10:11:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095811639999999997 2024-09-13T10:11:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037945+0000", 2024-09-13T10:11:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1891000000000003e-05 2024-09-13T10:11:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037971+0000", 2024-09-13T10:11:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5829e-05 2024-09-13T10:11:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038125+0000", 2024-09-13T10:11:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000153852 2024-09-13T10:11:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038193+0000", 2024-09-13T10:11:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7995000000000002e-05 2024-09-13T10:11:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.043904+0000", 2024-09-13T10:11:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0057110340000000002 2024-09-13T10:11:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.044001+0000", 2024-09-13T10:11:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7460000000000005e-05 2024-09-13T10:11:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:218 3.2 e38/30 3:5915c927:::smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'5, mlcod=38'2)", 2024-09-13T10:11:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:11:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.294592623, 2024-09-13T10:11:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012988942, 2024-09-13T10:11:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.360 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.360 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:11:21.360 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.360 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.360 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.360 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.360 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:11:21.360 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.360 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.360 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.360 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.360 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041685+0000", 2024-09-13T10:11:21.360 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7769999999999998e-06 2024-09-13T10:11:21.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.361 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.361 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046575+0000", 2024-09-13T10:11:21.361 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048899520000000004 2024-09-13T10:11:21.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.361 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.361 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046633+0000", 2024-09-13T10:11:21.361 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7561e-05 2024-09-13T10:11:21.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.361 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.362 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046657+0000", 2024-09-13T10:11:21.362 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4223000000000002e-05 2024-09-13T10:11:21.362 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.362 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.362 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.362 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046819+0000", 2024-09-13T10:11:21.362 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016167699999999999 2024-09-13T10:11:21.362 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.362 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.362 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.362 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046882+0000", 2024-09-13T10:11:21.362 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3467999999999996e-05 2024-09-13T10:11:21.362 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.362 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.362 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.363 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054529+0000", 2024-09-13T10:11:21.363 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0076467289999999997 2024-09-13T10:11:21.363 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.363 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.363 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.363 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054671+0000", 2024-09-13T10:11:21.363 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014255499999999999 2024-09-13T10:11:21.363 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.363 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.363 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.363 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.363 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.363 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:217 3.2 e38/30 3:554c406d:::smithi19530714-18 low_tier:head v 38'6, mlcod=38'2)", 2024-09-13T10:11:21.364 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:11:21.364 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.289545832999998, 2024-09-13T10:11:21.364 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013473855999999999, 2024-09-13T10:11:21.364 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.364 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.364 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.364 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.364 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.364 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.364 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:11:21.364 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.364 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.364 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.364 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.365 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:11:21.365 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.365 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.365 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.365 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046731+0000", 2024-09-13T10:11:21.365 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.813e-06 2024-09-13T10:11:21.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.365 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.365 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.365 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054994+0000", 2024-09-13T10:11:21.365 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0082631279999999998 2024-09-13T10:11:21.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.366 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.366 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.366 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055024+0000", 2024-09-13T10:11:21.366 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9839e-05 2024-09-13T10:11:21.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.366 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.366 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.366 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055044+0000", 2024-09-13T10:11:21.366 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9507000000000001e-05 2024-09-13T10:11:21.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.366 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.366 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.366 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055186+0000", 2024-09-13T10:11:21.367 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014243099999999999 2024-09-13T10:11:21.367 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.367 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.367 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.367 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055240+0000", 2024-09-13T10:11:21.367 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4444999999999998e-05 2024-09-13T10:11:21.367 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.367 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.367 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.367 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060135+0000", 2024-09-13T10:11:21.367 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048942949999999999 2024-09-13T10:11:21.367 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.367 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.368 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.368 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060203+0000", 2024-09-13T10:11:21.368 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8398000000000005e-05 2024-09-13T10:11:21.368 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.368 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.368 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.368 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.368 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.368 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:220 3.2 e38/30 3:753d5785:::smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'7, mlcod=38'3)", 2024-09-13T10:11:21.368 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:11:21.368 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.281164565000001, 2024-09-13T10:11:21.368 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.009287455, 2024-09-13T10:11:21.368 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.369 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.369 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.369 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.369 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.369 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.369 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:11:21.369 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.369 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.369 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.369 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.369 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:11:21.369 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.369 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.369 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.370 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.370 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055112+0000", 2024-09-13T10:11:21.370 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6589999999999999e-06 2024-09-13T10:11:21.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.370 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.370 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.370 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057327+0000", 2024-09-13T10:11:21.370 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0022145229999999999 2024-09-13T10:11:21.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.370 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.370 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.370 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057354+0000", 2024-09-13T10:11:21.370 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7642000000000001e-05 2024-09-13T10:11:21.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.371 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.371 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.371 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057373+0000", 2024-09-13T10:11:21.371 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8768999999999999e-05 2024-09-13T10:11:21.371 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.371 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.371 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.371 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057511+0000", 2024-09-13T10:11:21.371 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013793900000000001 2024-09-13T10:11:21.371 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.371 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.371 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.371 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057550+0000", 2024-09-13T10:11:21.372 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9301999999999997e-05 2024-09-13T10:11:21.372 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.372 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.372 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.372 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.064324+0000", 2024-09-13T10:11:21.372 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0067737429999999996 2024-09-13T10:11:21.372 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.372 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.372 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.372 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.064398+0000", 2024-09-13T10:11:21.372 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3878e-05 2024-09-13T10:11:21.372 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.373 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.373 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.373 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.373 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.373 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:227 3.2 e38/30 3:6ac7800c:::smithi19530714-28 low_tier:head v 38'8, mlcod=38'6)", 2024-09-13T10:11:21.373 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:11:21.373 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.261573915, 2024-09-13T10:11:21.373 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018812816, 2024-09-13T10:11:21.373 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.373 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.373 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.373 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.373 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.374 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.374 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:11:21.374 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.374 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.374 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.374 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.374 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:11:21.374 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.374 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.374 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.374 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.374 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074708+0000", 2024-09-13T10:11:21.375 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.968e-06 2024-09-13T10:11:21.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.375 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.375 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.375 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085262+0000", 2024-09-13T10:11:21.375 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010553842000000001 2024-09-13T10:11:21.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.375 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.375 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.375 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085289+0000", 2024-09-13T10:11:21.375 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7288999999999999e-05 2024-09-13T10:11:21.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.375 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.376 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.376 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085305+0000", 2024-09-13T10:11:21.376 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5925000000000001e-05 2024-09-13T10:11:21.376 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.376 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.376 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.376 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085418+0000", 2024-09-13T10:11:21.376 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112584 2024-09-13T10:11:21.376 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.376 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.376 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.376 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085451+0000", 2024-09-13T10:11:21.376 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3605000000000001e-05 2024-09-13T10:11:21.376 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.377 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.377 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.377 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093417+0000", 2024-09-13T10:11:21.377 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0079654249999999999 2024-09-13T10:11:21.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.377 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.377 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.377 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093514+0000", 2024-09-13T10:11:21.377 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7177999999999997e-05 2024-09-13T10:11:21.377 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.377 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.377 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.378 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.378 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:230 3.2 e38/30 3:5d108080:::smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'9, mlcod=38'6)", 2024-09-13T10:11:21.378 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:11:21.378 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.247290885000002, 2024-09-13T10:11:21.378 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01953365, 2024-09-13T10:11:21.378 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.378 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.378 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.378 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.378 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.378 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.378 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:11:21.378 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.378 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.379 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.379 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:11:21.379 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.379 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.379 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.379 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088987+0000", 2024-09-13T10:11:21.379 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4600000000000002e-06 2024-09-13T10:11:21.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.379 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.379 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.379 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095842+0000", 2024-09-13T10:11:21.379 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0068548760000000002 2024-09-13T10:11:21.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.379 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.380 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.380 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095872+0000", 2024-09-13T10:11:21.380 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0476000000000001e-05 2024-09-13T10:11:21.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.380 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.380 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.380 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095890+0000", 2024-09-13T10:11:21.380 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7554000000000002e-05 2024-09-13T10:11:21.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.380 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.380 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.380 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.096006+0000", 2024-09-13T10:11:21.380 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011597199999999999 2024-09-13T10:11:21.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.380 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.381 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.096058+0000", 2024-09-13T10:11:21.381 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2274000000000001e-05 2024-09-13T10:11:21.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.381 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.381 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.108451+0000", 2024-09-13T10:11:21.381 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012392815 2024-09-13T10:11:21.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.381 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.381 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.108518+0000", 2024-09-13T10:11:21.381 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7223000000000003e-05 2024-09-13T10:11:21.382 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.382 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.382 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.382 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.382 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.382 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:232 3.2 e38/30 3:6538713c:::smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'10, mlcod=38'6)", 2024-09-13T10:11:21.382 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:11:21.382 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.240340004, 2024-09-13T10:11:21.382 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015307497, 2024-09-13T10:11:21.382 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.382 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.382 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.382 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.382 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.383 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.383 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:11:21.383 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.383 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.383 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.383 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.383 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:11:21.383 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.383 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.383 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.383 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.383 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095937+0000", 2024-09-13T10:11:21.383 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.367e-06 2024-09-13T10:11:21.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.384 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.384 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.384 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099841+0000", 2024-09-13T10:11:21.384 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0039043960000000001 2024-09-13T10:11:21.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.384 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.384 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.384 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099865+0000", 2024-09-13T10:11:21.384 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4538000000000001e-05 2024-09-13T10:11:21.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.384 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.384 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.385 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099881+0000", 2024-09-13T10:11:21.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5809000000000001e-05 2024-09-13T10:11:21.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.385 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.385 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099995+0000", 2024-09-13T10:11:21.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113843 2024-09-13T10:11:21.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.385 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.385 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.100043+0000", 2024-09-13T10:11:21.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8291999999999998e-05 2024-09-13T10:11:21.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111200+0000", 2024-09-13T10:11:21.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011156291 2024-09-13T10:11:21.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111243+0000", 2024-09-13T10:11:21.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2960999999999997e-05 2024-09-13T10:11:21.386 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.386 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.386 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.387 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:236 3.2 e38/30 3:73c2893b:::smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'11, mlcod=38'7)", 2024-09-13T10:11:21.387 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:11:21.387 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.230839885999998, 2024-09-13T10:11:21.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0098707670000000008, 2024-09-13T10:11:21.387 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.387 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.387 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.387 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.388 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:11:21.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.388 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.388 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.388 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:11:21.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.388 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.388 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.388 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105438+0000", 2024-09-13T10:11:21.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3010000000000001e-06 2024-09-13T10:11:21.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112262+0000", 2024-09-13T10:11:21.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.006824793 2024-09-13T10:11:21.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112279+0000", 2024-09-13T10:11:21.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6829000000000001e-05 2024-09-13T10:11:21.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112293+0000", 2024-09-13T10:11:21.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3801000000000001e-05 2024-09-13T10:11:21.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112381+0000", 2024-09-13T10:11:21.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7825999999999996e-05 2024-09-13T10:11:21.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112411+0000", 2024-09-13T10:11:21.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0434e-05 2024-09-13T10:11:21.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.115246+0000", 2024-09-13T10:11:21.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028352360000000001 2024-09-13T10:11:21.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.115306+0000", 2024-09-13T10:11:21.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9546999999999998e-05 2024-09-13T10:11:21.391 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.391 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.391 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.392 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:240 3.2 e38/30 3:7eee2f4a:::smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'12, mlcod=38'7)", 2024-09-13T10:11:21.392 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:11:21.392 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.223930099, 2024-09-13T10:11:21.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011743134000000001, 2024-09-13T10:11:21.392 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.392 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.392 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.392 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.392 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.392 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.392 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:11:21.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:11:21.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112346+0000", 2024-09-13T10:11:21.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.037e-06 2024-09-13T10:11:21.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120152+0000", 2024-09-13T10:11:21.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0078054700000000001 2024-09-13T10:11:21.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120173+0000", 2024-09-13T10:11:21.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1498999999999999e-05 2024-09-13T10:11:21.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120188+0000", 2024-09-13T10:11:21.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5159e-05 2024-09-13T10:11:21.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120313+0000", 2024-09-13T10:11:21.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012440199999999999 2024-09-13T10:11:21.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120358+0000", 2024-09-13T10:11:21.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-13T10:11:21.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.123945+0000", 2024-09-13T10:11:21.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035865490000000001 2024-09-13T10:11:21.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.124088+0000", 2024-09-13T10:11:21.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014318999999999999 2024-09-13T10:11:21.396 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.396 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.396 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.396 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:242 3.2 e38/30 3:501d22a1:::smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'13, mlcod=38'7)", 2024-09-13T10:11:21.396 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:11:21.396 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.216037091, 2024-09-13T10:11:21.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0088394330000000007, 2024-09-13T10:11:21.397 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.397 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.397 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.397 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.397 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.397 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.397 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:11:21.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.397 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.397 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.397 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:11:21.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120240+0000", 2024-09-13T10:11:21.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.731e-06 2024-09-13T10:11:21.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125395+0000", 2024-09-13T10:11:21.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0051547490000000001 2024-09-13T10:11:21.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125412+0000", 2024-09-13T10:11:21.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7833e-05 2024-09-13T10:11:21.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125426+0000", 2024-09-13T10:11:21.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3237999999999999e-05 2024-09-13T10:11:21.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125541+0000", 2024-09-13T10:11:21.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011566299999999999 2024-09-13T10:11:21.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125578+0000", 2024-09-13T10:11:21.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7197000000000003e-05 2024-09-13T10:11:21.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.128885+0000", 2024-09-13T10:11:21.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0033067119999999998 2024-09-13T10:11:21.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.401 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.401 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.129077+0000", 2024-09-13T10:11:21.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019231000000000001 2024-09-13T10:11:21.401 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.401 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.401 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.401 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:244 3.2 e38/30 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'14, mlcod=38'10)", 2024-09-13T10:11:21.401 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:11:21.401 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.210017723, 2024-09-13T10:11:21.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01536599, 2024-09-13T10:11:21.402 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.402 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.402 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.402 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:11:21.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:11:21.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126259+0000", 2024-09-13T10:11:21.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.528e-06 2024-09-13T10:11:21.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134336+0000", 2024-09-13T10:11:21.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080767410000000001 2024-09-13T10:11:21.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134356+0000", 2024-09-13T10:11:21.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9936999999999999e-05 2024-09-13T10:11:21.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134367+0000", 2024-09-13T10:11:21.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1352000000000001e-05 2024-09-13T10:11:21.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134532+0000", 2024-09-13T10:11:21.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001651 2024-09-13T10:11:21.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134578+0000", 2024-09-13T10:11:21.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5442000000000003e-05 2024-09-13T10:11:21.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.141521+0000", 2024-09-13T10:11:21.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069432720000000003 2024-09-13T10:11:21.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.141623+0000", 2024-09-13T10:11:21.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102618 2024-09-13T10:11:21.406 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.406 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.406 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.406 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:250 3.2 e38/30 3:78d40eb9:::smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'15, mlcod=38'10)", 2024-09-13T10:11:21.406 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:11:21.406 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.201821251999998, 2024-09-13T10:11:21.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095447210000000008, 2024-09-13T10:11:21.406 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.406 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.406 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.406 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.407 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:11:21.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.407 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:11:21.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.407 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134455+0000", 2024-09-13T10:11:21.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2699999999999999e-07 2024-09-13T10:11:21.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139352+0000", 2024-09-13T10:11:21.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048977810000000004 2024-09-13T10:11:21.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139374+0000", 2024-09-13T10:11:21.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1339e-05 2024-09-13T10:11:21.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139384+0000", 2024-09-13T10:11:21.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0183999999999999e-05 2024-09-13T10:11:21.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139477+0000", 2024-09-13T10:11:21.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2590999999999995e-05 2024-09-13T10:11:21.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139515+0000", 2024-09-13T10:11:21.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8624999999999997e-05 2024-09-13T10:11:21.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.143939+0000", 2024-09-13T10:11:21.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0044241189999999998 2024-09-13T10:11:21.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.143999+0000", 2024-09-13T10:11:21.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9354999999999997e-05 2024-09-13T10:11:21.410 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.410 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.410 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.411 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:245 3.2 e38/30 3:7837e849:::smithi19530714-46 low_tier:head v 38'16, mlcod=38'10)", 2024-09-13T10:11:21.411 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:11:21.411 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.196864898000001, 2024-09-13T10:11:21.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012687423999999999, 2024-09-13T10:11:21.411 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.411 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.411 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.411 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:11:21.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:11:21.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139411+0000", 2024-09-13T10:11:21.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6300000000000005e-07 2024-09-13T10:11:21.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144635+0000", 2024-09-13T10:11:21.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0052242169999999997 2024-09-13T10:11:21.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144651+0000", 2024-09-13T10:11:21.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6371e-05 2024-09-13T10:11:21.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144662+0000", 2024-09-13T10:11:21.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0003000000000001e-05 2024-09-13T10:11:21.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144741+0000", 2024-09-13T10:11:21.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9614e-05 2024-09-13T10:11:21.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144768+0000", 2024-09-13T10:11:21.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6826999999999999e-05 2024-09-13T10:11:21.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152015+0000", 2024-09-13T10:11:21.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0072470879999999996 2024-09-13T10:11:21.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152098+0000", 2024-09-13T10:11:21.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2640999999999997e-05 2024-09-13T10:11:21.415 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.415 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.415 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.415 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(osd.7.22:1 3.2 e38/30 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'17, mlcod=38'12)", 2024-09-13T10:11:21.415 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.144693+0000", 2024-09-13T10:11:21.416 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.191582126, 2024-09-13T10:11:21.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0075754180000000004, 2024-09-13T10:11:21.416 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.416 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.416 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.416 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144693+0000", 2024-09-13T10:11:21.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144693+0000", 2024-09-13T10:11:21.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144694+0000", 2024-09-13T10:11:21.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6500000000000008e-07 2024-09-13T10:11:21.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144699+0000", 2024-09-13T10:11:21.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6820000000000002e-06 2024-09-13T10:11:21.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144704+0000", 2024-09-13T10:11:21.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7869999999999998e-06 2024-09-13T10:11:21.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144710+0000", 2024-09-13T10:11:21.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3079999999999998e-06 2024-09-13T10:11:21.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.146857+0000", 2024-09-13T10:11:21.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021475209999999999 2024-09-13T10:11:21.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.146883+0000", 2024-09-13T10:11:21.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5947000000000001e-05 2024-09-13T10:11:21.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152225+0000", 2024-09-13T10:11:21.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053421450000000004 2024-09-13T10:11:21.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152268+0000", 2024-09-13T10:11:21.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3062999999999997e-05 2024-09-13T10:11:21.420 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.420 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.420 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.420 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:779 3.2 e39/30 3:7cce43d8:::4d6751ba15e95c360cd3577cdb9af7c034dc546188a4da3e77ae41469aed654d:head v 39'67, mlcod=39'65)", 2024-09-13T10:11:21.420 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.575055+0000", 2024-09-13T10:11:21.420 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 29.761220446999999, 2024-09-13T10:11:21.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0063294359999999999, 2024-09-13T10:11:21.421 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.421 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.421 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.421 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.575055+0000", 2024-09-13T10:11:21.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.575055+0000", 2024-09-13T10:11:21.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.575057+0000", 2024-09-13T10:11:21.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5830000000000001e-06 2024-09-13T10:11:21.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.577075+0000", 2024-09-13T10:11:21.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0020175879999999998 2024-09-13T10:11:21.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.577108+0000", 2024-09-13T10:11:21.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2659e-05 2024-09-13T10:11:21.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.577128+0000", 2024-09-13T10:11:21.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0585999999999999e-05 2024-09-13T10:11:21.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.577228+0000", 2024-09-13T10:11:21.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9925000000000003e-05 2024-09-13T10:11:21.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.577252+0000", 2024-09-13T10:11:21.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4450000000000001e-05 2024-09-13T10:11:21.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.581303+0000", 2024-09-13T10:11:21.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040503079999999999 2024-09-13T10:11:21.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.581384+0000", 2024-09-13T10:11:21.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1336999999999997e-05 2024-09-13T10:11:21.424 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.424 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.425 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.425 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:943 4.5 e42/41 4:a59e88f7:::smithi19530714-21:head v 42'36, mlcod=42'34)", 2024-09-13T10:11:21.425 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:54.640469+0000", 2024-09-13T10:11:21.425 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 26.695806454, 2024-09-13T10:11:21.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053006540000000001, 2024-09-13T10:11:21.425 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.425 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.425 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.425 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.640469+0000", 2024-09-13T10:11:21.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.640469+0000", 2024-09-13T10:11:21.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.640471+0000", 2024-09-13T10:11:21.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9369999999999998e-06 2024-09-13T10:11:21.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.641389+0000", 2024-09-13T10:11:21.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00091834700000000002 2024-09-13T10:11:21.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.641418+0000", 2024-09-13T10:11:21.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8682000000000001e-05 2024-09-13T10:11:21.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.641431+0000", 2024-09-13T10:11:21.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3349999999999999e-05 2024-09-13T10:11:21.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.641572+0000", 2024-09-13T10:11:21.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014117000000000001 2024-09-13T10:11:21.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.641593+0000", 2024-09-13T10:11:21.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0647000000000001e-05 2024-09-13T10:11:21.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.645732+0000", 2024-09-13T10:11:21.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00413949 2024-09-13T10:11:21.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:54.645769+0000", 2024-09-13T10:11:21.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7030999999999998e-05 2024-09-13T10:11:21.429 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.429 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.429 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.429 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(osd.6.0:104 4.5 e42/41 4:a7549961:::smithi19530714-27:head v 42'40, mlcod=42'38)", 2024-09-13T10:11:21.430 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:56.665769+0000", 2024-09-13T10:11:21.430 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 24.670506561, 2024-09-13T10:11:21.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065613520000000003, 2024-09-13T10:11:21.430 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.430 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.430 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.430 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.665769+0000", 2024-09-13T10:11:21.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.665769+0000", 2024-09-13T10:11:21.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.665771+0000", 2024-09-13T10:11:21.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.317e-06 2024-09-13T10:11:21.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.669348+0000", 2024-09-13T10:11:21.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035773689999999999 2024-09-13T10:11:21.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.669365+0000", 2024-09-13T10:11:21.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6623999999999999e-05 2024-09-13T10:11:21.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.669373+0000", 2024-09-13T10:11:21.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1820000000000006e-06 2024-09-13T10:11:21.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.669438+0000", 2024-09-13T10:11:21.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5265999999999995e-05 2024-09-13T10:11:21.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.669462+0000", 2024-09-13T10:11:21.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3555000000000001e-05 2024-09-13T10:11:21.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.672241+0000", 2024-09-13T10:11:21.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0027788650000000002 2024-09-13T10:11:21.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.672330+0000", 2024-09-13T10:11:21.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9173999999999997e-05 2024-09-13T10:11:21.434 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.434 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.434 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.434 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.434 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.434 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:21.435 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:21.436 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:21.434+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:21.437 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T10:11:21.454 INFO:tasks.rados.rados.0.smithi195.stdout:1140: finishing write tid 1 to smithi19530714-25 2024-09-13T10:11:21.454 INFO:tasks.rados.rados.0.smithi195.stdout:1140: finishing write tid 2 to smithi19530714-25 2024-09-13T10:11:21.454 INFO:tasks.rados.rados.0.smithi195.stdout:1140: finishing write tid 3 to smithi19530714-25 2024-09-13T10:11:21.454 INFO:tasks.rados.rados.0.smithi195.stdout:1140: finishing write tid 4 to smithi19530714-25 2024-09-13T10:11:21.454 INFO:tasks.rados.rados.0.smithi195.stdout:1140: finishing write tid 5 to smithi19530714-25 2024-09-13T10:11:21.454 INFO:tasks.rados.rados.0.smithi195.stdout:1140: oid 25 updating version 0 to 114 2024-09-13T10:11:21.455 INFO:tasks.rados.rados.0.smithi195.stdout:1140: oid 25 version 114 is already newer than 111 2024-09-13T10:11:21.455 INFO:tasks.rados.rados.0.smithi195.stdout:1140: oid 25 version 114 is already newer than 112 2024-09-13T10:11:21.455 INFO:tasks.rados.rados.0.smithi195.stdout:1140: oid 25 version 114 is already newer than 113 2024-09-13T10:11:21.455 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 114 (ObjNum 210 snap 0 seq_num 210) dirty exists 2024-09-13T10:11:21.455 INFO:tasks.rados.rados.0.smithi195.stdout:1140: left oid 25 (ObjNum 210 snap 0 seq_num 210) 2024-09-13T10:11:21.455 INFO:tasks.rados.rados.0.smithi195.stdout:1141: expect (ObjNum 184 snap 0 seq_num 184) 2024-09-13T10:11:21.456 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:21.456 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:21.464 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1041182 2024-09-13T10:11:21.523 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:21.537 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:21.534+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:21.572 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:21.573 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:21.573 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:11:21.573 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:21.587 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T10:11:21.596 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:21.638 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:21.632+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:21.669 INFO:tasks.rados.rados.0.smithi195.stdout:1144: expect (ObjNum 207 snap 0 seq_num 207) 2024-09-13T10:11:21.710 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:21.710 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:21.710 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:11:21.710 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:11:21.711 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:21.725 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T10:11:21.739 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:21.736+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:21.784 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-13T10:11:21.832 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2071804 2024-09-13T10:11:21.839 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:21.835+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:21.849 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:21.849 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:11:21.849 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:11:21.849 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:11:21.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.849 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:203 3.1 e38/30 3:b033e801:::smithi19530714-4 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:21.849 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.002504+0000", 2024-09-13T10:11:21.849 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.844716108, 2024-09-13T10:11:21.849 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016371101999999998, 2024-09-13T10:11:21.850 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.850 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.850 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.850 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.850 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.850 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002504+0000", 2024-09-13T10:11:21.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.850 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.850 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002504+0000", 2024-09-13T10:11:21.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.851 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002507+0000", 2024-09-13T10:11:21.851 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2699999999999999e-06 2024-09-13T10:11:21.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.851 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007079+0000", 2024-09-13T10:11:21.851 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045722200000000001 2024-09-13T10:11:21.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.852 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007114+0000", 2024-09-13T10:11:21.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5389000000000003e-05 2024-09-13T10:11:21.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.852 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.852 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.852 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007132+0000", 2024-09-13T10:11:21.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7489000000000001e-05 2024-09-13T10:11:21.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.852 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.852 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.852 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007266+0000", 2024-09-13T10:11:21.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013401900000000001 2024-09-13T10:11:21.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007299+0000", 2024-09-13T10:11:21.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3136000000000003e-05 2024-09-13T10:11:21.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.018741+0000", 2024-09-13T10:11:21.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011441957000000001 2024-09-13T10:11:21.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.854 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.854 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.018875+0000", 2024-09-13T10:11:21.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013462200000000001 2024-09-13T10:11:21.854 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.854 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.854 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.854 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:211 3.0 e38/30 3:0596651d:::smithi19530714-12 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:21.854 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.007453+0000", 2024-09-13T10:11:21.854 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.839767284000001, 2024-09-13T10:11:21.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018632359000000001, 2024-09-13T10:11:21.854 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.854 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.855 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.855 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.855 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.855 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007453+0000", 2024-09-13T10:11:21.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.855 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.856 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007453+0000", 2024-09-13T10:11:21.856 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.856 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.856 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007455+0000", 2024-09-13T10:11:21.856 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4030000000000001e-06 2024-09-13T10:11:21.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.856 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.856 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020746+0000", 2024-09-13T10:11:21.856 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013290219000000001 2024-09-13T10:11:21.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020788+0000", 2024-09-13T10:11:21.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2342000000000002e-05 2024-09-13T10:11:21.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020868+0000", 2024-09-13T10:11:21.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9519999999999998e-05 2024-09-13T10:11:21.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021080+0000", 2024-09-13T10:11:21.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021249600000000001 2024-09-13T10:11:21.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021113+0000", 2024-09-13T10:11:21.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2786000000000001e-05 2024-09-13T10:11:21.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.025974+0000", 2024-09-13T10:11:21.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048614929999999997 2024-09-13T10:11:21.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026085+0000", 2024-09-13T10:11:21.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001111 2024-09-13T10:11:21.859 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.859 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.859 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.859 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:205 3.1 e38/30 3:aec11a92:::smithi19530714-6 low_tier:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:21.859 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.014358+0000", 2024-09-13T10:11:21.859 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.83286266, 2024-09-13T10:11:21.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033491923999999999, 2024-09-13T10:11:21.859 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.859 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.859 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.860 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.014358+0000", 2024-09-13T10:11:21.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.014358+0000", 2024-09-13T10:11:21.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.014360+0000", 2024-09-13T10:11:21.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.057e-06 2024-09-13T10:11:21.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026358+0000", 2024-09-13T10:11:21.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011998688 2024-09-13T10:11:21.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026392+0000", 2024-09-13T10:11:21.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3575999999999998e-05 2024-09-13T10:11:21.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.862 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027340+0000", 2024-09-13T10:11:21.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00094826100000000005 2024-09-13T10:11:21.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.862 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.862 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027449+0000", 2024-09-13T10:11:21.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108797 2024-09-13T10:11:21.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.862 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.862 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027500+0000", 2024-09-13T10:11:21.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0757999999999998e-05 2024-09-13T10:11:21.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047748+0000", 2024-09-13T10:11:21.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020247884000000001 2024-09-13T10:11:21.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047850+0000", 2024-09-13T10:11:21.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101903 2024-09-13T10:11:21.863 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.863 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.863 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.864 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:212 3.0 e38/30 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:21.864 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.021018+0000", 2024-09-13T10:11:21.864 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.826201852000001, 2024-09-13T10:11:21.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044503290000000001, 2024-09-13T10:11:21.864 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.864 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.864 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.864 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021018+0000", 2024-09-13T10:11:21.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.865 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021018+0000", 2024-09-13T10:11:21.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021019+0000", 2024-09-13T10:11:21.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7400000000000002e-07 2024-09-13T10:11:21.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038659+0000", 2024-09-13T10:11:21.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017639555000000001 2024-09-13T10:11:21.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038678+0000", 2024-09-13T10:11:21.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8912000000000001e-05 2024-09-13T10:11:21.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038693+0000", 2024-09-13T10:11:21.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5393e-05 2024-09-13T10:11:21.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038829+0000", 2024-09-13T10:11:21.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000135912 2024-09-13T10:11:21.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038887+0000", 2024-09-13T10:11:21.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7858000000000001e-05 2024-09-13T10:11:21.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065395+0000", 2024-09-13T10:11:21.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02650773 2024-09-13T10:11:21.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.869 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065522+0000", 2024-09-13T10:11:21.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012705600000000001 2024-09-13T10:11:21.869 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.869 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.869 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.869 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:213 3.1 e38/30 3:accf4e56:::smithi19530714-14 low_tier:head v 38'3, mlcod=0'0)", 2024-09-13T10:11:21.870 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.027392+0000", 2024-09-13T10:11:21.870 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.819828343000001, 2024-09-13T10:11:21.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025423449000000001, 2024-09-13T10:11:21.870 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.870 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.870 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.870 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027392+0000", 2024-09-13T10:11:21.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027392+0000", 2024-09-13T10:11:21.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027394+0000", 2024-09-13T10:11:21.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7370000000000001e-06 2024-09-13T10:11:21.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032354+0000", 2024-09-13T10:11:21.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049601589999999996 2024-09-13T10:11:21.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032413+0000", 2024-09-13T10:11:21.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9095000000000003e-05 2024-09-13T10:11:21.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032426+0000", 2024-09-13T10:11:21.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3465999999999999e-05 2024-09-13T10:11:21.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032805+0000", 2024-09-13T10:11:21.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000378704 2024-09-13T10:11:21.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032862+0000", 2024-09-13T10:11:21.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6878e-05 2024-09-13T10:11:21.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.052688+0000", 2024-09-13T10:11:21.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019826308000000001 2024-09-13T10:11:21.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.052815+0000", 2024-09-13T10:11:21.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127102 2024-09-13T10:11:21.874 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.874 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.874 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.875 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.875 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:209 3.1 e38/30 3:9798d58d:::smithi19530714-10 low_tier:head v 38'4, mlcod=0'0)", 2024-09-13T10:11:21.875 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.032594+0000", 2024-09-13T10:11:21.875 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.814625908, 2024-09-13T10:11:21.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028120108000000001, 2024-09-13T10:11:21.875 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.875 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.875 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.875 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.875 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.875 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.875 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032594+0000", 2024-09-13T10:11:21.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032594+0000", 2024-09-13T10:11:21.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032596+0000", 2024-09-13T10:11:21.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2330000000000001e-06 2024-09-13T10:11:21.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.035217+0000", 2024-09-13T10:11:21.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026212679999999999 2024-09-13T10:11:21.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.035250+0000", 2024-09-13T10:11:21.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3065000000000003e-05 2024-09-13T10:11:21.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.035272+0000", 2024-09-13T10:11:21.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2163000000000001e-05 2024-09-13T10:11:21.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038562+0000", 2024-09-13T10:11:21.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003290322 2024-09-13T10:11:21.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038642+0000", 2024-09-13T10:11:21.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9457999999999997e-05 2024-09-13T10:11:21.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060605+0000", 2024-09-13T10:11:21.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021962751999999999 2024-09-13T10:11:21.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060715+0000", 2024-09-13T10:11:21.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010984699999999999 2024-09-13T10:11:21.879 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.879 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.879 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.879 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:207 3.0 e38/30 3:332cd44b:::smithi19530714-8 low_tier:head v 38'3, mlcod=0'0)", 2024-09-13T10:11:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.038735+0000", 2024-09-13T10:11:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.808485329, 2024-09-13T10:11:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.026872200999999998, 2024-09-13T10:11:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038735+0000", 2024-09-13T10:11:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038735+0000", 2024-09-13T10:11:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038737+0000", 2024-09-13T10:11:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5430000000000001e-06 2024-09-13T10:11:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046860+0000", 2024-09-13T10:11:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081236629999999997 2024-09-13T10:11:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046895+0000", 2024-09-13T10:11:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4989e-05 2024-09-13T10:11:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046915+0000", 2024-09-13T10:11:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0026000000000001e-05 2024-09-13T10:11:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047057+0000", 2024-09-13T10:11:21.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014151800000000001 2024-09-13T10:11:21.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047111+0000", 2024-09-13T10:11:21.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3885000000000003e-05 2024-09-13T10:11:21.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065539+0000", 2024-09-13T10:11:21.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018428225999999999 2024-09-13T10:11:21.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065607+0000", 2024-09-13T10:11:21.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8350999999999997e-05 2024-09-13T10:11:21.884 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.884 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.884 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.884 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:216 3.0 e38/30 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:11:21.884 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.046965+0000", 2024-09-13T10:11:21.884 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.800254866, 2024-09-13T10:11:21.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.022018617000000001, 2024-09-13T10:11:21.884 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.885 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.885 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.885 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046965+0000", 2024-09-13T10:11:21.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046965+0000", 2024-09-13T10:11:21.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046967+0000", 2024-09-13T10:11:21.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3689999999999999e-06 2024-09-13T10:11:21.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054950+0000", 2024-09-13T10:11:21.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0079834910000000005 2024-09-13T10:11:21.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054967+0000", 2024-09-13T10:11:21.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6382e-05 2024-09-13T10:11:21.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054981+0000", 2024-09-13T10:11:21.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3974999999999999e-05 2024-09-13T10:11:21.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055057+0000", 2024-09-13T10:11:21.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5964000000000007e-05 2024-09-13T10:11:21.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055100+0000", 2024-09-13T10:11:21.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3522000000000001e-05 2024-09-13T10:11:21.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.068892+0000", 2024-09-13T10:11:21.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01379175 2024-09-13T10:11:21.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.068984+0000", 2024-09-13T10:11:21.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2163999999999994e-05 2024-09-13T10:11:21.889 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.889 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.889 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.889 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:219 3.1 e38/30 3:87bfc7b1:::smithi19530714-20 low_tier:head v 38'5, mlcod=0'0)", 2024-09-13T10:11:21.889 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.048853+0000", 2024-09-13T10:11:21.889 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.798367394, 2024-09-13T10:11:21.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017128925999999999, 2024-09-13T10:11:21.889 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.889 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.889 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.889 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.048853+0000", 2024-09-13T10:11:21.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.048853+0000", 2024-09-13T10:11:21.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.048855+0000", 2024-09-13T10:11:21.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.877e-06 2024-09-13T10:11:21.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053357+0000", 2024-09-13T10:11:21.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045020149999999998 2024-09-13T10:11:21.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053378+0000", 2024-09-13T10:11:21.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1639000000000001e-05 2024-09-13T10:11:21.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053396+0000", 2024-09-13T10:11:21.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7135000000000001e-05 2024-09-13T10:11:21.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053526+0000", 2024-09-13T10:11:21.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013083999999999999 2024-09-13T10:11:21.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053590+0000", 2024-09-13T10:11:21.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3355000000000006e-05 2024-09-13T10:11:21.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065909+0000", 2024-09-13T10:11:21.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012319583 2024-09-13T10:11:21.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065982+0000", 2024-09-13T10:11:21.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2482000000000002e-05 2024-09-13T10:11:21.893 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.893 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.893 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.894 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:221 3.0 e38/30 3:18ae852b:::smithi19530714-22 low_tier:head v 38'5, mlcod=0'0)", 2024-09-13T10:11:21.894 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.058895+0000", 2024-09-13T10:11:21.894 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.788325179000001, 2024-09-13T10:11:21.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019544733000000002, 2024-09-13T10:11:21.894 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.894 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.894 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.894 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.058895+0000", 2024-09-13T10:11:21.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.058895+0000", 2024-09-13T10:11:21.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.058897+0000", 2024-09-13T10:11:21.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0420000000000001e-06 2024-09-13T10:11:21.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.061589+0000", 2024-09-13T10:11:21.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026919869999999999 2024-09-13T10:11:21.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.061614+0000", 2024-09-13T10:11:21.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4984999999999999e-05 2024-09-13T10:11:21.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.061629+0000", 2024-09-13T10:11:21.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4501999999999999e-05 2024-09-13T10:11:21.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065668+0000", 2024-09-13T10:11:21.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040393260000000002 2024-09-13T10:11:21.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065711+0000", 2024-09-13T10:11:21.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2673999999999998e-05 2024-09-13T10:11:21.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.078336+0000", 2024-09-13T10:11:21.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012625113 2024-09-13T10:11:21.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.078440+0000", 2024-09-13T10:11:21.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000104104 2024-09-13T10:11:21.898 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.898 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.898 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.899 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:231 3.1 e38/30 3:9d247594:::smithi19530714-32 low_tier:head v 38'7, mlcod=38'4)", 2024-09-13T10:11:21.899 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.076614+0000", 2024-09-13T10:11:21.899 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.770606216000001, 2024-09-13T10:11:21.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017222986999999999, 2024-09-13T10:11:21.899 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.899 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.899 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.899 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.076614+0000", 2024-09-13T10:11:21.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.076614+0000", 2024-09-13T10:11:21.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.076616+0000", 2024-09-13T10:11:21.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5069999999999999e-06 2024-09-13T10:11:21.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082649+0000", 2024-09-13T10:11:21.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0060329499999999996 2024-09-13T10:11:21.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082674+0000", 2024-09-13T10:11:21.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5624000000000001e-05 2024-09-13T10:11:21.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082690+0000", 2024-09-13T10:11:21.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6152999999999999e-05 2024-09-13T10:11:21.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082812+0000", 2024-09-13T10:11:21.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012209499999999999 2024-09-13T10:11:21.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082863+0000", 2024-09-13T10:11:21.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0781000000000001e-05 2024-09-13T10:11:21.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093763+0000", 2024-09-13T10:11:21.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010899382000000001 2024-09-13T10:11:21.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093837+0000", 2024-09-13T10:11:21.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4494999999999997e-05 2024-09-13T10:11:21.903 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.904 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.904 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.904 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:234 3.1 e38/30 3:bdce9caa:::smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'8, mlcod=38'4)", 2024-09-13T10:11:21.904 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.086591+0000", 2024-09-13T10:11:21.904 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.760629102999999, 2024-09-13T10:11:21.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025699538000000001, 2024-09-13T10:11:21.904 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.904 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.904 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.904 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.086591+0000", 2024-09-13T10:11:21.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.086591+0000", 2024-09-13T10:11:21.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.086623+0000", 2024-09-13T10:11:21.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1335999999999999e-05 2024-09-13T10:11:21.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094695+0000", 2024-09-13T10:11:21.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080726919999999994 2024-09-13T10:11:21.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.906 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094716+0000", 2024-09-13T10:11:21.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1015000000000001e-05 2024-09-13T10:11:21.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.906 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094732+0000", 2024-09-13T10:11:21.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5532999999999998e-05 2024-09-13T10:11:21.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094807+0000", 2024-09-13T10:11:21.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4837000000000002e-05 2024-09-13T10:11:21.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094836+0000", 2024-09-13T10:11:21.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8912999999999999e-05 2024-09-13T10:11:21.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112239+0000", 2024-09-13T10:11:21.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017402995000000001 2024-09-13T10:11:21.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112291+0000", 2024-09-13T10:11:21.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2216999999999998e-05 2024-09-13T10:11:21.908 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.908 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.908 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.908 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:233 3.0 e38/30 3:24ead787:::smithi19530714-34 low_tier:head v 38'9, mlcod=38'4)", 2024-09-13T10:11:21.909 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.090071+0000", 2024-09-13T10:11:21.909 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.757149096999999, 2024-09-13T10:11:21.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.023396641999999999, 2024-09-13T10:11:21.909 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.909 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.909 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.909 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090071+0000", 2024-09-13T10:11:21.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090071+0000", 2024-09-13T10:11:21.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090073+0000", 2024-09-13T10:11:21.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0470000000000001e-06 2024-09-13T10:11:21.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097736+0000", 2024-09-13T10:11:21.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007662416 2024-09-13T10:11:21.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097761+0000", 2024-09-13T10:11:21.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4919e-05 2024-09-13T10:11:21.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097773+0000", 2024-09-13T10:11:21.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2182e-05 2024-09-13T10:11:21.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097877+0000", 2024-09-13T10:11:21.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010438900000000001 2024-09-13T10:11:21.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.912 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.912 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097910+0000", 2024-09-13T10:11:21.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2808999999999997e-05 2024-09-13T10:11:21.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.912 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.912 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113429+0000", 2024-09-13T10:11:21.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015518557000000001 2024-09-13T10:11:21.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.912 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.912 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113468+0000", 2024-09-13T10:11:21.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9323000000000003e-05 2024-09-13T10:11:21.912 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.913 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.913 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.913 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:223 3.1 e38/30 3:8759f4e0:::smithi19530714-24 low_tier:head v 38'9, mlcod=38'4)", 2024-09-13T10:11:21.913 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.094775+0000", 2024-09-13T10:11:21.913 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.75244558, 2024-09-13T10:11:21.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018476023000000001, 2024-09-13T10:11:21.913 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.913 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.913 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.913 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.913 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094775+0000", 2024-09-13T10:11:21.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094775+0000", 2024-09-13T10:11:21.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094776+0000", 2024-09-13T10:11:21.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.466e-06 2024-09-13T10:11:21.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097100+0000", 2024-09-13T10:11:21.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002323435 2024-09-13T10:11:21.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097123+0000", 2024-09-13T10:11:21.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3756000000000001e-05 2024-09-13T10:11:21.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097139+0000", 2024-09-13T10:11:21.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5424e-05 2024-09-13T10:11:21.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.098642+0000", 2024-09-13T10:11:21.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015034359999999999 2024-09-13T10:11:21.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.098683+0000", 2024-09-13T10:11:21.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0756000000000001e-05 2024-09-13T10:11:21.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113193+0000", 2024-09-13T10:11:21.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014509831000000001 2024-09-13T10:11:21.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113251+0000", 2024-09-13T10:11:21.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7918999999999999e-05 2024-09-13T10:11:21.917 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.917 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.917 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.917 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:241 3.0 e38/30 3:388ca573:::smithi19530714-42 low_tier:head v 38'12, mlcod=38'9)", 2024-09-13T10:11:21.917 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.137896+0000", 2024-09-13T10:11:21.918 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.709323927, 2024-09-13T10:11:21.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015462800000000001, 2024-09-13T10:11:21.918 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.918 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.918 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:21.918 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.137896+0000", 2024-09-13T10:11:21.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.137896+0000", 2024-09-13T10:11:21.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.137897+0000", 2024-09-13T10:11:21.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5099999999999998e-07 2024-09-13T10:11:21.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.919 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.919 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.142234+0000", 2024-09-13T10:11:21.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0043362160000000004 2024-09-13T10:11:21.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.142248+0000", 2024-09-13T10:11:21.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4854e-05 2024-09-13T10:11:21.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.142257+0000", 2024-09-13T10:11:21.920 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5820000000000002e-06 2024-09-13T10:11:21.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.920 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.142334+0000", 2024-09-13T10:11:21.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6518000000000002e-05 2024-09-13T10:11:21.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:21.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.142367+0000", 2024-09-13T10:11:21.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3667000000000001e-05 2024-09-13T10:11:21.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:21.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152035+0000", 2024-09-13T10:11:21.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0096680550000000001 2024-09-13T10:11:21.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.153359+0000", 2024-09-13T10:11:21.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001323957 2024-09-13T10:11:21.922 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.922 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.922 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.922 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1959 4.13 4.e5457733 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:21.922 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.866379+0000", 2024-09-13T10:11:21.922 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.9808412149999999, 2024-09-13T10:11:21.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.83266194900000001, 2024-09-13T10:11:21.922 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.922 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.923 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:11:21.923 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:21.923 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:21.923 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:21.923 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1959 2024-09-13T10:11:21.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.923 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.866379+0000", 2024-09-13T10:11:21.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.866379+0000", 2024-09-13T10:11:21.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.866381+0000", 2024-09-13T10:11:21.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.125e-06 2024-09-13T10:11:21.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867454+0000", 2024-09-13T10:11:21.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001072649 2024-09-13T10:11:21.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867475+0000", 2024-09-13T10:11:21.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1299999999999999e-05 2024-09-13T10:11:21.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867486+0000", 2024-09-13T10:11:21.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1167e-05 2024-09-13T10:11:21.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867581+0000", 2024-09-13T10:11:21.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4561999999999993e-05 2024-09-13T10:11:21.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:11:21.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867592+0000", 2024-09-13T10:11:21.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1031e-05 2024-09-13T10:11:21.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.926 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.926 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.926 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699018+0000", 2024-09-13T10:11:21.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.83142604099999995 2024-09-13T10:11:21.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699041+0000", 2024-09-13T10:11:21.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3074e-05 2024-09-13T10:11:21.927 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.927 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.927 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.927 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1960 4.13 4.e5457733 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:21.927 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.874165+0000", 2024-09-13T10:11:21.927 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.9730555750000001, 2024-09-13T10:11:21.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82492870799999996, 2024-09-13T10:11:21.928 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.928 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.928 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:11:21.928 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:21.928 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:21.928 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:21.928 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1960 2024-09-13T10:11:21.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.928 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874165+0000", 2024-09-13T10:11:21.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874165+0000", 2024-09-13T10:11:21.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874166+0000", 2024-09-13T10:11:21.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.325e-06 2024-09-13T10:11:21.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874706+0000", 2024-09-13T10:11:21.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00053996899999999995 2024-09-13T10:11:21.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874719+0000", 2024-09-13T10:11:21.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2866e-05 2024-09-13T10:11:21.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874726+0000", 2024-09-13T10:11:21.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7560000000000002e-06 2024-09-13T10:11:21.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874801+0000", 2024-09-13T10:11:21.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5612999999999996e-05 2024-09-13T10:11:21.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:11:21.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874810+0000", 2024-09-13T10:11:21.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2619999999999998e-06 2024-09-13T10:11:21.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699082+0000", 2024-09-13T10:11:21.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82427217699999999 2024-09-13T10:11:21.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699093+0000", 2024-09-13T10:11:21.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1739999999999999e-05 2024-09-13T10:11:21.932 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.932 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.932 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.932 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1961 4.13 4.e5457733 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:21.933 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.877465+0000", 2024-09-13T10:11:21.933 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.9697553239999994, 2024-09-13T10:11:21.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82170111099999998, 2024-09-13T10:11:21.933 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.933 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.933 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:11:21.933 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:21.933 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:21.933 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:21.933 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1961 2024-09-13T10:11:21.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.933 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877465+0000", 2024-09-13T10:11:21.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877465+0000", 2024-09-13T10:11:21.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877466+0000", 2024-09-13T10:11:21.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0100000000000004e-07 2024-09-13T10:11:21.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877640+0000", 2024-09-13T10:11:21.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000173795 2024-09-13T10:11:21.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877650+0000", 2024-09-13T10:11:21.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0465000000000001e-05 2024-09-13T10:11:21.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877656+0000", 2024-09-13T10:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6849999999999998e-06 2024-09-13T10:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877714+0000", 2024-09-13T10:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8097999999999998e-05 2024-09-13T10:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877726+0000", 2024-09-13T10:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1857e-05 2024-09-13T10:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699146+0000", 2024-09-13T10:11:21.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82142019700000002 2024-09-13T10:11:21.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699166+0000", 2024-09-13T10:11:21.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0213000000000001e-05 2024-09-13T10:11:21.937 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.937 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.937 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.938 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1962 4.13 4.e5457733 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:21.938 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.877669+0000", 2024-09-13T10:11:21.938 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.9695511890000006, 2024-09-13T10:11:21.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82156359899999998, 2024-09-13T10:11:21.938 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.938 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.938 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:11:21.938 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:21.938 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:21.938 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:21.938 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1962 2024-09-13T10:11:21.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.939 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877669+0000", 2024-09-13T10:11:21.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877669+0000", 2024-09-13T10:11:21.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877670+0000", 2024-09-13T10:11:21.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9899999999999999e-07 2024-09-13T10:11:21.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877675+0000", 2024-09-13T10:11:21.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5510000000000001e-06 2024-09-13T10:11:21.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877684+0000", 2024-09-13T10:11:21.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5829999999999996e-06 2024-09-13T10:11:21.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877694+0000", 2024-09-13T10:11:21.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0207000000000001e-05 2024-09-13T10:11:21.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877784+0000", 2024-09-13T10:11:21.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9990999999999999e-05 2024-09-13T10:11:21.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:11:21.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877798+0000", 2024-09-13T10:11:21.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3444e-05 2024-09-13T10:11:21.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699222+0000", 2024-09-13T10:11:21.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82142437999999995 2024-09-13T10:11:21.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699233+0000", 2024-09-13T10:11:21.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0644e-05 2024-09-13T10:11:21.942 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.942 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.943 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.943 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1963 4.13 4.e5457733 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:21.943 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.877721+0000", 2024-09-13T10:11:21.943 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.9694989310000004, 2024-09-13T10:11:21.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82158741499999999, 2024-09-13T10:11:21.943 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:21.943 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:21.943 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:11:21.943 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:21.943 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:21.943 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:21.944 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1963 2024-09-13T10:11:21.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.944 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:21.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:21.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877721+0000", 2024-09-13T10:11:21.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:21.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877721+0000", 2024-09-13T10:11:21.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:21.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:21.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877722+0000", 2024-09-13T10:11:21.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4099999999999998e-07 2024-09-13T10:11:21.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:21.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877728+0000", 2024-09-13T10:11:21.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1129999999999997e-06 2024-09-13T10:11:21.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:21.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877737+0000", 2024-09-13T10:11:21.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3969999999999994e-06 2024-09-13T10:11:21.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:21.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877743+0000", 2024-09-13T10:11:21.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7950000000000002e-06 2024-09-13T10:11:21.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877826+0000", 2024-09-13T10:11:21.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2213999999999996e-05 2024-09-13T10:11:21.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:11:21.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877831+0000", 2024-09-13T10:11:21.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.604e-06 2024-09-13T10:11:21.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:21.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699295+0000", 2024-09-13T10:11:21.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82146349299999999 2024-09-13T10:11:21.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:21.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:21.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:21.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699309+0000", 2024-09-13T10:11:21.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4058e-05 2024-09-13T10:11:21.947 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.948 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.948 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.948 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:21.948 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:21.948 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:21.949 INFO:tasks.rados.rados.0.smithi195.stdout:1143: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:21.949 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 133647 2024-09-13T10:11:21.950 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-13T10:11:21.950 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:21.947+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:21.980 INFO:tasks.rados.rados.0.smithi195.stdout:1145: finishing set_chunk to oid 20 2024-09-13T10:11:21.980 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 58 (ObjNum 194 snap 0 seq_num 194) dirty exists 2024-09-13T10:11:21.981 INFO:tasks.rados.rados.0.smithi195.stdout:1146: expect (ObjNum 176 snap 0 seq_num 176) 2024-09-13T10:11:22.050 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:22.048+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:22.094 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-13T10:11:22.128 INFO:tasks.rados.rados.0.smithi195.stdout:1148: finishing set_chunk to oid 42 2024-09-13T10:11:22.129 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 42 set_chunk returned error code -95 offset: 291328 length: 1497600 2024-09-13T10:11:22.129 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-13T10:11:22.129 INFO:tasks.rados.rados.0.smithi195.stdout:1115: expect (ObjNum 102 snap 0 seq_num 102) 2024-09-13T10:11:22.151 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:22.148+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:22.184 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 299511 2024-09-13T10:11:22.219 INFO:tasks.rados.rados.0.smithi195.stdout:1115: done (15 left) 2024-09-13T10:11:22.220 INFO:tasks.rados.rados.0.smithi195.stdout:1134: done (14 left) 2024-09-13T10:11:22.220 INFO:tasks.rados.rados.0.smithi195.stdout:1135: done (13 left) 2024-09-13T10:11:22.220 INFO:tasks.rados.rados.0.smithi195.stdout:1136: done (12 left) 2024-09-13T10:11:22.220 INFO:tasks.rados.rados.0.smithi195.stdout:1137: done (11 left) 2024-09-13T10:11:22.220 INFO:tasks.rados.rados.0.smithi195.stdout:1138: done (10 left) 2024-09-13T10:11:22.220 INFO:tasks.rados.rados.0.smithi195.stdout:1139: done (9 left) 2024-09-13T10:11:22.220 INFO:tasks.rados.rados.0.smithi195.stdout:1140: done (8 left) 2024-09-13T10:11:22.220 INFO:tasks.rados.rados.0.smithi195.stdout:1141: done (7 left) 2024-09-13T10:11:22.220 INFO:tasks.rados.rados.0.smithi195.stdout:1142: done (6 left) 2024-09-13T10:11:22.220 INFO:tasks.rados.rados.0.smithi195.stdout:1143: done (5 left) 2024-09-13T10:11:22.220 INFO:tasks.rados.rados.0.smithi195.stdout:1144: done (4 left) 2024-09-13T10:11:22.220 INFO:tasks.rados.rados.0.smithi195.stdout:1145: done (3 left) 2024-09-13T10:11:22.220 INFO:tasks.rados.rados.0.smithi195.stdout:1146: done (2 left) 2024-09-13T10:11:22.220 INFO:tasks.rados.rados.0.smithi195.stdout:1147: done (1 left) 2024-09-13T10:11:22.221 INFO:tasks.rados.rados.0.smithi195.stdout:1148: done (0 left) 2024-09-13T10:11:22.221 INFO:tasks.rados.rados.0.smithi195.stdout:1149: set_chunk oid 5 target oid 2024-09-13T10:11:22.221 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:22.221 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:22.221 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:11:22.221 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:11:22.221 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:22.232 INFO:tasks.rados.rados.0.smithi195.stdout:1149: set_chunk oid 5 offset: 2278912 length: 354816 target oid 9aeceb9845561eff20338338a6b5a53aee2d550b22ae99a937785d193e80939b offset: 0 snap -1 2024-09-13T10:11:22.233 INFO:tasks.rados.rados.0.smithi195.stdout:1150: read oid 21 snap -1 2024-09-13T10:11:22.233 INFO:tasks.rados.rados.0.smithi195.stdout:1150: expect (ObjNum 197 snap 0 seq_num 197) 2024-09-13T10:11:22.233 INFO:tasks.rados.rados.0.smithi195.stdout:1151: read oid 6 snap -1 2024-09-13T10:11:22.233 INFO:tasks.rados.rados.0.smithi195.stdout:1151: expect (ObjNum 168 snap 0 seq_num 168) 2024-09-13T10:11:22.233 INFO:tasks.rados.rados.0.smithi195.stdout:1152: read oid 34 snap -1 2024-09-13T10:11:22.233 INFO:tasks.rados.rados.0.smithi195.stdout:1152: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:22.233 INFO:tasks.rados.rados.0.smithi195.stdout:1153: tier_flush oid 17 2024-09-13T10:11:22.233 INFO:tasks.rados.rados.0.smithi195.stdout:1153: tier_flush oid 17 snap -1 2024-09-13T10:11:22.233 INFO:tasks.rados.rados.0.smithi195.stdout:1154: read oid 39 snap -1 2024-09-13T10:11:22.233 INFO:tasks.rados.rados.0.smithi195.stdout:1154: expect (ObjNum 174 snap 0 seq_num 174) 2024-09-13T10:11:22.233 INFO:tasks.rados.rados.0.smithi195.stdout:1155: delete oid 33 current snap is 0 2024-09-13T10:11:22.235 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-13T10:11:22.236 INFO:tasks.rados.rados.0.smithi195.stdout:1151: expect (ObjNum 168 snap 0 seq_num 168) 2024-09-13T10:11:22.252 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:22.251+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:22.353 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:22.352+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:22.400 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:22.400 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:11:22.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:11:22.400 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:11:22.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.400 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:22.400 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:11:22.400 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 52.015277767000001, 2024-09-13T10:11:22.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034624456999999997, 2024-09-13T10:11:22.401 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.401 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.401 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.401 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.401 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:22.401 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:22.401 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 70 2024-09-13T10:11:22.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.401 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.401 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.401 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:11:22.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:11:22.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381723+0000", 2024-09-13T10:11:22.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.540000000000001e-07 2024-09-13T10:11:22.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381731+0000", 2024-09-13T10:11:22.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.486e-06 2024-09-13T10:11:22.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381742+0000", 2024-09-13T10:11:22.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1269e-05 2024-09-13T10:11:22.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381750+0000", 2024-09-13T10:11:22.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8830000000000006e-06 2024-09-13T10:11:22.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401255+0000", 2024-09-13T10:11:22.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019505110999999999 2024-09-13T10:11:22.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401388+0000", 2024-09-13T10:11:22.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013253200000000001 2024-09-13T10:11:22.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:22.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401609+0000", 2024-09-13T10:11:22.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000220981 2024-09-13T10:11:22.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407538+0000", 2024-09-13T10:11:22.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059291220000000002 2024-09-13T10:11:22.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407626+0000", 2024-09-13T10:11:22.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8361000000000003e-05 2024-09-13T10:11:22.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:22.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416154+0000", 2024-09-13T10:11:22.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0085281000000000003 2024-09-13T10:11:22.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416234+0000", 2024-09-13T10:11:22.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9594999999999999e-05 2024-09-13T10:11:22.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416347+0000", 2024-09-13T10:11:22.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113063 2024-09-13T10:11:22.406 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.407 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.407 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.407 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:71 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:22.407 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:11:22.407 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 52.015237184999997, 2024-09-13T10:11:22.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035004, 2024-09-13T10:11:22.407 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.407 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.407 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.407 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.407 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:22.407 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:22.407 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 71 2024-09-13T10:11:22.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.408 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:11:22.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:11:22.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381764+0000", 2024-09-13T10:11:22.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4600000000000004e-07 2024-09-13T10:11:22.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381769+0000", 2024-09-13T10:11:22.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.48e-06 2024-09-13T10:11:22.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381780+0000", 2024-09-13T10:11:22.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0893000000000001e-05 2024-09-13T10:11:22.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381788+0000", 2024-09-13T10:11:22.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6389999999999992e-06 2024-09-13T10:11:22.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407896+0000", 2024-09-13T10:11:22.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.026107807 2024-09-13T10:11:22.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.408029+0000", 2024-09-13T10:11:22.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013383899999999999 2024-09-13T10:11:22.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:22.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.408250+0000", 2024-09-13T10:11:22.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022105399999999999 2024-09-13T10:11:22.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416130+0000", 2024-09-13T10:11:22.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0078798500000000007 2024-09-13T10:11:22.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416488+0000", 2024-09-13T10:11:22.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00035819299999999998 2024-09-13T10:11:22.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:22.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416595+0000", 2024-09-13T10:11:22.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001064 2024-09-13T10:11:22.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416662+0000", 2024-09-13T10:11:22.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6775999999999999e-05 2024-09-13T10:11:22.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416767+0000", 2024-09-13T10:11:22.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000105423 2024-09-13T10:11:22.413 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.413 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.413 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.413 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:72 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:22.413 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:11:22.413 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 52.015200151000002, 2024-09-13T10:11:22.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039399762999999997, 2024-09-13T10:11:22.413 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.413 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.413 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.413 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.413 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:22.414 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:22.414 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 72 2024-09-13T10:11:22.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.414 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:11:22.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:11:22.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381801+0000", 2024-09-13T10:11:22.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7700000000000004e-07 2024-09-13T10:11:22.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381898+0000", 2024-09-13T10:11:22.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7066999999999997e-05 2024-09-13T10:11:22.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381908+0000", 2024-09-13T10:11:22.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0336000000000001e-05 2024-09-13T10:11:22.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381917+0000", 2024-09-13T10:11:22.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5169999999999999e-06 2024-09-13T10:11:22.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.411900+0000", 2024-09-13T10:11:22.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029983381999999999 2024-09-13T10:11:22.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.412028+0000", 2024-09-13T10:11:22.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127916 2024-09-13T10:11:22.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:22.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.412248+0000", 2024-09-13T10:11:22.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022014699999999999 2024-09-13T10:11:22.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416576+0000", 2024-09-13T10:11:22.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004327549 2024-09-13T10:11:22.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416860+0000", 2024-09-13T10:11:22.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028391000000000001 2024-09-13T10:11:22.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:22.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421014+0000", 2024-09-13T10:11:22.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041539330000000003 2024-09-13T10:11:22.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421093+0000", 2024-09-13T10:11:22.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9097999999999994e-05 2024-09-13T10:11:22.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421200+0000", 2024-09-13T10:11:22.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000107231 2024-09-13T10:11:22.419 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.419 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.419 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.419 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:73 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:22.419 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:11:22.419 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 52.015066439000002, 2024-09-13T10:11:22.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044034528000000003, 2024-09-13T10:11:22.419 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.419 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.419 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.420 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.420 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:22.420 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:22.420 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 73 2024-09-13T10:11:22.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.420 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:11:22.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:11:22.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381935+0000", 2024-09-13T10:11:22.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0999999999999997e-07 2024-09-13T10:11:22.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381940+0000", 2024-09-13T10:11:22.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7030000000000003e-06 2024-09-13T10:11:22.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381949+0000", 2024-09-13T10:11:22.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5359999999999992e-06 2024-09-13T10:11:22.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381961+0000", 2024-09-13T10:11:22.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2539e-05 2024-09-13T10:11:22.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416942+0000", 2024-09-13T10:11:22.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034980270000000001 2024-09-13T10:11:22.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.417074+0000", 2024-09-13T10:11:22.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013246199999999999 2024-09-13T10:11:22.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:22.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.417298+0000", 2024-09-13T10:11:22.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000223823 2024-09-13T10:11:22.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425459+0000", 2024-09-13T10:11:22.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081613750000000002 2024-09-13T10:11:22.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425575+0000", 2024-09-13T10:11:22.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115513 2024-09-13T10:11:22.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:22.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425774+0000", 2024-09-13T10:11:22.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019959399999999999 2024-09-13T10:11:22.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425854+0000", 2024-09-13T10:11:22.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9532999999999999e-05 2024-09-13T10:11:22.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425968+0000", 2024-09-13T10:11:22.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011437 2024-09-13T10:11:22.425 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.425 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.425 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.426 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:74 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:22.426 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:11:22.426 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 52.015023565, 2024-09-13T10:11:22.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052701496, 2024-09-13T10:11:22.426 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.426 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.426 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.426 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.426 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:22.426 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:22.426 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 74 2024-09-13T10:11:22.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.426 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:11:22.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:11:22.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:11:22.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7999999999999995e-07 2024-09-13T10:11:22.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381983+0000", 2024-09-13T10:11:22.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4430000000000002e-06 2024-09-13T10:11:22.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381991+0000", 2024-09-13T10:11:22.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8960000000000003e-06 2024-09-13T10:11:22.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381998+0000", 2024-09-13T10:11:22.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7729999999999999e-06 2024-09-13T10:11:22.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421291+0000", 2024-09-13T10:11:22.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039292753 2024-09-13T10:11:22.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421424+0000", 2024-09-13T10:11:22.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013291900000000001 2024-09-13T10:11:22.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:22.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421646+0000", 2024-09-13T10:11:22.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000221843 2024-09-13T10:11:22.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425667+0000", 2024-09-13T10:11:22.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040206779999999998 2024-09-13T10:11:22.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425755+0000", 2024-09-13T10:11:22.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8848e-05 2024-09-13T10:11:22.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:22.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434472+0000", 2024-09-13T10:11:22.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0087164879999999997 2024-09-13T10:11:22.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434544+0000", 2024-09-13T10:11:22.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2273000000000004e-05 2024-09-13T10:11:22.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434678+0000", 2024-09-13T10:11:22.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013390200000000001 2024-09-13T10:11:22.432 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.432 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.432 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.432 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:75 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:22.432 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:11:22.432 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 52.014986700000001, 2024-09-13T10:11:22.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052864585999999998, 2024-09-13T10:11:22.433 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.433 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.433 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.433 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.433 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:22.433 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:22.433 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 75 2024-09-13T10:11:22.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.433 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:11:22.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:11:22.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:11:22.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8299999999999996e-07 2024-09-13T10:11:22.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382120+0000", 2024-09-13T10:11:22.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010627400000000001 2024-09-13T10:11:22.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382131+0000", 2024-09-13T10:11:22.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0796e-05 2024-09-13T10:11:22.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382140+0000", 2024-09-13T10:11:22.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5560000000000008e-06 2024-09-13T10:11:22.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426014+0000", 2024-09-13T10:11:22.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.043873843000000003 2024-09-13T10:11:22.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426205+0000", 2024-09-13T10:11:22.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019154600000000001 2024-09-13T10:11:22.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:22.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426429+0000", 2024-09-13T10:11:22.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022344699999999999 2024-09-13T10:11:22.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434352+0000", 2024-09-13T10:11:22.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007923355 2024-09-13T10:11:22.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434452+0000", 2024-09-13T10:11:22.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9961000000000001e-05 2024-09-13T10:11:22.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:22.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434700+0000", 2024-09-13T10:11:22.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024790899999999999 2024-09-13T10:11:22.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434774+0000", 2024-09-13T10:11:22.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4457999999999998e-05 2024-09-13T10:11:22.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434878+0000", 2024-09-13T10:11:22.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010375800000000001 2024-09-13T10:11:22.439 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.439 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.439 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.439 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:76 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:22.439 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:11:22.439 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 52.014819719999998, 2024-09-13T10:11:22.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053104138000000002, 2024-09-13T10:11:22.439 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.440 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.440 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.440 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.440 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:22.440 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:22.440 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 76 2024-09-13T10:11:22.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.440 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:11:22.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:11:22.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:11:22.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4600000000000003e-07 2024-09-13T10:11:22.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382188+0000", 2024-09-13T10:11:22.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4869999999999999e-06 2024-09-13T10:11:22.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382197+0000", 2024-09-13T10:11:22.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5519999999999993e-06 2024-09-13T10:11:22.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382207+0000", 2024-09-13T10:11:22.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7240000000000004e-06 2024-09-13T10:11:22.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430285+0000", 2024-09-13T10:11:22.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.048078093000000002 2024-09-13T10:11:22.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430429+0000", 2024-09-13T10:11:22.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000143981 2024-09-13T10:11:22.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:22.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430717+0000", 2024-09-13T10:11:22.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028802 2024-09-13T10:11:22.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435015+0000", 2024-09-13T10:11:22.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042973150000000003 2024-09-13T10:11:22.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435083+0000", 2024-09-13T10:11:22.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8448999999999995e-05 2024-09-13T10:11:22.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:22.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435109+0000", 2024-09-13T10:11:22.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6064999999999999e-05 2024-09-13T10:11:22.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435176+0000", 2024-09-13T10:11:22.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7362999999999999e-05 2024-09-13T10:11:22.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435285+0000", 2024-09-13T10:11:22.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108243 2024-09-13T10:11:22.445 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.445 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.445 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.446 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:96 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:22.446 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:11:22.446 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.873188290000002, 2024-09-13T10:11:22.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032701419000000002, 2024-09-13T10:11:22.446 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.446 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.446 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.446 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.446 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:22.446 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:22.446 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 96 2024-09-13T10:11:22.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.447 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:11:22.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:11:22.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523813+0000", 2024-09-13T10:11:22.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.364e-06 2024-09-13T10:11:22.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523913+0000", 2024-09-13T10:11:22.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9827000000000005e-05 2024-09-13T10:11:22.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523927+0000", 2024-09-13T10:11:22.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4029000000000001e-05 2024-09-13T10:11:22.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523936+0000", 2024-09-13T10:11:22.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1030000000000008e-06 2024-09-13T10:11:22.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.544707+0000", 2024-09-13T10:11:22.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0207706 2024-09-13T10:11:22.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.544834+0000", 2024-09-13T10:11:22.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012683200000000001 2024-09-13T10:11:22.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:22.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.545057+0000", 2024-09-13T10:11:22.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022338100000000001 2024-09-13T10:11:22.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549279+0000", 2024-09-13T10:11:22.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042223080000000001 2024-09-13T10:11:22.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549556+0000", 2024-09-13T10:11:22.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000276307 2024-09-13T10:11:22.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:22.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556364+0000", 2024-09-13T10:11:22.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0068082510000000004 2024-09-13T10:11:22.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556445+0000", 2024-09-13T10:11:22.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1081000000000005e-05 2024-09-13T10:11:22.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556513+0000", 2024-09-13T10:11:22.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8336000000000004e-05 2024-09-13T10:11:22.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.452 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.452 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:97 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:22.452 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:11:22.452 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.873045404999999, 2024-09-13T10:11:22.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035410807000000002, 2024-09-13T10:11:22.453 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.453 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.453 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.453 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.453 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:22.453 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:22.453 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 97 2024-09-13T10:11:22.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.453 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:11:22.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:11:22.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523956+0000", 2024-09-13T10:11:22.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8599999999999997e-07 2024-09-13T10:11:22.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523962+0000", 2024-09-13T10:11:22.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1240000000000002e-06 2024-09-13T10:11:22.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523971+0000", 2024-09-13T10:11:22.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7320000000000004e-06 2024-09-13T10:11:22.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523978+0000", 2024-09-13T10:11:22.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0899999999999999e-06 2024-09-13T10:11:22.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549637+0000", 2024-09-13T10:11:22.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025658876000000001 2024-09-13T10:11:22.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549757+0000", 2024-09-13T10:11:22.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000120837 2024-09-13T10:11:22.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:22.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549971+0000", 2024-09-13T10:11:22.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021356000000000001 2024-09-13T10:11:22.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556335+0000", 2024-09-13T10:11:22.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0063639329999999996 2024-09-13T10:11:22.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556725+0000", 2024-09-13T10:11:22.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000390221 2024-09-13T10:11:22.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:22.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559219+0000", 2024-09-13T10:11:22.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0024943500000000002 2024-09-13T10:11:22.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559296+0000", 2024-09-13T10:11:22.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7065999999999997e-05 2024-09-13T10:11:22.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559366+0000", 2024-09-13T10:11:22.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9132000000000002e-05 2024-09-13T10:11:22.459 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.459 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.459 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.459 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:98 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:22.459 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:11:22.459 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.873010796999999, 2024-09-13T10:11:22.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035839604999999997, 2024-09-13T10:11:22.459 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.459 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.459 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.459 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.459 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:22.460 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:22.460 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 98 2024-09-13T10:11:22.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.460 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.460 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.460 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:11:22.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.460 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.460 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:11:22.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523990+0000", 2024-09-13T10:11:22.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6300000000000005e-07 2024-09-13T10:11:22.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523995+0000", 2024-09-13T10:11:22.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2669999999999997e-06 2024-09-13T10:11:22.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524007+0000", 2024-09-13T10:11:22.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2002e-05 2024-09-13T10:11:22.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524015+0000", 2024-09-13T10:11:22.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4869999999999996e-06 2024-09-13T10:11:22.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.553854+0000", 2024-09-13T10:11:22.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029839633000000001 2024-09-13T10:11:22.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.553985+0000", 2024-09-13T10:11:22.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013032899999999999 2024-09-13T10:11:22.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:22.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.554212+0000", 2024-09-13T10:11:22.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022723400000000001 2024-09-13T10:11:22.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559523+0000", 2024-09-13T10:11:22.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053113509999999997 2024-09-13T10:11:22.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559662+0000", 2024-09-13T10:11:22.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013833500000000001 2024-09-13T10:11:22.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:22.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559698+0000", 2024-09-13T10:11:22.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5982000000000002e-05 2024-09-13T10:11:22.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559764+0000", 2024-09-13T10:11:22.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6019000000000006e-05 2024-09-13T10:11:22.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559829+0000", 2024-09-13T10:11:22.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5302999999999995e-05 2024-09-13T10:11:22.465 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.466 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.466 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.466 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:99 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:22.466 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:11:22.466 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.872973103, 2024-09-13T10:11:22.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.037637084000000001, 2024-09-13T10:11:22.466 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.466 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.467 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.467 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.467 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:22.467 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:22.467 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 99 2024-09-13T10:11:22.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.467 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:11:22.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:11:22.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524028+0000", 2024-09-13T10:11:22.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7999999999999995e-07 2024-09-13T10:11:22.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524109+0000", 2024-09-13T10:11:22.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0834999999999994e-05 2024-09-13T10:11:22.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524119+0000", 2024-09-13T10:11:22.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0149999999999999e-05 2024-09-13T10:11:22.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524127+0000", 2024-09-13T10:11:22.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8660000000000006e-06 2024-09-13T10:11:22.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556868+0000", 2024-09-13T10:11:22.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.03274146 2024-09-13T10:11:22.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.557001+0000", 2024-09-13T10:11:22.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000133249 2024-09-13T10:11:22.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:22.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.557224+0000", 2024-09-13T10:11:22.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000222914 2024-09-13T10:11:22.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561387+0000", 2024-09-13T10:11:22.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041627019999999999 2024-09-13T10:11:22.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:22.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561419+0000", 2024-09-13T10:11:22.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1924999999999997e-05 2024-09-13T10:11:22.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561476+0000", 2024-09-13T10:11:22.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7346999999999998e-05 2024-09-13T10:11:22.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561552+0000", 2024-09-13T10:11:22.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5847999999999996e-05 2024-09-13T10:11:22.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561664+0000", 2024-09-13T10:11:22.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112108 2024-09-13T10:11:22.473 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.473 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.474 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.474 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:100 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:22.474 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:11:22.474 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.872796952999998, 2024-09-13T10:11:22.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.038825510000000001, 2024-09-13T10:11:22.474 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.474 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.474 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.474 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.474 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:22.474 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:22.475 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 100 2024-09-13T10:11:22.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.475 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:11:22.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:11:22.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524204+0000", 2024-09-13T10:11:22.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.54e-07 2024-09-13T10:11:22.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524211+0000", 2024-09-13T10:11:22.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6919999999999996e-06 2024-09-13T10:11:22.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524221+0000", 2024-09-13T10:11:22.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0223e-05 2024-09-13T10:11:22.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524229+0000", 2024-09-13T10:11:22.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9859999999999994e-06 2024-09-13T10:11:22.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559920+0000", 2024-09-13T10:11:22.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035690854000000001 2024-09-13T10:11:22.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.560052+0000", 2024-09-13T10:11:22.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013199899999999999 2024-09-13T10:11:22.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:22.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.560356+0000", 2024-09-13T10:11:22.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030450200000000001 2024-09-13T10:11:22.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:22.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562115+0000", 2024-09-13T10:11:22.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0017586430000000001 2024-09-13T10:11:22.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562512+0000", 2024-09-13T10:11:22.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00039728099999999998 2024-09-13T10:11:22.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562828+0000", 2024-09-13T10:11:22.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00031528799999999999 2024-09-13T10:11:22.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562913+0000", 2024-09-13T10:11:22.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5628000000000001e-05 2024-09-13T10:11:22.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.563029+0000", 2024-09-13T10:11:22.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011556 2024-09-13T10:11:22.481 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.481 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.481 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.481 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:122 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:22.481 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:11:22.481 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.756112471999998, 2024-09-13T10:11:22.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.027900708, 2024-09-13T10:11:22.482 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.482 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.482 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.482 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.482 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:22.482 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:22.482 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 122 2024-09-13T10:11:22.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.482 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:11:22.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:11:22.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640889+0000", 2024-09-13T10:11:22.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4139999999999999e-06 2024-09-13T10:11:22.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640900+0000", 2024-09-13T10:11:22.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0559e-05 2024-09-13T10:11:22.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640916+0000", 2024-09-13T10:11:22.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6399e-05 2024-09-13T10:11:22.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640931+0000", 2024-09-13T10:11:22.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4659e-05 2024-09-13T10:11:22.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663602+0000", 2024-09-13T10:11:22.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.022671674999999999 2024-09-13T10:11:22.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663734+0000", 2024-09-13T10:11:22.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013189899999999999 2024-09-13T10:11:22.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:22.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663946+0000", 2024-09-13T10:11:22.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021198699999999999 2024-09-13T10:11:22.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668523+0000", 2024-09-13T10:11:22.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045765789999999999 2024-09-13T10:11:22.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668638+0000", 2024-09-13T10:11:22.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504 2024-09-13T10:11:22.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:22.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668657+0000", 2024-09-13T10:11:22.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9298999999999999e-05 2024-09-13T10:11:22.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668722+0000", 2024-09-13T10:11:22.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5126e-05 2024-09-13T10:11:22.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668788+0000", 2024-09-13T10:11:22.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6072e-05 2024-09-13T10:11:22.487 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.487 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.487 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.488 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:123 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:22.488 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:11:22.488 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.756046595000001, 2024-09-13T10:11:22.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028273795000000001, 2024-09-13T10:11:22.488 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.488 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.488 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.488 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.488 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:22.488 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:22.488 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 123 2024-09-13T10:11:22.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.488 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:11:22.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:11:22.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640955+0000", 2024-09-13T10:11:22.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.356e-06 2024-09-13T10:11:22.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641083+0000", 2024-09-13T10:11:22.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012763600000000001 2024-09-13T10:11:22.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641102+0000", 2024-09-13T10:11:22.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9055e-05 2024-09-13T10:11:22.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641117+0000", 2024-09-13T10:11:22.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5665999999999999e-05 2024-09-13T10:11:22.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.665747+0000", 2024-09-13T10:11:22.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.024629796999999998 2024-09-13T10:11:22.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.665876+0000", 2024-09-13T10:11:22.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012925600000000001 2024-09-13T10:11:22.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:22.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.666105+0000", 2024-09-13T10:11:22.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000228543 2024-09-13T10:11:22.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668924+0000", 2024-09-13T10:11:22.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028194309999999998 2024-09-13T10:11:22.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669070+0000", 2024-09-13T10:11:22.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014560199999999999 2024-09-13T10:11:22.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:22.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669104+0000", 2024-09-13T10:11:22.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3654e-05 2024-09-13T10:11:22.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669173+0000", 2024-09-13T10:11:22.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9819000000000004e-05 2024-09-13T10:11:22.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669227+0000", 2024-09-13T10:11:22.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3980000000000002e-05 2024-09-13T10:11:22.494 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.494 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.494 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.494 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:124 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:22.494 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:11:22.494 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.755853553999998, 2024-09-13T10:11:22.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030852801999999999, 2024-09-13T10:11:22.494 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.494 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.494 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.494 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.494 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:22.494 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:22.495 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 124 2024-09-13T10:11:22.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.495 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:11:22.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:11:22.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641148+0000", 2024-09-13T10:11:22.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4529999999999999e-06 2024-09-13T10:11:22.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641158+0000", 2024-09-13T10:11:22.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0003999999999999e-05 2024-09-13T10:11:22.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641199+0000", 2024-09-13T10:11:22.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0426999999999998e-05 2024-09-13T10:11:22.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641214+0000", 2024-09-13T10:11:22.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5716e-05 2024-09-13T10:11:22.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669267+0000", 2024-09-13T10:11:22.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028053112000000002 2024-09-13T10:11:22.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669394+0000", 2024-09-13T10:11:22.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012656000000000001 2024-09-13T10:11:22.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:22.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669614+0000", 2024-09-13T10:11:22.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021970400000000001 2024-09-13T10:11:22.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671267+0000", 2024-09-13T10:11:22.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001652976 2024-09-13T10:11:22.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671383+0000", 2024-09-13T10:11:22.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000116113 2024-09-13T10:11:22.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:22.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671828+0000", 2024-09-13T10:11:22.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000444871 2024-09-13T10:11:22.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671935+0000", 2024-09-13T10:11:22.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010703 2024-09-13T10:11:22.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671999+0000", 2024-09-13T10:11:22.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4836000000000001e-05 2024-09-13T10:11:22.500 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.500 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.500 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.500 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:201 3.3 e38/30 3:d4fb61be:::smithi19530714-2 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:22.500 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:11:22.500 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.383402661000002, 2024-09-13T10:11:22.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032127035999999998, 2024-09-13T10:11:22.501 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.501 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.501 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.501 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:11:22.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:11:22.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:11:22.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2600000000000001e-07 2024-09-13T10:11:22.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030449+0000", 2024-09-13T10:11:22.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016850477999999999 2024-09-13T10:11:22.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030499+0000", 2024-09-13T10:11:22.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0212e-05 2024-09-13T10:11:22.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030524+0000", 2024-09-13T10:11:22.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5222e-05 2024-09-13T10:11:22.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030739+0000", 2024-09-13T10:11:22.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000214396 2024-09-13T10:11:22.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030818+0000", 2024-09-13T10:11:22.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8941000000000002e-05 2024-09-13T10:11:22.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.045512+0000", 2024-09-13T10:11:22.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014694248 2024-09-13T10:11:22.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.045725+0000", 2024-09-13T10:11:22.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021261300000000001 2024-09-13T10:11:22.504 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.504 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.505 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.505 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:226 3.3 e38/30 3:fe5bfa3a:::smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'5, mlcod=38'3)", 2024-09-13T10:11:22.505 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:11:22.505 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.316542418000001, 2024-09-13T10:11:22.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030943633000000002, 2024-09-13T10:11:22.505 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.505 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.505 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.505 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:11:22.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:11:22.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080461+0000", 2024-09-13T10:11:22.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7049999999999999e-06 2024-09-13T10:11:22.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097571+0000", 2024-09-13T10:11:22.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017110658000000001 2024-09-13T10:11:22.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097615+0000", 2024-09-13T10:11:22.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3404e-05 2024-09-13T10:11:22.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097635+0000", 2024-09-13T10:11:22.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0279e-05 2024-09-13T10:11:22.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097995+0000", 2024-09-13T10:11:22.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00036038699999999999 2024-09-13T10:11:22.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.098029+0000", 2024-09-13T10:11:22.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4162000000000003e-05 2024-09-13T10:11:22.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111330+0000", 2024-09-13T10:11:22.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013300268000000001 2024-09-13T10:11:22.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111401+0000", 2024-09-13T10:11:22.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1769999999999999e-05 2024-09-13T10:11:22.509 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.509 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.509 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.509 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:237 3.3 e38/30 3:de40747e:::smithi19530714-38 low_tier:head v 38'7, mlcod=38'3)", 2024-09-13T10:11:22.509 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:11:22.509 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.282309626, 2024-09-13T10:11:22.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.037747504000000001, 2024-09-13T10:11:22.510 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.510 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.510 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.510 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:11:22.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:11:22.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:11:22.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7299999999999995e-07 2024-09-13T10:11:22.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145311+0000", 2024-09-13T10:11:22.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030619621 2024-09-13T10:11:22.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145336+0000", 2024-09-13T10:11:22.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5009999999999999e-05 2024-09-13T10:11:22.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145349+0000", 2024-09-13T10:11:22.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2724999999999999e-05 2024-09-13T10:11:22.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145445+0000", 2024-09-13T10:11:22.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6108999999999997e-05 2024-09-13T10:11:22.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145470+0000", 2024-09-13T10:11:22.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5514e-05 2024-09-13T10:11:22.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152389+0000", 2024-09-13T10:11:22.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069190739999999999 2024-09-13T10:11:22.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152438+0000", 2024-09-13T10:11:22.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8778e-05 2024-09-13T10:11:22.513 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.514 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.514 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.514 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:286 4.a 4:5ad8883b:::smithi19530714-37:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:22.514 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:11:22.514 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.148712999000001, 2024-09-13T10:11:22.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032980435000000002, 2024-09-13T10:11:22.514 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.514 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.514 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:22.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:22.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:22.515 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 286 2024-09-13T10:11:22.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.515 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:11:22.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:11:22.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248290+0000", 2024-09-13T10:11:22.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0170000000000001e-06 2024-09-13T10:11:22.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248302+0000", 2024-09-13T10:11:22.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1602e-05 2024-09-13T10:11:22.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248328+0000", 2024-09-13T10:11:22.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.622e-05 2024-09-13T10:11:22.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248344+0000", 2024-09-13T10:11:22.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6192999999999999e-05 2024-09-13T10:11:22.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248466+0000", 2024-09-13T10:11:22.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012186899999999999 2024-09-13T10:11:22.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248602+0000", 2024-09-13T10:11:22.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013623500000000001 2024-09-13T10:11:22.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 6,13", 2024-09-13T10:11:22.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274091+0000", 2024-09-13T10:11:22.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025489050999999999 2024-09-13T10:11:22.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275976+0000", 2024-09-13T10:11:22.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018847289999999999 2024-09-13T10:11:22.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:22.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276801+0000", 2024-09-13T10:11:22.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00082477899999999999 2024-09-13T10:11:22.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:22.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281064+0000", 2024-09-13T10:11:22.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042625950000000001 2024-09-13T10:11:22.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:22.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281173+0000", 2024-09-13T10:11:22.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000109391 2024-09-13T10:11:22.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281268+0000", 2024-09-13T10:11:22.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4753999999999994e-05 2024-09-13T10:11:22.520 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.520 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.520 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.520 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1631 4.a 4:5ad8883b:::smithi19530714-37:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:22.520 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:06.099753+0000", 2024-09-13T10:11:22.520 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 16.297247425999998, 2024-09-13T10:11:22.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033125612999999998, 2024-09-13T10:11:22.520 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:22.521 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:22.521 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:11:22.521 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:22.521 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:22.521 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:22.521 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1631 2024-09-13T10:11:22.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.521 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:22.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:22.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099753+0000", 2024-09-13T10:11:22.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:22.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099753+0000", 2024-09-13T10:11:22.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:22.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:22.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099754+0000", 2024-09-13T10:11:22.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2699999999999999e-06 2024-09-13T10:11:22.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:22.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099758+0000", 2024-09-13T10:11:22.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6540000000000001e-06 2024-09-13T10:11:22.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:22.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099768+0000", 2024-09-13T10:11:22.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0412e-05 2024-09-13T10:11:22.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:22.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099777+0000", 2024-09-13T10:11:22.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4600000000000003e-06 2024-09-13T10:11:22.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:22.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099830+0000", 2024-09-13T10:11:22.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3192000000000002e-05 2024-09-13T10:11:22.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:22.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099889+0000", 2024-09-13T10:11:22.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9576e-05 2024-09-13T10:11:22.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:22.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:22.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:22.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.132878+0000", 2024-09-13T10:11:22.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032989048999999999 2024-09-13T10:11:22.524 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.524 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.524 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.524 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:22.525 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:22.525 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:22.526 INFO:tasks.rados.rados.0.smithi195.stdout:1149: finishing set_chunk to oid 5 2024-09-13T10:11:22.526 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 8 (ObjNum 173 snap 0 seq_num 173) dirty exists 2024-09-13T10:11:22.526 INFO:tasks.rados.rados.0.smithi195.stdout:1154: expect (ObjNum 174 snap 0 seq_num 174) 2024-09-13T10:11:22.527 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-13T10:11:22.527 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:22.522+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:22.561 INFO:tasks.rados.rados.0.smithi195.stdout:1152: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:22.583 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1917514 2024-09-13T10:11:22.593 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 152136 2024-09-13T10:11:22.627 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:22.624+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:22.670 INFO:tasks.rados.rados.0.smithi195.stdout:1150: expect (ObjNum 197 snap 0 seq_num 197) 2024-09-13T10:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:22.717 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-13T10:11:22.718 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1476024 2024-09-13T10:11:22.766 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:22.764+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:22.791 INFO:tasks.rados.rados.0.smithi195.stdout:1153: got (0) Success 2024-09-13T10:11:22.791 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 0 (ObjNum 153 snap 0 seq_num 153) dirty exists 2024-09-13T10:11:22.791 INFO:tasks.rados.rados.0.smithi195.stdout:1149: done (6 left) 2024-09-13T10:11:22.791 INFO:tasks.rados.rados.0.smithi195.stdout:1150: done (5 left) 2024-09-13T10:11:22.791 INFO:tasks.rados.rados.0.smithi195.stdout:1151: done (4 left) 2024-09-13T10:11:22.791 INFO:tasks.rados.rados.0.smithi195.stdout:1152: done (3 left) 2024-09-13T10:11:22.792 INFO:tasks.rados.rados.0.smithi195.stdout:1153: done (2 left) 2024-09-13T10:11:22.792 INFO:tasks.rados.rados.0.smithi195.stdout:1154: done (1 left) 2024-09-13T10:11:22.792 INFO:tasks.rados.rados.0.smithi195.stdout:1155: done (0 left) 2024-09-13T10:11:22.792 INFO:tasks.rados.rados.0.smithi195.stdout:1156: delete oid 20 current snap is 0 2024-09-13T10:11:22.795 INFO:tasks.rados.rados.0.smithi195.stdout:1156: done (0 left) 2024-09-13T10:11:22.795 INFO:tasks.rados.rados.0.smithi195.stdout:1157: read oid 4 snap -1 2024-09-13T10:11:22.796 INFO:tasks.rados.rados.0.smithi195.stdout:1157: expect (ObjNum 191 snap 0 seq_num 191) 2024-09-13T10:11:22.796 INFO:tasks.rados.rados.0.smithi195.stdout:1158: write (excl) oid 27 current snap is 0 2024-09-13T10:11:22.796 INFO:tasks.rados.rados.0.smithi195.stdout:1158: seq_num 213 ranges {779891=485921,1947841=685395,3383215=196676} 2024-09-13T10:11:22.802 INFO:tasks.rados.rados.0.smithi195.stdout:1158: writing smithi19530714-27 from 779891 to 1265812 tid 1 2024-09-13T10:11:22.812 INFO:tasks.rados.rados.0.smithi195.stdout:1158: writing smithi19530714-27 from 1947841 to 2633236 tid 2 2024-09-13T10:11:22.815 INFO:tasks.rados.rados.0.smithi195.stdout:1158: writing smithi19530714-27 from 3383215 to 3579891 tid 3 2024-09-13T10:11:22.815 INFO:tasks.rados.rados.0.smithi195.stdout:1159: read oid 48 snap -1 2024-09-13T10:11:22.815 INFO:tasks.rados.rados.0.smithi195.stdout:1159: expect (ObjNum 185 snap 0 seq_num 185) 2024-09-13T10:11:22.815 INFO:tasks.rados.rados.0.smithi195.stdout:1160: write oid 6 current snap is 0 2024-09-13T10:11:22.815 INFO:tasks.rados.rados.0.smithi195.stdout:1160: seq_num 214 ranges {796698=561170,2006323=545406,3126991=469707} 2024-09-13T10:11:22.822 INFO:tasks.rados.rados.0.smithi195.stdout:1160: writing smithi19530714-6 from 796698 to 1357868 tid 1 2024-09-13T10:11:22.828 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:22.826+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:22.829 INFO:tasks.rados.rados.0.smithi195.stdout:1160: writing smithi19530714-6 from 2006323 to 2551729 tid 2 2024-09-13T10:11:22.835 INFO:tasks.rados.rados.0.smithi195.stdout:1160: writing smithi19530714-6 from 3126991 to 3596698 tid 3 2024-09-13T10:11:22.835 INFO:tasks.rados.rados.0.smithi195.stdout:1161: read oid 14 snap -1 2024-09-13T10:11:22.835 INFO:tasks.rados.rados.0.smithi195.stdout:1161: expect (ObjNum 209 snap 0 seq_num 209) 2024-09-13T10:11:22.835 INFO:tasks.rados.rados.0.smithi195.stdout:1162: tier_flush oid 19 2024-09-13T10:11:22.835 INFO:tasks.rados.rados.0.smithi195.stdout:1162: tier_flush oid 19 snap -1 2024-09-13T10:11:22.835 INFO:tasks.rados.rados.0.smithi195.stdout:1157: expect (ObjNum 191 snap 0 seq_num 191) 2024-09-13T10:11:22.894 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:22.894 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:22.894 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:11:22.894 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:11:22.895 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:22.896 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:22.909 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-13T10:11:22.929 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:22.927+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:22.985 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2635772 2024-09-13T10:11:22.997 INFO:tasks.rados.rados.0.smithi195.stdout:1158: finishing write tid 1 to smithi19530714-27 2024-09-13T10:11:22.997 INFO:tasks.rados.rados.0.smithi195.stdout:1158: finishing write tid 2 to smithi19530714-27 2024-09-13T10:11:22.997 INFO:tasks.rados.rados.0.smithi195.stdout:1158: finishing write tid 3 to smithi19530714-27 2024-09-13T10:11:22.998 INFO:tasks.rados.rados.0.smithi195.stdout:1158: finishing write tid 4 to smithi19530714-27 2024-09-13T10:11:22.998 INFO:tasks.rados.rados.0.smithi195.stdout:1158: finishing write tid 5 to smithi19530714-27 2024-09-13T10:11:22.998 INFO:tasks.rados.rados.0.smithi195.stdout:1158: oid 27 updating version 0 to 103 2024-09-13T10:11:22.998 INFO:tasks.rados.rados.0.smithi195.stdout:1158: oid 27 version 103 is already newer than 100 2024-09-13T10:11:22.998 INFO:tasks.rados.rados.0.smithi195.stdout:1158: oid 27 version 103 is already newer than 102 2024-09-13T10:11:22.998 INFO:tasks.rados.rados.0.smithi195.stdout:1158: oid 27 version 103 is already newer than 101 2024-09-13T10:11:22.999 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 103 (ObjNum 213 snap 0 seq_num 213) dirty exists 2024-09-13T10:11:22.999 INFO:tasks.rados.rados.0.smithi195.stdout:1158: left oid 27 (ObjNum 213 snap 0 seq_num 213) 2024-09-13T10:11:22.999 INFO:tasks.rados.rados.0.smithi195.stdout:1159: expect (ObjNum 185 snap 0 seq_num 185) 2024-09-13T10:11:23.030 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:23.024+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:23.033 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 584358 2024-09-13T10:11:23.034 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:23.034 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:11:23.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:11:23.034 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:11:23.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.034 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:53 4.c 4:3e6706e5:::smithi19530714-14:head [write 618491~694590,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.035 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:50.494212+0000", 2024-09-13T10:11:23.035 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.537287323999998, 2024-09-13T10:11:23.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0086614210000000007, 2024-09-13T10:11:23.035 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.035 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.035 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.035 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.035 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.035 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.035 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53 2024-09-13T10:11:23.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.035 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494212+0000", 2024-09-13T10:11:23.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494212+0000", 2024-09-13T10:11:23.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494213+0000", 2024-09-13T10:11:23.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.84e-06 2024-09-13T10:11:23.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494889+0000", 2024-09-13T10:11:23.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00067531799999999995 2024-09-13T10:11:23.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494907+0000", 2024-09-13T10:11:23.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7884999999999999e-05 2024-09-13T10:11:23.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494917+0000", 2024-09-13T10:11:23.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0043e-05 2024-09-13T10:11:23.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494971+0000", 2024-09-13T10:11:23.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4775000000000003e-05 2024-09-13T10:11:23.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.495081+0000", 2024-09-13T10:11:23.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110055 2024-09-13T10:11:23.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:23.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.495184+0000", 2024-09-13T10:11:23.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010221 2024-09-13T10:11:23.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.496618+0000", 2024-09-13T10:11:23.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014343590000000001 2024-09-13T10:11:23.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.041 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.041 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.498806+0000", 2024-09-13T10:11:23.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021880269999999999 2024-09-13T10:11:23.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.502814+0000", 2024-09-13T10:11:23.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040077890000000003 2024-09-13T10:11:23.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.502844+0000", 2024-09-13T10:11:23.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0462000000000001e-05 2024-09-13T10:11:23.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.502873+0000", 2024-09-13T10:11:23.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8657999999999999e-05 2024-09-13T10:11:23.043 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.043 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.043 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.044 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:161 4.d 4:b19b932a:::smithi19530714-41:head [write 672280~791725,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.045 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:50.852563+0000", 2024-09-13T10:11:23.045 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.178935357, 2024-09-13T10:11:23.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011425286999999999, 2024-09-13T10:11:23.045 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.045 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.045 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.046 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.046 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.046 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.046 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 161 2024-09-13T10:11:23.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.046 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.852563+0000", 2024-09-13T10:11:23.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.852563+0000", 2024-09-13T10:11:23.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.852567+0000", 2024-09-13T10:11:23.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1269999999999999e-06 2024-09-13T10:11:23.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853363+0000", 2024-09-13T10:11:23.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00079620899999999998 2024-09-13T10:11:23.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853394+0000", 2024-09-13T10:11:23.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1192000000000002e-05 2024-09-13T10:11:23.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853410+0000", 2024-09-13T10:11:23.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5896000000000002e-05 2024-09-13T10:11:23.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853574+0000", 2024-09-13T10:11:23.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000164358 2024-09-13T10:11:23.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853920+0000", 2024-09-13T10:11:23.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034612500000000002 2024-09-13T10:11:23.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:23.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.854305+0000", 2024-09-13T10:11:23.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038458900000000001 2024-09-13T10:11:23.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.858018+0000", 2024-09-13T10:11:23.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0037128619999999999 2024-09-13T10:11:23.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.858905+0000", 2024-09-13T10:11:23.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00088737899999999999 2024-09-13T10:11:23.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.863681+0000", 2024-09-13T10:11:23.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004775311 2024-09-13T10:11:23.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.863813+0000", 2024-09-13T10:11:23.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013254900000000001 2024-09-13T10:11:23.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.863989+0000", 2024-09-13T10:11:23.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017568999999999999 2024-09-13T10:11:23.052 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.052 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.052 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.052 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(osd.14.23:4 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.052 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:50.996997+0000", 2024-09-13T10:11:23.052 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.034501951000003, 2024-09-13T10:11:23.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0083262170000000003, 2024-09-13T10:11:23.053 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.053 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.053 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:11:23.053 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.996997+0000", 2024-09-13T10:11:23.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.996997+0000", 2024-09-13T10:11:23.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997000+0000", 2024-09-13T10:11:23.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3560000000000001e-06 2024-09-13T10:11:23.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997015+0000", 2024-09-13T10:11:23.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.523e-05 2024-09-13T10:11:23.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997048+0000", 2024-09-13T10:11:23.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2162000000000001e-05 2024-09-13T10:11:23.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997071+0000", 2024-09-13T10:11:23.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2918999999999999e-05 2024-09-13T10:11:23.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.001996+0000", 2024-09-13T10:11:23.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049256559999999996 2024-09-13T10:11:23.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002198+0000", 2024-09-13T10:11:23.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020196600000000001 2024-09-13T10:11:23.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.005323+0000", 2024-09-13T10:11:23.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0031249279999999999 2024-09-13T10:11:23.057 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.057 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.057 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.057 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:261 4.c 4:3d56d753:::smithi19530714-12:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.057 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.215641+0000", 2024-09-13T10:11:23.057 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.815857633, 2024-09-13T10:11:23.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010509116000000001, 2024-09-13T10:11:23.057 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.058 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.058 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.058 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.058 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.058 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.058 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 261 2024-09-13T10:11:23.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.058 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.058 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.058 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215641+0000", 2024-09-13T10:11:23.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215641+0000", 2024-09-13T10:11:23.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215646+0000", 2024-09-13T10:11:23.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9300000000000002e-06 2024-09-13T10:11:23.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215666+0000", 2024-09-13T10:11:23.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0095999999999999e-05 2024-09-13T10:11:23.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215713+0000", 2024-09-13T10:11:23.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7125e-05 2024-09-13T10:11:23.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215739+0000", 2024-09-13T10:11:23.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5488999999999999e-05 2024-09-13T10:11:23.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215918+0000", 2024-09-13T10:11:23.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017948399999999999 2024-09-13T10:11:23.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216165+0000", 2024-09-13T10:11:23.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024625200000000003 2024-09-13T10:11:23.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:23.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.223467+0000", 2024-09-13T10:11:23.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0073023840000000003 2024-09-13T10:11:23.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.225020+0000", 2024-09-13T10:11:23.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015529809999999999 2024-09-13T10:11:23.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.225406+0000", 2024-09-13T10:11:23.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038614699999999999 2024-09-13T10:11:23.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.225903+0000", 2024-09-13T10:11:23.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00049721900000000002 2024-09-13T10:11:23.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.226026+0000", 2024-09-13T10:11:23.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000123163 2024-09-13T10:11:23.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.226150+0000", 2024-09-13T10:11:23.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012384599999999999 2024-09-13T10:11:23.064 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.064 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.064 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.065 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:263 4.c 4:3e6706e5:::smithi19530714-14:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.065 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.215802+0000", 2024-09-13T10:11:23.065 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.815696424999999, 2024-09-13T10:11:23.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015708573999999999, 2024-09-13T10:11:23.065 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.065 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.065 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.065 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.065 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.066 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.066 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 263 2024-09-13T10:11:23.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.066 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215802+0000", 2024-09-13T10:11:23.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215802+0000", 2024-09-13T10:11:23.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215804+0000", 2024-09-13T10:11:23.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.624e-06 2024-09-13T10:11:23.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215813+0000", 2024-09-13T10:11:23.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4159999999999993e-06 2024-09-13T10:11:23.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215832+0000", 2024-09-13T10:11:23.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8768999999999999e-05 2024-09-13T10:11:23.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215843+0000", 2024-09-13T10:11:23.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0336999999999999e-05 2024-09-13T10:11:23.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216612+0000", 2024-09-13T10:11:23.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00076899399999999995 2024-09-13T10:11:23.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216767+0000", 2024-09-13T10:11:23.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015566399999999999 2024-09-13T10:11:23.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:23.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.227138+0000", 2024-09-13T10:11:23.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010370701 2024-09-13T10:11:23.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.229299+0000", 2024-09-13T10:11:23.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021611870000000002 2024-09-13T10:11:23.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.229896+0000", 2024-09-13T10:11:23.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00059659700000000003 2024-09-13T10:11:23.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231283+0000", 2024-09-13T10:11:23.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001387133 2024-09-13T10:11:23.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231392+0000", 2024-09-13T10:11:23.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010878999999999999 2024-09-13T10:11:23.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231511+0000", 2024-09-13T10:11:23.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119362 2024-09-13T10:11:23.072 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.072 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.072 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.072 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:264 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.072 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.215936+0000", 2024-09-13T10:11:23.072 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.815562446000001, 2024-09-13T10:11:23.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054954390999999998, 2024-09-13T10:11:23.072 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.072 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.073 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.073 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 264 2024-09-13T10:11:23.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.073 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215936+0000", 2024-09-13T10:11:23.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215936+0000", 2024-09-13T10:11:23.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215938+0000", 2024-09-13T10:11:23.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9860000000000001e-06 2024-09-13T10:11:23.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215951+0000", 2024-09-13T10:11:23.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2449999999999999e-05 2024-09-13T10:11:23.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215982+0000", 2024-09-13T10:11:23.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1133000000000002e-05 2024-09-13T10:11:23.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216009+0000", 2024-09-13T10:11:23.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6652000000000002e-05 2024-09-13T10:11:23.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.217072+0000", 2024-09-13T10:11:23.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010634240000000001 2024-09-13T10:11:23.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.217268+0000", 2024-09-13T10:11:23.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019641 2024-09-13T10:11:23.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:23.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.265415+0000", 2024-09-13T10:11:23.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.048146807999999999 2024-09-13T10:11:23.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.267067+0000", 2024-09-13T10:11:23.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00165211 2024-09-13T10:11:23.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.267319+0000", 2024-09-13T10:11:23.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025160500000000001 2024-09-13T10:11:23.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270591+0000", 2024-09-13T10:11:23.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0032720959999999999 2024-09-13T10:11:23.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270753+0000", 2024-09-13T10:11:23.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016215199999999999 2024-09-13T10:11:23.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270891+0000", 2024-09-13T10:11:23.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000137565 2024-09-13T10:11:23.078 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.078 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.079 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.079 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:278 4.c 4:39f17ee9:::smithi19530714-29:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.079 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.231988+0000", 2024-09-13T10:11:23.079 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.79951058, 2024-09-13T10:11:23.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011030108, 2024-09-13T10:11:23.079 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.079 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.079 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.079 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.079 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.079 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.079 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 278 2024-09-13T10:11:23.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.080 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231988+0000", 2024-09-13T10:11:23.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231988+0000", 2024-09-13T10:11:23.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231990+0000", 2024-09-13T10:11:23.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0499999999999999e-06 2024-09-13T10:11:23.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232002+0000", 2024-09-13T10:11:23.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2189e-05 2024-09-13T10:11:23.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232029+0000", 2024-09-13T10:11:23.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6673e-05 2024-09-13T10:11:23.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232043+0000", 2024-09-13T10:11:23.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4188999999999999e-05 2024-09-13T10:11:23.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232239+0000", 2024-09-13T10:11:23.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019557 2024-09-13T10:11:23.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232472+0000", 2024-09-13T10:11:23.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00023310900000000001 2024-09-13T10:11:23.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:23.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.237292+0000", 2024-09-13T10:11:23.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048195069999999998 2024-09-13T10:11:23.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.238595+0000", 2024-09-13T10:11:23.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013033260000000001 2024-09-13T10:11:23.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.239056+0000", 2024-09-13T10:11:23.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00046106800000000002 2024-09-13T10:11:23.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.242820+0000", 2024-09-13T10:11:23.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003763897 2024-09-13T10:11:23.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.242905+0000", 2024-09-13T10:11:23.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5580000000000004e-05 2024-09-13T10:11:23.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.243018+0000", 2024-09-13T10:11:23.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011294999999999999 2024-09-13T10:11:23.085 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.086 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.086 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.086 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:290 4.d 4:b19b932a:::smithi19530714-41:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.086 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.255335+0000", 2024-09-13T10:11:23.086 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.776164057999999, 2024-09-13T10:11:23.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021335703000000001, 2024-09-13T10:11:23.086 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.086 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.086 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.086 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.086 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.087 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.087 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 290 2024-09-13T10:11:23.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.087 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255335+0000", 2024-09-13T10:11:23.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255335+0000", 2024-09-13T10:11:23.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255338+0000", 2024-09-13T10:11:23.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6940000000000001e-06 2024-09-13T10:11:23.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255353+0000", 2024-09-13T10:11:23.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4136e-05 2024-09-13T10:11:23.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255390+0000", 2024-09-13T10:11:23.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7364999999999999e-05 2024-09-13T10:11:23.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255413+0000", 2024-09-13T10:11:23.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2690000000000001e-05 2024-09-13T10:11:23.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255615+0000", 2024-09-13T10:11:23.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000202306 2024-09-13T10:11:23.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255813+0000", 2024-09-13T10:11:23.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019769300000000001 2024-09-13T10:11:23.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:23.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274190+0000", 2024-09-13T10:11:23.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01837714 2024-09-13T10:11:23.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275400+0000", 2024-09-13T10:11:23.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001210275 2024-09-13T10:11:23.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276346+0000", 2024-09-13T10:11:23.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00094586699999999998 2024-09-13T10:11:23.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276497+0000", 2024-09-13T10:11:23.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015150299999999999 2024-09-13T10:11:23.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276597+0000", 2024-09-13T10:11:23.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9092999999999995e-05 2024-09-13T10:11:23.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276670+0000", 2024-09-13T10:11:23.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3941000000000002e-05 2024-09-13T10:11:23.096 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.096 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.097 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.097 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:295 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.097 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.261467+0000", 2024-09-13T10:11:23.097 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.770031918000001, 2024-09-13T10:11:23.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020013564000000001, 2024-09-13T10:11:23.097 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.097 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.097 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.097 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.098 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.098 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.098 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 295 2024-09-13T10:11:23.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.098 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261467+0000", 2024-09-13T10:11:23.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261467+0000", 2024-09-13T10:11:23.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261470+0000", 2024-09-13T10:11:23.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8200000000000001e-06 2024-09-13T10:11:23.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261487+0000", 2024-09-13T10:11:23.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7646e-05 2024-09-13T10:11:23.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261523+0000", 2024-09-13T10:11:23.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5299000000000002e-05 2024-09-13T10:11:23.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261545+0000", 2024-09-13T10:11:23.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2436e-05 2024-09-13T10:11:23.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261717+0000", 2024-09-13T10:11:23.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017231000000000001 2024-09-13T10:11:23.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261980+0000", 2024-09-13T10:11:23.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000262959 2024-09-13T10:11:23.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:23.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277104+0000", 2024-09-13T10:11:23.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015123983000000001 2024-09-13T10:11:23.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.279042+0000", 2024-09-13T10:11:23.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001937338 2024-09-13T10:11:23.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.279396+0000", 2024-09-13T10:11:23.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00035408199999999998 2024-09-13T10:11:23.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281302+0000", 2024-09-13T10:11:23.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001906346 2024-09-13T10:11:23.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281400+0000", 2024-09-13T10:11:23.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7528000000000005e-05 2024-09-13T10:11:23.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281480+0000", 2024-09-13T10:11:23.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0816999999999996e-05 2024-09-13T10:11:23.104 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.104 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.104 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.104 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:297 4.c 4:381e677b:::smithi19530714-48:head [assert-version v15,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.104 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.262493+0000", 2024-09-13T10:11:23.104 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.769005749000002, 2024-09-13T10:11:23.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012458446, 2024-09-13T10:11:23.105 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.105 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.105 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.105 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.105 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.105 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.105 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 297 2024-09-13T10:11:23.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.105 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262493+0000", 2024-09-13T10:11:23.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262493+0000", 2024-09-13T10:11:23.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262496+0000", 2024-09-13T10:11:23.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4729999999999999e-06 2024-09-13T10:11:23.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262510+0000", 2024-09-13T10:11:23.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4205999999999999e-05 2024-09-13T10:11:23.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262545+0000", 2024-09-13T10:11:23.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4789000000000002e-05 2024-09-13T10:11:23.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262566+0000", 2024-09-13T10:11:23.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1789000000000001e-05 2024-09-13T10:11:23.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262748+0000", 2024-09-13T10:11:23.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000181848 2024-09-13T10:11:23.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262986+0000", 2024-09-13T10:11:23.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00023747500000000001 2024-09-13T10:11:23.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:23.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.267794+0000", 2024-09-13T10:11:23.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048086479999999996 2024-09-13T10:11:23.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.269878+0000", 2024-09-13T10:11:23.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0020833900000000001 2024-09-13T10:11:23.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270928+0000", 2024-09-13T10:11:23.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001049901 2024-09-13T10:11:23.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274724+0000", 2024-09-13T10:11:23.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0037964880000000002 2024-09-13T10:11:23.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274862+0000", 2024-09-13T10:11:23.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000138286 2024-09-13T10:11:23.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274952+0000", 2024-09-13T10:11:23.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9153000000000005e-05 2024-09-13T10:11:23.111 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.111 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.111 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.111 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:313 4.c 4:3e6706e5:::smithi19530714-14:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.111 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.275533+0000", 2024-09-13T10:11:23.111 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.755965532000001, 2024-09-13T10:11:23.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019582932000000001, 2024-09-13T10:11:23.112 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.112 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.112 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.112 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.112 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.112 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.112 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 313 2024-09-13T10:11:23.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.112 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275533+0000", 2024-09-13T10:11:23.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275533+0000", 2024-09-13T10:11:23.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275536+0000", 2024-09-13T10:11:23.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4899999999999999e-06 2024-09-13T10:11:23.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275549+0000", 2024-09-13T10:11:23.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3699e-05 2024-09-13T10:11:23.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275574+0000", 2024-09-13T10:11:23.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4586000000000001e-05 2024-09-13T10:11:23.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275588+0000", 2024-09-13T10:11:23.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3746e-05 2024-09-13T10:11:23.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275705+0000", 2024-09-13T10:11:23.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000117352 2024-09-13T10:11:23.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275898+0000", 2024-09-13T10:11:23.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019293000000000001 2024-09-13T10:11:23.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:23.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286176+0000", 2024-09-13T10:11:23.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010277578000000001 2024-09-13T10:11:23.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.287526+0000", 2024-09-13T10:11:23.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013505349999999999 2024-09-13T10:11:23.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.288025+0000", 2024-09-13T10:11:23.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00049897500000000003 2024-09-13T10:11:23.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294958+0000", 2024-09-13T10:11:23.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069327980000000004 2024-09-13T10:11:23.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.295047+0000", 2024-09-13T10:11:23.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9154999999999996e-05 2024-09-13T10:11:23.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.295116+0000", 2024-09-13T10:11:23.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9088e-05 2024-09-13T10:11:23.118 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.118 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.118 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.119 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:314 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.119 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.276956+0000", 2024-09-13T10:11:23.119 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.754542853, 2024-09-13T10:11:23.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017940078000000002, 2024-09-13T10:11:23.119 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.119 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.119 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.119 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.119 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.119 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.119 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 314 2024-09-13T10:11:23.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.119 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276956+0000", 2024-09-13T10:11:23.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276956+0000", 2024-09-13T10:11:23.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276959+0000", 2024-09-13T10:11:23.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5730000000000002e-06 2024-09-13T10:11:23.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276971+0000", 2024-09-13T10:11:23.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2680000000000001e-05 2024-09-13T10:11:23.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276997+0000", 2024-09-13T10:11:23.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5579e-05 2024-09-13T10:11:23.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277018+0000", 2024-09-13T10:11:23.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1409000000000001e-05 2024-09-13T10:11:23.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277136+0000", 2024-09-13T10:11:23.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000117353 2024-09-13T10:11:23.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277237+0000", 2024-09-13T10:11:23.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101412 2024-09-13T10:11:23.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:23.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.285332+0000", 2024-09-13T10:11:23.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080951770000000003 2024-09-13T10:11:23.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.287120+0000", 2024-09-13T10:11:23.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001787828 2024-09-13T10:11:23.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.287200+0000", 2024-09-13T10:11:23.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0279999999999997e-05 2024-09-13T10:11:23.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294808+0000", 2024-09-13T10:11:23.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0076072400000000004 2024-09-13T10:11:23.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294853+0000", 2024-09-13T10:11:23.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5438000000000001e-05 2024-09-13T10:11:23.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294896+0000", 2024-09-13T10:11:23.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3108999999999997e-05 2024-09-13T10:11:23.125 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.125 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.125 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.126 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:340 4.d 4:b19b932a:::smithi19530714-41:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.126 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.284107+0000", 2024-09-13T10:11:23.126 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.747391622999999, 2024-09-13T10:11:23.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010293416, 2024-09-13T10:11:23.126 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.126 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.126 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.126 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.126 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.126 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.126 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 340 2024-09-13T10:11:23.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.127 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284107+0000", 2024-09-13T10:11:23.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284107+0000", 2024-09-13T10:11:23.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284109+0000", 2024-09-13T10:11:23.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.007e-06 2024-09-13T10:11:23.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284120+0000", 2024-09-13T10:11:23.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0346999999999999e-05 2024-09-13T10:11:23.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284144+0000", 2024-09-13T10:11:23.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3998999999999998e-05 2024-09-13T10:11:23.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284157+0000", 2024-09-13T10:11:23.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3045e-05 2024-09-13T10:11:23.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284244+0000", 2024-09-13T10:11:23.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7882999999999999e-05 2024-09-13T10:11:23.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284360+0000", 2024-09-13T10:11:23.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115467 2024-09-13T10:11:23.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:23.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.292368+0000", 2024-09-13T10:11:23.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080080380000000003 2024-09-13T10:11:23.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.293486+0000", 2024-09-13T10:11:23.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001117776 2024-09-13T10:11:23.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294261+0000", 2024-09-13T10:11:23.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00077481100000000003 2024-09-13T10:11:23.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294309+0000", 2024-09-13T10:11:23.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8086e-05 2024-09-13T10:11:23.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294358+0000", 2024-09-13T10:11:23.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8853999999999997e-05 2024-09-13T10:11:23.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294401+0000", 2024-09-13T10:11:23.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3102999999999997e-05 2024-09-13T10:11:23.132 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.132 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.132 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.132 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:345 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.133 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.286309+0000", 2024-09-13T10:11:23.133 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.745189445000001, 2024-09-13T10:11:23.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010478568000000001, 2024-09-13T10:11:23.133 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.133 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.133 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.133 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.133 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.133 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.133 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 345 2024-09-13T10:11:23.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.133 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286309+0000", 2024-09-13T10:11:23.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286309+0000", 2024-09-13T10:11:23.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286311+0000", 2024-09-13T10:11:23.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-13T10:11:23.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286320+0000", 2024-09-13T10:11:23.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8370000000000003e-06 2024-09-13T10:11:23.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286342+0000", 2024-09-13T10:11:23.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1446000000000001e-05 2024-09-13T10:11:23.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286385+0000", 2024-09-13T10:11:23.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3083000000000001e-05 2024-09-13T10:11:23.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286549+0000", 2024-09-13T10:11:23.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016463 2024-09-13T10:11:23.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286739+0000", 2024-09-13T10:11:23.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018965400000000001 2024-09-13T10:11:23.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:23.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294808+0000", 2024-09-13T10:11:23.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080692609999999994 2024-09-13T10:11:23.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.295388+0000", 2024-09-13T10:11:23.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00057994000000000003 2024-09-13T10:11:23.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296321+0000", 2024-09-13T10:11:23.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00093279099999999998 2024-09-13T10:11:23.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296622+0000", 2024-09-13T10:11:23.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030084300000000002 2024-09-13T10:11:23.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296702+0000", 2024-09-13T10:11:23.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0547e-05 2024-09-13T10:11:23.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296788+0000", 2024-09-13T10:11:23.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5612999999999995e-05 2024-09-13T10:11:23.139 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.139 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.139 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.139 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:364 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:23.139 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.297175+0000", 2024-09-13T10:11:23.140 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.734324268999998, 2024-09-13T10:11:23.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0085499329999999991, 2024-09-13T10:11:23.140 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.140 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.140 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.140 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.140 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.140 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.140 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 364 2024-09-13T10:11:23.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.140 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297175+0000", 2024-09-13T10:11:23.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297175+0000", 2024-09-13T10:11:23.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297176+0000", 2024-09-13T10:11:23.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9199999999999999e-07 2024-09-13T10:11:23.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297181+0000", 2024-09-13T10:11:23.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5149999999999997e-06 2024-09-13T10:11:23.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297191+0000", 2024-09-13T10:11:23.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8260000000000005e-06 2024-09-13T10:11:23.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297198+0000", 2024-09-13T10:11:23.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6950000000000003e-06 2024-09-13T10:11:23.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297244+0000", 2024-09-13T10:11:23.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6397999999999999e-05 2024-09-13T10:11:23.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297291+0000", 2024-09-13T10:11:23.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7342999999999997e-05 2024-09-13T10:11:23.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:23.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.303658+0000", 2024-09-13T10:11:23.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0063669 2024-09-13T10:11:23.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.304437+0000", 2024-09-13T10:11:23.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00077861399999999996 2024-09-13T10:11:23.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.304902+0000", 2024-09-13T10:11:23.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00046505899999999998 2024-09-13T10:11:23.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.305534+0000", 2024-09-13T10:11:23.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00063224199999999998 2024-09-13T10:11:23.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.305643+0000", 2024-09-13T10:11:23.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108384 2024-09-13T10:11:23.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.305725+0000", 2024-09-13T10:11:23.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1965000000000006e-05 2024-09-13T10:11:23.146 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.146 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.146 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.146 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:378 4.c 4:39f17ee9:::smithi19530714-29:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:23.147 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.320710+0000", 2024-09-13T10:11:23.147 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.710788898000001, 2024-09-13T10:11:23.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0079049020000000001, 2024-09-13T10:11:23.147 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.147 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.147 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.147 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.147 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.147 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.147 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 378 2024-09-13T10:11:23.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.147 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320710+0000", 2024-09-13T10:11:23.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320710+0000", 2024-09-13T10:11:23.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320712+0000", 2024-09-13T10:11:23.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4099999999999998e-06 2024-09-13T10:11:23.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320724+0000", 2024-09-13T10:11:23.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.134e-05 2024-09-13T10:11:23.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320746+0000", 2024-09-13T10:11:23.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2342999999999999e-05 2024-09-13T10:11:23.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320758+0000", 2024-09-13T10:11:23.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1557e-05 2024-09-13T10:11:23.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320840+0000", 2024-09-13T10:11:23.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2274000000000005e-05 2024-09-13T10:11:23.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320933+0000", 2024-09-13T10:11:23.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3375999999999995e-05 2024-09-13T10:11:23.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:23.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.326291+0000", 2024-09-13T10:11:23.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053575089999999999 2024-09-13T10:11:23.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.327254+0000", 2024-09-13T10:11:23.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00096337099999999998 2024-09-13T10:11:23.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.327902+0000", 2024-09-13T10:11:23.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00064762100000000003 2024-09-13T10:11:23.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328501+0000", 2024-09-13T10:11:23.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000599232 2024-09-13T10:11:23.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328547+0000", 2024-09-13T10:11:23.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5973000000000002e-05 2024-09-13T10:11:23.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328615+0000", 2024-09-13T10:11:23.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7896000000000002e-05 2024-09-13T10:11:23.153 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.153 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.153 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.154 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:395 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:23.154 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.322105+0000", 2024-09-13T10:11:23.154 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.709393825999999, 2024-09-13T10:11:23.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0084827089999999997, 2024-09-13T10:11:23.154 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.154 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.154 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.154 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.154 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.154 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.154 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 395 2024-09-13T10:11:23.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.155 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322105+0000", 2024-09-13T10:11:23.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322105+0000", 2024-09-13T10:11:23.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322106+0000", 2024-09-13T10:11:23.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2099999999999995e-07 2024-09-13T10:11:23.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322110+0000", 2024-09-13T10:11:23.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.827e-06 2024-09-13T10:11:23.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322120+0000", 2024-09-13T10:11:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0227999999999999e-05 2024-09-13T10:11:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322160+0000", 2024-09-13T10:11:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9941999999999998e-05 2024-09-13T10:11:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322232+0000", 2024-09-13T10:11:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2451000000000001e-05 2024-09-13T10:11:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322314+0000", 2024-09-13T10:11:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2089000000000006e-05 2024-09-13T10:11:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328374+0000", 2024-09-13T10:11:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0060591350000000002 2024-09-13T10:11:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.329439+0000", 2024-09-13T10:11:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010654830000000001 2024-09-13T10:11:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330219+0000", 2024-09-13T10:11:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00078003700000000003 2024-09-13T10:11:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330488+0000", 2024-09-13T10:11:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000269411 2024-09-13T10:11:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330544+0000", 2024-09-13T10:11:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5704000000000001e-05 2024-09-13T10:11:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330588+0000", 2024-09-13T10:11:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3581000000000002e-05 2024-09-13T10:11:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:595 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.370445+0000", 2024-09-13T10:11:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.661053792000001, 2024-09-13T10:11:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0084586149999999992, 2024-09-13T10:11:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 595 2024-09-13T10:11:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370445+0000", 2024-09-13T10:11:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370445+0000", 2024-09-13T10:11:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370447+0000", 2024-09-13T10:11:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.66e-06 2024-09-13T10:11:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370455+0000", 2024-09-13T10:11:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3059999999999998e-06 2024-09-13T10:11:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370472+0000", 2024-09-13T10:11:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7479999999999999e-05 2024-09-13T10:11:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370484+0000", 2024-09-13T10:11:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1882000000000001e-05 2024-09-13T10:11:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370562+0000", 2024-09-13T10:11:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7531e-05 2024-09-13T10:11:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370738+0000", 2024-09-13T10:11:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017587200000000001 2024-09-13T10:11:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.377463+0000", 2024-09-13T10:11:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0067247829999999998 2024-09-13T10:11:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378047+0000", 2024-09-13T10:11:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00058493100000000001 2024-09-13T10:11:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378730+0000", 2024-09-13T10:11:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068235799999999999 2024-09-13T10:11:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378840+0000", 2024-09-13T10:11:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110296 2024-09-13T10:11:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378875+0000", 2024-09-13T10:11:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4552999999999999e-05 2024-09-13T10:11:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378904+0000", 2024-09-13T10:11:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8963e-05 2024-09-13T10:11:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:782 4.c 4:39f17ee9:::smithi19530714-29:head [write 1543557~550004] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.601833+0000", 2024-09-13T10:11:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.429665918000001, 2024-09-13T10:11:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081663250000000003, 2024-09-13T10:11:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 782 2024-09-13T10:11:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.601833+0000", 2024-09-13T10:11:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.601833+0000", 2024-09-13T10:11:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.601835+0000", 2024-09-13T10:11:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.576e-06 2024-09-13T10:11:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602350+0000", 2024-09-13T10:11:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00051449700000000004 2024-09-13T10:11:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602379+0000", 2024-09-13T10:11:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8598999999999998e-05 2024-09-13T10:11:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602394+0000", 2024-09-13T10:11:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4952e-05 2024-09-13T10:11:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602534+0000", 2024-09-13T10:11:23.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014037899999999999 2024-09-13T10:11:23.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602796+0000", 2024-09-13T10:11:23.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000262415 2024-09-13T10:11:23.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:23.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.603214+0000", 2024-09-13T10:11:23.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00041785599999999998 2024-09-13T10:11:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.606389+0000", 2024-09-13T10:11:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0031751549999999998 2024-09-13T10:11:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.608306+0000", 2024-09-13T10:11:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0019163540000000001 2024-09-13T10:11:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.609577+0000", 2024-09-13T10:11:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001271466 2024-09-13T10:11:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.609872+0000", 2024-09-13T10:11:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00029501400000000002 2024-09-13T10:11:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.609999+0000", 2024-09-13T10:11:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127062 2024-09-13T10:11:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1085 4.c 4:3e6706e5:::smithi19530714-14:head [sparse-read 0~906036] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:56.648267+0000", 2024-09-13T10:11:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 26.383231394999999, 2024-09-13T10:11:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0083262089999999993, 2024-09-13T10:11:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:11:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1085 2024-09-13T10:11:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648267+0000", 2024-09-13T10:11:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648267+0000", 2024-09-13T10:11:23.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:23.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:23.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648269+0000", 2024-09-13T10:11:23.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1769999999999999e-06 2024-09-13T10:11:23.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:23.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648276+0000", 2024-09-13T10:11:23.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0530000000000001e-06 2024-09-13T10:11:23.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:23.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648292+0000", 2024-09-13T10:11:23.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6209e-05 2024-09-13T10:11:23.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648307+0000", 2024-09-13T10:11:23.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4647000000000001e-05 2024-09-13T10:11:23.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:23.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.655806+0000", 2024-09-13T10:11:23.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0074996500000000001 2024-09-13T10:11:23.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:23.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.656060+0000", 2024-09-13T10:11:23.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000253639 2024-09-13T10:11:23.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:23.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:23.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:23.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.656594+0000", 2024-09-13T10:11:23.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00053383399999999996 2024-09-13T10:11:23.180 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.180 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.180 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.180 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:23.180 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:23.180 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:23.182 INFO:tasks.rados.rados.0.smithi195.stdout:1160: finishing write tid 1 to smithi19530714-6 2024-09-13T10:11:23.182 INFO:tasks.rados.rados.0.smithi195.stdout:1160: finishing write tid 2 to smithi19530714-6 2024-09-13T10:11:23.182 INFO:tasks.rados.rados.0.smithi195.stdout:1160: finishing write tid 3 to smithi19530714-6 2024-09-13T10:11:23.182 INFO:tasks.rados.rados.0.smithi195.stdout:1160: finishing write tid 4 to smithi19530714-6 2024-09-13T10:11:23.182 INFO:tasks.rados.rados.0.smithi195.stdout:1160: finishing write tid 5 to smithi19530714-6 2024-09-13T10:11:23.182 INFO:tasks.rados.rados.0.smithi195.stdout:1160: oid 6 updating version 0 to 59 2024-09-13T10:11:23.182 INFO:tasks.rados.rados.0.smithi195.stdout:1160: oid 6 updating version 59 to 62 2024-09-13T10:11:23.182 INFO:tasks.rados.rados.0.smithi195.stdout:1160: oid 6 version 62 is already newer than 61 2024-09-13T10:11:23.182 INFO:tasks.rados.rados.0.smithi195.stdout:1160: oid 6 version 62 is already newer than 60 2024-09-13T10:11:23.182 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 62 (ObjNum 214 snap 0 seq_num 214) dirty exists 2024-09-13T10:11:23.182 INFO:tasks.rados.rados.0.smithi195.stdout:1160: left oid 6 (ObjNum 214 snap 0 seq_num 214) 2024-09-13T10:11:23.183 INFO:tasks.rados.rados.0.smithi195.stdout:1161: expect (ObjNum 209 snap 0 seq_num 209) 2024-09-13T10:11:23.183 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 164311 2024-09-13T10:11:23.183 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:23.178+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:23.183 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T10:11:23.227 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:23.282 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:23.280+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:23.291 INFO:tasks.rados.rados.0.smithi195.stdout:1157: done (5 left) 2024-09-13T10:11:23.291 INFO:tasks.rados.rados.0.smithi195.stdout:1158: done (4 left) 2024-09-13T10:11:23.291 INFO:tasks.rados.rados.0.smithi195.stdout:1159: done (3 left) 2024-09-13T10:11:23.291 INFO:tasks.rados.rados.0.smithi195.stdout:1160: done (2 left) 2024-09-13T10:11:23.292 INFO:tasks.rados.rados.0.smithi195.stdout:1161: done (1 left) 2024-09-13T10:11:23.292 INFO:tasks.rados.rados.0.smithi195.stdout:1163: read oid 34 snap -1 2024-09-13T10:11:23.292 INFO:tasks.rados.rados.0.smithi195.stdout:1163: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:23.292 INFO:tasks.rados.rados.0.smithi195.stdout:1164: read oid 32 snap -1 2024-09-13T10:11:23.292 INFO:tasks.rados.rados.0.smithi195.stdout:1164: expect deleted 2024-09-13T10:11:23.292 INFO:tasks.rados.rados.0.smithi195.stdout:1165: tier_promote oid 18 2024-09-13T10:11:23.292 INFO:tasks.rados.rados.0.smithi195.stdout:1166: read oid 47 snap -1 2024-09-13T10:11:23.292 INFO:tasks.rados.rados.0.smithi195.stdout:1166: expect (ObjNum 182 snap 0 seq_num 182) 2024-09-13T10:11:23.292 INFO:tasks.rados.rados.0.smithi195.stdout:1167: read oid 27 snap -1 2024-09-13T10:11:23.292 INFO:tasks.rados.rados.0.smithi195.stdout:1167: expect (ObjNum 213 snap 0 seq_num 213) 2024-09-13T10:11:23.292 INFO:tasks.rados.rados.0.smithi195.stdout:1168: write (excl) oid 25 current snap is 0 2024-09-13T10:11:23.292 INFO:tasks.rados.rados.0.smithi195.stdout:1168: seq_num 215 ranges {413505=636419,1664805=689064,3038061=492071} 2024-09-13T10:11:23.304 INFO:tasks.rados.rados.0.smithi195.stdout:1168: writing smithi19530714-25 from 413505 to 1049924 tid 1 2024-09-13T10:11:23.320 INFO:tasks.rados.rados.0.smithi195.stdout:1168: writing smithi19530714-25 from 1664805 to 2353869 tid 2 2024-09-13T10:11:23.321 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:23.321 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:23.321 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:11:23.321 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:23.335 INFO:tasks.rados.rados.0.smithi195.stdout:1168: writing smithi19530714-25 from 3038061 to 3530132 tid 3 2024-09-13T10:11:23.335 INFO:tasks.rados.rados.0.smithi195.stdout:1169: delete oid 9 current snap is 0 2024-09-13T10:11:23.335 INFO:tasks.rados.rados.0.smithi195.stdout:1166: expect (ObjNum 182 snap 0 seq_num 182) 2024-09-13T10:11:23.336 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T10:11:23.383 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:23.380+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:23.419 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 585815 2024-09-13T10:11:23.457 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:23.457 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:23.457 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:11:23.457 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:11:23.457 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:23.458 INFO:tasks.rados.rados.0.smithi195.stdout:1163: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:23.458 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:23.472 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T10:11:23.490 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:11:23.519 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:23.516+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:23.573 INFO:tasks.rados.rados.0.smithi195.stdout:1165: got (0) Success 2024-09-13T10:11:23.573 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 69 (ObjNum 189 snap 0 seq_num 189) dirty exists 2024-09-13T10:11:23.573 INFO:tasks.rados.rados.0.smithi195.stdout:1167: expect (ObjNum 213 snap 0 seq_num 213) 2024-09-13T10:11:23.585 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:23.582+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:23.599 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:23.599 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:11:23.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:11:23.599 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:11:23.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.599 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:246 3.3 e38/30 3:f1be28e7:::smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'10, mlcod=38'5)", 2024-09-13T10:11:23.599 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.130216+0000", 2024-09-13T10:11:23.599 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.465195381999997, 2024-09-13T10:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011932478, 2024-09-13T10:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.130216+0000", 2024-09-13T10:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.130216+0000", 2024-09-13T10:11:23.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.130217+0000", 2024-09-13T10:11:23.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.654e-06 2024-09-13T10:11:23.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.140044+0000", 2024-09-13T10:11:23.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0098266870000000006 2024-09-13T10:11:23.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.140059+0000", 2024-09-13T10:11:23.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5304000000000001e-05 2024-09-13T10:11:23.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.140067+0000", 2024-09-13T10:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6750000000000002e-06 2024-09-13T10:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.140118+0000", 2024-09-13T10:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1462000000000003e-05 2024-09-13T10:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.140138+0000", 2024-09-13T10:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9542999999999999e-05 2024-09-13T10:11:23.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.142085+0000", 2024-09-13T10:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019473959999999999 2024-09-13T10:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.142148+0000", 2024-09-13T10:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2756999999999996e-05 2024-09-13T10:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:253 4.7 4:e81d9ad0:::smithi19530714-4:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.134475+0000", 2024-09-13T10:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.460936029000003, 2024-09-13T10:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013417535, 2024-09-13T10:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 253 2024-09-13T10:11:23.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.606 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134475+0000", 2024-09-13T10:11:23.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134475+0000", 2024-09-13T10:11:23.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134477+0000", 2024-09-13T10:11:23.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1859999999999999e-06 2024-09-13T10:11:23.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134486+0000", 2024-09-13T10:11:23.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3139999999999993e-06 2024-09-13T10:11:23.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134506+0000", 2024-09-13T10:11:23.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9165999999999999e-05 2024-09-13T10:11:23.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134515+0000", 2024-09-13T10:11:23.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3670000000000003e-06 2024-09-13T10:11:23.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134579+0000", 2024-09-13T10:11:23.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4200000000000002e-05 2024-09-13T10:11:23.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134661+0000", 2024-09-13T10:11:23.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1632999999999996e-05 2024-09-13T10:11:23.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:23.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.144025+0000", 2024-09-13T10:11:23.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093644190000000006 2024-09-13T10:11:23.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:23.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.144574+0000", 2024-09-13T10:11:23.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054873699999999999 2024-09-13T10:11:23.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.146021+0000", 2024-09-13T10:11:23.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001446826 2024-09-13T10:11:23.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.147761+0000", 2024-09-13T10:11:23.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017406399999999999 2024-09-13T10:11:23.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:23.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.147835+0000", 2024-09-13T10:11:23.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3708999999999995e-05 2024-09-13T10:11:23.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.147893+0000", 2024-09-13T10:11:23.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7337999999999999e-05 2024-09-13T10:11:23.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.619 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.620 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.620 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:256 4.7 4:e8c2cbec:::smithi19530714-7:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.620 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.214859+0000", 2024-09-13T10:11:23.621 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.380551670999999, 2024-09-13T10:11:23.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049512370999999999, 2024-09-13T10:11:23.621 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.621 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.621 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.621 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.622 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.622 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.622 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 256 2024-09-13T10:11:23.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.622 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214859+0000", 2024-09-13T10:11:23.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214859+0000", 2024-09-13T10:11:23.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214863+0000", 2024-09-13T10:11:23.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.343e-06 2024-09-13T10:11:23.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214876+0000", 2024-09-13T10:11:23.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.308e-05 2024-09-13T10:11:23.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214909+0000", 2024-09-13T10:11:23.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2882999999999997e-05 2024-09-13T10:11:23.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214925+0000", 2024-09-13T10:11:23.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6002e-05 2024-09-13T10:11:23.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215088+0000", 2024-09-13T10:11:23.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016304199999999999 2024-09-13T10:11:23.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215261+0000", 2024-09-13T10:11:23.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000173804 2024-09-13T10:11:23.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:23.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258838+0000", 2024-09-13T10:11:23.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043576675000000002 2024-09-13T10:11:23.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:23.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.260096+0000", 2024-09-13T10:11:23.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012580410000000001 2024-09-13T10:11:23.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.263959+0000", 2024-09-13T10:11:23.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038630840000000001 2024-09-13T10:11:23.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264262+0000", 2024-09-13T10:11:23.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030279699999999999 2024-09-13T10:11:23.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:23.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264321+0000", 2024-09-13T10:11:23.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-13T10:11:23.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264372+0000", 2024-09-13T10:11:23.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0513000000000002e-05 2024-09-13T10:11:23.635 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.635 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.635 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.636 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:266 4.7 4:e55883d8:::smithi19530714-17:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.636 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.215454+0000", 2024-09-13T10:11:23.636 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.379957009999998, 2024-09-13T10:11:23.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020880112999999999, 2024-09-13T10:11:23.637 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.637 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.637 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.637 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.638 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.638 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.638 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 266 2024-09-13T10:11:23.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.638 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215454+0000", 2024-09-13T10:11:23.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215454+0000", 2024-09-13T10:11:23.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215457+0000", 2024-09-13T10:11:23.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6029999999999999e-06 2024-09-13T10:11:23.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215471+0000", 2024-09-13T10:11:23.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4629e-05 2024-09-13T10:11:23.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215503+0000", 2024-09-13T10:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1449000000000003e-05 2024-09-13T10:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215528+0000", 2024-09-13T10:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5273e-05 2024-09-13T10:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215632+0000", 2024-09-13T10:11:23.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010387699999999999 2024-09-13T10:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215898+0000", 2024-09-13T10:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026582100000000002 2024-09-13T10:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228364+0000", 2024-09-13T10:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012466672 2024-09-13T10:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.229463+0000", 2024-09-13T10:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001098344 2024-09-13T10:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.230814+0000", 2024-09-13T10:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013513959999999999 2024-09-13T10:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236141+0000", 2024-09-13T10:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053270160000000004 2024-09-13T10:11:23.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236223+0000", 2024-09-13T10:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1799999999999996e-05 2024-09-13T10:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236334+0000", 2024-09-13T10:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111233 2024-09-13T10:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:267 4.7 4:ea79a1ec:::smithi19530714-18:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.215583+0000", 2024-09-13T10:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.379828037999999, 2024-09-13T10:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.051600035000000002, 2024-09-13T10:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 267 2024-09-13T10:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.646 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215583+0000", 2024-09-13T10:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215583+0000", 2024-09-13T10:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215585+0000", 2024-09-13T10:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9099999999999999e-06 2024-09-13T10:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215594+0000", 2024-09-13T10:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8799999999999997e-06 2024-09-13T10:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215613+0000", 2024-09-13T10:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9473000000000001e-05 2024-09-13T10:11:23.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215630+0000", 2024-09-13T10:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6466000000000001e-05 2024-09-13T10:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.216360+0000", 2024-09-13T10:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073001200000000004 2024-09-13T10:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.216526+0000", 2024-09-13T10:11:23.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000166775 2024-09-13T10:11:23.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:23.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.260535+0000", 2024-09-13T10:11:23.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044008310000000002 2024-09-13T10:11:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261671+0000", 2024-09-13T10:11:23.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011362169999999999 2024-09-13T10:11:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.266819+0000", 2024-09-13T10:11:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051479940000000004 2024-09-13T10:11:23.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.267087+0000", 2024-09-13T10:11:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026847299999999998 2024-09-13T10:11:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:23.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.267140+0000", 2024-09-13T10:11:23.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2054e-05 2024-09-13T10:11:23.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.267183+0000", 2024-09-13T10:11:23.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3470999999999998e-05 2024-09-13T10:11:23.655 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.655 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.655 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.656 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:303 4.7 4:e81d9ad0:::smithi19530714-4:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:23.656 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.268636+0000", 2024-09-13T10:11:23.656 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.326775423000001, 2024-09-13T10:11:23.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018691609000000001, 2024-09-13T10:11:23.657 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.657 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.657 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.657 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.658 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.658 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.658 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 303 2024-09-13T10:11:23.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.658 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268636+0000", 2024-09-13T10:11:23.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268636+0000", 2024-09-13T10:11:23.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268638+0000", 2024-09-13T10:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8990000000000001e-06 2024-09-13T10:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268651+0000", 2024-09-13T10:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2556e-05 2024-09-13T10:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268682+0000", 2024-09-13T10:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0979000000000003e-05 2024-09-13T10:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268707+0000", 2024-09-13T10:11:23.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4680000000000001e-05 2024-09-13T10:11:23.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268787+0000", 2024-09-13T10:11:23.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9878999999999999e-05 2024-09-13T10:11:23.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268881+0000", 2024-09-13T10:11:23.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4242999999999999e-05 2024-09-13T10:11:23.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:23.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.283577+0000", 2024-09-13T10:11:23.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014695952999999999 2024-09-13T10:11:23.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:23.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.284599+0000", 2024-09-13T10:11:23.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001022329 2024-09-13T10:11:23.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285985+0000", 2024-09-13T10:11:23.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013861430000000001 2024-09-13T10:11:23.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.287150+0000", 2024-09-13T10:11:23.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001164382 2024-09-13T10:11:23.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:23.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.287245+0000", 2024-09-13T10:11:23.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5643999999999994e-05 2024-09-13T10:11:23.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.287327+0000", 2024-09-13T10:11:23.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1922000000000006e-05 2024-09-13T10:11:23.668 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.668 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.668 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.669 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:567 4.7 4:ea79a1ec:::smithi19530714-18:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:23.669 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:11:23.669 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.228110270999998, 2024-09-13T10:11:23.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012585694999999999, 2024-09-13T10:11:23.670 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.670 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.670 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.670 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.670 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.671 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.671 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 567 2024-09-13T10:11:23.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.671 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:11:23.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1200000000000003e-07 2024-09-13T10:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367306+0000", 2024-09-13T10:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0309999999999998e-06 2024-09-13T10:11:23.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367314+0000", 2024-09-13T10:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8960000000000003e-06 2024-09-13T10:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367321+0000", 2024-09-13T10:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4880000000000004e-06 2024-09-13T10:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367612+0000", 2024-09-13T10:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029163099999999999 2024-09-13T10:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367715+0000", 2024-09-13T10:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102837 2024-09-13T10:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.374040+0000", 2024-09-13T10:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063245539999999996 2024-09-13T10:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.374609+0000", 2024-09-13T10:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056937299999999997 2024-09-13T10:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379774+0000", 2024-09-13T10:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005164737 2024-09-13T10:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379810+0000", 2024-09-13T10:11:23.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6587000000000001e-05 2024-09-13T10:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379845+0000", 2024-09-13T10:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.468e-05 2024-09-13T10:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379886+0000", 2024-09-13T10:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1269000000000001e-05 2024-09-13T10:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.677 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:762 4.7 4:ea79a1ec:::smithi19530714-18:head [write 457157~535817,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.449773+0000", 2024-09-13T10:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.145637833999999, 2024-09-13T10:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012291262000000001, 2024-09-13T10:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.678 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 762 2024-09-13T10:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.449773+0000", 2024-09-13T10:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.449773+0000", 2024-09-13T10:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.449777+0000", 2024-09-13T10:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1969999999999998e-06 2024-09-13T10:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.450154+0000", 2024-09-13T10:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037642099999999999 2024-09-13T10:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.450175+0000", 2024-09-13T10:11:23.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0891e-05 2024-09-13T10:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.450185+0000", 2024-09-13T10:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362e-05 2024-09-13T10:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.450251+0000", 2024-09-13T10:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5712999999999999e-05 2024-09-13T10:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.450324+0000", 2024-09-13T10:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2885000000000004e-05 2024-09-13T10:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.450451+0000", 2024-09-13T10:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012726600000000001 2024-09-13T10:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.451966+0000", 2024-09-13T10:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001515408 2024-09-13T10:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.461707+0000", 2024-09-13T10:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097404339999999992 2024-09-13T10:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.461837+0000", 2024-09-13T10:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013055900000000001 2024-09-13T10:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.461995+0000", 2024-09-13T10:11:23.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015793400000000001 2024-09-13T10:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.462064+0000", 2024-09-13T10:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9191999999999998e-05 2024-09-13T10:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1039 4.17 4:ea79a1ec:::smithi19530714-18:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.096166+0000", 2024-09-13T10:11:23.684 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.499245014, 2024-09-13T10:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.051040927, 2024-09-13T10:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1039 2024-09-13T10:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096166+0000", 2024-09-13T10:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096166+0000", 2024-09-13T10:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096169+0000", 2024-09-13T10:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8100000000000002e-06 2024-09-13T10:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096180+0000", 2024-09-13T10:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1537e-05 2024-09-13T10:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096212+0000", 2024-09-13T10:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2104999999999998e-05 2024-09-13T10:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096268+0000", 2024-09-13T10:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5127999999999998e-05 2024-09-13T10:11:23.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096480+0000", 2024-09-13T10:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021249100000000001 2024-09-13T10:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096692+0000", 2024-09-13T10:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021200400000000001 2024-09-13T10:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.147207+0000", 2024-09-13T10:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.050514851999999999 2024-09-13T10:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1573 4.7 4:e79f1505:::smithi19530714-32:head [write 601717~635620] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.711520+0000", 2024-09-13T10:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.883890749999999, 2024-09-13T10:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011180353000000001, 2024-09-13T10:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.689 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1573 2024-09-13T10:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.711520+0000", 2024-09-13T10:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.711520+0000", 2024-09-13T10:11:23.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.711523+0000", 2024-09-13T10:11:23.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1499999999999999e-06 2024-09-13T10:11:23.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.712421+0000", 2024-09-13T10:11:23.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089757400000000005 2024-09-13T10:11:23.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.712452+0000", 2024-09-13T10:11:23.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0858999999999997e-05 2024-09-13T10:11:23.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.712468+0000", 2024-09-13T10:11:23.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6135999999999999e-05 2024-09-13T10:11:23.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.712618+0000", 2024-09-13T10:11:23.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014958699999999999 2024-09-13T10:11:23.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.712768+0000", 2024-09-13T10:11:23.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150536 2024-09-13T10:11:23.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:23.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.713000+0000", 2024-09-13T10:11:23.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023144100000000001 2024-09-13T10:11:23.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:23.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.716577+0000", 2024-09-13T10:11:23.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003577532 2024-09-13T10:11:23.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722251+0000", 2024-09-13T10:11:23.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0056736219999999997 2024-09-13T10:11:23.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722426+0000", 2024-09-13T10:11:23.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017517300000000001 2024-09-13T10:11:23.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:23.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722574+0000", 2024-09-13T10:11:23.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014806899999999999 2024-09-13T10:11:23.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722701+0000", 2024-09-13T10:11:23.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012667399999999999 2024-09-13T10:11:23.695 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.695 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.695 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.696 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1574 4.7 4:e79f1505:::smithi19530714-32:head [write 1799494~402223] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:11:23.696 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.716930+0000", 2024-09-13T10:11:23.696 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.878480748000001, 2024-09-13T10:11:23.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017286329999999999, 2024-09-13T10:11:23.696 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.696 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.696 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.696 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1574 2024-09-13T10:11:23.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.697 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.716930+0000", 2024-09-13T10:11:23.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.716930+0000", 2024-09-13T10:11:23.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.716962+0000", 2024-09-13T10:11:23.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1279999999999999e-05 2024-09-13T10:11:23.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717453+0000", 2024-09-13T10:11:23.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049160399999999998 2024-09-13T10:11:23.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717480+0000", 2024-09-13T10:11:23.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7313000000000001e-05 2024-09-13T10:11:23.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717496+0000", 2024-09-13T10:11:23.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5435999999999999e-05 2024-09-13T10:11:23.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717592+0000", 2024-09-13T10:11:23.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5851000000000001e-05 2024-09-13T10:11:23.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717732+0000", 2024-09-13T10:11:23.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013985000000000001 2024-09-13T10:11:23.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:23.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717962+0000", 2024-09-13T10:11:23.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000230518 2024-09-13T10:11:23.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:23.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722934+0000", 2024-09-13T10:11:23.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049717099999999998 2024-09-13T10:11:23.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724179+0000", 2024-09-13T10:11:23.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012455389999999999 2024-09-13T10:11:23.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.733963+0000", 2024-09-13T10:11:23.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097835349999999995 2024-09-13T10:11:23.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:23.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.734101+0000", 2024-09-13T10:11:23.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013852 2024-09-13T10:11:23.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.734217+0000", 2024-09-13T10:11:23.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011517400000000001 2024-09-13T10:11:23.702 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.702 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.704 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.704 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1575 4.7 4:e79f1505:::smithi19530714-32:head [setxattr _header (56) in=7b,truncate 2201717] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:11:23.704 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.717530+0000", 2024-09-13T10:11:23.704 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.877880592, 2024-09-13T10:11:23.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018674682000000001, 2024-09-13T10:11:23.705 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.705 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.705 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.705 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.705 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.705 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.705 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1575 2024-09-13T10:11:23.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.705 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717530+0000", 2024-09-13T10:11:23.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717530+0000", 2024-09-13T10:11:23.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717532+0000", 2024-09-13T10:11:23.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.813e-06 2024-09-13T10:11:23.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717542+0000", 2024-09-13T10:11:23.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4930000000000004e-06 2024-09-13T10:11:23.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717559+0000", 2024-09-13T10:11:23.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7337e-05 2024-09-13T10:11:23.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717575+0000", 2024-09-13T10:11:23.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6385999999999999e-05 2024-09-13T10:11:23.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.719954+0000", 2024-09-13T10:11:23.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023782679999999998 2024-09-13T10:11:23.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.720308+0000", 2024-09-13T10:11:23.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035397100000000002 2024-09-13T10:11:23.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:23.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.720827+0000", 2024-09-13T10:11:23.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00051909999999999999 2024-09-13T10:11:23.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:23.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722944+0000", 2024-09-13T10:11:23.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021173210000000001 2024-09-13T10:11:23.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724274+0000", 2024-09-13T10:11:23.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013303099999999999 2024-09-13T10:11:23.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.735926+0000", 2024-09-13T10:11:23.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011651846 2024-09-13T10:11:23.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:23.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736061+0000", 2024-09-13T10:11:23.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013452000000000001 2024-09-13T10:11:23.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736205+0000", 2024-09-13T10:11:23.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014431700000000001 2024-09-13T10:11:23.711 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.711 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.711 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.711 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1576 4.7 4:e79f1505:::smithi19530714-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:11:23.711 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.717602+0000", 2024-09-13T10:11:23.711 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.877809407000001, 2024-09-13T10:11:23.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019134776999999999, 2024-09-13T10:11:23.711 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.711 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.711 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:23.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.712 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1576 2024-09-13T10:11:23.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.712 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717602+0000", 2024-09-13T10:11:23.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717602+0000", 2024-09-13T10:11:23.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717603+0000", 2024-09-13T10:11:23.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8899999999999999e-06 2024-09-13T10:11:23.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717610+0000", 2024-09-13T10:11:23.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8900000000000001e-06 2024-09-13T10:11:23.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717631+0000", 2024-09-13T10:11:23.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0741999999999999e-05 2024-09-13T10:11:23.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717644+0000", 2024-09-13T10:11:23.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2680000000000001e-05 2024-09-13T10:11:23.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722783+0000", 2024-09-13T10:11:23.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005139429 2024-09-13T10:11:23.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:23.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722912+0000", 2024-09-13T10:11:23.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012886399999999999 2024-09-13T10:11:23.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736301+0000", 2024-09-13T10:11:23.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013389082 2024-09-13T10:11:23.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736498+0000", 2024-09-13T10:11:23.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000197059 2024-09-13T10:11:23.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736736+0000", 2024-09-13T10:11:23.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023814100000000001 2024-09-13T10:11:23.716 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.716 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.716 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.717 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1580 4.17 4:e8c2cbec:::smithi19530714-7:head [write 618524~710869,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:11:23.717 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.724425+0000", 2024-09-13T10:11:23.717 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.870985665999999, 2024-09-13T10:11:23.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013470263999999999, 2024-09-13T10:11:23.717 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.717 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.717 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.717 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.717 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1580 2024-09-13T10:11:23.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.717 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724425+0000", 2024-09-13T10:11:23.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724425+0000", 2024-09-13T10:11:23.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724427+0000", 2024-09-13T10:11:23.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9530000000000002e-06 2024-09-13T10:11:23.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725799+0000", 2024-09-13T10:11:23.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001371466 2024-09-13T10:11:23.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725823+0000", 2024-09-13T10:11:23.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4332999999999999e-05 2024-09-13T10:11:23.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725844+0000", 2024-09-13T10:11:23.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0539000000000001e-05 2024-09-13T10:11:23.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725963+0000", 2024-09-13T10:11:23.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119524 2024-09-13T10:11:23.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.726194+0000", 2024-09-13T10:11:23.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023045800000000001 2024-09-13T10:11:23.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:23.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.726619+0000", 2024-09-13T10:11:23.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042517199999999997 2024-09-13T10:11:23.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:23.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.735852+0000", 2024-09-13T10:11:23.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0092335560000000004 2024-09-13T10:11:23.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.735959+0000", 2024-09-13T10:11:23.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106351 2024-09-13T10:11:23.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.737757+0000", 2024-09-13T10:11:23.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017985119999999999 2024-09-13T10:11:23.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:23.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.737838+0000", 2024-09-13T10:11:23.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0457999999999994e-05 2024-09-13T10:11:23.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.737896+0000", 2024-09-13T10:11:23.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7942000000000002e-05 2024-09-13T10:11:23.723 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.723 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.723 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.723 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1855 4.17 4:e81d9ad0:::smithi19530714-4:head [write 739541~709000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:23.723 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.758272+0000", 2024-09-13T10:11:23.723 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.837138962000001, 2024-09-13T10:11:23.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01875696, 2024-09-13T10:11:23.723 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.724 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.724 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.724 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.724 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1855 2024-09-13T10:11:23.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.724 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.758272+0000", 2024-09-13T10:11:23.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.758272+0000", 2024-09-13T10:11:23.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.758275+0000", 2024-09-13T10:11:23.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4429999999999999e-06 2024-09-13T10:11:23.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.759964+0000", 2024-09-13T10:11:23.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001688557 2024-09-13T10:11:23.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.759996+0000", 2024-09-13T10:11:23.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1705999999999998e-05 2024-09-13T10:11:23.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.760013+0000", 2024-09-13T10:11:23.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6846000000000001e-05 2024-09-13T10:11:23.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.760156+0000", 2024-09-13T10:11:23.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014331700000000001 2024-09-13T10:11:23.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-13T10:11:23.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.760167+0000", 2024-09-13T10:11:23.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.062e-05 2024-09-13T10:11:23.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.763490+0000", 2024-09-13T10:11:23.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033235309999999998 2024-09-13T10:11:23.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.763611+0000", 2024-09-13T10:11:23.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012099699999999999 2024-09-13T10:11:23.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:23.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.763830+0000", 2024-09-13T10:11:23.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000218519 2024-09-13T10:11:23.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:23.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773195+0000", 2024-09-13T10:11:23.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093654340000000006 2024-09-13T10:11:23.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.774701+0000", 2024-09-13T10:11:23.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00150588 2024-09-13T10:11:23.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.776817+0000", 2024-09-13T10:11:23.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021156180000000001 2024-09-13T10:11:23.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:23.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.776910+0000", 2024-09-13T10:11:23.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3194000000000003e-05 2024-09-13T10:11:23.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.777029+0000", 2024-09-13T10:11:23.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119298 2024-09-13T10:11:23.730 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.730 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.730 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.731 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1856 4.17 4:e81d9ad0:::smithi19530714-4:head [write 2121859~426376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:23.731 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.765194+0000", 2024-09-13T10:11:23.731 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.830217169000001, 2024-09-13T10:11:23.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013074073, 2024-09-13T10:11:23.731 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.731 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.731 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.731 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1856 2024-09-13T10:11:23.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.732 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765194+0000", 2024-09-13T10:11:23.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765194+0000", 2024-09-13T10:11:23.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765196+0000", 2024-09-13T10:11:23.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.317e-06 2024-09-13T10:11:23.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765559+0000", 2024-09-13T10:11:23.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036326199999999999 2024-09-13T10:11:23.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765581+0000", 2024-09-13T10:11:23.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1676e-05 2024-09-13T10:11:23.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765595+0000", 2024-09-13T10:11:23.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4025999999999999e-05 2024-09-13T10:11:23.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.767613+0000", 2024-09-13T10:11:23.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00201768 2024-09-13T10:11:23.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.767751+0000", 2024-09-13T10:11:23.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013809 2024-09-13T10:11:23.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:23.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.767983+0000", 2024-09-13T10:11:23.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023206499999999999 2024-09-13T10:11:23.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:23.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773203+0000", 2024-09-13T10:11:23.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052195599999999998 2024-09-13T10:11:23.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.777481+0000", 2024-09-13T10:11:23.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042782840000000003 2024-09-13T10:11:23.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.778120+0000", 2024-09-13T10:11:23.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063947299999999999 2024-09-13T10:11:23.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:23.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.778203+0000", 2024-09-13T10:11:23.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2857999999999998e-05 2024-09-13T10:11:23.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.778268+0000", 2024-09-13T10:11:23.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4782000000000004e-05 2024-09-13T10:11:23.737 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.737 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.737 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.737 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1857 4.17 4:e81d9ad0:::smithi19530714-4:head [setxattr _header (55) in=7b,truncate 2739541] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:23.737 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.765630+0000", 2024-09-13T10:11:23.737 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.829781462, 2024-09-13T10:11:23.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013989379999999999, 2024-09-13T10:11:23.738 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.738 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.738 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.738 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.738 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1857 2024-09-13T10:11:23.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.738 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765630+0000", 2024-09-13T10:11:23.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765630+0000", 2024-09-13T10:11:23.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765631+0000", 2024-09-13T10:11:23.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.64e-06 2024-09-13T10:11:23.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765641+0000", 2024-09-13T10:11:23.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0047000000000001e-05 2024-09-13T10:11:23.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765658+0000", 2024-09-13T10:11:23.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6909e-05 2024-09-13T10:11:23.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765673+0000", 2024-09-13T10:11:23.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4980000000000001e-05 2024-09-13T10:11:23.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.771667+0000", 2024-09-13T10:11:23.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059938209999999999 2024-09-13T10:11:23.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.771795+0000", 2024-09-13T10:11:23.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012816999999999999 2024-09-13T10:11:23.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:23.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.772028+0000", 2024-09-13T10:11:23.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023327800000000001 2024-09-13T10:11:23.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:23.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773657+0000", 2024-09-13T10:11:23.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016287039999999999 2024-09-13T10:11:23.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.777579+0000", 2024-09-13T10:11:23.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039218589999999998 2024-09-13T10:11:23.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779159+0000", 2024-09-13T10:11:23.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001580176 2024-09-13T10:11:23.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:23.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779343+0000", 2024-09-13T10:11:23.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000183769 2024-09-13T10:11:23.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779619+0000", 2024-09-13T10:11:23.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027602699999999998 2024-09-13T10:11:23.744 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.744 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.744 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.744 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1858 4.17 4:e81d9ad0:::smithi19530714-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:23.744 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.765692+0000", 2024-09-13T10:11:23.744 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.829719376, 2024-09-13T10:11:23.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014311129000000001, 2024-09-13T10:11:23.744 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.744 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.744 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:23.745 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.745 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.745 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.745 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1858 2024-09-13T10:11:23.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.745 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765692+0000", 2024-09-13T10:11:23.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765692+0000", 2024-09-13T10:11:23.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765693+0000", 2024-09-13T10:11:23.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-13T10:11:23.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765697+0000", 2024-09-13T10:11:23.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6770000000000003e-06 2024-09-13T10:11:23.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765707+0000", 2024-09-13T10:11:23.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0129000000000001e-05 2024-09-13T10:11:23.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765715+0000", 2024-09-13T10:11:23.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0469999999999994e-06 2024-09-13T10:11:23.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773032+0000", 2024-09-13T10:11:23.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073166680000000001 2024-09-13T10:11:23.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:23.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773169+0000", 2024-09-13T10:11:23.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136964 2024-09-13T10:11:23.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779726+0000", 2024-09-13T10:11:23.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065572499999999997 2024-09-13T10:11:23.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779851+0000", 2024-09-13T10:11:23.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012477500000000001 2024-09-13T10:11:23.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.780003+0000", 2024-09-13T10:11:23.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015165299999999999 2024-09-13T10:11:23.749 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.749 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.749 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.750 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2113 4.17 4:e81d9ad0:::smithi19530714-4:head [write 1410296~761389] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T10:11:23.750 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:15.709580+0000", 2024-09-13T10:11:23.750 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 7.885830994, 2024-09-13T10:11:23.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012946568, 2024-09-13T10:11:23.750 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.750 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.750 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:23.750 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.750 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.750 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.750 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2113 2024-09-13T10:11:23.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.751 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.709580+0000", 2024-09-13T10:11:23.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.709580+0000", 2024-09-13T10:11:23.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.709582+0000", 2024-09-13T10:11:23.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4530000000000001e-06 2024-09-13T10:11:23.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710399+0000", 2024-09-13T10:11:23.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081639799999999997 2024-09-13T10:11:23.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710428+0000", 2024-09-13T10:11:23.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8736000000000001e-05 2024-09-13T10:11:23.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710443+0000", 2024-09-13T10:11:23.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5216e-05 2024-09-13T10:11:23.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710572+0000", 2024-09-13T10:11:23.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012882799999999999 2024-09-13T10:11:23.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710808+0000", 2024-09-13T10:11:23.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023585500000000001 2024-09-13T10:11:23.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:23.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.711245+0000", 2024-09-13T10:11:23.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043798099999999999 2024-09-13T10:11:23.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:23.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.716561+0000", 2024-09-13T10:11:23.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053156760000000001 2024-09-13T10:11:23.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.718008+0000", 2024-09-13T10:11:23.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014466380000000001 2024-09-13T10:11:23.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:23.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.722191+0000", 2024-09-13T10:11:23.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041829229999999998 2024-09-13T10:11:23.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:23.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.722439+0000", 2024-09-13T10:11:23.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024823899999999998 2024-09-13T10:11:23.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.722527+0000", 2024-09-13T10:11:23.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7625000000000002e-05 2024-09-13T10:11:23.756 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.756 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.756 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.756 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2412 4.7 4:e55883d8:::smithi19530714-17:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e63)", 2024-09-13T10:11:23.756 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:22.233147+0000", 2024-09-13T10:11:23.757 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.362264028, 2024-09-13T10:11:23.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042005107999999999, 2024-09-13T10:11:23.757 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:23.757 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:23.757 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:23.757 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:23.757 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:23.757 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:23.757 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2412 2024-09-13T10:11:23.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.757 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:23.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:23.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233147+0000", 2024-09-13T10:11:23.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:23.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233147+0000", 2024-09-13T10:11:23.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:23.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:23.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233150+0000", 2024-09-13T10:11:23.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0029999999999999e-06 2024-09-13T10:11:23.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:23.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233162+0000", 2024-09-13T10:11:23.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2449000000000001e-05 2024-09-13T10:11:23.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:23.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233195+0000", 2024-09-13T10:11:23.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2455e-05 2024-09-13T10:11:23.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:23.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233213+0000", 2024-09-13T10:11:23.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8185999999999998e-05 2024-09-13T10:11:23.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:23.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233351+0000", 2024-09-13T10:11:23.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000137942 2024-09-13T10:11:23.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:23.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233497+0000", 2024-09-13T10:11:23.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014627100000000001 2024-09-13T10:11:23.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:23.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:23.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:23.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.275152+0000", 2024-09-13T10:11:23.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041654801999999998 2024-09-13T10:11:23.761 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.761 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.761 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.761 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:23.761 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:23.761 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:23.762 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:23.763 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:23.763 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:23.763 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:23.764 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:23.758+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:23.764 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:23.764 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:23.765 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T10:11:23.765 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:23.766 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:23.766 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:23.767 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:23.774 INFO:tasks.rados.rados.0.smithi195.stdout:1168: finishing write tid 1 to smithi19530714-25 2024-09-13T10:11:23.774 INFO:tasks.rados.rados.0.smithi195.stdout:1168: finishing write tid 2 to smithi19530714-25 2024-09-13T10:11:23.774 INFO:tasks.rados.rados.0.smithi195.stdout:1168: finishing write tid 3 to smithi19530714-25 2024-09-13T10:11:23.774 INFO:tasks.rados.rados.0.smithi195.stdout:1168: finishing write tid 4 to smithi19530714-25 2024-09-13T10:11:23.774 INFO:tasks.rados.rados.0.smithi195.stdout:1168: finishing write tid 5 to smithi19530714-25 2024-09-13T10:11:23.775 INFO:tasks.rados.rados.0.smithi195.stdout:1168: oid 25 updating version 0 to 115 2024-09-13T10:11:23.775 INFO:tasks.rados.rados.0.smithi195.stdout:1168: oid 25 updating version 115 to 118 2024-09-13T10:11:23.775 INFO:tasks.rados.rados.0.smithi195.stdout:1168: oid 25 version 118 is already newer than 116 2024-09-13T10:11:23.775 INFO:tasks.rados.rados.0.smithi195.stdout:1168: oid 25 version 118 is already newer than 117 2024-09-13T10:11:23.775 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 118 (ObjNum 215 snap 0 seq_num 215) dirty exists 2024-09-13T10:11:23.775 INFO:tasks.rados.rados.0.smithi195.stdout:1168: left oid 25 (ObjNum 215 snap 0 seq_num 215) 2024-09-13T10:11:23.775 INFO:tasks.rados.rados.0.smithi195.stdout:1163: done (7 left) 2024-09-13T10:11:23.775 INFO:tasks.rados.rados.0.smithi195.stdout:1164: done (6 left) 2024-09-13T10:11:23.775 INFO:tasks.rados.rados.0.smithi195.stdout:1165: done (5 left) 2024-09-13T10:11:23.775 INFO:tasks.rados.rados.0.smithi195.stdout:1166: done (4 left) 2024-09-13T10:11:23.775 INFO:tasks.rados.rados.0.smithi195.stdout:1167: done (3 left) 2024-09-13T10:11:23.775 INFO:tasks.rados.rados.0.smithi195.stdout:1168: done (2 left) 2024-09-13T10:11:23.775 INFO:tasks.rados.rados.0.smithi195.stdout:1169: done (1 left) 2024-09-13T10:11:23.776 INFO:tasks.rados.rados.0.smithi195.stdout:1170: read oid 25 snap -1 2024-09-13T10:11:23.776 INFO:tasks.rados.rados.0.smithi195.stdout:1170: expect (ObjNum 215 snap 0 seq_num 215) 2024-09-13T10:11:23.776 INFO:tasks.rados.rados.0.smithi195.stdout:1171: write oid 11 current snap is 0 2024-09-13T10:11:23.776 INFO:tasks.rados.rados.0.smithi195.stdout:1171: seq_num 217 ranges {447119=786917,1665416=692733,2976554=514615} 2024-09-13T10:11:23.776 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:23.776 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:23.777 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:11:23.777 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:11:23.777 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:11:23.777 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T10:11:23.777 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T10:11:23.793 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:23.793 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:23.794 INFO:tasks.rados.rados.0.smithi195.stdout:1171: writing smithi19530714-11 from 447119 to 1234036 tid 1 2024-09-13T10:11:23.806 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T10:11:23.811 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:23.811 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:23.813 INFO:tasks.rados.rados.0.smithi195.stdout:1171: writing smithi19530714-11 from 1665416 to 2358149 tid 2 2024-09-13T10:11:23.830 INFO:tasks.rados.rados.0.smithi195.stdout:1171: writing smithi19530714-11 from 2976554 to 3491169 tid 3 2024-09-13T10:11:23.830 INFO:tasks.rados.rados.0.smithi195.stdout:1172: tier_promote oid 5 2024-09-13T10:11:23.830 INFO:tasks.rados.rados.0.smithi195.stdout:1170: expect (ObjNum 215 snap 0 seq_num 215) 2024-09-13T10:11:23.831 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:23.832 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:23.856 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:23.856 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:23.863 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:23.860+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:23.878 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:23.878 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:23.896 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:23.896 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:23.914 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:23.915 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:23.937 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:23.937 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:23.948 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:23.949 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:23.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:23.952+0000 7f5dfacc9640 0 -- 172.21.15.73:0/3710144953 >> v2:172.21.15.195:6805/848686366 conn(0x7f5df4224b80 msgr2=0x7f5df4226f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:11:23.957 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:23.958 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:23.964 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T10:11:23.965 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:23.960+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:24.015 INFO:tasks.rados.rados.0.smithi195.stdout:1171: finishing write tid 1 to smithi19530714-11 2024-09-13T10:11:24.015 INFO:tasks.rados.rados.0.smithi195.stdout:1171: finishing write tid 2 to smithi19530714-11 2024-09-13T10:11:24.015 INFO:tasks.rados.rados.0.smithi195.stdout:1171: finishing write tid 3 to smithi19530714-11 2024-09-13T10:11:24.015 INFO:tasks.rados.rados.0.smithi195.stdout:1171: finishing write tid 4 to smithi19530714-11 2024-09-13T10:11:24.015 INFO:tasks.rados.rados.0.smithi195.stdout:1171: finishing write tid 5 to smithi19530714-11 2024-09-13T10:11:24.015 INFO:tasks.rados.rados.0.smithi195.stdout:1171: oid 11 updating version 0 to 72 2024-09-13T10:11:24.016 INFO:tasks.rados.rados.0.smithi195.stdout:1171: oid 11 version 72 is already newer than 69 2024-09-13T10:11:24.016 INFO:tasks.rados.rados.0.smithi195.stdout:1171: oid 11 version 72 is already newer than 70 2024-09-13T10:11:24.016 INFO:tasks.rados.rados.0.smithi195.stdout:1171: oid 11 version 72 is already newer than 71 2024-09-13T10:11:24.016 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 72 (ObjNum 217 snap 0 seq_num 217) dirty exists 2024-09-13T10:11:24.016 INFO:tasks.rados.rados.0.smithi195.stdout:1171: left oid 11 (ObjNum 217 snap 0 seq_num 217) 2024-09-13T10:11:24.016 INFO:tasks.rados.rados.0.smithi195.stdout:1172: got (0) Success 2024-09-13T10:11:24.016 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 8 (ObjNum 173 snap 0 seq_num 173) dirty exists 2024-09-13T10:11:24.016 INFO:tasks.rados.rados.0.smithi195.stdout:1162: got (0) Success 2024-09-13T10:11:24.016 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 49 (ObjNum 160 snap 0 seq_num 160) dirty exists 2024-09-13T10:11:24.016 INFO:tasks.rados.rados.0.smithi195.stdout:1162: done (3 left) 2024-09-13T10:11:24.016 INFO:tasks.rados.rados.0.smithi195.stdout:1170: done (2 left) 2024-09-13T10:11:24.016 INFO:tasks.rados.rados.0.smithi195.stdout:1171: done (1 left) 2024-09-13T10:11:24.016 INFO:tasks.rados.rados.0.smithi195.stdout:1172: done (0 left) 2024-09-13T10:11:24.017 INFO:tasks.rados.rados.0.smithi195.stdout:1173: read oid 13 snap -1 2024-09-13T10:11:24.017 INFO:tasks.rados.rados.0.smithi195.stdout:1173: expect (ObjNum 187 snap 0 seq_num 187) 2024-09-13T10:11:24.017 INFO:tasks.rados.rados.0.smithi195.stdout:1174: tier_evict oid 22 2024-09-13T10:11:24.017 INFO:tasks.rados.rados.0.smithi195.stdout:1174: tier_evict oid 22 snap -1 2024-09-13T10:11:24.017 INFO:tasks.rados.rados.0.smithi195.stdout:1175: set_chunk oid 2 target oid 2024-09-13T10:11:24.053 INFO:tasks.rados.rados.0.smithi195.stdout:1175: set_chunk oid 2 offset: 693760 length: 2111488 target oid 2e0fa59229e6ca6cea933c713e4b9dc6003a351abd6ff530a0ad51c4e65e5e5c offset: 0 snap -1 2024-09-13T10:11:24.053 INFO:tasks.rados.rados.0.smithi195.stdout:1176: write (excl) oid 7 current snap is 0 2024-09-13T10:11:24.053 INFO:tasks.rados.rados.0.smithi195.stdout:1174: got (22) Invalid argument 2024-09-13T10:11:24.053 INFO:tasks.rados.rados.0.smithi195.stdout:1173: expect (ObjNum 187 snap 0 seq_num 187) 2024-09-13T10:11:24.065 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:24.063+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:24.086 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:24.086 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:24.096 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:11:24.096 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:24.096 INFO:teuthology.orchestra.run.smithi073.stdout: "epoch": 65, 2024-09-13T10:11:24.096 INFO:teuthology.orchestra.run.smithi073.stdout: "fsid": "c1e91d47-9492-4461-a043-8698fd74ccf3", 2024-09-13T10:11:24.097 INFO:teuthology.orchestra.run.smithi073.stdout: "created": "2024-09-13T10:10:14.019912+0000", 2024-09-13T10:11:24.097 INFO:teuthology.orchestra.run.smithi073.stdout: "modified": "2024-09-13T10:11:23.691683+0000", 2024-09-13T10:11:24.097 INFO:teuthology.orchestra.run.smithi073.stdout: "last_up_change": "2024-09-13T10:10:28.089885+0000", 2024-09-13T10:11:24.097 INFO:teuthology.orchestra.run.smithi073.stdout: "last_in_change": "2024-09-13T10:10:18.489546+0000", 2024-09-13T10:11:24.097 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T10:11:24.098 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_num": 5806080, 2024-09-13T10:11:24.098 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_set": [ 2024-09-13T10:11:24.098 INFO:teuthology.orchestra.run.smithi073.stdout: "nodeep-scrub", 2024-09-13T10:11:24.098 INFO:teuthology.orchestra.run.smithi073.stdout: "noscrub", 2024-09-13T10:11:24.098 INFO:teuthology.orchestra.run.smithi073.stdout: "pglog_hardlimit", 2024-09-13T10:11:24.098 INFO:teuthology.orchestra.run.smithi073.stdout: "purged_snapdirs", 2024-09-13T10:11:24.098 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_deletes", 2024-09-13T10:11:24.098 INFO:teuthology.orchestra.run.smithi073.stdout: "sortbitwise" 2024-09-13T10:11:24.098 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:24.098 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_version": 6, 2024-09-13T10:11:24.098 INFO:teuthology.orchestra.run.smithi073.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T10:11:24.099 INFO:teuthology.orchestra.run.smithi073.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T10:11:24.099 INFO:teuthology.orchestra.run.smithi073.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T10:11:24.099 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_snapshot": "", 2024-09-13T10:11:24.099 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_max": 4, 2024-09-13T10:11:24.099 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd": 16, 2024-09-13T10:11:24.099 INFO:teuthology.orchestra.run.smithi073.stdout: "require_min_compat_client": "luminous", 2024-09-13T10:11:24.099 INFO:teuthology.orchestra.run.smithi073.stdout: "min_compat_client": "luminous", 2024-09-13T10:11:24.099 INFO:teuthology.orchestra.run.smithi073.stdout: "require_osd_release": "squid", 2024-09-13T10:11:24.099 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_crimson": false, 2024-09-13T10:11:24.099 INFO:teuthology.orchestra.run.smithi073.stdout: "pools": [ 2024-09-13T10:11:24.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.099 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 1, 2024-09-13T10:11:24.099 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "rbd", 2024-09-13T10:11:24.100 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:28.924860+0000", 2024-09-13T10:11:24.100 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-13T10:11:24.100 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T10:11:24.100 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:11:24.100 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:11:24.100 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:11:24.100 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:11:24.100 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:11:24.100 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:11:24.100 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:11:24.100 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:11:24.100 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:11:24.101 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:11:24.101 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:11:24.101 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 8, 2024-09-13T10:11:24.101 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 8, 2024-09-13T10:11:24.101 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 8, 2024-09-13T10:11:24.101 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 8, 2024-09-13T10:11:24.101 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 8, 2024-09-13T10:11:24.101 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:11:24.102 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:11:24.102 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:11:24.102 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:11:24.102 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:11:24.102 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:11:24.103 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:11:24.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.103 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "27", 2024-09-13T10:11:24.103 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:11:24.103 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:11:24.103 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:11:24.103 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:11:24.103 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:11:24.103 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 2, 2024-09-13T10:11:24.103 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 27, 2024-09-13T10:11:24.103 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:11:24.103 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:11:24.103 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:11:24.104 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:11:24.104 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:11:24.104 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:11:24.104 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:11:24.104 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:11:24.104 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:11:24.104 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:11:24.104 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:11:24.104 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:11:24.104 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:11:24.104 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:11:24.104 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:11:24.105 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:11:24.105 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:11:24.105 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:11:24.105 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:11:24.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.105 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:11:24.105 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:11:24.105 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:11:24.105 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:11:24.105 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:11:24.105 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:11:24.105 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:11:24.105 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:11:24.106 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:11:24.106 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:11:24.106 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:11:24.106 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-13T10:11:24.106 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-13T10:11:24.106 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-13T10:11:24.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.106 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:11:24.106 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:11:24.106 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 3.7699999809265137, 2024-09-13T10:11:24.106 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 3.7699999809265137, 2024-09-13T10:11:24.106 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.73000001907348633, 2024-09-13T10:11:24.106 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2.75, 2024-09-13T10:11:24.107 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2.75, 2024-09-13T10:11:24.107 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8997650146484375, 2024-09-13T10:11:24.107 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:11:24.107 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-13T10:11:24.107 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.107 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-13T10:11:24.107 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": ".mgr", 2024-09-13T10:11:24.107 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:28.972519+0000", 2024-09-13T10:11:24.107 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-13T10:11:24.107 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-13T10:11:24.107 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:11:24.108 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:11:24.108 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:11:24.108 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:11:24.108 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:11:24.108 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:11:24.108 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:11:24.108 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:11:24.108 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:11:24.108 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:11:24.108 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:11:24.108 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 1, 2024-09-13T10:11:24.108 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 1, 2024-09-13T10:11:24.109 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 1, 2024-09-13T10:11:24.109 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 1, 2024-09-13T10:11:24.109 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 1, 2024-09-13T10:11:24.109 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:11:24.109 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:11:24.109 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:11:24.109 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:11:24.109 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:11:24.109 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:11:24.109 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:11:24.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.110 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "26", 2024-09-13T10:11:24.110 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:11:24.110 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:11:24.110 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:11:24.110 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:11:24.110 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:11:24.110 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-13T10:11:24.110 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-13T10:11:24.110 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:11:24.110 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:11:24.110 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:11:24.111 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:11:24.111 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:11:24.111 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:11:24.111 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:11:24.111 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:11:24.111 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:11:24.111 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:11:24.111 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:11:24.111 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:11:24.111 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:11:24.111 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:11:24.111 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:11:24.111 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:11:24.111 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:11:24.112 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:11:24.112 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:11:24.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.112 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:11:24.112 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:11:24.112 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:11:24.112 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:11:24.112 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:11:24.112 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:11:24.112 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:11:24.112 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:11:24.112 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:11:24.113 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:11:24.113 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:11:24.113 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-13T10:11:24.113 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_max": 32, 2024-09-13T10:11:24.113 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_min": 1 2024-09-13T10:11:24.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.113 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-13T10:11:24.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": {} 2024-09-13T10:11:24.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.113 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:11:24.113 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:11:24.113 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 15, 2024-09-13T10:11:24.113 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 15, 2024-09-13T10:11:24.114 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.20000000298023224, 2024-09-13T10:11:24.114 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 3, 2024-09-13T10:11:24.114 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 3, 2024-09-13T10:11:24.114 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8997650146484375, 2024-09-13T10:11:24.114 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:11:24.114 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-13T10:11:24.114 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.114 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-13T10:11:24.114 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "low_tier", 2024-09-13T10:11:24.114 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:41.320719+0000", 2024-09-13T10:11:24.114 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-13T10:11:24.115 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-13T10:11:24.115 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:11:24.115 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:11:24.115 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:11:24.115 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:11:24.115 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:11:24.115 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:11:24.115 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:11:24.115 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:11:24.115 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:11:24.115 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:11:24.115 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:11:24.115 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 4, 2024-09-13T10:11:24.116 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 4, 2024-09-13T10:11:24.116 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 4, 2024-09-13T10:11:24.116 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 4, 2024-09-13T10:11:24.116 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 4, 2024-09-13T10:11:24.116 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:11:24.116 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:11:24.116 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:11:24.116 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:11:24.116 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:11:24.116 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:11:24.117 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:11:24.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.117 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "32", 2024-09-13T10:11:24.117 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:11:24.117 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:11:24.117 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:11:24.117 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:11:24.117 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:11:24.117 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-13T10:11:24.117 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-13T10:11:24.117 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:11:24.118 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:11:24.118 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:11:24.118 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:11:24.118 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:11:24.118 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:11:24.118 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:11:24.118 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:11:24.118 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:11:24.118 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:11:24.118 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:11:24.118 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:11:24.118 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:11:24.119 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:11:24.119 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:11:24.119 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:11:24.119 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:11:24.119 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:11:24.119 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:11:24.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.119 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:11:24.119 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:11:24.119 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:11:24.119 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:11:24.119 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:11:24.119 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:11:24.120 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:11:24.120 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:11:24.120 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:11:24.120 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:11:24.120 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:11:24.120 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-13T10:11:24.120 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": {}, 2024-09-13T10:11:24.120 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:11:24.120 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:11:24.120 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 3.75, 2024-09-13T10:11:24.120 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 3.75, 2024-09-13T10:11:24.120 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.60000002384185791, 2024-09-13T10:11:24.121 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2.25, 2024-09-13T10:11:24.121 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2.25, 2024-09-13T10:11:24.121 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8997650146484375, 2024-09-13T10:11:24.121 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:11:24.121 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-13T10:11:24.121 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.121 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 4, 2024-09-13T10:11:24.121 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "unique_pool_0", 2024-09-13T10:11:24.121 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:42.575395+0000", 2024-09-13T10:11:24.121 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-13T10:11:24.122 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-13T10:11:24.122 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:11:24.122 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:11:24.122 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:11:24.122 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:11:24.122 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:11:24.122 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:11:24.122 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:11:24.122 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:11:24.122 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:11:24.122 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:11:24.122 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:11:24.122 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 26, 2024-09-13T10:11:24.123 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 23, 2024-09-13T10:11:24.123 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 26, 2024-09-13T10:11:24.123 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 26, 2024-09-13T10:11:24.123 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 26, 2024-09-13T10:11:24.123 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:11:24.123 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:11:24.123 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:11:24.123 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:11:24.123 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:11:24.123 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:11:24.123 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:11:24.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.123 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "63", 2024-09-13T10:11:24.124 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:11:24.124 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:11:24.124 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "41", 2024-09-13T10:11:24.124 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:11:24.124 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:11:24.124 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-13T10:11:24.124 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-13T10:11:24.124 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:11:24.124 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:11:24.124 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:11:24.124 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:11:24.124 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:11:24.125 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:11:24.125 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:11:24.125 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:11:24.125 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:11:24.125 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:11:24.125 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:11:24.125 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:11:24.125 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:11:24.125 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:11:24.125 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:11:24.125 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:11:24.125 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:11:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:11:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:11:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:11:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:11:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:11:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:11:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:11:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:11:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:11:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:11:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:11:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:11:24.127 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:11:24.127 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-13T10:11:24.127 INFO:teuthology.orchestra.run.smithi073.stdout: "dedup_cdc_chunk_size": 131072, 2024-09-13T10:11:24.127 INFO:teuthology.orchestra.run.smithi073.stdout: "dedup_chunk_algorithm": "fastcdc", 2024-09-13T10:11:24.127 INFO:teuthology.orchestra.run.smithi073.stdout: "dedup_tier": 3, 2024-09-13T10:11:24.127 INFO:teuthology.orchestra.run.smithi073.stdout: "fingerprint_algorithm": "sha256" 2024-09-13T10:11:24.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.127 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-13T10:11:24.127 INFO:teuthology.orchestra.run.smithi073.stdout: "rados": {} 2024-09-13T10:11:24.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.127 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:11:24.127 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:11:24.127 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 4.0300002098083496, 2024-09-13T10:11:24.128 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 3.4500000476837158, 2024-09-13T10:11:24.128 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 1.0700000524520874, 2024-09-13T10:11:24.128 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 4.309999942779541, 2024-09-13T10:11:24.128 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 3.690000057220459, 2024-09-13T10:11:24.128 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8997650146484375, 2024-09-13T10:11:24.128 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:11:24.128 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-13T10:11:24.128 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.128 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.128 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:24.128 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:24.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.128 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-13T10:11:24.129 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "f7d92188-c27c-4744-a330-4b7105e0f2d6", 2024-09-13T10:11:24.129 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:24.129 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:24.129 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:24.129 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:24.129 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:24.129 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:24.129 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:24.129 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 0, 2024-09-13T10:11:24.129 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:24.129 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:24.129 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:24.130 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.130 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.130 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6812", 2024-09-13T10:11:24.130 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:11:24.130 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.130 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.130 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:24.130 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.130 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.130 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6813", 2024-09-13T10:11:24.131 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:11:24.131 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.131 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.131 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:24.131 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.131 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.131 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6815", 2024-09-13T10:11:24.131 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:11:24.131 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.131 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.132 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:24.132 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.132 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.132 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6814", 2024-09-13T10:11:24.132 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:11:24.132 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.132 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.132 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6812/1977392321", 2024-09-13T10:11:24.132 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6813/1977392321", 2024-09-13T10:11:24.132 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6815/1977392321", 2024-09-13T10:11:24.133 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6814/1977392321", 2024-09-13T10:11:24.133 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:24.133 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:24.133 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:24.133 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.133 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-13T10:11:24.133 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e900b771-7c96-4b5d-93d7-35a8ea6c026a", 2024-09-13T10:11:24.133 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:24.133 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:24.133 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:24.133 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:24.134 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:24.134 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:24.134 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:24.134 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 64, 2024-09-13T10:11:24.134 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:24.134 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:24.134 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:24.134 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.134 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.134 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6808", 2024-09-13T10:11:24.134 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:11:24.134 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.135 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.135 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:24.135 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.135 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.135 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6809", 2024-09-13T10:11:24.135 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:11:24.135 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.135 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.135 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:24.135 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.136 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.136 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6811", 2024-09-13T10:11:24.136 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:11:24.136 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.136 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.136 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:24.136 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.136 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.136 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6810", 2024-09-13T10:11:24.136 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:11:24.137 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.137 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.137 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6808/1228102864", 2024-09-13T10:11:24.137 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6809/1228102864", 2024-09-13T10:11:24.137 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6811/1228102864", 2024-09-13T10:11:24.137 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6810/1228102864", 2024-09-13T10:11:24.137 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:24.137 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:24.137 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:24.137 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.138 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-13T10:11:24.138 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "28b7aa59-33b2-4e24-8f3d-758f90e25f68", 2024-09-13T10:11:24.138 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:24.138 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:24.139 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:24.139 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:24.139 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:24.139 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:24.139 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:24.139 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 61, 2024-09-13T10:11:24.139 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:24.140 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:24.140 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:24.140 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.140 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.140 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6800", 2024-09-13T10:11:24.140 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:11:24.140 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.141 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.141 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:24.141 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.141 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.141 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6801", 2024-09-13T10:11:24.141 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:11:24.141 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.141 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.142 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:24.142 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.142 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.142 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6803", 2024-09-13T10:11:24.142 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:11:24.142 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.142 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.142 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:24.142 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.143 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.143 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6802", 2024-09-13T10:11:24.143 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:11:24.143 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.143 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.143 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6800/3314820936", 2024-09-13T10:11:24.143 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6801/3314820936", 2024-09-13T10:11:24.143 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6803/3314820936", 2024-09-13T10:11:24.143 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6802/3314820936", 2024-09-13T10:11:24.143 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:24.143 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:24.143 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:24.144 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-13T10:11:24.144 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "a93d2df5-781e-49e1-a507-1718b6f954ed", 2024-09-13T10:11:24.144 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:24.144 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:24.144 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:24.144 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:24.144 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:24.144 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:24.144 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:24.144 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 60, 2024-09-13T10:11:24.145 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:24.145 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:24.145 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:24.145 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.145 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.145 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6801", 2024-09-13T10:11:24.145 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:11:24.145 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.145 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.145 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:24.145 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.146 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.146 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6802", 2024-09-13T10:11:24.146 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:11:24.146 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.146 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.146 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:24.146 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.146 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.146 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6804", 2024-09-13T10:11:24.147 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:11:24.147 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.147 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.147 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:24.147 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.147 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.147 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6803", 2024-09-13T10:11:24.147 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:11:24.147 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.147 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.148 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6801/2558565460", 2024-09-13T10:11:24.148 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6802/2558565460", 2024-09-13T10:11:24.148 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6804/2558565460", 2024-09-13T10:11:24.148 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6803/2558565460", 2024-09-13T10:11:24.148 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:24.148 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:24.148 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:24.148 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-13T10:11:24.148 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e69053fd-f6d9-437e-a9d0-8896de6b927f", 2024-09-13T10:11:24.148 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:24.149 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:24.149 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:24.149 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:24.149 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:24.149 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:24.149 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:24.149 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 58, 2024-09-13T10:11:24.149 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:24.149 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:24.149 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:24.149 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.150 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.150 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6800", 2024-09-13T10:11:24.150 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:11:24.150 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.150 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.150 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:24.150 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.150 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.150 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6801", 2024-09-13T10:11:24.150 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:11:24.150 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.151 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.151 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:24.151 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.151 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.151 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6803", 2024-09-13T10:11:24.151 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:11:24.151 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.151 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.151 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:24.151 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.152 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.152 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6802", 2024-09-13T10:11:24.152 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:11:24.152 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.152 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.152 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6800/858330493", 2024-09-13T10:11:24.152 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6801/858330493", 2024-09-13T10:11:24.152 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6803/858330493", 2024-09-13T10:11:24.152 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6802/858330493", 2024-09-13T10:11:24.152 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:24.153 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:24.153 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:24.153 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-13T10:11:24.153 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "3667ec10-561e-4a23-8197-6357db4af315", 2024-09-13T10:11:24.153 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:24.153 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:24.153 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:24.153 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:24.153 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:24.154 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:24.154 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:24.154 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 58, 2024-09-13T10:11:24.154 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:24.154 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:24.154 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:24.154 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.154 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.154 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6804", 2024-09-13T10:11:24.154 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:11:24.154 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.155 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.155 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:24.155 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.155 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.155 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6805", 2024-09-13T10:11:24.155 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:11:24.155 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.155 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.155 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:24.156 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.156 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.156 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6807", 2024-09-13T10:11:24.156 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:11:24.156 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.156 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.156 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:24.156 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.156 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.157 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6806", 2024-09-13T10:11:24.157 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:11:24.157 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.157 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.157 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6804/984313481", 2024-09-13T10:11:24.157 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6805/984313481", 2024-09-13T10:11:24.157 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6807/984313481", 2024-09-13T10:11:24.157 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6806/984313481", 2024-09-13T10:11:24.157 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:24.157 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:24.157 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:24.158 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.158 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-13T10:11:24.158 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "30d4d63c-f04f-4423-91a6-816a7cbc0287", 2024-09-13T10:11:24.158 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:24.158 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:24.158 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:24.158 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:24.158 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:24.158 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:24.159 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:24.159 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 64, 2024-09-13T10:11:24.159 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:24.159 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:24.159 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:24.159 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.159 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.159 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6812", 2024-09-13T10:11:24.159 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:11:24.160 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.160 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.160 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:24.160 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.160 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.160 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6813", 2024-09-13T10:11:24.160 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:11:24.160 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.160 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.161 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:24.161 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.161 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.161 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6815", 2024-09-13T10:11:24.161 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:11:24.161 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.161 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.161 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:24.161 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.161 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.162 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6814", 2024-09-13T10:11:24.162 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:11:24.162 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.162 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.162 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6812/1505939978", 2024-09-13T10:11:24.162 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6813/1505939978", 2024-09-13T10:11:24.162 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6815/1505939978", 2024-09-13T10:11:24.162 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6814/1505939978", 2024-09-13T10:11:24.162 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:24.162 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:24.162 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:24.162 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.163 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-13T10:11:24.163 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "48812f05-82f1-4263-8ec5-4861a6e91b61", 2024-09-13T10:11:24.163 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:24.163 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:24.163 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:24.163 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:24.163 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:24.163 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:24.163 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:24.163 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 41, 2024-09-13T10:11:24.164 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:24.164 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:24.164 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:24.164 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.164 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.164 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6809", 2024-09-13T10:11:24.164 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:11:24.164 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.164 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.164 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:24.165 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.165 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.165 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6810", 2024-09-13T10:11:24.165 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:11:24.165 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.165 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.165 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:24.165 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.165 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.165 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6812", 2024-09-13T10:11:24.166 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:11:24.166 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.166 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.166 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:24.166 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.166 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.166 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6811", 2024-09-13T10:11:24.166 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:11:24.166 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.166 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.167 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6809/3679292757", 2024-09-13T10:11:24.167 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6810/3679292757", 2024-09-13T10:11:24.167 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6812/3679292757", 2024-09-13T10:11:24.167 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6811/3679292757", 2024-09-13T10:11:24.167 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:24.167 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:24.167 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:24.167 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.167 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 8, 2024-09-13T10:11:24.168 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "72056e61-d04b-4fd3-94bf-57ee3df4b466", 2024-09-13T10:11:24.168 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:24.168 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:24.168 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:24.168 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0.396240234375, 2024-09-13T10:11:24.168 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:24.168 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:24.168 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:24.168 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 31, 2024-09-13T10:11:24.168 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:24.168 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:24.168 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:24.168 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.169 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.169 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6804", 2024-09-13T10:11:24.169 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:11:24.169 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.169 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.169 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:24.169 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.169 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.169 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6805", 2024-09-13T10:11:24.170 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:11:24.170 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.170 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.170 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:24.170 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.170 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.170 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6807", 2024-09-13T10:11:24.170 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:11:24.170 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.170 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.171 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:24.171 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.171 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.171 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6806", 2024-09-13T10:11:24.171 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:11:24.171 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.171 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.171 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6804/4052106885", 2024-09-13T10:11:24.171 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6805/4052106885", 2024-09-13T10:11:24.171 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6807/4052106885", 2024-09-13T10:11:24.172 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6806/4052106885", 2024-09-13T10:11:24.172 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:24.172 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:24.172 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:24.172 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.172 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 9, 2024-09-13T10:11:24.172 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "20e7f118-7148-4c27-9ec4-a33954709515", 2024-09-13T10:11:24.172 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:24.172 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:24.172 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:24.172 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:24.173 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:24.173 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:24.173 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 21, 2024-09-13T10:11:24.173 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 48, 2024-09-13T10:11:24.173 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:24.173 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:24.173 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:24.173 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.173 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.173 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6800", 2024-09-13T10:11:24.173 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:11:24.174 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.174 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.174 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:24.174 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.174 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.174 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6801", 2024-09-13T10:11:24.174 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:11:24.174 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.174 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.174 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:24.175 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.175 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.175 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6803", 2024-09-13T10:11:24.175 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:11:24.175 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.175 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.175 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:24.175 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.175 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.175 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6802", 2024-09-13T10:11:24.176 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:11:24.176 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.176 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.176 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6800/4266571820", 2024-09-13T10:11:24.176 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6801/4266571820", 2024-09-13T10:11:24.176 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6803/4266571820", 2024-09-13T10:11:24.176 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6802/4266571820", 2024-09-13T10:11:24.177 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:24.177 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:24.177 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:24.177 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.177 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 10, 2024-09-13T10:11:24.177 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "012409e8-51c2-4584-aba7-95996f0d72ad", 2024-09-13T10:11:24.177 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:24.177 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:24.177 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:24.177 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:24.177 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:24.177 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:24.178 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:24.178 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 30, 2024-09-13T10:11:24.178 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:24.178 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:24.178 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:24.178 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.178 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.178 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6808", 2024-09-13T10:11:24.178 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:11:24.178 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.178 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.178 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:24.179 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.179 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.179 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6809", 2024-09-13T10:11:24.179 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:11:24.179 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.179 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.179 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:24.179 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.179 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.179 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6811", 2024-09-13T10:11:24.179 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:11:24.180 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.180 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.180 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:24.180 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.180 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.180 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6810", 2024-09-13T10:11:24.180 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:11:24.180 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.180 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.181 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6808/958255259", 2024-09-13T10:11:24.181 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6809/958255259", 2024-09-13T10:11:24.181 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6811/958255259", 2024-09-13T10:11:24.181 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6810/958255259", 2024-09-13T10:11:24.181 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:24.181 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:24.181 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:24.181 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.181 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 11, 2024-09-13T10:11:24.181 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "88b137b1-7d30-45f7-ac5c-ba026704ef24", 2024-09-13T10:11:24.182 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:24.182 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:24.182 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:24.182 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:24.182 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:24.182 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:24.182 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:24.182 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 57, 2024-09-13T10:11:24.182 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:24.182 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:24.182 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:24.182 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.182 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.182 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6813", 2024-09-13T10:11:24.183 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:11:24.183 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.183 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.183 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:24.183 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.183 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.183 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6814", 2024-09-13T10:11:24.183 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:11:24.183 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.183 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.183 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:24.184 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.184 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.184 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6816", 2024-09-13T10:11:24.184 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:11:24.184 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.184 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.184 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:24.184 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.184 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.184 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6815", 2024-09-13T10:11:24.185 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:11:24.185 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.185 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.185 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6813/281884267", 2024-09-13T10:11:24.185 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6814/281884267", 2024-09-13T10:11:24.185 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6816/281884267", 2024-09-13T10:11:24.185 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6815/281884267", 2024-09-13T10:11:24.185 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:24.185 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:24.185 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:24.185 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.186 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 12, 2024-09-13T10:11:24.186 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "b54732a2-9dff-42e1-b4ef-281cbca3ad7c", 2024-09-13T10:11:24.186 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:24.186 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:24.186 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:24.186 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0, 2024-09-13T10:11:24.186 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:24.186 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:24.186 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:24.186 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 31, 2024-09-13T10:11:24.186 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:24.187 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:24.187 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:24.187 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.187 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.187 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6808", 2024-09-13T10:11:24.187 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:11:24.187 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.187 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.187 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:24.187 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6809", 2024-09-13T10:11:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:11:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6811", 2024-09-13T10:11:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:11:24.188 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6810", 2024-09-13T10:11:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:11:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6808/3532043806", 2024-09-13T10:11:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6809/3532043806", 2024-09-13T10:11:24.189 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6811/3532043806", 2024-09-13T10:11:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6810/3532043806", 2024-09-13T10:11:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 13, 2024-09-13T10:11:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "20b48864-44dc-4732-8dff-d0d21cc08032", 2024-09-13T10:11:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:24.190 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 21, 2024-09-13T10:11:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 53, 2024-09-13T10:11:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6812", 2024-09-13T10:11:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:11:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6813", 2024-09-13T10:11:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:11:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.192 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6815", 2024-09-13T10:11:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:11:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6814", 2024-09-13T10:11:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:11:24.193 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.194 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.194 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6812/4083982131", 2024-09-13T10:11:24.194 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6813/4083982131", 2024-09-13T10:11:24.194 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6815/4083982131", 2024-09-13T10:11:24.194 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6814/4083982131", 2024-09-13T10:11:24.194 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:24.194 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:24.194 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:24.194 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.194 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 14, 2024-09-13T10:11:24.195 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "9fac3ef4-e9be-4e8b-9a21-0de35b543d65", 2024-09-13T10:11:24.195 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:24.195 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:24.195 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:24.195 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:24.195 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:24.195 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:24.195 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:24.195 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 63, 2024-09-13T10:11:24.195 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:24.195 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:24.195 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:24.195 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.196 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.196 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6804", 2024-09-13T10:11:24.196 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:11:24.196 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.196 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.196 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:24.196 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.196 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.196 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6805", 2024-09-13T10:11:24.197 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:11:24.197 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.197 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.197 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:24.197 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.197 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.197 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6807", 2024-09-13T10:11:24.197 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:11:24.197 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.197 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.198 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:24.198 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.198 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.198 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6806", 2024-09-13T10:11:24.198 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:11:24.198 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.198 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.198 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6804/1173379175", 2024-09-13T10:11:24.198 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6805/1173379175", 2024-09-13T10:11:24.198 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6807/1173379175", 2024-09-13T10:11:24.198 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6806/1173379175", 2024-09-13T10:11:24.199 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:24.199 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:24.199 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:24.199 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.199 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 15, 2024-09-13T10:11:24.199 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "16b34f90-3148-4d41-a403-ba0f900991e1", 2024-09-13T10:11:24.199 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:24.199 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:24.199 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:24.199 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:24.199 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:24.199 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:24.200 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:24.200 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 44, 2024-09-13T10:11:24.200 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:24.200 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:24.200 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:24.200 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.200 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.200 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6805", 2024-09-13T10:11:24.200 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:11:24.200 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.200 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.200 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:24.201 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.201 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.201 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6806", 2024-09-13T10:11:24.201 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:11:24.201 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.201 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.201 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:24.201 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.202 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.202 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6808", 2024-09-13T10:11:24.202 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:11:24.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.202 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.202 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:24.202 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:24.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.202 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:24.202 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6807", 2024-09-13T10:11:24.202 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:11:24.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.203 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.203 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6805/848686366", 2024-09-13T10:11:24.203 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6806/848686366", 2024-09-13T10:11:24.203 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6808/848686366", 2024-09-13T10:11:24.203 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6807/848686366", 2024-09-13T10:11:24.203 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:24.203 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:24.203 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:24.204 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.204 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.204 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:24.204 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_xinfo": [ 2024-09-13T10:11:24.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.204 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-13T10:11:24.204 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:24.204 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:24.204 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:24.204 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:24.204 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:24.205 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.372908+0000", 2024-09-13T10:11:24.205 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:24.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.205 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-13T10:11:24.205 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:24.205 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:24.205 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:24.205 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:24.205 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:24.206 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.641263+0000", 2024-09-13T10:11:24.206 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:24.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.206 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-13T10:11:24.206 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:24.206 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:24.206 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:24.206 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:24.206 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:24.206 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.237394+0000", 2024-09-13T10:11:24.207 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:24.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.207 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-13T10:11:24.207 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:24.207 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:24.207 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:24.207 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:24.207 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:24.207 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:24.991965+0000", 2024-09-13T10:11:24.207 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:24.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.208 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-13T10:11:24.208 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:24.208 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:24.208 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:24.208 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:24.208 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:24.208 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.066350+0000", 2024-09-13T10:11:24.208 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:24.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.209 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-13T10:11:24.209 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:24.209 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:24.209 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:24.209 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:24.209 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:24.209 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.003317+0000", 2024-09-13T10:11:24.209 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:24.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.209 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-13T10:11:24.209 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:24.209 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:24.210 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:24.210 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:24.210 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:24.210 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.353521+0000", 2024-09-13T10:11:24.210 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:24.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.210 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-13T10:11:24.210 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:24.210 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:24.210 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:24.210 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:24.211 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:24.211 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.263941+0000", 2024-09-13T10:11:24.211 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:24.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.211 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 8, 2024-09-13T10:11:24.211 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:24.211 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:24.211 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:24.211 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:24.211 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:24.212 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:24.922976+0000", 2024-09-13T10:11:24.212 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:24.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.212 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 9, 2024-09-13T10:11:24.212 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:24.212 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:24.212 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:24.212 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:24.212 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:24.212 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:24.743520+0000", 2024-09-13T10:11:24.213 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:24.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.213 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 10, 2024-09-13T10:11:24.213 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:24.213 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:24.213 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:24.213 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:24.213 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:24.213 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.407426+0000", 2024-09-13T10:11:24.213 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:24.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.214 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 11, 2024-09-13T10:11:24.214 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:24.214 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:24.214 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:24.214 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:24.214 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:24.214 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.542266+0000", 2024-09-13T10:11:24.214 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:24.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.214 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 12, 2024-09-13T10:11:24.215 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:24.215 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:24.215 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:24.215 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:24.215 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:24.215 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.457347+0000", 2024-09-13T10:11:24.215 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:24.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.215 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 13, 2024-09-13T10:11:24.215 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:24.215 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:24.216 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:24.216 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:24.216 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:24.216 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:24.923863+0000", 2024-09-13T10:11:24.216 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:24.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.216 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 14, 2024-09-13T10:11:24.216 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:24.216 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:24.216 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:24.216 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:24.217 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:24.217 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:24.980077+0000", 2024-09-13T10:11:24.217 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:24.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.217 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 15, 2024-09-13T10:11:24.217 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:24.217 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:24.217 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:24.217 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:24.217 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:24.217 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.070162+0000", 2024-09-13T10:11:24.217 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:24.218 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.218 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:24.218 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap": [ 2024-09-13T10:11:24.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.218 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.e", 2024-09-13T10:11:24.218 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:24.218 INFO:teuthology.orchestra.run.smithi073.stdout: 8, 2024-09-13T10:11:24.218 INFO:teuthology.orchestra.run.smithi073.stdout: 3, 2024-09-13T10:11:24.218 INFO:teuthology.orchestra.run.smithi073.stdout: 15 2024-09-13T10:11:24.218 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.218 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.218 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:24.218 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_items": [], 2024-09-13T10:11:24.219 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_primaries": [], 2024-09-13T10:11:24.219 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_temp": [ 2024-09-13T10:11:24.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.219 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.e", 2024-09-13T10:11:24.219 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:24.219 INFO:teuthology.orchestra.run.smithi073.stdout: 2, 2024-09-13T10:11:24.219 INFO:teuthology.orchestra.run.smithi073.stdout: 8, 2024-09-13T10:11:24.219 INFO:teuthology.orchestra.run.smithi073.stdout: 4 2024-09-13T10:11:24.219 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.219 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.10", 2024-09-13T10:11:24.219 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:24.219 INFO:teuthology.orchestra.run.smithi073.stdout: 4, 2024-09-13T10:11:24.220 INFO:teuthology.orchestra.run.smithi073.stdout: 5, 2024-09-13T10:11:24.220 INFO:teuthology.orchestra.run.smithi073.stdout: 6 2024-09-13T10:11:24.220 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.220 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.11", 2024-09-13T10:11:24.220 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:24.220 INFO:teuthology.orchestra.run.smithi073.stdout: 2, 2024-09-13T10:11:24.220 INFO:teuthology.orchestra.run.smithi073.stdout: 12, 2024-09-13T10:11:24.220 INFO:teuthology.orchestra.run.smithi073.stdout: 8 2024-09-13T10:11:24.220 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.221 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.12", 2024-09-13T10:11:24.221 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:24.221 INFO:teuthology.orchestra.run.smithi073.stdout: 6, 2024-09-13T10:11:24.221 INFO:teuthology.orchestra.run.smithi073.stdout: 8, 2024-09-13T10:11:24.221 INFO:teuthology.orchestra.run.smithi073.stdout: 14 2024-09-13T10:11:24.221 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.221 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.14", 2024-09-13T10:11:24.221 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:24.221 INFO:teuthology.orchestra.run.smithi073.stdout: 5, 2024-09-13T10:11:24.221 INFO:teuthology.orchestra.run.smithi073.stdout: 6, 2024-09-13T10:11:24.221 INFO:teuthology.orchestra.run.smithi073.stdout: 10 2024-09-13T10:11:24.222 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:24.222 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.16", 2024-09-13T10:11:24.222 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:24.222 INFO:teuthology.orchestra.run.smithi073.stdout: 1, 2024-09-13T10:11:24.222 INFO:teuthology.orchestra.run.smithi073.stdout: 13, 2024-09-13T10:11:24.222 INFO:teuthology.orchestra.run.smithi073.stdout: 7 2024-09-13T10:11:24.222 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:24.222 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.222 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:24.222 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_temp": [], 2024-09-13T10:11:24.222 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklist": {}, 2024-09-13T10:11:24.223 INFO:teuthology.orchestra.run.smithi073.stdout: "range_blocklist": {}, 2024-09-13T10:11:24.223 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profiles": { 2024-09-13T10:11:24.223 INFO:teuthology.orchestra.run.smithi073.stdout: "default": { 2024-09-13T10:11:24.223 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-failure-domain": "osd", 2024-09-13T10:11:24.223 INFO:teuthology.orchestra.run.smithi073.stdout: "k": "2", 2024-09-13T10:11:24.223 INFO:teuthology.orchestra.run.smithi073.stdout: "m": "1", 2024-09-13T10:11:24.223 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin": "jerasure", 2024-09-13T10:11:24.223 INFO:teuthology.orchestra.run.smithi073.stdout: "technique": "reed_sol_van" 2024-09-13T10:11:24.223 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:24.223 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps_queue": [], 2024-09-13T10:11:24.223 INFO:teuthology.orchestra.run.smithi073.stdout: "new_removed_snaps": [], 2024-09-13T10:11:24.223 INFO:teuthology.orchestra.run.smithi073.stdout: "new_purged_snaps": [], 2024-09-13T10:11:24.223 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_node_flags": {}, 2024-09-13T10:11:24.223 INFO:teuthology.orchestra.run.smithi073.stdout: "device_class_flags": {}, 2024-09-13T10:11:24.224 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode": { 2024-09-13T10:11:24.224 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_enabled": false, 2024-09-13T10:11:24.224 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_bucket_count": 0, 2024-09-13T10:11:24.224 INFO:teuthology.orchestra.run.smithi073.stdout: "degraded_stretch_mode": 0, 2024-09-13T10:11:24.224 INFO:teuthology.orchestra.run.smithi073.stdout: "recovering_stretch_mode": 0, 2024-09-13T10:11:24.224 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_bucket": 0 2024-09-13T10:11:24.224 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:24.224 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:24.225 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:24.227 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T10:11:24.227 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:24.220+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:24.228 INFO:tasks.thrashosds.thrasher:j is {'epoch': 65, 'fsid': 'c1e91d47-9492-4461-a043-8698fd74ccf3', 'created': '2024-09-13T10:10:14.019912+0000', 'modified': '2024-09-13T10:11:23.691683+0000', 'last_up_change': '2024-09-13T10:10:28.089885+0000', 'last_in_change': '2024-09-13T10:10:18.489546+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T10:10:28.924860+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 27, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.7699999809265137, 'score_stable': 3.7699999809265137, 'optimal_score': 0.7300000190734863, 'raw_score_acting': 2.75, 'raw_score_stable': 2.75, 'primary_affinity_weighted': 0.8997650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T10:10:28.972519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 15, 'score_stable': 15, 'optimal_score': 0.20000000298023224, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.8997650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 3, 'pool_name': 'low_tier', 'create_time': '2024-09-13T10:10:41.320719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.75, 'score_stable': 3.75, 'optimal_score': 0.6000000238418579, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8997650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 4, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-13T10:10:42.575395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 23, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '41', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'dedup_cdc_chunk_size': 131072, 'dedup_chunk_algorithm': 'fastcdc', 'dedup_tier': 3, 'fingerprint_algorithm': 'sha256'}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.03000020980835, 'score_stable': 3.450000047683716, 'optimal_score': 1.0700000524520874, 'raw_score_acting': 4.309999942779541, 'raw_score_stable': 3.690000057220459, 'primary_affinity_weighted': 0.8997650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8999999761581421}}], 'osds': [{'osd': 0, 'uuid': 'f7d92188-c27c-4744-a330-4b7105e0f2d6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6812', 'nonce': 1977392321}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6813', 'nonce': 1977392321}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6815', 'nonce': 1977392321}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6814', 'nonce': 1977392321}]}, 'public_addr': '172.21.15.73:6812/1977392321', 'cluster_addr': '172.21.15.73:6813/1977392321', 'heartbeat_back_addr': '172.21.15.73:6815/1977392321', 'heartbeat_front_addr': '172.21.15.73:6814/1977392321', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e900b771-7c96-4b5d-93d7-35a8ea6c026a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 64, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6808', 'nonce': 1228102864}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6809', 'nonce': 1228102864}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6811', 'nonce': 1228102864}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6810', 'nonce': 1228102864}]}, 'public_addr': '172.21.15.136:6808/1228102864', 'cluster_addr': '172.21.15.136:6809/1228102864', 'heartbeat_back_addr': '172.21.15.136:6811/1228102864', 'heartbeat_front_addr': '172.21.15.136:6810/1228102864', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '28b7aa59-33b2-4e24-8f3d-758f90e25f68', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6800', 'nonce': 3314820936}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6801', 'nonce': 3314820936}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6803', 'nonce': 3314820936}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6802', 'nonce': 3314820936}]}, 'public_addr': '172.21.15.181:6800/3314820936', 'cluster_addr': '172.21.15.181:6801/3314820936', 'heartbeat_back_addr': '172.21.15.181:6803/3314820936', 'heartbeat_front_addr': '172.21.15.181:6802/3314820936', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a93d2df5-781e-49e1-a507-1718b6f954ed', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6801', 'nonce': 2558565460}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6802', 'nonce': 2558565460}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6804', 'nonce': 2558565460}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6803', 'nonce': 2558565460}]}, 'public_addr': '172.21.15.195:6801/2558565460', 'cluster_addr': '172.21.15.195:6802/2558565460', 'heartbeat_back_addr': '172.21.15.195:6804/2558565460', 'heartbeat_front_addr': '172.21.15.195:6803/2558565460', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e69053fd-f6d9-437e-a9d0-8896de6b927f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6800', 'nonce': 858330493}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6801', 'nonce': 858330493}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6803', 'nonce': 858330493}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6802', 'nonce': 858330493}]}, 'public_addr': '172.21.15.73:6800/858330493', 'cluster_addr': '172.21.15.73:6801/858330493', 'heartbeat_back_addr': '172.21.15.73:6803/858330493', 'heartbeat_front_addr': '172.21.15.73:6802/858330493', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '3667ec10-561e-4a23-8197-6357db4af315', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6804', 'nonce': 984313481}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6805', 'nonce': 984313481}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6807', 'nonce': 984313481}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6806', 'nonce': 984313481}]}, 'public_addr': '172.21.15.136:6804/984313481', 'cluster_addr': '172.21.15.136:6805/984313481', 'heartbeat_back_addr': '172.21.15.136:6807/984313481', 'heartbeat_front_addr': '172.21.15.136:6806/984313481', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '30d4d63c-f04f-4423-91a6-816a7cbc0287', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 64, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6812', 'nonce': 1505939978}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6813', 'nonce': 1505939978}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6815', 'nonce': 1505939978}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6814', 'nonce': 1505939978}]}, 'public_addr': '172.21.15.181:6812/1505939978', 'cluster_addr': '172.21.15.181:6813/1505939978', 'heartbeat_back_addr': '172.21.15.181:6815/1505939978', 'heartbeat_front_addr': '172.21.15.181:6814/1505939978', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '48812f05-82f1-4263-8ec5-4861a6e91b61', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6809', 'nonce': 3679292757}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6810', 'nonce': 3679292757}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6812', 'nonce': 3679292757}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6811', 'nonce': 3679292757}]}, 'public_addr': '172.21.15.195:6809/3679292757', 'cluster_addr': '172.21.15.195:6810/3679292757', 'heartbeat_back_addr': '172.21.15.195:6812/3679292757', 'heartbeat_front_addr': '172.21.15.195:6811/3679292757', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '72056e61-d04b-4fd3-94bf-57ee3df4b466', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.396240234375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6804', 'nonce': 4052106885}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6805', 'nonce': 4052106885}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6807', 'nonce': 4052106885}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6806', 'nonce': 4052106885}]}, 'public_addr': '172.21.15.73:6804/4052106885', 'cluster_addr': '172.21.15.73:6805/4052106885', 'heartbeat_back_addr': '172.21.15.73:6807/4052106885', 'heartbeat_front_addr': '172.21.15.73:6806/4052106885', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '20e7f118-7148-4c27-9ec4-a33954709515', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6800', 'nonce': 4266571820}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6801', 'nonce': 4266571820}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6803', 'nonce': 4266571820}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6802', 'nonce': 4266571820}]}, 'public_addr': '172.21.15.136:6800/4266571820', 'cluster_addr': '172.21.15.136:6801/4266571820', 'heartbeat_back_addr': '172.21.15.136:6803/4266571820', 'heartbeat_front_addr': '172.21.15.136:6802/4266571820', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '012409e8-51c2-4584-aba7-95996f0d72ad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6808', 'nonce': 958255259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6809', 'nonce': 958255259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6811', 'nonce': 958255259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6810', 'nonce': 958255259}]}, 'public_addr': '172.21.15.181:6808/958255259', 'cluster_addr': '172.21.15.181:6809/958255259', 'heartbeat_back_addr': '172.21.15.181:6811/958255259', 'heartbeat_front_addr': '172.21.15.181:6810/958255259', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '88b137b1-7d30-45f7-ac5c-ba026704ef24', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6813', 'nonce': 281884267}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6814', 'nonce': 281884267}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6816', 'nonce': 281884267}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6815', 'nonce': 281884267}]}, 'public_addr': '172.21.15.195:6813/281884267', 'cluster_addr': '172.21.15.195:6814/281884267', 'heartbeat_back_addr': '172.21.15.195:6816/281884267', 'heartbeat_front_addr': '172.21.15.195:6815/281884267', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': 'b54732a2-9dff-42e1-b4ef-281cbca3ad7c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6808', 'nonce': 3532043806}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6809', 'nonce': 3532043806}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6811', 'nonce': 3532043806}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6810', 'nonce': 3532043806}]}, 'public_addr': '172.21.15.73:6808/3532043806', 'cluster_addr': '172.21.15.73:6809/3532043806', 'heartbeat_back_addr': '172.21.15.73:6811/3532043806', 'heartbeat_front_addr': '172.21.15.73:6810/3532043806', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '20b48864-44dc-4732-8dff-d0d21cc08032', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6812', 'nonce': 4083982131}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6813', 'nonce': 4083982131}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6815', 'nonce': 4083982131}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6814', 'nonce': 4083982131}]}, 'public_addr': '172.21.15.136:6812/4083982131', 'cluster_addr': '172.21.15.136:6813/4083982131', 'heartbeat_back_addr': '172.21.15.136:6815/4083982131', 'heartbeat_front_addr': '172.21.15.136:6814/4083982131', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '9fac3ef4-e9be-4e8b-9a21-0de35b543d65', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 63, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6804', 'nonce': 1173379175}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6805', 'nonce': 1173379175}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6807', 'nonce': 1173379175}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6806', 'nonce': 1173379175}]}, 'public_addr': '172.21.15.181:6804/1173379175', 'cluster_addr': '172.21.15.181:6805/1173379175', 'heartbeat_back_addr': '172.21.15.181:6807/1173379175', 'heartbeat_front_addr': '172.21.15.181:6806/1173379175', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '16b34f90-3148-4d41-a403-ba0f900991e1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6805', 'nonce': 848686366}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6806', 'nonce': 848686366}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6808', 'nonce': 848686366}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6807', 'nonce': 848686366}]}, 'public_addr': '172.21.15.195:6805/848686366', 'cluster_addr': '172.21.15.195:6806/848686366', 'heartbeat_back_addr': '172.21.15.195:6808/848686366', 'heartbeat_front_addr': '172.21.15.195:6807/848686366', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.372908+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.641263+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.237394+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:24.991965+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.066350+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.003317+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.353521+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.263941+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:24.922976+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:24.743520+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.407426+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.542266+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.457347+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:24.923863+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:24.980077+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.070162+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '4.e', 'osds': [8, 3, 15]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '4.e', 'osds': [2, 8, 4]}, {'pgid': '4.10', 'osds': [4, 5, 6]}, {'pgid': '4.11', 'osds': [2, 12, 8]}, {'pgid': '4.12', 'osds': [6, 8, 14]}, {'pgid': '4.14', 'osds': [5, 6, 10]}, {'pgid': '4.16', 'osds': [1, 13, 7]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T10:11:24.228 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:11:24.235 INFO:tasks.rados.rados.0.smithi195.stdout:1175: finishing set_chunk to oid 2 2024-09-13T10:11:24.235 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 93 (ObjNum 176 snap 0 seq_num 176) dirty exists 2024-09-13T10:11:24.236 INFO:tasks.rados.rados.0.smithi195.stdout:1176: seq_num 218 ranges {463926=462166,1723898=436391,2603977=725887} 2024-09-13T10:11:24.250 INFO:tasks.rados.rados.0.smithi195.stdout:1176: writing smithi19530714-7 from 463926 to 926092 tid 1 2024-09-13T10:11:24.270 INFO:tasks.rados.rados.0.smithi195.stdout:1176: writing smithi19530714-7 from 1723898 to 2160289 tid 2 2024-09-13T10:11:24.289 INFO:tasks.rados.rados.0.smithi195.stdout:1176: writing smithi19530714-7 from 2603977 to 3329864 tid 3 2024-09-13T10:11:24.289 INFO:tasks.rados.rados.0.smithi195.stdout:1173: done (3 left) 2024-09-13T10:11:24.289 INFO:tasks.rados.rados.0.smithi195.stdout:1174: done (2 left) 2024-09-13T10:11:24.289 INFO:tasks.rados.rados.0.smithi195.stdout:1175: done (1 left) 2024-09-13T10:11:24.289 INFO:tasks.rados.rados.0.smithi195.stdout:1177: read oid 48 snap -1 2024-09-13T10:11:24.289 INFO:tasks.rados.rados.0.smithi195.stdout:1177: expect (ObjNum 185 snap 0 seq_num 185) 2024-09-13T10:11:24.289 INFO:tasks.rados.rados.0.smithi195.stdout:1178: read oid 31 snap -1 2024-09-13T10:11:24.290 INFO:tasks.rados.rados.0.smithi195.stdout:1178: expect deleted 2024-09-13T10:11:24.290 INFO:tasks.rados.rados.0.smithi195.stdout:1179: read oid 30 snap -1 2024-09-13T10:11:24.290 INFO:tasks.rados.rados.0.smithi195.stdout:1179: expect (ObjNum 102 snap 0 seq_num 102) 2024-09-13T10:11:24.290 INFO:tasks.rados.rados.0.smithi195.stdout:1180: write oid 32 current snap is 0 2024-09-13T10:11:24.290 INFO:tasks.rados.rados.0.smithi195.stdout:1180: seq_num 219 ranges {480733=537415,1782380=696402,3031400=537159} 2024-09-13T10:11:24.301 INFO:tasks.rados.rados.0.smithi195.stdout:1180: writing smithi19530714-32 from 480733 to 1018148 tid 1 2024-09-13T10:11:24.315 INFO:tasks.rados.rados.0.smithi195.stdout:1180: writing smithi19530714-32 from 1782380 to 2478782 tid 2 2024-09-13T10:11:24.324 INFO:tasks.rados.rados.0.smithi195.stdout:1180: writing smithi19530714-32 from 3031400 to 3568559 tid 3 2024-09-13T10:11:24.324 INFO:tasks.rados.rados.0.smithi195.stdout:1181: set_chunk oid 19 target oid 2024-09-13T10:11:24.327 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:24.324+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:24.347 INFO:tasks.rados.rados.0.smithi195.stdout:1181: set_chunk oid 19 offset: 550400 length: 1920512 target oid 7ab0d74b0ceb1bc1c9f11d1dcf3f91407638dd326777477b5be5839b17128936 offset: 0 snap -1 2024-09-13T10:11:24.347 INFO:tasks.rados.rados.0.smithi195.stdout:1182: tier_flush oid 10 2024-09-13T10:11:24.347 INFO:tasks.rados.rados.0.smithi195.stdout:1182: tier_flush oid 10 snap -1 2024-09-13T10:11:24.347 INFO:tasks.rados.rados.0.smithi195.stdout:1176: finishing write tid 1 to smithi19530714-7 2024-09-13T10:11:24.347 INFO:tasks.rados.rados.0.smithi195.stdout:1183: write (excl) oid 29 current snap is 0 2024-09-13T10:11:24.347 INFO:tasks.rados.rados.0.smithi195.stdout:1183: seq_num 220 ranges {497540=612664,1840862=440060,3058823=464784} 2024-09-13T10:11:24.355 INFO:tasks.rados.rados.0.smithi195.stdout:1183: writing smithi19530714-29 from 497540 to 1110204 tid 1 2024-09-13T10:11:24.364 INFO:tasks.rados.rados.0.smithi195.stdout:1183: writing smithi19530714-29 from 1840862 to 2280922 tid 2 2024-09-13T10:11:24.373 INFO:tasks.rados.rados.0.smithi195.stdout:1183: writing smithi19530714-29 from 3058823 to 3523607 tid 3 2024-09-13T10:11:24.373 INFO:tasks.rados.rados.0.smithi195.stdout:1184: read oid 39 snap -1 2024-09-13T10:11:24.373 INFO:tasks.rados.rados.0.smithi195.stdout:1184: expect (ObjNum 174 snap 0 seq_num 174) 2024-09-13T10:11:24.373 INFO:tasks.rados.rados.0.smithi195.stdout:1185: set_chunk oid 50 target oid 2024-09-13T10:11:24.400 INFO:tasks.rados.rados.0.smithi195.stdout:1185: set_chunk oid 50 offset: 215552 length: 2212352 target oid 8fbc276d1864477ee2602826e46e110082d4a253146958f0c6e18a76d18f0cd8 offset: 0 snap -1 2024-09-13T10:11:24.400 INFO:tasks.rados.rados.0.smithi195.stdout:1186: write oid 27 current snap is 0 2024-09-13T10:11:24.400 INFO:tasks.rados.rados.0.smithi195.stdout:1186: seq_num 221 ranges {514347=404266,1499344=700071,2686246=676056} 2024-09-13T10:11:24.409 INFO:tasks.rados.rados.0.smithi195.stdout:1186: writing smithi19530714-27 from 514347 to 918613 tid 1 2024-09-13T10:11:24.411 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:24.411 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:24.420 INFO:tasks.rados.rados.0.smithi195.stdout:1186: writing smithi19530714-27 from 1499344 to 2199415 tid 2 2024-09-13T10:11:24.427 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-13T10:11:24.428 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:24.424+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:24.430 INFO:tasks.rados.rados.0.smithi195.stdout:1186: writing smithi19530714-27 from 2686246 to 3362302 tid 3 2024-09-13T10:11:24.430 INFO:tasks.rados.rados.0.smithi195.stdout:1187: set_chunk oid 9 target oid 2024-09-13T10:11:24.430 INFO:tasks.rados.rados.0.smithi195.stdout:1187: set_chunk oid 9 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:24.430 INFO:tasks.rados.rados.0.smithi195.stdout:1188: write oid 3 current snap is 0 2024-09-13T10:11:24.430 INFO:tasks.rados.rados.0.smithi195.stdout:1188: seq_num 222 ranges {531154=479515,1557826=443729,2713669=487328,3679295=51859} 2024-09-13T10:11:24.437 INFO:tasks.rados.rados.0.smithi195.stdout:1188: writing smithi19530714-3 from 531154 to 1010669 tid 1 2024-09-13T10:11:24.444 INFO:tasks.rados.rados.0.smithi195.stdout:1188: writing smithi19530714-3 from 1557826 to 2001555 tid 2 2024-09-13T10:11:24.451 INFO:tasks.rados.rados.0.smithi195.stdout:1188: writing smithi19530714-3 from 2713669 to 3200997 tid 3 2024-09-13T10:11:24.455 INFO:tasks.rados.rados.0.smithi195.stdout:1188: writing smithi19530714-3 from 3679295 to 3731154 tid 4 2024-09-13T10:11:24.455 INFO:tasks.rados.rados.0.smithi195.stdout:1189: tier_promote oid 36 2024-09-13T10:11:24.455 INFO:tasks.rados.rados.0.smithi195.stdout:1176: finishing write tid 2 to smithi19530714-7 2024-09-13T10:11:24.455 INFO:tasks.rados.rados.0.smithi195.stdout:1177: expect (ObjNum 185 snap 0 seq_num 185) 2024-09-13T10:11:24.529 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:24.526+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:24.541 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:11:24.542 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:11:24.558 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-09-13T10:11:23.694089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":211586196,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":23,"num_objects_unfound":0,"num_objects_dirty":190,"num_whiteouts":0,"num_read":2219,"num_read_kb":1264391,"num_write":1722,"num_write_kb":381840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1493592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":26,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101,"ondisk_log_size":101,"up":117,"acting":117,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":916764,"kb_used_data":450968,"kb_used_omap":24,"kb_used_meta":461223,"kb_avail":1509032676,"statfs":{"total":1546188226560,"available":1545249460224,"internally_reserved":0,"allocated":461791232,"data_stored":455414884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":472292431},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8737002,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":201,"num_read_kb":136014,"num_write":154,"num_write_kb":26918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":-1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.926920"},"pg_stats":[{"pgid":"4.19","version":"63'47","reported_seq":184,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.235607+0000","last_change":"2024-09-13T10:11:00.388311+0000","last_active":"2024-09-13T10:11:22.235607+0000","last_peered":"2024-09-13T10:11:22.235607+0000","last_clean":"2024-09-13T10:11:22.235607+0000","last_became_active":"2024-09-13T10:10:54.324793+0000","last_became_peered":"2024-09-13T10:10:54.324793+0000","last_unstale":"2024-09-13T10:11:22.235607+0000","last_undegraded":"2024-09-13T10:11:22.235607+0000","last_fullsized":"2024-09-13T10:11:22.235607+0000","mapping_epoch":41,"log_start":"57'44","ondisk_log_start":"57'44","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'28","last_scrub_stamp":"2024-09-13T10:11:00.388132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.388132+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":44,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:05.303384+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3058874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":52,"num_read_kb":42529,"num_write":23,"num_write_kb":6099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,9],"acting":[11,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.18","version":"63'10","reported_seq":135,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.800124+0000","last_change":"2024-09-13T10:11:01.783685+0000","last_active":"2024-09-13T10:11:22.800124+0000","last_peered":"2024-09-13T10:11:22.800124+0000","last_clean":"2024-09-13T10:11:22.800124+0000","last_became_active":"2024-09-13T10:10:54.319034+0000","last_became_peered":"2024-09-13T10:10:54.319034+0000","last_unstale":"2024-09-13T10:11:22.800124+0000","last_undegraded":"2024-09-13T10:11:22.800124+0000","last_fullsized":"2024-09-13T10:11:22.800124+0000","mapping_epoch":41,"log_start":"55'7","ondisk_log_start":"55'7","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'5","last_scrub_stamp":"2024-09-13T10:11:01.783496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.783496+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":7,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:16.865515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2907611,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":37,"num_read_kb":28310,"num_write":4,"num_write_kb":1195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,0],"acting":[3,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"58'73","reported_seq":238,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.694850+0000","last_change":"2024-09-13T10:11:00.442471+0000","last_active":"2024-09-13T10:11:22.694850+0000","last_peered":"2024-09-13T10:11:22.694850+0000","last_clean":"2024-09-13T10:11:22.694850+0000","last_became_active":"2024-09-13T10:10:54.318781+0000","last_became_peered":"2024-09-13T10:10:54.318781+0000","last_unstale":"2024-09-13T10:11:22.694850+0000","last_undegraded":"2024-09-13T10:11:22.694850+0000","last_fullsized":"2024-09-13T10:11:22.694850+0000","mapping_epoch":41,"log_start":"58'70","ondisk_log_start":"58'70","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'34","last_scrub_stamp":"2024-09-13T10:11:00.442413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.442413+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":70,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:15.671215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9462341,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":67591,"num_write":60,"num_write_kb":16288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,10],"acting":[1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"61'56","reported_seq":194,"reported_epoch":64,"state":"remapped+peering","last_fresh":"2024-09-13T10:11:22.695941+0000","last_change":"2024-09-13T10:11:22.695941+0000","last_active":"2024-09-13T10:11:22.695941+0000","last_peered":"2024-09-13T10:11:19.869542+0000","last_clean":"2024-09-13T10:11:19.869542+0000","last_became_active":"2024-09-13T10:10:54.324059+0000","last_became_peered":"2024-09-13T10:10:54.324059+0000","last_unstale":"2024-09-13T10:11:22.695941+0000","last_undegraded":"2024-09-13T10:11:22.695941+0000","last_fullsized":"2024-09-13T10:11:22.695941+0000","mapping_epoch":64,"log_start":"59'53","ondisk_log_start":"59'53","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"46'44","last_scrub_stamp":"2024-09-13T10:11:00.456457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.456457+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":50,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5983292,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":41481,"num_write":12,"num_write_kb":2989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13,5],"acting":[1,13,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"59'37","reported_seq":158,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T10:11:15.753729+0000","last_change":"2024-09-13T10:11:05.611718+0000","last_active":"2024-09-13T10:11:15.753729+0000","last_peered":"2024-09-13T10:11:15.753729+0000","last_clean":"2024-09-13T10:11:15.753729+0000","last_became_active":"2024-09-13T10:10:54.326181+0000","last_became_peered":"2024-09-13T10:10:54.326181+0000","last_unstale":"2024-09-13T10:11:15.753729+0000","last_undegraded":"2024-09-13T10:11:15.753729+0000","last_fullsized":"2024-09-13T10:11:15.753729+0000","mapping_epoch":41,"log_start":"45'33","ondisk_log_start":"45'33","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'36","last_scrub_stamp":"2024-09-13T10:11:05.611551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.611551+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":31,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:32.246300+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3714314,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4993,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,0],"acting":[6,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"39'40","reported_seq":107,"reported_epoch":64,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:11:22.700553+0000","last_change":"2024-09-13T10:11:15.854375+0000","last_active":"2024-09-13T10:11:22.700553+0000","last_peered":"2024-09-13T10:11:22.700553+0000","last_clean":"2024-09-13T10:10:51.933477+0000","last_became_active":"2024-09-13T10:11:15.850712+0000","last_became_peered":"2024-09-13T10:11:15.850712+0000","last_unstale":"2024-09-13T10:11:22.700553+0000","last_undegraded":"2024-09-13T10:11:22.700553+0000","last_fullsized":"2024-09-13T10:11:22.700553+0000","mapping_epoch":58,"log_start":"39'37","ondisk_log_start":"39'37","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'40","last_scrub_stamp":"2024-09-13T10:11:04.595221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.595221+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":37,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7831963,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":112,"num_write":16,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[5,6,10],"avail_no_missing":["5","6","10"],"object_location_counts":[{"shards":"5,6,10","objects":3}],"blocked_by":[],"up_primary":11,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"57'24","reported_seq":176,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.820285+0000","last_change":"2024-09-13T10:11:15.761185+0000","last_active":"2024-09-13T10:11:22.820285+0000","last_peered":"2024-09-13T10:11:22.820285+0000","last_clean":"2024-09-13T10:11:22.820285+0000","last_became_active":"2024-09-13T10:11:15.757364+0000","last_became_peered":"2024-09-13T10:11:15.757364+0000","last_unstale":"2024-09-13T10:11:22.820285+0000","last_undegraded":"2024-09-13T10:11:22.820285+0000","last_fullsized":"2024-09-13T10:11:22.820285+0000","mapping_epoch":58,"log_start":"57'21","ondisk_log_start":"57'21","created":41,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'17","last_scrub_stamp":"2024-09-13T10:11:03.220846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:03.220846+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":19,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:14.071053+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2924418,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":28,"num_read_kb":25554,"num_write":9,"num_write_kb":2174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1493592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"58'26","reported_seq":181,"reported_epoch":59,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:11:18.133555+0000","last_change":"2024-09-13T10:11:11.763857+0000","last_active":"2024-09-13T10:11:18.133555+0000","last_peered":"2024-09-13T10:11:18.133555+0000","last_clean":"2024-09-13T10:11:08.694521+0000","last_became_active":"2024-09-13T10:11:11.753624+0000","last_became_peered":"2024-09-13T10:11:11.753624+0000","last_unstale":"2024-09-13T10:11:18.133555+0000","last_undegraded":"2024-09-13T10:11:18.133555+0000","last_fullsized":"2024-09-13T10:11:18.133555+0000","mapping_epoch":54,"log_start":"58'23","ondisk_log_start":"58'23","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'17","last_scrub_stamp":"2024-09-13T10:11:02.582829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:02.582829+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":23,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5243784,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":92,"num_read_kb":75068,"num_write":10,"num_write_kb":2527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,9],"acting":[6,8,14],"avail_no_missing":["6","8","14"],"object_location_counts":[{"shards":"6,8,14","objects":2}],"blocked_by":[],"up_primary":13,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"40'13","reported_seq":133,"reported_epoch":60,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:11:18.759362+0000","last_change":"2024-09-13T10:11:08.740093+0000","last_active":"2024-09-13T10:11:18.759362+0000","last_peered":"2024-09-13T10:11:18.759362+0000","last_clean":"2024-09-13T10:11:06.437508+0000","last_became_active":"2024-09-13T10:11:08.737198+0000","last_became_peered":"2024-09-13T10:11:08.737198+0000","last_unstale":"2024-09-13T10:11:18.759362+0000","last_undegraded":"2024-09-13T10:11:18.759362+0000","last_fullsized":"2024-09-13T10:11:18.759362+0000","mapping_epoch":51,"log_start":"39'10","ondisk_log_start":"39'10","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"40'13","last_scrub_stamp":"2024-09-13T10:11:04.242723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.242723+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":10,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2689087,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":24610,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,12],"acting":[2,12,8],"avail_no_missing":["2","8","12"],"object_location_counts":[{"shards":"2,8,12","objects":1}],"blocked_by":[],"up_primary":13,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.10","version":"53'33","reported_seq":186,"reported_epoch":64,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:11:22.820441+0000","last_change":"2024-09-13T10:11:07.791128+0000","last_active":"2024-09-13T10:11:22.820441+0000","last_peered":"2024-09-13T10:11:22.820441+0000","last_clean":"2024-09-13T10:11:04.408213+0000","last_became_active":"2024-09-13T10:11:07.782322+0000","last_became_peered":"2024-09-13T10:11:07.782322+0000","last_unstale":"2024-09-13T10:11:22.820441+0000","last_undegraded":"2024-09-13T10:11:22.820441+0000","last_fullsized":"2024-09-13T10:11:22.820441+0000","mapping_epoch":49,"log_start":"52'30","ondisk_log_start":"52'30","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'16","last_scrub_stamp":"2024-09-13T10:11:00.706874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.706874+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":28,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4773188,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":75,"num_read_kb":61619,"num_write":21,"num_write_kb":5962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,14,3],"acting":[4,5,6],"avail_no_missing":["4","5","6"],"object_location_counts":[{"shards":"4,5,6","objects":2}],"blocked_by":[],"up_primary":9,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":29,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.800030+0000","last_change":"2024-09-13T10:11:12.744453+0000","last_active":"2024-09-13T10:11:22.800030+0000","last_peered":"2024-09-13T10:11:22.800030+0000","last_clean":"2024-09-13T10:11:22.800030+0000","last_became_active":"2024-09-13T10:11:12.744090+0000","last_became_peered":"2024-09-13T10:11:12.744090+0000","last_unstale":"2024-09-13T10:11:22.800030+0000","last_undegraded":"2024-09-13T10:11:22.800030+0000","last_fullsized":"2024-09-13T10:11:22.800030+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:57.001325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,12],"acting":[3,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"39'12","reported_seq":117,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T10:11:15.753517+0000","last_change":"2024-09-13T10:11:01.537080+0000","last_active":"2024-09-13T10:11:15.753517+0000","last_peered":"2024-09-13T10:11:15.753517+0000","last_clean":"2024-09-13T10:11:15.753517+0000","last_became_active":"2024-09-13T10:10:54.318519+0000","last_became_peered":"2024-09-13T10:10:54.318519+0000","last_unstale":"2024-09-13T10:11:15.753517+0000","last_undegraded":"2024-09-13T10:11:15.753517+0000","last_fullsized":"2024-09-13T10:11:15.753517+0000","mapping_epoch":31,"log_start":"38'9","ondisk_log_start":"38'9","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'12","last_scrub_stamp":"2024-09-13T10:11:01.536963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.536963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":9,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:19.892837+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,14],"acting":[6,8,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.800030+0000","last_change":"2024-09-13T10:10:32.932079+0000","last_active":"2024-09-13T10:11:22.800030+0000","last_peered":"2024-09-13T10:11:22.800030+0000","last_clean":"2024-09-13T10:11:22.800030+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:11:22.800030+0000","last_undegraded":"2024-09-13T10:11:22.800030+0000","last_fullsized":"2024-09-13T10:11:22.800030+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:57.533673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"59'62","reported_seq":198,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T10:11:18.759452+0000","last_change":"2024-09-13T10:11:01.219554+0000","last_active":"2024-09-13T10:11:18.759452+0000","last_peered":"2024-09-13T10:11:18.759452+0000","last_clean":"2024-09-13T10:11:18.759452+0000","last_became_active":"2024-09-13T10:10:54.314545+0000","last_became_peered":"2024-09-13T10:10:54.314545+0000","last_unstale":"2024-09-13T10:11:18.759452+0000","last_undegraded":"2024-09-13T10:11:18.759452+0000","last_fullsized":"2024-09-13T10:11:18.759452+0000","mapping_epoch":31,"log_start":"59'59","ondisk_log_start":"59'59","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'27","last_scrub_stamp":"2024-09-13T10:11:01.219486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.219486+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":58,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:06.525021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6537923,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":27068,"num_write":50,"num_write_kb":14446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,8],"acting":[2,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.819858+0000","last_change":"2024-09-13T10:10:32.177488+0000","last_active":"2024-09-13T10:11:22.819858+0000","last_peered":"2024-09-13T10:11:22.819858+0000","last_clean":"2024-09-13T10:11:22.819858+0000","last_became_active":"2024-09-13T10:10:30.128658+0000","last_became_peered":"2024-09-13T10:10:30.128658+0000","last_unstale":"2024-09-13T10:11:22.819858+0000","last_undegraded":"2024-09-13T10:11:22.819858+0000","last_fullsized":"2024-09-13T10:11:22.819858+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:50.278429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.0","version":"60'24","reported_seq":165,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.700397+0000","last_change":"2024-09-13T10:11:05.681812+0000","last_active":"2024-09-13T10:11:22.700397+0000","last_peered":"2024-09-13T10:11:22.700397+0000","last_clean":"2024-09-13T10:11:22.700397+0000","last_became_active":"2024-09-13T10:10:54.320014+0000","last_became_peered":"2024-09-13T10:10:54.320014+0000","last_unstale":"2024-09-13T10:11:22.700397+0000","last_undegraded":"2024-09-13T10:11:22.700397+0000","last_fullsized":"2024-09-13T10:11:22.700397+0000","mapping_epoch":31,"log_start":"60'21","ondisk_log_start":"60'21","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'14","last_scrub_stamp":"2024-09-13T10:11:05.681653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.681653+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":20,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:35.041022+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3411821,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":34456,"num_write":10,"num_write_kb":3373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T10:11:18.752896+0000","last_change":"2024-09-13T10:10:32.701581+0000","last_active":"2024-09-13T10:11:18.752896+0000","last_peered":"2024-09-13T10:11:18.752896+0000","last_clean":"2024-09-13T10:11:18.752896+0000","last_became_active":"2024-09-13T10:10:30.126588+0000","last_became_peered":"2024-09-13T10:10:30.126588+0000","last_unstale":"2024-09-13T10:11:18.752896+0000","last_undegraded":"2024-09-13T10:11:18.752896+0000","last_fullsized":"2024-09-13T10:11:18.752896+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:40.095103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089379399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.1","version":"44'18","reported_seq":139,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T10:11:18.759581+0000","last_change":"2024-09-13T10:11:02.190704+0000","last_active":"2024-09-13T10:11:18.759581+0000","last_peered":"2024-09-13T10:11:18.759581+0000","last_clean":"2024-09-13T10:11:18.759581+0000","last_became_active":"2024-09-13T10:10:54.317600+0000","last_became_peered":"2024-09-13T10:10:54.317600+0000","last_unstale":"2024-09-13T10:11:18.759581+0000","last_undegraded":"2024-09-13T10:11:18.759581+0000","last_fullsized":"2024-09-13T10:11:18.759581+0000","mapping_epoch":31,"log_start":"43'14","ondisk_log_start":"43'14","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'18","last_scrub_stamp":"2024-09-13T10:11:02.190506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:02.190506+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":14,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:28.399688+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3411788,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":45401,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,12],"acting":[2,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"62'173","reported_seq":258,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T10:11:20.794962+0000","last_change":"2024-09-13T10:10:43.231291+0000","last_active":"2024-09-13T10:11:20.794962+0000","last_peered":"2024-09-13T10:11:20.794962+0000","last_clean":"2024-09-13T10:11:20.794962+0000","last_became_active":"2024-09-13T10:10:43.230866+0000","last_became_peered":"2024-09-13T10:10:43.230866+0000","last_unstale":"2024-09-13T10:11:20.794962+0000","last_undegraded":"2024-09-13T10:11:20.794962+0000","last_fullsized":"2024-09-13T10:11:20.794962+0000","mapping_epoch":30,"log_start":"61'170","ondisk_log_start":"61'170","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":185,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:54.564637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17972410,"num_objects":35,"num_object_clones":0,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":163,"num_read_kb":2774,"num_write":231,"num_write_kb":48089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,4,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":176,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.800288+0000","last_change":"2024-09-13T10:11:12.699917+0000","last_active":"2024-09-13T10:11:22.800288+0000","last_peered":"2024-09-13T10:11:22.800288+0000","last_clean":"2024-09-13T10:11:22.800288+0000","last_became_active":"2024-09-13T10:11:12.699043+0000","last_became_peered":"2024-09-13T10:11:12.699043+0000","last_unstale":"2024-09-13T10:11:22.800288+0000","last_undegraded":"2024-09-13T10:11:22.800288+0000","last_fullsized":"2024-09-13T10:11:22.800288+0000","mapping_epoch":55,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:41.120070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,7],"acting":[3,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":92,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T10:11:20.793258+0000","last_change":"2024-09-13T10:10:32.702214+0000","last_active":"2024-09-13T10:11:20.793258+0000","last_peered":"2024-09-13T10:11:20.793258+0000","last_clean":"2024-09-13T10:11:20.793258+0000","last_became_active":"2024-09-13T10:10:30.124759+0000","last_became_peered":"2024-09-13T10:10:30.124759+0000","last_unstale":"2024-09-13T10:11:20.793258+0000","last_undegraded":"2024-09-13T10:11:20.793258+0000","last_fullsized":"2024-09-13T10:11:20.793258+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:45.386731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090474700000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.6","version":"64'62","reported_seq":214,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.841497+0000","last_change":"2024-09-13T10:11:05.397056+0000","last_active":"2024-09-13T10:11:22.841497+0000","last_peered":"2024-09-13T10:11:22.841497+0000","last_clean":"2024-09-13T10:11:22.841497+0000","last_became_active":"2024-09-13T10:10:54.319456+0000","last_became_peered":"2024-09-13T10:10:54.319456+0000","last_unstale":"2024-09-13T10:11:22.841497+0000","last_undegraded":"2024-09-13T10:11:22.841497+0000","last_fullsized":"2024-09-13T10:11:22.841497+0000","mapping_epoch":31,"log_start":"64'59","ondisk_log_start":"64'59","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'44","last_scrub_stamp":"2024-09-13T10:11:05.396838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.396838+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":59,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:30.673063+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7092554,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":24140,"num_write":22,"num_write_kb":6053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,13,1],"acting":[7,13,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"63'210","reported_seq":292,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.800027+0000","last_change":"2024-09-13T10:10:43.220235+0000","last_active":"2024-09-13T10:11:22.800027+0000","last_peered":"2024-09-13T10:11:22.800027+0000","last_clean":"2024-09-13T10:11:22.800027+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:11:22.800027+0000","last_undegraded":"2024-09-13T10:11:22.800027+0000","last_fullsized":"2024-09-13T10:11:22.800027+0000","mapping_epoch":30,"log_start":"63'207","ondisk_log_start":"63'207","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":219,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:55.166722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24126617,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":185,"num_read_kb":2584,"num_write":284,"num_write_kb":63361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":89,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T10:11:18.759282+0000","last_change":"2024-09-13T10:10:32.146042+0000","last_active":"2024-09-13T10:11:18.759282+0000","last_peered":"2024-09-13T10:11:18.759282+0000","last_clean":"2024-09-13T10:11:18.759282+0000","last_became_active":"2024-09-13T10:10:30.127232+0000","last_became_peered":"2024-09-13T10:10:30.127232+0000","last_unstale":"2024-09-13T10:11:18.759282+0000","last_undegraded":"2024-09-13T10:11:18.759282+0000","last_fullsized":"2024-09-13T10:11:18.759282+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:29.955802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034622780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"63'51","reported_seq":199,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.695108+0000","last_change":"2024-09-13T10:11:05.436449+0000","last_active":"2024-09-13T10:11:22.695108+0000","last_peered":"2024-09-13T10:11:22.695108+0000","last_clean":"2024-09-13T10:11:22.695108+0000","last_became_active":"2024-09-13T10:10:54.314465+0000","last_became_peered":"2024-09-13T10:10:54.314465+0000","last_unstale":"2024-09-13T10:11:22.695108+0000","last_undegraded":"2024-09-13T10:11:22.695108+0000","last_fullsized":"2024-09-13T10:11:22.695108+0000","mapping_epoch":31,"log_start":"55'47","ondisk_log_start":"55'47","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'40","last_scrub_stamp":"2024-09-13T10:11:05.436272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.436272+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":44,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:30.799300+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2571471,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1654,"num_write":11,"num_write_kb":2520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,10],"acting":[1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"63'179","reported_seq":261,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.701054+0000","last_change":"2024-09-13T10:10:43.220955+0000","last_active":"2024-09-13T10:11:22.701054+0000","last_peered":"2024-09-13T10:11:22.701054+0000","last_clean":"2024-09-13T10:11:22.701054+0000","last_became_active":"2024-09-13T10:10:43.220074+0000","last_became_peered":"2024-09-13T10:10:43.220074+0000","last_unstale":"2024-09-13T10:11:22.701054+0000","last_undegraded":"2024-09-13T10:11:22.701054+0000","last_fullsized":"2024-09-13T10:11:22.701054+0000","mapping_epoch":30,"log_start":"63'176","ondisk_log_start":"63'176","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":188,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:54.562129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21794410,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":154,"num_read_kb":2658,"num_write":233,"num_write_kb":55059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":94,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.698398+0000","last_change":"2024-09-13T10:10:33.223611+0000","last_active":"2024-09-13T10:11:22.698398+0000","last_peered":"2024-09-13T10:11:22.698398+0000","last_clean":"2024-09-13T10:11:22.698398+0000","last_became_active":"2024-09-13T10:10:30.128310+0000","last_became_peered":"2024-09-13T10:10:30.128310+0000","last_unstale":"2024-09-13T10:11:22.698398+0000","last_undegraded":"2024-09-13T10:11:22.698398+0000","last_fullsized":"2024-09-13T10:11:22.698398+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:46.008600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081844899999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"59'106","reported_seq":329,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T10:11:18.153219+0000","last_change":"2024-09-13T10:11:03.633738+0000","last_active":"2024-09-13T10:11:18.153219+0000","last_peered":"2024-09-13T10:11:18.153219+0000","last_clean":"2024-09-13T10:11:18.153219+0000","last_became_active":"2024-09-13T10:10:54.316518+0000","last_became_peered":"2024-09-13T10:10:54.316518+0000","last_unstale":"2024-09-13T10:11:18.153219+0000","last_undegraded":"2024-09-13T10:11:18.153219+0000","last_fullsized":"2024-09-13T10:11:18.153219+0000","mapping_epoch":31,"log_start":"59'102","ondisk_log_start":"59'102","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'66","last_scrub_stamp":"2024-09-13T10:11:03.633499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:03.633499+0000","objects_scrubbed":5,"log_size":4,"log_dups_size":102,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:20.115625+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16605316,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":148,"num_read_kb":120004,"num_write":45,"num_write_kb":12304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10],"acting":[6,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"59'203","reported_seq":283,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T10:11:18.753124+0000","last_change":"2024-09-13T10:10:43.230767+0000","last_active":"2024-09-13T10:11:18.753124+0000","last_peered":"2024-09-13T10:11:18.753124+0000","last_clean":"2024-09-13T10:11:18.753124+0000","last_became_active":"2024-09-13T10:10:43.229961+0000","last_became_peered":"2024-09-13T10:10:43.229961+0000","last_unstale":"2024-09-13T10:11:18.753124+0000","last_undegraded":"2024-09-13T10:11:18.753124+0000","last_fullsized":"2024-09-13T10:11:18.753124+0000","mapping_epoch":30,"log_start":"59'200","ondisk_log_start":"59'200","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":209,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:53.856674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10730671,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":189,"num_read_kb":2855,"num_write":251,"num_write_kb":53051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T10:11:15.752807+0000","last_change":"2024-09-13T10:10:32.149042+0000","last_active":"2024-09-13T10:11:15.752807+0000","last_peered":"2024-09-13T10:11:15.752807+0000","last_clean":"2024-09-13T10:11:15.752807+0000","last_became_active":"2024-09-13T10:10:30.272007+0000","last_became_peered":"2024-09-13T10:10:30.272007+0000","last_unstale":"2024-09-13T10:11:15.752807+0000","last_undegraded":"2024-09-13T10:11:15.752807+0000","last_fullsized":"2024-09-13T10:11:15.752807+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:49.552306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000987604,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"59'98","reported_seq":292,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T10:11:18.535587+0000","last_change":"2024-09-13T10:11:00.541038+0000","last_active":"2024-09-13T10:11:18.535587+0000","last_peered":"2024-09-13T10:11:18.535587+0000","last_clean":"2024-09-13T10:11:18.535587+0000","last_became_active":"2024-09-13T10:10:54.315196+0000","last_became_peered":"2024-09-13T10:10:54.315196+0000","last_unstale":"2024-09-13T10:11:18.535587+0000","last_undegraded":"2024-09-13T10:11:18.535587+0000","last_fullsized":"2024-09-13T10:11:18.535587+0000","mapping_epoch":31,"log_start":"59'95","ondisk_log_start":"59'95","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'48","last_scrub_stamp":"2024-09-13T10:11:00.540905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.540905+0000","objects_scrubbed":5,"log_size":3,"log_dups_size":94,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:04.265928+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10924550,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":83,"num_read_kb":43545,"num_write":60,"num_write_kb":16923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,0],"acting":[6,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"39'5","reported_seq":111,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.799824+0000","last_change":"2024-09-13T10:11:00.786261+0000","last_active":"2024-09-13T10:11:22.799824+0000","last_peered":"2024-09-13T10:11:22.799824+0000","last_clean":"2024-09-13T10:11:22.799824+0000","last_became_active":"2024-09-13T10:10:54.314935+0000","last_became_peered":"2024-09-13T10:10:54.314935+0000","last_unstale":"2024-09-13T10:11:22.799824+0000","last_undegraded":"2024-09-13T10:11:22.799824+0000","last_fullsized":"2024-09-13T10:11:22.799824+0000","mapping_epoch":31,"log_start":"38'2","ondisk_log_start":"38'2","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'5","last_scrub_stamp":"2024-09-13T10:11:00.786110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.786110+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:18.670276+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,0],"acting":[3,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"61'34","reported_seq":159,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.235861+0000","last_change":"2024-09-13T10:11:04.446852+0000","last_active":"2024-09-13T10:11:22.235861+0000","last_peered":"2024-09-13T10:11:22.235861+0000","last_clean":"2024-09-13T10:11:22.235861+0000","last_became_active":"2024-09-13T10:10:54.316057+0000","last_became_peered":"2024-09-13T10:10:54.316057+0000","last_unstale":"2024-09-13T10:11:22.235861+0000","last_undegraded":"2024-09-13T10:11:22.235861+0000","last_fullsized":"2024-09-13T10:11:22.235861+0000","mapping_epoch":31,"log_start":"61'31","ondisk_log_start":"61'31","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'29","last_scrub_stamp":"2024-09-13T10:11:04.446717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.446717+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":28,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:12.946954+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3462242,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":14221,"num_write":5,"num_write_kb":1299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,9],"acting":[11,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.a","version":"54'19","reported_seq":119,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T10:11:15.753213+0000","last_change":"2024-09-13T10:11:12.708135+0000","last_active":"2024-09-13T10:11:15.753213+0000","last_peered":"2024-09-13T10:11:15.753213+0000","last_clean":"2024-09-13T10:11:15.753213+0000","last_became_active":"2024-09-13T10:11:12.707306+0000","last_became_peered":"2024-09-13T10:11:12.707306+0000","last_unstale":"2024-09-13T10:11:15.753213+0000","last_undegraded":"2024-09-13T10:11:15.753213+0000","last_fullsized":"2024-09-13T10:11:15.753213+0000","mapping_epoch":55,"log_start":"51'16","ondisk_log_start":"51'16","created":31,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":16,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:06.200023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2437015,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":34314,"num_write":18,"num_write_kb":5695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,13],"acting":[6,8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"59'52","reported_seq":191,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-13T10:11:18.752919+0000","last_change":"2024-09-13T10:10:44.227804+0000","last_active":"2024-09-13T10:11:18.752919+0000","last_peered":"2024-09-13T10:11:18.752919+0000","last_clean":"2024-09-13T10:11:18.752919+0000","last_became_active":"2024-09-13T10:10:44.226956+0000","last_became_peered":"2024-09-13T10:10:44.226956+0000","last_unstale":"2024-09-13T10:11:18.752919+0000","last_undegraded":"2024-09-13T10:11:18.752919+0000","last_fullsized":"2024-09-13T10:11:18.752919+0000","mapping_epoch":31,"log_start":"54'48","ondisk_log_start":"54'48","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":48,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:44.906351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8403500,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":142,"num_read_kb":122916,"num_write":49,"num_write_kb":13728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,4],"acting":[14,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.c","version":"58'59","reported_seq":229,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T10:11:15.754035+0000","last_change":"2024-09-13T10:10:59.373229+0000","last_active":"2024-09-13T10:11:15.754035+0000","last_peered":"2024-09-13T10:11:15.754035+0000","last_clean":"2024-09-13T10:11:15.754035+0000","last_became_active":"2024-09-13T10:10:59.366341+0000","last_became_peered":"2024-09-13T10:10:59.366341+0000","last_unstale":"2024-09-13T10:11:15.754035+0000","last_undegraded":"2024-09-13T10:11:15.754035+0000","last_fullsized":"2024-09-13T10:11:15.754035+0000","mapping_epoch":44,"log_start":"57'56","ondisk_log_start":"57'56","created":31,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":56,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:48.299027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10336239,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":202,"num_read_kb":187173,"num_write":51,"num_write_kb":13508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,12,3],"acting":[6,12,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"49'32","reported_seq":154,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-13T10:11:22.819890+0000","last_change":"2024-09-13T10:10:59.365358+0000","last_active":"2024-09-13T10:11:22.819890+0000","last_peered":"2024-09-13T10:11:22.819890+0000","last_clean":"2024-09-13T10:11:22.819890+0000","last_became_active":"2024-09-13T10:10:59.364443+0000","last_became_peered":"2024-09-13T10:10:59.364443+0000","last_unstale":"2024-09-13T10:11:22.819890+0000","last_undegraded":"2024-09-13T10:11:22.819890+0000","last_fullsized":"2024-09-13T10:11:22.819890+0000","mapping_epoch":44,"log_start":"47'29","ondisk_log_start":"47'29","created":31,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":29,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:04.976156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2369787,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":65,"num_read_kb":55051,"num_write":27,"num_write_kb":7702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,12,7],"acting":[15,12,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.e","version":"46'10","reported_seq":104,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-13T10:11:15.750618+0000","last_change":"2024-09-13T10:10:44.218427+0000","last_active":"2024-09-13T10:11:15.750618+0000","last_peered":"2024-09-13T10:11:15.750618+0000","last_clean":"2024-09-13T10:11:15.750618+0000","last_became_active":"2024-09-13T10:10:44.217784+0000","last_became_peered":"2024-09-13T10:10:44.217784+0000","last_unstale":"2024-09-13T10:11:15.750618+0000","last_undegraded":"2024-09-13T10:11:15.750618+0000","last_fullsized":"2024-09-13T10:11:15.750618+0000","mapping_epoch":31,"log_start":"46'7","ondisk_log_start":"46'7","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":7,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:49.240011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3747928,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":33,"num_read_kb":29838,"num_write":9,"num_write_kb":2294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,8],"acting":[4,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"60'60","reported_seq":202,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-13T10:11:20.809904+0000","last_change":"2024-09-13T10:10:44.224347+0000","last_active":"2024-09-13T10:11:20.809904+0000","last_peered":"2024-09-13T10:11:20.809904+0000","last_clean":"2024-09-13T10:11:20.809904+0000","last_became_active":"2024-09-13T10:10:44.223700+0000","last_became_peered":"2024-09-13T10:10:44.223700+0000","last_unstale":"2024-09-13T10:11:20.809904+0000","last_undegraded":"2024-09-13T10:11:20.809904+0000","last_fullsized":"2024-09-13T10:11:20.809904+0000","mapping_epoch":31,"log_start":"60'57","ondisk_log_start":"60'57","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":57,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:47.288641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6470695,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":135,"num_read_kb":141799,"num_write":64,"num_write_kb":17823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,7],"acting":[13,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":26,"stat_sum":{"num_bytes":136371701,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":23,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":1442,"num_read_kb":1253447,"num_write":576,"num_write_kb":159606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1493592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":26,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":242503680,"data_stored":241712512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":78,"acting":78,"num_store_stats":16},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":74624108,"num_objects":149,"num_object_clones":0,"num_object_copies":447,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":0,"num_read":691,"num_read_kb":10871,"num_write":999,"num_write_kb":219560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":198934528,"data_stored":198158643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":11}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247821,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68432,"kb_used_data":37248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94303408,"statfs":{"total":96636764160,"available":96566689792,"internally_reserved":0,"allocated":38141952,"data_stored":37702962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247823,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38004,"kb_used_data":10980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333836,"statfs":{"total":96636764160,"available":96597848064,"internally_reserved":0,"allocated":11243520,"data_stored":10930533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247823,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66948,"kb_used_data":35764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94304892,"statfs":{"total":96636764160,"available":96568209408,"internally_reserved":0,"allocated":36622336,"data_stored":36221213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313231,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33680,"kb_used_data":6656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338160,"statfs":{"total":96636764160,"available":96602275840,"internally_reserved":0,"allocated":6815744,"data_stored":6497211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,5,6,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313230,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58836,"kb_used_data":31812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94313004,"statfs":{"total":96636764160,"available":96576516096,"internally_reserved":0,"allocated":32575488,"data_stored":32206931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,3,4,6,7,8,9,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247821,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50800,"kb_used_data":18768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321040,"statfs":{"total":96636764160,"available":96584744960,"internally_reserved":0,"allocated":19218432,"data_stored":18803493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280526,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60408,"kb_used_data":29216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311432,"statfs":{"total":96636764160,"available":96574906368,"internally_reserved":0,"allocated":29917184,"data_stored":29479563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,6,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280526,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73592,"kb_used_data":42408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94298248,"statfs":{"total":96636764160,"available":96561405952,"internally_reserved":0,"allocated":43425792,"data_stored":42966708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,8,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280527,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82616,"kb_used_data":55592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94289224,"statfs":{"total":96636764160,"available":96552165376,"internally_reserved":0,"allocated":56926208,"data_stored":56453668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,3,5,6,8,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280525,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48596,"kb_used_data":21572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94323244,"statfs":{"total":96636764160,"available":96587001856,"internally_reserved":0,"allocated":22089728,"data_stored":21738016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280526,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68156,"kb_used_data":41132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94303684,"statfs":{"total":96636764160,"available":96566972416,"internally_reserved":0,"allocated":42119168,"data_stored":41719639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280526,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70212,"kb_used_data":42516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94301628,"statfs":{"total":96636764160,"available":96564867072,"internally_reserved":0,"allocated":43536384,"data_stored":43018967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280526,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42928,"kb_used_data":11744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328912,"statfs":{"total":96636764160,"available":96592805888,"internally_reserved":0,"allocated":12025856,"data_stored":11706789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,5,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280526,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29372,"kb_used_data":2348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342468,"statfs":{"total":96636764160,"available":96606687232,"internally_reserved":0,"allocated":2404352,"data_stored":2101891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247821,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46520,"kb_used_data":19496,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94325320,"statfs":{"total":96636764160,"available":96589127680,"internally_reserved":0,"allocated":19963904,"data_stored":19629101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247820,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77664,"kb_used_data":43716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94294176,"statfs":{"total":96636764160,"available":96557236224,"internally_reserved":0,"allocated":44765184,"data_stored":44238199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6237871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12972032,"data_stored":12908073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":31784960,"data_stored":31653708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":42586112,"data_stored":42451801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24203264,"data_stored":24126617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20566016,"data_stored":20487535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":12972032,"data_stored":12908073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":18026496,"data_stored":17972410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":29532160,"data_stored":29412555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":10317824,"data_stored":10255653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16248832,"data_stored":16179413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":19312640,"data_stored":19271023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8630272,"data_stored":8614331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12546048,"data_stored":12512574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":21381120,"data_stored":21337577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":44113920,"data_stored":43880121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16269312,"data_stored":16251051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":14802944,"data_stored":14776992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":6127616,"data_stored":6113824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":30449664,"data_stored":30297093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8159232,"data_stored":8148362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":10596352,"data_stored":10572474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":13864960,"data_stored":13851153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":7962624,"data_stored":7932348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":1720320,"data_stored":1718523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:11:24.561 INFO:tasks.thrashosds.thrasher:Setting 1.3 to [4, 9, 8] 2024-09-13T10:11:24.561 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '1.3', '4', '9', '8'] 2024-09-13T10:11:24.561 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 1.3 4 9 8 2024-09-13T10:11:24.589 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:24.589 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:24.603 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-13T10:11:24.627 INFO:tasks.rados.rados.0.smithi195.stdout:1176: finishing write tid 3 to smithi19530714-7 2024-09-13T10:11:24.628 INFO:tasks.rados.rados.0.smithi195.stdout:1177: done (13 left) 2024-09-13T10:11:24.628 INFO:tasks.rados.rados.0.smithi195.stdout:1178: done (12 left) 2024-09-13T10:11:24.628 INFO:tasks.rados.rados.0.smithi195.stdout:1190: read oid 40 snap -1 2024-09-13T10:11:24.628 INFO:tasks.rados.rados.0.smithi195.stdout:1190: expect (ObjNum 203 snap 0 seq_num 203) 2024-09-13T10:11:24.628 INFO:tasks.rados.rados.0.smithi195.stdout:1191: read oid 20 snap -1 2024-09-13T10:11:24.628 INFO:tasks.rados.rados.0.smithi195.stdout:1191: expect deleted 2024-09-13T10:11:24.628 INFO:tasks.rados.rados.0.smithi195.stdout:1192: read oid 28 snap -1 2024-09-13T10:11:24.628 INFO:tasks.rados.rados.0.smithi195.stdout:1192: expect (ObjNum 180 snap 0 seq_num 180) 2024-09-13T10:11:24.629 INFO:tasks.rados.rados.0.smithi195.stdout:1193: tier_evict oid 24 2024-09-13T10:11:24.629 INFO:tasks.rados.rados.0.smithi195.stdout:1193: tier_evict oid 24 snap -1 2024-09-13T10:11:24.629 INFO:tasks.rados.rados.0.smithi195.stdout:1176: finishing write tid 4 to smithi19530714-7 2024-09-13T10:11:24.629 INFO:tasks.rados.rados.0.smithi195.stdout:1176: finishing write tid 5 to smithi19530714-7 2024-09-13T10:11:24.629 INFO:tasks.rados.rados.0.smithi195.stdout:1176: oid 7 updating version 0 to 76 2024-09-13T10:11:24.629 INFO:tasks.rados.rados.0.smithi195.stdout:1176: oid 7 updating version 76 to 79 2024-09-13T10:11:24.629 INFO:tasks.rados.rados.0.smithi195.stdout:1176: oid 7 version 79 is already newer than 77 2024-09-13T10:11:24.629 INFO:tasks.rados.rados.0.smithi195.stdout:1176: oid 7 version 79 is already newer than 78 2024-09-13T10:11:24.629 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 79 (ObjNum 218 snap 0 seq_num 218) dirty exists 2024-09-13T10:11:24.629 INFO:tasks.rados.rados.0.smithi195.stdout:1176: left oid 7 (ObjNum 218 snap 0 seq_num 218) 2024-09-13T10:11:24.629 INFO:tasks.rados.rados.0.smithi195.stdout:1179: expect (ObjNum 102 snap 0 seq_num 102) 2024-09-13T10:11:24.631 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:24.627+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:24.669 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3080560 2024-09-13T10:11:24.677 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:24.717 INFO:tasks.rados.rados.0.smithi195.stdout:1180: finishing write tid 1 to smithi19530714-32 2024-09-13T10:11:24.717 INFO:tasks.rados.rados.0.smithi195.stdout:1180: finishing write tid 2 to smithi19530714-32 2024-09-13T10:11:24.717 INFO:tasks.rados.rados.0.smithi195.stdout:1180: finishing write tid 3 to smithi19530714-32 2024-09-13T10:11:24.717 INFO:tasks.rados.rados.0.smithi195.stdout:1180: finishing write tid 4 to smithi19530714-32 2024-09-13T10:11:24.717 INFO:tasks.rados.rados.0.smithi195.stdout:1180: finishing write tid 5 to smithi19530714-32 2024-09-13T10:11:24.717 INFO:tasks.rados.rados.0.smithi195.stdout:1180: oid 32 updating version 0 to 55 2024-09-13T10:11:24.718 INFO:tasks.rados.rados.0.smithi195.stdout:1180: oid 32 version 55 is already newer than 52 2024-09-13T10:11:24.718 INFO:tasks.rados.rados.0.smithi195.stdout:1180: oid 32 version 55 is already newer than 53 2024-09-13T10:11:24.718 INFO:tasks.rados.rados.0.smithi195.stdout:1180: oid 32 version 55 is already newer than 54 2024-09-13T10:11:24.718 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 55 (ObjNum 219 snap 0 seq_num 219) dirty exists 2024-09-13T10:11:24.718 INFO:tasks.rados.rados.0.smithi195.stdout:1180: left oid 32 (ObjNum 219 snap 0 seq_num 219) 2024-09-13T10:11:24.718 INFO:tasks.rados.rados.0.smithi195.stdout:1181: finishing set_chunk to oid 19 2024-09-13T10:11:24.718 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 19 set_chunk returned error code -95 offset: 550400 length: 1920512 2024-09-13T10:11:24.718 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 0 (ObjNum 160 snap 0 seq_num 160) dirty exists 2024-09-13T10:11:24.718 INFO:tasks.rados.rados.0.smithi195.stdout:1183: finishing write tid 1 to smithi19530714-29 2024-09-13T10:11:24.719 INFO:tasks.rados.rados.0.smithi195.stdout:1176: done (15 left) 2024-09-13T10:11:24.719 INFO:tasks.rados.rados.0.smithi195.stdout:1179: done (14 left) 2024-09-13T10:11:24.719 INFO:tasks.rados.rados.0.smithi195.stdout:1180: done (13 left) 2024-09-13T10:11:24.719 INFO:tasks.rados.rados.0.smithi195.stdout:1181: done (12 left) 2024-09-13T10:11:24.719 INFO:tasks.rados.rados.0.smithi195.stdout:1194: write oid 17 current snap is 0 2024-09-13T10:11:24.719 INFO:tasks.rados.rados.0.smithi195.stdout:1183: finishing write tid 2 to smithi19530714-29 2024-09-13T10:11:24.719 INFO:tasks.rados.rados.0.smithi195.stdout:1183: finishing write tid 3 to smithi19530714-29 2024-09-13T10:11:24.719 INFO:tasks.rados.rados.0.smithi195.stdout:1194: seq_num 223 ranges {547961=554764,1616308=703740,2741092=698600} 2024-09-13T10:11:24.723 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:24.723 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:24.727 INFO:tasks.rados.rados.0.smithi195.stdout:1194: writing smithi19530714-17 from 547961 to 1102725 tid 1 2024-09-13T10:11:24.731 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:24.727+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:24.737 INFO:tasks.rados.rados.0.smithi195.stdout:1194: writing smithi19530714-17 from 1616308 to 2320048 tid 2 2024-09-13T10:11:24.740 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-13T10:11:24.747 INFO:tasks.rados.rados.0.smithi195.stdout:1194: writing smithi19530714-17 from 2741092 to 3439692 tid 3 2024-09-13T10:11:24.747 INFO:tasks.rados.rados.0.smithi195.stdout:1195: read oid 37 snap -1 2024-09-13T10:11:24.747 INFO:tasks.rados.rados.0.smithi195.stdout:1195: expect (ObjNum 145 snap 0 seq_num 145) 2024-09-13T10:11:24.747 INFO:tasks.rados.rados.0.smithi195.stdout:1196: read oid 26 snap -1 2024-09-13T10:11:24.747 INFO:tasks.rados.rados.0.smithi195.stdout:1196: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:24.747 INFO:tasks.rados.rados.0.smithi195.stdout:1197: write oid 8 current snap is 0 2024-09-13T10:11:24.747 INFO:tasks.rados.rados.0.smithi195.stdout:1197: seq_num 224 ranges {564768=630013,1674790=447398,2768515=626225} 2024-09-13T10:11:24.753 INFO:tasks.rados.rados.0.smithi195.stdout:1197: writing smithi19530714-8 from 564768 to 1194781 tid 1 2024-09-13T10:11:24.759 INFO:tasks.rados.rados.0.smithi195.stdout:1197: writing smithi19530714-8 from 1674790 to 2122188 tid 2 2024-09-13T10:11:24.766 INFO:tasks.rados.rados.0.smithi195.stdout:1197: writing smithi19530714-8 from 2768515 to 3394740 tid 3 2024-09-13T10:11:24.766 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:24.767 INFO:tasks.rados.rados.0.smithi195.stdout:1183: finishing write tid 4 to smithi19530714-29 2024-09-13T10:11:24.767 INFO:tasks.rados.rados.0.smithi195.stdout:1183: finishing write tid 5 to smithi19530714-29 2024-09-13T10:11:24.767 INFO:tasks.rados.rados.0.smithi195.stdout:1183: oid 29 updating version 0 to 69 2024-09-13T10:11:24.767 INFO:tasks.rados.rados.0.smithi195.stdout:1183: oid 29 updating version 69 to 71 2024-09-13T10:11:24.767 INFO:tasks.rados.rados.0.smithi195.stdout:1183: oid 29 version 71 is already newer than 68 2024-09-13T10:11:24.767 INFO:tasks.rados.rados.0.smithi195.stdout:1183: oid 29 version 71 is already newer than 70 2024-09-13T10:11:24.767 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 71 (ObjNum 220 snap 0 seq_num 220) dirty exists 2024-09-13T10:11:24.767 INFO:tasks.rados.rados.0.smithi195.stdout:1183: left oid 29 (ObjNum 220 snap 0 seq_num 220) 2024-09-13T10:11:24.767 INFO:tasks.rados.rados.0.smithi195.stdout:1184: expect (ObjNum 174 snap 0 seq_num 174) 2024-09-13T10:11:24.768 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:24.805 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-13T10:11:24.832 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:24.828+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:24.859 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:24.859 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:24.873 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-13T10:11:24.898 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2353340 2024-09-13T10:11:24.909 INFO:tasks.rados.rados.0.smithi195.stdout:1182: got (0) Success 2024-09-13T10:11:24.909 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 0 (ObjNum 207 snap 0 seq_num 207) dirty exists 2024-09-13T10:11:24.909 INFO:tasks.rados.rados.0.smithi195.stdout:1185: finishing set_chunk to oid 50 2024-09-13T10:11:24.909 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 33 (ObjNum 156 snap 0 seq_num 156) dirty exists 2024-09-13T10:11:24.909 INFO:tasks.rados.rados.0.smithi195.stdout:1186: finishing write tid 1 to smithi19530714-27 2024-09-13T10:11:24.909 INFO:tasks.rados.rados.0.smithi195.stdout:1186: finishing write tid 2 to smithi19530714-27 2024-09-13T10:11:24.909 INFO:tasks.rados.rados.0.smithi195.stdout:1187: finishing set_chunk to oid 9 2024-09-13T10:11:24.909 INFO:tasks.rados.rados.0.smithi195.stdout:1187: got expected ENOENT (src dne) 2024-09-13T10:11:24.909 INFO:tasks.rados.rados.0.smithi195.stdout:1186: finishing write tid 3 to smithi19530714-27 2024-09-13T10:11:24.909 INFO:tasks.rados.rados.0.smithi195.stdout:1186: finishing write tid 4 to smithi19530714-27 2024-09-13T10:11:24.909 INFO:tasks.rados.rados.0.smithi195.stdout:1186: finishing write tid 5 to smithi19530714-27 2024-09-13T10:11:24.910 INFO:tasks.rados.rados.0.smithi195.stdout:1186: oid 27 updating version 0 to 106 2024-09-13T10:11:24.910 INFO:tasks.rados.rados.0.smithi195.stdout:1186: oid 27 updating version 106 to 109 2024-09-13T10:11:24.910 INFO:tasks.rados.rados.0.smithi195.stdout:1186: oid 27 version 109 is already newer than 107 2024-09-13T10:11:24.910 INFO:tasks.rados.rados.0.smithi195.stdout:1186: oid 27 version 109 is already newer than 108 2024-09-13T10:11:24.910 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 109 (ObjNum 221 snap 0 seq_num 221) dirty exists 2024-09-13T10:11:24.910 INFO:tasks.rados.rados.0.smithi195.stdout:1186: left oid 27 (ObjNum 221 snap 0 seq_num 221) 2024-09-13T10:11:24.910 INFO:tasks.rados.rados.0.smithi195.stdout:1188: finishing write tid 1 to smithi19530714-3 2024-09-13T10:11:24.910 INFO:tasks.rados.rados.0.smithi195.stdout:1188: finishing write tid 2 to smithi19530714-3 2024-09-13T10:11:24.910 INFO:tasks.rados.rados.0.smithi195.stdout:1188: finishing write tid 3 to smithi19530714-3 2024-09-13T10:11:24.910 INFO:tasks.rados.rados.0.smithi195.stdout:1182: done (15 left) 2024-09-13T10:11:24.910 INFO:tasks.rados.rados.0.smithi195.stdout:1183: done (14 left) 2024-09-13T10:11:24.910 INFO:tasks.rados.rados.0.smithi195.stdout:1184: done (13 left) 2024-09-13T10:11:24.910 INFO:tasks.rados.rados.0.smithi195.stdout:1185: done (12 left) 2024-09-13T10:11:24.911 INFO:tasks.rados.rados.0.smithi195.stdout:1186: done (11 left) 2024-09-13T10:11:24.911 INFO:tasks.rados.rados.0.smithi195.stdout:1187: done (10 left) 2024-09-13T10:11:24.911 INFO:tasks.rados.rados.0.smithi195.stdout:1198: set_chunk oid 27 target oid 2024-09-13T10:11:24.921 INFO:tasks.rados.rados.0.smithi195.stdout:1198: set_chunk oid 27 offset: 2697728 length: 484864 target oid ce698bbfe9e330a63f2288e9e68dce4ae65d50bcda2d3259421a5cd991bce4ce offset: 0 snap -1 2024-09-13T10:11:24.921 INFO:tasks.rados.rados.0.smithi195.stdout:1199: set_chunk oid 23 target oid 2024-09-13T10:11:24.921 INFO:tasks.rados.rados.0.smithi195.stdout:1188: finishing write tid 4 to smithi19530714-3 2024-09-13T10:11:24.921 INFO:tasks.rados.rados.0.smithi195.stdout:1188: finishing write tid 5 to smithi19530714-3 2024-09-13T10:11:24.921 INFO:tasks.rados.rados.0.smithi195.stdout:1189: got (0) Success 2024-09-13T10:11:24.922 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 26 (ObjNum 190 snap 0 seq_num 190) dirty exists 2024-09-13T10:11:24.922 INFO:tasks.rados.rados.0.smithi195.stdout:1188: finishing write tid 6 to smithi19530714-3 2024-09-13T10:11:24.922 INFO:tasks.rados.rados.0.smithi195.stdout:1188: oid 3 updating version 0 to 60 2024-09-13T10:11:24.922 INFO:tasks.rados.rados.0.smithi195.stdout:1188: oid 3 updating version 60 to 61 2024-09-13T10:11:24.922 INFO:tasks.rados.rados.0.smithi195.stdout:1188: oid 3 version 61 is already newer than 59 2024-09-13T10:11:24.922 INFO:tasks.rados.rados.0.smithi195.stdout:1188: oid 3 version 61 is already newer than 58 2024-09-13T10:11:24.922 INFO:tasks.rados.rados.0.smithi195.stdout:1188: oid 3 version 61 is already newer than 57 2024-09-13T10:11:24.922 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 61 (ObjNum 222 snap 0 seq_num 222) dirty exists 2024-09-13T10:11:24.922 INFO:tasks.rados.rados.0.smithi195.stdout:1188: left oid 3 (ObjNum 222 snap 0 seq_num 222) 2024-09-13T10:11:24.922 INFO:tasks.rados.rados.0.smithi195.stdout:1190: expect (ObjNum 203 snap 0 seq_num 203) 2024-09-13T10:11:24.932 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1237300 2024-09-13T10:11:24.932 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:24.930+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:25.033 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:25.028+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:25.036 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:25.036 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:25.051 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-13T10:11:25.078 INFO:tasks.rados.rados.0.smithi195.stdout:1192: expect (ObjNum 180 snap 0 seq_num 180) 2024-09-13T10:11:25.085 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 953023 2024-09-13T10:11:25.133 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 893304 2024-09-13T10:11:25.134 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:25.130+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:25.219 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:25.219 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:25.227 INFO:tasks.rados.rados.0.smithi195.stdout:1193: got (22) Invalid argument 2024-09-13T10:11:25.227 INFO:tasks.rados.rados.0.smithi195.stdout:1194: finishing write tid 1 to smithi19530714-17 2024-09-13T10:11:25.227 INFO:tasks.rados.rados.0.smithi195.stdout:1194: finishing write tid 2 to smithi19530714-17 2024-09-13T10:11:25.227 INFO:tasks.rados.rados.0.smithi195.stdout:1194: finishing write tid 3 to smithi19530714-17 2024-09-13T10:11:25.227 INFO:tasks.rados.rados.0.smithi195.stdout:1194: finishing write tid 4 to smithi19530714-17 2024-09-13T10:11:25.227 INFO:tasks.rados.rados.0.smithi195.stdout:1194: finishing write tid 5 to smithi19530714-17 2024-09-13T10:11:25.227 INFO:tasks.rados.rados.0.smithi195.stdout:1194: oid 17 updating version 0 to 59 2024-09-13T10:11:25.227 INFO:tasks.rados.rados.0.smithi195.stdout:1194: oid 17 version 59 is already newer than 58 2024-09-13T10:11:25.228 INFO:tasks.rados.rados.0.smithi195.stdout:1194: oid 17 version 59 is already newer than 57 2024-09-13T10:11:25.228 INFO:tasks.rados.rados.0.smithi195.stdout:1194: oid 17 version 59 is already newer than 56 2024-09-13T10:11:25.228 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 59 (ObjNum 223 snap 0 seq_num 223) dirty exists 2024-09-13T10:11:25.228 INFO:tasks.rados.rados.0.smithi195.stdout:1194: left oid 17 (ObjNum 223 snap 0 seq_num 223) 2024-09-13T10:11:25.228 INFO:tasks.rados.rados.0.smithi195.stdout:1197: finishing write tid 1 to smithi19530714-8 2024-09-13T10:11:25.228 INFO:tasks.rados.rados.0.smithi195.stdout:1197: finishing write tid 2 to smithi19530714-8 2024-09-13T10:11:25.233 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-13T10:11:25.234 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:25.228+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:25.276 INFO:tasks.rados.rados.0.smithi195.stdout:1199: set_chunk oid 23 offset: 68608 length: 3249664 target oid c4df49d87a472c2ff73e2c64956418917110feb34c999f63d2bffb08c67f5c40 offset: 0 snap -1 2024-09-13T10:11:25.276 INFO:tasks.rados.rados.0.smithi195.stdout:1188: done (11 left) 2024-09-13T10:11:25.276 INFO:tasks.rados.rados.0.smithi195.stdout:1189: done (10 left) 2024-09-13T10:11:25.276 INFO:tasks.rados.rados.0.smithi195.stdout:1190: done (9 left) 2024-09-13T10:11:25.276 INFO:tasks.rados.rados.0.smithi195.stdout:1191: done (8 left) 2024-09-13T10:11:25.276 INFO:tasks.rados.rados.0.smithi195.stdout:1192: done (7 left) 2024-09-13T10:11:25.277 INFO:tasks.rados.rados.0.smithi195.stdout:1193: done (6 left) 2024-09-13T10:11:25.277 INFO:tasks.rados.rados.0.smithi195.stdout:1194: done (5 left) 2024-09-13T10:11:25.277 INFO:tasks.rados.rados.0.smithi195.stdout:1200: set_chunk oid 13 target oid 2024-09-13T10:11:25.283 INFO:tasks.rados.rados.0.smithi195.stdout:1200: set_chunk oid 13 offset: 2873344 length: 44032 target oid f65e8e8531efa479e302f4f377acb27c20864c743f37e61fe874439d7799c74d offset: 0 snap -1 2024-09-13T10:11:25.283 INFO:tasks.rados.rados.0.smithi195.stdout:1201: read oid 40 snap -1 2024-09-13T10:11:25.284 INFO:tasks.rados.rados.0.smithi195.stdout:1201: expect (ObjNum 203 snap 0 seq_num 203) 2024-09-13T10:11:25.284 INFO:tasks.rados.rados.0.smithi195.stdout:1202: read oid 36 snap -1 2024-09-13T10:11:25.284 INFO:tasks.rados.rados.0.smithi195.stdout:1202: expect (ObjNum 190 snap 0 seq_num 190) 2024-09-13T10:11:25.284 INFO:tasks.rados.rados.0.smithi195.stdout:1203: write oid 32 current snap is 0 2024-09-13T10:11:25.284 INFO:tasks.rados.rados.0.smithi195.stdout:1203: seq_num 225 ranges {581575=705262,1733272=707409,3195938=437497} 2024-09-13T10:11:25.296 INFO:tasks.rados.rados.0.smithi195.stdout:1203: writing smithi19530714-32 from 581575 to 1286837 tid 1 2024-09-13T10:11:25.307 INFO:tasks.rados.rados.0.smithi195.stdout:1203: writing smithi19530714-32 from 1733272 to 2440681 tid 2 2024-09-13T10:11:25.319 INFO:tasks.rados.rados.0.smithi195.stdout:1203: writing smithi19530714-32 from 3195938 to 3633435 tid 3 2024-09-13T10:11:25.320 INFO:tasks.rados.rados.0.smithi195.stdout:1204: read oid 29 snap -1 2024-09-13T10:11:25.320 INFO:tasks.rados.rados.0.smithi195.stdout:1204: expect (ObjNum 220 snap 0 seq_num 220) 2024-09-13T10:11:25.320 INFO:tasks.rados.rados.0.smithi195.stdout:1205: read oid 34 snap -1 2024-09-13T10:11:25.320 INFO:tasks.rados.rados.0.smithi195.stdout:1205: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:25.320 INFO:tasks.rados.rados.0.smithi195.stdout:1206: tier_promote oid 42 2024-09-13T10:11:25.320 INFO:tasks.rados.rados.0.smithi195.stdout:1195: expect (ObjNum 145 snap 0 seq_num 145) 2024-09-13T10:11:25.334 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:25.332+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:25.356 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1896803 2024-09-13T10:11:25.357 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:25.357 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:25.371 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T10:11:25.375 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:25.436 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:25.432+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:25.495 INFO:tasks.rados.rados.0.smithi195.stdout:1196: expect (ObjNum 122 snap 0 seq_num 122) 2024-09-13T10:11:25.495 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:25.505 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:25.505 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:25.519 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T10:11:25.537 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:25.561 INFO:tasks.rados.rados.0.smithi195.stdout:1197: finishing write tid 3 to smithi19530714-8 2024-09-13T10:11:25.561 INFO:tasks.rados.rados.0.smithi195.stdout:1197: finishing write tid 4 to smithi19530714-8 2024-09-13T10:11:25.561 INFO:tasks.rados.rados.0.smithi195.stdout:1197: finishing write tid 5 to smithi19530714-8 2024-09-13T10:11:25.561 INFO:tasks.rados.rados.0.smithi195.stdout:1197: oid 8 updating version 0 to 12 2024-09-13T10:11:25.561 INFO:tasks.rados.rados.0.smithi195.stdout:1197: oid 8 version 12 is already newer than 11 2024-09-13T10:11:25.561 INFO:tasks.rados.rados.0.smithi195.stdout:1197: oid 8 updating version 12 to 14 2024-09-13T10:11:25.561 INFO:tasks.rados.rados.0.smithi195.stdout:1197: oid 8 version 14 is already newer than 13 2024-09-13T10:11:25.561 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 14 (ObjNum 224 snap 0 seq_num 224) dirty exists 2024-09-13T10:11:25.561 INFO:tasks.rados.rados.0.smithi195.stdout:1197: left oid 8 (ObjNum 224 snap 0 seq_num 224) 2024-09-13T10:11:25.562 INFO:tasks.rados.rados.0.smithi195.stdout:1198: finishing set_chunk to oid 27 2024-09-13T10:11:25.562 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 109 (ObjNum 221 snap 0 seq_num 221) dirty exists 2024-09-13T10:11:25.562 INFO:tasks.rados.rados.0.smithi195.stdout:1199: finishing set_chunk to oid 23 2024-09-13T10:11:25.562 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 34 (ObjNum 206 snap 0 seq_num 206) dirty exists 2024-09-13T10:11:25.562 INFO:tasks.rados.rados.0.smithi195.stdout:1200: finishing set_chunk to oid 13 2024-09-13T10:11:25.562 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 13 set_chunk returned error code -95 offset: 2873344 length: 44032 2024-09-13T10:11:25.562 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 0 (ObjNum 187 snap 0 seq_num 187) dirty exists 2024-09-13T10:11:25.562 INFO:tasks.rados.rados.0.smithi195.stdout:1201: expect (ObjNum 203 snap 0 seq_num 203) 2024-09-13T10:11:25.568 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:25.567+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:25.595 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3010653 2024-09-13T10:11:25.639 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:25.635+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:25.642 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:25.642 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:25.657 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T10:11:25.718 INFO:tasks.rados.rados.0.smithi195.stdout:1202: expect (ObjNum 190 snap 0 seq_num 190) 2024-09-13T10:11:25.744 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:25.743+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:25.782 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:25.812 INFO:teuthology.orchestra.run.smithi073.stderr:set 1.3 pg_upmap mapping to [4,9,8] 2024-09-13T10:11:25.816 INFO:tasks.rados.rados.0.smithi195.stdout:1203: finishing write tid 1 to smithi19530714-32 2024-09-13T10:11:25.816 INFO:tasks.rados.rados.0.smithi195.stdout:1203: finishing write tid 2 to smithi19530714-32 2024-09-13T10:11:25.816 INFO:tasks.rados.rados.0.smithi195.stdout:1203: finishing write tid 3 to smithi19530714-32 2024-09-13T10:11:25.816 INFO:tasks.rados.rados.0.smithi195.stdout:1203: finishing write tid 4 to smithi19530714-32 2024-09-13T10:11:25.816 INFO:tasks.rados.rados.0.smithi195.stdout:1203: finishing write tid 5 to smithi19530714-32 2024-09-13T10:11:25.816 INFO:tasks.rados.rados.0.smithi195.stdout:1203: oid 32 updating version 0 to 62 2024-09-13T10:11:25.817 INFO:tasks.rados.rados.0.smithi195.stdout:1203: oid 32 updating version 62 to 63 2024-09-13T10:11:25.817 INFO:tasks.rados.rados.0.smithi195.stdout:1203: oid 32 version 63 is already newer than 61 2024-09-13T10:11:25.817 INFO:tasks.rados.rados.0.smithi195.stdout:1203: oid 32 version 63 is already newer than 60 2024-09-13T10:11:25.817 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 63 (ObjNum 225 snap 0 seq_num 225) dirty exists 2024-09-13T10:11:25.817 INFO:tasks.rados.rados.0.smithi195.stdout:1203: left oid 32 (ObjNum 225 snap 0 seq_num 225) 2024-09-13T10:11:25.817 INFO:tasks.rados.rados.0.smithi195.stdout:1205: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:25.834 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:25.835 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:25.840 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:25.838+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:25.849 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T10:11:25.937 INFO:tasks.rados.rados.0.smithi195.stdout:1206: got (0) Success 2024-09-13T10:11:25.937 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 6 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-13T10:11:25.937 INFO:tasks.rados.rados.0.smithi195.stdout:1204: expect (ObjNum 220 snap 0 seq_num 220) 2024-09-13T10:11:25.938 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:25.941 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:25.938+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:25.974 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:11:26.010 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:26.010 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:26.014 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1263754 2024-09-13T10:11:26.024 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T10:11:26.042 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:26.038+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:26.044 INFO:tasks.rados.rados.0.smithi195.stdout:1195: done (11 left) 2024-09-13T10:11:26.044 INFO:tasks.rados.rados.0.smithi195.stdout:1196: done (10 left) 2024-09-13T10:11:26.044 INFO:tasks.rados.rados.0.smithi195.stdout:1197: done (9 left) 2024-09-13T10:11:26.044 INFO:tasks.rados.rados.0.smithi195.stdout:1198: done (8 left) 2024-09-13T10:11:26.044 INFO:tasks.rados.rados.0.smithi195.stdout:1199: done (7 left) 2024-09-13T10:11:26.045 INFO:tasks.rados.rados.0.smithi195.stdout:1200: done (6 left) 2024-09-13T10:11:26.045 INFO:tasks.rados.rados.0.smithi195.stdout:1201: done (5 left) 2024-09-13T10:11:26.045 INFO:tasks.rados.rados.0.smithi195.stdout:1202: done (4 left) 2024-09-13T10:11:26.045 INFO:tasks.rados.rados.0.smithi195.stdout:1203: done (3 left) 2024-09-13T10:11:26.045 INFO:tasks.rados.rados.0.smithi195.stdout:1204: done (2 left) 2024-09-13T10:11:26.045 INFO:tasks.rados.rados.0.smithi195.stdout:1205: done (1 left) 2024-09-13T10:11:26.045 INFO:tasks.rados.rados.0.smithi195.stdout:1206: done (0 left) 2024-09-13T10:11:26.045 INFO:tasks.rados.rados.0.smithi195.stdout:1207: read oid 37 snap -1 2024-09-13T10:11:26.045 INFO:tasks.rados.rados.0.smithi195.stdout:1207: expect (ObjNum 145 snap 0 seq_num 145) 2024-09-13T10:11:26.045 INFO:tasks.rados.rados.0.smithi195.stdout:1208: read oid 2 snap -1 2024-09-13T10:11:26.045 INFO:tasks.rados.rados.0.smithi195.stdout:1208: expect (ObjNum 176 snap 0 seq_num 176) 2024-09-13T10:11:26.046 INFO:tasks.rados.rados.0.smithi195.stdout:1209: write (excl) oid 8 current snap is 0 2024-09-13T10:11:26.046 INFO:tasks.rados.rados.0.smithi195.stdout:1209: seq_num 226 ranges {598382=780511,2075401=451067,3107008=648769} 2024-09-13T10:11:26.066 INFO:tasks.rados.rados.0.smithi195.stdout:1209: writing smithi19530714-8 from 598382 to 1378893 tid 1 2024-09-13T10:11:26.085 INFO:tasks.rados.rados.0.smithi195.stdout:1209: writing smithi19530714-8 from 2075401 to 2526468 tid 2 2024-09-13T10:11:26.102 INFO:tasks.rados.rados.0.smithi195.stdout:1209: writing smithi19530714-8 from 3107008 to 3755777 tid 3 2024-09-13T10:11:26.103 INFO:tasks.rados.rados.0.smithi195.stdout:1210: write oid 31 current snap is 0 2024-09-13T10:11:26.103 INFO:tasks.rados.rados.0.smithi195.stdout:1210: seq_num 227 ranges {615189=455760,1733883=711078,3134431=460041} 2024-09-13T10:11:26.114 INFO:tasks.rados.rados.0.smithi195.stdout:1210: writing smithi19530714-31 from 615189 to 1070949 tid 1 2024-09-13T10:11:26.126 INFO:tasks.rados.rados.0.smithi195.stdout:1210: writing smithi19530714-31 from 1733883 to 2444961 tid 2 2024-09-13T10:11:26.131 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:26.132 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:26.135 INFO:tasks.rados.rados.0.smithi195.stdout:1210: writing smithi19530714-31 from 3134431 to 3594472 tid 3 2024-09-13T10:11:26.136 INFO:tasks.rados.rados.0.smithi195.stdout:1211: write oid 13 current snap is 0 2024-09-13T10:11:26.136 INFO:tasks.rados.rados.0.smithi195.stdout:1211: seq_num 228 ranges {631996=531009,1792365=454736,2761854=671313} 2024-09-13T10:11:26.143 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:26.140+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:26.143 INFO:tasks.rados.rados.0.smithi195.stdout:1211: writing smithi19530714-13 from 631996 to 1163005 tid 1 2024-09-13T10:11:26.151 INFO:tasks.rados.rados.0.smithi195.stdout:1211: writing smithi19530714-13 from 1792365 to 2247101 tid 2 2024-09-13T10:11:26.152 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:26.152 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:26.156 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:26.156 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:26.160 INFO:tasks.rados.rados.0.smithi195.stdout:1211: writing smithi19530714-13 from 2761854 to 3433167 tid 3 2024-09-13T10:11:26.160 INFO:tasks.rados.rados.0.smithi195.stdout:1212: set_chunk oid 12 target oid 2024-09-13T10:11:26.160 INFO:tasks.rados.rados.0.smithi195.stdout:1212: set_chunk oid 12 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:26.160 INFO:tasks.rados.rados.0.smithi195.stdout:1213: tier_evict oid 3 2024-09-13T10:11:26.161 INFO:tasks.rados.rados.0.smithi195.stdout:1213: tier_evict oid 3 snap -1 2024-09-13T10:11:26.161 INFO:tasks.rados.rados.0.smithi195.stdout:1214: read oid 24 snap -1 2024-09-13T10:11:26.161 INFO:tasks.rados.rados.0.smithi195.stdout:1214: expect (ObjNum 121 snap 0 seq_num 121) 2024-09-13T10:11:26.161 INFO:tasks.rados.rados.0.smithi195.stdout:1215: set_chunk oid 33 target oid 2024-09-13T10:11:26.161 INFO:tasks.rados.rados.0.smithi195.stdout:1215: set_chunk oid 33 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:26.161 INFO:tasks.rados.rados.0.smithi195.stdout:1216: write oid 6 current snap is 0 2024-09-13T10:11:26.162 INFO:tasks.rados.rados.0.smithi195.stdout:1207: expect (ObjNum 145 snap 0 seq_num 145) 2024-09-13T10:11:26.162 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:26.175 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:26.176 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:26.177 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T10:11:26.194 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:26.195 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:26.211 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:26.212 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:26.233 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:26.233 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:26.244 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:26.240+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:26.252 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:26.252 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:26.274 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:26.274 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:26.291 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:26.292 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:26.292 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:26.301 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:26.302 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:11:26.302 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:11:26.302 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:11:26.302 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.302 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:251 4.5 4:a71ca386:::smithi19530714-2:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.302 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:11:26.302 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.180296357000003, 2024-09-13T10:11:26.302 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041813326999999997, 2024-09-13T10:11:26.302 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.302 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.302 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.303 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.303 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.303 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.303 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 251 2024-09-13T10:11:26.303 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.303 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.303 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.303 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.303 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:11:26.303 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.303 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.303 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.303 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.304 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:11:26.304 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.304 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.304 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.304 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.304 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117990+0000", 2024-09-13T10:11:26.304 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.203e-06 2024-09-13T10:11:26.304 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.304 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.305 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.305 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117997+0000", 2024-09-13T10:11:26.305 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9750000000000001e-06 2024-09-13T10:11:26.305 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.305 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.305 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.305 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118009+0000", 2024-09-13T10:11:26.305 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2478e-05 2024-09-13T10:11:26.305 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.305 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.305 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.305 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118017+0000", 2024-09-13T10:11:26.306 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6909999999999997e-06 2024-09-13T10:11:26.306 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.306 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.306 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.306 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118106+0000", 2024-09-13T10:11:26.306 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.9331000000000002e-05 2024-09-13T10:11:26.306 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.306 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.306 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.306 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118184+0000", 2024-09-13T10:11:26.306 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7448999999999997e-05 2024-09-13T10:11:26.306 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.306 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.307 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,10", 2024-09-13T10:11:26.307 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158604+0000", 2024-09-13T10:11:26.307 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.040419954000000001 2024-09-13T10:11:26.307 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.307 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.307 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.307 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159306+0000", 2024-09-13T10:11:26.307 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00070229299999999999 2024-09-13T10:11:26.307 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.307 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.307 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.307 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159539+0000", 2024-09-13T10:11:26.307 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023353600000000001 2024-09-13T10:11:26.308 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.308 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.308 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.308 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159696+0000", 2024-09-13T10:11:26.308 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015616 2024-09-13T10:11:26.308 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.308 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.308 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.308 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159754+0000", 2024-09-13T10:11:26.308 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8513e-05 2024-09-13T10:11:26.308 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.308 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.308 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.309 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159802+0000", 2024-09-13T10:11:26.309 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.7744000000000002e-05 2024-09-13T10:11:26.309 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.309 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.309 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.309 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.309 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.309 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:270 4.5 4:a59e88f7:::smithi19530714-21:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.309 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:11:26.309 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.072250230999998, 2024-09-13T10:11:26.309 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041472594000000002, 2024-09-13T10:11:26.309 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.310 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.310 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.310 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.310 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.310 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.310 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 270 2024-09-13T10:11:26.310 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.310 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.310 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.310 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.310 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:11:26.310 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.310 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.311 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.311 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.311 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:11:26.311 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.311 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.311 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.311 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.311 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226037+0000", 2024-09-13T10:11:26.311 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6560000000000001e-06 2024-09-13T10:11:26.311 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.311 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.311 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.311 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226048+0000", 2024-09-13T10:11:26.311 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0893000000000001e-05 2024-09-13T10:11:26.312 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.312 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.312 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.312 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226079+0000", 2024-09-13T10:11:26.312 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0412999999999998e-05 2024-09-13T10:11:26.312 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.312 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.312 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.312 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226106+0000", 2024-09-13T10:11:26.312 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7160000000000001e-05 2024-09-13T10:11:26.312 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.312 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.313 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.313 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226234+0000", 2024-09-13T10:11:26.313 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000128393 2024-09-13T10:11:26.313 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.313 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.313 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.313 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226388+0000", 2024-09-13T10:11:26.313 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015424300000000001 2024-09-13T10:11:26.313 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.313 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.313 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,10", 2024-09-13T10:11:26.313 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262647+0000", 2024-09-13T10:11:26.313 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036258460999999999 2024-09-13T10:11:26.314 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.314 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.314 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.314 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264690+0000", 2024-09-13T10:11:26.314 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020429889999999998 2024-09-13T10:11:26.314 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.314 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.314 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.314 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.266720+0000", 2024-09-13T10:11:26.314 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002029669 2024-09-13T10:11:26.314 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.314 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.315 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.315 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267350+0000", 2024-09-13T10:11:26.315 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00063076699999999996 2024-09-13T10:11:26.315 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.315 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.315 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.315 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267409+0000", 2024-09-13T10:11:26.315 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8366000000000003e-05 2024-09-13T10:11:26.315 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.315 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.316 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.316 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267507+0000", 2024-09-13T10:11:26.316 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.8584000000000003e-05 2024-09-13T10:11:26.316 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.316 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.316 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.316 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.316 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.316 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:275 4.2 4:4ac406d9:::smithi19530714-26:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.316 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:11:26.317 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.068503167999999, 2024-09-13T10:11:26.317 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024603505000000001, 2024-09-13T10:11:26.317 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.317 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.317 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.317 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.317 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.317 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.317 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 275 2024-09-13T10:11:26.317 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.317 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.317 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.317 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.318 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:11:26.318 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.318 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.318 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.318 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.318 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:11:26.318 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.318 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.318 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.318 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.318 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229783+0000", 2024-09-13T10:11:26.318 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.44e-06 2024-09-13T10:11:26.318 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.319 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.319 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.319 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229793+0000", 2024-09-13T10:11:26.319 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0163000000000001e-05 2024-09-13T10:11:26.319 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.319 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.319 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.319 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229814+0000", 2024-09-13T10:11:26.319 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0564000000000002e-05 2024-09-13T10:11:26.319 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.319 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.319 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.319 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229831+0000", 2024-09-13T10:11:26.320 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7142e-05 2024-09-13T10:11:26.320 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.320 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.320 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.320 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229949+0000", 2024-09-13T10:11:26.320 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000118014 2024-09-13T10:11:26.320 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.320 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.320 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.320 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230081+0000", 2024-09-13T10:11:26.320 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013181299999999999 2024-09-13T10:11:26.320 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.320 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.321 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,14", 2024-09-13T10:11:26.321 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.236541+0000", 2024-09-13T10:11:26.321 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0064597680000000003 2024-09-13T10:11:26.321 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.321 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.321 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.321 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.238773+0000", 2024-09-13T10:11:26.321 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022326680000000002 2024-09-13T10:11:26.321 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.321 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.321 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.321 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.241978+0000", 2024-09-13T10:11:26.321 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00320455 2024-09-13T10:11:26.322 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.322 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.322 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.322 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254159+0000", 2024-09-13T10:11:26.322 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012181149 2024-09-13T10:11:26.322 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.322 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.322 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.322 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254259+0000", 2024-09-13T10:11:26.322 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000100184 2024-09-13T10:11:26.322 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.322 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.322 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.323 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254385+0000", 2024-09-13T10:11:26.323 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012605 2024-09-13T10:11:26.323 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.323 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.323 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.323 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.323 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.323 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:270 4.5 4:a59e88f7:::smithi19530714-21:head [assert-version v9,set-chunk] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.323 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:11:26.323 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.064848738000002, 2024-09-13T10:11:26.323 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033985986000000003, 2024-09-13T10:11:26.323 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.323 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.324 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "waiting for ondisk", 2024-09-13T10:11:26.324 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.324 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.324 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.324 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 270 2024-09-13T10:11:26.324 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.324 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.324 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.324 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.324 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:11:26.324 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.324 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.325 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.325 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.325 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:11:26.325 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.325 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.325 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.325 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.325 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233438+0000", 2024-09-13T10:11:26.325 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.187e-06 2024-09-13T10:11:26.326 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.326 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.326 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.326 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233449+0000", 2024-09-13T10:11:26.326 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0926e-05 2024-09-13T10:11:26.326 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.326 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.326 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.326 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233477+0000", 2024-09-13T10:11:26.326 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7515999999999999e-05 2024-09-13T10:11:26.326 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.326 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.327 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.327 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233499+0000", 2024-09-13T10:11:26.327 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1719999999999999e-05 2024-09-13T10:11:26.327 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.327 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.327 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.327 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233670+0000", 2024-09-13T10:11:26.327 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000171146 2024-09-13T10:11:26.327 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.327 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.327 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for blocked object", 2024-09-13T10:11:26.327 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233823+0000", 2024-09-13T10:11:26.328 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015309300000000001 2024-09-13T10:11:26.328 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.328 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.328 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.328 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.263191+0000", 2024-09-13T10:11:26.328 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029368157999999998 2024-09-13T10:11:26.328 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.328 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.328 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for ondisk", 2024-09-13T10:11:26.328 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.263301+0000", 2024-09-13T10:11:26.328 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110214 2024-09-13T10:11:26.328 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.329 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.329 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.329 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267422+0000", 2024-09-13T10:11:26.329 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0041210259999999999 2024-09-13T10:11:26.329 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.329 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.329 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.329 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.329 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.329 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:275 4.2 4:4ac406d9:::smithi19530714-26:head [assert-version v3,set-chunk] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.329 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.233895+0000", 2024-09-13T10:11:26.329 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.064390351999997, 2024-09-13T10:11:26.330 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020388462, 2024-09-13T10:11:26.330 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.330 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.330 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "waiting for ondisk", 2024-09-13T10:11:26.330 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.330 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.330 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.330 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 275 2024-09-13T10:11:26.330 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.330 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.330 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.330 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.330 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233895+0000", 2024-09-13T10:11:26.331 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.331 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.331 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.331 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.331 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233895+0000", 2024-09-13T10:11:26.331 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.331 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.331 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.331 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.331 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233896+0000", 2024-09-13T10:11:26.331 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3230000000000001e-06 2024-09-13T10:11:26.331 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.332 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.332 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.332 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233904+0000", 2024-09-13T10:11:26.332 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6930000000000007e-06 2024-09-13T10:11:26.332 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.332 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.332 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.332 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233921+0000", 2024-09-13T10:11:26.332 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7739000000000001e-05 2024-09-13T10:11:26.332 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.332 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.332 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.332 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233937+0000", 2024-09-13T10:11:26.333 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5444e-05 2024-09-13T10:11:26.333 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.333 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.333 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.333 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234067+0000", 2024-09-13T10:11:26.333 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013029 2024-09-13T10:11:26.333 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.333 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.333 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for blocked object", 2024-09-13T10:11:26.333 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234173+0000", 2024-09-13T10:11:26.334 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010631 2024-09-13T10:11:26.334 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.334 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.334 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.334 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.237380+0000", 2024-09-13T10:11:26.334 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0032067369999999999 2024-09-13T10:11:26.334 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.334 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.334 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for ondisk", 2024-09-13T10:11:26.334 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.237533+0000", 2024-09-13T10:11:26.334 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000152748 2024-09-13T10:11:26.335 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.335 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.335 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.335 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254283+0000", 2024-09-13T10:11:26.335 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016750178000000001 2024-09-13T10:11:26.335 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.335 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.335 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.335 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.335 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.335 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:277 4.4 4:20c7201f:::smithi19530714-28:head [assert-version v9,set-chunk] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.335 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:11:26.335 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.064242653000001, 2024-09-13T10:11:26.336 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.042430906999999997, 2024-09-13T10:11:26.336 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.336 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.336 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.336 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.336 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.336 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.336 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 277 2024-09-13T10:11:26.336 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.336 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.336 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.336 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.336 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:11:26.337 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.337 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.337 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.337 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.337 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:11:26.337 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.337 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.337 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.337 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.337 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234043+0000", 2024-09-13T10:11:26.337 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.13e-06 2024-09-13T10:11:26.337 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.338 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.338 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.338 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234050+0000", 2024-09-13T10:11:26.338 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.263e-06 2024-09-13T10:11:26.338 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.338 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.338 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.338 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234065+0000", 2024-09-13T10:11:26.338 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5207e-05 2024-09-13T10:11:26.338 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.338 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.338 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.338 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234078+0000", 2024-09-13T10:11:26.339 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3643e-05 2024-09-13T10:11:26.339 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.339 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.339 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.339 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.235383+0000", 2024-09-13T10:11:26.339 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0013048269999999999 2024-09-13T10:11:26.339 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.339 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.339 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.339 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.235604+0000", 2024-09-13T10:11:26.339 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022116900000000001 2024-09-13T10:11:26.339 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.339 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.339 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:26.340 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.271455+0000", 2024-09-13T10:11:26.340 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035850173999999999 2024-09-13T10:11:26.340 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.340 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.340 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.340 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273083+0000", 2024-09-13T10:11:26.340 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0016287649999999999 2024-09-13T10:11:26.340 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.340 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.340 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.340 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.275756+0000", 2024-09-13T10:11:26.340 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0026721140000000002 2024-09-13T10:11:26.341 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.341 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.341 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.341 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276255+0000", 2024-09-13T10:11:26.341 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00049961600000000004 2024-09-13T10:11:26.341 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.341 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.341 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.341 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276401+0000", 2024-09-13T10:11:26.341 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014584099999999999 2024-09-13T10:11:26.341 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.341 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.341 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.342 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276473+0000", 2024-09-13T10:11:26.342 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2157999999999996e-05 2024-09-13T10:11:26.342 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.342 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.342 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.342 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.342 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.342 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:294 4.4 4:25c904b0:::smithi19530714-45:head [assert-version v15,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.342 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.260461+0000", 2024-09-13T10:11:26.342 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.037823824999997, 2024-09-13T10:11:26.342 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020509151, 2024-09-13T10:11:26.342 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.342 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.343 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.343 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.343 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.343 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.343 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 294 2024-09-13T10:11:26.343 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.343 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.343 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.343 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.343 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260461+0000", 2024-09-13T10:11:26.343 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.343 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.343 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.343 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.344 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260461+0000", 2024-09-13T10:11:26.344 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.344 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.344 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.344 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.344 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260464+0000", 2024-09-13T10:11:26.344 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4279999999999999e-06 2024-09-13T10:11:26.344 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.344 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.344 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.344 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260474+0000", 2024-09-13T10:11:26.344 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.031e-05 2024-09-13T10:11:26.345 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.345 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.345 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.345 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260494+0000", 2024-09-13T10:11:26.345 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9894e-05 2024-09-13T10:11:26.345 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.345 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.345 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.345 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260509+0000", 2024-09-13T10:11:26.345 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5135e-05 2024-09-13T10:11:26.345 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.345 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.345 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.346 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260654+0000", 2024-09-13T10:11:26.346 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014543100000000001 2024-09-13T10:11:26.346 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.346 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.346 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.346 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260980+0000", 2024-09-13T10:11:26.346 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00032611599999999999 2024-09-13T10:11:26.346 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.346 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.346 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:26.346 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.274012+0000", 2024-09-13T10:11:26.346 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013031205000000001 2024-09-13T10:11:26.346 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.347 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.347 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.347 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.275836+0000", 2024-09-13T10:11:26.347 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0018243630000000001 2024-09-13T10:11:26.347 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.347 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.347 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.347 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280160+0000", 2024-09-13T10:11:26.347 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0043236020000000002 2024-09-13T10:11:26.347 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.347 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.347 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.347 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280818+0000", 2024-09-13T10:11:26.347 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00065824600000000003 2024-09-13T10:11:26.348 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.348 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.348 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.348 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280897+0000", 2024-09-13T10:11:26.348 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9623000000000007e-05 2024-09-13T10:11:26.348 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.348 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.348 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.348 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280970+0000", 2024-09-13T10:11:26.348 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2798000000000003e-05 2024-09-13T10:11:26.348 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.348 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.349 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.349 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.349 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.349 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:298 4.4 4:24da8222:::smithi19530714-49:head [assert-version v18,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.349 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.262102+0000", 2024-09-13T10:11:26.349 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.036183268999999, 2024-09-13T10:11:26.349 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023401737999999998, 2024-09-13T10:11:26.349 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.349 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.349 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.349 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.349 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.350 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.350 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 298 2024-09-13T10:11:26.350 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.350 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.350 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.350 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.350 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262102+0000", 2024-09-13T10:11:26.350 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.350 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.350 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.350 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.350 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262102+0000", 2024-09-13T10:11:26.350 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.351 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.351 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.351 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.351 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262103+0000", 2024-09-13T10:11:26.351 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7719999999999999e-06 2024-09-13T10:11:26.351 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.351 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.351 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.351 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262112+0000", 2024-09-13T10:11:26.351 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.242e-06 2024-09-13T10:11:26.351 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.351 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.351 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.352 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262130+0000", 2024-09-13T10:11:26.352 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8057999999999999e-05 2024-09-13T10:11:26.352 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.352 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.352 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.352 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262141+0000", 2024-09-13T10:11:26.352 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0893000000000001e-05 2024-09-13T10:11:26.352 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.352 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.352 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.352 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262226+0000", 2024-09-13T10:11:26.352 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5694999999999999e-05 2024-09-13T10:11:26.352 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.353 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.353 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.353 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262321+0000", 2024-09-13T10:11:26.353 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.4394999999999993e-05 2024-09-13T10:11:26.353 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.353 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.353 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:26.353 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.274901+0000", 2024-09-13T10:11:26.353 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012580071999999999 2024-09-13T10:11:26.353 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.353 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.353 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.353 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276773+0000", 2024-09-13T10:11:26.354 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0018726579999999999 2024-09-13T10:11:26.354 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.354 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.354 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.354 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283725+0000", 2024-09-13T10:11:26.354 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0069512460000000003 2024-09-13T10:11:26.354 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.354 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.354 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.354 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.285294+0000", 2024-09-13T10:11:26.354 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0015694859999999999 2024-09-13T10:11:26.354 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.354 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.355 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.355 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.285420+0000", 2024-09-13T10:11:26.355 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012607699999999999 2024-09-13T10:11:26.355 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.355 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.355 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.355 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.285503+0000", 2024-09-13T10:11:26.355 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.3144000000000002e-05 2024-09-13T10:11:26.355 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.355 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.355 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.355 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.355 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.355 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:324 4.4 4:26b855f3:::smithi19530714-25:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.356 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:11:26.356 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.017580406999997, 2024-09-13T10:11:26.356 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033888489000000001, 2024-09-13T10:11:26.356 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.356 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.356 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.356 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.356 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.356 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.356 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 324 2024-09-13T10:11:26.357 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.357 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.357 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.357 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.357 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:11:26.357 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.357 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.357 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.357 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.357 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:11:26.357 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.357 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.358 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.358 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.358 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280706+0000", 2024-09-13T10:11:26.358 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6479999999999999e-06 2024-09-13T10:11:26.358 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.358 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.358 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.358 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280715+0000", 2024-09-13T10:11:26.358 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1500000000000005e-06 2024-09-13T10:11:26.358 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.358 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.358 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.358 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280733+0000", 2024-09-13T10:11:26.359 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8168e-05 2024-09-13T10:11:26.359 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.359 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.359 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.359 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280747+0000", 2024-09-13T10:11:26.359 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3661e-05 2024-09-13T10:11:26.359 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.359 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.359 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.359 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281013+0000", 2024-09-13T10:11:26.359 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026565099999999999 2024-09-13T10:11:26.359 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.359 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.360 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.360 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281124+0000", 2024-09-13T10:11:26.360 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000111715 2024-09-13T10:11:26.360 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.360 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.360 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:26.360 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286502+0000", 2024-09-13T10:11:26.360 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0053775259999999997 2024-09-13T10:11:26.360 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.360 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.360 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.360 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.289334+0000", 2024-09-13T10:11:26.360 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0028316779999999998 2024-09-13T10:11:26.361 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.361 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.361 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.361 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.290331+0000", 2024-09-13T10:11:26.361 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0009974929999999999 2024-09-13T10:11:26.361 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.361 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.361 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.361 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314453+0000", 2024-09-13T10:11:26.361 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024121417999999999 2024-09-13T10:11:26.361 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.361 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.361 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.362 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314531+0000", 2024-09-13T10:11:26.362 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8841e-05 2024-09-13T10:11:26.362 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.362 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.362 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.362 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314593+0000", 2024-09-13T10:11:26.362 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.1539999999999997e-05 2024-09-13T10:11:26.362 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.362 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.362 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.362 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.362 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.362 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:327 4.4 4:20c7201f:::smithi19530714-28:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.363 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:11:26.363 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.017112578000003, 2024-09-13T10:11:26.363 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035855512999999999, 2024-09-13T10:11:26.363 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.363 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.363 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.363 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.363 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.363 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.363 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 327 2024-09-13T10:11:26.363 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.363 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.364 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.364 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.364 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:11:26.364 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.364 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.364 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.364 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.364 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:11:26.364 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.364 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.364 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.364 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.365 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281173+0000", 2024-09-13T10:11:26.365 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0020000000000001e-06 2024-09-13T10:11:26.365 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.365 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.365 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.365 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281180+0000", 2024-09-13T10:11:26.365 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.8260000000000004e-06 2024-09-13T10:11:26.365 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.365 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.365 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.365 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281192+0000", 2024-09-13T10:11:26.366 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1725e-05 2024-09-13T10:11:26.366 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.366 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.366 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.366 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281202+0000", 2024-09-13T10:11:26.366 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0492000000000001e-05 2024-09-13T10:11:26.366 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.366 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.366 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.367 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281396+0000", 2024-09-13T10:11:26.367 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019404200000000001 2024-09-13T10:11:26.367 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.367 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.367 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.367 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281533+0000", 2024-09-13T10:11:26.368 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000136267 2024-09-13T10:11:26.368 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.368 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.368 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:26.368 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.313834+0000", 2024-09-13T10:11:26.368 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032301491000000002 2024-09-13T10:11:26.368 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.368 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.369 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.369 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314993+0000", 2024-09-13T10:11:26.369 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001158585 2024-09-13T10:11:26.369 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.369 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.369 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.369 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.315097+0000", 2024-09-13T10:11:26.369 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000104685 2024-09-13T10:11:26.369 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.369 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.369 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.369 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.316808+0000", 2024-09-13T10:11:26.369 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0017110529999999999 2024-09-13T10:11:26.370 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.370 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.370 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.370 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.316946+0000", 2024-09-13T10:11:26.370 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000137737 2024-09-13T10:11:26.370 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.370 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.370 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.370 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.317028+0000", 2024-09-13T10:11:26.370 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.1607999999999995e-05 2024-09-13T10:11:26.370 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.370 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.370 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.371 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.371 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.371 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:335 4.2 4:4a24ecac:::smithi19530714-36:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.371 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:11:26.371 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.015793832, 2024-09-13T10:11:26.371 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032260373000000002, 2024-09-13T10:11:26.371 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.371 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.371 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.371 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.371 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.371 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.371 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 335 2024-09-13T10:11:26.372 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.372 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.372 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.372 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.372 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:11:26.372 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.372 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.372 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.372 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.372 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:11:26.372 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.372 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.373 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.373 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.373 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282494+0000", 2024-09-13T10:11:26.373 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5390000000000002e-06 2024-09-13T10:11:26.373 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.373 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.373 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.373 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282506+0000", 2024-09-13T10:11:26.373 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2646000000000001e-05 2024-09-13T10:11:26.374 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.374 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.374 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.374 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282539+0000", 2024-09-13T10:11:26.374 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3176000000000002e-05 2024-09-13T10:11:26.374 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.374 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.374 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.374 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282558+0000", 2024-09-13T10:11:26.374 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9065000000000001e-05 2024-09-13T10:11:26.374 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.374 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.375 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.375 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282697+0000", 2024-09-13T10:11:26.375 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013857999999999999 2024-09-13T10:11:26.375 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.375 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.375 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.375 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282874+0000", 2024-09-13T10:11:26.375 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017656900000000001 2024-09-13T10:11:26.375 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.375 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.375 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,14", 2024-09-13T10:11:26.375 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.287911+0000", 2024-09-13T10:11:26.376 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0050369220000000001 2024-09-13T10:11:26.376 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.376 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.376 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.376 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.290491+0000", 2024-09-13T10:11:26.376 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025802500000000001 2024-09-13T10:11:26.376 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.376 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.376 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.376 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.291374+0000", 2024-09-13T10:11:26.377 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00088367800000000005 2024-09-13T10:11:26.377 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.377 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.377 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.377 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314557+0000", 2024-09-13T10:11:26.377 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023182094 2024-09-13T10:11:26.377 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.377 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.377 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.377 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314664+0000", 2024-09-13T10:11:26.377 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010751099999999999 2024-09-13T10:11:26.377 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.377 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.378 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.378 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314751+0000", 2024-09-13T10:11:26.378 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7342999999999994e-05 2024-09-13T10:11:26.378 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.378 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.378 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.378 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.378 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.378 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:337 4.4 4:24275569:::smithi19530714-38:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.378 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:11:26.378 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.015137281000001, 2024-09-13T10:11:26.378 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.031660147, 2024-09-13T10:11:26.378 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.378 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.379 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.379 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.379 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.379 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.379 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 337 2024-09-13T10:11:26.379 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.379 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.379 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.379 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.379 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:11:26.379 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.379 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.379 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.380 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.380 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:11:26.380 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.380 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.380 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.380 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.380 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283149+0000", 2024-09-13T10:11:26.380 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.655e-06 2024-09-13T10:11:26.380 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.380 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.380 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.380 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283159+0000", 2024-09-13T10:11:26.380 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.9529999999999994e-06 2024-09-13T10:11:26.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.381 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.381 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.381 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283177+0000", 2024-09-13T10:11:26.381 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8168e-05 2024-09-13T10:11:26.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.381 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.381 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.381 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283193+0000", 2024-09-13T10:11:26.381 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5563e-05 2024-09-13T10:11:26.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.381 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.381 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.382 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283317+0000", 2024-09-13T10:11:26.382 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012421700000000001 2024-09-13T10:11:26.382 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.382 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.382 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.382 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283478+0000", 2024-09-13T10:11:26.382 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000160378 2024-09-13T10:11:26.382 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.382 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.382 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:26.382 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.289593+0000", 2024-09-13T10:11:26.382 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0061152200000000002 2024-09-13T10:11:26.382 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.383 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.383 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.383 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.291636+0000", 2024-09-13T10:11:26.383 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020435430000000001 2024-09-13T10:11:26.383 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.383 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.383 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.383 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294186+0000", 2024-09-13T10:11:26.383 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025494760000000002 2024-09-13T10:11:26.383 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.383 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.384 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.384 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314672+0000", 2024-09-13T10:11:26.384 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020486392999999999 2024-09-13T10:11:26.384 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.384 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.384 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.384 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314741+0000", 2024-09-13T10:11:26.384 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9003000000000003e-05 2024-09-13T10:11:26.384 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.384 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.384 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.384 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314808+0000", 2024-09-13T10:11:26.385 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.6577999999999999e-05 2024-09-13T10:11:26.385 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.385 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.385 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.385 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.385 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.385 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1089 4.5 4:a7549961:::smithi19530714-27:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:26.385 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:56.657202+0000", 2024-09-13T10:11:26.385 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 29.641082726, 2024-09-13T10:11:26.385 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020240790000000002, 2024-09-13T10:11:26.385 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.385 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.385 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:26.386 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.386 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.386 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.386 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1089 2024-09-13T10:11:26.386 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.386 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.386 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.386 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.386 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657202+0000", 2024-09-13T10:11:26.386 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.386 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.386 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.386 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.387 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657202+0000", 2024-09-13T10:11:26.387 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.387 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.387 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.387 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.387 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657205+0000", 2024-09-13T10:11:26.387 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.373e-06 2024-09-13T10:11:26.387 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.387 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.387 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.387 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657215+0000", 2024-09-13T10:11:26.387 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0563e-05 2024-09-13T10:11:26.387 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.388 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.388 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.388 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657243+0000", 2024-09-13T10:11:26.388 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8065999999999999e-05 2024-09-13T10:11:26.388 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.388 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.388 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.388 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657259+0000", 2024-09-13T10:11:26.388 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5415999999999999e-05 2024-09-13T10:11:26.388 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.388 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.388 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.388 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657404+0000", 2024-09-13T10:11:26.389 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014504000000000001 2024-09-13T10:11:26.389 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.389 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.389 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.389 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.657619+0000", 2024-09-13T10:11:26.389 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021532499999999999 2024-09-13T10:11:26.389 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.389 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.389 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.389 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:56.677443+0000", 2024-09-13T10:11:26.389 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.019824007000000001 2024-09-13T10:11:26.389 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.390 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.390 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.390 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.390 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.390 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1579 4.5 4:a59e88f7:::smithi19530714-21:head [sparse-read 1584598~750371] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:11:26.390 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:04.717221+0000", 2024-09-13T10:11:26.390 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 21.581063721, 2024-09-13T10:11:26.390 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016269826000000001, 2024-09-13T10:11:26.390 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.390 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.390 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1579 2024-09-13T10:11:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717221+0000", 2024-09-13T10:11:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.391 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.392 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.392 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717221+0000", 2024-09-13T10:11:26.392 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.392 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.392 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.392 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.392 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717223+0000", 2024-09-13T10:11:26.392 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6339999999999999e-06 2024-09-13T10:11:26.392 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.392 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.392 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.392 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717229+0000", 2024-09-13T10:11:26.392 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.2330000000000003e-06 2024-09-13T10:11:26.393 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.393 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.393 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.393 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717240+0000", 2024-09-13T10:11:26.393 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0606e-05 2024-09-13T10:11:26.393 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.393 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.393 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.393 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.717253+0000", 2024-09-13T10:11:26.393 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3743e-05 2024-09-13T10:11:26.393 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.393 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.394 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.394 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.732724+0000", 2024-09-13T10:11:26.394 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.01547045 2024-09-13T10:11:26.394 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.394 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.394 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.394 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.732946+0000", 2024-09-13T10:11:26.394 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000221676 2024-09-13T10:11:26.394 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.394 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.394 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.394 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:04.733491+0000", 2024-09-13T10:11:26.395 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00054548400000000005 2024-09-13T10:11:26.395 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.395 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.395 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.395 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.395 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.395 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1865 4.12 4:4ac406d9:::smithi19530714-26:head [read 0~2050454,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:26.395 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.728406+0000", 2024-09-13T10:11:26.395 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 14.569878706000001, 2024-09-13T10:11:26.395 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029384652000000001, 2024-09-13T10:11:26.395 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.395 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.396 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:26.396 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.396 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.396 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.396 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1865 2024-09-13T10:11:26.396 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.396 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.396 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.396 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.397 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728406+0000", 2024-09-13T10:11:26.397 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.397 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.397 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.397 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.397 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728406+0000", 2024-09-13T10:11:26.397 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.397 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.397 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.397 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.397 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728410+0000", 2024-09-13T10:11:26.397 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9129999999999996e-06 2024-09-13T10:11:26.397 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.398 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.398 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.398 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728423+0000", 2024-09-13T10:11:26.398 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2425999999999999e-05 2024-09-13T10:11:26.398 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.398 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.398 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.398 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728453+0000", 2024-09-13T10:11:26.398 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0673999999999998e-05 2024-09-13T10:11:26.398 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.398 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.398 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.398 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728478+0000", 2024-09-13T10:11:26.399 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5099000000000001e-05 2024-09-13T10:11:26.399 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.399 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.399 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.399 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730729+0000", 2024-09-13T10:11:26.399 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002251009 2024-09-13T10:11:26.399 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.399 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.399 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:11:26.399 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730758+0000", 2024-09-13T10:11:26.399 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8623e-05 2024-09-13T10:11:26.399 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.399 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.400 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.400 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.754517+0000", 2024-09-13T10:11:26.400 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023759199000000002 2024-09-13T10:11:26.400 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.400 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.400 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.400 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.754761+0000", 2024-09-13T10:11:26.400 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000243923 2024-09-13T10:11:26.401 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.401 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.401 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.401 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.757791+0000", 2024-09-13T10:11:26.401 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003029786 2024-09-13T10:11:26.401 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.401 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.401 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.401 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.401 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.401 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1866 4.12 4:4ac406d9:::smithi19530714-26:head [sparse-read 0~1216897] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:26.401 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.728515+0000", 2024-09-13T10:11:26.402 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 14.569769651, 2024-09-13T10:11:26.402 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.031206358999999999, 2024-09-13T10:11:26.402 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.402 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.402 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:26.402 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.402 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.402 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.402 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1866 2024-09-13T10:11:26.402 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.402 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.402 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.402 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.402 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728515+0000", 2024-09-13T10:11:26.403 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.403 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.403 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.403 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728515+0000", 2024-09-13T10:11:26.403 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.403 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.403 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.403 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728516+0000", 2024-09-13T10:11:26.403 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1429999999999999e-06 2024-09-13T10:11:26.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.403 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.404 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.404 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728522+0000", 2024-09-13T10:11:26.404 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5740000000000003e-06 2024-09-13T10:11:26.404 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.404 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.404 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.404 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728534+0000", 2024-09-13T10:11:26.404 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1539000000000001e-05 2024-09-13T10:11:26.404 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.405 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.405 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.405 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728543+0000", 2024-09-13T10:11:26.405 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6059999999999992e-06 2024-09-13T10:11:26.405 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.405 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.405 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.405 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730840+0000", 2024-09-13T10:11:26.405 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022966789999999998 2024-09-13T10:11:26.405 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.405 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.405 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:11:26.405 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730859+0000", 2024-09-13T10:11:26.406 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9613e-05 2024-09-13T10:11:26.406 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.406 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.406 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.406 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.758405+0000", 2024-09-13T10:11:26.406 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027545656000000002 2024-09-13T10:11:26.406 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.406 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.406 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.406 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.758574+0000", 2024-09-13T10:11:26.406 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000168808 2024-09-13T10:11:26.406 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.406 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.407 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.407 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759722+0000", 2024-09-13T10:11:26.407 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011477410000000001 2024-09-13T10:11:26.407 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.407 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.407 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.407 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.407 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.407 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1867 4.12 4:4ac406d9:::smithi19530714-26:head [read 116628~1933826,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:26.407 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.728557+0000", 2024-09-13T10:11:26.407 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 14.569728168999999, 2024-09-13T10:11:26.407 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033656772000000001, 2024-09-13T10:11:26.407 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.407 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.408 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:26.408 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.408 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.408 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.408 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1867 2024-09-13T10:11:26.408 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.408 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.408 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.408 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.408 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728557+0000", 2024-09-13T10:11:26.408 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.408 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.408 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.409 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.409 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728557+0000", 2024-09-13T10:11:26.409 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.409 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.409 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.409 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.409 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728558+0000", 2024-09-13T10:11:26.409 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8400000000000003e-07 2024-09-13T10:11:26.409 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.409 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.409 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.409 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728562+0000", 2024-09-13T10:11:26.409 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.8029999999999996e-06 2024-09-13T10:11:26.410 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.410 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.410 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.410 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728589+0000", 2024-09-13T10:11:26.410 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6536000000000001e-05 2024-09-13T10:11:26.410 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.410 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.410 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.410 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728603+0000", 2024-09-13T10:11:26.410 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4483e-05 2024-09-13T10:11:26.410 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.410 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.410 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.410 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730933+0000", 2024-09-13T10:11:26.411 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023295939999999999 2024-09-13T10:11:26.411 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.411 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.411 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:11:26.411 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730952+0000", 2024-09-13T10:11:26.411 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8614000000000002e-05 2024-09-13T10:11:26.411 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.411 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.411 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.411 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759881+0000", 2024-09-13T10:11:26.411 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.02892962 2024-09-13T10:11:26.411 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.412 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.412 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.412 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.760061+0000", 2024-09-13T10:11:26.412 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00018033 2024-09-13T10:11:26.412 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.412 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.412 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.412 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.762214+0000", 2024-09-13T10:11:26.412 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002152008 2024-09-13T10:11:26.412 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.412 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.412 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.412 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.413 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.413 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1932 4.5 4:a3098556:::smithi19530714-22:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:26.413 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.759351+0000", 2024-09-13T10:11:26.413 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 14.538933756, 2024-09-13T10:11:26.413 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.043979880999999998, 2024-09-13T10:11:26.413 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.413 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.413 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:26.413 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.413 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.413 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.413 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1932 2024-09-13T10:11:26.414 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.414 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.414 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.414 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.414 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759351+0000", 2024-09-13T10:11:26.414 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.414 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.414 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.414 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.414 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759351+0000", 2024-09-13T10:11:26.414 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.414 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.414 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.415 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.415 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759352+0000", 2024-09-13T10:11:26.415 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.28e-06 2024-09-13T10:11:26.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.415 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.415 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.415 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759358+0000", 2024-09-13T10:11:26.415 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8699999999999997e-06 2024-09-13T10:11:26.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.415 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.415 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.415 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759376+0000", 2024-09-13T10:11:26.415 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7532000000000001e-05 2024-09-13T10:11:26.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.416 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.416 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.416 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759391+0000", 2024-09-13T10:11:26.416 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5590000000000002e-05 2024-09-13T10:11:26.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.416 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.416 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.416 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759607+0000", 2024-09-13T10:11:26.416 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021536999999999999 2024-09-13T10:11:26.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.416 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.416 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.417 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759831+0000", 2024-09-13T10:11:26.417 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000224243 2024-09-13T10:11:26.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.417 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.417 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.417 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.803331+0000", 2024-09-13T10:11:26.417 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.043499995999999999 2024-09-13T10:11:26.417 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.417 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.417 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.418 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.418 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.418 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2189 4.4 4:26b855f3:::smithi19530714-25:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:11:26.418 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:17.462885+0000", 2024-09-13T10:11:26.418 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 8.8354000500000005, 2024-09-13T10:11:26.418 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041082475, 2024-09-13T10:11:26.418 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.418 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.418 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:26.418 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.418 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.418 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.418 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2189 2024-09-13T10:11:26.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.419 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.419 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.419 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.419 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462885+0000", 2024-09-13T10:11:26.419 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.419 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.419 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.419 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462885+0000", 2024-09-13T10:11:26.419 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.419 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.420 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.420 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462888+0000", 2024-09-13T10:11:26.420 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.74e-06 2024-09-13T10:11:26.420 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.420 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.420 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.420 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462898+0000", 2024-09-13T10:11:26.420 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0618999999999999e-05 2024-09-13T10:11:26.420 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.420 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.420 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.420 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462928+0000", 2024-09-13T10:11:26.420 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9685000000000001e-05 2024-09-13T10:11:26.421 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.421 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.421 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.421 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462944+0000", 2024-09-13T10:11:26.421 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6175999999999999e-05 2024-09-13T10:11:26.421 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.421 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.421 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.421 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.463063+0000", 2024-09-13T10:11:26.421 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000119393 2024-09-13T10:11:26.421 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.421 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.421 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.422 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.463204+0000", 2024-09-13T10:11:26.422 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014020600000000001 2024-09-13T10:11:26.422 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.422 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.422 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.422 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.503967+0000", 2024-09-13T10:11:26.422 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.040763656000000002 2024-09-13T10:11:26.422 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.422 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.422 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.422 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.422 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.422 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2302 4.15 4:ab73fdc2:::smithi19530714-30:head [read 299511~3414803,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e62)", 2024-09-13T10:11:26.423 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:21.796919+0000", 2024-09-13T10:11:26.423 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 4.5013658420000002, 2024-09-13T10:11:26.423 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017004148, 2024-09-13T10:11:26.423 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.423 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.423 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:26.423 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.423 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.423 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.423 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2302 2024-09-13T10:11:26.423 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.423 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.423 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.423 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.424 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:21.796919+0000", 2024-09-13T10:11:26.424 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.424 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.424 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.424 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.424 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:21.796919+0000", 2024-09-13T10:11:26.424 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.424 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.424 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.424 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.424 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:21.796920+0000", 2024-09-13T10:11:26.424 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.28e-06 2024-09-13T10:11:26.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.425 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.425 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.425 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:21.796928+0000", 2024-09-13T10:11:26.425 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5959999999999998e-06 2024-09-13T10:11:26.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.425 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.425 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.425 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:21.796943+0000", 2024-09-13T10:11:26.425 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5478999999999999e-05 2024-09-13T10:11:26.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.425 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.426 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.426 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:21.796958+0000", 2024-09-13T10:11:26.426 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4185999999999999e-05 2024-09-13T10:11:26.426 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.426 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.426 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.426 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:21.797737+0000", 2024-09-13T10:11:26.426 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00077914499999999997 2024-09-13T10:11:26.426 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.426 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.426 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:11:26.426 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:21.797760+0000", 2024-09-13T10:11:26.426 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3057999999999998e-05 2024-09-13T10:11:26.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.427 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.427 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.427 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:21.808829+0000", 2024-09-13T10:11:26.427 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011068704 2024-09-13T10:11:26.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.427 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.427 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.427 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:21.809016+0000", 2024-09-13T10:11:26.427 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000187157 2024-09-13T10:11:26.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.427 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.427 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.427 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:21.813923+0000", 2024-09-13T10:11:26.428 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0049075430000000003 2024-09-13T10:11:26.428 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.428 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.428 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.428 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.428 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.428 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:26.429 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:26.430 INFO:tasks.rados.rados.0.smithi195.stdout:1208: expect (ObjNum 176 snap 0 seq_num 176) 2024-09-13T10:11:26.430 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:26.430 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:26.430 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:26.426+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:26.431 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:26.431 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:26.431 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-13T10:11:26.432 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:26.432 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:26.433 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:26.433 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:26.433 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:26.433 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:26.477 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2891299 2024-09-13T10:11:26.483 INFO:tasks.rados.rados.0.smithi195.stdout:1209: finishing write tid 1 to smithi19530714-8 2024-09-13T10:11:26.484 INFO:tasks.rados.rados.0.smithi195.stdout:1209: finishing write tid 2 to smithi19530714-8 2024-09-13T10:11:26.484 INFO:tasks.rados.rados.0.smithi195.stdout:1209: finishing write tid 3 to smithi19530714-8 2024-09-13T10:11:26.484 INFO:tasks.rados.rados.0.smithi195.stdout:1209: finishing write tid 4 to smithi19530714-8 2024-09-13T10:11:26.484 INFO:tasks.rados.rados.0.smithi195.stdout:1209: finishing write tid 5 to smithi19530714-8 2024-09-13T10:11:26.484 INFO:tasks.rados.rados.0.smithi195.stdout:1209: oid 8 updating version 0 to 16 2024-09-13T10:11:26.484 INFO:tasks.rados.rados.0.smithi195.stdout:1209: oid 8 version 16 is already newer than 15 2024-09-13T10:11:26.484 INFO:tasks.rados.rados.0.smithi195.stdout:1209: oid 8 updating version 16 to 18 2024-09-13T10:11:26.485 INFO:tasks.rados.rados.0.smithi195.stdout:1209: oid 8 version 18 is already newer than 17 2024-09-13T10:11:26.485 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 18 (ObjNum 226 snap 0 seq_num 226) dirty exists 2024-09-13T10:11:26.485 INFO:tasks.rados.rados.0.smithi195.stdout:1209: left oid 8 (ObjNum 226 snap 0 seq_num 226) 2024-09-13T10:11:26.485 INFO:tasks.rados.rados.0.smithi195.stdout:1210: finishing write tid 1 to smithi19530714-31 2024-09-13T10:11:26.485 INFO:tasks.rados.rados.0.smithi195.stdout:1210: finishing write tid 2 to smithi19530714-31 2024-09-13T10:11:26.485 INFO:tasks.rados.rados.0.smithi195.stdout:1210: finishing write tid 3 to smithi19530714-31 2024-09-13T10:11:26.485 INFO:tasks.rados.rados.0.smithi195.stdout:1210: finishing write tid 4 to smithi19530714-31 2024-09-13T10:11:26.486 INFO:tasks.rados.rados.0.smithi195.stdout:1210: finishing write tid 5 to smithi19530714-31 2024-09-13T10:11:26.486 INFO:tasks.rados.rados.0.smithi195.stdout:1210: oid 31 updating version 0 to 64 2024-09-13T10:11:26.486 INFO:tasks.rados.rados.0.smithi195.stdout:1210: oid 31 version 64 is already newer than 61 2024-09-13T10:11:26.486 INFO:tasks.rados.rados.0.smithi195.stdout:1210: oid 31 version 64 is already newer than 63 2024-09-13T10:11:26.486 INFO:tasks.rados.rados.0.smithi195.stdout:1210: oid 31 version 64 is already newer than 62 2024-09-13T10:11:26.486 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 64 (ObjNum 227 snap 0 seq_num 227) dirty exists 2024-09-13T10:11:26.486 INFO:tasks.rados.rados.0.smithi195.stdout:1210: left oid 31 (ObjNum 227 snap 0 seq_num 227) 2024-09-13T10:11:26.486 INFO:tasks.rados.rados.0.smithi195.stdout:1211: finishing write tid 1 to smithi19530714-13 2024-09-13T10:11:26.486 INFO:tasks.rados.rados.0.smithi195.stdout:1211: finishing write tid 2 to smithi19530714-13 2024-09-13T10:11:26.486 INFO:tasks.rados.rados.0.smithi195.stdout:1212: finishing set_chunk to oid 12 2024-09-13T10:11:26.486 INFO:tasks.rados.rados.0.smithi195.stdout:1212: got expected ENOENT (src dne) 2024-09-13T10:11:26.486 INFO:tasks.rados.rados.0.smithi195.stdout:1215: finishing set_chunk to oid 33 2024-09-13T10:11:26.486 INFO:tasks.rados.rados.0.smithi195.stdout:1215: got expected ENOENT (src dne) 2024-09-13T10:11:26.487 INFO:tasks.rados.rados.0.smithi195.stdout:1213: got (22) Invalid argument 2024-09-13T10:11:26.487 INFO:tasks.rados.rados.0.smithi195.stdout:1216: seq_num 229 ranges {648803=722611,1967200=714747,3421983=426820} 2024-09-13T10:11:26.495 INFO:tasks.rados.rados.0.smithi195.stdout:1216: writing smithi19530714-6 from 648803 to 1371414 tid 1 2024-09-13T10:11:26.509 INFO:tasks.rados.rados.0.smithi195.stdout:1216: writing smithi19530714-6 from 1967200 to 2681947 tid 2 2024-09-13T10:11:26.522 INFO:tasks.rados.rados.0.smithi195.stdout:1216: writing smithi19530714-6 from 3421983 to 3848803 tid 3 2024-09-13T10:11:26.522 INFO:tasks.rados.rados.0.smithi195.stdout:1207: done (9 left) 2024-09-13T10:11:26.522 INFO:tasks.rados.rados.0.smithi195.stdout:1208: done (8 left) 2024-09-13T10:11:26.522 INFO:tasks.rados.rados.0.smithi195.stdout:1209: done (7 left) 2024-09-13T10:11:26.522 INFO:tasks.rados.rados.0.smithi195.stdout:1210: done (6 left) 2024-09-13T10:11:26.522 INFO:tasks.rados.rados.0.smithi195.stdout:1212: done (5 left) 2024-09-13T10:11:26.523 INFO:tasks.rados.rados.0.smithi195.stdout:1213: done (4 left) 2024-09-13T10:11:26.523 INFO:tasks.rados.rados.0.smithi195.stdout:1215: done (3 left) 2024-09-13T10:11:26.523 INFO:tasks.rados.rados.0.smithi195.stdout:1217: read oid 1 snap -1 2024-09-13T10:11:26.523 INFO:tasks.rados.rados.0.smithi195.stdout:1217: expect (ObjNum 184 snap 0 seq_num 184) 2024-09-13T10:11:26.523 INFO:tasks.rados.rados.0.smithi195.stdout:1218: read oid 41 snap -1 2024-09-13T10:11:26.523 INFO:tasks.rados.rados.0.smithi195.stdout:1218: expect (ObjNum 141 snap 0 seq_num 141) 2024-09-13T10:11:26.523 INFO:tasks.rados.rados.0.smithi195.stdout:1219: read oid 5 snap -1 2024-09-13T10:11:26.523 INFO:tasks.rados.rados.0.smithi195.stdout:1219: expect (ObjNum 173 snap 0 seq_num 173) 2024-09-13T10:11:26.523 INFO:tasks.rados.rados.0.smithi195.stdout:1220: read oid 14 snap -1 2024-09-13T10:11:26.523 INFO:tasks.rados.rados.0.smithi195.stdout:1220: expect (ObjNum 209 snap 0 seq_num 209) 2024-09-13T10:11:26.523 INFO:tasks.rados.rados.0.smithi195.stdout:1221: write (excl) oid 43 current snap is 0 2024-09-13T10:11:26.523 INFO:tasks.rados.rados.0.smithi195.stdout:1211: finishing write tid 3 to smithi19530714-13 2024-09-13T10:11:26.523 INFO:tasks.rados.rados.0.smithi195.stdout:1211: finishing write tid 4 to smithi19530714-13 2024-09-13T10:11:26.523 INFO:tasks.rados.rados.0.smithi195.stdout:1211: finishing write tid 5 to smithi19530714-13 2024-09-13T10:11:26.524 INFO:tasks.rados.rados.0.smithi195.stdout:1211: oid 13 updating version 0 to 61 2024-09-13T10:11:26.524 INFO:tasks.rados.rados.0.smithi195.stdout:1211: oid 13 version 61 is already newer than 60 2024-09-13T10:11:26.524 INFO:tasks.rados.rados.0.smithi195.stdout:1211: oid 13 updating version 61 to 63 2024-09-13T10:11:26.524 INFO:tasks.rados.rados.0.smithi195.stdout:1211: oid 13 version 63 is already newer than 62 2024-09-13T10:11:26.524 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 63 (ObjNum 228 snap 0 seq_num 228) dirty exists 2024-09-13T10:11:26.524 INFO:tasks.rados.rados.0.smithi195.stdout:1211: left oid 13 (ObjNum 228 snap 0 seq_num 228) 2024-09-13T10:11:26.524 INFO:tasks.rados.rados.0.smithi195.stdout:1214: expect (ObjNum 121 snap 0 seq_num 121) 2024-09-13T10:11:26.530 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:26.526+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:26.562 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:26.562 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:26.562 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:11:26.562 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:26.576 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-13T10:11:26.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:26.596+0000 7f13ad142640 0 -- 172.21.15.73:0/3210624370 >> v2:172.21.15.181:6804/1173379175 conn(0x7f13a8222040 msgr2=0x7f13a8224490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:11:26.624 INFO:tasks.rados.rados.0.smithi195.stdout:1216: finishing write tid 1 to smithi19530714-6 2024-09-13T10:11:26.624 INFO:tasks.rados.rados.0.smithi195.stdout:1216: finishing write tid 2 to smithi19530714-6 2024-09-13T10:11:26.624 INFO:tasks.rados.rados.0.smithi195.stdout:1216: finishing write tid 3 to smithi19530714-6 2024-09-13T10:11:26.624 INFO:tasks.rados.rados.0.smithi195.stdout:1218: expect (ObjNum 141 snap 0 seq_num 141) 2024-09-13T10:11:26.631 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:26.626+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:26.660 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:26.697 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:26.698 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:26.698 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:11:26.698 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:11:26.698 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:26.711 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-13T10:11:26.732 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:26.728+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:26.752 INFO:tasks.rados.rados.0.smithi195.stdout:1216: finishing write tid 4 to smithi19530714-6 2024-09-13T10:11:26.752 INFO:tasks.rados.rados.0.smithi195.stdout:1216: finishing write tid 5 to smithi19530714-6 2024-09-13T10:11:26.752 INFO:tasks.rados.rados.0.smithi195.stdout:1216: oid 6 updating version 0 to 66 2024-09-13T10:11:26.752 INFO:tasks.rados.rados.0.smithi195.stdout:1216: oid 6 version 66 is already newer than 63 2024-09-13T10:11:26.752 INFO:tasks.rados.rados.0.smithi195.stdout:1216: oid 6 version 66 is already newer than 65 2024-09-13T10:11:26.752 INFO:tasks.rados.rados.0.smithi195.stdout:1216: oid 6 version 66 is already newer than 64 2024-09-13T10:11:26.752 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 66 (ObjNum 229 snap 0 seq_num 229) dirty exists 2024-09-13T10:11:26.752 INFO:tasks.rados.rados.0.smithi195.stdout:1216: left oid 6 (ObjNum 229 snap 0 seq_num 229) 2024-09-13T10:11:26.752 INFO:tasks.rados.rados.0.smithi195.stdout:1219: expect (ObjNum 173 snap 0 seq_num 173) 2024-09-13T10:11:26.758 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 716348 2024-09-13T10:11:26.826 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T10:11:26.833 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:26.831+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:26.841 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:26.841 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:11:26.841 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:11:26.841 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:11:26.841 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.841 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:201 3.3 3:d4fb61be:::smithi19530714-2 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.841 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:11:26.841 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.843635011000003, 2024-09-13T10:11:26.841 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.052937043000000003, 2024-09-13T10:11:26.841 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.841 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 201 2024-09-13T10:11:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:11:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.842 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.843 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.843 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:11:26.843 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.843 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.843 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.843 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.843 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993192+0000", 2024-09-13T10:11:26.843 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8559999999999996e-06 2024-09-13T10:11:26.843 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.843 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.843 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.844 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993203+0000", 2024-09-13T10:11:26.844 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1113e-05 2024-09-13T10:11:26.844 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.844 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.844 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.844 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993225+0000", 2024-09-13T10:11:26.844 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2793e-05 2024-09-13T10:11:26.844 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.844 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.844 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.844 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993241+0000", 2024-09-13T10:11:26.844 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5806000000000001e-05 2024-09-13T10:11:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993401+0000", 2024-09-13T10:11:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000160149 2024-09-13T10:11:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994047+0000", 2024-09-13T10:11:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00064594100000000003 2024-09-13T10:11:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.845 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.846 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.011243+0000", 2024-09-13T10:11:26.846 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017196026999999999 2024-09-13T10:11:26.846 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.846 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.846 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.846 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029096+0000", 2024-09-13T10:11:26.846 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017852115000000002 2024-09-13T10:11:26.846 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.846 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.846 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.846 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029271+0000", 2024-09-13T10:11:26.846 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017589299999999999 2024-09-13T10:11:26.846 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.847 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.847 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.847 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.045867+0000", 2024-09-13T10:11:26.847 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016595625999999999 2024-09-13T10:11:26.847 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.847 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.847 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.847 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.046017+0000", 2024-09-13T10:11:26.847 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014974199999999999 2024-09-13T10:11:26.847 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.848 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.848 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.848 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.046120+0000", 2024-09-13T10:11:26.848 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000102982 2024-09-13T10:11:26.848 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.848 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.848 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.848 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.848 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.848 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:208 3.3 3:dbb0d8aa:::smithi19530714-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.848 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:11:26.848 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.843539200999999, 2024-09-13T10:11:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.071216475000000001, 2024-09-13T10:11:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 208 2024-09-13T10:11:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.849 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:11:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:11:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993280+0000", 2024-09-13T10:11:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1999999999999999e-06 2024-09-13T10:11:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.850 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993289+0000", 2024-09-13T10:11:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8200000000000003e-06 2024-09-13T10:11:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994000+0000", 2024-09-13T10:11:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00071192499999999997 2024-09-13T10:11:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994024+0000", 2024-09-13T10:11:26.851 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3578999999999999e-05 2024-09-13T10:11:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994468+0000", 2024-09-13T10:11:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00044410599999999999 2024-09-13T10:11:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994842+0000", 2024-09-13T10:11:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037380300000000003 2024-09-13T10:11:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.852 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031572+0000", 2024-09-13T10:11:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036730479000000003 2024-09-13T10:11:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.039007+0000", 2024-09-13T10:11:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.007434871 2024-09-13T10:11:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.041001+0000", 2024-09-13T10:11:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001994104 2024-09-13T10:11:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.853 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064330+0000", 2024-09-13T10:11:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023328912 2024-09-13T10:11:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064401+0000", 2024-09-13T10:11:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.1123000000000003e-05 2024-09-13T10:11:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064495+0000", 2024-09-13T10:11:26.854 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.3553000000000004e-05 2024-09-13T10:11:26.855 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.855 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.855 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.855 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.855 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.855 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:215 3.3 3:ed6c0a17:::smithi19530714-16 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.855 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:11:26.855 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.842758947, 2024-09-13T10:11:26.855 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.055806517999999999, 2024-09-13T10:11:26.855 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.855 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.855 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.856 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.856 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.856 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.856 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 215 2024-09-13T10:11:26.856 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.856 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.856 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.856 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.856 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:11:26.856 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.856 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.857 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.857 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.857 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:11:26.857 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.857 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.857 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.857 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.857 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994060+0000", 2024-09-13T10:11:26.857 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5030000000000001e-06 2024-09-13T10:11:26.857 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.857 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.857 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.858 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994070+0000", 2024-09-13T10:11:26.858 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0046e-05 2024-09-13T10:11:26.858 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.858 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.858 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.858 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994087+0000", 2024-09-13T10:11:26.858 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6283999999999999e-05 2024-09-13T10:11:26.858 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.858 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.858 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.858 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994096+0000", 2024-09-13T10:11:26.859 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.969e-06 2024-09-13T10:11:26.859 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.859 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.859 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.859 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.995226+0000", 2024-09-13T10:11:26.859 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011299540000000001 2024-09-13T10:11:26.859 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.859 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.859 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.859 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.995548+0000", 2024-09-13T10:11:26.859 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00032266199999999998 2024-09-13T10:11:26.859 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.860 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.860 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.860 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.018922+0000", 2024-09-13T10:11:26.860 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023373485999999999 2024-09-13T10:11:26.860 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.860 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.860 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.860 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029400+0000", 2024-09-13T10:11:26.860 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010478721 2024-09-13T10:11:26.860 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.861 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.861 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.861 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038932+0000", 2024-09-13T10:11:26.861 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0095316859999999993 2024-09-13T10:11:26.861 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.861 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.861 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.861 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049620+0000", 2024-09-13T10:11:26.861 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010687688000000001 2024-09-13T10:11:26.861 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.861 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.862 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.862 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049754+0000", 2024-09-13T10:11:26.862 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013457700000000001 2024-09-13T10:11:26.862 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.862 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.862 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.862 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049865+0000", 2024-09-13T10:11:26.862 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110942 2024-09-13T10:11:26.862 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.862 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.862 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.862 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.863 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.863 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:224 3.3 3:f941658c:::smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.863 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:11:26.863 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.78625091, 2024-09-13T10:11:26.863 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.025014027000000001, 2024-09-13T10:11:26.863 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.863 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.863 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.863 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.863 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.863 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.863 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 224 2024-09-13T10:11:26.863 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.864 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.864 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.864 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.864 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:11:26.864 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.864 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.864 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.864 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.864 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:11:26.864 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.864 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.865 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.865 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.865 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050570+0000", 2024-09-13T10:11:26.865 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9799999999999998e-06 2024-09-13T10:11:26.865 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.865 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.865 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.865 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050582+0000", 2024-09-13T10:11:26.865 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2170999999999999e-05 2024-09-13T10:11:26.865 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.865 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.866 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.866 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050605+0000", 2024-09-13T10:11:26.866 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3543000000000001e-05 2024-09-13T10:11:26.866 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.866 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.866 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.866 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050619+0000", 2024-09-13T10:11:26.866 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3478999999999999e-05 2024-09-13T10:11:26.866 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.866 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.866 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.866 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050725+0000", 2024-09-13T10:11:26.866 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000105585 2024-09-13T10:11:26.867 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.867 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.867 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.867 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.051086+0000", 2024-09-13T10:11:26.867 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036162100000000001 2024-09-13T10:11:26.867 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.867 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.867 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.867 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.058628+0000", 2024-09-13T10:11:26.867 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075415569999999999 2024-09-13T10:11:26.867 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.868 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.868 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.868 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065344+0000", 2024-09-13T10:11:26.868 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0067161740000000001 2024-09-13T10:11:26.868 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.868 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.868 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.868 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.066222+0000", 2024-09-13T10:11:26.868 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00087765599999999999 2024-09-13T10:11:26.868 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.868 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.868 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.869 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075372+0000", 2024-09-13T10:11:26.869 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0091499609999999999 2024-09-13T10:11:26.869 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.869 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.869 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.869 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075489+0000", 2024-09-13T10:11:26.869 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011718300000000001 2024-09-13T10:11:26.869 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.869 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.869 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.869 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075581+0000", 2024-09-13T10:11:26.869 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.2116999999999999e-05 2024-09-13T10:11:26.869 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.869 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.870 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.870 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.870 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.870 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:226 3.3 3:fe5bfa3a:::smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.870 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:11:26.870 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.776717900999998, 2024-09-13T10:11:26.870 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051515848000000003, 2024-09-13T10:11:26.870 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.870 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.870 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.870 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.870 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.870 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.871 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 226 2024-09-13T10:11:26.871 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.871 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.871 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.871 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.871 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:11:26.871 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.871 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.871 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.871 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.871 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:11:26.871 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.871 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.872 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.872 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.872 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060102+0000", 2024-09-13T10:11:26.872 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0480000000000001e-06 2024-09-13T10:11:26.872 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.872 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.872 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.872 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060112+0000", 2024-09-13T10:11:26.872 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6649999999999998e-06 2024-09-13T10:11:26.872 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.872 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.872 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.872 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060132+0000", 2024-09-13T10:11:26.873 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0262e-05 2024-09-13T10:11:26.873 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.873 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.873 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.873 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060143+0000", 2024-09-13T10:11:26.873 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1420000000000001e-05 2024-09-13T10:11:26.873 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.873 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.873 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.873 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064563+0000", 2024-09-13T10:11:26.873 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044199119999999998 2024-09-13T10:11:26.874 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.874 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.874 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.874 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064882+0000", 2024-09-13T10:11:26.874 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000318735 2024-09-13T10:11:26.874 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.874 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.874 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.874 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.078858+0000", 2024-09-13T10:11:26.874 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013976455 2024-09-13T10:11:26.874 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.874 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.874 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.875 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088977+0000", 2024-09-13T10:11:26.875 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010118544 2024-09-13T10:11:26.875 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.875 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.875 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.875 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.090451+0000", 2024-09-13T10:11:26.875 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001474026 2024-09-13T10:11:26.875 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.875 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.875 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.875 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111539+0000", 2024-09-13T10:11:26.875 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.02108786 2024-09-13T10:11:26.876 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.876 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.876 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.876 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111580+0000", 2024-09-13T10:11:26.876 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1245000000000002e-05 2024-09-13T10:11:26.876 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.876 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.876 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.876 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111616+0000", 2024-09-13T10:11:26.876 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5676000000000002e-05 2024-09-13T10:11:26.876 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.876 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.876 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.877 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.877 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.877 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:235 3.3 3:e217e96e:::smithi19530714-36 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.877 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:11:26.877 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.765940272000002, 2024-09-13T10:11:26.877 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051184682000000002, 2024-09-13T10:11:26.877 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.877 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.877 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.877 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.877 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.877 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.877 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 235 2024-09-13T10:11:26.878 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.878 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.878 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.878 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.878 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:11:26.878 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.878 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.878 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.878 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.878 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:11:26.878 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.878 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.878 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.879 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.879 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070881+0000", 2024-09-13T10:11:26.879 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0579999999999999e-06 2024-09-13T10:11:26.879 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.879 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.879 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.879 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070897+0000", 2024-09-13T10:11:26.879 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5967000000000001e-05 2024-09-13T10:11:26.879 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.879 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.879 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.879 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070942+0000", 2024-09-13T10:11:26.879 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4947000000000002e-05 2024-09-13T10:11:26.880 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.880 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.880 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.880 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070963+0000", 2024-09-13T10:11:26.880 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0098e-05 2024-09-13T10:11:26.880 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.880 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.880 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.880 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.071099+0000", 2024-09-13T10:11:26.880 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013697199999999999 2024-09-13T10:11:26.880 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.880 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.880 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.881 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.071330+0000", 2024-09-13T10:11:26.881 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000230881 2024-09-13T10:11:26.881 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.881 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.881 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.881 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.084668+0000", 2024-09-13T10:11:26.881 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013337663 2024-09-13T10:11:26.881 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.881 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.881 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.881 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.092656+0000", 2024-09-13T10:11:26.881 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0079878880000000003 2024-09-13T10:11:26.881 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.882 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.882 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.882 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.100238+0000", 2024-09-13T10:11:26.882 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075819449999999997 2024-09-13T10:11:26.882 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.882 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.882 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.882 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121877+0000", 2024-09-13T10:11:26.882 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021639104999999999 2024-09-13T10:11:26.882 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.882 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.882 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.882 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121978+0000", 2024-09-13T10:11:26.883 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000101363 2024-09-13T10:11:26.883 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.883 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.883 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.883 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.122062+0000", 2024-09-13T10:11:26.883 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.3795000000000007e-05 2024-09-13T10:11:26.883 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.883 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.883 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.883 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.883 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.883 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:237 3.3 3:de40747e:::smithi19530714-38 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.883 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:11:26.884 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.757751067999997, 2024-09-13T10:11:26.884 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.073341331999999995, 2024-09-13T10:11:26.884 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.884 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.884 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.884 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.884 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.884 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.884 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 237 2024-09-13T10:11:26.884 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.884 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.884 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.884 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.885 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:11:26.885 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.885 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.885 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.885 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.885 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:11:26.885 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.885 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.885 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.885 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.885 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079069+0000", 2024-09-13T10:11:26.886 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6979999999999998e-06 2024-09-13T10:11:26.886 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.886 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.886 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.886 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079083+0000", 2024-09-13T10:11:26.886 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.384e-05 2024-09-13T10:11:26.886 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.886 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.886 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.886 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079105+0000", 2024-09-13T10:11:26.886 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1736e-05 2024-09-13T10:11:26.886 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.886 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.887 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.887 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079123+0000", 2024-09-13T10:11:26.887 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7889000000000001e-05 2024-09-13T10:11:26.887 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.887 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.887 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.887 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088459+0000", 2024-09-13T10:11:26.887 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0093359700000000007 2024-09-13T10:11:26.887 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.887 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.887 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.887 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088726+0000", 2024-09-13T10:11:26.888 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026732299999999997 2024-09-13T10:11:26.888 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.888 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.888 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.888 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.097522+0000", 2024-09-13T10:11:26.888 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0087962630000000003 2024-09-13T10:11:26.888 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.888 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.888 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.888 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.103972+0000", 2024-09-13T10:11:26.888 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0064494840000000001 2024-09-13T10:11:26.888 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.888 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.889 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.889 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.104160+0000", 2024-09-13T10:11:26.889 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00018782200000000001 2024-09-13T10:11:26.889 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.889 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.889 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.889 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152294+0000", 2024-09-13T10:11:26.889 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048134458999999997 2024-09-13T10:11:26.889 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.889 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.889 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.889 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152373+0000", 2024-09-13T10:11:26.890 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8574999999999999e-05 2024-09-13T10:11:26.890 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.890 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.890 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.890 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152408+0000", 2024-09-13T10:11:26.890 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5272999999999999e-05 2024-09-13T10:11:26.890 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.890 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.890 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.890 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.890 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.890 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:238 3.3 3:c43dd60b:::smithi19530714-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.890 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:11:26.891 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.748165421000003, 2024-09-13T10:11:26.891 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.069426360000000006, 2024-09-13T10:11:26.891 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.891 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.891 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.891 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.891 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.891 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.891 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 238 2024-09-13T10:11:26.891 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.891 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.891 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.891 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.892 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:11:26.892 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.892 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.892 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.892 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.892 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:11:26.892 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.892 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.892 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.892 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.892 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088655+0000", 2024-09-13T10:11:26.892 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2359999999999999e-06 2024-09-13T10:11:26.893 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.893 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.893 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.893 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088663+0000", 2024-09-13T10:11:26.893 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7800000000000006e-06 2024-09-13T10:11:26.893 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.893 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.893 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.893 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088688+0000", 2024-09-13T10:11:26.893 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4507000000000001e-05 2024-09-13T10:11:26.893 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.893 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.893 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.893 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088717+0000", 2024-09-13T10:11:26.894 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9204e-05 2024-09-13T10:11:26.894 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.894 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.894 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.894 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.089038+0000", 2024-09-13T10:11:26.894 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00032087199999999999 2024-09-13T10:11:26.894 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.894 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.894 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.894 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.089435+0000", 2024-09-13T10:11:26.894 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039756600000000002 2024-09-13T10:11:26.894 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.894 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.895 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.895 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.101672+0000", 2024-09-13T10:11:26.895 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012236528 2024-09-13T10:11:26.895 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.895 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.895 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.895 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.105256+0000", 2024-09-13T10:11:26.895 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0035837159999999998 2024-09-13T10:11:26.895 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.895 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.895 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.895 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111504+0000", 2024-09-13T10:11:26.895 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.006248577 2024-09-13T10:11:26.896 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.896 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.896 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.896 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.157950+0000", 2024-09-13T10:11:26.896 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.046446031999999998 2024-09-13T10:11:26.896 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.896 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.896 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.896 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158027+0000", 2024-09-13T10:11:26.896 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6402999999999993e-05 2024-09-13T10:11:26.896 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.897 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.897 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.897 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158079+0000", 2024-09-13T10:11:26.897 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.1938999999999999e-05 2024-09-13T10:11:26.897 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.897 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.897 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.897 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.897 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.897 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:246 3.3 3:f1be28e7:::smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.897 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:11:26.897 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.723454674000003, 2024-09-13T10:11:26.897 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051630810999999999, 2024-09-13T10:11:26.898 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.898 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.898 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.898 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.898 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.898 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.898 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 246 2024-09-13T10:11:26.898 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.898 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.898 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.898 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.898 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:11:26.898 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.898 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.899 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.899 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.899 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:11:26.899 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.899 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.899 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.899 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.899 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113365+0000", 2024-09-13T10:11:26.899 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.762e-06 2024-09-13T10:11:26.899 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.899 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.899 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.899 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113372+0000", 2024-09-13T10:11:26.900 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5020000000000003e-06 2024-09-13T10:11:26.900 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.900 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.900 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.900 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113390+0000", 2024-09-13T10:11:26.900 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8056000000000001e-05 2024-09-13T10:11:26.900 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.900 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.900 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.900 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113400+0000", 2024-09-13T10:11:26.900 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5119999999999997e-06 2024-09-13T10:11:26.900 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.900 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.901 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.901 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113456+0000", 2024-09-13T10:11:26.901 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.6221999999999998e-05 2024-09-13T10:11:26.901 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.901 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.901 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.901 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113576+0000", 2024-09-13T10:11:26.901 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000119466 2024-09-13T10:11:26.901 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.901 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.901 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.901 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.129127+0000", 2024-09-13T10:11:26.901 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015551003000000001 2024-09-13T10:11:26.902 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.902 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.902 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.902 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142482+0000", 2024-09-13T10:11:26.902 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013355104 2024-09-13T10:11:26.902 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.902 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.902 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.902 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142518+0000", 2024-09-13T10:11:26.902 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6532000000000002e-05 2024-09-13T10:11:26.902 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.902 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.902 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.903 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164919+0000", 2024-09-13T10:11:26.903 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022401038000000002 2024-09-13T10:11:26.903 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.903 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.903 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.903 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164962+0000", 2024-09-13T10:11:26.903 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.2329000000000001e-05 2024-09-13T10:11:26.903 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.903 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.903 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.903 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164994+0000", 2024-09-13T10:11:26.903 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2285e-05 2024-09-13T10:11:26.903 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.904 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.904 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.904 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.904 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.904 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:249 3.3 3:f565152a:::smithi19530714-50 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.904 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:11:26.904 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.721852196999997, 2024-09-13T10:11:26.904 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051233251, 2024-09-13T10:11:26.904 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.904 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.905 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.905 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:26.905 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:26.905 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:26.905 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 249 2024-09-13T10:11:26.905 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.905 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.905 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.905 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.905 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:11:26.905 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.905 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.905 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.906 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.906 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:11:26.906 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.906 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.906 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.906 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.906 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114967+0000", 2024-09-13T10:11:26.906 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.313e-06 2024-09-13T10:11:26.906 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.906 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.906 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.906 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114972+0000", 2024-09-13T10:11:26.906 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5620000000000003e-06 2024-09-13T10:11:26.907 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.907 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.907 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.907 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114997+0000", 2024-09-13T10:11:26.907 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4845999999999999e-05 2024-09-13T10:11:26.907 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.907 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.907 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.907 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115007+0000", 2024-09-13T10:11:26.908 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7200000000000001e-06 2024-09-13T10:11:26.908 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.908 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.908 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.908 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115076+0000", 2024-09-13T10:11:26.908 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9473000000000004e-05 2024-09-13T10:11:26.908 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.908 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.908 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.908 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115221+0000", 2024-09-13T10:11:26.908 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000144604 2024-09-13T10:11:26.908 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.908 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.909 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.909 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.136862+0000", 2024-09-13T10:11:26.909 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021641365999999999 2024-09-13T10:11:26.909 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.909 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.909 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.909 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.145775+0000", 2024-09-13T10:11:26.909 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0089130649999999995 2024-09-13T10:11:26.909 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.909 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.909 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.909 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.150578+0000", 2024-09-13T10:11:26.909 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0048025999999999998 2024-09-13T10:11:26.910 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.910 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.910 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.910 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166133+0000", 2024-09-13T10:11:26.910 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015554853 2024-09-13T10:11:26.910 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.910 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.910 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.910 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166167+0000", 2024-09-13T10:11:26.910 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3973999999999997e-05 2024-09-13T10:11:26.910 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.910 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.910 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.911 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166199+0000", 2024-09-13T10:11:26.911 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1875000000000002e-05 2024-09-13T10:11:26.911 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.911 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.911 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.911 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.911 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.911 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.6.23:1 3.3 3:d4fb61be:::smithi19530714-2 low_tier:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.911 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:11:26.911 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.716689148, 2024-09-13T10:11:26.911 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.038168746000000003, 2024-09-13T10:11:26.911 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.911 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.912 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.912 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.912 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.912 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.912 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:11:26.912 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.912 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.912 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.912 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.912 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:11:26.912 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.912 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.912 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.913 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.913 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120131+0000", 2024-09-13T10:11:26.913 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5620000000000002e-06 2024-09-13T10:11:26.913 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.913 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.913 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.913 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120140+0000", 2024-09-13T10:11:26.913 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1889999999999996e-06 2024-09-13T10:11:26.913 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.913 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.913 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.913 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120161+0000", 2024-09-13T10:11:26.913 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0485000000000001e-05 2024-09-13T10:11:26.914 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.914 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.914 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.914 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120200+0000", 2024-09-13T10:11:26.914 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9648999999999999e-05 2024-09-13T10:11:26.914 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.914 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.914 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.914 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120285+0000", 2024-09-13T10:11:26.914 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.4488999999999997e-05 2024-09-13T10:11:26.914 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.914 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.914 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.914 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120399+0000", 2024-09-13T10:11:26.915 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000113857 2024-09-13T10:11:26.915 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.915 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.915 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.915 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120640+0000", 2024-09-13T10:11:26.915 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024112600000000001 2024-09-13T10:11:26.915 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.915 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.915 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.915 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121715+0000", 2024-09-13T10:11:26.915 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001074885 2024-09-13T10:11:26.915 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.916 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.916 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.916 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.122870+0000", 2024-09-13T10:11:26.916 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00115504 2024-09-13T10:11:26.916 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.916 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.916 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.916 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158160+0000", 2024-09-13T10:11:26.916 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035289760000000003 2024-09-13T10:11:26.917 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.917 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.917 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.917 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158211+0000", 2024-09-13T10:11:26.917 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.1005999999999999e-05 2024-09-13T10:11:26.917 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.917 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.917 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.918 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158297+0000", 2024-09-13T10:11:26.918 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.6698000000000002e-05 2024-09-13T10:11:26.918 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.918 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.918 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.918 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.918 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.918 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.5.22:3 3.3 3:f565152a:::smithi19530714-50 low_tier:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:26.918 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.264828+0000", 2024-09-13T10:11:26.918 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 35.571990130000003, 2024-09-13T10:11:26.918 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011635411, 2024-09-13T10:11:26.918 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.918 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.919 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.919 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.919 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.919 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.919 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264828+0000", 2024-09-13T10:11:26.919 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.919 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.919 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.919 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.919 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264828+0000", 2024-09-13T10:11:26.919 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.919 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.919 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.920 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.920 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264830+0000", 2024-09-13T10:11:26.920 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.283e-06 2024-09-13T10:11:26.920 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.920 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.920 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.920 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264841+0000", 2024-09-13T10:11:26.920 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0893999999999999e-05 2024-09-13T10:11:26.920 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.920 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.920 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.920 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264865+0000", 2024-09-13T10:11:26.920 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4516999999999999e-05 2024-09-13T10:11:26.921 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.921 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.921 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.921 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264883+0000", 2024-09-13T10:11:26.921 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7759000000000001e-05 2024-09-13T10:11:26.921 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.921 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.921 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.921 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.265917+0000", 2024-09-13T10:11:26.921 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0010340390000000001 2024-09-13T10:11:26.921 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.921 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.922 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.922 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.266107+0000", 2024-09-13T10:11:26.922 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000189435 2024-09-13T10:11:26.922 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.922 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.922 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.922 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.266566+0000", 2024-09-13T10:11:26.923 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00045961599999999999 2024-09-13T10:11:26.923 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.923 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.923 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.923 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.268500+0000", 2024-09-13T10:11:26.923 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019334549999999999 2024-09-13T10:11:26.923 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.923 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.924 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.924 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.269782+0000", 2024-09-13T10:11:26.924 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001282444 2024-09-13T10:11:26.924 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.924 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.924 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.924 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276234+0000", 2024-09-13T10:11:26.924 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0064523580000000001 2024-09-13T10:11:26.924 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.924 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.924 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.924 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276372+0000", 2024-09-13T10:11:26.924 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013788099999999999 2024-09-13T10:11:26.925 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.925 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.925 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.925 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276463+0000", 2024-09-13T10:11:26.925 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.0729999999999994e-05 2024-09-13T10:11:26.925 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.925 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.925 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.925 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.925 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.925 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.8.23:15 3.3 3:e24de40a:::8acc92edede4f51df2e7506dbd189cab8b5b01cc17028d04a4fc401eaf5ce1ea:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:26.925 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:06.115725+0000", 2024-09-13T10:11:26.926 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 20.721092281000001, 2024-09-13T10:11:26.926 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012080047999999999, 2024-09-13T10:11:26.926 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.926 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.926 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.926 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.926 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.926 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.926 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.115725+0000", 2024-09-13T10:11:26.927 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.927 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.927 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.927 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.927 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.115725+0000", 2024-09-13T10:11:26.927 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.927 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.927 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.927 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.927 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.115727+0000", 2024-09-13T10:11:26.927 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9939999999999999e-06 2024-09-13T10:11:26.927 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.927 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.928 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.928 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116297+0000", 2024-09-13T10:11:26.928 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00056970499999999999 2024-09-13T10:11:26.928 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.928 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.928 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.928 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116330+0000", 2024-09-13T10:11:26.928 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2727e-05 2024-09-13T10:11:26.928 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.928 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.928 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.928 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116352+0000", 2024-09-13T10:11:26.928 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2569e-05 2024-09-13T10:11:26.929 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.929 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.929 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.929 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.119140+0000", 2024-09-13T10:11:26.929 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0027870780000000001 2024-09-13T10:11:26.929 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.929 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.929 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.929 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.119363+0000", 2024-09-13T10:11:26.929 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022384299999999999 2024-09-13T10:11:26.929 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.929 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.929 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.930 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.119746+0000", 2024-09-13T10:11:26.930 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000382533 2024-09-13T10:11:26.930 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.930 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.930 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.930 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.127251+0000", 2024-09-13T10:11:26.930 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075055720000000003 2024-09-13T10:11:26.930 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.930 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.931 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.931 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.127366+0000", 2024-09-13T10:11:26.931 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000114695 2024-09-13T10:11:26.931 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.931 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.931 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.931 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.127516+0000", 2024-09-13T10:11:26.931 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014997 2024-09-13T10:11:26.931 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.931 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.931 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.931 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.127700+0000", 2024-09-13T10:11:26.931 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000184267 2024-09-13T10:11:26.932 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.932 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.932 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.932 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.127805+0000", 2024-09-13T10:11:26.932 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000105095 2024-09-13T10:11:26.932 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.933 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.933 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.933 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.933 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.933 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.8.23:21 3.3 3:c7dc066d:::f97d511732b2d9ef6d00b0eccb7f53f5d47aafd7ca868e5742459233eacb7726:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:26.933 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:06.116395+0000", 2024-09-13T10:11:26.933 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 20.720422620000001, 2024-09-13T10:11:26.933 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012048580999999999, 2024-09-13T10:11:26.934 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.934 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.934 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.934 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.934 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.934 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.934 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116395+0000", 2024-09-13T10:11:26.934 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.934 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.934 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.934 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.934 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116395+0000", 2024-09-13T10:11:26.934 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.935 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.935 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.935 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.935 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116397+0000", 2024-09-13T10:11:26.935 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1299999999999999e-06 2024-09-13T10:11:26.935 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.935 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.935 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.935 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116589+0000", 2024-09-13T10:11:26.935 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019133 2024-09-13T10:11:26.935 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.935 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.935 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.936 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116611+0000", 2024-09-13T10:11:26.936 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2759e-05 2024-09-13T10:11:26.936 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.936 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.936 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.936 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116633+0000", 2024-09-13T10:11:26.936 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1353e-05 2024-09-13T10:11:26.936 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.936 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.936 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.937 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.122898+0000", 2024-09-13T10:11:26.937 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0062655569999999997 2024-09-13T10:11:26.937 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.937 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.937 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.937 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.123242+0000", 2024-09-13T10:11:26.937 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00034350899999999998 2024-09-13T10:11:26.937 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.937 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.937 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.937 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.123797+0000", 2024-09-13T10:11:26.937 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00055495700000000002 2024-09-13T10:11:26.938 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.938 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.938 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.938 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.127843+0000", 2024-09-13T10:11:26.938 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0040462609999999998 2024-09-13T10:11:26.938 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.938 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.938 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.938 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.127966+0000", 2024-09-13T10:11:26.938 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012303100000000001 2024-09-13T10:11:26.938 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.938 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.938 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.939 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.128218+0000", 2024-09-13T10:11:26.939 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025239499999999998 2024-09-13T10:11:26.939 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.939 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.939 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.939 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.128330+0000", 2024-09-13T10:11:26.939 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000111365 2024-09-13T10:11:26.939 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.939 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.939 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.939 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.128444+0000", 2024-09-13T10:11:26.940 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000113934 2024-09-13T10:11:26.940 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.940 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.940 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.940 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.940 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.940 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.8.23:23 3.3 3:e316fc5a:::252a0f0dcd13ac7cc8768c369d40341dfd148109baa17147b0e0aef9e7d09876:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:26.940 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:06.116669+0000", 2024-09-13T10:11:26.940 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 20.720149095, 2024-09-13T10:11:26.940 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012469427, 2024-09-13T10:11:26.940 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.941 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.941 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.941 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.941 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.941 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.941 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116669+0000", 2024-09-13T10:11:26.941 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.941 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.941 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.941 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.941 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116669+0000", 2024-09-13T10:11:26.941 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.941 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.942 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.942 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.942 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116670+0000", 2024-09-13T10:11:26.942 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.756e-06 2024-09-13T10:11:26.942 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.942 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.942 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.942 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116850+0000", 2024-09-13T10:11:26.942 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017943700000000001 2024-09-13T10:11:26.942 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.942 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.942 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.943 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116878+0000", 2024-09-13T10:11:26.943 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8119e-05 2024-09-13T10:11:26.943 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.943 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.943 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.943 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116901+0000", 2024-09-13T10:11:26.943 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2963999999999999e-05 2024-09-13T10:11:26.943 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.943 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.943 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.943 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.125063+0000", 2024-09-13T10:11:26.943 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.008162117 2024-09-13T10:11:26.944 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.944 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.944 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.944 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.125376+0000", 2024-09-13T10:11:26.944 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000312627 2024-09-13T10:11:26.944 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.944 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.944 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.944 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.125950+0000", 2024-09-13T10:11:26.944 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00057483899999999995 2024-09-13T10:11:26.944 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.944 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.945 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.945 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.128105+0000", 2024-09-13T10:11:26.945 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0021549630000000002 2024-09-13T10:11:26.945 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.945 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.945 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.945 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.128661+0000", 2024-09-13T10:11:26.945 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00055533700000000004 2024-09-13T10:11:26.945 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.945 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.945 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.945 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.128932+0000", 2024-09-13T10:11:26.945 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00027158199999999998 2024-09-13T10:11:26.946 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.946 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.946 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.946 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.129040+0000", 2024-09-13T10:11:26.946 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000107871 2024-09-13T10:11:26.946 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.946 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.946 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.946 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.129138+0000", 2024-09-13T10:11:26.946 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7814999999999998e-05 2024-09-13T10:11:26.946 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.946 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.947 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.947 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.947 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.947 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.14.23:42 3.3 3:ce7ec0f0:::302c6a3bdf6d249958d305a37c5742c0de5d7cad722f2dabd19f570d2f795527:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:26.947 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:10.780924+0000", 2024-09-13T10:11:26.947 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 16.055893447999999, 2024-09-13T10:11:26.947 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013436228, 2024-09-13T10:11:26.947 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.947 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.947 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.947 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.947 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.947 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.948 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.780924+0000", 2024-09-13T10:11:26.948 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.948 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.948 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.948 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.948 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.780924+0000", 2024-09-13T10:11:26.948 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.948 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.948 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.949 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.949 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.780927+0000", 2024-09-13T10:11:26.949 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3860000000000001e-06 2024-09-13T10:11:26.949 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.949 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.949 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.949 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781100+0000", 2024-09-13T10:11:26.949 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000172967 2024-09-13T10:11:26.949 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.949 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.949 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.949 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781123+0000", 2024-09-13T10:11:26.949 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.302e-05 2024-09-13T10:11:26.950 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.950 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.950 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.950 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781141+0000", 2024-09-13T10:11:26.950 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8692999999999999e-05 2024-09-13T10:11:26.950 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.950 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.950 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.950 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.785827+0000", 2024-09-13T10:11:26.950 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0046856850000000002 2024-09-13T10:11:26.950 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.950 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.950 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.951 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.786033+0000", 2024-09-13T10:11:26.951 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00020591000000000001 2024-09-13T10:11:26.951 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.951 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.951 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.951 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.786402+0000", 2024-09-13T10:11:26.951 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036895 2024-09-13T10:11:26.951 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.951 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.951 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.951 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.793957+0000", 2024-09-13T10:11:26.951 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075548940000000004 2024-09-13T10:11:26.951 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.951 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.952 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.952 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794103+0000", 2024-09-13T10:11:26.952 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014615799999999999 2024-09-13T10:11:26.952 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.952 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.952 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.952 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794217+0000", 2024-09-13T10:11:26.952 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011437799999999999 2024-09-13T10:11:26.952 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.952 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.952 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.952 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794297+0000", 2024-09-13T10:11:26.953 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9431999999999994e-05 2024-09-13T10:11:26.953 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.953 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.953 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.953 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794360+0000", 2024-09-13T10:11:26.953 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.3755000000000002e-05 2024-09-13T10:11:26.953 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.953 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.953 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.953 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.953 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.953 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.14.23:44 3.3 3:e2816a99:::d4bb749789c90c884ffd20ac62f804c7a6b80502d14432fa627e1239249ee2da:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:26.954 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:10.781168+0000", 2024-09-13T10:11:26.954 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 16.055649539000001, 2024-09-13T10:11:26.954 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013562833, 2024-09-13T10:11:26.954 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.954 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.954 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.954 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.954 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.954 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.954 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781168+0000", 2024-09-13T10:11:26.954 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.954 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.954 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.955 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.955 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781168+0000", 2024-09-13T10:11:26.955 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.955 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.955 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.955 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.955 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781170+0000", 2024-09-13T10:11:26.955 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5400000000000001e-06 2024-09-13T10:11:26.955 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.955 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.955 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.955 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781470+0000", 2024-09-13T10:11:26.955 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00030077100000000002 2024-09-13T10:11:26.955 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.956 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.956 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.956 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781508+0000", 2024-09-13T10:11:26.956 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.7517e-05 2024-09-13T10:11:26.956 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.956 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.956 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.956 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781527+0000", 2024-09-13T10:11:26.956 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8766e-05 2024-09-13T10:11:26.957 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.957 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.957 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.957 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.788166+0000", 2024-09-13T10:11:26.957 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0066389450000000003 2024-09-13T10:11:26.957 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.957 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.957 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.957 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.788515+0000", 2024-09-13T10:11:26.957 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00034976099999999997 2024-09-13T10:11:26.957 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.957 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.957 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.958 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.789180+0000", 2024-09-13T10:11:26.958 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00066460200000000003 2024-09-13T10:11:26.958 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.958 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.958 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.958 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794399+0000", 2024-09-13T10:11:26.958 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0052185340000000004 2024-09-13T10:11:26.958 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.958 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.958 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.958 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794487+0000", 2024-09-13T10:11:26.958 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8660999999999997e-05 2024-09-13T10:11:26.958 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.958 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.959 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.959 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794601+0000", 2024-09-13T10:11:26.959 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000114188 2024-09-13T10:11:26.959 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.959 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.959 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.959 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794678+0000", 2024-09-13T10:11:26.959 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6058999999999998e-05 2024-09-13T10:11:26.959 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.959 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.959 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.960 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794731+0000", 2024-09-13T10:11:26.960 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.3489000000000002e-05 2024-09-13T10:11:26.960 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.960 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.960 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.960 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.960 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.960 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.14.23:47 3.3 3:ffec9690:::b0d2dba9f365561c17cd4cce8c8a8f4d8f931d05ca998f81ae8f5caa986fcf1e:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:26.960 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:10.781559+0000", 2024-09-13T10:11:26.960 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 16.055258242000001, 2024-09-13T10:11:26.960 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015576100000000001, 2024-09-13T10:11:26.960 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.961 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.961 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.961 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.961 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.961 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.961 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781559+0000", 2024-09-13T10:11:26.961 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.961 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.961 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.961 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.961 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781559+0000", 2024-09-13T10:11:26.962 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.962 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.962 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.962 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.962 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781561+0000", 2024-09-13T10:11:26.962 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5460000000000001e-06 2024-09-13T10:11:26.962 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.962 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.962 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.962 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781708+0000", 2024-09-13T10:11:26.962 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000147291 2024-09-13T10:11:26.963 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.963 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.963 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.963 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781735+0000", 2024-09-13T10:11:26.963 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7175999999999999e-05 2024-09-13T10:11:26.963 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.963 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.963 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.963 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781755+0000", 2024-09-13T10:11:26.963 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9653e-05 2024-09-13T10:11:26.963 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.963 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.963 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.964 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.790861+0000", 2024-09-13T10:11:26.964 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0091056920000000003 2024-09-13T10:11:26.964 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.964 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.964 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.964 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.791072+0000", 2024-09-13T10:11:26.964 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000211103 2024-09-13T10:11:26.964 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.964 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.964 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.965 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.791511+0000", 2024-09-13T10:11:26.965 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00043908899999999998 2024-09-13T10:11:26.965 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.965 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.965 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.965 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794857+0000", 2024-09-13T10:11:26.965 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0033462940000000001 2024-09-13T10:11:26.965 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.965 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.965 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.965 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794945+0000", 2024-09-13T10:11:26.965 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7626000000000004e-05 2024-09-13T10:11:26.966 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.966 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.966 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.966 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.797001+0000", 2024-09-13T10:11:26.966 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020557280000000002 2024-09-13T10:11:26.966 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.966 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.966 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.966 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.797080+0000", 2024-09-13T10:11:26.966 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9169e-05 2024-09-13T10:11:26.967 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.967 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.967 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.967 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.797136+0000", 2024-09-13T10:11:26.967 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5733000000000003e-05 2024-09-13T10:11:26.967 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.968 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.968 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.968 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.968 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.968 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.6.23:158 3.3 3:f5f18922:::5fd497cf663f539dfd95b3997c5a776f4928ee513a2033637dac7a26a4ef0dd6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:11:26.969 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:18.185966+0000", 2024-09-13T10:11:26.969 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 8.6508515579999994, 2024-09-13T10:11:26.969 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012329359999999999, 2024-09-13T10:11:26.969 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.969 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.969 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.970 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.970 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.970 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.970 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.185966+0000", 2024-09-13T10:11:26.970 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.970 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.970 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.971 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.971 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.185966+0000", 2024-09-13T10:11:26.971 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.971 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.971 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.971 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.972 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.185967+0000", 2024-09-13T10:11:26.972 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2200000000000003e-07 2024-09-13T10:11:26.972 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.972 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.972 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.973 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186094+0000", 2024-09-13T10:11:26.973 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012747100000000001 2024-09-13T10:11:26.973 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.973 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.973 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.973 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186104+0000", 2024-09-13T10:11:26.974 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.2560000000000008e-06 2024-09-13T10:11:26.974 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.974 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.974 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.974 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186108+0000", 2024-09-13T10:11:26.974 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.583e-06 2024-09-13T10:11:26.975 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.975 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.975 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.975 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.192945+0000", 2024-09-13T10:11:26.975 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0068368170000000002 2024-09-13T10:11:26.975 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.976 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.976 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.976 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.193177+0000", 2024-09-13T10:11:26.976 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000232458 2024-09-13T10:11:26.976 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.977 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.977 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.977 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.193568+0000", 2024-09-13T10:11:26.977 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039028799999999997 2024-09-13T10:11:26.977 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.980 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.980 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.980 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.196406+0000", 2024-09-13T10:11:26.981 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0028379529999999998 2024-09-13T10:11:26.981 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.981 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.981 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.981 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.196839+0000", 2024-09-13T10:11:26.981 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00043299899999999999 2024-09-13T10:11:26.982 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.982 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.982 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.982 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.198197+0000", 2024-09-13T10:11:26.982 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0013580560000000001 2024-09-13T10:11:26.982 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.983 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.983 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.983 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.198250+0000", 2024-09-13T10:11:26.983 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.3563999999999997e-05 2024-09-13T10:11:26.983 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.983 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.984 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.984 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.198296+0000", 2024-09-13T10:11:26.984 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.5192999999999999e-05 2024-09-13T10:11:26.984 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.984 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.984 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.985 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.985 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.985 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.6.23:159 3.3 3:d330ea58:::b1e0a8f1f87dd65498385354f65ab6aa4dbeb3cb948fa25b5d7b0975437eef68:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:11:26.985 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:18.186115+0000", 2024-09-13T10:11:26.985 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 8.6507022409999994, 2024-09-13T10:11:26.986 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.01270717, 2024-09-13T10:11:26.986 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:26.986 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:26.986 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:26.986 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:26.986 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.987 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:26.987 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186115+0000", 2024-09-13T10:11:26.987 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.987 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.987 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.987 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:26.988 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186115+0000", 2024-09-13T10:11:26.988 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:26.988 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.988 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.988 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:26.988 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186116+0000", 2024-09-13T10:11:26.989 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5599999999999995e-07 2024-09-13T10:11:26.989 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.989 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.989 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:26.989 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186174+0000", 2024-09-13T10:11:26.989 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8397999999999999e-05 2024-09-13T10:11:26.990 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.990 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.990 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:26.990 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186182+0000", 2024-09-13T10:11:26.990 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.729e-06 2024-09-13T10:11:26.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.991 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.991 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:26.991 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186189+0000", 2024-09-13T10:11:26.991 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4629999999999998e-06 2024-09-13T10:11:26.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.992 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.992 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:26.992 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.195175+0000", 2024-09-13T10:11:26.992 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0089862550000000003 2024-09-13T10:11:26.992 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.992 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.993 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:26.993 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.195287+0000", 2024-09-13T10:11:26.993 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011233700000000001 2024-09-13T10:11:26.993 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.993 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.994 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:26.994 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.195457+0000", 2024-09-13T10:11:26.994 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016981799999999999 2024-09-13T10:11:26.994 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.994 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.994 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.994 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.196713+0000", 2024-09-13T10:11:26.994 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001256463 2024-09-13T10:11:26.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.995 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.995 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:26.995 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.197341+0000", 2024-09-13T10:11:26.995 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00062791400000000003 2024-09-13T10:11:26.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.995 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.995 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:26.995 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.198733+0000", 2024-09-13T10:11:26.995 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001391403 2024-09-13T10:11:26.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.995 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.995 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:26.996 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.198781+0000", 2024-09-13T10:11:26.996 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.8680000000000001e-05 2024-09-13T10:11:26.996 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:26.996 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:26.996 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:26.996 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.198823+0000", 2024-09-13T10:11:26.996 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1153999999999999e-05 2024-09-13T10:11:26.996 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.996 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.996 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.996 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:26.996 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:26.996 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:26.998 INFO:tasks.rados.rados.0.smithi195.stdout:1217: expect (ObjNum 184 snap 0 seq_num 184) 2024-09-13T10:11:26.998 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:26.998 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:26.999 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-13T10:11:26.999 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:26.995+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:27.019 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 510718 2024-09-13T10:11:27.024 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:27.024 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:27.047 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:27.048 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:27.087 INFO:tasks.rados.rados.0.smithi195.stdout:1220: expect (ObjNum 209 snap 0 seq_num 209) 2024-09-13T10:11:27.087 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:27.099 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:27.096+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:27.150 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:27.170 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:27.170 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:27.170 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:11:27.171 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:27.185 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-13T10:11:27.200 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:27.196+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:27.213 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:27.277 INFO:tasks.rados.rados.0.smithi195.stdout:1221: seq_num 230 ranges {665610=797860,1909329=458405,2816700=410210} 2024-09-13T10:11:27.300 INFO:tasks.rados.rados.0.smithi195.stdout:1221: writing smithi19530714-43 from 665610 to 1463470 tid 1 2024-09-13T10:11:27.301 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:27.299+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:27.302 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:27.302 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:27.302 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:11:27.303 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:11:27.303 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:27.317 INFO:tasks.rados.rados.0.smithi195.stdout:1221: writing smithi19530714-43 from 1909329 to 2367734 tid 2 2024-09-13T10:11:27.318 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-13T10:11:27.330 INFO:tasks.rados.rados.0.smithi195.stdout:1221: writing smithi19530714-43 from 2816700 to 3226910 tid 3 2024-09-13T10:11:27.330 INFO:tasks.rados.rados.0.smithi195.stdout:1211: done (7 left) 2024-09-13T10:11:27.330 INFO:tasks.rados.rados.0.smithi195.stdout:1214: done (6 left) 2024-09-13T10:11:27.330 INFO:tasks.rados.rados.0.smithi195.stdout:1216: done (5 left) 2024-09-13T10:11:27.330 INFO:tasks.rados.rados.0.smithi195.stdout:1217: done (4 left) 2024-09-13T10:11:27.330 INFO:tasks.rados.rados.0.smithi195.stdout:1218: done (3 left) 2024-09-13T10:11:27.330 INFO:tasks.rados.rados.0.smithi195.stdout:1219: done (2 left) 2024-09-13T10:11:27.330 INFO:tasks.rados.rados.0.smithi195.stdout:1220: done (1 left) 2024-09-13T10:11:27.330 INFO:tasks.rados.rados.0.smithi195.stdout:1222: read oid 29 snap -1 2024-09-13T10:11:27.330 INFO:tasks.rados.rados.0.smithi195.stdout:1222: expect (ObjNum 220 snap 0 seq_num 220) 2024-09-13T10:11:27.330 INFO:tasks.rados.rados.0.smithi195.stdout:1223: write oid 5 current snap is 0 2024-09-13T10:11:27.331 INFO:tasks.rados.rados.0.smithi195.stdout:1221: finishing write tid 1 to smithi19530714-43 2024-09-13T10:11:27.331 INFO:tasks.rados.rados.0.smithi195.stdout:1221: finishing write tid 2 to smithi19530714-43 2024-09-13T10:11:27.331 INFO:tasks.rados.rados.0.smithi195.stdout:1223: seq_num 231 ranges {682417=473109,1567811=602063,2844123=621482} 2024-09-13T10:11:27.345 INFO:tasks.rados.rados.0.smithi195.stdout:1223: writing smithi19530714-5 from 682417 to 1155526 tid 1 2024-09-13T10:11:27.359 INFO:tasks.rados.rados.0.smithi195.stdout:1223: writing smithi19530714-5 from 1567811 to 2169874 tid 2 2024-09-13T10:11:27.373 INFO:tasks.rados.rados.0.smithi195.stdout:1223: writing smithi19530714-5 from 2844123 to 3465605 tid 3 2024-09-13T10:11:27.374 INFO:tasks.rados.rados.0.smithi195.stdout:1224: set_chunk oid 46 target oid 2024-09-13T10:11:27.374 INFO:tasks.rados.rados.0.smithi195.stdout:1224: set_chunk oid 46 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:27.374 INFO:tasks.rados.rados.0.smithi195.stdout:1225: read oid 28 snap -1 2024-09-13T10:11:27.374 INFO:tasks.rados.rados.0.smithi195.stdout:1225: expect (ObjNum 180 snap 0 seq_num 180) 2024-09-13T10:11:27.374 INFO:tasks.rados.rados.0.smithi195.stdout:1226: tier_evict oid 3 2024-09-13T10:11:27.374 INFO:tasks.rados.rados.0.smithi195.stdout:1226: tier_evict oid 3 snap -1 2024-09-13T10:11:27.374 INFO:tasks.rados.rados.0.smithi195.stdout:1221: finishing write tid 3 to smithi19530714-43 2024-09-13T10:11:27.374 INFO:tasks.rados.rados.0.smithi195.stdout:1221: finishing write tid 4 to smithi19530714-43 2024-09-13T10:11:27.374 INFO:tasks.rados.rados.0.smithi195.stdout:1221: finishing write tid 5 to smithi19530714-43 2024-09-13T10:11:27.374 INFO:tasks.rados.rados.0.smithi195.stdout:1221: oid 43 updating version 0 to 68 2024-09-13T10:11:27.375 INFO:tasks.rados.rados.0.smithi195.stdout:1221: oid 43 version 68 is already newer than 65 2024-09-13T10:11:27.375 INFO:tasks.rados.rados.0.smithi195.stdout:1221: oid 43 version 68 is already newer than 66 2024-09-13T10:11:27.375 INFO:tasks.rados.rados.0.smithi195.stdout:1221: oid 43 version 68 is already newer than 67 2024-09-13T10:11:27.375 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 68 (ObjNum 230 snap 0 seq_num 230) dirty exists 2024-09-13T10:11:27.375 INFO:tasks.rados.rados.0.smithi195.stdout:1221: left oid 43 (ObjNum 230 snap 0 seq_num 230) 2024-09-13T10:11:27.375 INFO:tasks.rados.rados.0.smithi195.stdout:1222: expect (ObjNum 220 snap 0 seq_num 220) 2024-09-13T10:11:27.402 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:27.400+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:27.470 INFO:tasks.rados.rados.0.smithi195.stdout:1223: finishing write tid 1 to smithi19530714-5 2024-09-13T10:11:27.470 INFO:tasks.rados.rados.0.smithi195.stdout:1223: finishing write tid 2 to smithi19530714-5 2024-09-13T10:11:27.470 INFO:tasks.rados.rados.0.smithi195.stdout:1226: got (22) Invalid argument 2024-09-13T10:11:27.471 INFO:tasks.rados.rados.0.smithi195.stdout:1224: finishing set_chunk to oid 46 2024-09-13T10:11:27.471 INFO:tasks.rados.rados.0.smithi195.stdout:1224: got expected ENOENT (src dne) 2024-09-13T10:11:27.471 INFO:tasks.rados.rados.0.smithi195.stdout:1223: finishing write tid 3 to smithi19530714-5 2024-09-13T10:11:27.471 INFO:tasks.rados.rados.0.smithi195.stdout:1225: expect (ObjNum 180 snap 0 seq_num 180) 2024-09-13T10:11:27.488 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:27.488 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:11:27.488 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:11:27.489 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:11:27.489 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.489 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:202 3.2 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.489 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.991310+0000", 2024-09-13T10:11:27.489 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.493004829, 2024-09-13T10:11:27.489 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.040945251000000002, 2024-09-13T10:11:27.489 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.489 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.489 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.489 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.489 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.490 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.490 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 202 2024-09-13T10:11:27.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.490 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.490 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.490 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.490 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991310+0000", 2024-09-13T10:11:27.490 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.490 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.490 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.490 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991310+0000", 2024-09-13T10:11:27.490 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.491 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.491 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.491 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991314+0000", 2024-09-13T10:11:27.491 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.7100000000000001e-06 2024-09-13T10:11:27.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.491 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.491 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.491 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991328+0000", 2024-09-13T10:11:27.491 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.413e-05 2024-09-13T10:11:27.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.491 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.491 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.492 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991361+0000", 2024-09-13T10:11:27.492 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2512999999999999e-05 2024-09-13T10:11:27.492 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.492 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.492 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.492 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991377+0000", 2024-09-13T10:11:27.492 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6478999999999999e-05 2024-09-13T10:11:27.492 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.492 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.493 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.493 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991546+0000", 2024-09-13T10:11:27.493 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016856600000000001 2024-09-13T10:11:27.493 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.493 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.493 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.493 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991850+0000", 2024-09-13T10:11:27.493 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00030478800000000001 2024-09-13T10:11:27.493 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.493 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.493 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.493 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.001416+0000", 2024-09-13T10:11:27.493 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0095659279999999996 2024-09-13T10:11:27.494 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.494 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.494 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.494 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031502+0000", 2024-09-13T10:11:27.494 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.030085526000000001 2024-09-13T10:11:27.494 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.494 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.494 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.494 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031585+0000", 2024-09-13T10:11:27.494 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.3175000000000002e-05 2024-09-13T10:11:27.494 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.495 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.495 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.495 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031775+0000", 2024-09-13T10:11:27.495 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000190161 2024-09-13T10:11:27.495 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.495 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.495 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.495 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031982+0000", 2024-09-13T10:11:27.495 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000207245 2024-09-13T10:11:27.495 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.495 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.495 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.496 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032255+0000", 2024-09-13T10:11:27.496 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00027303000000000001 2024-09-13T10:11:27.496 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.496 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.496 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.496 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.496 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.497 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:206 3.2 3:4cbdbe9e:::smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.497 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.991649+0000", 2024-09-13T10:11:27.497 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.492666501000002, 2024-09-13T10:11:27.497 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041234240999999998, 2024-09-13T10:11:27.497 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.497 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.497 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.497 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.497 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.498 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.498 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 206 2024-09-13T10:11:27.498 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.498 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.498 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.498 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.498 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991649+0000", 2024-09-13T10:11:27.498 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.498 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.498 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.498 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.499 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991649+0000", 2024-09-13T10:11:27.499 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.499 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.499 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.499 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.499 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991650+0000", 2024-09-13T10:11:27.499 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.483e-06 2024-09-13T10:11:27.499 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.499 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.499 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.499 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991657+0000", 2024-09-13T10:11:27.499 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.0899999999999999e-06 2024-09-13T10:11:27.499 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.500 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.500 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.500 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991672+0000", 2024-09-13T10:11:27.500 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4514e-05 2024-09-13T10:11:27.500 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.500 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.500 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.500 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991681+0000", 2024-09-13T10:11:27.500 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6360000000000006e-06 2024-09-13T10:11:27.500 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.500 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.500 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.500 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992159+0000", 2024-09-13T10:11:27.501 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00047820099999999998 2024-09-13T10:11:27.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.501 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.501 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.501 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992411+0000", 2024-09-13T10:11:27.501 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025199700000000001 2024-09-13T10:11:27.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.501 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.501 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.501 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.009775+0000", 2024-09-13T10:11:27.501 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017363841000000001 2024-09-13T10:11:27.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.501 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.502 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.502 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032446+0000", 2024-09-13T10:11:27.502 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022670490000000001 2024-09-13T10:11:27.502 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.502 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.502 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.502 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032482+0000", 2024-09-13T10:11:27.502 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5902000000000003e-05 2024-09-13T10:11:27.502 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.502 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.502 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.502 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032612+0000", 2024-09-13T10:11:27.502 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013022700000000001 2024-09-13T10:11:27.502 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032767+0000", 2024-09-13T10:11:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015477000000000001 2024-09-13T10:11:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032883+0000", 2024-09-13T10:11:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011608999999999999 2024-09-13T10:11:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.503 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.504 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:210 3.2 3:46ab5247:::smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.504 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.992337+0000", 2024-09-13T10:11:27.504 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.491978357999997, 2024-09-13T10:11:27.504 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.057985696000000003, 2024-09-13T10:11:27.504 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.504 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.504 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.504 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.504 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.504 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.505 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 210 2024-09-13T10:11:27.505 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.505 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.505 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.505 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.505 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992337+0000", 2024-09-13T10:11:27.505 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.505 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.505 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.505 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.505 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992337+0000", 2024-09-13T10:11:27.506 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.506 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.506 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.506 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.506 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992339+0000", 2024-09-13T10:11:27.506 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2400000000000002e-06 2024-09-13T10:11:27.506 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.506 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.506 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.506 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992350+0000", 2024-09-13T10:11:27.507 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1259e-05 2024-09-13T10:11:27.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.507 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.507 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.507 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992368+0000", 2024-09-13T10:11:27.507 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8133000000000001e-05 2024-09-13T10:11:27.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.507 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.507 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.508 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992382+0000", 2024-09-13T10:11:27.508 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3837e-05 2024-09-13T10:11:27.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.508 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.508 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.508 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992752+0000", 2024-09-13T10:11:27.508 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037008000000000001 2024-09-13T10:11:27.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.508 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.508 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.508 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992980+0000", 2024-09-13T10:11:27.509 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022799100000000001 2024-09-13T10:11:27.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.509 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.509 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.509 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027651+0000", 2024-09-13T10:11:27.509 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.034671027 2024-09-13T10:11:27.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.509 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.509 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.509 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.034868+0000", 2024-09-13T10:11:27.509 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0072169920000000002 2024-09-13T10:11:27.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.509 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.510 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.510 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038033+0000", 2024-09-13T10:11:27.510 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003164966 2024-09-13T10:11:27.510 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.510 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.510 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.510 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050187+0000", 2024-09-13T10:11:27.510 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012153815 2024-09-13T10:11:27.510 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.511 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.511 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.511 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050266+0000", 2024-09-13T10:11:27.511 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9359999999999999e-05 2024-09-13T10:11:27.511 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.511 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.511 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.511 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050322+0000", 2024-09-13T10:11:27.511 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5995999999999997e-05 2024-09-13T10:11:27.511 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.511 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.511 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.511 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.512 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.512 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:214 3.2 3:68ba95ba:::smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.512 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.992479+0000", 2024-09-13T10:11:27.512 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.491836284999998, 2024-09-13T10:11:27.512 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.045987867000000002, 2024-09-13T10:11:27.512 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.512 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.512 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.512 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.512 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.512 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.512 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 214 2024-09-13T10:11:27.513 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.513 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.513 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.513 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.513 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992479+0000", 2024-09-13T10:11:27.513 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.513 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.513 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.513 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.513 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992479+0000", 2024-09-13T10:11:27.513 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.513 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.513 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.514 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.514 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992481+0000", 2024-09-13T10:11:27.514 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2960000000000001e-06 2024-09-13T10:11:27.514 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.514 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.514 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.514 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992493+0000", 2024-09-13T10:11:27.514 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2077e-05 2024-09-13T10:11:27.514 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.514 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.514 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.514 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992519+0000", 2024-09-13T10:11:27.514 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6282e-05 2024-09-13T10:11:27.514 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.515 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.515 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.515 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992538+0000", 2024-09-13T10:11:27.515 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9083e-05 2024-09-13T10:11:27.515 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.515 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.515 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.515 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993393+0000", 2024-09-13T10:11:27.515 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00085445200000000001 2024-09-13T10:11:27.515 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.515 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.515 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.515 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993682+0000", 2024-09-13T10:11:27.516 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000288895 2024-09-13T10:11:27.516 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.516 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.516 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.516 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.019700+0000", 2024-09-13T10:11:27.516 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.02601794 2024-09-13T10:11:27.516 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.516 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.516 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.516 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032498+0000", 2024-09-13T10:11:27.516 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012798535999999999 2024-09-13T10:11:27.516 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.516 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.517 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.517 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033641+0000", 2024-09-13T10:11:27.517 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001142569 2024-09-13T10:11:27.517 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.517 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.517 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.517 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038259+0000", 2024-09-13T10:11:27.517 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0046180939999999997 2024-09-13T10:11:27.517 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.517 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.517 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.517 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038379+0000", 2024-09-13T10:11:27.517 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000119914 2024-09-13T10:11:27.518 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.518 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.518 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.518 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038467+0000", 2024-09-13T10:11:27.518 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7729e-05 2024-09-13T10:11:27.518 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.518 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.519 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.519 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.519 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.519 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:217 3.2 3:554c406d:::smithi19530714-18 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.519 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.027515+0000", 2024-09-13T10:11:27.519 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.456800450999999, 2024-09-13T10:11:27.519 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032787180999999999, 2024-09-13T10:11:27.519 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.519 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.519 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.519 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.519 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.520 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.520 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 217 2024-09-13T10:11:27.520 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.520 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.520 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.520 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.520 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027515+0000", 2024-09-13T10:11:27.520 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.520 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.520 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.520 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.520 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027515+0000", 2024-09-13T10:11:27.520 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.521 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.521 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.521 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.521 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027518+0000", 2024-09-13T10:11:27.521 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.4429999999999999e-06 2024-09-13T10:11:27.521 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.521 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.521 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.521 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027532+0000", 2024-09-13T10:11:27.521 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3517e-05 2024-09-13T10:11:27.521 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.521 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.522 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.522 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027563+0000", 2024-09-13T10:11:27.522 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1158999999999998e-05 2024-09-13T10:11:27.522 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.522 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.522 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.522 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027585+0000", 2024-09-13T10:11:27.522 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.228e-05 2024-09-13T10:11:27.522 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.522 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.522 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.522 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033014+0000", 2024-09-13T10:11:27.522 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0054288649999999997 2024-09-13T10:11:27.523 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.523 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.523 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.523 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033269+0000", 2024-09-13T10:11:27.523 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000255054 2024-09-13T10:11:27.523 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.523 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.523 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.523 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.044618+0000", 2024-09-13T10:11:27.523 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011349391 2024-09-13T10:11:27.523 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.523 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.523 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.523 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.054627+0000", 2024-09-13T10:11:27.524 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010008992 2024-09-13T10:11:27.524 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.524 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.524 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.524 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.058047+0000", 2024-09-13T10:11:27.524 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0034193000000000001 2024-09-13T10:11:27.524 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.524 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.524 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.524 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060139+0000", 2024-09-13T10:11:27.525 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020925240000000001 2024-09-13T10:11:27.525 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.525 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.525 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.525 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060222+0000", 2024-09-13T10:11:27.525 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.2917999999999995e-05 2024-09-13T10:11:27.525 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.525 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.525 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.525 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060302+0000", 2024-09-13T10:11:27.525 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9738000000000001e-05 2024-09-13T10:11:27.525 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.525 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.526 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.526 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.526 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.526 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:218 3.2 3:5915c927:::smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.526 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.032754+0000", 2024-09-13T10:11:27.526 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.451561183000003, 2024-09-13T10:11:27.526 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023630241, 2024-09-13T10:11:27.526 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.526 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.526 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.526 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.526 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.527 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.527 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 218 2024-09-13T10:11:27.527 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.527 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.527 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.527 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.527 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032754+0000", 2024-09-13T10:11:27.527 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.527 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.527 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.527 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.528 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032754+0000", 2024-09-13T10:11:27.528 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.528 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.528 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.528 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.528 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032757+0000", 2024-09-13T10:11:27.528 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6330000000000002e-06 2024-09-13T10:11:27.528 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.528 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.528 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.528 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032772+0000", 2024-09-13T10:11:27.528 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4636e-05 2024-09-13T10:11:27.529 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.529 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.529 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.529 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032796+0000", 2024-09-13T10:11:27.529 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3963000000000001e-05 2024-09-13T10:11:27.529 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.529 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.529 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.529 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032815+0000", 2024-09-13T10:11:27.529 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9159000000000001e-05 2024-09-13T10:11:27.529 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.530 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.530 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.530 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033742+0000", 2024-09-13T10:11:27.530 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00092672700000000002 2024-09-13T10:11:27.530 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.530 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.531 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.531 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.034124+0000", 2024-09-13T10:11:27.531 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00038220399999999997 2024-09-13T10:11:27.531 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.531 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.532 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.532 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.040963+0000", 2024-09-13T10:11:27.532 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0068392230000000002 2024-09-13T10:11:27.532 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.532 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.532 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.533 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050362+0000", 2024-09-13T10:11:27.533 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0093986060000000003 2024-09-13T10:11:27.533 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.533 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.533 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.533 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.054585+0000", 2024-09-13T10:11:27.533 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0042225429999999996 2024-09-13T10:11:27.534 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.534 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.534 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.534 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.056250+0000", 2024-09-13T10:11:27.534 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001665251 2024-09-13T10:11:27.534 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.534 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.534 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.534 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.056320+0000", 2024-09-13T10:11:27.534 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9913999999999995e-05 2024-09-13T10:11:27.534 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.534 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.534 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.535 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.056384+0000", 2024-09-13T10:11:27.535 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4381999999999994e-05 2024-09-13T10:11:27.535 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.535 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.535 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.535 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.535 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.535 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:220 3.2 3:753d5785:::smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.535 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.035716+0000", 2024-09-13T10:11:27.535 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.448598689999997, 2024-09-13T10:11:27.535 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.030171203000000001, 2024-09-13T10:11:27.536 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.536 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.536 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.536 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.536 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.536 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.536 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 220 2024-09-13T10:11:27.537 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.537 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.537 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.537 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.537 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035716+0000", 2024-09-13T10:11:27.537 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.537 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.537 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.537 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.537 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035716+0000", 2024-09-13T10:11:27.537 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.537 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.537 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.538 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.538 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035720+0000", 2024-09-13T10:11:27.538 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5310000000000002e-06 2024-09-13T10:11:27.538 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.538 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.538 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.538 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035735+0000", 2024-09-13T10:11:27.538 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5517000000000001e-05 2024-09-13T10:11:27.538 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.538 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.538 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.538 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035765+0000", 2024-09-13T10:11:27.538 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9771e-05 2024-09-13T10:11:27.539 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.539 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.539 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.539 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035786+0000", 2024-09-13T10:11:27.539 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0928e-05 2024-09-13T10:11:27.539 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.539 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.539 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.539 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035945+0000", 2024-09-13T10:11:27.539 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015871900000000001 2024-09-13T10:11:27.539 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.539 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.539 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.540 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.036337+0000", 2024-09-13T10:11:27.540 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039231299999999998 2024-09-13T10:11:27.540 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.540 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.540 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.540 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.051063+0000", 2024-09-13T10:11:27.540 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014725607 2024-09-13T10:11:27.540 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.540 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.540 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.540 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.055802+0000", 2024-09-13T10:11:27.540 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0047387929999999998 2024-09-13T10:11:27.540 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.541 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.541 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.541 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064353+0000", 2024-09-13T10:11:27.541 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0085516629999999993 2024-09-13T10:11:27.541 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.541 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.541 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.541 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065750+0000", 2024-09-13T10:11:27.541 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001396748 2024-09-13T10:11:27.541 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.541 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.541 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.542 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065811+0000", 2024-09-13T10:11:27.542 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.0770000000000003e-05 2024-09-13T10:11:27.542 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.542 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.542 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.542 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065888+0000", 2024-09-13T10:11:27.542 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6842999999999996e-05 2024-09-13T10:11:27.542 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.542 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.542 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.542 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.542 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.542 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:227 3.2 3:6ac7800c:::smithi19530714-28 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.543 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.060091+0000", 2024-09-13T10:11:27.543 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.424223625000003, 2024-09-13T10:11:27.543 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.034993362, 2024-09-13T10:11:27.543 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.543 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.543 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.543 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.543 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.543 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.543 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 227 2024-09-13T10:11:27.543 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.543 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.544 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.544 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.544 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060091+0000", 2024-09-13T10:11:27.544 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.544 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.544 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.545 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.545 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060091+0000", 2024-09-13T10:11:27.545 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.545 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.545 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.546 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.546 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060094+0000", 2024-09-13T10:11:27.546 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5330000000000001e-06 2024-09-13T10:11:27.546 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.546 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.547 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.547 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060103+0000", 2024-09-13T10:11:27.547 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7020000000000007e-06 2024-09-13T10:11:27.547 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.547 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.547 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.548 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060121+0000", 2024-09-13T10:11:27.548 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.791e-05 2024-09-13T10:11:27.548 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.548 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.548 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.548 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060135+0000", 2024-09-13T10:11:27.549 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4059999999999999e-05 2024-09-13T10:11:27.549 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.549 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.549 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.549 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060346+0000", 2024-09-13T10:11:27.549 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021175600000000001 2024-09-13T10:11:27.550 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.550 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.550 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.550 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060559+0000", 2024-09-13T10:11:27.550 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021239499999999999 2024-09-13T10:11:27.550 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.551 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.551 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.551 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.073003+0000", 2024-09-13T10:11:27.551 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012444255 2024-09-13T10:11:27.551 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.551 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.552 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.552 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.081388+0000", 2024-09-13T10:11:27.552 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0083852119999999995 2024-09-13T10:11:27.552 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.552 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.552 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.552 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094289+0000", 2024-09-13T10:11:27.553 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012900271 2024-09-13T10:11:27.553 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.553 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.553 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.553 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094913+0000", 2024-09-13T10:11:27.553 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00062429200000000003 2024-09-13T10:11:27.554 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.554 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.554 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.554 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094972+0000", 2024-09-13T10:11:27.554 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9457999999999999e-05 2024-09-13T10:11:27.554 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.555 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.555 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.555 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095085+0000", 2024-09-13T10:11:27.555 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000112518 2024-09-13T10:11:27.555 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.555 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.555 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.555 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.556 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.556 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:230 3.2 3:5d108080:::smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.556 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.065002+0000", 2024-09-13T10:11:27.556 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.419313181, 2024-09-13T10:11:27.556 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.049477431000000002, 2024-09-13T10:11:27.556 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.556 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.557 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.557 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.557 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.557 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.557 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 230 2024-09-13T10:11:27.557 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.557 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.558 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.558 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.558 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065002+0000", 2024-09-13T10:11:27.558 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.558 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.558 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.559 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.559 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065002+0000", 2024-09-13T10:11:27.559 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.559 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.559 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.559 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.559 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065004+0000", 2024-09-13T10:11:27.560 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4269999999999998e-06 2024-09-13T10:11:27.560 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.560 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.560 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.560 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065014+0000", 2024-09-13T10:11:27.561 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.9799999999999993e-06 2024-09-13T10:11:27.561 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.561 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.561 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.561 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065038+0000", 2024-09-13T10:11:27.562 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4229000000000001e-05 2024-09-13T10:11:27.562 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.562 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.562 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.562 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065054+0000", 2024-09-13T10:11:27.562 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5764999999999999e-05 2024-09-13T10:11:27.563 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.563 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.563 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.563 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065171+0000", 2024-09-13T10:11:27.563 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117032 2024-09-13T10:11:27.564 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.564 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.564 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.564 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065513+0000", 2024-09-13T10:11:27.564 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00034131 2024-09-13T10:11:27.564 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.565 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.565 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.565 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.087042+0000", 2024-09-13T10:11:27.565 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021529534999999999 2024-09-13T10:11:27.565 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.565 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.565 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.565 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.097903+0000", 2024-09-13T10:11:27.566 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010861282999999999 2024-09-13T10:11:27.566 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.566 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.566 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.566 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.099922+0000", 2024-09-13T10:11:27.566 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020186539999999999 2024-09-13T10:11:27.566 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.566 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.567 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.567 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114391+0000", 2024-09-13T10:11:27.567 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014468880999999999 2024-09-13T10:11:27.567 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.567 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.568 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.568 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114434+0000", 2024-09-13T10:11:27.568 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.3511999999999999e-05 2024-09-13T10:11:27.568 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.568 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.568 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.568 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114479+0000", 2024-09-13T10:11:27.569 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4823000000000001e-05 2024-09-13T10:11:27.569 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.569 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.569 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.569 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.570 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.570 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:232 3.2 3:6538713c:::smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.570 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.065891+0000", 2024-09-13T10:11:27.570 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.418424309999999, 2024-09-13T10:11:27.570 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048804489999999999, 2024-09-13T10:11:27.570 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.571 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.571 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.571 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.571 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.571 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.571 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 232 2024-09-13T10:11:27.572 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.572 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.572 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.572 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.572 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065891+0000", 2024-09-13T10:11:27.572 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.573 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.573 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.573 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.573 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065891+0000", 2024-09-13T10:11:27.573 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.573 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.573 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.574 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.574 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065892+0000", 2024-09-13T10:11:27.574 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6249999999999999e-06 2024-09-13T10:11:27.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.574 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.574 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.574 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065902+0000", 2024-09-13T10:11:27.574 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.4150000000000005e-06 2024-09-13T10:11:27.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.574 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.574 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.574 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065920+0000", 2024-09-13T10:11:27.575 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7907e-05 2024-09-13T10:11:27.575 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.575 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.575 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065932+0000", 2024-09-13T10:11:27.575 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2469999999999999e-05 2024-09-13T10:11:27.575 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.575 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.575 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.066072+0000", 2024-09-13T10:11:27.575 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013935999999999999 2024-09-13T10:11:27.575 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.576 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.576 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.066435+0000", 2024-09-13T10:11:27.576 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036323300000000002 2024-09-13T10:11:27.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.576 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.576 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.576 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091133+0000", 2024-09-13T10:11:27.576 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024698494000000001 2024-09-13T10:11:27.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.577 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.577 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.577 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.101407+0000", 2024-09-13T10:11:27.577 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010273835 2024-09-13T10:11:27.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.577 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.577 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.577 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114342+0000", 2024-09-13T10:11:27.577 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012934826 2024-09-13T10:11:27.578 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.578 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.578 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.578 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114548+0000", 2024-09-13T10:11:27.578 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000206243 2024-09-13T10:11:27.579 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.579 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.579 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.579 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114630+0000", 2024-09-13T10:11:27.579 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.2127999999999997e-05 2024-09-13T10:11:27.579 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.580 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.580 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.580 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114695+0000", 2024-09-13T10:11:27.580 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4954000000000002e-05 2024-09-13T10:11:27.580 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.580 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.580 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.581 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.581 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:236 3.2 3:73c2893b:::smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.581 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.076316+0000", 2024-09-13T10:11:27.581 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.407999189000002, 2024-09-13T10:11:27.581 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041217193999999999, 2024-09-13T10:11:27.581 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.581 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.581 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.581 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.581 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.581 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.582 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 236 2024-09-13T10:11:27.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.582 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.582 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.582 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076316+0000", 2024-09-13T10:11:27.582 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.582 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.583 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076316+0000", 2024-09-13T10:11:27.583 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.583 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.584 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076319+0000", 2024-09-13T10:11:27.584 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1750000000000001e-06 2024-09-13T10:11:27.584 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.584 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.584 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.585 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076333+0000", 2024-09-13T10:11:27.585 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3862e-05 2024-09-13T10:11:27.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.585 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.585 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.585 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076359+0000", 2024-09-13T10:11:27.586 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5641999999999999e-05 2024-09-13T10:11:27.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.586 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.586 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076378+0000", 2024-09-13T10:11:27.586 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9406999999999999e-05 2024-09-13T10:11:27.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.586 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.586 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.078831+0000", 2024-09-13T10:11:27.587 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0024532590000000002 2024-09-13T10:11:27.587 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.587 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.587 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.587 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079137+0000", 2024-09-13T10:11:27.587 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00030614500000000002 2024-09-13T10:11:27.587 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.587 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.587 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.587 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.102393+0000", 2024-09-13T10:11:27.587 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023255175999999999 2024-09-13T10:11:27.587 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.588 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.588 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.588 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114728+0000", 2024-09-13T10:11:27.588 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012335376 2024-09-13T10:11:27.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.588 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.588 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.588 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114735+0000", 2024-09-13T10:11:27.588 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2969999999999998e-06 2024-09-13T10:11:27.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.589 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.589 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117465+0000", 2024-09-13T10:11:27.589 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0027298019999999999 2024-09-13T10:11:27.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.589 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.589 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117501+0000", 2024-09-13T10:11:27.589 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5837999999999998e-05 2024-09-13T10:11:27.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.590 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.590 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117533+0000", 2024-09-13T10:11:27.590 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2215000000000002e-05 2024-09-13T10:11:27.590 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.590 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.590 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.590 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.590 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:240 3.2 3:7eee2f4a:::smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.590 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.091046+0000", 2024-09-13T10:11:27.590 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.393269347999997, 2024-09-13T10:11:27.590 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048376970999999998, 2024-09-13T10:11:27.590 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.591 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.591 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.591 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.591 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.591 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.591 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 240 2024-09-13T10:11:27.591 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.591 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.591 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.591 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.591 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091046+0000", 2024-09-13T10:11:27.591 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.591 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.591 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.592 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.592 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091046+0000", 2024-09-13T10:11:27.592 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.592 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.592 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091048+0000", 2024-09-13T10:11:27.592 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6299999999999998e-06 2024-09-13T10:11:27.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.593 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.593 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091060+0000", 2024-09-13T10:11:27.593 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1229999999999999e-05 2024-09-13T10:11:27.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.593 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.593 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.593 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091075+0000", 2024-09-13T10:11:27.593 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5313000000000001e-05 2024-09-13T10:11:27.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.593 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.593 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.593 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091085+0000", 2024-09-13T10:11:27.593 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0284e-05 2024-09-13T10:11:27.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.594 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.594 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.594 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094402+0000", 2024-09-13T10:11:27.594 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00331708 2024-09-13T10:11:27.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.594 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.594 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.594 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094649+0000", 2024-09-13T10:11:27.596 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024695300000000001 2024-09-13T10:11:27.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.596 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.596 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.107510+0000", 2024-09-13T10:11:27.596 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012860778999999999 2024-09-13T10:11:27.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.596 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.596 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114739+0000", 2024-09-13T10:11:27.596 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0072289700000000004 2024-09-13T10:11:27.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.597 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.597 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.597 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118693+0000", 2024-09-13T10:11:27.597 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0039540560000000001 2024-09-13T10:11:27.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.597 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.597 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.597 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139282+0000", 2024-09-13T10:11:27.597 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020589423999999999 2024-09-13T10:11:27.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.597 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.597 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.598 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139357+0000", 2024-09-13T10:11:27.598 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.4276999999999994e-05 2024-09-13T10:11:27.598 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.598 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.598 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.598 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139423+0000", 2024-09-13T10:11:27.598 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.5975000000000004e-05 2024-09-13T10:11:27.598 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.598 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.598 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.599 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.599 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:242 3.2 3:501d22a1:::smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.599 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.095846+0000", 2024-09-13T10:11:27.599 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.388469143999998, 2024-09-13T10:11:27.599 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.043813831999999997, 2024-09-13T10:11:27.599 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.599 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.599 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.599 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.599 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.599 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.599 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 242 2024-09-13T10:11:27.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.600 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.600 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.600 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.600 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095846+0000", 2024-09-13T10:11:27.600 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.600 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.600 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.600 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.600 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095846+0000", 2024-09-13T10:11:27.600 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.600 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.600 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.600 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.601 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095848+0000", 2024-09-13T10:11:27.601 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3760000000000002e-06 2024-09-13T10:11:27.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.601 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.601 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.601 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095859+0000", 2024-09-13T10:11:27.601 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0429e-05 2024-09-13T10:11:27.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.601 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.601 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.601 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095876+0000", 2024-09-13T10:11:27.601 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7504e-05 2024-09-13T10:11:27.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.602 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.602 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.602 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095886+0000", 2024-09-13T10:11:27.602 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0182e-05 2024-09-13T10:11:27.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.602 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.602 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.602 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.096010+0000", 2024-09-13T10:11:27.602 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012336700000000001 2024-09-13T10:11:27.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.602 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.602 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.603 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.096252+0000", 2024-09-13T10:11:27.603 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024189200000000001 2024-09-13T10:11:27.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.603 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.603 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.603 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111492+0000", 2024-09-13T10:11:27.603 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015239920000000001 2024-09-13T10:11:27.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.603 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.603 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.603 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115691+0000", 2024-09-13T10:11:27.603 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0041992410000000003 2024-09-13T10:11:27.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.604 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.604 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139492+0000", 2024-09-13T10:11:27.604 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023801495999999998 2024-09-13T10:11:27.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.604 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.604 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139560+0000", 2024-09-13T10:11:27.604 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.8163000000000005e-05 2024-09-13T10:11:27.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.604 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.605 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139619+0000", 2024-09-13T10:11:27.605 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8678999999999998e-05 2024-09-13T10:11:27.605 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.605 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.605 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.605 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139660+0000", 2024-09-13T10:11:27.605 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0583000000000001e-05 2024-09-13T10:11:27.605 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.605 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.605 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.605 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.605 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.606 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:244 3.2 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.606 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.112158+0000", 2024-09-13T10:11:27.606 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.372156681, 2024-09-13T10:11:27.606 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.030058202999999999, 2024-09-13T10:11:27.606 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.606 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.606 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.606 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.606 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.606 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.606 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 244 2024-09-13T10:11:27.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.606 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.606 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.607 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.607 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112158+0000", 2024-09-13T10:11:27.607 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.607 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.607 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.607 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112158+0000", 2024-09-13T10:11:27.607 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.607 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.607 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.607 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112161+0000", 2024-09-13T10:11:27.607 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.384e-06 2024-09-13T10:11:27.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.608 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.608 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112168+0000", 2024-09-13T10:11:27.608 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.7780000000000002e-06 2024-09-13T10:11:27.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.608 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.608 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112183+0000", 2024-09-13T10:11:27.608 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5917999999999999e-05 2024-09-13T10:11:27.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.608 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.609 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112195+0000", 2024-09-13T10:11:27.609 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1578e-05 2024-09-13T10:11:27.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.609 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.609 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.609 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114766+0000", 2024-09-13T10:11:27.609 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025710659999999999 2024-09-13T10:11:27.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.609 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.609 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.609 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114995+0000", 2024-09-13T10:11:27.609 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022844 2024-09-13T10:11:27.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.609 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.610 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.610 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.125677+0000", 2024-09-13T10:11:27.610 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010682677999999999 2024-09-13T10:11:27.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.610 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.610 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.610 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.140355+0000", 2024-09-13T10:11:27.610 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014677773999999999 2024-09-13T10:11:27.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.610 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.610 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.611 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.141368+0000", 2024-09-13T10:11:27.611 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0010134040000000001 2024-09-13T10:11:27.611 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.611 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.611 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.611 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142112+0000", 2024-09-13T10:11:27.611 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00074347599999999997 2024-09-13T10:11:27.611 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.611 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.611 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.611 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142172+0000", 2024-09-13T10:11:27.612 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9749e-05 2024-09-13T10:11:27.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.612 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.612 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.612 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142217+0000", 2024-09-13T10:11:27.612 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4957999999999999e-05 2024-09-13T10:11:27.612 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.612 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.612 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.612 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.612 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:245 3.2 3:7837e849:::smithi19530714-46 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.612 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.112398+0000", 2024-09-13T10:11:27.612 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.371916624000001, 2024-09-13T10:11:27.613 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.039535731999999997, 2024-09-13T10:11:27.613 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.613 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.613 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.613 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.613 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.613 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.613 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 245 2024-09-13T10:11:27.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.613 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.613 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.613 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112398+0000", 2024-09-13T10:11:27.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.614 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.614 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.614 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112398+0000", 2024-09-13T10:11:27.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.614 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.614 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.614 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112399+0000", 2024-09-13T10:11:27.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0470000000000001e-06 2024-09-13T10:11:27.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.614 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.615 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.615 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112405+0000", 2024-09-13T10:11:27.615 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.6899999999999997e-06 2024-09-13T10:11:27.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.615 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.615 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.615 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112417+0000", 2024-09-13T10:11:27.615 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2334000000000001e-05 2024-09-13T10:11:27.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.615 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.615 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.615 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112427+0000", 2024-09-13T10:11:27.615 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.0929999999999992e-06 2024-09-13T10:11:27.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.616 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.616 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115182+0000", 2024-09-13T10:11:27.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0027552739999999998 2024-09-13T10:11:27.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.616 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.616 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115305+0000", 2024-09-13T10:11:27.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012364200000000001 2024-09-13T10:11:27.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.136435+0000", 2024-09-13T10:11:27.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021129479999999999 2024-09-13T10:11:27.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.617 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143376+0000", 2024-09-13T10:11:27.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0069408600000000001 2024-09-13T10:11:27.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.617 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.150416+0000", 2024-09-13T10:11:27.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0070403669999999996 2024-09-13T10:11:27.618 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.618 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.618 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.618 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.151872+0000", 2024-09-13T10:11:27.618 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001456286 2024-09-13T10:11:27.618 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.618 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.618 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.618 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.151906+0000", 2024-09-13T10:11:27.619 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3167999999999998e-05 2024-09-13T10:11:27.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.619 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.619 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.619 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.151934+0000", 2024-09-13T10:11:27.619 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8490999999999999e-05 2024-09-13T10:11:27.620 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.620 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.620 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.620 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.620 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:250 3.2 3:78d40eb9:::smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.621 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.115099+0000", 2024-09-13T10:11:27.621 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.369216381000001, 2024-09-13T10:11:27.621 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.028750412999999999, 2024-09-13T10:11:27.621 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.621 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.621 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.622 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.622 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.622 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.622 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 250 2024-09-13T10:11:27.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.622 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.623 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.623 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.623 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115099+0000", 2024-09-13T10:11:27.623 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.623 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.624 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.624 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115099+0000", 2024-09-13T10:11:27.624 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.625 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.625 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115101+0000", 2024-09-13T10:11:27.625 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1789999999999998e-06 2024-09-13T10:11:27.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.625 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.626 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.626 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115132+0000", 2024-09-13T10:11:27.626 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1163e-05 2024-09-13T10:11:27.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.626 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.627 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115150+0000", 2024-09-13T10:11:27.627 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7864999999999999e-05 2024-09-13T10:11:27.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.627 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.627 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.628 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115159+0000", 2024-09-13T10:11:27.628 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1719999999999996e-06 2024-09-13T10:11:27.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.628 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.628 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.628 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115440+0000", 2024-09-13T10:11:27.629 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028068199999999998 2024-09-13T10:11:27.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.629 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.629 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115579+0000", 2024-09-13T10:11:27.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013910800000000001 2024-09-13T10:11:27.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.630 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.132849+0000", 2024-09-13T10:11:27.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017270127999999999 2024-09-13T10:11:27.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.631 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.631 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142332+0000", 2024-09-13T10:11:27.631 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0094831859999999994 2024-09-13T10:11:27.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.632 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.632 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.632 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143034+0000", 2024-09-13T10:11:27.632 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00070226499999999999 2024-09-13T10:11:27.632 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.633 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.633 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143793+0000", 2024-09-13T10:11:27.633 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00075876700000000004 2024-09-13T10:11:27.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.634 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.634 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143825+0000", 2024-09-13T10:11:27.634 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2017999999999997e-05 2024-09-13T10:11:27.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.634 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.634 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.634 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143849+0000", 2024-09-13T10:11:27.635 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3879999999999998e-05 2024-09-13T10:11:27.635 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.635 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.635 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.635 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.636 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.7.22:1 3.2 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.636 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.130657+0000", 2024-09-13T10:11:27.636 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.353657986000002, 2024-09-13T10:11:27.636 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021425745, 2024-09-13T10:11:27.636 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.636 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.636 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.636 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.636 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.636 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.637 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130657+0000", 2024-09-13T10:11:27.637 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.637 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.637 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.637 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.637 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130657+0000", 2024-09-13T10:11:27.637 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.637 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.637 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.637 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.637 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130660+0000", 2024-09-13T10:11:27.637 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.095e-06 2024-09-13T10:11:27.638 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.638 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.638 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.638 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130671+0000", 2024-09-13T10:11:27.638 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0455000000000001e-05 2024-09-13T10:11:27.638 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.638 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.638 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.638 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130684+0000", 2024-09-13T10:11:27.638 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2933e-05 2024-09-13T10:11:27.638 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.638 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.639 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.639 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130695+0000", 2024-09-13T10:11:27.639 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1138000000000001e-05 2024-09-13T10:11:27.639 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.639 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.639 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.639 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139699+0000", 2024-09-13T10:11:27.639 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0090041779999999998 2024-09-13T10:11:27.639 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.639 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.639 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.639 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139816+0000", 2024-09-13T10:11:27.640 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117484 2024-09-13T10:11:27.640 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.640 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.640 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.640 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.140032+0000", 2024-09-13T10:11:27.640 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021595699999999999 2024-09-13T10:11:27.640 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.640 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.640 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.640 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143538+0000", 2024-09-13T10:11:27.640 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0035057180000000001 2024-09-13T10:11:27.640 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.640 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.641 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.641 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.150610+0000", 2024-09-13T10:11:27.641 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0070718969999999997 2024-09-13T10:11:27.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.641 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.641 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.641 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152002+0000", 2024-09-13T10:11:27.641 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001391834 2024-09-13T10:11:27.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.641 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.642 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.642 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152049+0000", 2024-09-13T10:11:27.642 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.7097e-05 2024-09-13T10:11:27.642 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.642 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.642 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.642 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152083+0000", 2024-09-13T10:11:27.642 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3958999999999998e-05 2024-09-13T10:11:27.642 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.642 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.642 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.642 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.643 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.643 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.7.22:3 3.2 3:5915c927:::smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.643 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.216433+0000", 2024-09-13T10:11:27.643 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.267882002999997, 2024-09-13T10:11:27.643 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.037069469000000001, 2024-09-13T10:11:27.643 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.643 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.643 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.643 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.643 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.643 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.643 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216433+0000", 2024-09-13T10:11:27.643 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.644 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.644 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.644 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216433+0000", 2024-09-13T10:11:27.644 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.644 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.644 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.644 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216436+0000", 2024-09-13T10:11:27.644 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7800000000000001e-06 2024-09-13T10:11:27.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.645 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.645 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.645 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216473+0000", 2024-09-13T10:11:27.645 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6931999999999998e-05 2024-09-13T10:11:27.645 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.645 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.645 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.645 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216512+0000", 2024-09-13T10:11:27.645 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.8831999999999997e-05 2024-09-13T10:11:27.645 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.645 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.646 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.646 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216539+0000", 2024-09-13T10:11:27.646 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7705999999999999e-05 2024-09-13T10:11:27.646 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.646 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.646 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.646 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216732+0000", 2024-09-13T10:11:27.647 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019224500000000001 2024-09-13T10:11:27.647 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.647 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.647 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.647 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217000+0000", 2024-09-13T10:11:27.647 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026813100000000001 2024-09-13T10:11:27.647 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.647 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.647 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.647 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217638+0000", 2024-09-13T10:11:27.647 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00063874400000000001 2024-09-13T10:11:27.647 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.648 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.648 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.648 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.219940+0000", 2024-09-13T10:11:27.648 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023015890000000002 2024-09-13T10:11:27.648 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.648 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.648 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.648 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.220304+0000", 2024-09-13T10:11:27.648 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036399099999999998 2024-09-13T10:11:27.648 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.649 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.649 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.649 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253232+0000", 2024-09-13T10:11:27.649 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032927640000000001 2024-09-13T10:11:27.649 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.649 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.649 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.649 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253393+0000", 2024-09-13T10:11:27.649 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000161142 2024-09-13T10:11:27.649 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.649 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.649 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.650 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253503+0000", 2024-09-13T10:11:27.650 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000109737 2024-09-13T10:11:27.650 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.650 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.650 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.650 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.650 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.650 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.2.23:2 3.2 3:46ab5247:::smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:27.650 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.217995+0000", 2024-09-13T10:11:27.650 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.266320532000002, 2024-09-13T10:11:27.650 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036081690999999999, 2024-09-13T10:11:27.650 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.650 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.650 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:27.651 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.651 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.651 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.651 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217995+0000", 2024-09-13T10:11:27.651 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.651 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.651 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.651 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.651 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217995+0000", 2024-09-13T10:11:27.651 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.651 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.651 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.651 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.652 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217997+0000", 2024-09-13T10:11:27.652 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3429999999999998e-06 2024-09-13T10:11:27.652 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.652 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.652 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.652 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218007+0000", 2024-09-13T10:11:27.652 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.91e-06 2024-09-13T10:11:27.652 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.652 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.652 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.652 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218024+0000", 2024-09-13T10:11:27.652 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7549000000000001e-05 2024-09-13T10:11:27.652 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.653 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.653 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.653 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218038+0000", 2024-09-13T10:11:27.653 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3509999999999999e-05 2024-09-13T10:11:27.653 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.653 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.653 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.653 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218384+0000", 2024-09-13T10:11:27.653 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00034618499999999999 2024-09-13T10:11:27.653 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.653 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.653 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.653 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218605+0000", 2024-09-13T10:11:27.654 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022059400000000001 2024-09-13T10:11:27.654 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.654 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.654 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:27.654 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.219124+0000", 2024-09-13T10:11:27.654 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00051934699999999997 2024-09-13T10:11:27.654 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.654 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.654 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.654 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.222285+0000", 2024-09-13T10:11:27.654 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0031605790000000002 2024-09-13T10:11:27.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.655 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.655 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:27.655 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.224618+0000", 2024-09-13T10:11:27.655 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023338209999999998 2024-09-13T10:11:27.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.655 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.655 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:27.655 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253766+0000", 2024-09-13T10:11:27.655 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029147890999999999 2024-09-13T10:11:27.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.655 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.656 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:27.656 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253920+0000", 2024-09-13T10:11:27.656 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015327299999999999 2024-09-13T10:11:27.656 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.656 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.656 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.656 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254076+0000", 2024-09-13T10:11:27.656 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000156689 2024-09-13T10:11:27.656 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.656 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.656 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.657 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.657 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.657 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1854 4.b 4:d8e9927b:::smithi19530714-44:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:27.657 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:10.749110+0000", 2024-09-13T10:11:27.657 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 16.735205442000002, 2024-09-13T10:11:27.657 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.049047407000000001, 2024-09-13T10:11:27.657 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:27.657 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:27.657 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:27.658 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:27.658 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:27.658 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:27.658 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1854 2024-09-13T10:11:27.658 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.658 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:27.658 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.658 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:27.659 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749110+0000", 2024-09-13T10:11:27.659 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.659 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.659 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.659 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:27.659 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749110+0000", 2024-09-13T10:11:27.659 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:27.659 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.659 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.659 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:27.659 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749113+0000", 2024-09-13T10:11:27.660 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0970000000000002e-06 2024-09-13T10:11:27.660 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.660 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.660 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:27.660 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749125+0000", 2024-09-13T10:11:27.660 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2048999999999999e-05 2024-09-13T10:11:27.660 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.660 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.661 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:27.661 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749164+0000", 2024-09-13T10:11:27.661 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9579999999999997e-05 2024-09-13T10:11:27.661 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.661 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.661 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:27.661 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749188+0000", 2024-09-13T10:11:27.662 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4026e-05 2024-09-13T10:11:27.662 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.662 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.662 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:27.662 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749320+0000", 2024-09-13T10:11:27.662 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013178500000000001 2024-09-13T10:11:27.662 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.662 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.662 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:27.662 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749506+0000", 2024-09-13T10:11:27.662 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00018623700000000001 2024-09-13T10:11:27.662 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:27.663 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:27.663 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:27.663 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.798157+0000", 2024-09-13T10:11:27.663 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048650632999999999 2024-09-13T10:11:27.663 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.663 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.663 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.663 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:27.663 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:27.663 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:27.666 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T10:11:27.667 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 937764 2024-09-13T10:11:27.667 INFO:tasks.rados.rados.0.smithi195.stdout:1223: finishing write tid 4 to smithi19530714-5 2024-09-13T10:11:27.667 INFO:tasks.rados.rados.0.smithi195.stdout:1223: finishing write tid 5 to smithi19530714-5 2024-09-13T10:11:27.667 INFO:tasks.rados.rados.0.smithi195.stdout:1223: oid 5 updating version 0 to 15 2024-09-13T10:11:27.667 INFO:tasks.rados.rados.0.smithi195.stdout:1223: oid 5 updating version 15 to 16 2024-09-13T10:11:27.668 INFO:tasks.rados.rados.0.smithi195.stdout:1223: oid 5 version 16 is already newer than 14 2024-09-13T10:11:27.668 INFO:tasks.rados.rados.0.smithi195.stdout:1223: oid 5 version 16 is already newer than 13 2024-09-13T10:11:27.668 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 16 (ObjNum 231 snap 0 seq_num 231) dirty exists 2024-09-13T10:11:27.668 INFO:tasks.rados.rados.0.smithi195.stdout:1223: left oid 5 (ObjNum 231 snap 0 seq_num 231) 2024-09-13T10:11:27.668 INFO:tasks.rados.rados.0.smithi195.stdout:1221: done (5 left) 2024-09-13T10:11:27.668 INFO:tasks.rados.rados.0.smithi195.stdout:1222: done (4 left) 2024-09-13T10:11:27.668 INFO:tasks.rados.rados.0.smithi195.stdout:1223: done (3 left) 2024-09-13T10:11:27.668 INFO:tasks.rados.rados.0.smithi195.stdout:1224: done (2 left) 2024-09-13T10:11:27.669 INFO:tasks.rados.rados.0.smithi195.stdout:1225: done (1 left) 2024-09-13T10:11:27.669 INFO:tasks.rados.rados.0.smithi195.stdout:1226: done (0 left) 2024-09-13T10:11:27.669 INFO:tasks.rados.rados.0.smithi195.stdout:1227: set_chunk oid 8 target oid 2024-09-13T10:11:27.669 INFO:tasks.rados.rados.0.smithi195.stdout:1227: set_chunk oid 8 offset: 3142656 length: 644096 target oid 2720ade6989541e1874ce4b3cc9c39c8f5ba38069b5c81f7251aa8119f6c930e offset: 0 snap -1 2024-09-13T10:11:27.669 INFO:tasks.rados.rados.0.smithi195.stdout:1228: set_chunk oid 29 target oid 2024-09-13T10:11:27.669 INFO:tasks.rados.rados.0.smithi195.stdout:1228: set_chunk oid 29 offset: 1310208 length: 504320 target oid 7373d55317b84a4bb1648a7f6730f3fb68f0cf856036a47c79b9a3ba10bb2e8e offset: 0 snap -1 2024-09-13T10:11:27.669 INFO:tasks.rados.rados.0.smithi195.stdout:1229: tier_evict oid 41 2024-09-13T10:11:27.669 INFO:tasks.rados.rados.0.smithi195.stdout:1229: tier_evict oid 41 snap -1 2024-09-13T10:11:27.669 INFO:tasks.rados.rados.0.smithi195.stdout:1227: finishing set_chunk to oid 8 2024-09-13T10:11:27.669 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 18 (ObjNum 226 snap 0 seq_num 226) dirty exists 2024-09-13T10:11:27.669 INFO:tasks.rados.rados.0.smithi195.stdout:1227: done (2 left) 2024-09-13T10:11:27.669 INFO:tasks.rados.rados.0.smithi195.stdout:1230: read oid 32 snap -1 2024-09-13T10:11:27.670 INFO:tasks.rados.rados.0.smithi195.stdout:1230: expect (ObjNum 225 snap 0 seq_num 225) 2024-09-13T10:11:27.670 INFO:tasks.rados.rados.0.smithi195.stdout:1231: set_chunk oid 31 target oid 2024-09-13T10:11:27.670 INFO:tasks.rados.rados.0.smithi195.stdout:1231: set_chunk oid 31 offset: 336384 length: 630784 target oid b379c49cd75851fb64c7dee69a4568784353b9d6d10357c507a3867749103bd3 offset: 0 snap -1 2024-09-13T10:11:27.670 INFO:tasks.rados.rados.0.smithi195.stdout:1232: read oid 25 snap -1 2024-09-13T10:11:27.670 INFO:tasks.rados.rados.0.smithi195.stdout:1232: expect (ObjNum 215 snap 0 seq_num 215) 2024-09-13T10:11:27.670 INFO:tasks.rados.rados.0.smithi195.stdout:1233: tier_promote oid 37 2024-09-13T10:11:27.670 INFO:tasks.rados.rados.0.smithi195.stdout:1229: got (22) Invalid argument 2024-09-13T10:11:27.670 INFO:tasks.rados.rados.0.smithi195.stdout:1228: finishing set_chunk to oid 29 2024-09-13T10:11:27.670 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 71 (ObjNum 220 snap 0 seq_num 220) dirty exists 2024-09-13T10:11:27.670 INFO:tasks.rados.rados.0.smithi195.stdout:1230: expect (ObjNum 225 snap 0 seq_num 225) 2024-09-13T10:11:27.671 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:27.660+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:27.694 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-13T10:11:27.726 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1603477 2024-09-13T10:11:27.765 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:27.763+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:27.790 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:27.791 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:27.791 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:11:27.791 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:27.802 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3396364 2024-09-13T10:11:27.804 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T10:11:27.809 INFO:tasks.rados.rados.0.smithi195.stdout:1231: finishing set_chunk to oid 31 2024-09-13T10:11:27.810 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 64 (ObjNum 227 snap 0 seq_num 227) dirty exists 2024-09-13T10:11:27.810 INFO:tasks.rados.rados.0.smithi195.stdout:1232: expect (ObjNum 215 snap 0 seq_num 215) 2024-09-13T10:11:27.866 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:27.867 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:27.862+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:27.918 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:27.919 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:27.919 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:11:27.919 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:11:27.919 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:27.932 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T10:11:27.954 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1910161 2024-09-13T10:11:28.021 INFO:tasks.rados.rados.0.smithi195.stdout:1233: got (0) Success 2024-09-13T10:11:28.021 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 18 (ObjNum 145 snap 0 seq_num 145) dirty exists 2024-09-13T10:11:28.021 INFO:tasks.rados.rados.0.smithi195.stdout:1228: done (5 left) 2024-09-13T10:11:28.021 INFO:tasks.rados.rados.0.smithi195.stdout:1229: done (4 left) 2024-09-13T10:11:28.021 INFO:tasks.rados.rados.0.smithi195.stdout:1230: done (3 left) 2024-09-13T10:11:28.021 INFO:tasks.rados.rados.0.smithi195.stdout:1231: done (2 left) 2024-09-13T10:11:28.021 INFO:tasks.rados.rados.0.smithi195.stdout:1232: done (1 left) 2024-09-13T10:11:28.021 INFO:tasks.rados.rados.0.smithi195.stdout:1233: done (0 left) 2024-09-13T10:11:28.021 INFO:tasks.rados.rados.0.smithi195.stdout:1234: read oid 14 snap -1 2024-09-13T10:11:28.022 INFO:tasks.rados.rados.0.smithi195.stdout:1234: expect (ObjNum 209 snap 0 seq_num 209) 2024-09-13T10:11:28.022 INFO:tasks.rados.rados.0.smithi195.stdout:1235: write oid 20 current snap is 0 2024-09-13T10:11:28.022 INFO:tasks.rados.rados.0.smithi195.stdout:1235: seq_num 232 ranges {699224=548358,2026293=462074,3271546=432754} 2024-09-13T10:11:28.022 INFO:tasks.rados.rados.0.smithi195.stdout:1235: writing smithi19530714-20 from 699224 to 1247582 tid 1 2024-09-13T10:11:28.022 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:28.019+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:28.031 INFO:tasks.rados.rados.0.smithi195.stdout:1235: writing smithi19530714-20 from 2026293 to 2488367 tid 2 2024-09-13T10:11:28.048 INFO:tasks.rados.rados.0.smithi195.stdout:1235: writing smithi19530714-20 from 3271546 to 3704300 tid 3 2024-09-13T10:11:28.048 INFO:tasks.rados.rados.0.smithi195.stdout:1236: read oid 15 snap -1 2024-09-13T10:11:28.049 INFO:tasks.rados.rados.0.smithi195.stdout:1236: expect (ObjNum 204 snap 0 seq_num 204) 2024-09-13T10:11:28.049 INFO:tasks.rados.rados.0.smithi195.stdout:1237: read oid 11 snap -1 2024-09-13T10:11:28.049 INFO:tasks.rados.rados.0.smithi195.stdout:1237: expect (ObjNum 217 snap 0 seq_num 217) 2024-09-13T10:11:28.049 INFO:tasks.rados.rados.0.smithi195.stdout:1238: tier_promote oid 26 2024-09-13T10:11:28.049 INFO:tasks.rados.rados.0.smithi195.stdout:1234: expect (ObjNum 209 snap 0 seq_num 209) 2024-09-13T10:11:28.063 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1267302 2024-09-13T10:11:28.122 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:28.119+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:28.140 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:28.140 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:11:28.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:11:28.141 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:11:28.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.141 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:207 3.0 3:332cd44b:::smithi19530714-8 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.141 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:11:28.141 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.145960455999997, 2024-09-13T10:11:28.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074034500000000003, 2024-09-13T10:11:28.141 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.141 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.141 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.141 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.141 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.141 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.142 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 207 2024-09-13T10:11:28.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.142 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:11:28.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:11:28.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991271+0000", 2024-09-13T10:11:28.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9289999999999998e-06 2024-09-13T10:11:28.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991284+0000", 2024-09-13T10:11:28.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2597e-05 2024-09-13T10:11:28.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.144 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991313+0000", 2024-09-13T10:11:28.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8982000000000001e-05 2024-09-13T10:11:28.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991334+0000", 2024-09-13T10:11:28.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1046000000000001e-05 2024-09-13T10:11:28.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991416+0000", 2024-09-13T10:11:28.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.153e-05 2024-09-13T10:11:28.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991504+0000", 2024-09-13T10:11:28.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8220999999999994e-05 2024-09-13T10:11:28.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:28.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017232+0000", 2024-09-13T10:11:28.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025728328000000002 2024-09-13T10:11:28.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.031588+0000", 2024-09-13T10:11:28.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014355557 2024-09-13T10:11:28.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034211+0000", 2024-09-13T10:11:28.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026232790000000001 2024-09-13T10:11:28.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065168+0000", 2024-09-13T10:11:28.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030956833999999999 2024-09-13T10:11:28.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065260+0000", 2024-09-13T10:11:28.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2044000000000002e-05 2024-09-13T10:11:28.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065303+0000", 2024-09-13T10:11:28.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3152999999999998e-05 2024-09-13T10:11:28.148 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.148 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.148 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.148 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:211 3.0 3:0596651d:::smithi19530714-12 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.148 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:11:28.149 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.145237078000001, 2024-09-13T10:11:28.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.042923122000000001, 2024-09-13T10:11:28.149 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.149 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.149 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.149 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.149 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.149 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.149 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 211 2024-09-13T10:11:28.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.149 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:11:28.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:11:28.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991995+0000", 2024-09-13T10:11:28.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7640000000000001e-06 2024-09-13T10:11:28.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992007+0000", 2024-09-13T10:11:28.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2058999999999999e-05 2024-09-13T10:11:28.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992040+0000", 2024-09-13T10:11:28.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3466000000000001e-05 2024-09-13T10:11:28.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.151 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.151 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992059+0000", 2024-09-13T10:11:28.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8485999999999999e-05 2024-09-13T10:11:28.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992136+0000", 2024-09-13T10:11:28.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6994000000000002e-05 2024-09-13T10:11:28.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992207+0000", 2024-09-13T10:11:28.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1357000000000001e-05 2024-09-13T10:11:28.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:28.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.006377+0000", 2024-09-13T10:11:28.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014170455 2024-09-13T10:11:28.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.022844+0000", 2024-09-13T10:11:28.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016466275999999998 2024-09-13T10:11:28.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023031+0000", 2024-09-13T10:11:28.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018686000000000001 2024-09-13T10:11:28.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034788+0000", 2024-09-13T10:11:28.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011756967 2024-09-13T10:11:28.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034857+0000", 2024-09-13T10:11:28.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9169000000000001e-05 2024-09-13T10:11:28.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034915+0000", 2024-09-13T10:11:28.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8269000000000001e-05 2024-09-13T10:11:28.155 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.155 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.155 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.155 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:212 3.0 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.156 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:11:28.156 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.145130745000003, 2024-09-13T10:11:28.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.073011809999999996, 2024-09-13T10:11:28.156 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.156 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.156 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.156 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.156 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.156 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.156 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 212 2024-09-13T10:11:28.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.156 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:11:28.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:11:28.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992099+0000", 2024-09-13T10:11:28.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4899999999999995e-07 2024-09-13T10:11:28.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992103+0000", 2024-09-13T10:11:28.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.33e-06 2024-09-13T10:11:28.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992110+0000", 2024-09-13T10:11:28.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8399999999999997e-06 2024-09-13T10:11:28.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992116+0000", 2024-09-13T10:11:28.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6790000000000002e-06 2024-09-13T10:11:28.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992292+0000", 2024-09-13T10:11:28.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017659199999999999 2024-09-13T10:11:28.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992385+0000", 2024-09-13T10:11:28.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2608000000000005e-05 2024-09-13T10:11:28.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:28.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011428+0000", 2024-09-13T10:11:28.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.019042631000000001 2024-09-13T10:11:28.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.022852+0000", 2024-09-13T10:11:28.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011424528999999999 2024-09-13T10:11:28.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.031660+0000", 2024-09-13T10:11:28.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088079899999999999 2024-09-13T10:11:28.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065032+0000", 2024-09-13T10:11:28.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033372247000000001 2024-09-13T10:11:28.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065073+0000", 2024-09-13T10:11:28.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0598e-05 2024-09-13T10:11:28.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.163 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065110+0000", 2024-09-13T10:11:28.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7116999999999997e-05 2024-09-13T10:11:28.163 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.163 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.163 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.163 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:202 3.2 e38/30 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:28.163 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:11:28.163 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.135640139000003, 2024-09-13T10:11:28.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017280298999999999, 2024-09-13T10:11:28.163 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.163 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.164 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.164 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:11:28.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:11:28.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001594+0000", 2024-09-13T10:11:28.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5550000000000002e-06 2024-09-13T10:11:28.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009486+0000", 2024-09-13T10:11:28.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078916799999999999 2024-09-13T10:11:28.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009519+0000", 2024-09-13T10:11:28.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2898999999999998e-05 2024-09-13T10:11:28.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009540+0000", 2024-09-13T10:11:28.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1559000000000001e-05 2024-09-13T10:11:28.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009655+0000", 2024-09-13T10:11:28.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114271 2024-09-13T10:11:28.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009683+0000", 2024-09-13T10:11:28.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8379000000000001e-05 2024-09-13T10:11:28.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.018766+0000", 2024-09-13T10:11:28.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0090829589999999998 2024-09-13T10:11:28.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.018869+0000", 2024-09-13T10:11:28.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102997 2024-09-13T10:11:28.168 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.168 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.168 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.168 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:216 3.0 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.169 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:11:28.169 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.132627603000003, 2024-09-13T10:11:28.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.063936142000000001, 2024-09-13T10:11:28.169 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.169 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.169 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.169 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.169 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.169 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.169 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 216 2024-09-13T10:11:28.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.169 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:11:28.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:11:28.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004603+0000", 2024-09-13T10:11:28.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5540000000000001e-06 2024-09-13T10:11:28.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004615+0000", 2024-09-13T10:11:28.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2597e-05 2024-09-13T10:11:28.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004637+0000", 2024-09-13T10:11:28.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1713e-05 2024-09-13T10:11:28.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004651+0000", 2024-09-13T10:11:28.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3878e-05 2024-09-13T10:11:28.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004985+0000", 2024-09-13T10:11:28.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033347299999999998 2024-09-13T10:11:28.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.005706+0000", 2024-09-13T10:11:28.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00072122199999999999 2024-09-13T10:11:28.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:28.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023801+0000", 2024-09-13T10:11:28.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018095466000000001 2024-09-13T10:11:28.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.032153+0000", 2024-09-13T10:11:28.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083516010000000002 2024-09-13T10:11:28.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.035250+0000", 2024-09-13T10:11:28.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030974409999999998 2024-09-13T10:11:28.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068439+0000", 2024-09-13T10:11:28.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033189042000000002 2024-09-13T10:11:28.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068499+0000", 2024-09-13T10:11:28.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9969000000000001e-05 2024-09-13T10:11:28.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068537+0000", 2024-09-13T10:11:28.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8186000000000003e-05 2024-09-13T10:11:28.176 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.176 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.176 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.176 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:221 3.0 3:18ae852b:::smithi19530714-22 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.176 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:11:28.176 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.098703096000001, 2024-09-13T10:11:28.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039809298, 2024-09-13T10:11:28.176 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.177 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.177 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.177 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.177 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.177 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.177 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 221 2024-09-13T10:11:28.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.177 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:11:28.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:11:28.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038528+0000", 2024-09-13T10:11:28.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8819999999999999e-06 2024-09-13T10:11:28.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038534+0000", 2024-09-13T10:11:28.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0730000000000001e-06 2024-09-13T10:11:28.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038546+0000", 2024-09-13T10:11:28.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.219e-05 2024-09-13T10:11:28.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038566+0000", 2024-09-13T10:11:28.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9867000000000001e-05 2024-09-13T10:11:28.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038608+0000", 2024-09-13T10:11:28.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1977999999999997e-05 2024-09-13T10:11:28.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038707+0000", 2024-09-13T10:11:28.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8708000000000004e-05 2024-09-13T10:11:28.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:28.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.057374+0000", 2024-09-13T10:11:28.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018667652999999999 2024-09-13T10:11:28.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061983+0000", 2024-09-13T10:11:28.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0046087089999999999 2024-09-13T10:11:28.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.063027+0000", 2024-09-13T10:11:28.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001044239 2024-09-13T10:11:28.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078233+0000", 2024-09-13T10:11:28.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015205639999999999 2024-09-13T10:11:28.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.183 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078298+0000", 2024-09-13T10:11:28.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5180999999999998e-05 2024-09-13T10:11:28.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078335+0000", 2024-09-13T10:11:28.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7178000000000002e-05 2024-09-13T10:11:28.184 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.184 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.184 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.185 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:225 3.0 3:26960251:::smithi19530714-26 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.185 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:11:28.185 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.078776822000002, 2024-09-13T10:11:28.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030704875999999999, 2024-09-13T10:11:28.185 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.185 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.185 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.185 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.186 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.186 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.186 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 225 2024-09-13T10:11:28.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.186 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:11:28.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:11:28.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058455+0000", 2024-09-13T10:11:28.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3220000000000001e-06 2024-09-13T10:11:28.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058467+0000", 2024-09-13T10:11:28.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1987999999999999e-05 2024-09-13T10:11:28.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058502+0000", 2024-09-13T10:11:28.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4873000000000003e-05 2024-09-13T10:11:28.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058520+0000", 2024-09-13T10:11:28.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7977000000000001e-05 2024-09-13T10:11:28.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060066+0000", 2024-09-13T10:11:28.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001545261 2024-09-13T10:11:28.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060680+0000", 2024-09-13T10:11:28.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00061431799999999998 2024-09-13T10:11:28.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:28.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070650+0000", 2024-09-13T10:11:28.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0099703099999999996 2024-09-13T10:11:28.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078192+0000", 2024-09-13T10:11:28.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0075419709999999997 2024-09-13T10:11:28.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084486+0000", 2024-09-13T10:11:28.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0062943779999999998 2024-09-13T10:11:28.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089029+0000", 2024-09-13T10:11:28.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045422379999999997 2024-09-13T10:11:28.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.194 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089092+0000", 2024-09-13T10:11:28.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3435999999999994e-05 2024-09-13T10:11:28.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.194 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089157+0000", 2024-09-13T10:11:28.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4803999999999998e-05 2024-09-13T10:11:28.195 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.195 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.195 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.195 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:228 3.0 3:0168a995:::smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.196 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:11:28.196 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.076975863999998, 2024-09-13T10:11:28.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030139446, 2024-09-13T10:11:28.196 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.196 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.196 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.196 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.197 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.197 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.197 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 228 2024-09-13T10:11:28.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.197 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:11:28.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:11:28.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060255+0000", 2024-09-13T10:11:28.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.787e-06 2024-09-13T10:11:28.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060262+0000", 2024-09-13T10:11:28.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8630000000000002e-06 2024-09-13T10:11:28.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060285+0000", 2024-09-13T10:11:28.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2935e-05 2024-09-13T10:11:28.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060294+0000", 2024-09-13T10:11:28.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3940000000000002e-06 2024-09-13T10:11:28.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061041+0000", 2024-09-13T10:11:28.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00074706700000000002 2024-09-13T10:11:28.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061148+0000", 2024-09-13T10:11:28.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107364 2024-09-13T10:11:28.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:28.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074199+0000", 2024-09-13T10:11:28.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013050447999999999 2024-09-13T10:11:28.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.203 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084506+0000", 2024-09-13T10:11:28.203 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010306789 2024-09-13T10:11:28.203 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.203 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.203 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089222+0000", 2024-09-13T10:11:28.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047163480000000004 2024-09-13T10:11:28.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090310+0000", 2024-09-13T10:11:28.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010880919999999999 2024-09-13T10:11:28.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.205 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.205 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090356+0000", 2024-09-13T10:11:28.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5522000000000002e-05 2024-09-13T10:11:28.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090393+0000", 2024-09-13T10:11:28.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6837e-05 2024-09-13T10:11:28.206 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.206 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.206 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.207 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:229 3.0 3:1e6cc58c:::smithi19530714-30 low_tier:head [copy-from ver 18] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.207 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:11:28.207 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.075669488999999, 2024-09-13T10:11:28.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033410479, 2024-09-13T10:11:28.207 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.207 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.207 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.207 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.208 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.208 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.208 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 229 2024-09-13T10:11:28.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.208 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:11:28.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:11:28.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061562+0000", 2024-09-13T10:11:28.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9730000000000002e-06 2024-09-13T10:11:28.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061577+0000", 2024-09-13T10:11:28.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5044e-05 2024-09-13T10:11:28.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061602+0000", 2024-09-13T10:11:28.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4494999999999998e-05 2024-09-13T10:11:28.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061615+0000", 2024-09-13T10:11:28.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3207000000000001e-05 2024-09-13T10:11:28.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061701+0000", 2024-09-13T10:11:28.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5694999999999999e-05 2024-09-13T10:11:28.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061871+0000", 2024-09-13T10:11:28.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017004100000000001 2024-09-13T10:11:28.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:28.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.079511+0000", 2024-09-13T10:11:28.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017639708 2024-09-13T10:11:28.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.212 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.212 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089295+0000", 2024-09-13T10:11:28.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0097846359999999993 2024-09-13T10:11:28.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089316+0000", 2024-09-13T10:11:28.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0288999999999998e-05 2024-09-13T10:11:28.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094875+0000", 2024-09-13T10:11:28.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0055598389999999996 2024-09-13T10:11:28.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094931+0000", 2024-09-13T10:11:28.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5556999999999997e-05 2024-09-13T10:11:28.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094970+0000", 2024-09-13T10:11:28.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8995000000000002e-05 2024-09-13T10:11:28.215 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.215 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.215 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.215 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:233 3.0 3:24ead787:::smithi19530714-34 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.215 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:11:28.216 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.071492984999999, 2024-09-13T10:11:28.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047200234000000001, 2024-09-13T10:11:28.216 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.216 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.216 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.216 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.216 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.216 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.216 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 233 2024-09-13T10:11:28.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.216 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:11:28.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:11:28.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065737+0000", 2024-09-13T10:11:28.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.283e-06 2024-09-13T10:11:28.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065745+0000", 2024-09-13T10:11:28.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3130000000000001e-06 2024-09-13T10:11:28.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065757+0000", 2024-09-13T10:11:28.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2627e-05 2024-09-13T10:11:28.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065765+0000", 2024-09-13T10:11:28.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4669999999999998e-06 2024-09-13T10:11:28.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065816+0000", 2024-09-13T10:11:28.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1545000000000002e-05 2024-09-13T10:11:28.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.219 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.219 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065940+0000", 2024-09-13T10:11:28.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012366699999999999 2024-09-13T10:11:28.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.219 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:28.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.086064+0000", 2024-09-13T10:11:28.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.020124487 2024-09-13T10:11:28.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090747+0000", 2024-09-13T10:11:28.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004682795 2024-09-13T10:11:28.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.091235+0000", 2024-09-13T10:11:28.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048779900000000002 2024-09-13T10:11:28.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112809+0000", 2024-09-13T10:11:28.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.021573967999999999 2024-09-13T10:11:28.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112878+0000", 2024-09-13T10:11:28.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9413999999999996e-05 2024-09-13T10:11:28.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112936+0000", 2024-09-13T10:11:28.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7868999999999998e-05 2024-09-13T10:11:28.222 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.222 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.222 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.222 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:227 3.2 e38/30 3:6ac7800c:::smithi19530714-28 low_tier:head v 38'8, mlcod=38'6)", 2024-09-13T10:11:28.222 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:11:28.222 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.063129922000002, 2024-09-13T10:11:28.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014433425999999999, 2024-09-13T10:11:28.222 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.222 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.223 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.223 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.223 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:11:28.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.223 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:11:28.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074101+0000", 2024-09-13T10:11:28.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.511e-06 2024-09-13T10:11:28.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083754+0000", 2024-09-13T10:11:28.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096533799999999996 2024-09-13T10:11:28.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083772+0000", 2024-09-13T10:11:28.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8241e-05 2024-09-13T10:11:28.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.225 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083782+0000", 2024-09-13T10:11:28.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.004e-05 2024-09-13T10:11:28.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083875+0000", 2024-09-13T10:11:28.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.268e-05 2024-09-13T10:11:28.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083918+0000", 2024-09-13T10:11:28.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-13T10:11:28.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.088456+0000", 2024-09-13T10:11:28.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045385540000000002 2024-09-13T10:11:28.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.228 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.228 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.088532+0000", 2024-09-13T10:11:28.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6069000000000006e-05 2024-09-13T10:11:28.228 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.228 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.228 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.228 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:239 3.0 3:078df4f4:::smithi19530714-40 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.229 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:11:28.229 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.047926410000002, 2024-09-13T10:11:28.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025192791999999999, 2024-09-13T10:11:28.229 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.229 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.229 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.229 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.229 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.229 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.229 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 239 2024-09-13T10:11:28.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.230 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:11:28.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:11:28.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089305+0000", 2024-09-13T10:11:28.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1710000000000001e-06 2024-09-13T10:11:28.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089312+0000", 2024-09-13T10:11:28.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6860000000000006e-06 2024-09-13T10:11:28.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.231 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.231 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089323+0000", 2024-09-13T10:11:28.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0467e-05 2024-09-13T10:11:28.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.232 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.232 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089330+0000", 2024-09-13T10:11:28.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3939999999999998e-06 2024-09-13T10:11:28.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.232 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.232 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089379+0000", 2024-09-13T10:11:28.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8990999999999999e-05 2024-09-13T10:11:28.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.232 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.232 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089543+0000", 2024-09-13T10:11:28.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016351700000000001 2024-09-13T10:11:28.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:28.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.098886+0000", 2024-09-13T10:11:28.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0093434980000000004 2024-09-13T10:11:28.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.105855+0000", 2024-09-13T10:11:28.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.006968279 2024-09-13T10:11:28.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.105868+0000", 2024-09-13T10:11:28.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3560999999999999e-05 2024-09-13T10:11:28.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114382+0000", 2024-09-13T10:11:28.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0085134730000000006 2024-09-13T10:11:28.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114458+0000", 2024-09-13T10:11:28.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6143999999999995e-05 2024-09-13T10:11:28.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114495+0000", 2024-09-13T10:11:28.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7611000000000003e-05 2024-09-13T10:11:28.235 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.235 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.235 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.235 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:241 3.0 3:388ca573:::smithi19530714-42 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.235 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:11:28.236 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.042052687999998, 2024-09-13T10:11:28.236 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057595649999999998, 2024-09-13T10:11:28.236 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.236 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.236 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.236 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.236 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.236 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.236 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 241 2024-09-13T10:11:28.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.236 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.236 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:11:28.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:11:28.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095178+0000", 2024-09-13T10:11:28.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.345e-06 2024-09-13T10:11:28.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095184+0000", 2024-09-13T10:11:28.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4239999999999998e-06 2024-09-13T10:11:28.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.239 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.239 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.239 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095198+0000", 2024-09-13T10:11:28.239 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4037e-05 2024-09-13T10:11:28.239 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.240 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.240 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.240 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095206+0000", 2024-09-13T10:11:28.240 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1629999999999996e-06 2024-09-13T10:11:28.240 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.241 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.241 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095274+0000", 2024-09-13T10:11:28.241 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7948999999999996e-05 2024-09-13T10:11:28.241 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.241 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095485+0000", 2024-09-13T10:11:28.242 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021050100000000001 2024-09-13T10:11:28.242 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.242 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.242 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:28.242 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.134651+0000", 2024-09-13T10:11:28.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039166200999999998 2024-09-13T10:11:28.243 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.243 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.243 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.243 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.140536+0000", 2024-09-13T10:11:28.243 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058849719999999996 2024-09-13T10:11:28.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.244 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.244 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.244 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.140980+0000", 2024-09-13T10:11:28.244 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044394200000000001 2024-09-13T10:11:28.244 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.245 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.245 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.245 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152700+0000", 2024-09-13T10:11:28.245 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011720431999999999 2024-09-13T10:11:28.245 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.245 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.245 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.245 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152739+0000", 2024-09-13T10:11:28.246 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8448999999999997e-05 2024-09-13T10:11:28.246 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.246 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.246 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.246 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152772+0000", 2024-09-13T10:11:28.246 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3235000000000003e-05 2024-09-13T10:11:28.246 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.246 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.246 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.246 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.246 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.246 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:247 3.0 3:0b30fd85:::smithi19530714-48 low_tier:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.246 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:11:28.247 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.024153036999998, 2024-09-13T10:11:28.247 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032448926000000003, 2024-09-13T10:11:28.247 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.247 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.247 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.247 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.247 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.247 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.247 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 247 2024-09-13T10:11:28.247 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.247 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.247 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.247 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.247 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:11:28.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.248 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.248 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.248 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.248 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:11:28.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.248 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.248 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.248 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.248 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113078+0000", 2024-09-13T10:11:28.248 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8670000000000001e-06 2024-09-13T10:11:28.248 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.249 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113086+0000", 2024-09-13T10:11:28.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2260000000000005e-06 2024-09-13T10:11:28.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.249 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113104+0000", 2024-09-13T10:11:28.249 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7887e-05 2024-09-13T10:11:28.249 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.249 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.249 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.249 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113115+0000", 2024-09-13T10:11:28.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0763e-05 2024-09-13T10:11:28.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.250 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.250 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113163+0000", 2024-09-13T10:11:28.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8541999999999997e-05 2024-09-13T10:11:28.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.250 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.250 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.250 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113322+0000", 2024-09-13T10:11:28.250 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000159121 2024-09-13T10:11:28.250 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.250 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.251 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:28.251 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.125165+0000", 2024-09-13T10:11:28.251 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011842320999999999 2024-09-13T10:11:28.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.251 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.251 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.251 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.131189+0000", 2024-09-13T10:11:28.251 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.006024728 2024-09-13T10:11:28.251 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.251 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.251 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.251 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.131203+0000", 2024-09-13T10:11:28.251 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3551e-05 2024-09-13T10:11:28.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.252 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145435+0000", 2024-09-13T10:11:28.252 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014232273 2024-09-13T10:11:28.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.252 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145481+0000", 2024-09-13T10:11:28.252 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5918000000000003e-05 2024-09-13T10:11:28.252 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.252 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.252 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.253 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145525+0000", 2024-09-13T10:11:28.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3729000000000001e-05 2024-09-13T10:11:28.253 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.253 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.253 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.253 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.253 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.253 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:244 3.2 e38/30 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'14, mlcod=38'10)", 2024-09-13T10:11:28.253 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:11:28.253 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.010687234000002, 2024-09-13T10:11:28.253 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014967827, 2024-09-13T10:11:28.253 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.253 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.254 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.254 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.254 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:11:28.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.254 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.254 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:11:28.254 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.254 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.254 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.255 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126544+0000", 2024-09-13T10:11:28.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.03e-06 2024-09-13T10:11:28.255 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.255 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133374+0000", 2024-09-13T10:11:28.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068299119999999996 2024-09-13T10:11:28.255 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.255 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.255 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.255 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133398+0000", 2024-09-13T10:11:28.255 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4209000000000001e-05 2024-09-13T10:11:28.255 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133404+0000", 2024-09-13T10:11:28.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3570000000000003e-06 2024-09-13T10:11:28.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133457+0000", 2024-09-13T10:11:28.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2320000000000001e-05 2024-09-13T10:11:28.256 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.256 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.256 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.256 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133477+0000", 2024-09-13T10:11:28.256 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0721999999999999e-05 2024-09-13T10:11:28.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.141475+0000", 2024-09-13T10:11:28.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0079973649999999993 2024-09-13T10:11:28.257 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.257 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.257 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.257 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.141510+0000", 2024-09-13T10:11:28.257 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4912e-05 2024-09-13T10:11:28.257 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.257 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.257 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.258 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.258 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.258 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:254 4.8 4:1de9da87:::smithi19530714-5:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.258 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.214446+0000", 2024-09-13T10:11:28.258 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 36.922782896000001, 2024-09-13T10:11:28.258 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014408274, 2024-09-13T10:11:28.258 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.258 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.258 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.258 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.258 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.258 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.259 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 254 2024-09-13T10:11:28.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.259 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214446+0000", 2024-09-13T10:11:28.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.259 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.259 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214446+0000", 2024-09-13T10:11:28.259 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.259 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.259 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214449+0000", 2024-09-13T10:11:28.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7860000000000001e-06 2024-09-13T10:11:28.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214464+0000", 2024-09-13T10:11:28.260 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5290000000000001e-05 2024-09-13T10:11:28.260 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.260 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.260 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.260 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214499+0000", 2024-09-13T10:11:28.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5119e-05 2024-09-13T10:11:28.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214517+0000", 2024-09-13T10:11:28.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7563000000000001e-05 2024-09-13T10:11:28.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.261 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.261 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214680+0000", 2024-09-13T10:11:28.261 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016348799999999999 2024-09-13T10:11:28.261 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.261 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214937+0000", 2024-09-13T10:11:28.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025682599999999998 2024-09-13T10:11:28.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0,11", 2024-09-13T10:11:28.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225660+0000", 2024-09-13T10:11:28.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010722768000000001 2024-09-13T10:11:28.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.262 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.262 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.262 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.227197+0000", 2024-09-13T10:11:28.262 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015373749999999999 2024-09-13T10:11:28.262 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.263 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.263 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228428+0000", 2024-09-13T10:11:28.263 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001230863 2024-09-13T10:11:28.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.263 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.263 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228646+0000", 2024-09-13T10:11:28.263 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021769700000000001 2024-09-13T10:11:28.263 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.263 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.263 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.263 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228755+0000", 2024-09-13T10:11:28.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109551 2024-09-13T10:11:28.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.264 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.264 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228854+0000", 2024-09-13T10:11:28.264 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8948000000000002e-05 2024-09-13T10:11:28.264 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.264 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.264 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.264 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.264 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.264 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1836 3.0 3:0cf17fa4:::e22ed036f41f6a390924cf1e5e56fa98e0edea9e6c33080c396679f1537fbc7a:head [write 0~2655744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-13T10:11:28.264 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:10.675934+0000", 2024-09-13T10:11:28.265 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 17.461294599999999, 2024-09-13T10:11:28.265 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015481818, 2024-09-13T10:11:28.265 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.265 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.265 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.265 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.265 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.265 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.265 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1836 2024-09-13T10:11:28.265 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.265 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.265 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.675934+0000", 2024-09-13T10:11:28.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.675934+0000", 2024-09-13T10:11:28.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.266 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.266 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.266 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.266 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.675939+0000", 2024-09-13T10:11:28.266 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2830000000000003e-06 2024-09-13T10:11:28.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677769+0000", 2024-09-13T10:11:28.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0018299830000000001 2024-09-13T10:11:28.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.267 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677796+0000", 2024-09-13T10:11:28.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7858999999999999e-05 2024-09-13T10:11:28.267 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.267 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677809+0000", 2024-09-13T10:11:28.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2291e-05 2024-09-13T10:11:28.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677895+0000", 2024-09-13T10:11:28.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6581000000000003e-05 2024-09-13T10:11:28.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.678079+0000", 2024-09-13T10:11:28.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000183802 2024-09-13T10:11:28.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:28.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.678268+0000", 2024-09-13T10:11:28.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000188383 2024-09-13T10:11:28.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.683389+0000", 2024-09-13T10:11:28.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0051215840000000002 2024-09-13T10:11:28.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.684549+0000", 2024-09-13T10:11:28.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0011600169999999999 2024-09-13T10:11:28.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.691333+0000", 2024-09-13T10:11:28.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0067840510000000001 2024-09-13T10:11:28.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.691377+0000", 2024-09-13T10:11:28.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4313000000000001e-05 2024-09-13T10:11:28.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.691416+0000", 2024-09-13T10:11:28.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8671000000000003e-05 2024-09-13T10:11:28.271 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.271 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.271 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.271 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(osd.6.23:97 3.0 3:129f02f0:::86c832e6e94279c78c969e8f58073998e1521a4d8ea062f95fd8fe010a0a109b:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:28.271 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:11.785962+0000", 2024-09-13T10:11:28.271 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.35126739, 2024-09-13T10:11:28.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013473951, 2024-09-13T10:11:28.271 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.271 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.271 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.272 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.785962+0000", 2024-09-13T10:11:28.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.785962+0000", 2024-09-13T10:11:28.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.785965+0000", 2024-09-13T10:11:28.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2169999999999999e-06 2024-09-13T10:11:28.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.787438+0000", 2024-09-13T10:11:28.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001473689 2024-09-13T10:11:28.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.787476+0000", 2024-09-13T10:11:28.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7866e-05 2024-09-13T10:11:28.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.787502+0000", 2024-09-13T10:11:28.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5695999999999999e-05 2024-09-13T10:11:28.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.787610+0000", 2024-09-13T10:11:28.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010799899999999999 2024-09-13T10:11:28.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.787729+0000", 2024-09-13T10:11:28.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119094 2024-09-13T10:11:28.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:28.275 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.787929+0000", 2024-09-13T10:11:28.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019958700000000001 2024-09-13T10:11:28.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.275 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.794652+0000", 2024-09-13T10:11:28.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0067231000000000001 2024-09-13T10:11:28.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.797887+0000", 2024-09-13T10:11:28.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032356770000000002 2024-09-13T10:11:28.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.799299+0000", 2024-09-13T10:11:28.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0014111239999999999 2024-09-13T10:11:28.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.799383+0000", 2024-09-13T10:11:28.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4439999999999998e-05 2024-09-13T10:11:28.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.277 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.277 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.799435+0000", 2024-09-13T10:11:28.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2462e-05 2024-09-13T10:11:28.277 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.277 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.277 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.277 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(osd.6.23:143 3.0 3:134e9915:::7d92d912696ecf782205efce7a4497b55ab02697829ddde6b89e83aa65ed2839:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:11:28.277 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:17.489085+0000", 2024-09-13T10:11:28.277 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.648143665999999, 2024-09-13T10:11:28.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013459029000000001, 2024-09-13T10:11:28.278 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.278 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.278 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.278 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489085+0000", 2024-09-13T10:11:28.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489085+0000", 2024-09-13T10:11:28.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489086+0000", 2024-09-13T10:11:28.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.2399999999999998e-07 2024-09-13T10:11:28.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489176+0000", 2024-09-13T10:11:28.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9863000000000003e-05 2024-09-13T10:11:28.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489183+0000", 2024-09-13T10:11:28.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9519999999999997e-06 2024-09-13T10:11:28.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489189+0000", 2024-09-13T10:11:28.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8930000000000002e-06 2024-09-13T10:11:28.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.498361+0000", 2024-09-13T10:11:28.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.009172553 2024-09-13T10:11:28.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.498492+0000", 2024-09-13T10:11:28.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013119800000000001 2024-09-13T10:11:28.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:28.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.498693+0000", 2024-09-13T10:11:28.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020111599999999999 2024-09-13T10:11:28.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.502117+0000", 2024-09-13T10:11:28.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034239280000000001 2024-09-13T10:11:28.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.502275+0000", 2024-09-13T10:11:28.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015722699999999999 2024-09-13T10:11:28.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.502371+0000", 2024-09-13T10:11:28.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5895000000000003e-05 2024-09-13T10:11:28.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.502456+0000", 2024-09-13T10:11:28.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5692000000000006e-05 2024-09-13T10:11:28.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.502544+0000", 2024-09-13T10:11:28.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8088000000000001e-05 2024-09-13T10:11:28.283 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.283 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.284 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.284 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(osd.6.23:144 3.0 3:037b20b5:::49b862fdd3fdabd6bd09ef286a067d3d49fec0ab014d5ef65dd18c71075e148b:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:11:28.284 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:17.489205+0000", 2024-09-13T10:11:28.284 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.648024404000001, 2024-09-13T10:11:28.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013662446, 2024-09-13T10:11:28.284 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.284 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.285 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.285 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.285 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489205+0000", 2024-09-13T10:11:28.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.285 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489205+0000", 2024-09-13T10:11:28.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.286 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.286 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489205+0000", 2024-09-13T10:11:28.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9500000000000003e-07 2024-09-13T10:11:28.286 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.286 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.286 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489258+0000", 2024-09-13T10:11:28.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2784000000000001e-05 2024-09-13T10:11:28.286 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.286 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.286 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489264+0000", 2024-09-13T10:11:28.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6969999999999996e-06 2024-09-13T10:11:28.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489270+0000", 2024-09-13T10:11:28.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2789999999999998e-06 2024-09-13T10:11:28.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.499601+0000", 2024-09-13T10:11:28.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010331252000000001 2024-09-13T10:11:28.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.499863+0000", 2024-09-13T10:11:28.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172599999999999 2024-09-13T10:11:28.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:28.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.500329+0000", 2024-09-13T10:11:28.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00046600299999999999 2024-09-13T10:11:28.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.502566+0000", 2024-09-13T10:11:28.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002237447 2024-09-13T10:11:28.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.502638+0000", 2024-09-13T10:11:28.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2166e-05 2024-09-13T10:11:28.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.502704+0000", 2024-09-13T10:11:28.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5209000000000005e-05 2024-09-13T10:11:28.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.502782+0000", 2024-09-13T10:11:28.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8801999999999995e-05 2024-09-13T10:11:28.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.502867+0000", 2024-09-13T10:11:28.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4586000000000006e-05 2024-09-13T10:11:28.290 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.290 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.290 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.290 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.290 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.291 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:28.292 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2500654 2024-09-13T10:11:28.293 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T10:11:28.293 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 628744 2024-09-13T10:11:28.293 INFO:tasks.rados.rados.0.smithi195.stdout:1235: finishing write tid 1 to smithi19530714-20 2024-09-13T10:11:28.293 INFO:tasks.rados.rados.0.smithi195.stdout:1235: finishing write tid 2 to smithi19530714-20 2024-09-13T10:11:28.293 INFO:tasks.rados.rados.0.smithi195.stdout:1238: got (0) Success 2024-09-13T10:11:28.293 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 20 (ObjNum 122 snap 0 seq_num 122) dirty exists 2024-09-13T10:11:28.294 INFO:tasks.rados.rados.0.smithi195.stdout:1235: finishing write tid 3 to smithi19530714-20 2024-09-13T10:11:28.294 INFO:tasks.rados.rados.0.smithi195.stdout:1237: expect (ObjNum 217 snap 0 seq_num 217) 2024-09-13T10:11:28.294 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:28.288+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:28.342 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1980162 2024-09-13T10:11:28.393 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:28.388+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:28.433 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1679743 2024-09-13T10:11:28.450 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:28.451 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:28.451 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:11:28.451 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:28.463 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T10:11:28.493 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:28.492+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:28.497 INFO:tasks.rados.rados.0.smithi195.stdout:1235: finishing write tid 4 to smithi19530714-20 2024-09-13T10:11:28.498 INFO:tasks.rados.rados.0.smithi195.stdout:1235: finishing write tid 5 to smithi19530714-20 2024-09-13T10:11:28.498 INFO:tasks.rados.rados.0.smithi195.stdout:1235: oid 20 updating version 0 to 74 2024-09-13T10:11:28.498 INFO:tasks.rados.rados.0.smithi195.stdout:1235: oid 20 updating version 74 to 75 2024-09-13T10:11:28.498 INFO:tasks.rados.rados.0.smithi195.stdout:1235: oid 20 updating version 75 to 76 2024-09-13T10:11:28.498 INFO:tasks.rados.rados.0.smithi195.stdout:1235: oid 20 version 76 is already newer than 73 2024-09-13T10:11:28.499 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 76 (ObjNum 232 snap 0 seq_num 232) dirty exists 2024-09-13T10:11:28.499 INFO:tasks.rados.rados.0.smithi195.stdout:1235: left oid 20 (ObjNum 232 snap 0 seq_num 232) 2024-09-13T10:11:28.499 INFO:tasks.rados.rados.0.smithi195.stdout:1236: expect (ObjNum 204 snap 0 seq_num 204) 2024-09-13T10:11:28.548 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1132908 2024-09-13T10:11:28.594 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:28.591+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:28.618 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:28.618 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:28.618 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:11:28.618 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:11:28.619 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:28.632 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T10:11:28.655 INFO:tasks.rados.rados.0.smithi195.stdout:1234: done (4 left) 2024-09-13T10:11:28.655 INFO:tasks.rados.rados.0.smithi195.stdout:1235: done (3 left) 2024-09-13T10:11:28.655 INFO:tasks.rados.rados.0.smithi195.stdout:1236: done (2 left) 2024-09-13T10:11:28.655 INFO:tasks.rados.rados.0.smithi195.stdout:1237: done (1 left) 2024-09-13T10:11:28.655 INFO:tasks.rados.rados.0.smithi195.stdout:1238: done (0 left) 2024-09-13T10:11:28.655 INFO:tasks.rados.rados.0.smithi195.stdout:1239: read oid 7 snap -1 2024-09-13T10:11:28.655 INFO:tasks.rados.rados.0.smithi195.stdout:1239: expect (ObjNum 218 snap 0 seq_num 218) 2024-09-13T10:11:28.656 INFO:tasks.rados.rados.0.smithi195.stdout:1240: read oid 50 snap -1 2024-09-13T10:11:28.656 INFO:tasks.rados.rados.0.smithi195.stdout:1240: expect (ObjNum 156 snap 0 seq_num 156) 2024-09-13T10:11:28.656 INFO:tasks.rados.rados.0.smithi195.stdout:1241: set_chunk oid 6 target oid 2024-09-13T10:11:28.663 INFO:tasks.rados.rados.0.smithi195.stdout:1241: set_chunk oid 6 offset: 2736640 length: 428032 target oid 0edc8fe76113ea5bdec805e565ee47709b3eae1f0055b1623ffb6ea6080bba47 offset: 0 snap -1 2024-09-13T10:11:28.663 INFO:tasks.rados.rados.0.smithi195.stdout:1242: read oid 17 snap -1 2024-09-13T10:11:28.663 INFO:tasks.rados.rados.0.smithi195.stdout:1242: expect (ObjNum 223 snap 0 seq_num 223) 2024-09-13T10:11:28.663 INFO:tasks.rados.rados.0.smithi195.stdout:1243: write oid 48 current snap is 0 2024-09-13T10:11:28.663 INFO:tasks.rados.rados.0.smithi195.stdout:1243: seq_num 233 ranges {716031=623607,2084775=605732,3298969=617062} 2024-09-13T10:11:28.670 INFO:tasks.rados.rados.0.smithi195.stdout:1243: writing smithi19530714-48 from 716031 to 1339638 tid 1 2024-09-13T10:11:28.678 INFO:tasks.rados.rados.0.smithi195.stdout:1243: writing smithi19530714-48 from 2084775 to 2690507 tid 2 2024-09-13T10:11:28.686 INFO:tasks.rados.rados.0.smithi195.stdout:1243: writing smithi19530714-48 from 3298969 to 3916031 tid 3 2024-09-13T10:11:28.686 INFO:tasks.rados.rados.0.smithi195.stdout:1244: tier_promote oid 31 2024-09-13T10:11:28.686 INFO:tasks.rados.rados.0.smithi195.stdout:1241: finishing set_chunk to oid 6 2024-09-13T10:11:28.686 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 66 (ObjNum 229 snap 0 seq_num 229) dirty exists 2024-09-13T10:11:28.686 INFO:tasks.rados.rados.0.smithi195.stdout:1240: expect (ObjNum 156 snap 0 seq_num 156) 2024-09-13T10:11:28.686 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:28.694 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:28.690+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:28.751 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:28.754 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:11:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:11:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:11:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:45 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'27, mlcod=25'19)", 2024-09-13T10:11:28.754 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.286596+0000", 2024-09-13T10:11:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 58.464197724000002, 2024-09-13T10:11:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0093831250000000008, 2024-09-13T10:11:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286596+0000", 2024-09-13T10:11:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286596+0000", 2024-09-13T10:11:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286598+0000", 2024-09-13T10:11:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8500000000000001e-06 2024-09-13T10:11:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286611+0000", 2024-09-13T10:11:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3456e-05 2024-09-13T10:11:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286635+0000", 2024-09-13T10:11:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4165e-05 2024-09-13T10:11:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286655+0000", 2024-09-13T10:11:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9383000000000001e-05 2024-09-13T10:11:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289627+0000", 2024-09-13T10:11:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029728900000000002 2024-09-13T10:11:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289685+0000", 2024-09-13T10:11:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7040000000000003e-05 2024-09-13T10:11:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.295871+0000", 2024-09-13T10:11:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061868629999999999 2024-09-13T10:11:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.295979+0000", 2024-09-13T10:11:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107478 2024-09-13T10:11:28.759 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:46 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'28, mlcod=25'19)", 2024-09-13T10:11:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.286979+0000", 2024-09-13T10:11:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 58.463814812999999, 2024-09-13T10:11:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0090737700000000001, 2024-09-13T10:11:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286979+0000", 2024-09-13T10:11:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286979+0000", 2024-09-13T10:11:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286981+0000", 2024-09-13T10:11:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0880000000000002e-06 2024-09-13T10:11:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286994+0000", 2024-09-13T10:11:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2884999999999999e-05 2024-09-13T10:11:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287017+0000", 2024-09-13T10:11:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3408e-05 2024-09-13T10:11:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.287033+0000", 2024-09-13T10:11:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.641e-05 2024-09-13T10:11:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.290659+0000", 2024-09-13T10:11:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036255749999999998 2024-09-13T10:11:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.290692+0000", 2024-09-13T10:11:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3154999999999997e-05 2024-09-13T10:11:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.295986+0000", 2024-09-13T10:11:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0052934180000000003 2024-09-13T10:11:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.296052+0000", 2024-09-13T10:11:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6830999999999998e-05 2024-09-13T10:11:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:211 3.0 e38/30 3:0596651d:::smithi19530714-12 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:11:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.742474559999998, 2024-09-13T10:11:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013316957000000001, 2024-09-13T10:11:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.765 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:11:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:11:28.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008321+0000", 2024-09-13T10:11:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-13T10:11:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011069+0000", 2024-09-13T10:11:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002747893 2024-09-13T10:11:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011113+0000", 2024-09-13T10:11:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3921000000000002e-05 2024-09-13T10:11:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011140+0000", 2024-09-13T10:11:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7662999999999999e-05 2024-09-13T10:11:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011313+0000", 2024-09-13T10:11:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017232200000000001 2024-09-13T10:11:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011366+0000", 2024-09-13T10:11:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3888000000000003e-05 2024-09-13T10:11:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021577+0000", 2024-09-13T10:11:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010210472999999999 2024-09-13T10:11:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021636+0000", 2024-09-13T10:11:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8953999999999998e-05 2024-09-13T10:11:28.770 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:212 3.0 e38/30 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:11:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.739157454000001, 2024-09-13T10:11:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014217904, 2024-09-13T10:11:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:11:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:11:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011638+0000", 2024-09-13T10:11:28.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.672e-06 2024-09-13T10:11:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013289+0000", 2024-09-13T10:11:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001651511 2024-09-13T10:11:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013310+0000", 2024-09-13T10:11:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0917999999999999e-05 2024-09-13T10:11:28.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013322+0000", 2024-09-13T10:11:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2067e-05 2024-09-13T10:11:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.016882+0000", 2024-09-13T10:11:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035599130000000001 2024-09-13T10:11:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.016934+0000", 2024-09-13T10:11:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1715999999999997e-05 2024-09-13T10:11:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025787+0000", 2024-09-13T10:11:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088531809999999999 2024-09-13T10:11:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025854+0000", 2024-09-13T10:11:28.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6926000000000003e-05 2024-09-13T10:11:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:207 3.0 e38/30 3:332cd44b:::smithi19530714-8 low_tier:head v 38'3, mlcod=0'0)", 2024-09-13T10:11:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:11:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.733364012999999, 2024-09-13T10:11:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016660892, 2024-09-13T10:11:28.776 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:11:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:11:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017432+0000", 2024-09-13T10:11:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1490000000000001e-06 2024-09-13T10:11:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021310+0000", 2024-09-13T10:11:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003877985 2024-09-13T10:11:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021351+0000", 2024-09-13T10:11:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1105e-05 2024-09-13T10:11:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021376+0000", 2024-09-13T10:11:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5185999999999999e-05 2024-09-13T10:11:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021727+0000", 2024-09-13T10:11:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035131300000000002 2024-09-13T10:11:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021792+0000", 2024-09-13T10:11:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4368000000000004e-05 2024-09-13T10:11:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034027+0000", 2024-09-13T10:11:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012235582 2024-09-13T10:11:28.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034090+0000", 2024-09-13T10:11:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3204e-05 2024-09-13T10:11:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:216 3.0 e38/30 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:11:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:11:28.781 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.726798443, 2024-09-13T10:11:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011044523000000001, 2024-09-13T10:11:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:11:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:11:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023997+0000", 2024-09-13T10:11:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.533e-06 2024-09-13T10:11:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.783 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024872+0000", 2024-09-13T10:11:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087517699999999999 2024-09-13T10:11:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024894+0000", 2024-09-13T10:11:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2571000000000001e-05 2024-09-13T10:11:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.784 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024909+0000", 2024-09-13T10:11:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5158e-05 2024-09-13T10:11:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025979+0000", 2024-09-13T10:11:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010692329999999999 2024-09-13T10:11:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.026027+0000", 2024-09-13T10:11:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8158000000000002e-05 2024-09-13T10:11:28.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034992+0000", 2024-09-13T10:11:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0089650960000000005 2024-09-13T10:11:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.035040+0000", 2024-09-13T10:11:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7596999999999998e-05 2024-09-13T10:11:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:225 3.0 e38/30 3:26960251:::smithi19530714-26 low_tier:head v 38'6, mlcod=38'3)", 2024-09-13T10:11:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:11:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.679901366000003, 2024-09-13T10:11:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013205889, 2024-09-13T10:11:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:11:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:11:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070894+0000", 2024-09-13T10:11:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5e-06 2024-09-13T10:11:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072562+0000", 2024-09-13T10:11:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016683469999999999 2024-09-13T10:11:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072632+0000", 2024-09-13T10:11:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9665000000000005e-05 2024-09-13T10:11:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072661+0000", 2024-09-13T10:11:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9714999999999999e-05 2024-09-13T10:11:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072800+0000", 2024-09-13T10:11:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013886000000000001 2024-09-13T10:11:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072828+0000", 2024-09-13T10:11:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7886e-05 2024-09-13T10:11:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084041+0000", 2024-09-13T10:11:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011213064999999999 2024-09-13T10:11:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084098+0000", 2024-09-13T10:11:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6851000000000002e-05 2024-09-13T10:11:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.791 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:228 3.0 e38/30 3:0168a995:::smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'7, mlcod=38'3)", 2024-09-13T10:11:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:11:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.676117785999999, 2024-09-13T10:11:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094647819999999997, 2024-09-13T10:11:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:11:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:11:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074677+0000", 2024-09-13T10:11:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7099999999999999e-06 2024-09-13T10:11:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076599+0000", 2024-09-13T10:11:28.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019212459999999999 2024-09-13T10:11:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076616+0000", 2024-09-13T10:11:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7008999999999999e-05 2024-09-13T10:11:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076641+0000", 2024-09-13T10:11:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5380000000000001e-05 2024-09-13T10:11:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076772+0000", 2024-09-13T10:11:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013101599999999999 2024-09-13T10:11:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076867+0000", 2024-09-13T10:11:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5262999999999999e-05 2024-09-13T10:11:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084106+0000", 2024-09-13T10:11:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0072390859999999996 2024-09-13T10:11:28.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084140+0000", 2024-09-13T10:11:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4072000000000002e-05 2024-09-13T10:11:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:252 4.6 4:6ddb2b23:::smithi19530714-3:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.796 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:11:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.628729888000002, 2024-09-13T10:11:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.03110711, 2024-09-13T10:11:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 252 2024-09-13T10:11:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:11:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:11:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122066+0000", 2024-09-13T10:11:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4619999999999999e-06 2024-09-13T10:11:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122097+0000", 2024-09-13T10:11:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0488999999999999e-05 2024-09-13T10:11:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122120+0000", 2024-09-13T10:11:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3649999999999999e-05 2024-09-13T10:11:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122130+0000", 2024-09-13T10:11:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7950000000000002e-06 2024-09-13T10:11:28.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122212+0000", 2024-09-13T10:11:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2090999999999997e-05 2024-09-13T10:11:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122317+0000", 2024-09-13T10:11:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104954 2024-09-13T10:11:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:28.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.151952+0000", 2024-09-13T10:11:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.029634908000000001 2024-09-13T10:11:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152580+0000", 2024-09-13T10:11:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062796499999999997 2024-09-13T10:11:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152891+0000", 2024-09-13T10:11:28.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000310747 2024-09-13T10:11:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153077+0000", 2024-09-13T10:11:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018588200000000001 2024-09-13T10:11:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153130+0000", 2024-09-13T10:11:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3016000000000002e-05 2024-09-13T10:11:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153171+0000", 2024-09-13T10:11:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1151e-05 2024-09-13T10:11:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:255 4.6 4:6150ba38:::smithi19530714-6:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:11:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.536315684000002, 2024-09-13T10:11:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011708387000000001, 2024-09-13T10:11:28.804 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 255 2024-09-13T10:11:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:11:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:11:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214482+0000", 2024-09-13T10:11:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8139999999999998e-06 2024-09-13T10:11:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214498+0000", 2024-09-13T10:11:28.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6636000000000001e-05 2024-09-13T10:11:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214537+0000", 2024-09-13T10:11:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8668999999999999e-05 2024-09-13T10:11:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214562+0000", 2024-09-13T10:11:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5242999999999999e-05 2024-09-13T10:11:28.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214770+0000", 2024-09-13T10:11:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020771299999999999 2024-09-13T10:11:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214996+0000", 2024-09-13T10:11:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022615999999999999 2024-09-13T10:11:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:28.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.223455+0000", 2024-09-13T10:11:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0084591960000000004 2024-09-13T10:11:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225042+0000", 2024-09-13T10:11:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001586896 2024-09-13T10:11:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225719+0000", 2024-09-13T10:11:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00067701700000000003 2024-09-13T10:11:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225845+0000", 2024-09-13T10:11:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012626299999999999 2024-09-13T10:11:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226070+0000", 2024-09-13T10:11:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022411600000000001 2024-09-13T10:11:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226186+0000", 2024-09-13T10:11:28.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011666400000000001 2024-09-13T10:11:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:268 4.6 4:6fb28ed4:::smithi19530714-19:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:11:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.535397123999999, 2024-09-13T10:11:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041608707000000002, 2024-09-13T10:11:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.811 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 268 2024-09-13T10:11:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:11:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:11:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215399+0000", 2024-09-13T10:11:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8669999999999998e-06 2024-09-13T10:11:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215416+0000", 2024-09-13T10:11:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6792000000000001e-05 2024-09-13T10:11:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215480+0000", 2024-09-13T10:11:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3805000000000003e-05 2024-09-13T10:11:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215499+0000", 2024-09-13T10:11:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9349999999999999e-05 2024-09-13T10:11:28.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215600+0000", 2024-09-13T10:11:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010119 2024-09-13T10:11:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215715+0000", 2024-09-13T10:11:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011491 2024-09-13T10:11:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:28.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.253967+0000", 2024-09-13T10:11:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038251470000000003 2024-09-13T10:11:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.255748+0000", 2024-09-13T10:11:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017815089999999999 2024-09-13T10:11:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256188+0000", 2024-09-13T10:11:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00043959799999999997 2024-09-13T10:11:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256840+0000", 2024-09-13T10:11:28.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000652591 2024-09-13T10:11:28.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256932+0000", 2024-09-13T10:11:28.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1691e-05 2024-09-13T10:11:28.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.257005+0000", 2024-09-13T10:11:28.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2934000000000003e-05 2024-09-13T10:11:28.821 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.821 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.821 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.821 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:305 4.6 4:6150ba38:::smithi19530714-6:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:28.821 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:11:28.821 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.482166431000003, 2024-09-13T10:11:28.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010323297, 2024-09-13T10:11:28.821 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.821 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.822 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.822 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.822 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.822 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.822 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 305 2024-09-13T10:11:28.822 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.822 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:11:28.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:11:28.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.823 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268630+0000", 2024-09-13T10:11:28.823 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.548e-06 2024-09-13T10:11:28.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268644+0000", 2024-09-13T10:11:28.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.394e-05 2024-09-13T10:11:28.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268672+0000", 2024-09-13T10:11:28.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8242000000000002e-05 2024-09-13T10:11:28.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268689+0000", 2024-09-13T10:11:28.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7062e-05 2024-09-13T10:11:28.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268789+0000", 2024-09-13T10:11:28.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010026600000000001 2024-09-13T10:11:28.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268905+0000", 2024-09-13T10:11:28.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115908 2024-09-13T10:11:28.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:28.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.276776+0000", 2024-09-13T10:11:28.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078712000000000001 2024-09-13T10:11:28.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278412+0000", 2024-09-13T10:11:28.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016358670000000001 2024-09-13T10:11:28.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278664+0000", 2024-09-13T10:11:28.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025162600000000002 2024-09-13T10:11:28.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278785+0000", 2024-09-13T10:11:28.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012162100000000001 2024-09-13T10:11:28.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278895+0000", 2024-09-13T10:11:28.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109774 2024-09-13T10:11:28.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278950+0000", 2024-09-13T10:11:28.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5242999999999999e-05 2024-09-13T10:11:28.830 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.830 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.831 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.831 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1023 4.16 4:6fb28ed4:::smithi19530714-19:head [write 1798066~477679] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:28.831 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.078200+0000", 2024-09-13T10:11:28.831 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.672593071000001, 2024-09-13T10:11:28.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0095557869999999996, 2024-09-13T10:11:28.831 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.831 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.831 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.832 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.832 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.832 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.832 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1023 2024-09-13T10:11:28.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.832 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078200+0000", 2024-09-13T10:11:28.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078200+0000", 2024-09-13T10:11:28.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078202+0000", 2024-09-13T10:11:28.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2529999999999999e-06 2024-09-13T10:11:28.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078505+0000", 2024-09-13T10:11:28.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000303228 2024-09-13T10:11:28.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078519+0000", 2024-09-13T10:11:28.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3918e-05 2024-09-13T10:11:28.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078526+0000", 2024-09-13T10:11:28.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.311e-06 2024-09-13T10:11:28.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078577+0000", 2024-09-13T10:11:28.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1125999999999998e-05 2024-09-13T10:11:28.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078682+0000", 2024-09-13T10:11:28.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104671 2024-09-13T10:11:28.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:28.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078864+0000", 2024-09-13T10:11:28.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018166999999999999 2024-09-13T10:11:28.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.080583+0000", 2024-09-13T10:11:28.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001719138 2024-09-13T10:11:28.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.085104+0000", 2024-09-13T10:11:28.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045216950000000001 2024-09-13T10:11:28.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.087686+0000", 2024-09-13T10:11:28.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025817499999999998 2024-09-13T10:11:28.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.087721+0000", 2024-09-13T10:11:28.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5009999999999999e-05 2024-09-13T10:11:28.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.838 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.838 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.087756+0000", 2024-09-13T10:11:28.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5017e-05 2024-09-13T10:11:28.838 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.838 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.838 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.838 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1040 4.6 4:61ec6a5a:::smithi19530714-35:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:28.838 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.095747+0000", 2024-09-13T10:11:28.838 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.655046093999999, 2024-09-13T10:11:28.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036815318999999999, 2024-09-13T10:11:28.839 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.839 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.839 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:28.839 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.839 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.839 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.839 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1040 2024-09-13T10:11:28.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.839 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095747+0000", 2024-09-13T10:11:28.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095747+0000", 2024-09-13T10:11:28.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095749+0000", 2024-09-13T10:11:28.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9039999999999999e-06 2024-09-13T10:11:28.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095757+0000", 2024-09-13T10:11:28.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6660000000000008e-06 2024-09-13T10:11:28.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095784+0000", 2024-09-13T10:11:28.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6860999999999999e-05 2024-09-13T10:11:28.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095794+0000", 2024-09-13T10:11:28.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7149999999999993e-06 2024-09-13T10:11:28.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095860+0000", 2024-09-13T10:11:28.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6699999999999995e-05 2024-09-13T10:11:28.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095984+0000", 2024-09-13T10:11:28.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123676 2024-09-13T10:11:28.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.132563+0000", 2024-09-13T10:11:28.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036578797000000003 2024-09-13T10:11:28.843 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.843 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.843 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.843 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1045 4.6 4:6150ba38:::smithi19530714-6:head [write 527753~708201,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:28.843 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.313424+0000", 2024-09-13T10:11:28.843 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.437369160000003, 2024-09-13T10:11:28.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011098433, 2024-09-13T10:11:28.843 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.843 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.844 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.844 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.844 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.844 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.844 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1045 2024-09-13T10:11:28.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.844 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313424+0000", 2024-09-13T10:11:28.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313424+0000", 2024-09-13T10:11:28.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313427+0000", 2024-09-13T10:11:28.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0230000000000001e-06 2024-09-13T10:11:28.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.845 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.845 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313756+0000", 2024-09-13T10:11:28.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032905500000000003 2024-09-13T10:11:28.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313789+0000", 2024-09-13T10:11:28.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2359e-05 2024-09-13T10:11:28.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313807+0000", 2024-09-13T10:11:28.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8130000000000001e-05 2024-09-13T10:11:28.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313953+0000", 2024-09-13T10:11:28.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014562200000000001 2024-09-13T10:11:28.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.314185+0000", 2024-09-13T10:11:28.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023246399999999999 2024-09-13T10:11:28.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:28.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.314442+0000", 2024-09-13T10:11:28.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000257023 2024-09-13T10:11:28.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.317258+0000", 2024-09-13T10:11:28.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0028160149999999998 2024-09-13T10:11:28.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.319440+0000", 2024-09-13T10:11:28.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021817149999999999 2024-09-13T10:11:28.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.324214+0000", 2024-09-13T10:11:28.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047739189999999997 2024-09-13T10:11:28.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.324440+0000", 2024-09-13T10:11:28.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000225944 2024-09-13T10:11:28.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.324523+0000", 2024-09-13T10:11:28.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3164000000000005e-05 2024-09-13T10:11:28.849 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.850 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.850 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.850 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2211 4.16 4:6ddb2b23:::smithi19530714-3:head [write 2749730~544442] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:11:28.850 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:18.121021+0000", 2024-09-13T10:11:28.850 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.62977248, 2024-09-13T10:11:28.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0091761099999999995, 2024-09-13T10:11:28.850 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.850 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.850 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.850 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.850 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.850 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.851 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2211 2024-09-13T10:11:28.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.851 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121021+0000", 2024-09-13T10:11:28.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121021+0000", 2024-09-13T10:11:28.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121023+0000", 2024-09-13T10:11:28.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.827e-06 2024-09-13T10:11:28.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121330+0000", 2024-09-13T10:11:28.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030740699999999999 2024-09-13T10:11:28.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121355+0000", 2024-09-13T10:11:28.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5198000000000002e-05 2024-09-13T10:11:28.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121369+0000", 2024-09-13T10:11:28.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3115999999999999e-05 2024-09-13T10:11:28.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121433+0000", 2024-09-13T10:11:28.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3979999999999994e-05 2024-09-13T10:11:28.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121531+0000", 2024-09-13T10:11:28.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8482999999999999e-05 2024-09-13T10:11:28.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:28.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121666+0000", 2024-09-13T10:11:28.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000134628 2024-09-13T10:11:28.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.124458+0000", 2024-09-13T10:11:28.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027928670000000001 2024-09-13T10:11:28.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.127165+0000", 2024-09-13T10:11:28.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027069110000000002 2024-09-13T10:11:28.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.130090+0000", 2024-09-13T10:11:28.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029247919999999998 2024-09-13T10:11:28.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.130146+0000", 2024-09-13T10:11:28.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5995000000000002e-05 2024-09-13T10:11:28.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.130197+0000", 2024-09-13T10:11:28.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0905999999999997e-05 2024-09-13T10:11:28.857 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.857 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.857 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.858 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2212 4.16 4:6ddb2b23:::smithi19530714-3:head [setxattr _header (55) in=7b,truncate 3294172] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:11:28.858 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:18.121414+0000", 2024-09-13T10:11:28.858 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.629379631000001, 2024-09-13T10:11:28.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00920319, 2024-09-13T10:11:28.858 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.858 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.858 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.858 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.858 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.858 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.858 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2212 2024-09-13T10:11:28.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.859 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121414+0000", 2024-09-13T10:11:28.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121414+0000", 2024-09-13T10:11:28.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121415+0000", 2024-09-13T10:11:28.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.08e-06 2024-09-13T10:11:28.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121421+0000", 2024-09-13T10:11:28.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5030000000000002e-06 2024-09-13T10:11:28.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121433+0000", 2024-09-13T10:11:28.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1392999999999999e-05 2024-09-13T10:11:28.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121442+0000", 2024-09-13T10:11:28.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7029999999999995e-06 2024-09-13T10:11:28.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.122797+0000", 2024-09-13T10:11:28.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001355247 2024-09-13T10:11:28.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.123026+0000", 2024-09-13T10:11:28.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000229044 2024-09-13T10:11:28.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:28.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.123405+0000", 2024-09-13T10:11:28.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000378797 2024-09-13T10:11:28.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.124514+0000", 2024-09-13T10:11:28.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001109547 2024-09-13T10:11:28.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.128894+0000", 2024-09-13T10:11:28.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0043794979999999999 2024-09-13T10:11:28.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.130328+0000", 2024-09-13T10:11:28.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00143436 2024-09-13T10:11:28.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.130476+0000", 2024-09-13T10:11:28.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014754199999999999 2024-09-13T10:11:28.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.130617+0000", 2024-09-13T10:11:28.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000141476 2024-09-13T10:11:28.867 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.867 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.867 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.867 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2213 4.16 4:6ddb2b23:::smithi19530714-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:11:28.868 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:18.121459+0000", 2024-09-13T10:11:28.868 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 10.629334765999999, 2024-09-13T10:11:28.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0093728219999999994, 2024-09-13T10:11:28.868 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.868 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.868 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:28.868 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.868 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.868 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.868 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2213 2024-09-13T10:11:28.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.869 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121459+0000", 2024-09-13T10:11:28.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121459+0000", 2024-09-13T10:11:28.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121460+0000", 2024-09-13T10:11:28.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3600000000000002e-07 2024-09-13T10:11:28.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121463+0000", 2024-09-13T10:11:28.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6500000000000002e-06 2024-09-13T10:11:28.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121473+0000", 2024-09-13T10:11:28.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0113e-05 2024-09-13T10:11:28.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.121481+0000", 2024-09-13T10:11:28.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8660000000000006e-06 2024-09-13T10:11:28.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.124305+0000", 2024-09-13T10:11:28.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0028241189999999999 2024-09-13T10:11:28.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:28.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.124433+0000", 2024-09-13T10:11:28.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012770300000000001 2024-09-13T10:11:28.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.130655+0000", 2024-09-13T10:11:28.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0062215530000000003 2024-09-13T10:11:28.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.130737+0000", 2024-09-13T10:11:28.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2676999999999995e-05 2024-09-13T10:11:28.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:18.130832+0000", 2024-09-13T10:11:28.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4204999999999997e-05 2024-09-13T10:11:28.873 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.873 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.873 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.873 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2603 4.6 4:6150ba38:::smithi19530714-6:head [setxattr _header (55) in=7b,truncate 3848803] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T10:11:28.874 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:26.522327+0000", 2024-09-13T10:11:28.874 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.228466353, 2024-09-13T10:11:28.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0099854200000000001, 2024-09-13T10:11:28.874 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.874 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.874 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:28.874 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.874 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.874 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.874 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2603 2024-09-13T10:11:28.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.874 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522327+0000", 2024-09-13T10:11:28.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522327+0000", 2024-09-13T10:11:28.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522329+0000", 2024-09-13T10:11:28.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.813e-06 2024-09-13T10:11:28.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522338+0000", 2024-09-13T10:11:28.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1269999999999992e-06 2024-09-13T10:11:28.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522359+0000", 2024-09-13T10:11:28.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1242000000000001e-05 2024-09-13T10:11:28.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522374+0000", 2024-09-13T10:11:28.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5085999999999999e-05 2024-09-13T10:11:28.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.523648+0000", 2024-09-13T10:11:28.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0012740049999999999 2024-09-13T10:11:28.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.523917+0000", 2024-09-13T10:11:28.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026876100000000001 2024-09-13T10:11:28.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:28.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.524319+0000", 2024-09-13T10:11:28.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040155200000000001 2024-09-13T10:11:28.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:28.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.525257+0000", 2024-09-13T10:11:28.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00093831600000000002 2024-09-13T10:11:28.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532117+0000", 2024-09-13T10:11:28.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068596530000000003 2024-09-13T10:11:28.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:28.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532185+0000", 2024-09-13T10:11:28.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8785e-05 2024-09-13T10:11:28.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:28.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532243+0000", 2024-09-13T10:11:28.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7627999999999998e-05 2024-09-13T10:11:28.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532313+0000", 2024-09-13T10:11:28.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9451999999999998e-05 2024-09-13T10:11:28.880 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.880 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.880 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.880 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2604 4.6 4:6150ba38:::smithi19530714-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-09-13T10:11:28.880 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:26.522408+0000", 2024-09-13T10:11:28.880 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 2.2283858859999999, 2024-09-13T10:11:28.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010226071999999999, 2024-09-13T10:11:28.880 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:28.881 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:28.881 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:28.881 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:28.881 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:28.881 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:28.881 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2604 2024-09-13T10:11:28.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.881 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:28.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:28.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522408+0000", 2024-09-13T10:11:28.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:28.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522408+0000", 2024-09-13T10:11:28.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:28.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:28.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522409+0000", 2024-09-13T10:11:28.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3999999999999999e-06 2024-09-13T10:11:28.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:28.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522415+0000", 2024-09-13T10:11:28.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7790000000000001e-06 2024-09-13T10:11:28.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:28.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522433+0000", 2024-09-13T10:11:28.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7972999999999999e-05 2024-09-13T10:11:28.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:28.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522449+0000", 2024-09-13T10:11:28.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5899000000000001e-05 2024-09-13T10:11:28.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.524993+0000", 2024-09-13T10:11:28.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002544499 2024-09-13T10:11:28.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:28.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.525196+0000", 2024-09-13T10:11:28.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202721 2024-09-13T10:11:28.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:28.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532374+0000", 2024-09-13T10:11:28.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0071786259999999996 2024-09-13T10:11:28.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:28.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532498+0000", 2024-09-13T10:11:28.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012336899999999999 2024-09-13T10:11:28.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:28.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:28.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:28.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532634+0000", 2024-09-13T10:11:28.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013580600000000001 2024-09-13T10:11:28.885 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.886 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.886 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.886 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:28.886 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:28.886 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:28.887 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-13T10:11:28.888 INFO:tasks.rados.rados.0.smithi195.stdout:1239: expect (ObjNum 218 snap 0 seq_num 218) 2024-09-13T10:11:28.888 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:28.884+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:28.988 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:28.987+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:29.052 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:29.052 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:29.052 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:11:29.052 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:29.057 INFO:tasks.rados.rados.0.smithi195.stdout:1242: expect (ObjNum 223 snap 0 seq_num 223) 2024-09-13T10:11:29.065 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-13T10:11:29.067 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:11:29.089 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:29.087+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:29.140 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1756525 2024-09-13T10:11:29.178 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:29.178 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:29.178 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:11:29.178 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:11:29.178 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:29.191 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:29.188+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:29.193 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-13T10:11:29.224 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:29.224 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:29.245 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:29.245 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:29.247 INFO:tasks.rados.rados.0.smithi195.stdout:1243: finishing write tid 1 to smithi19530714-48 2024-09-13T10:11:29.247 INFO:tasks.rados.rados.0.smithi195.stdout:1243: finishing write tid 2 to smithi19530714-48 2024-09-13T10:11:29.247 INFO:tasks.rados.rados.0.smithi195.stdout:1243: finishing write tid 3 to smithi19530714-48 2024-09-13T10:11:29.247 INFO:tasks.rados.rados.0.smithi195.stdout:1243: finishing write tid 4 to smithi19530714-48 2024-09-13T10:11:29.247 INFO:tasks.rados.rados.0.smithi195.stdout:1243: finishing write tid 5 to smithi19530714-48 2024-09-13T10:11:29.247 INFO:tasks.rados.rados.0.smithi195.stdout:1243: oid 48 updating version 0 to 75 2024-09-13T10:11:29.247 INFO:tasks.rados.rados.0.smithi195.stdout:1243: oid 48 updating version 75 to 76 2024-09-13T10:11:29.247 INFO:tasks.rados.rados.0.smithi195.stdout:1243: oid 48 version 76 is already newer than 73 2024-09-13T10:11:29.247 INFO:tasks.rados.rados.0.smithi195.stdout:1243: oid 48 version 76 is already newer than 74 2024-09-13T10:11:29.248 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 76 (ObjNum 233 snap 0 seq_num 233) dirty exists 2024-09-13T10:11:29.248 INFO:tasks.rados.rados.0.smithi195.stdout:1243: left oid 48 (ObjNum 233 snap 0 seq_num 233) 2024-09-13T10:11:29.248 INFO:tasks.rados.rados.0.smithi195.stdout:1244: got (0) Success 2024-09-13T10:11:29.248 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 64 (ObjNum 227 snap 0 seq_num 227) dirty exists 2024-09-13T10:11:29.248 INFO:tasks.rados.rados.0.smithi195.stdout:1239: done (5 left) 2024-09-13T10:11:29.248 INFO:tasks.rados.rados.0.smithi195.stdout:1240: done (4 left) 2024-09-13T10:11:29.248 INFO:tasks.rados.rados.0.smithi195.stdout:1241: done (3 left) 2024-09-13T10:11:29.248 INFO:tasks.rados.rados.0.smithi195.stdout:1242: done (2 left) 2024-09-13T10:11:29.248 INFO:tasks.rados.rados.0.smithi195.stdout:1243: done (1 left) 2024-09-13T10:11:29.248 INFO:tasks.rados.rados.0.smithi195.stdout:1244: done (0 left) 2024-09-13T10:11:29.249 INFO:tasks.rados.rados.0.smithi195.stdout:1245: write (excl) oid 7 current snap is 0 2024-09-13T10:11:29.249 INFO:tasks.rados.rados.0.smithi195.stdout:1245: seq_num 234 ranges {732838=698856,2026904=465743,3210039=571651} 2024-09-13T10:11:29.258 INFO:tasks.rados.rados.0.smithi195.stdout:1245: writing smithi19530714-7 from 732838 to 1431694 tid 1 2024-09-13T10:11:29.267 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:29.267 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:29.276 INFO:tasks.rados.rados.0.smithi195.stdout:1245: writing smithi19530714-7 from 2026904 to 2492647 tid 2 2024-09-13T10:11:29.288 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:29.288 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:29.291 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:29.286+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:29.295 INFO:tasks.rados.rados.0.smithi195.stdout:1245: writing smithi19530714-7 from 3210039 to 3781690 tid 3 2024-09-13T10:11:29.295 INFO:tasks.rados.rados.0.smithi195.stdout:1246: tier_promote oid 44 2024-09-13T10:11:29.295 INFO:tasks.rados.rados.0.smithi195.stdout:1245: finishing write tid 1 to smithi19530714-7 2024-09-13T10:11:29.295 INFO:tasks.rados.rados.0.smithi195.stdout:1245: finishing write tid 2 to smithi19530714-7 2024-09-13T10:11:29.295 INFO:tasks.rados.rados.0.smithi195.stdout:1247: read oid 15 snap -1 2024-09-13T10:11:29.295 INFO:tasks.rados.rados.0.smithi195.stdout:1247: expect (ObjNum 204 snap 0 seq_num 204) 2024-09-13T10:11:29.295 INFO:tasks.rados.rados.0.smithi195.stdout:1248: read oid 8 snap -1 2024-09-13T10:11:29.295 INFO:tasks.rados.rados.0.smithi195.stdout:1248: expect (ObjNum 226 snap 0 seq_num 226) 2024-09-13T10:11:29.295 INFO:tasks.rados.rados.0.smithi195.stdout:1249: write (excl) oid 21 current snap is 0 2024-09-13T10:11:29.295 INFO:tasks.rados.rados.0.smithi195.stdout:1249: seq_num 235 ranges {749645=774105,2085386=609401,3237462=712183} 2024-09-13T10:11:29.310 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:29.310 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:29.313 INFO:tasks.rados.rados.0.smithi195.stdout:1249: writing smithi19530714-21 from 749645 to 1523750 tid 1 2024-09-13T10:11:29.317 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:29.317 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:11:29.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:11:29.318 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:11:29.318 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.318 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:89 4.9 4:96fcf081:::smithi19530714-23:head [write 769754~688184,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:29.318 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.809155+0000", 2024-09-13T10:11:29.318 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.504719043000001, 2024-09-13T10:11:29.318 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011562566, 2024-09-13T10:11:29.318 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.318 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.318 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:29.318 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.318 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.318 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.319 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 89 2024-09-13T10:11:29.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.319 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.809155+0000", 2024-09-13T10:11:29.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.809155+0000", 2024-09-13T10:11:29.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.809158+0000", 2024-09-13T10:11:29.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5770000000000001e-06 2024-09-13T10:11:29.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810210+0000", 2024-09-13T10:11:29.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010519889999999999 2024-09-13T10:11:29.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810238+0000", 2024-09-13T10:11:29.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8449000000000002e-05 2024-09-13T10:11:29.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810265+0000", 2024-09-13T10:11:29.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6832e-05 2024-09-13T10:11:29.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810374+0000", 2024-09-13T10:11:29.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109274 2024-09-13T10:11:29.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810519+0000", 2024-09-13T10:11:29.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014449299999999999 2024-09-13T10:11:29.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:29.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810684+0000", 2024-09-13T10:11:29.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000165744 2024-09-13T10:11:29.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816442+0000", 2024-09-13T10:11:29.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057578990000000004 2024-09-13T10:11:29.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:29.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816685+0000", 2024-09-13T10:11:29.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024254999999999999 2024-09-13T10:11:29.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820614+0000", 2024-09-13T10:11:29.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0039290970000000003 2024-09-13T10:11:29.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:29.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820668+0000", 2024-09-13T10:11:29.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3567999999999999e-05 2024-09-13T10:11:29.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820718+0000", 2024-09-13T10:11:29.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0093999999999998e-05 2024-09-13T10:11:29.324 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.324 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.324 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.325 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:90 4.9 4:96fcf081:::smithi19530714-23:head [write 2021192~348562] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:29.325 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810322+0000", 2024-09-13T10:11:29.325 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.503551807999997, 2024-09-13T10:11:29.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01166355, 2024-09-13T10:11:29.325 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.325 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.325 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:29.325 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.325 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.325 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.325 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 90 2024-09-13T10:11:29.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.326 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810322+0000", 2024-09-13T10:11:29.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810322+0000", 2024-09-13T10:11:29.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810324+0000", 2024-09-13T10:11:29.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.097e-06 2024-09-13T10:11:29.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810494+0000", 2024-09-13T10:11:29.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016961499999999999 2024-09-13T10:11:29.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810515+0000", 2024-09-13T10:11:29.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0939999999999999e-05 2024-09-13T10:11:29.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810533+0000", 2024-09-13T10:11:29.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7909000000000001e-05 2024-09-13T10:11:29.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.814429+0000", 2024-09-13T10:11:29.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0038957409999999999 2024-09-13T10:11:29.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.814603+0000", 2024-09-13T10:11:29.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174294 2024-09-13T10:11:29.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:29.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.814804+0000", 2024-09-13T10:11:29.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020149799999999999 2024-09-13T10:11:29.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:29.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820569+0000", 2024-09-13T10:11:29.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057648389999999999 2024-09-13T10:11:29.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820761+0000", 2024-09-13T10:11:29.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019123 2024-09-13T10:11:29.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821893+0000", 2024-09-13T10:11:29.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001132051 2024-09-13T10:11:29.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:29.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821949+0000", 2024-09-13T10:11:29.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6832000000000001e-05 2024-09-13T10:11:29.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821986+0000", 2024-09-13T10:11:29.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6504000000000002e-05 2024-09-13T10:11:29.331 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.331 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.331 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.331 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:91 4.9 4:96fcf081:::smithi19530714-23:head [setxattr _header (56) in=7b,truncate 2369754] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:29.331 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810564+0000", 2024-09-13T10:11:29.332 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.503310597999999, 2024-09-13T10:11:29.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01178625, 2024-09-13T10:11:29.332 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.332 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.332 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:29.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.332 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 91 2024-09-13T10:11:29.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.332 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.332 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810564+0000", 2024-09-13T10:11:29.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810564+0000", 2024-09-13T10:11:29.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810565+0000", 2024-09-13T10:11:29.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3960000000000001e-06 2024-09-13T10:11:29.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810574+0000", 2024-09-13T10:11:29.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9339999999999995e-06 2024-09-13T10:11:29.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810590+0000", 2024-09-13T10:11:29.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5858999999999998e-05 2024-09-13T10:11:29.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810604+0000", 2024-09-13T10:11:29.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4533e-05 2024-09-13T10:11:29.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816017+0000", 2024-09-13T10:11:29.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0054122390000000001 2024-09-13T10:11:29.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816092+0000", 2024-09-13T10:11:29.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5537999999999994e-05 2024-09-13T10:11:29.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:29.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816206+0000", 2024-09-13T10:11:29.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114298 2024-09-13T10:11:29.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820805+0000", 2024-09-13T10:11:29.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004598347 2024-09-13T10:11:29.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:29.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821735+0000", 2024-09-13T10:11:29.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000930348 2024-09-13T10:11:29.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822118+0000", 2024-09-13T10:11:29.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038271099999999999 2024-09-13T10:11:29.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.337 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:29.337 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822208+0000", 2024-09-13T10:11:29.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0271000000000004e-05 2024-09-13T10:11:29.337 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.337 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822350+0000", 2024-09-13T10:11:29.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014177600000000001 2024-09-13T10:11:29.338 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.338 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.338 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.338 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:92 4.9 4:96fcf081:::smithi19530714-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:29.338 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810624+0000", 2024-09-13T10:11:29.338 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.503250172999998, 2024-09-13T10:11:29.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012449517, 2024-09-13T10:11:29.338 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.339 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.339 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:29.339 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.339 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.339 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.339 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 92 2024-09-13T10:11:29.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.339 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810624+0000", 2024-09-13T10:11:29.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810624+0000", 2024-09-13T10:11:29.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810625+0000", 2024-09-13T10:11:29.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3459999999999999e-06 2024-09-13T10:11:29.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810638+0000", 2024-09-13T10:11:29.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2853e-05 2024-09-13T10:11:29.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810655+0000", 2024-09-13T10:11:29.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6370000000000001e-05 2024-09-13T10:11:29.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810668+0000", 2024-09-13T10:11:29.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3906e-05 2024-09-13T10:11:29.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816513+0000", 2024-09-13T10:11:29.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058443369999999998 2024-09-13T10:11:29.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:29.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816657+0000", 2024-09-13T10:11:29.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014419199999999999 2024-09-13T10:11:29.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822492+0000", 2024-09-13T10:11:29.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.005835107 2024-09-13T10:11:29.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822740+0000", 2024-09-13T10:11:29.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024784600000000003 2024-09-13T10:11:29.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823074+0000", 2024-09-13T10:11:29.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033356 2024-09-13T10:11:29.343 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.343 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.343 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.344 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:129 4.9 4:998d9bf1:::smithi19530714-33:head [write 537824~473380,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:29.344 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810689+0000", 2024-09-13T10:11:29.344 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.503185397999999, 2024-09-13T10:11:29.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01301894, 2024-09-13T10:11:29.344 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.344 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.344 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:29.344 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.344 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.344 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.344 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 129 2024-09-13T10:11:29.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.345 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810689+0000", 2024-09-13T10:11:29.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810689+0000", 2024-09-13T10:11:29.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810690+0000", 2024-09-13T10:11:29.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2330000000000001e-06 2024-09-13T10:11:29.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811247+0000", 2024-09-13T10:11:29.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055654999999999995 2024-09-13T10:11:29.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811264+0000", 2024-09-13T10:11:29.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7813e-05 2024-09-13T10:11:29.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811280+0000", 2024-09-13T10:11:29.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5676e-05 2024-09-13T10:11:29.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816755+0000", 2024-09-13T10:11:29.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0054754139999999996 2024-09-13T10:11:29.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816940+0000", 2024-09-13T10:11:29.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184091 2024-09-13T10:11:29.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:29.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.817141+0000", 2024-09-13T10:11:29.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020179099999999999 2024-09-13T10:11:29.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:29.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821746+0000", 2024-09-13T10:11:29.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004604543 2024-09-13T10:11:29.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823211+0000", 2024-09-13T10:11:29.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001464824 2024-09-13T10:11:29.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823536+0000", 2024-09-13T10:11:29.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032508599999999998 2024-09-13T10:11:29.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:29.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823620+0000", 2024-09-13T10:11:29.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4524999999999995e-05 2024-09-13T10:11:29.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.350 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.350 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823708+0000", 2024-09-13T10:11:29.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7393999999999997e-05 2024-09-13T10:11:29.350 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.350 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.350 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.350 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:130 4.9 4:998d9bf1:::smithi19530714-33:head [write 1806012~625998] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:29.350 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.811308+0000", 2024-09-13T10:11:29.351 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.502566434000002, 2024-09-13T10:11:29.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012775038000000001, 2024-09-13T10:11:29.351 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.351 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.351 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:29.351 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.351 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.351 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.351 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 130 2024-09-13T10:11:29.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.351 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811308+0000", 2024-09-13T10:11:29.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811308+0000", 2024-09-13T10:11:29.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811310+0000", 2024-09-13T10:11:29.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9400000000000001e-06 2024-09-13T10:11:29.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811989+0000", 2024-09-13T10:11:29.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00067975999999999996 2024-09-13T10:11:29.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812006+0000", 2024-09-13T10:11:29.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6175999999999999e-05 2024-09-13T10:11:29.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812024+0000", 2024-09-13T10:11:29.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8813000000000001e-05 2024-09-13T10:11:29.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.818639+0000", 2024-09-13T10:11:29.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066148639999999998 2024-09-13T10:11:29.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.818763+0000", 2024-09-13T10:11:29.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012372300000000001 2024-09-13T10:11:29.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:29.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.818947+0000", 2024-09-13T10:11:29.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184394 2024-09-13T10:11:29.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:29.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822370+0000", 2024-09-13T10:11:29.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034224989999999999 2024-09-13T10:11:29.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823413+0000", 2024-09-13T10:11:29.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001042739 2024-09-13T10:11:29.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823829+0000", 2024-09-13T10:11:29.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000416477 2024-09-13T10:11:29.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.356 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:29.356 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823982+0000", 2024-09-13T10:11:29.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015238599999999999 2024-09-13T10:11:29.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.356 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824083+0000", 2024-09-13T10:11:29.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000101267 2024-09-13T10:11:29.357 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.357 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.357 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.357 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:131 4.9 4:998d9bf1:::smithi19530714-33:head [setxattr _header (56) in=7b,truncate 2537824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:29.357 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.812049+0000", 2024-09-13T10:11:29.357 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.501824675999998, 2024-09-13T10:11:29.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012575461, 2024-09-13T10:11:29.357 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.358 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.358 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:29.358 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.358 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.358 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.358 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 131 2024-09-13T10:11:29.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.358 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812049+0000", 2024-09-13T10:11:29.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812049+0000", 2024-09-13T10:11:29.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812051+0000", 2024-09-13T10:11:29.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-13T10:11:29.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812060+0000", 2024-09-13T10:11:29.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9930000000000001e-06 2024-09-13T10:11:29.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812078+0000", 2024-09-13T10:11:29.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8266000000000001e-05 2024-09-13T10:11:29.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812093+0000", 2024-09-13T10:11:29.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5153e-05 2024-09-13T10:11:29.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820907+0000", 2024-09-13T10:11:29.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088138650000000006 2024-09-13T10:11:29.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821046+0000", 2024-09-13T10:11:29.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013850900000000001 2024-09-13T10:11:29.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:29.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821338+0000", 2024-09-13T10:11:29.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029241800000000002 2024-09-13T10:11:29.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:29.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823249+0000", 2024-09-13T10:11:29.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019105610000000001 2024-09-13T10:11:29.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824157+0000", 2024-09-13T10:11:29.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090782899999999997 2024-09-13T10:11:29.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824422+0000", 2024-09-13T10:11:29.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026587900000000001 2024-09-13T10:11:29.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:29.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824497+0000", 2024-09-13T10:11:29.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4443999999999994e-05 2024-09-13T10:11:29.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824625+0000", 2024-09-13T10:11:29.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012811699999999999 2024-09-13T10:11:29.363 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.363 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.363 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.364 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:132 4.9 4:998d9bf1:::smithi19530714-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:29.364 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.812113+0000", 2024-09-13T10:11:29.364 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.501760664000003, 2024-09-13T10:11:29.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01281326, 2024-09-13T10:11:29.364 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.364 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.364 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:29.364 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.364 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.364 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.364 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 132 2024-09-13T10:11:29.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.364 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812113+0000", 2024-09-13T10:11:29.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812113+0000", 2024-09-13T10:11:29.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812115+0000", 2024-09-13T10:11:29.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.826e-06 2024-09-13T10:11:29.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812121+0000", 2024-09-13T10:11:29.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9540000000000001e-06 2024-09-13T10:11:29.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812158+0000", 2024-09-13T10:11:29.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6619000000000003e-05 2024-09-13T10:11:29.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812175+0000", 2024-09-13T10:11:29.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6662000000000001e-05 2024-09-13T10:11:29.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821791+0000", 2024-09-13T10:11:29.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.009616187 2024-09-13T10:11:29.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:29.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821855+0000", 2024-09-13T10:11:29.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4183999999999994e-05 2024-09-13T10:11:29.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824680+0000", 2024-09-13T10:11:29.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0028252749999999999 2024-09-13T10:11:29.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824784+0000", 2024-09-13T10:11:29.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010365 2024-09-13T10:11:29.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824927+0000", 2024-09-13T10:11:29.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000142903 2024-09-13T10:11:29.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.369 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.369 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.369 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:272 4.9 4:96fcf081:::smithi19530714-23:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:29.369 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.226803+0000", 2024-09-13T10:11:29.369 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.087070951000001, 2024-09-13T10:11:29.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0087344910000000005, 2024-09-13T10:11:29.369 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.369 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.369 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:29.370 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.370 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.370 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.370 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 272 2024-09-13T10:11:29.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.370 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.370 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226803+0000", 2024-09-13T10:11:29.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.370 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226803+0000", 2024-09-13T10:11:29.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226807+0000", 2024-09-13T10:11:29.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8829999999999999e-06 2024-09-13T10:11:29.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226823+0000", 2024-09-13T10:11:29.371 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5979999999999999e-05 2024-09-13T10:11:29.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226859+0000", 2024-09-13T10:11:29.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6331999999999997e-05 2024-09-13T10:11:29.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226882+0000", 2024-09-13T10:11:29.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2452000000000001e-05 2024-09-13T10:11:29.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.227037+0000", 2024-09-13T10:11:29.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000155563 2024-09-13T10:11:29.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.227232+0000", 2024-09-13T10:11:29.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019476500000000001 2024-09-13T10:11:29.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:29.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.233676+0000", 2024-09-13T10:11:29.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0064435719999999998 2024-09-13T10:11:29.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235081+0000", 2024-09-13T10:11:29.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001404953 2024-09-13T10:11:29.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:29.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235114+0000", 2024-09-13T10:11:29.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2996000000000001e-05 2024-09-13T10:11:29.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235286+0000", 2024-09-13T10:11:29.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171965 2024-09-13T10:11:29.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:29.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235431+0000", 2024-09-13T10:11:29.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000145205 2024-09-13T10:11:29.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235538+0000", 2024-09-13T10:11:29.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010682499999999999 2024-09-13T10:11:29.375 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.375 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.375 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.375 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:282 4.9 4:998d9bf1:::smithi19530714-33:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:29.376 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.240459+0000", 2024-09-13T10:11:29.376 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.073415296, 2024-09-13T10:11:29.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.022877255999999999, 2024-09-13T10:11:29.376 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.376 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.376 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:29.376 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.376 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.376 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.376 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 282 2024-09-13T10:11:29.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.376 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240459+0000", 2024-09-13T10:11:29.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240459+0000", 2024-09-13T10:11:29.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.377 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240462+0000", 2024-09-13T10:11:29.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.971e-06 2024-09-13T10:11:29.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240479+0000", 2024-09-13T10:11:29.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6937000000000001e-05 2024-09-13T10:11:29.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240512+0000", 2024-09-13T10:11:29.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3012999999999997e-05 2024-09-13T10:11:29.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240533+0000", 2024-09-13T10:11:29.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1138999999999999e-05 2024-09-13T10:11:29.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240687+0000", 2024-09-13T10:11:29.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000154275 2024-09-13T10:11:29.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240829+0000", 2024-09-13T10:11:29.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000141619 2024-09-13T10:11:29.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:29.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.260691+0000", 2024-09-13T10:11:29.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.019861706 2024-09-13T10:11:29.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:29.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.262189+0000", 2024-09-13T10:11:29.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0014983069999999999 2024-09-13T10:11:29.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.262975+0000", 2024-09-13T10:11:29.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00078655900000000004 2024-09-13T10:11:29.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.263100+0000", 2024-09-13T10:11:29.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012431899999999999 2024-09-13T10:11:29.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:29.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.263232+0000", 2024-09-13T10:11:29.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013227399999999999 2024-09-13T10:11:29.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.263336+0000", 2024-09-13T10:11:29.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104137 2024-09-13T10:11:29.382 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.382 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.382 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.382 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:296 4.9 4:9bec18ef:::smithi19530714-47:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:29.382 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.261575+0000", 2024-09-13T10:11:29.382 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.052299353999999, 2024-09-13T10:11:29.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0097761170000000008, 2024-09-13T10:11:29.383 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.383 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.383 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:29.383 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.383 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.383 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.383 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 296 2024-09-13T10:11:29.383 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.383 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261575+0000", 2024-09-13T10:11:29.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261575+0000", 2024-09-13T10:11:29.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261579+0000", 2024-09-13T10:11:29.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7740000000000002e-06 2024-09-13T10:11:29.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261594+0000", 2024-09-13T10:11:29.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4973999999999999e-05 2024-09-13T10:11:29.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261625+0000", 2024-09-13T10:11:29.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1519000000000001e-05 2024-09-13T10:11:29.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261645+0000", 2024-09-13T10:11:29.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9568e-05 2024-09-13T10:11:29.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261758+0000", 2024-09-13T10:11:29.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011301700000000001 2024-09-13T10:11:29.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261902+0000", 2024-09-13T10:11:29.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014435300000000001 2024-09-13T10:11:29.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:29.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.269548+0000", 2024-09-13T10:11:29.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0076456850000000002 2024-09-13T10:11:29.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.270867+0000", 2024-09-13T10:11:29.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013193930000000001 2024-09-13T10:11:29.387 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.387 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.387 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:29.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.270884+0000", 2024-09-13T10:11:29.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7116999999999999e-05 2024-09-13T10:11:29.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.271218+0000", 2024-09-13T10:11:29.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033386800000000002 2024-09-13T10:11:29.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.388 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:29.388 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.271305+0000", 2024-09-13T10:11:29.388 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6738999999999997e-05 2024-09-13T10:11:29.388 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.388 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.389 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.389 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.271351+0000", 2024-09-13T10:11:29.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6109999999999997e-05 2024-09-13T10:11:29.389 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.389 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.389 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.389 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.389 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.389 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:346 4.9 4:9bec18ef:::smithi19530714-47:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:29.389 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.285952+0000", 2024-09-13T10:11:29.389 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 38.027922504999999, 2024-09-13T10:11:29.389 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0081934079999999992, 2024-09-13T10:11:29.390 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.390 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.390 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:29.390 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.390 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.390 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.390 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 346 2024-09-13T10:11:29.390 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.390 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.390 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.390 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.390 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285952+0000", 2024-09-13T10:11:29.390 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285952+0000", 2024-09-13T10:11:29.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.391 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285954+0000", 2024-09-13T10:11:29.391 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3530000000000002e-06 2024-09-13T10:11:29.391 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.391 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.391 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285965+0000", 2024-09-13T10:11:29.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0718e-05 2024-09-13T10:11:29.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285988+0000", 2024-09-13T10:11:29.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3095000000000002e-05 2024-09-13T10:11:29.392 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.392 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.392 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.392 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286005+0000", 2024-09-13T10:11:29.392 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7147999999999999e-05 2024-09-13T10:11:29.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286119+0000", 2024-09-13T10:11:29.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000113999 2024-09-13T10:11:29.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.393 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.393 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286281+0000", 2024-09-13T10:11:29.393 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000161948 2024-09-13T10:11:29.393 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.393 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:29.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.291713+0000", 2024-09-13T10:11:29.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0054324810000000003 2024-09-13T10:11:29.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293743+0000", 2024-09-13T10:11:29.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020298439999999998 2024-09-13T10:11:29.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.394 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.394 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.394 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293904+0000", 2024-09-13T10:11:29.394 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000160853 2024-09-13T10:11:29.394 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:29.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293952+0000", 2024-09-13T10:11:29.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7991999999999997e-05 2024-09-13T10:11:29.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:29.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294052+0000", 2024-09-13T10:11:29.395 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9893000000000001e-05 2024-09-13T10:11:29.395 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.395 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.395 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.395 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294145+0000", 2024-09-13T10:11:29.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3084000000000005e-05 2024-09-13T10:11:29.396 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.396 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.396 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.396 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.396 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.396 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:801 4.9 4:96fcf081:::smithi19530714-23:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:29.396 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.798334+0000", 2024-09-13T10:11:29.396 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.515540147000003, 2024-09-13T10:11:29.396 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056549887, 2024-09-13T10:11:29.396 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.396 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.397 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:29.397 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.397 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.397 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.397 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 801 2024-09-13T10:11:29.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.397 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.397 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.397 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798334+0000", 2024-09-13T10:11:29.397 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.397 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.397 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798334+0000", 2024-09-13T10:11:29.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798338+0000", 2024-09-13T10:11:29.398 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3239999999999996e-06 2024-09-13T10:11:29.398 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.398 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.398 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.398 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798355+0000", 2024-09-13T10:11:29.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7149000000000001e-05 2024-09-13T10:11:29.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798403+0000", 2024-09-13T10:11:29.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7132000000000002e-05 2024-09-13T10:11:29.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.399 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.399 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798431+0000", 2024-09-13T10:11:29.399 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8365999999999999e-05 2024-09-13T10:11:29.399 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.399 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798534+0000", 2024-09-13T10:11:29.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102545 2024-09-13T10:11:29.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798715+0000", 2024-09-13T10:11:29.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018161800000000001 2024-09-13T10:11:29.400 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.400 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.400 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.400 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.854884+0000", 2024-09-13T10:11:29.400 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056168753000000002 2024-09-13T10:11:29.401 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.401 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.401 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.401 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.401 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.401 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1065 4.19 4:998d9bf1:::smithi19530714-33:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:29.401 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.618527+0000", 2024-09-13T10:11:29.401 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.695347431000002, 2024-09-13T10:11:29.401 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010820649999999999, 2024-09-13T10:11:29.401 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.401 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.401 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:29.401 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.402 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.402 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.402 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1065 2024-09-13T10:11:29.402 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.402 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.402 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.402 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.402 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618527+0000", 2024-09-13T10:11:29.402 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618527+0000", 2024-09-13T10:11:29.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.403 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618531+0000", 2024-09-13T10:11:29.403 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9899999999999999e-06 2024-09-13T10:11:29.403 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.403 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.403 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618546+0000", 2024-09-13T10:11:29.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5733e-05 2024-09-13T10:11:29.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618603+0000", 2024-09-13T10:11:29.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6075000000000001e-05 2024-09-13T10:11:29.404 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.404 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.404 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.404 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618635+0000", 2024-09-13T10:11:29.404 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2053e-05 2024-09-13T10:11:29.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618731+0000", 2024-09-13T10:11:29.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6514000000000005e-05 2024-09-13T10:11:29.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.405 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618887+0000", 2024-09-13T10:11:29.405 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000155415 2024-09-13T10:11:29.405 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.405 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.405 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.406 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.629347+0000", 2024-09-13T10:11:29.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010460870000000001 2024-09-13T10:11:29.406 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.406 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.406 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.406 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.406 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.406 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(osd.2.0:47 4.3 e46/41 4:c5b5ff75:::smithi19530714-20:head v 46'28, mlcod=43'26)", 2024-09-13T10:11:29.406 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:01.500294+0000", 2024-09-13T10:11:29.406 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 27.813580268999999, 2024-09-13T10:11:29.406 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010309196, 2024-09-13T10:11:29.406 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.406 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.407 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:29.407 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.500294+0000", 2024-09-13T10:11:29.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.407 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.407 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.407 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.500294+0000", 2024-09-13T10:11:29.407 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.407 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.500296+0000", 2024-09-13T10:11:29.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2800000000000002e-06 2024-09-13T10:11:29.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505222+0000", 2024-09-13T10:11:29.408 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0049261269999999998 2024-09-13T10:11:29.408 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.408 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.408 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.408 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505252+0000", 2024-09-13T10:11:29.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9272999999999999e-05 2024-09-13T10:11:29.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505266+0000", 2024-09-13T10:11:29.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4459e-05 2024-09-13T10:11:29.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.409 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.409 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.409 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505445+0000", 2024-09-13T10:11:29.409 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017880899999999999 2024-09-13T10:11:29.409 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505501+0000", 2024-09-13T10:11:29.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5926e-05 2024-09-13T10:11:29.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:29.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.510523+0000", 2024-09-13T10:11:29.410 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0050222440000000004 2024-09-13T10:11:29.410 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.410 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.410 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.410 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.510603+0000", 2024-09-13T10:11:29.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0078000000000001e-05 2024-09-13T10:11:29.411 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.411 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.411 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.411 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.411 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.411 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1619 4.19 4:998d9bf1:::smithi19530714-33:head [write 1750386~495032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:29.411 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.073959+0000", 2024-09-13T10:11:29.411 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 23.239915262, 2024-09-13T10:11:29.411 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0081975420000000004, 2024-09-13T10:11:29.411 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.411 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.411 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:29.411 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.412 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.412 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.412 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1619 2024-09-13T10:11:29.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.412 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.412 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.073959+0000", 2024-09-13T10:11:29.412 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.412 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.412 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.412 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.073959+0000", 2024-09-13T10:11:29.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.073961+0000", 2024-09-13T10:11:29.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3829999999999999e-06 2024-09-13T10:11:29.413 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.413 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.413 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.413 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074180+0000", 2024-09-13T10:11:29.413 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000218352 2024-09-13T10:11:29.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074208+0000", 2024-09-13T10:11:29.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7880000000000001e-05 2024-09-13T10:11:29.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.414 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074222+0000", 2024-09-13T10:11:29.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4250000000000001e-05 2024-09-13T10:11:29.414 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.414 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074360+0000", 2024-09-13T10:11:29.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000138023 2024-09-13T10:11:29.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074587+0000", 2024-09-13T10:11:29.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000226789 2024-09-13T10:11:29.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:29.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074984+0000", 2024-09-13T10:11:29.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000397031 2024-09-13T10:11:29.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:29.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.078275+0000", 2024-09-13T10:11:29.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032917250000000001 2024-09-13T10:11:29.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.080241+0000", 2024-09-13T10:11:29.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019660910000000001 2024-09-13T10:11:29.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.081869+0000", 2024-09-13T10:11:29.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001627325 2024-09-13T10:11:29.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:29.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082041+0000", 2024-09-13T10:11:29.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017206599999999999 2024-09-13T10:11:29.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082156+0000", 2024-09-13T10:11:29.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115627 2024-09-13T10:11:29.417 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.417 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.418 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.418 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1620 4.19 4:998d9bf1:::smithi19530714-33:head [setxattr _header (56) in=7b,truncate 2302559] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:29.418 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.074286+0000", 2024-09-13T10:11:29.418 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 23.239587896, 2024-09-13T10:11:29.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0086728220000000002, 2024-09-13T10:11:29.418 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.418 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.418 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:29.418 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.418 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.418 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.419 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1620 2024-09-13T10:11:29.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.419 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074286+0000", 2024-09-13T10:11:29.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074286+0000", 2024-09-13T10:11:29.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074288+0000", 2024-09-13T10:11:29.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.677e-06 2024-09-13T10:11:29.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074298+0000", 2024-09-13T10:11:29.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0020000000000001e-05 2024-09-13T10:11:29.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074316+0000", 2024-09-13T10:11:29.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7546000000000001e-05 2024-09-13T10:11:29.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074330+0000", 2024-09-13T10:11:29.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4399999999999999e-05 2024-09-13T10:11:29.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.076843+0000", 2024-09-13T10:11:29.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025133289999999999 2024-09-13T10:11:29.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.076976+0000", 2024-09-13T10:11:29.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013269000000000001 2024-09-13T10:11:29.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:29.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.077195+0000", 2024-09-13T10:11:29.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000219278 2024-09-13T10:11:29.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:29.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.078998+0000", 2024-09-13T10:11:29.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0018027379999999999 2024-09-13T10:11:29.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082240+0000", 2024-09-13T10:11:29.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032419929999999999 2024-09-13T10:11:29.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082653+0000", 2024-09-13T10:11:29.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041340499999999999 2024-09-13T10:11:29.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:29.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082806+0000", 2024-09-13T10:11:29.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015261300000000001 2024-09-13T10:11:29.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082959+0000", 2024-09-13T10:11:29.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015313300000000001 2024-09-13T10:11:29.424 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.424 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.424 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.424 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1621 4.19 4:998d9bf1:::smithi19530714-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:29.425 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.074359+0000", 2024-09-13T10:11:29.425 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 23.239515117, 2024-09-13T10:11:29.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0092381359999999992, 2024-09-13T10:11:29.425 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.425 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.425 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:29.425 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.425 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.425 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.425 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1621 2024-09-13T10:11:29.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.425 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074359+0000", 2024-09-13T10:11:29.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074359+0000", 2024-09-13T10:11:29.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074361+0000", 2024-09-13T10:11:29.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4699999999999999e-06 2024-09-13T10:11:29.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074387+0000", 2024-09-13T10:11:29.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6472e-05 2024-09-13T10:11:29.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074402+0000", 2024-09-13T10:11:29.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5437000000000001e-05 2024-09-13T10:11:29.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074411+0000", 2024-09-13T10:11:29.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8699999999999998e-06 2024-09-13T10:11:29.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.078356+0000", 2024-09-13T10:11:29.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003944894 2024-09-13T10:11:29.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.428 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:29.428 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.078565+0000", 2024-09-13T10:11:29.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020851800000000001 2024-09-13T10:11:29.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.083093+0000", 2024-09-13T10:11:29.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045284660000000001 2024-09-13T10:11:29.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.083319+0000", 2024-09-13T10:11:29.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022566200000000001 2024-09-13T10:11:29.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.083597+0000", 2024-09-13T10:11:29.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027834700000000002 2024-09-13T10:11:29.432 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.432 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.432 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.433 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1989 4.19 4:998d9bf1:::smithi19530714-33:head [write 574839~795192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T10:11:29.433 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:13.093772+0000", 2024-09-13T10:11:29.433 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.220102519000001, 2024-09-13T10:11:29.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.008334045, 2024-09-13T10:11:29.433 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.433 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.433 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:29.433 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.433 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.433 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.434 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1989 2024-09-13T10:11:29.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.434 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.093772+0000", 2024-09-13T10:11:29.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.093772+0000", 2024-09-13T10:11:29.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.093774+0000", 2024-09-13T10:11:29.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1519999999999999e-06 2024-09-13T10:11:29.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094142+0000", 2024-09-13T10:11:29.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036777400000000002 2024-09-13T10:11:29.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094158+0000", 2024-09-13T10:11:29.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6574000000000001e-05 2024-09-13T10:11:29.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094167+0000", 2024-09-13T10:11:29.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9299999999999992e-06 2024-09-13T10:11:29.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094284+0000", 2024-09-13T10:11:29.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116895 2024-09-13T10:11:29.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094428+0000", 2024-09-13T10:11:29.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014431099999999999 2024-09-13T10:11:29.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:29.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094658+0000", 2024-09-13T10:11:29.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000230106 2024-09-13T10:11:29.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:29.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.098293+0000", 2024-09-13T10:11:29.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003634918 2024-09-13T10:11:29.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.100887+0000", 2024-09-13T10:11:29.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025936309999999999 2024-09-13T10:11:29.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.101823+0000", 2024-09-13T10:11:29.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00093637600000000005 2024-09-13T10:11:29.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:29.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.101980+0000", 2024-09-13T10:11:29.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015709400000000001 2024-09-13T10:11:29.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.102106+0000", 2024-09-13T10:11:29.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125284 2024-09-13T10:11:29.439 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.439 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.439 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.439 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2416 4.19 4:998d9bf1:::smithi19530714-33:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e63)", 2024-09-13T10:11:29.440 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:22.232954+0000", 2024-09-13T10:11:29.440 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 7.0809203820000004, 2024-09-13T10:11:29.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010811217999999999, 2024-09-13T10:11:29.440 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:29.440 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:29.440 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:29.440 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:29.440 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:29.440 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:29.440 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2416 2024-09-13T10:11:29.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.440 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:29.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:29.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.232954+0000", 2024-09-13T10:11:29.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:29.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.232954+0000", 2024-09-13T10:11:29.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:29.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:29.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.232957+0000", 2024-09-13T10:11:29.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0240000000000002e-06 2024-09-13T10:11:29.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:29.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.232969+0000", 2024-09-13T10:11:29.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2602000000000001e-05 2024-09-13T10:11:29.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:29.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.233001+0000", 2024-09-13T10:11:29.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1486000000000003e-05 2024-09-13T10:11:29.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:29.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.233017+0000", 2024-09-13T10:11:29.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6565000000000001e-05 2024-09-13T10:11:29.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.233160+0000", 2024-09-13T10:11:29.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014236000000000001 2024-09-13T10:11:29.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op must wait for map", 2024-09-13T10:11:29.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.233170+0000", 2024-09-13T10:11:29.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9599999999999995e-06 2024-09-13T10:11:29.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:29.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.236247+0000", 2024-09-13T10:11:29.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030776990000000001 2024-09-13T10:11:29.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:29.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.236509+0000", 2024-09-13T10:11:29.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026104399999999999 2024-09-13T10:11:29.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:29.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.236883+0000", 2024-09-13T10:11:29.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00037440500000000002 2024-09-13T10:11:29.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:29.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.240643+0000", 2024-09-13T10:11:29.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003760354 2024-09-13T10:11:29.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.241525+0000", 2024-09-13T10:11:29.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00088204699999999995 2024-09-13T10:11:29.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:29.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.243591+0000", 2024-09-13T10:11:29.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020659039999999999 2024-09-13T10:11:29.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:29.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.243723+0000", 2024-09-13T10:11:29.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013196 2024-09-13T10:11:29.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:29.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:29.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:29.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.243765+0000", 2024-09-13T10:11:29.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1807999999999997e-05 2024-09-13T10:11:29.447 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.447 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.447 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.447 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:29.447 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:29.447 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:29.448 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:29.448 INFO:tasks.rados.rados.0.smithi195.stdout:1249: writing smithi19530714-21 from 2085386 to 2694787 tid 2 2024-09-13T10:11:29.449 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:29.449 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:29.450 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:29.450 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-13T10:11:29.451 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:29.451 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:29.452 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:29.452 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:29.453 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:29.453 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:29.454 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:29.454 INFO:tasks.rados.rados.0.smithi195.stdout:1249: writing smithi19530714-21 from 3237462 to 3949645 tid 3 2024-09-13T10:11:29.454 INFO:tasks.rados.rados.0.smithi195.stdout:1250: read oid 47 snap -1 2024-09-13T10:11:29.454 INFO:tasks.rados.rados.0.smithi195.stdout:1250: expect (ObjNum 182 snap 0 seq_num 182) 2024-09-13T10:11:29.454 INFO:tasks.rados.rados.0.smithi195.stdout:1251: set_chunk oid 37 target oid 2024-09-13T10:11:29.454 INFO:tasks.rados.rados.0.smithi195.stdout:1251: set_chunk oid 37 offset: 1130496 length: 1120768 target oid 2a5fc62cb5ad8dde0092f9f8f65432545589ccca16197221bb81db86e8292db4 offset: 0 snap -1 2024-09-13T10:11:29.454 INFO:tasks.rados.rados.0.smithi195.stdout:1246: got (0) Success 2024-09-13T10:11:29.455 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 57 (ObjNum 205 snap 0 seq_num 205) dirty exists 2024-09-13T10:11:29.455 INFO:tasks.rados.rados.0.smithi195.stdout:1252: read oid 48 snap -1 2024-09-13T10:11:29.455 INFO:tasks.rados.rados.0.smithi195.stdout:1252: expect (ObjNum 233 snap 0 seq_num 233) 2024-09-13T10:11:29.455 INFO:tasks.rados.rados.0.smithi195.stdout:1246: done (7 left) 2024-09-13T10:11:29.455 INFO:tasks.rados.rados.0.smithi195.stdout:1253: write oid 41 current snap is 0 2024-09-13T10:11:29.455 INFO:tasks.rados.rados.0.smithi195.stdout:1253: seq_num 236 ranges {766452=565707,1860221=469412,2981238=594195} 2024-09-13T10:11:29.455 INFO:tasks.rados.rados.0.smithi195.stdout:1253: writing smithi19530714-41 from 766452 to 1332159 tid 1 2024-09-13T10:11:29.455 INFO:tasks.rados.rados.0.smithi195.stdout:1253: writing smithi19530714-41 from 1860221 to 2329633 tid 2 2024-09-13T10:11:29.455 INFO:tasks.rados.rados.0.smithi195.stdout:1253: writing smithi19530714-41 from 2981238 to 3575433 tid 3 2024-09-13T10:11:29.455 INFO:tasks.rados.rados.0.smithi195.stdout:1254: read oid 10 snap -1 2024-09-13T10:11:29.455 INFO:tasks.rados.rados.0.smithi195.stdout:1254: expect (ObjNum 207 snap 0 seq_num 207) 2024-09-13T10:11:29.455 INFO:tasks.rados.rados.0.smithi195.stdout:1255: tier_flush oid 1 2024-09-13T10:11:29.455 INFO:tasks.rados.rados.0.smithi195.stdout:1255: tier_flush oid 1 snap -1 2024-09-13T10:11:29.456 INFO:tasks.rados.rados.0.smithi195.stdout:1245: finishing write tid 3 to smithi19530714-7 2024-09-13T10:11:29.456 INFO:tasks.rados.rados.0.smithi195.stdout:1256: tier_promote oid 38 2024-09-13T10:11:29.456 INFO:tasks.rados.rados.0.smithi195.stdout:1245: finishing write tid 4 to smithi19530714-7 2024-09-13T10:11:29.456 INFO:tasks.rados.rados.0.smithi195.stdout:1245: finishing write tid 5 to smithi19530714-7 2024-09-13T10:11:29.456 INFO:tasks.rados.rados.0.smithi195.stdout:1245: oid 7 updating version 0 to 83 2024-09-13T10:11:29.456 INFO:tasks.rados.rados.0.smithi195.stdout:1245: oid 7 version 83 is already newer than 80 2024-09-13T10:11:29.456 INFO:tasks.rados.rados.0.smithi195.stdout:1245: oid 7 version 83 is already newer than 81 2024-09-13T10:11:29.456 INFO:tasks.rados.rados.0.smithi195.stdout:1245: oid 7 version 83 is already newer than 82 2024-09-13T10:11:29.456 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 83 (ObjNum 234 snap 0 seq_num 234) dirty exists 2024-09-13T10:11:29.456 INFO:tasks.rados.rados.0.smithi195.stdout:1245: left oid 7 (ObjNum 234 snap 0 seq_num 234) 2024-09-13T10:11:29.456 INFO:tasks.rados.rados.0.smithi195.stdout:1247: expect (ObjNum 204 snap 0 seq_num 204) 2024-09-13T10:11:29.456 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1795222 2024-09-13T10:11:29.457 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:29.457 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:29.444+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:29.458 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:29.459 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:29.476 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:29.476 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:29.496 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:29.497 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:29.521 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:29.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:29.516+0000 7fa4d4ef7640 0 -- 172.21.15.73:0/1197378979 >> v2:172.21.15.195:6805/848686366 conn(0x7fa4d02251e0 msgr2=0x7fa4d02255e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:11:29.525 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:29.549 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:29.544+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:29.586 INFO:tasks.rados.rados.0.smithi195.stdout:1248: expect (ObjNum 226 snap 0 seq_num 226) 2024-09-13T10:11:29.586 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:29.611 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:29.611 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:29.611 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:11:29.611 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:29.624 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-13T10:11:29.650 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:29.648+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:29.673 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:29.718 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T10:11:29.747 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:29.748 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:29.751 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:29.748+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:29.785 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:29.785 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:29.798 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-13T10:11:29.818 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3146239 2024-09-13T10:11:29.834 INFO:tasks.rados.rados.0.smithi195.stdout:1249: finishing write tid 1 to smithi19530714-21 2024-09-13T10:11:29.834 INFO:tasks.rados.rados.0.smithi195.stdout:1249: finishing write tid 2 to smithi19530714-21 2024-09-13T10:11:29.835 INFO:tasks.rados.rados.0.smithi195.stdout:1250: expect (ObjNum 182 snap 0 seq_num 182) 2024-09-13T10:11:29.852 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:29.848+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:29.912 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 411766 2024-09-13T10:11:29.936 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:29.937 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:29.950 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T10:11:29.953 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:29.948+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:29.954 INFO:tasks.rados.rados.0.smithi195.stdout:1249: finishing write tid 3 to smithi19530714-21 2024-09-13T10:11:29.954 INFO:tasks.rados.rados.0.smithi195.stdout:1249: finishing write tid 4 to smithi19530714-21 2024-09-13T10:11:29.954 INFO:tasks.rados.rados.0.smithi195.stdout:1249: finishing write tid 5 to smithi19530714-21 2024-09-13T10:11:29.954 INFO:tasks.rados.rados.0.smithi195.stdout:1249: oid 21 updating version 0 to 112 2024-09-13T10:11:29.954 INFO:tasks.rados.rados.0.smithi195.stdout:1249: oid 21 updating version 112 to 113 2024-09-13T10:11:29.954 INFO:tasks.rados.rados.0.smithi195.stdout:1249: oid 21 version 113 is already newer than 111 2024-09-13T10:11:29.955 INFO:tasks.rados.rados.0.smithi195.stdout:1249: oid 21 updating version 113 to 114 2024-09-13T10:11:29.955 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 114 (ObjNum 235 snap 0 seq_num 235) dirty exists 2024-09-13T10:11:29.955 INFO:tasks.rados.rados.0.smithi195.stdout:1249: left oid 21 (ObjNum 235 snap 0 seq_num 235) 2024-09-13T10:11:29.955 INFO:tasks.rados.rados.0.smithi195.stdout:1245: done (10 left) 2024-09-13T10:11:29.955 INFO:tasks.rados.rados.0.smithi195.stdout:1247: done (9 left) 2024-09-13T10:11:29.955 INFO:tasks.rados.rados.0.smithi195.stdout:1248: done (8 left) 2024-09-13T10:11:29.955 INFO:tasks.rados.rados.0.smithi195.stdout:1249: done (7 left) 2024-09-13T10:11:29.955 INFO:tasks.rados.rados.0.smithi195.stdout:1250: done (6 left) 2024-09-13T10:11:29.955 INFO:tasks.rados.rados.0.smithi195.stdout:1257: tier_promote oid 28 2024-09-13T10:11:29.955 INFO:tasks.rados.rados.0.smithi195.stdout:1251: finishing set_chunk to oid 37 2024-09-13T10:11:29.955 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 37 set_chunk returned error code -95 offset: 1130496 length: 1120768 2024-09-13T10:11:29.955 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 0 (ObjNum 145 snap 0 seq_num 145) dirty exists 2024-09-13T10:11:29.955 INFO:tasks.rados.rados.0.smithi195.stdout:1252: expect (ObjNum 233 snap 0 seq_num 233) 2024-09-13T10:11:30.053 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:30.050+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:30.074 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:30.074 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:30.089 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T10:11:30.131 INFO:tasks.rados.rados.0.smithi195.stdout:1253: finishing write tid 1 to smithi19530714-41 2024-09-13T10:11:30.131 INFO:tasks.rados.rados.0.smithi195.stdout:1253: finishing write tid 2 to smithi19530714-41 2024-09-13T10:11:30.131 INFO:tasks.rados.rados.0.smithi195.stdout:1256: got (0) Success 2024-09-13T10:11:30.132 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 105 (ObjNum 193 snap 0 seq_num 193) dirty exists 2024-09-13T10:11:30.132 INFO:tasks.rados.rados.0.smithi195.stdout:1253: finishing write tid 3 to smithi19530714-41 2024-09-13T10:11:30.132 INFO:tasks.rados.rados.0.smithi195.stdout:1253: finishing write tid 4 to smithi19530714-41 2024-09-13T10:11:30.132 INFO:tasks.rados.rados.0.smithi195.stdout:1253: finishing write tid 5 to smithi19530714-41 2024-09-13T10:11:30.132 INFO:tasks.rados.rados.0.smithi195.stdout:1253: oid 41 updating version 0 to 33 2024-09-13T10:11:30.132 INFO:tasks.rados.rados.0.smithi195.stdout:1253: oid 41 updating version 33 to 36 2024-09-13T10:11:30.132 INFO:tasks.rados.rados.0.smithi195.stdout:1253: oid 41 version 36 is already newer than 34 2024-09-13T10:11:30.132 INFO:tasks.rados.rados.0.smithi195.stdout:1253: oid 41 version 36 is already newer than 35 2024-09-13T10:11:30.132 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 36 (ObjNum 236 snap 0 seq_num 236) dirty exists 2024-09-13T10:11:30.132 INFO:tasks.rados.rados.0.smithi195.stdout:1253: left oid 41 (ObjNum 236 snap 0 seq_num 236) 2024-09-13T10:11:30.132 INFO:tasks.rados.rados.0.smithi195.stdout:1254: expect (ObjNum 207 snap 0 seq_num 207) 2024-09-13T10:11:30.154 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:30.151+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:30.209 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:30.209 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:30.223 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T10:11:30.255 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:30.252+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:30.329 INFO:tasks.rados.rados.0.smithi195.stdout:1255: got (0) Success 2024-09-13T10:11:30.329 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 0 (ObjNum 184 snap 0 seq_num 184) dirty exists 2024-09-13T10:11:30.329 INFO:tasks.rados.rados.0.smithi195.stdout:1257: got (0) Success 2024-09-13T10:11:30.329 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 96 (ObjNum 180 snap 0 seq_num 180) dirty exists 2024-09-13T10:11:30.329 INFO:tasks.rados.rados.0.smithi195.stdout:1251: done (6 left) 2024-09-13T10:11:30.329 INFO:tasks.rados.rados.0.smithi195.stdout:1252: done (5 left) 2024-09-13T10:11:30.329 INFO:tasks.rados.rados.0.smithi195.stdout:1253: done (4 left) 2024-09-13T10:11:30.330 INFO:tasks.rados.rados.0.smithi195.stdout:1254: done (3 left) 2024-09-13T10:11:30.330 INFO:tasks.rados.rados.0.smithi195.stdout:1255: done (2 left) 2024-09-13T10:11:30.330 INFO:tasks.rados.rados.0.smithi195.stdout:1256: done (1 left) 2024-09-13T10:11:30.330 INFO:tasks.rados.rados.0.smithi195.stdout:1257: done (0 left) 2024-09-13T10:11:30.330 INFO:tasks.rados.rados.0.smithi195.stdout:1258: read oid 23 snap -1 2024-09-13T10:11:30.330 INFO:tasks.rados.rados.0.smithi195.stdout:1258: expect (ObjNum 206 snap 0 seq_num 206) 2024-09-13T10:11:30.330 INFO:tasks.rados.rados.0.smithi195.stdout:1259: set_chunk oid 27 target oid 2024-09-13T10:11:30.345 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:30.345 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:30.348 INFO:tasks.rados.rados.0.smithi195.stdout:1259: set_chunk oid 27 offset: 2611712 length: 872448 target oid d3901c599cd0a4dcc46f9c14ba77479701f5c6164f831a1598a7f5e5c5f0b8f2 offset: 0 snap -1 2024-09-13T10:11:30.348 INFO:tasks.rados.rados.0.smithi195.stdout:1260: write (excl) oid 3 current snap is 0 2024-09-13T10:11:30.348 INFO:tasks.rados.rados.0.smithi195.stdout:1260: seq_num 237 ranges {783259=640956,1918703=613070,3008661=405467,3821939=161320} 2024-09-13T10:11:30.355 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:30.350+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:30.360 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T10:11:30.371 INFO:tasks.rados.rados.0.smithi195.stdout:1260: writing smithi19530714-3 from 783259 to 1424215 tid 1 2024-09-13T10:11:30.388 INFO:tasks.rados.rados.0.smithi195.stdout:1260: writing smithi19530714-3 from 1918703 to 2531773 tid 2 2024-09-13T10:11:30.400 INFO:tasks.rados.rados.0.smithi195.stdout:1260: writing smithi19530714-3 from 3008661 to 3414128 tid 3 2024-09-13T10:11:30.405 INFO:tasks.rados.rados.0.smithi195.stdout:1260: writing smithi19530714-3 from 3821939 to 3983259 tid 4 2024-09-13T10:11:30.405 INFO:tasks.rados.rados.0.smithi195.stdout:1261: delete oid 7 current snap is 0 2024-09-13T10:11:30.405 INFO:tasks.rados.rados.0.smithi195.stdout:1258: expect (ObjNum 206 snap 0 seq_num 206) 2024-09-13T10:11:30.456 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:30.455+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:30.477 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:30.477 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:30.491 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T10:11:30.507 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2540287 2024-09-13T10:11:30.544 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2526380 2024-09-13T10:11:30.557 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:30.552+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:30.560 INFO:tasks.rados.rados.0.smithi195.stdout:1259: finishing set_chunk to oid 27 2024-09-13T10:11:30.560 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 27 set_chunk returned error code -95 offset: 2611712 length: 872448 2024-09-13T10:11:30.560 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 0 (ObjNum 221 snap 0 seq_num 221) dirty exists 2024-09-13T10:11:30.560 INFO:tasks.rados.rados.0.smithi195.stdout:1260: finishing write tid 1 to smithi19530714-3 2024-09-13T10:11:30.560 INFO:tasks.rados.rados.0.smithi195.stdout:1260: finishing write tid 2 to smithi19530714-3 2024-09-13T10:11:30.560 INFO:tasks.rados.rados.0.smithi195.stdout:1260: finishing write tid 3 to smithi19530714-3 2024-09-13T10:11:30.560 INFO:tasks.rados.rados.0.smithi195.stdout:1260: finishing write tid 4 to smithi19530714-3 2024-09-13T10:11:30.560 INFO:tasks.rados.rados.0.smithi195.stdout:1260: finishing write tid 5 to smithi19530714-3 2024-09-13T10:11:30.560 INFO:tasks.rados.rados.0.smithi195.stdout:1260: finishing write tid 6 to smithi19530714-3 2024-09-13T10:11:30.561 INFO:tasks.rados.rados.0.smithi195.stdout:1260: oid 3 updating version 0 to 64 2024-09-13T10:11:30.561 INFO:tasks.rados.rados.0.smithi195.stdout:1260: oid 3 updating version 64 to 65 2024-09-13T10:11:30.561 INFO:tasks.rados.rados.0.smithi195.stdout:1260: oid 3 updating version 65 to 66 2024-09-13T10:11:30.561 INFO:tasks.rados.rados.0.smithi195.stdout:1260: oid 3 version 66 is already newer than 62 2024-09-13T10:11:30.561 INFO:tasks.rados.rados.0.smithi195.stdout:1260: oid 3 version 66 is already newer than 63 2024-09-13T10:11:30.561 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 66 (ObjNum 237 snap 0 seq_num 237) dirty exists 2024-09-13T10:11:30.561 INFO:tasks.rados.rados.0.smithi195.stdout:1260: left oid 3 (ObjNum 237 snap 0 seq_num 237) 2024-09-13T10:11:30.561 INFO:tasks.rados.rados.0.smithi195.stdout:1258: done (3 left) 2024-09-13T10:11:30.561 INFO:tasks.rados.rados.0.smithi195.stdout:1259: done (2 left) 2024-09-13T10:11:30.561 INFO:tasks.rados.rados.0.smithi195.stdout:1260: done (1 left) 2024-09-13T10:11:30.561 INFO:tasks.rados.rados.0.smithi195.stdout:1261: done (0 left) 2024-09-13T10:11:30.562 INFO:tasks.rados.rados.0.smithi195.stdout:1262: read oid 45 snap -1 2024-09-13T10:11:30.562 INFO:tasks.rados.rados.0.smithi195.stdout:1262: expect (ObjNum 202 snap 0 seq_num 202) 2024-09-13T10:11:30.562 INFO:tasks.rados.rados.0.smithi195.stdout:1263: read oid 18 snap -1 2024-09-13T10:11:30.562 INFO:tasks.rados.rados.0.smithi195.stdout:1263: expect (ObjNum 189 snap 0 seq_num 189) 2024-09-13T10:11:30.562 INFO:tasks.rados.rados.0.smithi195.stdout:1264: set_chunk oid 41 target oid 2024-09-13T10:11:30.606 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:30.607 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:30.617 INFO:tasks.rados.rados.0.smithi195.stdout:1264: set_chunk oid 41 offset: 418816 length: 3412992 target oid 8bc69220bc832b3f62fb37124de6a1f3e8366e5fb3ca5fcc7f60f4f608949c42 offset: 0 snap -1 2024-09-13T10:11:30.617 INFO:tasks.rados.rados.0.smithi195.stdout:1265: set_chunk oid 33 target oid 2024-09-13T10:11:30.617 INFO:tasks.rados.rados.0.smithi195.stdout:1265: set_chunk oid 33 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:30.617 INFO:tasks.rados.rados.0.smithi195.stdout:1266: write (excl) oid 4 current snap is 0 2024-09-13T10:11:30.617 INFO:tasks.rados.rados.0.smithi195.stdout:1266: seq_num 239 ranges {416873=791454,1919314=97559} 2024-09-13T10:11:30.622 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T10:11:30.636 INFO:tasks.rados.rados.0.smithi195.stdout:1266: writing smithi19530714-4 from 416873 to 1208327 tid 1 2024-09-13T10:11:30.650 INFO:tasks.rados.rados.0.smithi195.stdout:1266: writing smithi19530714-4 from 1919314 to 2016873 tid 2 2024-09-13T10:11:30.650 INFO:tasks.rados.rados.0.smithi195.stdout:1267: read oid 47 snap -1 2024-09-13T10:11:30.650 INFO:tasks.rados.rados.0.smithi195.stdout:1267: expect (ObjNum 182 snap 0 seq_num 182) 2024-09-13T10:11:30.650 INFO:tasks.rados.rados.0.smithi195.stdout:1268: write oid 1 current snap is 0 2024-09-13T10:11:30.650 INFO:tasks.rados.rados.0.smithi195.stdout:1268: seq_num 240 ranges {433680=466703,1577796=455884} 2024-09-13T10:11:30.658 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:30.655+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:30.664 INFO:tasks.rados.rados.0.smithi195.stdout:1268: writing smithi19530714-1 from 433680 to 900383 tid 1 2024-09-13T10:11:30.676 INFO:tasks.rados.rados.0.smithi195.stdout:1268: writing smithi19530714-1 from 1577796 to 2033680 tid 2 2024-09-13T10:11:30.677 INFO:tasks.rados.rados.0.smithi195.stdout:1269: write oid 39 current snap is 0 2024-09-13T10:11:30.677 INFO:tasks.rados.rados.0.smithi195.stdout:1269: seq_num 241 ranges {450487=541952,1636278=414209} 2024-09-13T10:11:30.686 INFO:tasks.rados.rados.0.smithi195.stdout:1269: writing smithi19530714-39 from 450487 to 992439 tid 1 2024-09-13T10:11:30.695 INFO:tasks.rados.rados.0.smithi195.stdout:1269: writing smithi19530714-39 from 1636278 to 2050487 tid 2 2024-09-13T10:11:30.695 INFO:tasks.rados.rados.0.smithi195.stdout:1270: delete oid 37 current snap is 0 2024-09-13T10:11:30.695 INFO:tasks.rados.rados.0.smithi195.stdout:1262: expect (ObjNum 202 snap 0 seq_num 202) 2024-09-13T10:11:30.759 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:30.756+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:30.792 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:30.793 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:30.807 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-13T10:11:30.839 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:11:30.839 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:11:30.839 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:11:30.839 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T10:11:30.839 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T10:11:30.902 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:30.900+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:30.927 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:30.927 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:30.940 INFO:tasks.rados.rados.0.smithi195.stdout:1263: expect (ObjNum 189 snap 0 seq_num 189) 2024-09-13T10:11:30.941 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-13T10:11:30.946 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 672550 2024-09-13T10:11:30.982 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:30.980+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:31.010 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-13T10:11:31.062 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:31.058+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:31.093 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:31.093 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:31.096 INFO:tasks.rados.rados.0.smithi195.stdout:1264: finishing set_chunk to oid 41 2024-09-13T10:11:31.096 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 41 set_chunk returned error code -95 offset: 418816 length: 3412992 2024-09-13T10:11:31.096 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 0 (ObjNum 236 snap 0 seq_num 236) dirty exists 2024-09-13T10:11:31.096 INFO:tasks.rados.rados.0.smithi195.stdout:1265: finishing set_chunk to oid 33 2024-09-13T10:11:31.096 INFO:tasks.rados.rados.0.smithi195.stdout:1265: got expected ENOENT (src dne) 2024-09-13T10:11:31.096 INFO:tasks.rados.rados.0.smithi195.stdout:1266: finishing write tid 1 to smithi19530714-4 2024-09-13T10:11:31.096 INFO:tasks.rados.rados.0.smithi195.stdout:1267: expect (ObjNum 182 snap 0 seq_num 182) 2024-09-13T10:11:31.105 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1215781 2024-09-13T10:11:31.107 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-13T10:11:31.136 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:31.160 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:11:31.160 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:31.160 INFO:teuthology.orchestra.run.smithi073.stdout: "epoch": 70, 2024-09-13T10:11:31.160 INFO:teuthology.orchestra.run.smithi073.stdout: "fsid": "c1e91d47-9492-4461-a043-8698fd74ccf3", 2024-09-13T10:11:31.160 INFO:teuthology.orchestra.run.smithi073.stdout: "created": "2024-09-13T10:10:14.019912+0000", 2024-09-13T10:11:31.160 INFO:teuthology.orchestra.run.smithi073.stdout: "modified": "2024-09-13T10:11:31.001317+0000", 2024-09-13T10:11:31.160 INFO:teuthology.orchestra.run.smithi073.stdout: "last_up_change": "2024-09-13T10:10:28.089885+0000", 2024-09-13T10:11:31.160 INFO:teuthology.orchestra.run.smithi073.stdout: "last_in_change": "2024-09-13T10:10:18.489546+0000", 2024-09-13T10:11:31.160 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T10:11:31.160 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_num": 5801984, 2024-09-13T10:11:31.161 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_set": [ 2024-09-13T10:11:31.161 INFO:teuthology.orchestra.run.smithi073.stdout: "noscrub", 2024-09-13T10:11:31.161 INFO:teuthology.orchestra.run.smithi073.stdout: "pglog_hardlimit", 2024-09-13T10:11:31.161 INFO:teuthology.orchestra.run.smithi073.stdout: "purged_snapdirs", 2024-09-13T10:11:31.161 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_deletes", 2024-09-13T10:11:31.161 INFO:teuthology.orchestra.run.smithi073.stdout: "sortbitwise" 2024-09-13T10:11:31.161 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:31.161 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_version": 6, 2024-09-13T10:11:31.161 INFO:teuthology.orchestra.run.smithi073.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T10:11:31.161 INFO:teuthology.orchestra.run.smithi073.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T10:11:31.161 INFO:teuthology.orchestra.run.smithi073.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T10:11:31.161 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_snapshot": "", 2024-09-13T10:11:31.161 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_max": 4, 2024-09-13T10:11:31.162 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd": 16, 2024-09-13T10:11:31.162 INFO:teuthology.orchestra.run.smithi073.stdout: "require_min_compat_client": "luminous", 2024-09-13T10:11:31.162 INFO:teuthology.orchestra.run.smithi073.stdout: "min_compat_client": "luminous", 2024-09-13T10:11:31.162 INFO:teuthology.orchestra.run.smithi073.stdout: "require_osd_release": "squid", 2024-09-13T10:11:31.162 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_crimson": false, 2024-09-13T10:11:31.162 INFO:teuthology.orchestra.run.smithi073.stdout: "pools": [ 2024-09-13T10:11:31.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.162 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 1, 2024-09-13T10:11:31.162 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "rbd", 2024-09-13T10:11:31.162 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:28.924860+0000", 2024-09-13T10:11:31.162 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-13T10:11:31.162 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T10:11:31.162 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:11:31.163 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:11:31.163 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:11:31.163 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:11:31.163 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:11:31.163 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:11:31.163 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:11:31.163 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:11:31.163 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:11:31.163 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:11:31.163 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:11:31.163 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 8, 2024-09-13T10:11:31.163 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 8, 2024-09-13T10:11:31.163 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 8, 2024-09-13T10:11:31.164 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 8, 2024-09-13T10:11:31.164 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 8, 2024-09-13T10:11:31.164 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:11:31.164 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:11:31.164 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:11:31.164 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:11:31.164 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:11:31.164 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:11:31.164 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:11:31.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.164 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "27", 2024-09-13T10:11:31.164 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:11:31.164 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:11:31.165 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:11:31.165 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:11:31.165 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:11:31.165 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 2, 2024-09-13T10:11:31.165 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 27, 2024-09-13T10:11:31.165 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:11:31.165 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:11:31.165 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:11:31.165 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:11:31.165 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:11:31.165 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:11:31.165 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:11:31.166 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:11:31.166 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:11:31.166 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:11:31.166 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:11:31.166 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:11:31.166 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:11:31.166 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:11:31.166 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:11:31.166 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:11:31.166 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:11:31.166 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:11:31.166 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:11:31.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.166 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:11:31.167 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:11:31.167 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:11:31.167 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:11:31.167 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:11:31.167 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:11:31.167 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:11:31.167 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:11:31.167 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:11:31.167 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:11:31.167 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:11:31.167 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-13T10:11:31.167 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-13T10:11:31.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-13T10:11:31.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.168 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:11:31.168 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:11:31.168 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 3.7699999809265137, 2024-09-13T10:11:31.168 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 3.7699999809265137, 2024-09-13T10:11:31.168 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.73000001907348633, 2024-09-13T10:11:31.168 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2.75, 2024-09-13T10:11:31.168 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2.75, 2024-09-13T10:11:31.168 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8997650146484375, 2024-09-13T10:11:31.168 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:11:31.168 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-13T10:11:31.168 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.169 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-13T10:11:31.169 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": ".mgr", 2024-09-13T10:11:31.169 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:28.972519+0000", 2024-09-13T10:11:31.169 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-13T10:11:31.169 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-13T10:11:31.169 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:11:31.169 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:11:31.169 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:11:31.169 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:11:31.169 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:11:31.169 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:11:31.169 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:11:31.170 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:11:31.170 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:11:31.170 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:11:31.170 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:11:31.170 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 1, 2024-09-13T10:11:31.170 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 1, 2024-09-13T10:11:31.170 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 1, 2024-09-13T10:11:31.170 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 1, 2024-09-13T10:11:31.170 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 1, 2024-09-13T10:11:31.170 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:11:31.170 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:11:31.170 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:11:31.170 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:11:31.171 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:11:31.171 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:11:31.171 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:11:31.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.171 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "26", 2024-09-13T10:11:31.171 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:11:31.171 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:11:31.171 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:11:31.171 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:11:31.171 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:11:31.171 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-13T10:11:31.171 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-13T10:11:31.171 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:11:31.171 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:11:31.172 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:11:31.172 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:11:31.172 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:11:31.172 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:11:31.172 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:11:31.172 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:11:31.172 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:11:31.172 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:11:31.173 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:11:31.173 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:11:31.173 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:11:31.173 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:11:31.173 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:11:31.173 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:11:31.173 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:11:31.173 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:11:31.173 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:11:31.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.173 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:11:31.173 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:11:31.174 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:11:31.174 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:11:31.174 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:11:31.174 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:11:31.174 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:11:31.174 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:11:31.174 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:11:31.174 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:11:31.174 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:11:31.174 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-13T10:11:31.174 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_max": 32, 2024-09-13T10:11:31.174 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_min": 1 2024-09-13T10:11:31.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.175 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-13T10:11:31.175 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": {} 2024-09-13T10:11:31.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.175 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:11:31.175 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:11:31.175 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 15, 2024-09-13T10:11:31.175 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 15, 2024-09-13T10:11:31.175 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.20000000298023224, 2024-09-13T10:11:31.175 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 3, 2024-09-13T10:11:31.175 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 3, 2024-09-13T10:11:31.175 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8997650146484375, 2024-09-13T10:11:31.175 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:11:31.176 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-13T10:11:31.176 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.176 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-13T10:11:31.176 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "low_tier", 2024-09-13T10:11:31.176 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:41.320719+0000", 2024-09-13T10:11:31.176 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-13T10:11:31.176 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-13T10:11:31.176 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:11:31.176 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:11:31.176 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:11:31.177 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:11:31.177 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:11:31.177 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:11:31.177 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:11:31.177 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:11:31.177 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:11:31.177 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:11:31.177 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:11:31.177 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 4, 2024-09-13T10:11:31.177 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 4, 2024-09-13T10:11:31.177 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 4, 2024-09-13T10:11:31.177 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 4, 2024-09-13T10:11:31.177 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 4, 2024-09-13T10:11:31.178 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:11:31.178 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:11:31.178 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:11:31.178 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:11:31.178 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:11:31.178 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:11:31.178 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:11:31.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.178 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "32", 2024-09-13T10:11:31.178 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:11:31.178 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:11:31.178 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:11:31.178 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:11:31.179 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:11:31.179 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-13T10:11:31.179 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-13T10:11:31.179 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:11:31.179 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:11:31.179 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:11:31.179 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:11:31.179 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:11:31.179 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:11:31.179 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:11:31.179 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:11:31.179 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:11:31.179 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:11:31.180 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:11:31.180 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:11:31.180 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:11:31.180 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:11:31.180 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:11:31.180 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:11:31.180 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:11:31.180 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:11:31.180 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:11:31.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.180 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:11:31.180 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:11:31.180 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:11:31.181 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:11:31.181 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:11:31.181 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:11:31.181 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:11:31.181 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:11:31.181 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:11:31.181 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:11:31.181 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:11:31.181 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-13T10:11:31.181 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": {}, 2024-09-13T10:11:31.181 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:11:31.181 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:11:31.181 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 3.75, 2024-09-13T10:11:31.182 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 3.75, 2024-09-13T10:11:31.182 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.60000002384185791, 2024-09-13T10:11:31.182 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2.25, 2024-09-13T10:11:31.182 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2.25, 2024-09-13T10:11:31.182 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8997650146484375, 2024-09-13T10:11:31.182 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:11:31.182 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-13T10:11:31.182 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.182 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 4, 2024-09-13T10:11:31.182 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "unique_pool_0", 2024-09-13T10:11:31.182 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:42.575395+0000", 2024-09-13T10:11:31.182 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-13T10:11:31.183 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-13T10:11:31.183 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:11:31.183 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:11:31.183 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:11:31.183 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:11:31.183 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:11:31.183 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:11:31.183 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:11:31.183 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:11:31.183 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:11:31.183 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:11:31.183 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:11:31.183 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 26, 2024-09-13T10:11:31.183 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 24, 2024-09-13T10:11:31.184 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 26, 2024-09-13T10:11:31.184 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 26, 2024-09-13T10:11:31.184 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 26, 2024-09-13T10:11:31.184 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:11:31.184 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:11:31.184 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:11:31.184 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:11:31.184 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:11:31.184 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:11:31.184 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:11:31.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.184 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "70", 2024-09-13T10:11:31.184 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:11:31.184 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:11:31.185 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "41", 2024-09-13T10:11:31.185 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:11:31.185 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:11:31.185 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-13T10:11:31.185 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-13T10:11:31.185 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:11:31.185 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:11:31.185 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:11:31.185 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:11:31.185 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:11:31.185 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:11:31.185 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:11:31.185 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:11:31.186 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:11:31.186 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:11:31.186 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:11:31.186 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:11:31.186 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:11:31.186 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:11:31.186 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:11:31.186 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:11:31.186 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:11:31.186 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:11:31.186 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:11:31.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.187 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:11:31.187 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:11:31.187 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:11:31.187 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:11:31.187 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:11:31.187 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:11:31.187 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:11:31.187 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:11:31.187 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:11:31.187 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:11:31.187 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:11:31.187 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-13T10:11:31.188 INFO:teuthology.orchestra.run.smithi073.stdout: "dedup_cdc_chunk_size": 131072, 2024-09-13T10:11:31.188 INFO:teuthology.orchestra.run.smithi073.stdout: "dedup_chunk_algorithm": "fastcdc", 2024-09-13T10:11:31.188 INFO:teuthology.orchestra.run.smithi073.stdout: "dedup_tier": 3, 2024-09-13T10:11:31.188 INFO:teuthology.orchestra.run.smithi073.stdout: "fingerprint_algorithm": "sha256" 2024-09-13T10:11:31.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.188 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-13T10:11:31.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rados": {} 2024-09-13T10:11:31.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.188 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:11:31.188 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:11:31.188 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 4.0300002098083496, 2024-09-13T10:11:31.188 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 3.4500000476837158, 2024-09-13T10:11:31.189 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 1.0700000524520874, 2024-09-13T10:11:31.189 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 4.309999942779541, 2024-09-13T10:11:31.189 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 3.690000057220459, 2024-09-13T10:11:31.189 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8997650146484375, 2024-09-13T10:11:31.189 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:11:31.189 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.89999997615814209 2024-09-13T10:11:31.189 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.189 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.189 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:31.189 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:31.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.189 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-13T10:11:31.189 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "f7d92188-c27c-4744-a330-4b7105e0f2d6", 2024-09-13T10:11:31.190 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:31.190 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:31.190 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:31.190 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:31.190 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:31.190 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:31.190 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:31.190 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 0, 2024-09-13T10:11:31.190 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:31.190 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:31.190 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:31.190 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.191 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.191 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6812", 2024-09-13T10:11:31.191 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:11:31.191 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.191 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.191 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:31.191 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.191 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.191 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6813", 2024-09-13T10:11:31.191 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:11:31.192 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.192 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.192 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:31.192 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.192 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.192 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6815", 2024-09-13T10:11:31.192 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:11:31.192 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.192 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.192 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:31.193 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.193 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.193 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6814", 2024-09-13T10:11:31.193 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:11:31.193 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.193 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.193 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6812/1977392321", 2024-09-13T10:11:31.193 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6813/1977392321", 2024-09-13T10:11:31.193 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6815/1977392321", 2024-09-13T10:11:31.193 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6814/1977392321", 2024-09-13T10:11:31.193 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:31.194 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:31.194 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:31.194 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.194 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-13T10:11:31.194 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e900b771-7c96-4b5d-93d7-35a8ea6c026a", 2024-09-13T10:11:31.194 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:31.194 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:31.194 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:31.194 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:31.194 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:31.194 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:31.195 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:31.195 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 65, 2024-09-13T10:11:31.195 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:31.195 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:31.195 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:31.195 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.195 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.195 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6808", 2024-09-13T10:11:31.195 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:11:31.195 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.195 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.196 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:31.196 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.196 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.196 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6809", 2024-09-13T10:11:31.196 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:11:31.196 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.196 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.196 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:31.196 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.197 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.197 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6811", 2024-09-13T10:11:31.197 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:11:31.197 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.197 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.197 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:31.197 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.197 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.197 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6810", 2024-09-13T10:11:31.197 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:11:31.197 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.198 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.198 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6808/1228102864", 2024-09-13T10:11:31.198 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6809/1228102864", 2024-09-13T10:11:31.198 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6811/1228102864", 2024-09-13T10:11:31.198 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6810/1228102864", 2024-09-13T10:11:31.198 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:31.198 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:31.198 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:31.198 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.198 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-13T10:11:31.199 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "28b7aa59-33b2-4e24-8f3d-758f90e25f68", 2024-09-13T10:11:31.199 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:31.199 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:31.199 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:31.199 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:31.199 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:31.199 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:31.199 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:31.199 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 61, 2024-09-13T10:11:31.199 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:31.199 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:31.199 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:31.199 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.200 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.200 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6800", 2024-09-13T10:11:31.200 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:11:31.200 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.200 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.200 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:31.200 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.200 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.200 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6801", 2024-09-13T10:11:31.200 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:11:31.201 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.201 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.201 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:31.201 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.201 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.201 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6803", 2024-09-13T10:11:31.201 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:11:31.201 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.201 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.202 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:31.202 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.202 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.202 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6802", 2024-09-13T10:11:31.202 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:11:31.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.202 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.202 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6800/3314820936", 2024-09-13T10:11:31.202 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6801/3314820936", 2024-09-13T10:11:31.202 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6803/3314820936", 2024-09-13T10:11:31.202 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6802/3314820936", 2024-09-13T10:11:31.202 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:31.203 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:31.203 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:31.203 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.203 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-13T10:11:31.203 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "a93d2df5-781e-49e1-a507-1718b6f954ed", 2024-09-13T10:11:31.203 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:31.203 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:31.203 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:31.203 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:31.203 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:31.203 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:31.204 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:31.204 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 60, 2024-09-13T10:11:31.204 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:31.204 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:31.204 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:31.204 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.204 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.204 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6801", 2024-09-13T10:11:31.204 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:11:31.204 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.204 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.205 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:31.205 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.206 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.206 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6802", 2024-09-13T10:11:31.206 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:11:31.206 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.206 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.206 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:31.206 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.206 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.206 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6804", 2024-09-13T10:11:31.206 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:11:31.207 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.207 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.207 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:31.207 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.207 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.207 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6803", 2024-09-13T10:11:31.207 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:11:31.207 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.207 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.207 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6801/2558565460", 2024-09-13T10:11:31.208 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6802/2558565460", 2024-09-13T10:11:31.208 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6804/2558565460", 2024-09-13T10:11:31.208 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6803/2558565460", 2024-09-13T10:11:31.208 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:31.208 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:31.208 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:31.208 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.208 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-13T10:11:31.208 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e69053fd-f6d9-437e-a9d0-8896de6b927f", 2024-09-13T10:11:31.208 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:31.208 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:31.209 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:31.209 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:31.209 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:31.209 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:31.209 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:31.209 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 67, 2024-09-13T10:11:31.209 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:31.209 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:31.209 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:31.209 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.209 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.209 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6800", 2024-09-13T10:11:31.210 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:11:31.210 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.210 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.210 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:31.210 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.210 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.210 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6801", 2024-09-13T10:11:31.210 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:11:31.210 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.210 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.210 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:31.211 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.211 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.211 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6803", 2024-09-13T10:11:31.211 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:11:31.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.211 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.211 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:31.211 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.211 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.211 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6802", 2024-09-13T10:11:31.212 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:11:31.212 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.212 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.212 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6800/858330493", 2024-09-13T10:11:31.212 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6801/858330493", 2024-09-13T10:11:31.212 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6803/858330493", 2024-09-13T10:11:31.212 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6802/858330493", 2024-09-13T10:11:31.212 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:31.212 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:31.212 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:31.212 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.213 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-13T10:11:31.213 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "3667ec10-561e-4a23-8197-6357db4af315", 2024-09-13T10:11:31.213 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:31.213 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:31.213 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:31.213 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:31.213 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:31.213 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:31.213 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:31.213 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 58, 2024-09-13T10:11:31.213 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:31.213 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:31.214 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:31.214 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.214 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.214 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6804", 2024-09-13T10:11:31.214 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:11:31.214 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.214 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.214 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:31.214 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.214 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.214 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6805", 2024-09-13T10:11:31.215 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:11:31.215 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.215 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.215 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:31.215 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.215 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.215 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6807", 2024-09-13T10:11:31.215 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:11:31.215 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.215 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.216 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:31.216 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.216 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.216 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6806", 2024-09-13T10:11:31.216 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:11:31.216 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.216 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.216 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6804/984313481", 2024-09-13T10:11:31.216 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6805/984313481", 2024-09-13T10:11:31.216 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6807/984313481", 2024-09-13T10:11:31.216 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6806/984313481", 2024-09-13T10:11:31.217 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:31.217 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:31.217 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:31.217 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.217 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-13T10:11:31.217 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "30d4d63c-f04f-4423-91a6-816a7cbc0287", 2024-09-13T10:11:31.217 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:31.217 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:31.217 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:31.217 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:31.217 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:31.218 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:31.218 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:31.218 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 64, 2024-09-13T10:11:31.218 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:31.218 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:31.218 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:31.218 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.218 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.218 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6812", 2024-09-13T10:11:31.218 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:11:31.218 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.218 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.219 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:31.219 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.219 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.219 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6813", 2024-09-13T10:11:31.219 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:11:31.219 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.219 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.219 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:31.219 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.220 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.220 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6815", 2024-09-13T10:11:31.220 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:11:31.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.220 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.220 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:31.220 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.220 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.220 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6814", 2024-09-13T10:11:31.220 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:11:31.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.221 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.221 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6812/1505939978", 2024-09-13T10:11:31.221 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6813/1505939978", 2024-09-13T10:11:31.221 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6815/1505939978", 2024-09-13T10:11:31.221 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6814/1505939978", 2024-09-13T10:11:31.221 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:31.221 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:31.221 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:31.221 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.221 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-13T10:11:31.221 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "48812f05-82f1-4263-8ec5-4861a6e91b61", 2024-09-13T10:11:31.222 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:31.222 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:31.222 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:31.222 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:31.222 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:31.222 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:31.222 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:31.222 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 41, 2024-09-13T10:11:31.222 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:31.223 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:31.223 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:31.223 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.223 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.223 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6809", 2024-09-13T10:11:31.223 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:11:31.223 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.223 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.223 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:31.223 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.224 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.224 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6810", 2024-09-13T10:11:31.224 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:11:31.224 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.224 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.224 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:31.224 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.224 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.224 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6812", 2024-09-13T10:11:31.224 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:11:31.225 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.225 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.225 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:31.225 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.225 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.225 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6811", 2024-09-13T10:11:31.225 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:11:31.225 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.225 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.225 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6809/3679292757", 2024-09-13T10:11:31.226 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6810/3679292757", 2024-09-13T10:11:31.226 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6812/3679292757", 2024-09-13T10:11:31.226 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6811/3679292757", 2024-09-13T10:11:31.226 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:31.226 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:31.226 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:31.226 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.226 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 8, 2024-09-13T10:11:31.226 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "72056e61-d04b-4fd3-94bf-57ee3df4b466", 2024-09-13T10:11:31.226 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:31.226 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:31.226 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:31.227 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0.396240234375, 2024-09-13T10:11:31.227 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:31.227 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:31.227 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:31.227 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 31, 2024-09-13T10:11:31.227 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:31.227 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:31.227 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:31.227 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.227 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.227 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6804", 2024-09-13T10:11:31.227 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:11:31.228 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.228 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.228 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:31.228 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.228 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.228 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6805", 2024-09-13T10:11:31.228 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:11:31.228 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.228 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.228 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:31.229 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.229 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.229 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6807", 2024-09-13T10:11:31.229 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:11:31.229 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.229 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.229 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:31.229 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.229 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.229 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6806", 2024-09-13T10:11:31.230 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:11:31.230 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.230 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.230 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6804/4052106885", 2024-09-13T10:11:31.230 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6805/4052106885", 2024-09-13T10:11:31.230 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6807/4052106885", 2024-09-13T10:11:31.230 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6806/4052106885", 2024-09-13T10:11:31.230 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:31.230 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:31.230 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:31.230 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.231 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 9, 2024-09-13T10:11:31.231 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "20e7f118-7148-4c27-9ec4-a33954709515", 2024-09-13T10:11:31.231 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:31.231 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:31.231 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:31.231 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:31.231 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:31.231 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:31.231 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 21, 2024-09-13T10:11:31.231 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 48, 2024-09-13T10:11:31.231 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:31.231 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:31.231 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:31.231 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.232 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.232 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6800", 2024-09-13T10:11:31.232 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:11:31.232 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.232 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.232 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:31.232 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.232 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.232 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6801", 2024-09-13T10:11:31.232 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:11:31.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.233 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.233 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:31.233 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.233 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.233 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6803", 2024-09-13T10:11:31.234 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:11:31.234 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.234 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.234 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:31.234 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.234 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.234 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6802", 2024-09-13T10:11:31.234 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:11:31.234 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.234 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.234 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6800/4266571820", 2024-09-13T10:11:31.235 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6801/4266571820", 2024-09-13T10:11:31.235 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6803/4266571820", 2024-09-13T10:11:31.235 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6802/4266571820", 2024-09-13T10:11:31.235 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:31.235 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:31.235 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:31.235 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.235 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 10, 2024-09-13T10:11:31.235 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "012409e8-51c2-4584-aba7-95996f0d72ad", 2024-09-13T10:11:31.235 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:31.235 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:31.235 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:31.236 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:31.236 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:31.236 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:31.236 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:31.236 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 30, 2024-09-13T10:11:31.236 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:31.236 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:31.236 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:31.236 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.236 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.236 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6808", 2024-09-13T10:11:31.236 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:11:31.237 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.237 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.237 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:31.237 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.237 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.237 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6809", 2024-09-13T10:11:31.237 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:11:31.237 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.237 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.237 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:31.238 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.238 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.238 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6811", 2024-09-13T10:11:31.238 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:11:31.238 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.238 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.238 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:31.238 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.238 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.238 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6810", 2024-09-13T10:11:31.239 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:11:31.239 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.239 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.239 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6808/958255259", 2024-09-13T10:11:31.239 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6809/958255259", 2024-09-13T10:11:31.239 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6811/958255259", 2024-09-13T10:11:31.239 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6810/958255259", 2024-09-13T10:11:31.239 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:31.239 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:31.239 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:31.239 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.240 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 11, 2024-09-13T10:11:31.240 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "88b137b1-7d30-45f7-ac5c-ba026704ef24", 2024-09-13T10:11:31.240 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:31.240 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:31.240 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:31.240 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:31.240 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:31.240 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:31.240 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:31.240 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 57, 2024-09-13T10:11:31.240 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:31.240 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:31.241 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:31.241 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.241 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.241 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6813", 2024-09-13T10:11:31.241 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:11:31.241 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.241 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.241 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:31.241 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.241 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.241 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6814", 2024-09-13T10:11:31.242 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:11:31.242 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.242 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.242 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:31.242 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.242 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.242 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6816", 2024-09-13T10:11:31.242 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:11:31.242 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.242 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.242 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:31.243 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.243 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.243 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6815", 2024-09-13T10:11:31.243 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:11:31.243 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.243 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.243 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6813/281884267", 2024-09-13T10:11:31.243 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6814/281884267", 2024-09-13T10:11:31.243 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6816/281884267", 2024-09-13T10:11:31.243 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6815/281884267", 2024-09-13T10:11:31.243 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:31.244 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:31.244 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:31.244 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.244 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 12, 2024-09-13T10:11:31.244 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "b54732a2-9dff-42e1-b4ef-281cbca3ad7c", 2024-09-13T10:11:31.244 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:31.244 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:31.244 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:31.244 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0, 2024-09-13T10:11:31.244 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:31.245 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:31.245 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:31.245 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 31, 2024-09-13T10:11:31.245 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:31.245 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:31.245 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:31.245 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.245 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.245 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6808", 2024-09-13T10:11:31.245 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:11:31.245 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.245 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.246 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:31.246 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.246 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.246 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6809", 2024-09-13T10:11:31.246 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:11:31.246 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.246 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.246 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:31.246 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.247 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.247 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6811", 2024-09-13T10:11:31.247 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:11:31.247 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.247 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.247 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:31.247 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.247 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.247 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6810", 2024-09-13T10:11:31.247 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:11:31.247 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.247 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.248 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6808/3532043806", 2024-09-13T10:11:31.248 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6809/3532043806", 2024-09-13T10:11:31.248 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6811/3532043806", 2024-09-13T10:11:31.248 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6810/3532043806", 2024-09-13T10:11:31.248 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:31.248 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:31.248 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:31.248 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.248 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 13, 2024-09-13T10:11:31.249 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "20b48864-44dc-4732-8dff-d0d21cc08032", 2024-09-13T10:11:31.249 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:31.249 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:31.249 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:31.249 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:31.249 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:31.249 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:31.249 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 21, 2024-09-13T10:11:31.249 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 53, 2024-09-13T10:11:31.249 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:31.249 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:31.249 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:31.249 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.250 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.250 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6812", 2024-09-13T10:11:31.250 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:11:31.250 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.250 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.250 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:31.250 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.250 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.250 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6813", 2024-09-13T10:11:31.250 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:11:31.251 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.251 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.251 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:31.251 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.251 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.251 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6815", 2024-09-13T10:11:31.251 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:11:31.251 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.251 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.251 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:31.252 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.252 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.252 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6814", 2024-09-13T10:11:31.252 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:11:31.252 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.252 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.252 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6812/4083982131", 2024-09-13T10:11:31.252 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6813/4083982131", 2024-09-13T10:11:31.252 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6815/4083982131", 2024-09-13T10:11:31.252 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6814/4083982131", 2024-09-13T10:11:31.252 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:31.253 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:31.253 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:31.253 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.253 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 14, 2024-09-13T10:11:31.253 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "9fac3ef4-e9be-4e8b-9a21-0de35b543d65", 2024-09-13T10:11:31.253 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:31.253 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:31.253 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:31.253 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:31.253 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:31.253 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:31.253 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:31.254 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 66, 2024-09-13T10:11:31.254 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:31.254 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:31.254 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:31.254 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.254 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.254 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6804", 2024-09-13T10:11:31.254 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:11:31.254 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.254 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.254 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:31.255 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.255 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.255 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6805", 2024-09-13T10:11:31.255 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:11:31.255 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.255 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.255 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:31.255 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.256 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.256 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6807", 2024-09-13T10:11:31.256 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:11:31.256 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.256 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.256 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:31.256 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.256 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.257 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6806", 2024-09-13T10:11:31.257 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:11:31.257 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.257 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.257 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6804/1173379175", 2024-09-13T10:11:31.257 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6805/1173379175", 2024-09-13T10:11:31.257 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6807/1173379175", 2024-09-13T10:11:31.257 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6806/1173379175", 2024-09-13T10:11:31.257 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:31.257 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:31.257 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:31.258 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.258 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 15, 2024-09-13T10:11:31.258 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "16b34f90-3148-4d41-a403-ba0f900991e1", 2024-09-13T10:11:31.258 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:31.258 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:31.258 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:31.258 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:31.258 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:31.258 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:31.258 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:31.259 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 44, 2024-09-13T10:11:31.259 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:31.259 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:31.259 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:31.259 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.259 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.259 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6805", 2024-09-13T10:11:31.259 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:11:31.259 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.259 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.259 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:31.260 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.260 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.260 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6806", 2024-09-13T10:11:31.260 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:11:31.260 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.260 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.260 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:31.260 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.260 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.260 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6808", 2024-09-13T10:11:31.261 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:11:31.261 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.261 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.261 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:31.261 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:31.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.261 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:31.261 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6807", 2024-09-13T10:11:31.261 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:11:31.261 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.261 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.262 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6805/848686366", 2024-09-13T10:11:31.262 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6806/848686366", 2024-09-13T10:11:31.262 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6808/848686366", 2024-09-13T10:11:31.262 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6807/848686366", 2024-09-13T10:11:31.262 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:31.262 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:31.262 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:31.262 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.262 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.262 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:31.262 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_xinfo": [ 2024-09-13T10:11:31.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.262 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-13T10:11:31.263 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:31.263 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:31.263 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:31.263 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:31.263 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:31.263 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.372908+0000", 2024-09-13T10:11:31.263 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:31.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.263 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-13T10:11:31.263 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:31.263 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:31.263 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:31.263 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:31.264 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:31.264 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.641263+0000", 2024-09-13T10:11:31.264 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:31.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.264 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-13T10:11:31.264 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:31.264 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:31.264 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:31.264 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:31.264 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:31.264 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.237394+0000", 2024-09-13T10:11:31.264 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:31.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.265 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-13T10:11:31.265 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:31.265 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:31.265 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:31.265 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:31.265 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:31.265 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:24.991965+0000", 2024-09-13T10:11:31.265 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:31.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.265 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-13T10:11:31.265 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:31.266 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:31.266 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:31.266 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:31.266 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:31.266 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.066350+0000", 2024-09-13T10:11:31.266 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:31.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.266 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-13T10:11:31.266 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:31.266 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:31.266 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:31.266 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:31.267 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:31.267 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.003317+0000", 2024-09-13T10:11:31.267 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:31.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.267 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-13T10:11:31.267 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:31.267 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:31.267 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:31.267 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:31.267 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:31.267 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.353521+0000", 2024-09-13T10:11:31.267 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:31.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.268 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-13T10:11:31.268 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:31.268 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:31.268 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:31.268 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:31.268 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:31.268 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.263941+0000", 2024-09-13T10:11:31.268 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:31.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.268 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 8, 2024-09-13T10:11:31.269 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:31.269 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:31.269 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:31.269 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:31.269 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:31.269 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:24.922976+0000", 2024-09-13T10:11:31.269 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:31.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.269 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 9, 2024-09-13T10:11:31.269 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:31.269 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:31.269 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:31.270 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:31.270 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:31.270 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:25.726157+0000", 2024-09-13T10:11:31.270 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:31.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.270 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 10, 2024-09-13T10:11:31.270 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:31.270 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:31.270 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:31.270 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:31.270 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:31.270 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:27.329800+0000", 2024-09-13T10:11:31.271 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:31.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.271 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 11, 2024-09-13T10:11:31.271 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:31.271 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:31.271 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:31.271 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:31.271 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:31.271 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:26.722524+0000", 2024-09-13T10:11:31.271 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:31.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.271 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 12, 2024-09-13T10:11:31.272 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:31.272 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:31.272 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:31.272 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:31.272 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:31.272 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:25.954484+0000", 2024-09-13T10:11:31.272 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:31.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.272 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 13, 2024-09-13T10:11:31.272 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:31.272 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:31.272 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:31.273 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:31.273 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:31.273 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:25.900950+0000", 2024-09-13T10:11:31.273 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:31.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.273 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 14, 2024-09-13T10:11:31.273 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:31.273 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:31.273 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:31.273 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:31.273 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:31.274 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:24.980077+0000", 2024-09-13T10:11:31.274 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:31.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.274 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 15, 2024-09-13T10:11:31.274 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:31.274 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:31.274 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:31.274 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:31.274 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:31.274 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.070162+0000", 2024-09-13T10:11:31.274 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:31.274 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.274 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:31.275 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap": [ 2024-09-13T10:11:31.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.275 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "1.3", 2024-09-13T10:11:31.275 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:31.275 INFO:teuthology.orchestra.run.smithi073.stdout: 4, 2024-09-13T10:11:31.275 INFO:teuthology.orchestra.run.smithi073.stdout: 9, 2024-09-13T10:11:31.275 INFO:teuthology.orchestra.run.smithi073.stdout: 8 2024-09-13T10:11:31.275 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.275 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.e", 2024-09-13T10:11:31.275 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:31.275 INFO:teuthology.orchestra.run.smithi073.stdout: 8, 2024-09-13T10:11:31.276 INFO:teuthology.orchestra.run.smithi073.stdout: 3, 2024-09-13T10:11:31.276 INFO:teuthology.orchestra.run.smithi073.stdout: 15 2024-09-13T10:11:31.276 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.276 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.276 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:31.276 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_items": [], 2024-09-13T10:11:31.276 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_primaries": [], 2024-09-13T10:11:31.276 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_temp": [ 2024-09-13T10:11:31.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.276 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.e", 2024-09-13T10:11:31.276 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:31.276 INFO:teuthology.orchestra.run.smithi073.stdout: 2, 2024-09-13T10:11:31.276 INFO:teuthology.orchestra.run.smithi073.stdout: 8, 2024-09-13T10:11:31.277 INFO:teuthology.orchestra.run.smithi073.stdout: 4 2024-09-13T10:11:31.277 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.277 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.10", 2024-09-13T10:11:31.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:31.277 INFO:teuthology.orchestra.run.smithi073.stdout: 4, 2024-09-13T10:11:31.277 INFO:teuthology.orchestra.run.smithi073.stdout: 5, 2024-09-13T10:11:31.277 INFO:teuthology.orchestra.run.smithi073.stdout: 6 2024-09-13T10:11:31.277 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.277 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.11", 2024-09-13T10:11:31.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:31.278 INFO:teuthology.orchestra.run.smithi073.stdout: 2, 2024-09-13T10:11:31.278 INFO:teuthology.orchestra.run.smithi073.stdout: 12, 2024-09-13T10:11:31.278 INFO:teuthology.orchestra.run.smithi073.stdout: 8 2024-09-13T10:11:31.278 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.280 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.12", 2024-09-13T10:11:31.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:31.280 INFO:teuthology.orchestra.run.smithi073.stdout: 6, 2024-09-13T10:11:31.280 INFO:teuthology.orchestra.run.smithi073.stdout: 8, 2024-09-13T10:11:31.280 INFO:teuthology.orchestra.run.smithi073.stdout: 14 2024-09-13T10:11:31.280 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:31.281 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.14", 2024-09-13T10:11:31.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:31.281 INFO:teuthology.orchestra.run.smithi073.stdout: 5, 2024-09-13T10:11:31.281 INFO:teuthology.orchestra.run.smithi073.stdout: 6, 2024-09-13T10:11:31.281 INFO:teuthology.orchestra.run.smithi073.stdout: 10 2024-09-13T10:11:31.281 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:31.281 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.281 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:31.281 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_temp": [], 2024-09-13T10:11:31.281 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklist": {}, 2024-09-13T10:11:31.281 INFO:teuthology.orchestra.run.smithi073.stdout: "range_blocklist": {}, 2024-09-13T10:11:31.282 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profiles": { 2024-09-13T10:11:31.282 INFO:teuthology.orchestra.run.smithi073.stdout: "default": { 2024-09-13T10:11:31.282 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-failure-domain": "osd", 2024-09-13T10:11:31.282 INFO:teuthology.orchestra.run.smithi073.stdout: "k": "2", 2024-09-13T10:11:31.282 INFO:teuthology.orchestra.run.smithi073.stdout: "m": "1", 2024-09-13T10:11:31.282 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin": "jerasure", 2024-09-13T10:11:31.282 INFO:teuthology.orchestra.run.smithi073.stdout: "technique": "reed_sol_van" 2024-09-13T10:11:31.282 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:31.282 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps_queue": [], 2024-09-13T10:11:31.282 INFO:teuthology.orchestra.run.smithi073.stdout: "new_removed_snaps": [], 2024-09-13T10:11:31.282 INFO:teuthology.orchestra.run.smithi073.stdout: "new_purged_snaps": [], 2024-09-13T10:11:31.282 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_node_flags": {}, 2024-09-13T10:11:31.282 INFO:teuthology.orchestra.run.smithi073.stdout: "device_class_flags": {}, 2024-09-13T10:11:31.283 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode": { 2024-09-13T10:11:31.283 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_enabled": false, 2024-09-13T10:11:31.283 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_bucket_count": 0, 2024-09-13T10:11:31.283 INFO:teuthology.orchestra.run.smithi073.stdout: "degraded_stretch_mode": 0, 2024-09-13T10:11:31.283 INFO:teuthology.orchestra.run.smithi073.stdout: "recovering_stretch_mode": 0, 2024-09-13T10:11:31.283 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_bucket": 0 2024-09-13T10:11:31.283 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:31.283 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:31.286 INFO:tasks.thrashosds.thrasher:j is {'epoch': 70, 'fsid': 'c1e91d47-9492-4461-a043-8698fd74ccf3', 'created': '2024-09-13T10:10:14.019912+0000', 'modified': '2024-09-13T10:11:31.001317+0000', 'last_up_change': '2024-09-13T10:10:28.089885+0000', 'last_in_change': '2024-09-13T10:10:18.489546+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T10:10:28.924860+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 27, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.7699999809265137, 'score_stable': 3.7699999809265137, 'optimal_score': 0.7300000190734863, 'raw_score_acting': 2.75, 'raw_score_stable': 2.75, 'primary_affinity_weighted': 0.8997650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T10:10:28.972519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 15, 'score_stable': 15, 'optimal_score': 0.20000000298023224, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.8997650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 3, 'pool_name': 'low_tier', 'create_time': '2024-09-13T10:10:41.320719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.75, 'score_stable': 3.75, 'optimal_score': 0.6000000238418579, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8997650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8999999761581421}}, {'pool': 4, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-13T10:10:42.575395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 24, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '41', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'dedup_cdc_chunk_size': 131072, 'dedup_chunk_algorithm': 'fastcdc', 'dedup_tier': 3, 'fingerprint_algorithm': 'sha256'}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.03000020980835, 'score_stable': 3.450000047683716, 'optimal_score': 1.0700000524520874, 'raw_score_acting': 4.309999942779541, 'raw_score_stable': 3.690000057220459, 'primary_affinity_weighted': 0.8997650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8999999761581421}}], 'osds': [{'osd': 0, 'uuid': 'f7d92188-c27c-4744-a330-4b7105e0f2d6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6812', 'nonce': 1977392321}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6813', 'nonce': 1977392321}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6815', 'nonce': 1977392321}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6814', 'nonce': 1977392321}]}, 'public_addr': '172.21.15.73:6812/1977392321', 'cluster_addr': '172.21.15.73:6813/1977392321', 'heartbeat_back_addr': '172.21.15.73:6815/1977392321', 'heartbeat_front_addr': '172.21.15.73:6814/1977392321', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e900b771-7c96-4b5d-93d7-35a8ea6c026a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 65, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6808', 'nonce': 1228102864}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6809', 'nonce': 1228102864}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6811', 'nonce': 1228102864}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6810', 'nonce': 1228102864}]}, 'public_addr': '172.21.15.136:6808/1228102864', 'cluster_addr': '172.21.15.136:6809/1228102864', 'heartbeat_back_addr': '172.21.15.136:6811/1228102864', 'heartbeat_front_addr': '172.21.15.136:6810/1228102864', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '28b7aa59-33b2-4e24-8f3d-758f90e25f68', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6800', 'nonce': 3314820936}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6801', 'nonce': 3314820936}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6803', 'nonce': 3314820936}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6802', 'nonce': 3314820936}]}, 'public_addr': '172.21.15.181:6800/3314820936', 'cluster_addr': '172.21.15.181:6801/3314820936', 'heartbeat_back_addr': '172.21.15.181:6803/3314820936', 'heartbeat_front_addr': '172.21.15.181:6802/3314820936', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a93d2df5-781e-49e1-a507-1718b6f954ed', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 60, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6801', 'nonce': 2558565460}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6802', 'nonce': 2558565460}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6804', 'nonce': 2558565460}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6803', 'nonce': 2558565460}]}, 'public_addr': '172.21.15.195:6801/2558565460', 'cluster_addr': '172.21.15.195:6802/2558565460', 'heartbeat_back_addr': '172.21.15.195:6804/2558565460', 'heartbeat_front_addr': '172.21.15.195:6803/2558565460', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e69053fd-f6d9-437e-a9d0-8896de6b927f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 67, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6800', 'nonce': 858330493}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6801', 'nonce': 858330493}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6803', 'nonce': 858330493}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6802', 'nonce': 858330493}]}, 'public_addr': '172.21.15.73:6800/858330493', 'cluster_addr': '172.21.15.73:6801/858330493', 'heartbeat_back_addr': '172.21.15.73:6803/858330493', 'heartbeat_front_addr': '172.21.15.73:6802/858330493', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '3667ec10-561e-4a23-8197-6357db4af315', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6804', 'nonce': 984313481}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6805', 'nonce': 984313481}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6807', 'nonce': 984313481}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6806', 'nonce': 984313481}]}, 'public_addr': '172.21.15.136:6804/984313481', 'cluster_addr': '172.21.15.136:6805/984313481', 'heartbeat_back_addr': '172.21.15.136:6807/984313481', 'heartbeat_front_addr': '172.21.15.136:6806/984313481', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '30d4d63c-f04f-4423-91a6-816a7cbc0287', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 64, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6812', 'nonce': 1505939978}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6813', 'nonce': 1505939978}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6815', 'nonce': 1505939978}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6814', 'nonce': 1505939978}]}, 'public_addr': '172.21.15.181:6812/1505939978', 'cluster_addr': '172.21.15.181:6813/1505939978', 'heartbeat_back_addr': '172.21.15.181:6815/1505939978', 'heartbeat_front_addr': '172.21.15.181:6814/1505939978', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '48812f05-82f1-4263-8ec5-4861a6e91b61', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6809', 'nonce': 3679292757}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6810', 'nonce': 3679292757}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6812', 'nonce': 3679292757}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6811', 'nonce': 3679292757}]}, 'public_addr': '172.21.15.195:6809/3679292757', 'cluster_addr': '172.21.15.195:6810/3679292757', 'heartbeat_back_addr': '172.21.15.195:6812/3679292757', 'heartbeat_front_addr': '172.21.15.195:6811/3679292757', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '72056e61-d04b-4fd3-94bf-57ee3df4b466', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.396240234375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6804', 'nonce': 4052106885}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6805', 'nonce': 4052106885}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6807', 'nonce': 4052106885}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6806', 'nonce': 4052106885}]}, 'public_addr': '172.21.15.73:6804/4052106885', 'cluster_addr': '172.21.15.73:6805/4052106885', 'heartbeat_back_addr': '172.21.15.73:6807/4052106885', 'heartbeat_front_addr': '172.21.15.73:6806/4052106885', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '20e7f118-7148-4c27-9ec4-a33954709515', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6800', 'nonce': 4266571820}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6801', 'nonce': 4266571820}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6803', 'nonce': 4266571820}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6802', 'nonce': 4266571820}]}, 'public_addr': '172.21.15.136:6800/4266571820', 'cluster_addr': '172.21.15.136:6801/4266571820', 'heartbeat_back_addr': '172.21.15.136:6803/4266571820', 'heartbeat_front_addr': '172.21.15.136:6802/4266571820', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '012409e8-51c2-4584-aba7-95996f0d72ad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6808', 'nonce': 958255259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6809', 'nonce': 958255259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6811', 'nonce': 958255259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6810', 'nonce': 958255259}]}, 'public_addr': '172.21.15.181:6808/958255259', 'cluster_addr': '172.21.15.181:6809/958255259', 'heartbeat_back_addr': '172.21.15.181:6811/958255259', 'heartbeat_front_addr': '172.21.15.181:6810/958255259', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '88b137b1-7d30-45f7-ac5c-ba026704ef24', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 57, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6813', 'nonce': 281884267}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6814', 'nonce': 281884267}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6816', 'nonce': 281884267}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6815', 'nonce': 281884267}]}, 'public_addr': '172.21.15.195:6813/281884267', 'cluster_addr': '172.21.15.195:6814/281884267', 'heartbeat_back_addr': '172.21.15.195:6816/281884267', 'heartbeat_front_addr': '172.21.15.195:6815/281884267', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': 'b54732a2-9dff-42e1-b4ef-281cbca3ad7c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6808', 'nonce': 3532043806}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6809', 'nonce': 3532043806}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6811', 'nonce': 3532043806}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6810', 'nonce': 3532043806}]}, 'public_addr': '172.21.15.73:6808/3532043806', 'cluster_addr': '172.21.15.73:6809/3532043806', 'heartbeat_back_addr': '172.21.15.73:6811/3532043806', 'heartbeat_front_addr': '172.21.15.73:6810/3532043806', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '20b48864-44dc-4732-8dff-d0d21cc08032', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 53, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6812', 'nonce': 4083982131}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6813', 'nonce': 4083982131}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6815', 'nonce': 4083982131}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6814', 'nonce': 4083982131}]}, 'public_addr': '172.21.15.136:6812/4083982131', 'cluster_addr': '172.21.15.136:6813/4083982131', 'heartbeat_back_addr': '172.21.15.136:6815/4083982131', 'heartbeat_front_addr': '172.21.15.136:6814/4083982131', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '9fac3ef4-e9be-4e8b-9a21-0de35b543d65', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6804', 'nonce': 1173379175}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6805', 'nonce': 1173379175}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6807', 'nonce': 1173379175}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6806', 'nonce': 1173379175}]}, 'public_addr': '172.21.15.181:6804/1173379175', 'cluster_addr': '172.21.15.181:6805/1173379175', 'heartbeat_back_addr': '172.21.15.181:6807/1173379175', 'heartbeat_front_addr': '172.21.15.181:6806/1173379175', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '16b34f90-3148-4d41-a403-ba0f900991e1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6805', 'nonce': 848686366}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6806', 'nonce': 848686366}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6808', 'nonce': 848686366}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6807', 'nonce': 848686366}]}, 'public_addr': '172.21.15.195:6805/848686366', 'cluster_addr': '172.21.15.195:6806/848686366', 'heartbeat_back_addr': '172.21.15.195:6808/848686366', 'heartbeat_front_addr': '172.21.15.195:6807/848686366', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.372908+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.641263+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.237394+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:24.991965+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.066350+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.003317+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.353521+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.263941+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:24.922976+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:25.726157+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:27.329800+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:26.722524+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:25.954484+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:25.900950+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:24.980077+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.070162+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.3', 'osds': [4, 9, 8]}, {'pgid': '4.e', 'osds': [8, 3, 15]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '4.e', 'osds': [2, 8, 4]}, {'pgid': '4.10', 'osds': [4, 5, 6]}, {'pgid': '4.11', 'osds': [2, 12, 8]}, {'pgid': '4.12', 'osds': [6, 8, 14]}, {'pgid': '4.14', 'osds': [5, 6, 10]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T10:11:31.287 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 1.3 2024-09-13T10:11:31.287 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 1.3 2024-09-13T10:11:31.288 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:31.288 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:31.289 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-13T10:11:31.289 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2120543 2024-09-13T10:11:31.290 INFO:tasks.rados.rados.0.smithi195.stdout:1266: finishing write tid 2 to smithi19530714-4 2024-09-13T10:11:31.290 INFO:tasks.rados.rados.0.smithi195.stdout:1266: finishing write tid 3 to smithi19530714-4 2024-09-13T10:11:31.290 INFO:tasks.rados.rados.0.smithi195.stdout:1266: finishing write tid 4 to smithi19530714-4 2024-09-13T10:11:31.290 INFO:tasks.rados.rados.0.smithi195.stdout:1266: oid 4 updating version 0 to 87 2024-09-13T10:11:31.290 INFO:tasks.rados.rados.0.smithi195.stdout:1266: oid 4 version 87 is already newer than 86 2024-09-13T10:11:31.290 INFO:tasks.rados.rados.0.smithi195.stdout:1266: oid 4 version 87 is already newer than 85 2024-09-13T10:11:31.290 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 87 (ObjNum 239 snap 0 seq_num 239) dirty exists 2024-09-13T10:11:31.290 INFO:tasks.rados.rados.0.smithi195.stdout:1266: left oid 4 (ObjNum 239 snap 0 seq_num 239) 2024-09-13T10:11:31.290 INFO:tasks.rados.rados.0.smithi195.stdout:1268: finishing write tid 1 to smithi19530714-1 2024-09-13T10:11:31.290 INFO:tasks.rados.rados.0.smithi195.stdout:1268: finishing write tid 2 to smithi19530714-1 2024-09-13T10:11:31.290 INFO:tasks.rados.rados.0.smithi195.stdout:1268: finishing write tid 3 to smithi19530714-1 2024-09-13T10:11:31.290 INFO:tasks.rados.rados.0.smithi195.stdout:1268: finishing write tid 4 to smithi19530714-1 2024-09-13T10:11:31.290 INFO:tasks.rados.rados.0.smithi195.stdout:1268: oid 1 updating version 0 to 74 2024-09-13T10:11:31.291 INFO:tasks.rados.rados.0.smithi195.stdout:1268: oid 1 updating version 74 to 75 2024-09-13T10:11:31.291 INFO:tasks.rados.rados.0.smithi195.stdout:1268: oid 1 version 75 is already newer than 73 2024-09-13T10:11:31.291 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 75 (ObjNum 240 snap 0 seq_num 240) dirty exists 2024-09-13T10:11:31.291 INFO:tasks.rados.rados.0.smithi195.stdout:1268: left oid 1 (ObjNum 240 snap 0 seq_num 240) 2024-09-13T10:11:31.291 INFO:tasks.rados.rados.0.smithi195.stdout:1269: finishing write tid 1 to smithi19530714-39 2024-09-13T10:11:31.291 INFO:tasks.rados.rados.0.smithi195.stdout:1269: finishing write tid 2 to smithi19530714-39 2024-09-13T10:11:31.291 INFO:tasks.rados.rados.0.smithi195.stdout:1269: finishing write tid 3 to smithi19530714-39 2024-09-13T10:11:31.291 INFO:tasks.rados.rados.0.smithi195.stdout:1269: finishing write tid 4 to smithi19530714-39 2024-09-13T10:11:31.291 INFO:tasks.rados.rados.0.smithi195.stdout:1269: oid 39 updating version 0 to 25 2024-09-13T10:11:31.291 INFO:tasks.rados.rados.0.smithi195.stdout:1269: oid 39 updating version 25 to 27 2024-09-13T10:11:31.291 INFO:tasks.rados.rados.0.smithi195.stdout:1269: oid 39 version 27 is already newer than 26 2024-09-13T10:11:31.291 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 27 (ObjNum 241 snap 0 seq_num 241) dirty exists 2024-09-13T10:11:31.291 INFO:tasks.rados.rados.0.smithi195.stdout:1269: left oid 39 (ObjNum 241 snap 0 seq_num 241) 2024-09-13T10:11:31.292 INFO:tasks.rados.rados.0.smithi195.stdout:1262: done (8 left) 2024-09-13T10:11:31.292 INFO:tasks.rados.rados.0.smithi195.stdout:1263: done (7 left) 2024-09-13T10:11:31.292 INFO:tasks.rados.rados.0.smithi195.stdout:1264: done (6 left) 2024-09-13T10:11:31.292 INFO:tasks.rados.rados.0.smithi195.stdout:1265: done (5 left) 2024-09-13T10:11:31.292 INFO:tasks.rados.rados.0.smithi195.stdout:1266: done (4 left) 2024-09-13T10:11:31.292 INFO:tasks.rados.rados.0.smithi195.stdout:1267: done (3 left) 2024-09-13T10:11:31.292 INFO:tasks.rados.rados.0.smithi195.stdout:1268: done (2 left) 2024-09-13T10:11:31.292 INFO:tasks.rados.rados.0.smithi195.stdout:1269: done (1 left) 2024-09-13T10:11:31.292 INFO:tasks.rados.rados.0.smithi195.stdout:1270: done (0 left) 2024-09-13T10:11:31.292 INFO:tasks.rados.rados.0.smithi195.stdout:1271: write (excl) oid 14 current snap is 0 2024-09-13T10:11:31.292 INFO:tasks.rados.rados.0.smithi195.stdout:1271: seq_num 243 ranges {484101=692450,1636889=447212} 2024-09-13T10:11:31.292 INFO:tasks.rados.rados.0.smithi195.stdout:1271: writing smithi19530714-14 from 484101 to 1176551 tid 1 2024-09-13T10:11:31.292 INFO:tasks.rados.rados.0.smithi195.stdout:1271: writing smithi19530714-14 from 1636889 to 2084101 tid 2 2024-09-13T10:11:31.293 INFO:tasks.rados.rados.0.smithi195.stdout:1272: tier_evict oid 43 2024-09-13T10:11:31.293 INFO:tasks.rados.rados.0.smithi195.stdout:1272: tier_evict oid 43 snap -1 2024-09-13T10:11:31.293 INFO:tasks.rados.rados.0.smithi195.stdout:1271: finishing write tid 1 to smithi19530714-14 2024-09-13T10:11:31.293 INFO:tasks.rados.rados.0.smithi195.stdout:1273: read oid 4 snap -1 2024-09-13T10:11:31.293 INFO:tasks.rados.rados.0.smithi195.stdout:1273: expect (ObjNum 239 snap 0 seq_num 239) 2024-09-13T10:11:31.293 INFO:tasks.rados.rados.0.smithi195.stdout:1274: write oid 26 current snap is 0 2024-09-13T10:11:31.293 INFO:tasks.rados.rados.0.smithi195.stdout:1274: seq_num 244 ranges {500908=484052,1411724=689184} 2024-09-13T10:11:31.293 INFO:tasks.rados.rados.0.smithi195.stdout:1274: writing smithi19530714-26 from 500908 to 984960 tid 1 2024-09-13T10:11:31.293 INFO:tasks.rados.rados.0.smithi195.stdout:1274: writing smithi19530714-26 from 1411724 to 2100908 tid 2 2024-09-13T10:11:31.293 INFO:tasks.rados.rados.0.smithi195.stdout:1275: set_chunk oid 23 target oid 2024-09-13T10:11:31.293 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:31.282+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:31.304 INFO:tasks.rados.rados.0.smithi195.stdout:1275: set_chunk oid 23 offset: 1428992 length: 1222144 target oid 5224eb93274067dff2427d7b341ff98a83795426688c1ed33109dd24a6c6d211 offset: 0 snap -1 2024-09-13T10:11:31.304 INFO:tasks.rados.rados.0.smithi195.stdout:1276: write oid 2 current snap is 0 2024-09-13T10:11:31.304 INFO:tasks.rados.rados.0.smithi195.stdout:1272: got (22) Invalid argument 2024-09-13T10:11:31.304 INFO:tasks.rados.rados.0.smithi195.stdout:1276: seq_num 245 ranges {517715=559301,1870206=247509} 2024-09-13T10:11:31.317 INFO:tasks.rados.rados.0.smithi195.stdout:1276: writing smithi19530714-2 from 517715 to 1077016 tid 1 2024-09-13T10:11:31.330 INFO:tasks.rados.rados.0.smithi195.stdout:1276: writing smithi19530714-2 from 1870206 to 2117715 tid 2 2024-09-13T10:11:31.330 INFO:tasks.rados.rados.0.smithi195.stdout:1272: done (5 left) 2024-09-13T10:11:31.330 INFO:tasks.rados.rados.0.smithi195.stdout:1277: read oid 35 snap -1 2024-09-13T10:11:31.331 INFO:tasks.rados.rados.0.smithi195.stdout:1277: expect (ObjNum 208 snap 0 seq_num 208) 2024-09-13T10:11:31.331 INFO:tasks.rados.rados.0.smithi195.stdout:1278: read oid 34 snap -1 2024-09-13T10:11:31.331 INFO:tasks.rados.rados.0.smithi195.stdout:1278: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:31.331 INFO:tasks.rados.rados.0.smithi195.stdout:1279: read oid 27 snap -1 2024-09-13T10:11:31.331 INFO:tasks.rados.rados.0.smithi195.stdout:1279: expect (ObjNum 221 snap 0 seq_num 221) 2024-09-13T10:11:31.331 INFO:tasks.rados.rados.0.smithi195.stdout:1280: read oid 41 snap -1 2024-09-13T10:11:31.331 INFO:tasks.rados.rados.0.smithi195.stdout:1280: expect (ObjNum 236 snap 0 seq_num 236) 2024-09-13T10:11:31.331 INFO:tasks.rados.rados.0.smithi195.stdout:1281: read oid 25 snap -1 2024-09-13T10:11:31.331 INFO:tasks.rados.rados.0.smithi195.stdout:1281: expect (ObjNum 215 snap 0 seq_num 215) 2024-09-13T10:11:31.331 INFO:tasks.rados.rados.0.smithi195.stdout:1282: read oid 49 snap -1 2024-09-13T10:11:31.331 INFO:tasks.rados.rados.0.smithi195.stdout:1282: expect (ObjNum 181 snap 0 seq_num 181) 2024-09-13T10:11:31.331 INFO:tasks.rados.rados.0.smithi195.stdout:1283: write oid 15 current snap is 0 2024-09-13T10:11:31.331 INFO:tasks.rados.rados.0.smithi195.stdout:1283: seq_num 246 ranges {534522=634550,1812335=322187} 2024-09-13T10:11:31.344 INFO:tasks.rados.rados.0.smithi195.stdout:1283: writing smithi19530714-15 from 534522 to 1169072 tid 1 2024-09-13T10:11:31.354 INFO:tasks.rados.rados.0.smithi195.stdout:1283: writing smithi19530714-15 from 1812335 to 2134522 tid 2 2024-09-13T10:11:31.354 INFO:tasks.rados.rados.0.smithi195.stdout:1284: read oid 30 snap -1 2024-09-13T10:11:31.354 INFO:tasks.rados.rados.0.smithi195.stdout:1284: expect (ObjNum 102 snap 0 seq_num 102) 2024-09-13T10:11:31.354 INFO:tasks.rados.rados.0.smithi195.stdout:1285: delete oid 17 current snap is 0 2024-09-13T10:11:31.354 INFO:tasks.rados.rados.0.smithi195.stdout:1271: finishing write tid 2 to smithi19530714-14 2024-09-13T10:11:31.354 INFO:tasks.rados.rados.0.smithi195.stdout:1271: finishing write tid 3 to smithi19530714-14 2024-09-13T10:11:31.355 INFO:tasks.rados.rados.0.smithi195.stdout:1271: finishing write tid 4 to smithi19530714-14 2024-09-13T10:11:31.355 INFO:tasks.rados.rados.0.smithi195.stdout:1271: oid 14 updating version 0 to 79 2024-09-13T10:11:31.355 INFO:tasks.rados.rados.0.smithi195.stdout:1271: oid 14 version 79 is already newer than 77 2024-09-13T10:11:31.355 INFO:tasks.rados.rados.0.smithi195.stdout:1271: oid 14 version 79 is already newer than 78 2024-09-13T10:11:31.355 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 79 (ObjNum 243 snap 0 seq_num 243) dirty exists 2024-09-13T10:11:31.355 INFO:tasks.rados.rados.0.smithi195.stdout:1271: left oid 14 (ObjNum 243 snap 0 seq_num 243) 2024-09-13T10:11:31.355 INFO:tasks.rados.rados.0.smithi195.stdout:1274: finishing write tid 1 to smithi19530714-26 2024-09-13T10:11:31.355 INFO:tasks.rados.rados.0.smithi195.stdout:1274: finishing write tid 2 to smithi19530714-26 2024-09-13T10:11:31.355 INFO:tasks.rados.rados.0.smithi195.stdout:1274: finishing write tid 3 to smithi19530714-26 2024-09-13T10:11:31.355 INFO:tasks.rados.rados.0.smithi195.stdout:1274: finishing write tid 4 to smithi19530714-26 2024-09-13T10:11:31.355 INFO:tasks.rados.rados.0.smithi195.stdout:1274: oid 26 updating version 0 to 30 2024-09-13T10:11:31.355 INFO:tasks.rados.rados.0.smithi195.stdout:1274: oid 26 version 30 is already newer than 29 2024-09-13T10:11:31.356 INFO:tasks.rados.rados.0.smithi195.stdout:1274: oid 26 updating version 30 to 31 2024-09-13T10:11:31.356 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 31 (ObjNum 244 snap 0 seq_num 244) dirty exists 2024-09-13T10:11:31.356 INFO:tasks.rados.rados.0.smithi195.stdout:1274: left oid 26 (ObjNum 244 snap 0 seq_num 244) 2024-09-13T10:11:31.356 INFO:tasks.rados.rados.0.smithi195.stdout:1275: finishing set_chunk to oid 23 2024-09-13T10:11:31.356 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 23 set_chunk returned error code -95 offset: 1428992 length: 1222144 2024-09-13T10:11:31.356 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 0 (ObjNum 206 snap 0 seq_num 206) dirty exists 2024-09-13T10:11:31.356 INFO:tasks.rados.rados.0.smithi195.stdout:1276: finishing write tid 1 to smithi19530714-2 2024-09-13T10:11:31.356 INFO:tasks.rados.rados.0.smithi195.stdout:1278: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:31.385 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:31.382+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:31.432 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:31.461 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:31.461 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:31.476 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-13T10:11:31.477 INFO:tasks.rados.rados.0.smithi195.stdout:1276: finishing write tid 2 to smithi19530714-2 2024-09-13T10:11:31.477 INFO:tasks.rados.rados.0.smithi195.stdout:1276: finishing write tid 3 to smithi19530714-2 2024-09-13T10:11:31.477 INFO:tasks.rados.rados.0.smithi195.stdout:1276: finishing write tid 4 to smithi19530714-2 2024-09-13T10:11:31.477 INFO:tasks.rados.rados.0.smithi195.stdout:1276: oid 2 updating version 0 to 115 2024-09-13T10:11:31.477 INFO:tasks.rados.rados.0.smithi195.stdout:1276: oid 2 updating version 115 to 117 2024-09-13T10:11:31.477 INFO:tasks.rados.rados.0.smithi195.stdout:1276: oid 2 version 117 is already newer than 116 2024-09-13T10:11:31.477 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 117 (ObjNum 245 snap 0 seq_num 245) dirty exists 2024-09-13T10:11:31.477 INFO:tasks.rados.rados.0.smithi195.stdout:1276: left oid 2 (ObjNum 245 snap 0 seq_num 245) 2024-09-13T10:11:31.478 INFO:tasks.rados.rados.0.smithi195.stdout:1280: expect (ObjNum 236 snap 0 seq_num 236) 2024-09-13T10:11:31.489 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:31.486+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:31.543 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:31.588 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:31.588 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:31.590 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:31.588+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:31.602 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-13T10:11:31.693 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:31.688+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:31.702 INFO:tasks.rados.rados.0.smithi195.stdout:1277: expect (ObjNum 208 snap 0 seq_num 208) 2024-09-13T10:11:31.721 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:31.721 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:31.735 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T10:11:31.765 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 404831 2024-09-13T10:11:31.774 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:11:31.792 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:31.791+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:31.852 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1363182 2024-09-13T10:11:31.868 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:31.868 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:31.883 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T10:11:31.892 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:31.890+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:31.904 INFO:tasks.rados.rados.0.smithi195.stdout:1281: expect (ObjNum 215 snap 0 seq_num 215) 2024-09-13T10:11:31.906 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 455245 2024-09-13T10:11:31.918 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:31.918 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:31.935 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:31.936 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:31.956 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:31.957 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:31.978 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:31.978 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:31.993 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:31.988+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:31.995 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:31.995 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:32.006 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:32.006 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:32.006 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:11:32.007 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:11:32.007 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:32.011 INFO:teuthology.orchestra.run.smithi073.stderr:clear 1.3 pg_upmap mapping 2024-09-13T10:11:32.018 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:32.018 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:32.024 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T10:11:32.040 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:32.040 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:32.060 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:32.061 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:32.078 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:32.078 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:32.086 INFO:tasks.rados.rados.0.smithi195.stdout:1279: expect (ObjNum 221 snap 0 seq_num 221) 2024-09-13T10:11:32.093 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:32.090+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:32.096 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:32.097 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:32.118 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:32.118 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:32.139 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:32.139 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:32.151 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:32.152 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:11:32.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:11:32.152 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:11:32.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.152 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:246 3.3 e38/30 3:f1be28e7:::smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'10, mlcod=38'5)", 2024-09-13T10:11:32.152 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.130216+0000", 2024-09-13T10:11:32.152 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.017883826999999, 2024-09-13T10:11:32.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011932478, 2024-09-13T10:11:32.152 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.152 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.153 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.153 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.130216+0000", 2024-09-13T10:11:32.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.130216+0000", 2024-09-13T10:11:32.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.130217+0000", 2024-09-13T10:11:32.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.654e-06 2024-09-13T10:11:32.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.140044+0000", 2024-09-13T10:11:32.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0098266870000000006 2024-09-13T10:11:32.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.140059+0000", 2024-09-13T10:11:32.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5304000000000001e-05 2024-09-13T10:11:32.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.140067+0000", 2024-09-13T10:11:32.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6750000000000002e-06 2024-09-13T10:11:32.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.140118+0000", 2024-09-13T10:11:32.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1462000000000003e-05 2024-09-13T10:11:32.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.140138+0000", 2024-09-13T10:11:32.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9542999999999999e-05 2024-09-13T10:11:32.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.142085+0000", 2024-09-13T10:11:32.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019473959999999999 2024-09-13T10:11:32.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.142148+0000", 2024-09-13T10:11:32.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2756999999999996e-05 2024-09-13T10:11:32.157 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.157 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.157 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.157 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:253 4.7 4:e81d9ad0:::smithi19530714-4:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.157 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.134475+0000", 2024-09-13T10:11:32.157 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.013624473999997, 2024-09-13T10:11:32.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013417535, 2024-09-13T10:11:32.158 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.158 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.158 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.158 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.158 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 253 2024-09-13T10:11:32.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.158 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134475+0000", 2024-09-13T10:11:32.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134475+0000", 2024-09-13T10:11:32.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134477+0000", 2024-09-13T10:11:32.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1859999999999999e-06 2024-09-13T10:11:32.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134486+0000", 2024-09-13T10:11:32.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3139999999999993e-06 2024-09-13T10:11:32.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134506+0000", 2024-09-13T10:11:32.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9165999999999999e-05 2024-09-13T10:11:32.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134515+0000", 2024-09-13T10:11:32.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3670000000000003e-06 2024-09-13T10:11:32.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134579+0000", 2024-09-13T10:11:32.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4200000000000002e-05 2024-09-13T10:11:32.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134661+0000", 2024-09-13T10:11:32.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1632999999999996e-05 2024-09-13T10:11:32.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:32.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.144025+0000", 2024-09-13T10:11:32.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093644190000000006 2024-09-13T10:11:32.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.144574+0000", 2024-09-13T10:11:32.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054873699999999999 2024-09-13T10:11:32.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.146021+0000", 2024-09-13T10:11:32.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001446826 2024-09-13T10:11:32.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.147761+0000", 2024-09-13T10:11:32.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017406399999999999 2024-09-13T10:11:32.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.147835+0000", 2024-09-13T10:11:32.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3708999999999995e-05 2024-09-13T10:11:32.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.147893+0000", 2024-09-13T10:11:32.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7337999999999999e-05 2024-09-13T10:11:32.165 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.165 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.165 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.165 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:256 4.7 4:e8c2cbec:::smithi19530714-7:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.165 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.214859+0000", 2024-09-13T10:11:32.165 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 40.933240116, 2024-09-13T10:11:32.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049512370999999999, 2024-09-13T10:11:32.165 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.165 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.166 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.166 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.166 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.166 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.166 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 256 2024-09-13T10:11:32.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.166 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214859+0000", 2024-09-13T10:11:32.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214859+0000", 2024-09-13T10:11:32.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214863+0000", 2024-09-13T10:11:32.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.343e-06 2024-09-13T10:11:32.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214876+0000", 2024-09-13T10:11:32.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.308e-05 2024-09-13T10:11:32.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214909+0000", 2024-09-13T10:11:32.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2882999999999997e-05 2024-09-13T10:11:32.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214925+0000", 2024-09-13T10:11:32.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6002e-05 2024-09-13T10:11:32.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215088+0000", 2024-09-13T10:11:32.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016304199999999999 2024-09-13T10:11:32.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215261+0000", 2024-09-13T10:11:32.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000173804 2024-09-13T10:11:32.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:32.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258838+0000", 2024-09-13T10:11:32.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043576675000000002 2024-09-13T10:11:32.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.260096+0000", 2024-09-13T10:11:32.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012580410000000001 2024-09-13T10:11:32.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.263959+0000", 2024-09-13T10:11:32.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038630840000000001 2024-09-13T10:11:32.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264262+0000", 2024-09-13T10:11:32.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030279699999999999 2024-09-13T10:11:32.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264321+0000", 2024-09-13T10:11:32.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-13T10:11:32.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264372+0000", 2024-09-13T10:11:32.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0513000000000002e-05 2024-09-13T10:11:32.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.172 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.172 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.172 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:266 4.7 4:e55883d8:::smithi19530714-17:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.172 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.215454+0000", 2024-09-13T10:11:32.172 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 40.932645454999999, 2024-09-13T10:11:32.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020880112999999999, 2024-09-13T10:11:32.172 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.172 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.172 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.172 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.172 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.173 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.173 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 266 2024-09-13T10:11:32.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.173 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215454+0000", 2024-09-13T10:11:32.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215454+0000", 2024-09-13T10:11:32.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215457+0000", 2024-09-13T10:11:32.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6029999999999999e-06 2024-09-13T10:11:32.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215471+0000", 2024-09-13T10:11:32.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4629e-05 2024-09-13T10:11:32.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215503+0000", 2024-09-13T10:11:32.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1449000000000003e-05 2024-09-13T10:11:32.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215528+0000", 2024-09-13T10:11:32.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5273e-05 2024-09-13T10:11:32.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215632+0000", 2024-09-13T10:11:32.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010387699999999999 2024-09-13T10:11:32.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215898+0000", 2024-09-13T10:11:32.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026582100000000002 2024-09-13T10:11:32.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:32.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228364+0000", 2024-09-13T10:11:32.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012466672 2024-09-13T10:11:32.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.229463+0000", 2024-09-13T10:11:32.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001098344 2024-09-13T10:11:32.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.230814+0000", 2024-09-13T10:11:32.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013513959999999999 2024-09-13T10:11:32.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236141+0000", 2024-09-13T10:11:32.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053270160000000004 2024-09-13T10:11:32.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236223+0000", 2024-09-13T10:11:32.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1799999999999996e-05 2024-09-13T10:11:32.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236334+0000", 2024-09-13T10:11:32.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111233 2024-09-13T10:11:32.179 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.179 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.179 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.179 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:267 4.7 4:ea79a1ec:::smithi19530714-18:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.179 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.215583+0000", 2024-09-13T10:11:32.180 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 40.932516483000001, 2024-09-13T10:11:32.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.051600035000000002, 2024-09-13T10:11:32.180 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.180 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.180 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.180 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.180 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 267 2024-09-13T10:11:32.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.180 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215583+0000", 2024-09-13T10:11:32.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215583+0000", 2024-09-13T10:11:32.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215585+0000", 2024-09-13T10:11:32.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9099999999999999e-06 2024-09-13T10:11:32.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215594+0000", 2024-09-13T10:11:32.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8799999999999997e-06 2024-09-13T10:11:32.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215613+0000", 2024-09-13T10:11:32.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9473000000000001e-05 2024-09-13T10:11:32.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215630+0000", 2024-09-13T10:11:32.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6466000000000001e-05 2024-09-13T10:11:32.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.216360+0000", 2024-09-13T10:11:32.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073001200000000004 2024-09-13T10:11:32.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.216526+0000", 2024-09-13T10:11:32.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000166775 2024-09-13T10:11:32.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:32.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.260535+0000", 2024-09-13T10:11:32.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044008310000000002 2024-09-13T10:11:32.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261671+0000", 2024-09-13T10:11:32.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011362169999999999 2024-09-13T10:11:32.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.266819+0000", 2024-09-13T10:11:32.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051479940000000004 2024-09-13T10:11:32.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.267087+0000", 2024-09-13T10:11:32.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026847299999999998 2024-09-13T10:11:32.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.267140+0000", 2024-09-13T10:11:32.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2054e-05 2024-09-13T10:11:32.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.267183+0000", 2024-09-13T10:11:32.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3470999999999998e-05 2024-09-13T10:11:32.186 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.186 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.186 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.186 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:303 4.7 4:e81d9ad0:::smithi19530714-4:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.186 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.268636+0000", 2024-09-13T10:11:32.186 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 40.879463868000002, 2024-09-13T10:11:32.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018691609000000001, 2024-09-13T10:11:32.186 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.186 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.187 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.187 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.187 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.187 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.187 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 303 2024-09-13T10:11:32.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.187 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268636+0000", 2024-09-13T10:11:32.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268636+0000", 2024-09-13T10:11:32.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268638+0000", 2024-09-13T10:11:32.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8990000000000001e-06 2024-09-13T10:11:32.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268651+0000", 2024-09-13T10:11:32.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2556e-05 2024-09-13T10:11:32.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268682+0000", 2024-09-13T10:11:32.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0979000000000003e-05 2024-09-13T10:11:32.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268707+0000", 2024-09-13T10:11:32.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4680000000000001e-05 2024-09-13T10:11:32.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268787+0000", 2024-09-13T10:11:32.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9878999999999999e-05 2024-09-13T10:11:32.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268881+0000", 2024-09-13T10:11:32.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4242999999999999e-05 2024-09-13T10:11:32.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:32.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.283577+0000", 2024-09-13T10:11:32.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014695952999999999 2024-09-13T10:11:32.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.284599+0000", 2024-09-13T10:11:32.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001022329 2024-09-13T10:11:32.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285985+0000", 2024-09-13T10:11:32.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013861430000000001 2024-09-13T10:11:32.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.287150+0000", 2024-09-13T10:11:32.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001164382 2024-09-13T10:11:32.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.287245+0000", 2024-09-13T10:11:32.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5643999999999994e-05 2024-09-13T10:11:32.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.287327+0000", 2024-09-13T10:11:32.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1922000000000006e-05 2024-09-13T10:11:32.192 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.192 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.192 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.193 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:567 4.7 4:ea79a1ec:::smithi19530714-18:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:32.193 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:11:32.193 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 40.780798716, 2024-09-13T10:11:32.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012585694999999999, 2024-09-13T10:11:32.193 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.193 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.193 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.193 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.193 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 567 2024-09-13T10:11:32.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.194 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:11:32.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:11:32.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:11:32.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1200000000000003e-07 2024-09-13T10:11:32.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367306+0000", 2024-09-13T10:11:32.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0309999999999998e-06 2024-09-13T10:11:32.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367314+0000", 2024-09-13T10:11:32.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8960000000000003e-06 2024-09-13T10:11:32.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367321+0000", 2024-09-13T10:11:32.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4880000000000004e-06 2024-09-13T10:11:32.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367612+0000", 2024-09-13T10:11:32.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029163099999999999 2024-09-13T10:11:32.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367715+0000", 2024-09-13T10:11:32.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102837 2024-09-13T10:11:32.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:32.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.374040+0000", 2024-09-13T10:11:32.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063245539999999996 2024-09-13T10:11:32.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.374609+0000", 2024-09-13T10:11:32.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056937299999999997 2024-09-13T10:11:32.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379774+0000", 2024-09-13T10:11:32.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005164737 2024-09-13T10:11:32.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379810+0000", 2024-09-13T10:11:32.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6587000000000001e-05 2024-09-13T10:11:32.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379845+0000", 2024-09-13T10:11:32.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.468e-05 2024-09-13T10:11:32.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379886+0000", 2024-09-13T10:11:32.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1269000000000001e-05 2024-09-13T10:11:32.199 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.199 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.199 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.199 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:762 4.7 4:ea79a1ec:::smithi19530714-18:head [write 457157~535817,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:32.199 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.449773+0000", 2024-09-13T10:11:32.200 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 40.698326279, 2024-09-13T10:11:32.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012291262000000001, 2024-09-13T10:11:32.200 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.200 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.200 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.200 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.200 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.200 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.200 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 762 2024-09-13T10:11:32.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.200 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.449773+0000", 2024-09-13T10:11:32.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.449773+0000", 2024-09-13T10:11:32.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.449777+0000", 2024-09-13T10:11:32.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1969999999999998e-06 2024-09-13T10:11:32.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.450154+0000", 2024-09-13T10:11:32.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037642099999999999 2024-09-13T10:11:32.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.450175+0000", 2024-09-13T10:11:32.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0891e-05 2024-09-13T10:11:32.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.202 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.202 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.450185+0000", 2024-09-13T10:11:32.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0362e-05 2024-09-13T10:11:32.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.450251+0000", 2024-09-13T10:11:32.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5712999999999999e-05 2024-09-13T10:11:32.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.450324+0000", 2024-09-13T10:11:32.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2885000000000004e-05 2024-09-13T10:11:32.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:32.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.450451+0000", 2024-09-13T10:11:32.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012726600000000001 2024-09-13T10:11:32.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.451966+0000", 2024-09-13T10:11:32.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001515408 2024-09-13T10:11:32.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.461707+0000", 2024-09-13T10:11:32.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097404339999999992 2024-09-13T10:11:32.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.461837+0000", 2024-09-13T10:11:32.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013055900000000001 2024-09-13T10:11:32.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.461995+0000", 2024-09-13T10:11:32.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015793400000000001 2024-09-13T10:11:32.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.462064+0000", 2024-09-13T10:11:32.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9191999999999998e-05 2024-09-13T10:11:32.206 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.206 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.206 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.206 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1039 4.17 4:ea79a1ec:::smithi19530714-18:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:32.206 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.096166+0000", 2024-09-13T10:11:32.207 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.051933458999997, 2024-09-13T10:11:32.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.051040927, 2024-09-13T10:11:32.207 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.207 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.207 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:32.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.207 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1039 2024-09-13T10:11:32.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.207 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.207 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096166+0000", 2024-09-13T10:11:32.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096166+0000", 2024-09-13T10:11:32.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096169+0000", 2024-09-13T10:11:32.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8100000000000002e-06 2024-09-13T10:11:32.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096180+0000", 2024-09-13T10:11:32.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1537e-05 2024-09-13T10:11:32.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096212+0000", 2024-09-13T10:11:32.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2104999999999998e-05 2024-09-13T10:11:32.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096268+0000", 2024-09-13T10:11:32.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5127999999999998e-05 2024-09-13T10:11:32.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096480+0000", 2024-09-13T10:11:32.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021249100000000001 2024-09-13T10:11:32.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096692+0000", 2024-09-13T10:11:32.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021200400000000001 2024-09-13T10:11:32.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.147207+0000", 2024-09-13T10:11:32.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.050514851999999999 2024-09-13T10:11:32.211 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.211 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.211 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.211 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1574 4.7 4:e79f1505:::smithi19530714-32:head [write 1799494~402223] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:11:32.211 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.716930+0000", 2024-09-13T10:11:32.211 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.431169192999999, 2024-09-13T10:11:32.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017286329999999999, 2024-09-13T10:11:32.211 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.211 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.211 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.211 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.212 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.212 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.212 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1574 2024-09-13T10:11:32.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.212 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.716930+0000", 2024-09-13T10:11:32.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.212 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.212 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.716930+0000", 2024-09-13T10:11:32.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.716962+0000", 2024-09-13T10:11:32.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1279999999999999e-05 2024-09-13T10:11:32.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717453+0000", 2024-09-13T10:11:32.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049160399999999998 2024-09-13T10:11:32.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717480+0000", 2024-09-13T10:11:32.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7313000000000001e-05 2024-09-13T10:11:32.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717496+0000", 2024-09-13T10:11:32.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5435999999999999e-05 2024-09-13T10:11:32.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717592+0000", 2024-09-13T10:11:32.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5851000000000001e-05 2024-09-13T10:11:32.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717732+0000", 2024-09-13T10:11:32.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013985000000000001 2024-09-13T10:11:32.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:32.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717962+0000", 2024-09-13T10:11:32.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000230518 2024-09-13T10:11:32.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722934+0000", 2024-09-13T10:11:32.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049717099999999998 2024-09-13T10:11:32.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724179+0000", 2024-09-13T10:11:32.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012455389999999999 2024-09-13T10:11:32.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.733963+0000", 2024-09-13T10:11:32.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097835349999999995 2024-09-13T10:11:32.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.734101+0000", 2024-09-13T10:11:32.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013852 2024-09-13T10:11:32.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.734217+0000", 2024-09-13T10:11:32.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011517400000000001 2024-09-13T10:11:32.217 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.217 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.217 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.218 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1575 4.7 4:e79f1505:::smithi19530714-32:head [setxattr _header (56) in=7b,truncate 2201717] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:11:32.218 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.717530+0000", 2024-09-13T10:11:32.218 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.430569037000001, 2024-09-13T10:11:32.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018674682000000001, 2024-09-13T10:11:32.218 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.218 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.218 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.218 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.218 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.218 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.218 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1575 2024-09-13T10:11:32.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.219 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717530+0000", 2024-09-13T10:11:32.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717530+0000", 2024-09-13T10:11:32.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717532+0000", 2024-09-13T10:11:32.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.813e-06 2024-09-13T10:11:32.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717542+0000", 2024-09-13T10:11:32.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4930000000000004e-06 2024-09-13T10:11:32.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717559+0000", 2024-09-13T10:11:32.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7337e-05 2024-09-13T10:11:32.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717575+0000", 2024-09-13T10:11:32.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6385999999999999e-05 2024-09-13T10:11:32.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.719954+0000", 2024-09-13T10:11:32.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023782679999999998 2024-09-13T10:11:32.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.720308+0000", 2024-09-13T10:11:32.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035397100000000002 2024-09-13T10:11:32.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:32.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.720827+0000", 2024-09-13T10:11:32.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00051909999999999999 2024-09-13T10:11:32.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722944+0000", 2024-09-13T10:11:32.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021173210000000001 2024-09-13T10:11:32.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724274+0000", 2024-09-13T10:11:32.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013303099999999999 2024-09-13T10:11:32.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.735926+0000", 2024-09-13T10:11:32.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011651846 2024-09-13T10:11:32.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736061+0000", 2024-09-13T10:11:32.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013452000000000001 2024-09-13T10:11:32.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736205+0000", 2024-09-13T10:11:32.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014431700000000001 2024-09-13T10:11:32.224 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.224 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.224 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.224 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1576 4.7 4:e79f1505:::smithi19530714-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:11:32.224 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.717602+0000", 2024-09-13T10:11:32.224 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.430497851999998, 2024-09-13T10:11:32.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019134776999999999, 2024-09-13T10:11:32.225 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.225 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.225 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:32.225 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.225 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.225 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.225 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1576 2024-09-13T10:11:32.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.225 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717602+0000", 2024-09-13T10:11:32.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717602+0000", 2024-09-13T10:11:32.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717603+0000", 2024-09-13T10:11:32.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8899999999999999e-06 2024-09-13T10:11:32.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717610+0000", 2024-09-13T10:11:32.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8900000000000001e-06 2024-09-13T10:11:32.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717631+0000", 2024-09-13T10:11:32.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0741999999999999e-05 2024-09-13T10:11:32.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717644+0000", 2024-09-13T10:11:32.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2680000000000001e-05 2024-09-13T10:11:32.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722783+0000", 2024-09-13T10:11:32.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005139429 2024-09-13T10:11:32.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:32.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722912+0000", 2024-09-13T10:11:32.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012886399999999999 2024-09-13T10:11:32.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736301+0000", 2024-09-13T10:11:32.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013389082 2024-09-13T10:11:32.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736498+0000", 2024-09-13T10:11:32.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000197059 2024-09-13T10:11:32.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736736+0000", 2024-09-13T10:11:32.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023814100000000001 2024-09-13T10:11:32.230 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.230 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.230 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.230 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1580 4.17 4:e8c2cbec:::smithi19530714-7:head [write 618524~710869,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:11:32.230 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.724425+0000", 2024-09-13T10:11:32.230 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.423674111, 2024-09-13T10:11:32.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013470263999999999, 2024-09-13T10:11:32.230 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.230 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.230 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.230 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.231 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.231 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.231 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1580 2024-09-13T10:11:32.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.231 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724425+0000", 2024-09-13T10:11:32.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724425+0000", 2024-09-13T10:11:32.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724427+0000", 2024-09-13T10:11:32.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9530000000000002e-06 2024-09-13T10:11:32.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.232 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725799+0000", 2024-09-13T10:11:32.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001371466 2024-09-13T10:11:32.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725823+0000", 2024-09-13T10:11:32.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4332999999999999e-05 2024-09-13T10:11:32.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725844+0000", 2024-09-13T10:11:32.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0539000000000001e-05 2024-09-13T10:11:32.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725963+0000", 2024-09-13T10:11:32.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119524 2024-09-13T10:11:32.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.726194+0000", 2024-09-13T10:11:32.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023045800000000001 2024-09-13T10:11:32.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:32.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.726619+0000", 2024-09-13T10:11:32.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042517199999999997 2024-09-13T10:11:32.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.735852+0000", 2024-09-13T10:11:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0092335560000000004 2024-09-13T10:11:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.735959+0000", 2024-09-13T10:11:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106351 2024-09-13T10:11:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.737757+0000", 2024-09-13T10:11:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017985119999999999 2024-09-13T10:11:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.737838+0000", 2024-09-13T10:11:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0457999999999994e-05 2024-09-13T10:11:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.737896+0000", 2024-09-13T10:11:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7942000000000002e-05 2024-09-13T10:11:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.237 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1855 4.17 4:e81d9ad0:::smithi19530714-4:head [write 739541~709000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:32.237 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.758272+0000", 2024-09-13T10:11:32.237 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.389827406999999, 2024-09-13T10:11:32.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01875696, 2024-09-13T10:11:32.237 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.237 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.237 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.237 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.237 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.237 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.237 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1855 2024-09-13T10:11:32.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.237 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.758272+0000", 2024-09-13T10:11:32.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.758272+0000", 2024-09-13T10:11:32.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.758275+0000", 2024-09-13T10:11:32.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4429999999999999e-06 2024-09-13T10:11:32.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.759964+0000", 2024-09-13T10:11:32.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001688557 2024-09-13T10:11:32.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.759996+0000", 2024-09-13T10:11:32.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1705999999999998e-05 2024-09-13T10:11:32.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.760013+0000", 2024-09-13T10:11:32.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6846000000000001e-05 2024-09-13T10:11:32.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.760156+0000", 2024-09-13T10:11:32.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014331700000000001 2024-09-13T10:11:32.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-13T10:11:32.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.760167+0000", 2024-09-13T10:11:32.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.062e-05 2024-09-13T10:11:32.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.763490+0000", 2024-09-13T10:11:32.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033235309999999998 2024-09-13T10:11:32.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.763611+0000", 2024-09-13T10:11:32.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012099699999999999 2024-09-13T10:11:32.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:32.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.763830+0000", 2024-09-13T10:11:32.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000218519 2024-09-13T10:11:32.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773195+0000", 2024-09-13T10:11:32.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093654340000000006 2024-09-13T10:11:32.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.774701+0000", 2024-09-13T10:11:32.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00150588 2024-09-13T10:11:32.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.776817+0000", 2024-09-13T10:11:32.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021156180000000001 2024-09-13T10:11:32.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.776910+0000", 2024-09-13T10:11:32.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3194000000000003e-05 2024-09-13T10:11:32.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.777029+0000", 2024-09-13T10:11:32.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119298 2024-09-13T10:11:32.244 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.244 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.244 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.244 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1856 4.17 4:e81d9ad0:::smithi19530714-4:head [write 2121859~426376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:32.244 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.765194+0000", 2024-09-13T10:11:32.244 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.382905613999998, 2024-09-13T10:11:32.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013074073, 2024-09-13T10:11:32.244 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.244 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.244 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.244 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.245 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.245 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.245 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1856 2024-09-13T10:11:32.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.245 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765194+0000", 2024-09-13T10:11:32.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.245 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.245 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765194+0000", 2024-09-13T10:11:32.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765196+0000", 2024-09-13T10:11:32.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.317e-06 2024-09-13T10:11:32.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765559+0000", 2024-09-13T10:11:32.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036326199999999999 2024-09-13T10:11:32.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765581+0000", 2024-09-13T10:11:32.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1676e-05 2024-09-13T10:11:32.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765595+0000", 2024-09-13T10:11:32.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4025999999999999e-05 2024-09-13T10:11:32.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.767613+0000", 2024-09-13T10:11:32.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00201768 2024-09-13T10:11:32.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.767751+0000", 2024-09-13T10:11:32.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013809 2024-09-13T10:11:32.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:32.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.767983+0000", 2024-09-13T10:11:32.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023206499999999999 2024-09-13T10:11:32.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773203+0000", 2024-09-13T10:11:32.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052195599999999998 2024-09-13T10:11:32.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.777481+0000", 2024-09-13T10:11:32.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042782840000000003 2024-09-13T10:11:32.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.778120+0000", 2024-09-13T10:11:32.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063947299999999999 2024-09-13T10:11:32.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.778203+0000", 2024-09-13T10:11:32.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2857999999999998e-05 2024-09-13T10:11:32.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.778268+0000", 2024-09-13T10:11:32.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4782000000000004e-05 2024-09-13T10:11:32.250 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.250 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.250 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.251 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1857 4.17 4:e81d9ad0:::smithi19530714-4:head [setxattr _header (55) in=7b,truncate 2739541] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:32.251 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.765630+0000", 2024-09-13T10:11:32.251 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.382469907000001, 2024-09-13T10:11:32.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013989379999999999, 2024-09-13T10:11:32.251 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.251 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.251 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.251 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.251 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1857 2024-09-13T10:11:32.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.251 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765630+0000", 2024-09-13T10:11:32.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765630+0000", 2024-09-13T10:11:32.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765631+0000", 2024-09-13T10:11:32.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.64e-06 2024-09-13T10:11:32.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765641+0000", 2024-09-13T10:11:32.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0047000000000001e-05 2024-09-13T10:11:32.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765658+0000", 2024-09-13T10:11:32.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6909e-05 2024-09-13T10:11:32.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765673+0000", 2024-09-13T10:11:32.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4980000000000001e-05 2024-09-13T10:11:32.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.771667+0000", 2024-09-13T10:11:32.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059938209999999999 2024-09-13T10:11:32.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.771795+0000", 2024-09-13T10:11:32.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012816999999999999 2024-09-13T10:11:32.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:32.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.772028+0000", 2024-09-13T10:11:32.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023327800000000001 2024-09-13T10:11:32.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773657+0000", 2024-09-13T10:11:32.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016287039999999999 2024-09-13T10:11:32.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.777579+0000", 2024-09-13T10:11:32.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039218589999999998 2024-09-13T10:11:32.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779159+0000", 2024-09-13T10:11:32.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001580176 2024-09-13T10:11:32.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779343+0000", 2024-09-13T10:11:32.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000183769 2024-09-13T10:11:32.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779619+0000", 2024-09-13T10:11:32.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027602699999999998 2024-09-13T10:11:32.257 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.257 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.257 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.257 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1858 4.17 4:e81d9ad0:::smithi19530714-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:32.257 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.765692+0000", 2024-09-13T10:11:32.257 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.382407821000001, 2024-09-13T10:11:32.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014311129000000001, 2024-09-13T10:11:32.257 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.257 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.257 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:32.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.258 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.258 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1858 2024-09-13T10:11:32.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.258 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765692+0000", 2024-09-13T10:11:32.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765692+0000", 2024-09-13T10:11:32.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765693+0000", 2024-09-13T10:11:32.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-13T10:11:32.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765697+0000", 2024-09-13T10:11:32.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6770000000000003e-06 2024-09-13T10:11:32.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765707+0000", 2024-09-13T10:11:32.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0129000000000001e-05 2024-09-13T10:11:32.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765715+0000", 2024-09-13T10:11:32.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0469999999999994e-06 2024-09-13T10:11:32.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773032+0000", 2024-09-13T10:11:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073166680000000001 2024-09-13T10:11:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773169+0000", 2024-09-13T10:11:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136964 2024-09-13T10:11:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779726+0000", 2024-09-13T10:11:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065572499999999997 2024-09-13T10:11:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779851+0000", 2024-09-13T10:11:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012477500000000001 2024-09-13T10:11:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.780003+0000", 2024-09-13T10:11:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015165299999999999 2024-09-13T10:11:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.262 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2113 4.17 4:e81d9ad0:::smithi19530714-4:head [write 1410296~761389] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T10:11:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:15.709580+0000", 2024-09-13T10:11:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.438519439, 2024-09-13T10:11:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012946568, 2024-09-13T10:11:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2113 2024-09-13T10:11:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.709580+0000", 2024-09-13T10:11:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.709580+0000", 2024-09-13T10:11:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.709582+0000", 2024-09-13T10:11:32.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4530000000000001e-06 2024-09-13T10:11:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710399+0000", 2024-09-13T10:11:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081639799999999997 2024-09-13T10:11:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710428+0000", 2024-09-13T10:11:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8736000000000001e-05 2024-09-13T10:11:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710443+0000", 2024-09-13T10:11:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5216e-05 2024-09-13T10:11:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710572+0000", 2024-09-13T10:11:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012882799999999999 2024-09-13T10:11:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710808+0000", 2024-09-13T10:11:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023585500000000001 2024-09-13T10:11:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.711245+0000", 2024-09-13T10:11:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043798099999999999 2024-09-13T10:11:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.716561+0000", 2024-09-13T10:11:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053156760000000001 2024-09-13T10:11:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.718008+0000", 2024-09-13T10:11:32.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014466380000000001 2024-09-13T10:11:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.722191+0000", 2024-09-13T10:11:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041829229999999998 2024-09-13T10:11:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.722439+0000", 2024-09-13T10:11:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024823899999999998 2024-09-13T10:11:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.722527+0000", 2024-09-13T10:11:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7625000000000002e-05 2024-09-13T10:11:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2412 4.7 4:e55883d8:::smithi19530714-17:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e63)", 2024-09-13T10:11:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:22.233147+0000", 2024-09-13T10:11:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.9149524729999996, 2024-09-13T10:11:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042005107999999999, 2024-09-13T10:11:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.269 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2412 2024-09-13T10:11:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233147+0000", 2024-09-13T10:11:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233147+0000", 2024-09-13T10:11:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233150+0000", 2024-09-13T10:11:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0029999999999999e-06 2024-09-13T10:11:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233162+0000", 2024-09-13T10:11:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2449000000000001e-05 2024-09-13T10:11:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233195+0000", 2024-09-13T10:11:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2455e-05 2024-09-13T10:11:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233213+0000", 2024-09-13T10:11:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8185999999999998e-05 2024-09-13T10:11:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233351+0000", 2024-09-13T10:11:32.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000137942 2024-09-13T10:11:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233497+0000", 2024-09-13T10:11:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014627100000000001 2024-09-13T10:11:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.275152+0000", 2024-09-13T10:11:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041654801999999998 2024-09-13T10:11:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2669 4.17 4:e8c2cbec:::smithi19530714-7:head [sparse-read 0~2591864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:28.655647+0000", 2024-09-13T10:11:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.492452031, 2024-09-13T10:11:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01160226, 2024-09-13T10:11:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.274 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2669 2024-09-13T10:11:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:28.655647+0000", 2024-09-13T10:11:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:28.655647+0000", 2024-09-13T10:11:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:28.655648+0000", 2024-09-13T10:11:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3699999999999999e-07 2024-09-13T10:11:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:28.655652+0000", 2024-09-13T10:11:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1160000000000001e-06 2024-09-13T10:11:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:28.655675+0000", 2024-09-13T10:11:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3099e-05 2024-09-13T10:11:32.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:28.655684+0000", 2024-09-13T10:11:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9460000000000004e-06 2024-09-13T10:11:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:28.664152+0000", 2024-09-13T10:11:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0084673590000000007 2024-09-13T10:11:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:28.664273+0000", 2024-09-13T10:11:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000121321 2024-09-13T10:11:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:28.667250+0000", 2024-09-13T10:11:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029765820000000002 2024-09-13T10:11:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.278 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.278 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:32.279 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:32.280 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:32.280 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:32.281 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:32.281 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T10:11:32.282 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:32.282 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:32.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:32.220+0000 7fb9837fe640 0 -- 172.21.15.73:0/3925513680 >> v2:172.21.15.195:6805/848686366 conn(0x7fb9842256d0 msgr2=0x7fb984227ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:11:32.283 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:32.283 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:32.276+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:32.287 INFO:tasks.rados.rados.0.smithi195.stdout:1282: expect (ObjNum 181 snap 0 seq_num 181) 2024-09-13T10:11:32.381 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:32.379+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:32.392 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:32.416 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:32.417 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:32.417 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:11:32.417 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:32.428 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T10:11:32.453 INFO:tasks.rados.rados.0.smithi195.stdout:1283: finishing write tid 1 to smithi19530714-15 2024-09-13T10:11:32.453 INFO:tasks.rados.rados.0.smithi195.stdout:1283: finishing write tid 2 to smithi19530714-15 2024-09-13T10:11:32.453 INFO:tasks.rados.rados.0.smithi195.stdout:1283: finishing write tid 3 to smithi19530714-15 2024-09-13T10:11:32.453 INFO:tasks.rados.rados.0.smithi195.stdout:1283: finishing write tid 4 to smithi19530714-15 2024-09-13T10:11:32.453 INFO:tasks.rados.rados.0.smithi195.stdout:1283: oid 15 updating version 0 to 80 2024-09-13T10:11:32.454 INFO:tasks.rados.rados.0.smithi195.stdout:1283: oid 15 updating version 80 to 81 2024-09-13T10:11:32.454 INFO:tasks.rados.rados.0.smithi195.stdout:1283: oid 15 updating version 81 to 82 2024-09-13T10:11:32.454 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 82 (ObjNum 246 snap 0 seq_num 246) dirty exists 2024-09-13T10:11:32.454 INFO:tasks.rados.rados.0.smithi195.stdout:1283: left oid 15 (ObjNum 246 snap 0 seq_num 246) 2024-09-13T10:11:32.454 INFO:tasks.rados.rados.0.smithi195.stdout:1284: expect (ObjNum 102 snap 0 seq_num 102) 2024-09-13T10:11:32.481 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:32.476+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:32.482 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:32.519 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:32.550 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:32.551 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:32.551 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:11:32.551 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:11:32.551 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:32.556 INFO:tasks.rados.rados.0.smithi195.stdout:1271: done (13 left) 2024-09-13T10:11:32.556 INFO:tasks.rados.rados.0.smithi195.stdout:1274: done (12 left) 2024-09-13T10:11:32.556 INFO:tasks.rados.rados.0.smithi195.stdout:1275: done (11 left) 2024-09-13T10:11:32.556 INFO:tasks.rados.rados.0.smithi195.stdout:1276: done (10 left) 2024-09-13T10:11:32.557 INFO:tasks.rados.rados.0.smithi195.stdout:1277: done (9 left) 2024-09-13T10:11:32.557 INFO:tasks.rados.rados.0.smithi195.stdout:1278: done (8 left) 2024-09-13T10:11:32.557 INFO:tasks.rados.rados.0.smithi195.stdout:1279: done (7 left) 2024-09-13T10:11:32.557 INFO:tasks.rados.rados.0.smithi195.stdout:1280: done (6 left) 2024-09-13T10:11:32.557 INFO:tasks.rados.rados.0.smithi195.stdout:1281: done (5 left) 2024-09-13T10:11:32.557 INFO:tasks.rados.rados.0.smithi195.stdout:1282: done (4 left) 2024-09-13T10:11:32.557 INFO:tasks.rados.rados.0.smithi195.stdout:1283: done (3 left) 2024-09-13T10:11:32.557 INFO:tasks.rados.rados.0.smithi195.stdout:1284: done (2 left) 2024-09-13T10:11:32.557 INFO:tasks.rados.rados.0.smithi195.stdout:1285: done (1 left) 2024-09-13T10:11:32.557 INFO:tasks.rados.rados.0.smithi195.stdout:1286: read oid 45 snap -1 2024-09-13T10:11:32.557 INFO:tasks.rados.rados.0.smithi195.stdout:1286: expect (ObjNum 202 snap 0 seq_num 202) 2024-09-13T10:11:32.557 INFO:tasks.rados.rados.0.smithi195.stdout:1287: tier_promote oid 11 2024-09-13T10:11:32.557 INFO:tasks.rados.rados.0.smithi195.stdout:1288: set_chunk oid 47 target oid 2024-09-13T10:11:32.566 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T10:11:32.582 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:32.580+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:32.592 INFO:tasks.rados.rados.0.smithi195.stdout:1288: set_chunk oid 47 offset: 543232 length: 2057216 target oid 890eeb33aa7613d6935d7cb9ed4860e522183188e5dc23a51d7045443a8d23dd offset: 0 snap -1 2024-09-13T10:11:32.593 INFO:tasks.rados.rados.0.smithi195.stdout:1289: delete oid 40 current snap is 0 2024-09-13T10:11:32.593 INFO:tasks.rados.rados.0.smithi195.stdout:1287: got (0) Success 2024-09-13T10:11:32.593 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 72 (ObjNum 217 snap 0 seq_num 217) dirty exists 2024-09-13T10:11:32.593 INFO:tasks.rados.rados.0.smithi195.stdout:1286: expect (ObjNum 202 snap 0 seq_num 202) 2024-09-13T10:11:32.683 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:32.680+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:32.694 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:32.694 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:11:32.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:11:32.694 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:11:32.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.694 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:61 4.0 4:0b4c105e:::smithi19530714-16:head [write 652105~445088,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.694 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.720236+0000", 2024-09-13T10:11:32.695 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.970086801000001, 2024-09-13T10:11:32.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086548600000000003, 2024-09-13T10:11:32.695 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.695 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.695 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.695 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 61 2024-09-13T10:11:32.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.695 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720236+0000", 2024-09-13T10:11:32.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720236+0000", 2024-09-13T10:11:32.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720240+0000", 2024-09-13T10:11:32.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3629999999999998e-06 2024-09-13T10:11:32.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720942+0000", 2024-09-13T10:11:32.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070250300000000005 2024-09-13T10:11:32.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720968+0000", 2024-09-13T10:11:32.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5245999999999999e-05 2024-09-13T10:11:32.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.720984+0000", 2024-09-13T10:11:32.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6821999999999999e-05 2024-09-13T10:11:32.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721070+0000", 2024-09-13T10:11:32.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5660000000000003e-05 2024-09-13T10:11:32.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721221+0000", 2024-09-13T10:11:32.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015077499999999999 2024-09-13T10:11:32.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:32.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721364+0000", 2024-09-13T10:11:32.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014364299999999999 2024-09-13T10:11:32.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728585+0000", 2024-09-13T10:11:32.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072205680000000001 2024-09-13T10:11:32.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728595+0000", 2024-09-13T10:11:32.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0108e-05 2024-09-13T10:11:32.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728688+0000", 2024-09-13T10:11:32.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2405999999999996e-05 2024-09-13T10:11:32.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728769+0000", 2024-09-13T10:11:32.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1174000000000006e-05 2024-09-13T10:11:32.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728891+0000", 2024-09-13T10:11:32.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012259200000000001 2024-09-13T10:11:32.702 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.702 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.702 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.702 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:62 4.0 4:0b4c105e:::smithi19530714-16:head [write 1728171~452988] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.702 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.721021+0000", 2024-09-13T10:11:32.702 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.969301770999998, 2024-09-13T10:11:32.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096364669999999993, 2024-09-13T10:11:32.702 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.702 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.702 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.702 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.703 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.703 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.703 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 62 2024-09-13T10:11:32.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.703 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721021+0000", 2024-09-13T10:11:32.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721021+0000", 2024-09-13T10:11:32.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721023+0000", 2024-09-13T10:11:32.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-13T10:11:32.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721744+0000", 2024-09-13T10:11:32.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000721092 2024-09-13T10:11:32.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721766+0000", 2024-09-13T10:11:32.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2391999999999998e-05 2024-09-13T10:11:32.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721779+0000", 2024-09-13T10:11:32.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2864000000000001e-05 2024-09-13T10:11:32.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.724492+0000", 2024-09-13T10:11:32.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027128759999999999 2024-09-13T10:11:32.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.724746+0000", 2024-09-13T10:11:32.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000253914 2024-09-13T10:11:32.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:32.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.725111+0000", 2024-09-13T10:11:32.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036536400000000002 2024-09-13T10:11:32.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728978+0000", 2024-09-13T10:11:32.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038665969999999998 2024-09-13T10:11:32.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728991+0000", 2024-09-13T10:11:32.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3711e-05 2024-09-13T10:11:32.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.730564+0000", 2024-09-13T10:11:32.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001572564 2024-09-13T10:11:32.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.730617+0000", 2024-09-13T10:11:32.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2744999999999997e-05 2024-09-13T10:11:32.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.730658+0000", 2024-09-13T10:11:32.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1291999999999997e-05 2024-09-13T10:11:32.708 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.708 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.708 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.709 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:63 4.0 4:0b4c105e:::smithi19530714-16:head [setxattr _header (56) in=7b,truncate 2252105] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.709 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.721807+0000", 2024-09-13T10:11:32.709 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.968516227999999, 2024-09-13T10:11:32.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093328749999999992, 2024-09-13T10:11:32.709 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.709 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.709 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.712 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.712 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 63 2024-09-13T10:11:32.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.712 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721807+0000", 2024-09-13T10:11:32.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721807+0000", 2024-09-13T10:11:32.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721809+0000", 2024-09-13T10:11:32.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6559999999999999e-06 2024-09-13T10:11:32.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721818+0000", 2024-09-13T10:11:32.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1530000000000003e-06 2024-09-13T10:11:32.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721836+0000", 2024-09-13T10:11:32.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8240000000000002e-05 2024-09-13T10:11:32.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721854+0000", 2024-09-13T10:11:32.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7532000000000001e-05 2024-09-13T10:11:32.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.727557+0000", 2024-09-13T10:11:32.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057033259999999999 2024-09-13T10:11:32.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.727772+0000", 2024-09-13T10:11:32.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021469600000000001 2024-09-13T10:11:32.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:32.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728048+0000", 2024-09-13T10:11:32.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027671299999999999 2024-09-13T10:11:32.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.729819+0000", 2024-09-13T10:11:32.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017708489999999999 2024-09-13T10:11:32.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.730310+0000", 2024-09-13T10:11:32.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049081999999999999 2024-09-13T10:11:32.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731047+0000", 2024-09-13T10:11:32.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073742899999999995 2024-09-13T10:11:32.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731096+0000", 2024-09-13T10:11:32.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8573999999999999e-05 2024-09-13T10:11:32.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731140+0000", 2024-09-13T10:11:32.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3887000000000002e-05 2024-09-13T10:11:32.718 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.718 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.718 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.718 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:64 4.0 4:0b4c105e:::smithi19530714-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.718 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.721878+0000", 2024-09-13T10:11:32.718 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.968445088000003, 2024-09-13T10:11:32.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094623740000000008, 2024-09-13T10:11:32.719 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.719 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.719 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:32.719 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.719 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.719 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.719 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 64 2024-09-13T10:11:32.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.719 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721878+0000", 2024-09-13T10:11:32.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721878+0000", 2024-09-13T10:11:32.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721880+0000", 2024-09-13T10:11:32.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3939999999999999e-06 2024-09-13T10:11:32.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721886+0000", 2024-09-13T10:11:32.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0390000000000001e-06 2024-09-13T10:11:32.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721902+0000", 2024-09-13T10:11:32.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6520000000000001e-05 2024-09-13T10:11:32.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721917+0000", 2024-09-13T10:11:32.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4959e-05 2024-09-13T10:11:32.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.729034+0000", 2024-09-13T10:11:32.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071171100000000003 2024-09-13T10:11:32.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:32.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.729161+0000", 2024-09-13T10:11:32.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012695600000000001 2024-09-13T10:11:32.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731172+0000", 2024-09-13T10:11:32.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002010651 2024-09-13T10:11:32.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731241+0000", 2024-09-13T10:11:32.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9646000000000004e-05 2024-09-13T10:11:32.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731341+0000", 2024-09-13T10:11:32.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9098999999999994e-05 2024-09-13T10:11:32.723 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.724 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.724 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.724 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:150 4.4 e38/31 4:24275569:::smithi19530714-38:head v 38'11, mlcod=38'9)", 2024-09-13T10:11:32.724 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.815678+0000", 2024-09-13T10:11:32.724 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.874645641999997, 2024-09-13T10:11:32.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066202199999999996, 2024-09-13T10:11:32.724 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.724 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.724 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.724 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815678+0000", 2024-09-13T10:11:32.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815678+0000", 2024-09-13T10:11:32.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815689+0000", 2024-09-13T10:11:32.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1334e-05 2024-09-13T10:11:32.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816641+0000", 2024-09-13T10:11:32.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000952307 2024-09-13T10:11:32.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816667+0000", 2024-09-13T10:11:32.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5636e-05 2024-09-13T10:11:32.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816679+0000", 2024-09-13T10:11:32.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2373e-05 2024-09-13T10:11:32.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816864+0000", 2024-09-13T10:11:32.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018461399999999999 2024-09-13T10:11:32.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816918+0000", 2024-09-13T10:11:32.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4413999999999998e-05 2024-09-13T10:11:32.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822241+0000", 2024-09-13T10:11:32.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053230430000000004 2024-09-13T10:11:32.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822298+0000", 2024-09-13T10:11:32.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6499000000000002e-05 2024-09-13T10:11:32.729 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.729 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.729 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.729 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:157 4.0 4:03fab808:::smithi19530714-40:head [write 655473~716476,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.730 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.836742+0000", 2024-09-13T10:11:32.730 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.853580745999999, 2024-09-13T10:11:32.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0076813209999999996, 2024-09-13T10:11:32.730 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.730 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.730 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.730 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.730 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.730 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.730 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 157 2024-09-13T10:11:32.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.730 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.836742+0000", 2024-09-13T10:11:32.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.836742+0000", 2024-09-13T10:11:32.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.836746+0000", 2024-09-13T10:11:32.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1930000000000002e-06 2024-09-13T10:11:32.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837565+0000", 2024-09-13T10:11:32.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081902899999999998 2024-09-13T10:11:32.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837613+0000", 2024-09-13T10:11:32.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8640999999999997e-05 2024-09-13T10:11:32.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837628+0000", 2024-09-13T10:11:32.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4867e-05 2024-09-13T10:11:32.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837701+0000", 2024-09-13T10:11:32.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2862999999999997e-05 2024-09-13T10:11:32.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837831+0000", 2024-09-13T10:11:32.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012971599999999999 2024-09-13T10:11:32.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:32.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.837974+0000", 2024-09-13T10:11:32.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014289899999999999 2024-09-13T10:11:32.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.839710+0000", 2024-09-13T10:11:32.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00173613 2024-09-13T10:11:32.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.842419+0000", 2024-09-13T10:11:32.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027087539999999998 2024-09-13T10:11:32.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844336+0000", 2024-09-13T10:11:32.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001916947 2024-09-13T10:11:32.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844381+0000", 2024-09-13T10:11:32.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5377000000000003e-05 2024-09-13T10:11:32.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844424+0000", 2024-09-13T10:11:32.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2904999999999997e-05 2024-09-13T10:11:32.736 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.736 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.736 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.736 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:158 4.0 4:03fab808:::smithi19530714-40:head [write 2099033~556440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.736 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.843944+0000", 2024-09-13T10:11:32.736 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.846378852000001, 2024-09-13T10:11:32.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006981821, 2024-09-13T10:11:32.736 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.737 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.737 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.737 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 158 2024-09-13T10:11:32.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.737 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.843944+0000", 2024-09-13T10:11:32.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.843944+0000", 2024-09-13T10:11:32.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.843946+0000", 2024-09-13T10:11:32.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.75e-06 2024-09-13T10:11:32.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844474+0000", 2024-09-13T10:11:32.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00052765500000000003 2024-09-13T10:11:32.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844496+0000", 2024-09-13T10:11:32.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2552e-05 2024-09-13T10:11:32.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844509+0000", 2024-09-13T10:11:32.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2387e-05 2024-09-13T10:11:32.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844578+0000", 2024-09-13T10:11:32.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9664000000000003e-05 2024-09-13T10:11:32.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844657+0000", 2024-09-13T10:11:32.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8417000000000005e-05 2024-09-13T10:11:32.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:32.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844784+0000", 2024-09-13T10:11:32.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012675800000000001 2024-09-13T10:11:32.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.846495+0000", 2024-09-13T10:11:32.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017114350000000001 2024-09-13T10:11:32.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.848918+0000", 2024-09-13T10:11:32.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002422783 2024-09-13T10:11:32.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.850771+0000", 2024-09-13T10:11:32.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001853616 2024-09-13T10:11:32.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.850859+0000", 2024-09-13T10:11:32.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7786000000000003e-05 2024-09-13T10:11:32.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.850926+0000", 2024-09-13T10:11:32.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7018000000000001e-05 2024-09-13T10:11:32.743 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.743 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.743 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.743 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:159 4.0 4:03fab808:::smithi19530714-40:head [setxattr _header (56) in=7b,truncate 2655473] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.743 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.844538+0000", 2024-09-13T10:11:32.743 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.845785386000003, 2024-09-13T10:11:32.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067064480000000003, 2024-09-13T10:11:32.743 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.743 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.743 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.744 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.744 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.744 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.744 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 159 2024-09-13T10:11:32.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.744 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844538+0000", 2024-09-13T10:11:32.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844538+0000", 2024-09-13T10:11:32.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844539+0000", 2024-09-13T10:11:32.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.223e-06 2024-09-13T10:11:32.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844545+0000", 2024-09-13T10:11:32.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8100000000000003e-06 2024-09-13T10:11:32.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844555+0000", 2024-09-13T10:11:32.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6490000000000003e-06 2024-09-13T10:11:32.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844563+0000", 2024-09-13T10:11:32.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4370000000000007e-06 2024-09-13T10:11:32.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.845602+0000", 2024-09-13T10:11:32.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00103863 2024-09-13T10:11:32.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.845695+0000", 2024-09-13T10:11:32.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2913999999999998e-05 2024-09-13T10:11:32.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:32.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.845812+0000", 2024-09-13T10:11:32.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011715899999999999 2024-09-13T10:11:32.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.846904+0000", 2024-09-13T10:11:32.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001092733 2024-09-13T10:11:32.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.849487+0000", 2024-09-13T10:11:32.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025827459999999999 2024-09-13T10:11:32.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851147+0000", 2024-09-13T10:11:32.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001659878 2024-09-13T10:11:32.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851184+0000", 2024-09-13T10:11:32.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7435999999999999e-05 2024-09-13T10:11:32.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851244+0000", 2024-09-13T10:11:32.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9833000000000001e-05 2024-09-13T10:11:32.750 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.750 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.750 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.750 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:160 4.0 4:03fab808:::smithi19530714-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.750 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.844579+0000", 2024-09-13T10:11:32.750 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.845743798000001, 2024-09-13T10:11:32.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.006862506, 2024-09-13T10:11:32.750 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.751 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.751 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:32.751 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.751 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.751 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.751 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 160 2024-09-13T10:11:32.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.751 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844579+0000", 2024-09-13T10:11:32.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844579+0000", 2024-09-13T10:11:32.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844580+0000", 2024-09-13T10:11:32.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1699999999999997e-07 2024-09-13T10:11:32.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844585+0000", 2024-09-13T10:11:32.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6360000000000002e-06 2024-09-13T10:11:32.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844595+0000", 2024-09-13T10:11:32.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9739999999999997e-06 2024-09-13T10:11:32.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.844604+0000", 2024-09-13T10:11:32.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7560000000000005e-06 2024-09-13T10:11:32.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.846062+0000", 2024-09-13T10:11:32.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001458446 2024-09-13T10:11:32.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:32.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.846121+0000", 2024-09-13T10:11:32.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8697999999999999e-05 2024-09-13T10:11:32.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851297+0000", 2024-09-13T10:11:32.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051764810000000001 2024-09-13T10:11:32.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851355+0000", 2024-09-13T10:11:32.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7794999999999998e-05 2024-09-13T10:11:32.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.851442+0000", 2024-09-13T10:11:32.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6903000000000005e-05 2024-09-13T10:11:32.755 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.755 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.755 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.756 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:198 4.0 4:0bd0428c:::smithi19530714-50:head [write 1767500~799008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.756 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991140+0000", 2024-09-13T10:11:32.756 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.699182809, 2024-09-13T10:11:32.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0079451699999999997, 2024-09-13T10:11:32.756 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.756 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.756 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.756 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.756 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.756 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.756 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 198 2024-09-13T10:11:32.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.757 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991140+0000", 2024-09-13T10:11:32.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991140+0000", 2024-09-13T10:11:32.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991144+0000", 2024-09-13T10:11:32.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2499999999999998e-06 2024-09-13T10:11:32.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992036+0000", 2024-09-13T10:11:32.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000891931 2024-09-13T10:11:32.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992052+0000", 2024-09-13T10:11:32.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6070999999999999e-05 2024-09-13T10:11:32.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992060+0000", 2024-09-13T10:11:32.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9719999999999992e-06 2024-09-13T10:11:32.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992128+0000", 2024-09-13T10:11:32.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7858e-05 2024-09-13T10:11:32.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992208+0000", 2024-09-13T10:11:32.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0383000000000006e-05 2024-09-13T10:11:32.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:32.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992324+0000", 2024-09-13T10:11:32.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011587 2024-09-13T10:11:32.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997567+0000", 2024-09-13T10:11:32.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052430419999999998 2024-09-13T10:11:32.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.998107+0000", 2024-09-13T10:11:32.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054063099999999997 2024-09-13T10:11:32.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.998940+0000", 2024-09-13T10:11:32.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083243599999999998 2024-09-13T10:11:32.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.999021+0000", 2024-09-13T10:11:32.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1335999999999995e-05 2024-09-13T10:11:32.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.999086+0000", 2024-09-13T10:11:32.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4389999999999998e-05 2024-09-13T10:11:32.762 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.762 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.762 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.763 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:199 4.0 4:0bd0428c:::smithi19530714-50:head [setxattr _header (56) in=7b,truncate 2823543] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.763 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.992081+0000", 2024-09-13T10:11:32.763 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.698242307999998, 2024-09-13T10:11:32.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011866829000000001, 2024-09-13T10:11:32.763 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.763 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.763 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.763 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.763 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.763 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.763 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 199 2024-09-13T10:11:32.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.764 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992081+0000", 2024-09-13T10:11:32.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992081+0000", 2024-09-13T10:11:32.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992082+0000", 2024-09-13T10:11:32.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.004e-06 2024-09-13T10:11:32.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992090+0000", 2024-09-13T10:11:32.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7440000000000007e-06 2024-09-13T10:11:32.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:11:32.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3650000000000004e-06 2024-09-13T10:11:32.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992107+0000", 2024-09-13T10:11:32.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9230000000000008e-06 2024-09-13T10:11:32.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.995938+0000", 2024-09-13T10:11:32.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003830773 2024-09-13T10:11:32.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.996229+0000", 2024-09-13T10:11:32.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029133300000000001 2024-09-13T10:11:32.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:32.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.996692+0000", 2024-09-13T10:11:32.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046326700000000001 2024-09-13T10:11:32.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997974+0000", 2024-09-13T10:11:32.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001282016 2024-09-13T10:11:32.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.999243+0000", 2024-09-13T10:11:32.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012682360000000001 2024-09-13T10:11:32.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.003767+0000", 2024-09-13T10:11:32.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045243970000000003 2024-09-13T10:11:32.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.003851+0000", 2024-09-13T10:11:32.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4195000000000003e-05 2024-09-13T10:11:32.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.003948+0000", 2024-09-13T10:11:32.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6576000000000005e-05 2024-09-13T10:11:32.769 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.769 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.769 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.769 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:200 4.0 4:0bd0428c:::smithi19530714-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.769 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.992119+0000", 2024-09-13T10:11:32.770 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.698204148000002, 2024-09-13T10:11:32.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012149748, 2024-09-13T10:11:32.770 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.770 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.770 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:32.770 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.770 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.770 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.770 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 200 2024-09-13T10:11:32.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.770 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992119+0000", 2024-09-13T10:11:32.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992119+0000", 2024-09-13T10:11:32.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992120+0000", 2024-09-13T10:11:32.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9999999999999997e-07 2024-09-13T10:11:32.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992123+0000", 2024-09-13T10:11:32.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9569999999999998e-06 2024-09-13T10:11:32.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992131+0000", 2024-09-13T10:11:32.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2024-09-13T10:11:32.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992140+0000", 2024-09-13T10:11:32.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7900000000000005e-06 2024-09-13T10:11:32.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997418+0000", 2024-09-13T10:11:32.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052784349999999997 2024-09-13T10:11:32.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:32.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997546+0000", 2024-09-13T10:11:32.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012788900000000001 2024-09-13T10:11:32.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.004004+0000", 2024-09-13T10:11:32.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064580549999999999 2024-09-13T10:11:32.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.004128+0000", 2024-09-13T10:11:32.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123628 2024-09-13T10:11:32.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.004269+0000", 2024-09-13T10:11:32.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000141292 2024-09-13T10:11:32.775 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.775 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.775 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.775 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:265 4.0 4:0b4c105e:::smithi19530714-16:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.775 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.215372+0000", 2024-09-13T10:11:32.775 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.474951116, 2024-09-13T10:11:32.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013030243, 2024-09-13T10:11:32.775 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.775 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.776 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.776 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.776 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.776 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.776 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 265 2024-09-13T10:11:32.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.776 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215372+0000", 2024-09-13T10:11:32.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215372+0000", 2024-09-13T10:11:32.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215376+0000", 2024-09-13T10:11:32.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.393e-06 2024-09-13T10:11:32.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215387+0000", 2024-09-13T10:11:32.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1986e-05 2024-09-13T10:11:32.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215421+0000", 2024-09-13T10:11:32.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3355999999999997e-05 2024-09-13T10:11:32.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215438+0000", 2024-09-13T10:11:32.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7059e-05 2024-09-13T10:11:32.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215529+0000", 2024-09-13T10:11:32.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1571999999999997e-05 2024-09-13T10:11:32.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215621+0000", 2024-09-13T10:11:32.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1094999999999994e-05 2024-09-13T10:11:32.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:32.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.223688+0000", 2024-09-13T10:11:32.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080669859999999999 2024-09-13T10:11:32.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.225291+0000", 2024-09-13T10:11:32.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001603548 2024-09-13T10:11:32.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.225833+0000", 2024-09-13T10:11:32.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054185500000000005 2024-09-13T10:11:32.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228151+0000", 2024-09-13T10:11:32.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023176640000000001 2024-09-13T10:11:32.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228300+0000", 2024-09-13T10:11:32.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000149152 2024-09-13T10:11:32.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228402+0000", 2024-09-13T10:11:32.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102577 2024-09-13T10:11:32.782 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.782 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.782 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.782 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:289 4.0 4:03fab808:::smithi19530714-40:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.782 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.250617+0000", 2024-09-13T10:11:32.782 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.439705830000001, 2024-09-13T10:11:32.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010740764999999999, 2024-09-13T10:11:32.782 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.783 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.783 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.783 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.783 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.783 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.783 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 289 2024-09-13T10:11:32.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.783 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250617+0000", 2024-09-13T10:11:32.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250617+0000", 2024-09-13T10:11:32.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250621+0000", 2024-09-13T10:11:32.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4029999999999999e-06 2024-09-13T10:11:32.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250637+0000", 2024-09-13T10:11:32.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5950000000000001e-05 2024-09-13T10:11:32.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250673+0000", 2024-09-13T10:11:32.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6559e-05 2024-09-13T10:11:32.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250699+0000", 2024-09-13T10:11:32.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5479000000000001e-05 2024-09-13T10:11:32.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250825+0000", 2024-09-13T10:11:32.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012585600000000001 2024-09-13T10:11:32.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250971+0000", 2024-09-13T10:11:32.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000146002 2024-09-13T10:11:32.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:32.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257715+0000", 2024-09-13T10:11:32.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067440360000000001 2024-09-13T10:11:32.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258695+0000", 2024-09-13T10:11:32.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0009798509999999999 2024-09-13T10:11:32.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.260555+0000", 2024-09-13T10:11:32.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018600439999999999 2024-09-13T10:11:32.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261249+0000", 2024-09-13T10:11:32.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00069435899999999997 2024-09-13T10:11:32.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261307+0000", 2024-09-13T10:11:32.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7630000000000002e-05 2024-09-13T10:11:32.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261358+0000", 2024-09-13T10:11:32.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1595999999999999e-05 2024-09-13T10:11:32.789 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.789 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.789 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.789 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:299 4.0 4:0bd0428c:::smithi19530714-50:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:32.789 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.264107+0000", 2024-09-13T10:11:32.789 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.426216441000001, 2024-09-13T10:11:32.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020961186999999999, 2024-09-13T10:11:32.789 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.789 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.789 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.790 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.790 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.790 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.790 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 299 2024-09-13T10:11:32.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.790 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264107+0000", 2024-09-13T10:11:32.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264107+0000", 2024-09-13T10:11:32.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264111+0000", 2024-09-13T10:11:32.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0330000000000002e-06 2024-09-13T10:11:32.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264121+0000", 2024-09-13T10:11:32.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0543999999999999e-05 2024-09-13T10:11:32.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264150+0000", 2024-09-13T10:11:32.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8581999999999998e-05 2024-09-13T10:11:32.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264171+0000", 2024-09-13T10:11:32.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-13T10:11:32.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264369+0000", 2024-09-13T10:11:32.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019837000000000001 2024-09-13T10:11:32.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264535+0000", 2024-09-13T10:11:32.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000165324 2024-09-13T10:11:32.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:32.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.276997+0000", 2024-09-13T10:11:32.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012461842000000001 2024-09-13T10:11:32.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.277946+0000", 2024-09-13T10:11:32.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00094919299999999998 2024-09-13T10:11:32.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.281540+0000", 2024-09-13T10:11:32.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035946910000000001 2024-09-13T10:11:32.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.284932+0000", 2024-09-13T10:11:32.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033913799999999998 2024-09-13T10:11:32.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285010+0000", 2024-09-13T10:11:32.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8059000000000006e-05 2024-09-13T10:11:32.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285068+0000", 2024-09-13T10:11:32.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8072999999999998e-05 2024-09-13T10:11:32.795 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.795 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.795 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.796 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:489 4.0 4:03fab808:::smithi19530714-40:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:32.796 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.350324+0000", 2024-09-13T10:11:32.796 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.339999423999998, 2024-09-13T10:11:32.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014394344, 2024-09-13T10:11:32.796 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.796 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.796 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.796 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.796 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.796 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.796 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 489 2024-09-13T10:11:32.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.797 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350324+0000", 2024-09-13T10:11:32.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350324+0000", 2024-09-13T10:11:32.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350325+0000", 2024-09-13T10:11:32.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6270000000000001e-06 2024-09-13T10:11:32.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350333+0000", 2024-09-13T10:11:32.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3719999999999999e-06 2024-09-13T10:11:32.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350349+0000", 2024-09-13T10:11:32.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6004000000000001e-05 2024-09-13T10:11:32.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350359+0000", 2024-09-13T10:11:32.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.73e-06 2024-09-13T10:11:32.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350455+0000", 2024-09-13T10:11:32.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6260000000000003e-05 2024-09-13T10:11:32.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350526+0000", 2024-09-13T10:11:32.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0946999999999997e-05 2024-09-13T10:11:32.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:32.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.357046+0000", 2024-09-13T10:11:32.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065198510000000001 2024-09-13T10:11:32.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.357616+0000", 2024-09-13T10:11:32.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056994299999999995 2024-09-13T10:11:32.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.358085+0000", 2024-09-13T10:11:32.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046959399999999999 2024-09-13T10:11:32.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.364636+0000", 2024-09-13T10:11:32.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065509560000000001 2024-09-13T10:11:32.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.364680+0000", 2024-09-13T10:11:32.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3859999999999997e-05 2024-09-13T10:11:32.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.364718+0000", 2024-09-13T10:11:32.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.82e-05 2024-09-13T10:11:32.802 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.802 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.802 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.802 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.6.0:77 4.4 e39/31 4:244eab67:::smithi19530714-24:head v 39'35, mlcod=39'33)", 2024-09-13T10:11:32.802 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.572069+0000", 2024-09-13T10:11:32.802 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.118254057000001, 2024-09-13T10:11:32.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072872479999999996, 2024-09-13T10:11:32.803 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.803 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.803 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.803 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.572069+0000", 2024-09-13T10:11:32.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.572069+0000", 2024-09-13T10:11:32.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.572072+0000", 2024-09-13T10:11:32.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.909e-06 2024-09-13T10:11:32.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.574286+0000", 2024-09-13T10:11:32.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022137580000000001 2024-09-13T10:11:32.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.574318+0000", 2024-09-13T10:11:32.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2582000000000001e-05 2024-09-13T10:11:32.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.574335+0000", 2024-09-13T10:11:32.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6906999999999999e-05 2024-09-13T10:11:32.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.574487+0000", 2024-09-13T10:11:32.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015193499999999999 2024-09-13T10:11:32.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.574552+0000", 2024-09-13T10:11:32.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4721000000000006e-05 2024-09-13T10:11:32.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.579275+0000", 2024-09-13T10:11:32.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004723222 2024-09-13T10:11:32.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.579356+0000", 2024-09-13T10:11:32.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1213999999999999e-05 2024-09-13T10:11:32.807 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.807 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.807 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.807 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:1587 4.4 e47/41 4:24da8222:::smithi19530714-49:head v 47'70, mlcod=47'68)", 2024-09-13T10:11:32.807 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:05.127296+0000", 2024-09-13T10:11:32.807 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.563027002999998, 2024-09-13T10:11:32.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0066656880000000003, 2024-09-13T10:11:32.807 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.807 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.808 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.808 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.127296+0000", 2024-09-13T10:11:32.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.127296+0000", 2024-09-13T10:11:32.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.127298+0000", 2024-09-13T10:11:32.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0030000000000001e-06 2024-09-13T10:11:32.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.128661+0000", 2024-09-13T10:11:32.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001362858 2024-09-13T10:11:32.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.128690+0000", 2024-09-13T10:11:32.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9076000000000001e-05 2024-09-13T10:11:32.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.128705+0000", 2024-09-13T10:11:32.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4459e-05 2024-09-13T10:11:32.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.128856+0000", 2024-09-13T10:11:32.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000151273 2024-09-13T10:11:32.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.128918+0000", 2024-09-13T10:11:32.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2001999999999994e-05 2024-09-13T10:11:32.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.133766+0000", 2024-09-13T10:11:32.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048483850000000002 2024-09-13T10:11:32.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.133962+0000", 2024-09-13T10:11:32.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000195632 2024-09-13T10:11:32.812 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.812 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.812 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.812 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1919 4.0 4:03fab808:::smithi19530714-40:head [assert-version v17,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:32.812 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:11.758517+0000", 2024-09-13T10:11:32.812 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.931806008999999, 2024-09-13T10:11:32.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082282680000000004, 2024-09-13T10:11:32.812 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.812 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.812 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:32.813 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.813 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.813 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.813 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1919 2024-09-13T10:11:32.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.813 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.758517+0000", 2024-09-13T10:11:32.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.758517+0000", 2024-09-13T10:11:32.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.758520+0000", 2024-09-13T10:11:32.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8499999999999998e-06 2024-09-13T10:11:32.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.758534+0000", 2024-09-13T10:11:32.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3920000000000001e-05 2024-09-13T10:11:32.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.758566+0000", 2024-09-13T10:11:32.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2475000000000003e-05 2024-09-13T10:11:32.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.758582+0000", 2024-09-13T10:11:32.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6033999999999999e-05 2024-09-13T10:11:32.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.758717+0000", 2024-09-13T10:11:32.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013453299999999999 2024-09-13T10:11:32.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.758944+0000", 2024-09-13T10:11:32.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000227252 2024-09-13T10:11:32.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:32.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.763328+0000", 2024-09-13T10:11:32.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043840880000000004 2024-09-13T10:11:32.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.765136+0000", 2024-09-13T10:11:32.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018076850000000001 2024-09-13T10:11:32.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:32.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.765180+0000", 2024-09-13T10:11:32.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4119000000000002e-05 2024-09-13T10:11:32.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:32.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.766547+0000", 2024-09-13T10:11:32.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001367006 2024-09-13T10:11:32.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:32.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.766637+0000", 2024-09-13T10:11:32.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0241999999999994e-05 2024-09-13T10:11:32.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:11.766745+0000", 2024-09-13T10:11:32.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108064 2024-09-13T10:11:32.818 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.818 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.818 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.819 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2174 4.0 4:03fab808:::smithi19530714-40:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:11:32.819 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:17.423281+0000", 2024-09-13T10:11:32.819 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.267042281, 2024-09-13T10:11:32.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008981826, 2024-09-13T10:11:32.819 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:32.819 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:32.819 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:32.819 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:32.819 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:32.819 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:32.820 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2174 2024-09-13T10:11:32.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.820 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:32.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:32.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:17.423281+0000", 2024-09-13T10:11:32.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:32.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:17.423281+0000", 2024-09-13T10:11:32.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:32.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:32.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:17.423284+0000", 2024-09-13T10:11:32.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.027e-06 2024-09-13T10:11:32.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:32.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:17.423297+0000", 2024-09-13T10:11:32.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2669e-05 2024-09-13T10:11:32.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:32.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:17.423328+0000", 2024-09-13T10:11:32.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1603999999999998e-05 2024-09-13T10:11:32.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:32.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:17.423346+0000", 2024-09-13T10:11:32.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7589e-05 2024-09-13T10:11:32.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:32.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:17.423502+0000", 2024-09-13T10:11:32.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015632999999999999 2024-09-13T10:11:32.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:32.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:17.423661+0000", 2024-09-13T10:11:32.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015840200000000001 2024-09-13T10:11:32.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:32.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:32.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:32.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:17.432263+0000", 2024-09-13T10:11:32.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086022049999999999 2024-09-13T10:11:32.823 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.823 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.823 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.823 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:32.823 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:32.824 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:32.825 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1568798 2024-09-13T10:11:32.825 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-13T10:11:32.826 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:32.823+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:32.826 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2129586 2024-09-13T10:11:32.863 INFO:tasks.rados.rados.0.smithi195.stdout:1288: finishing set_chunk to oid 47 2024-09-13T10:11:32.863 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 45 (ObjNum 182 snap 0 seq_num 182) dirty exists 2024-09-13T10:11:32.863 INFO:tasks.rados.rados.0.smithi195.stdout:1286: done (4 left) 2024-09-13T10:11:32.863 INFO:tasks.rados.rados.0.smithi195.stdout:1287: done (3 left) 2024-09-13T10:11:32.863 INFO:tasks.rados.rados.0.smithi195.stdout:1288: done (2 left) 2024-09-13T10:11:32.863 INFO:tasks.rados.rados.0.smithi195.stdout:1289: done (1 left) 2024-09-13T10:11:32.863 INFO:tasks.rados.rados.0.smithi195.stdout:1290: set_chunk oid 20 target oid 2024-09-13T10:11:32.893 INFO:tasks.rados.rados.0.smithi195.stdout:1290: set_chunk oid 20 offset: 2291200 length: 1359872 target oid 1fa676dc3d1e506ea93eaeb504791933fdde520bb2cba670c5c248da82aeece8 offset: 0 snap -1 2024-09-13T10:11:32.893 INFO:tasks.rados.rados.0.smithi195.stdout:1291: read oid 11 snap -1 2024-09-13T10:11:32.893 INFO:tasks.rados.rados.0.smithi195.stdout:1291: expect (ObjNum 217 snap 0 seq_num 217) 2024-09-13T10:11:32.893 INFO:tasks.rados.rados.0.smithi195.stdout:1292: read oid 19 snap -1 2024-09-13T10:11:32.893 INFO:tasks.rados.rados.0.smithi195.stdout:1292: expect (ObjNum 160 snap 0 seq_num 160) 2024-09-13T10:11:32.893 INFO:tasks.rados.rados.0.smithi195.stdout:1293: tier_evict oid 5 2024-09-13T10:11:32.893 INFO:tasks.rados.rados.0.smithi195.stdout:1293: tier_evict oid 5 snap -1 2024-09-13T10:11:32.893 INFO:tasks.rados.rados.0.smithi195.stdout:1294: tier_evict oid 14 2024-09-13T10:11:32.893 INFO:tasks.rados.rados.0.smithi195.stdout:1294: tier_evict oid 14 snap -1 2024-09-13T10:11:32.894 INFO:tasks.rados.rados.0.smithi195.stdout:1295: read oid 9 snap -1 2024-09-13T10:11:32.894 INFO:tasks.rados.rados.0.smithi195.stdout:1295: expect deleted 2024-09-13T10:11:32.894 INFO:tasks.rados.rados.0.smithi195.stdout:1296: read oid 28 snap -1 2024-09-13T10:11:32.894 INFO:tasks.rados.rados.0.smithi195.stdout:1296: expect (ObjNum 180 snap 0 seq_num 180) 2024-09-13T10:11:32.894 INFO:tasks.rados.rados.0.smithi195.stdout:1297: read oid 25 snap -1 2024-09-13T10:11:32.894 INFO:tasks.rados.rados.0.smithi195.stdout:1297: expect (ObjNum 215 snap 0 seq_num 215) 2024-09-13T10:11:32.894 INFO:tasks.rados.rados.0.smithi195.stdout:1298: read oid 31 snap -1 2024-09-13T10:11:32.894 INFO:tasks.rados.rados.0.smithi195.stdout:1298: expect (ObjNum 227 snap 0 seq_num 227) 2024-09-13T10:11:32.894 INFO:tasks.rados.rados.0.smithi195.stdout:1299: read oid 16 snap -1 2024-09-13T10:11:32.894 INFO:tasks.rados.rados.0.smithi195.stdout:1299: expect (ObjNum 128 snap 0 seq_num 128) 2024-09-13T10:11:32.894 INFO:tasks.rados.rados.0.smithi195.stdout:1300: read oid 49 snap -1 2024-09-13T10:11:32.894 INFO:tasks.rados.rados.0.smithi195.stdout:1300: expect (ObjNum 181 snap 0 seq_num 181) 2024-09-13T10:11:32.894 INFO:tasks.rados.rados.0.smithi195.stdout:1301: read oid 26 snap -1 2024-09-13T10:11:32.895 INFO:tasks.rados.rados.0.smithi195.stdout:1301: expect (ObjNum 244 snap 0 seq_num 244) 2024-09-13T10:11:32.895 INFO:tasks.rados.rados.0.smithi195.stdout:1302: tier_evict oid 1 2024-09-13T10:11:32.895 INFO:tasks.rados.rados.0.smithi195.stdout:1302: tier_evict oid 1 snap -1 2024-09-13T10:11:32.895 INFO:tasks.rados.rados.0.smithi195.stdout:1303: read oid 18 snap -1 2024-09-13T10:11:32.895 INFO:tasks.rados.rados.0.smithi195.stdout:1303: expect (ObjNum 189 snap 0 seq_num 189) 2024-09-13T10:11:32.895 INFO:tasks.rados.rados.0.smithi195.stdout:1304: write oid 2 current snap is 0 2024-09-13T10:11:32.895 INFO:tasks.rados.rados.0.smithi195.stdout:1293: got (22) Invalid argument 2024-09-13T10:11:32.895 INFO:tasks.rados.rados.0.smithi195.stdout:1304: seq_num 249 ranges {584943=460297,1587781=597162} 2024-09-13T10:11:32.916 INFO:tasks.rados.rados.0.smithi195.stdout:1304: writing smithi19530714-2 from 584943 to 1045240 tid 1 2024-09-13T10:11:32.926 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:32.920+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:32.936 INFO:tasks.rados.rados.0.smithi195.stdout:1304: writing smithi19530714-2 from 1587781 to 2184943 tid 2 2024-09-13T10:11:32.936 INFO:tasks.rados.rados.0.smithi195.stdout:1293: done (15 left) 2024-09-13T10:11:32.936 INFO:tasks.rados.rados.0.smithi195.stdout:1305: read oid 33 snap -1 2024-09-13T10:11:32.936 INFO:tasks.rados.rados.0.smithi195.stdout:1305: expect deleted 2024-09-13T10:11:32.936 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:32.936 INFO:tasks.rados.rados.0.smithi195.stdout:1294: got (22) Invalid argument 2024-09-13T10:11:32.936 INFO:tasks.rados.rados.0.smithi195.stdout:1299: expect (ObjNum 128 snap 0 seq_num 128) 2024-09-13T10:11:32.993 INFO:tasks.rados.rados.0.smithi195.stdout:1273: expect (ObjNum 239 snap 0 seq_num 239) 2024-09-13T10:11:33.002 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:33.002 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:33.002 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:11:33.003 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:33.018 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-13T10:11:33.027 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:33.024+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:33.030 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T10:11:33.096 INFO:tasks.rados.rados.0.smithi195.stdout:1301: expect (ObjNum 244 snap 0 seq_num 244) 2024-09-13T10:11:33.105 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 758337 2024-09-13T10:11:33.128 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:33.124+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:33.140 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:33.140 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:33.140 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:11:33.140 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:11:33.140 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:33.154 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-13T10:11:33.172 INFO:tasks.rados.rados.0.smithi195.stdout:1303: expect (ObjNum 189 snap 0 seq_num 189) 2024-09-13T10:11:33.229 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:33.224+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:33.320 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:33.320 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:11:33.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:11:33.320 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:11:33.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4365.0:5 1.2 e26/24 1:5cea7035:::rbd_info:head v 26'1, mlcod=0'0)", 2024-09-13T10:11:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:31.142893+0000", 2024-09-13T10:11:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 62.174483252000002, 2024-09-13T10:11:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024212330000000001, 2024-09-13T10:11:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142893+0000", 2024-09-13T10:11:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142893+0000", 2024-09-13T10:11:33.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142893+0000", 2024-09-13T10:11:33.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1699999999999998e-07 2024-09-13T10:11:33.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142914+0000", 2024-09-13T10:11:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0378999999999999e-05 2024-09-13T10:11:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142951+0000", 2024-09-13T10:11:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7471000000000001e-05 2024-09-13T10:11:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.142983+0000", 2024-09-13T10:11:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2422000000000002e-05 2024-09-13T10:11:33.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.143202+0000", 2024-09-13T10:11:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021875799999999999 2024-09-13T10:11:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.143308+0000", 2024-09-13T10:11:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105487 2024-09-13T10:11:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.145207+0000", 2024-09-13T10:11:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001899848 2024-09-13T10:11:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:31.145314+0000", 2024-09-13T10:11:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106351 2024-09-13T10:11:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.325 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:89 4.9 e38/31 4:96fcf081:::smithi19530714-23:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.811705+0000", 2024-09-13T10:11:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.505671006999997, 2024-09-13T10:11:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053533510000000001, 2024-09-13T10:11:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.811705+0000", 2024-09-13T10:11:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.811705+0000", 2024-09-13T10:11:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.811706+0000", 2024-09-13T10:11:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7700000000000003e-07 2024-09-13T10:11:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813717+0000", 2024-09-13T10:11:33.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002011494 2024-09-13T10:11:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813743+0000", 2024-09-13T10:11:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5528999999999999e-05 2024-09-13T10:11:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813759+0000", 2024-09-13T10:11:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6178999999999999e-05 2024-09-13T10:11:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813873+0000", 2024-09-13T10:11:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000113863 2024-09-13T10:11:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.813922+0000", 2024-09-13T10:11:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9558999999999998e-05 2024-09-13T10:11:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.816994+0000", 2024-09-13T10:11:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003071299 2024-09-13T10:11:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817058+0000", 2024-09-13T10:11:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4550999999999999e-05 2024-09-13T10:11:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:90 4.9 e38/31 4:96fcf081:::smithi19530714-23:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:33.330 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.815985+0000", 2024-09-13T10:11:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.501390671999999, 2024-09-13T10:11:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00489822, 2024-09-13T10:11:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815985+0000", 2024-09-13T10:11:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815985+0000", 2024-09-13T10:11:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.815987+0000", 2024-09-13T10:11:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.277e-06 2024-09-13T10:11:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817028+0000", 2024-09-13T10:11:33.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001040664 2024-09-13T10:11:33.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817055+0000", 2024-09-13T10:11:33.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6835999999999999e-05 2024-09-13T10:11:33.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817074+0000", 2024-09-13T10:11:33.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8629000000000001e-05 2024-09-13T10:11:33.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817153+0000", 2024-09-13T10:11:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9794000000000002e-05 2024-09-13T10:11:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817183+0000", 2024-09-13T10:11:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9479e-05 2024-09-13T10:11:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820783+0000", 2024-09-13T10:11:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035997720000000002 2024-09-13T10:11:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820883+0000", 2024-09-13T10:11:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100769 2024-09-13T10:11:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:91 4.9 e38/31 4:96fcf081:::smithi19530714-23:head v 38'3, mlcod=0'0)", 2024-09-13T10:11:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.817117+0000", 2024-09-13T10:11:33.335 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.500258467000002, 2024-09-13T10:11:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038490590000000002, 2024-09-13T10:11:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817117+0000", 2024-09-13T10:11:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817117+0000", 2024-09-13T10:11:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817119+0000", 2024-09-13T10:11:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1999999999999999e-06 2024-09-13T10:11:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817130+0000", 2024-09-13T10:11:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1313e-05 2024-09-13T10:11:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817145+0000", 2024-09-13T10:11:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4786e-05 2024-09-13T10:11:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817157+0000", 2024-09-13T10:11:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2775999999999999e-05 2024-09-13T10:11:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818394+0000", 2024-09-13T10:11:33.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012367999999999999 2024-09-13T10:11:33.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818496+0000", 2024-09-13T10:11:33.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102098 2024-09-13T10:11:33.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820895+0000", 2024-09-13T10:11:33.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023989189999999998 2024-09-13T10:11:33.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820966+0000", 2024-09-13T10:11:33.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1167000000000005e-05 2024-09-13T10:11:33.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.340 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.340 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:129 4.9 e38/31 4:998d9bf1:::smithi19530714-33:head v 38'4, mlcod=0'0)", 2024-09-13T10:11:33.340 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.817768+0000", 2024-09-13T10:11:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.499607662999999, 2024-09-13T10:11:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047213589999999996, 2024-09-13T10:11:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817768+0000", 2024-09-13T10:11:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817768+0000", 2024-09-13T10:11:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.817769+0000", 2024-09-13T10:11:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.184e-06 2024-09-13T10:11:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818638+0000", 2024-09-13T10:11:33.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00086842199999999999 2024-09-13T10:11:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818665+0000", 2024-09-13T10:11:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6979e-05 2024-09-13T10:11:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818685+0000", 2024-09-13T10:11:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0063000000000001e-05 2024-09-13T10:11:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.818994+0000", 2024-09-13T10:11:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030959500000000002 2024-09-13T10:11:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819019+0000", 2024-09-13T10:11:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4508999999999999e-05 2024-09-13T10:11:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822389+0000", 2024-09-13T10:11:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033701970000000001 2024-09-13T10:11:33.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822490+0000", 2024-09-13T10:11:33.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010041 2024-09-13T10:11:33.345 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.345 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.345 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.345 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:130 4.9 e38/31 4:998d9bf1:::smithi19530714-33:head v 38'5, mlcod=0'0)", 2024-09-13T10:11:33.345 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.819657+0000", 2024-09-13T10:11:33.346 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.497718699000004, 2024-09-13T10:11:33.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032383609999999999, 2024-09-13T10:11:33.346 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.346 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.346 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.346 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819657+0000", 2024-09-13T10:11:33.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819657+0000", 2024-09-13T10:11:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.819659+0000", 2024-09-13T10:11:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.624e-06 2024-09-13T10:11:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820879+0000", 2024-09-13T10:11:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001220184 2024-09-13T10:11:33.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820906+0000", 2024-09-13T10:11:33.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6536000000000001e-05 2024-09-13T10:11:33.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.820930+0000", 2024-09-13T10:11:33.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4552000000000001e-05 2024-09-13T10:11:33.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821017+0000", 2024-09-13T10:11:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6539999999999995e-05 2024-09-13T10:11:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.821043+0000", 2024-09-13T10:11:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6018999999999999e-05 2024-09-13T10:11:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822848+0000", 2024-09-13T10:11:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001805673 2024-09-13T10:11:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.822896+0000", 2024-09-13T10:11:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7233e-05 2024-09-13T10:11:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:185 4.9 e38/31 4:9bec18ef:::smithi19530714-47:head v 38'7, mlcod=38'5)", 2024-09-13T10:11:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.941317+0000", 2024-09-13T10:11:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.376059069, 2024-09-13T10:11:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002782858, 2024-09-13T10:11:33.350 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.941317+0000", 2024-09-13T10:11:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.941317+0000", 2024-09-13T10:11:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.941319+0000", 2024-09-13T10:11:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9860000000000001e-06 2024-09-13T10:11:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942196+0000", 2024-09-13T10:11:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087739999999999997 2024-09-13T10:11:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942230+0000", 2024-09-13T10:11:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4118000000000001e-05 2024-09-13T10:11:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942252+0000", 2024-09-13T10:11:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2030000000000001e-05 2024-09-13T10:11:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942365+0000", 2024-09-13T10:11:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112689 2024-09-13T10:11:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.942398+0000", 2024-09-13T10:11:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2897e-05 2024-09-13T10:11:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.944044+0000", 2024-09-13T10:11:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016460800000000001 2024-09-13T10:11:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.944100+0000", 2024-09-13T10:11:33.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5658000000000001e-05 2024-09-13T10:11:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:186 4.9 e38/31 4:9bec18ef:::smithi19530714-47:head v 38'8, mlcod=38'6)", 2024-09-13T10:11:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.948439+0000", 2024-09-13T10:11:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.368936996000002, 2024-09-13T10:11:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002770018, 2024-09-13T10:11:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.355 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.948439+0000", 2024-09-13T10:11:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.948439+0000", 2024-09-13T10:11:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.948440+0000", 2024-09-13T10:11:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4440000000000001e-06 2024-09-13T10:11:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949519+0000", 2024-09-13T10:11:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010790039999999999 2024-09-13T10:11:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949537+0000", 2024-09-13T10:11:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7312e-05 2024-09-13T10:11:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949545+0000", 2024-09-13T10:11:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2570000000000007e-06 2024-09-13T10:11:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949615+0000", 2024-09-13T10:11:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9614000000000001e-05 2024-09-13T10:11:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.949633+0000", 2024-09-13T10:11:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8640000000000001e-05 2024-09-13T10:11:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.951146+0000", 2024-09-13T10:11:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015129570000000001 2024-09-13T10:11:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.951209+0000", 2024-09-13T10:11:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.279e-05 2024-09-13T10:11:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.359 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(osd.11.0:22 4.19 e42/41 4:998d9bf1:::smithi19530714-33:head v 42'23, mlcod=0'0)", 2024-09-13T10:11:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.624507+0000", 2024-09-13T10:11:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.692868464999997, 2024-09-13T10:11:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039005810000000002, 2024-09-13T10:11:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.624507+0000", 2024-09-13T10:11:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.624507+0000", 2024-09-13T10:11:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.624510+0000", 2024-09-13T10:11:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4770000000000002e-06 2024-09-13T10:11:33.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.625777+0000", 2024-09-13T10:11:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001267589 2024-09-13T10:11:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.625809+0000", 2024-09-13T10:11:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1288999999999998e-05 2024-09-13T10:11:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.625827+0000", 2024-09-13T10:11:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8312999999999999e-05 2024-09-13T10:11:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626008+0000", 2024-09-13T10:11:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000181421 2024-09-13T10:11:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626077+0000", 2024-09-13T10:11:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8471000000000002e-05 2024-09-13T10:11:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.628292+0000", 2024-09-13T10:11:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002214728 2024-09-13T10:11:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.628408+0000", 2024-09-13T10:11:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011629300000000001 2024-09-13T10:11:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "pg_update_log_missing(4.19 epoch 42/41 rep_tid 23 entries 42'24 (0'0) error 4:998d9bf1:::smithi19530714-33:head by client.4607.0:1065 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 40'21 roll_forward_to 0'0)", 2024-09-13T10:11:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.626821+0000", 2024-09-13T10:11:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.690554587000001, 2024-09-13T10:11:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022279040000000002, 2024-09-13T10:11:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:11:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626821+0000", 2024-09-13T10:11:33.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626821+0000", 2024-09-13T10:11:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626823+0000", 2024-09-13T10:11:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.77e-06 2024-09-13T10:11:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626840+0000", 2024-09-13T10:11:33.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7272999999999999e-05 2024-09-13T10:11:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626882+0000", 2024-09-13T10:11:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1936e-05 2024-09-13T10:11:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.626930+0000", 2024-09-13T10:11:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7284000000000003e-05 2024-09-13T10:11:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.627490+0000", 2024-09-13T10:11:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056048399999999998 2024-09-13T10:11:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.629049+0000", 2024-09-13T10:11:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001559157 2024-09-13T10:11:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:1276 4.19 e45/41 4:998d9bf1:::smithi19530714-33:head v 45'25, mlcod=42'23)", 2024-09-13T10:11:33.368 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:59.884039+0000", 2024-09-13T10:11:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.433336877999999, 2024-09-13T10:11:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0043044900000000002, 2024-09-13T10:11:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.884039+0000", 2024-09-13T10:11:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.884039+0000", 2024-09-13T10:11:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.884039+0000", 2024-09-13T10:11:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6199999999999999e-07 2024-09-13T10:11:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.885292+0000", 2024-09-13T10:11:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012528999999999999 2024-09-13T10:11:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.885310+0000", 2024-09-13T10:11:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8121000000000001e-05 2024-09-13T10:11:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.885323+0000", 2024-09-13T10:11:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2672e-05 2024-09-13T10:11:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.885441+0000", 2024-09-13T10:11:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011766599999999999 2024-09-13T10:11:33.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.885479+0000", 2024-09-13T10:11:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8124999999999998e-05 2024-09-13T10:11:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.888240+0000", 2024-09-13T10:11:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027611440000000001 2024-09-13T10:11:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.888343+0000", 2024-09-13T10:11:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010349999999999999 2024-09-13T10:11:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:1277 4.19 e45/41 4:998d9bf1:::smithi19530714-33:head v 45'26, mlcod=42'24)", 2024-09-13T10:11:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:59.891710+0000", 2024-09-13T10:11:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.425666024000002, 2024-09-13T10:11:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003460627, 2024-09-13T10:11:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.373 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.891710+0000", 2024-09-13T10:11:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.891710+0000", 2024-09-13T10:11:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.891711+0000", 2024-09-13T10:11:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3820000000000001e-06 2024-09-13T10:11:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.892660+0000", 2024-09-13T10:11:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00094837899999999995 2024-09-13T10:11:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.892683+0000", 2024-09-13T10:11:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3767000000000001e-05 2024-09-13T10:11:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.892697+0000", 2024-09-13T10:11:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3619e-05 2024-09-13T10:11:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.892795+0000", 2024-09-13T10:11:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7553000000000006e-05 2024-09-13T10:11:33.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.892831+0000", 2024-09-13T10:11:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5976000000000002e-05 2024-09-13T10:11:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.895127+0000", 2024-09-13T10:11:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022967830000000002 2024-09-13T10:11:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.895170+0000", 2024-09-13T10:11:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3167999999999997e-05 2024-09-13T10:11:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1600 4.10 4.31420bd0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:05.990716+0000", 2024-09-13T10:11:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.326659911, 2024-09-13T10:11:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.412517299, 2024-09-13T10:11:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.378 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:11:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1600 2024-09-13T10:11:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.990716+0000", 2024-09-13T10:11:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.990716+0000", 2024-09-13T10:11:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.990720+0000", 2024-09-13T10:11:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9689999999999996e-06 2024-09-13T10:11:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991825+0000", 2024-09-13T10:11:33.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001104764 2024-09-13T10:11:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991856+0000", 2024-09-13T10:11:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1473000000000002e-05 2024-09-13T10:11:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991872+0000", 2024-09-13T10:11:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6212999999999999e-05 2024-09-13T10:11:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991974+0000", 2024-09-13T10:11:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010160799999999999 2024-09-13T10:11:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991988+0000", 2024-09-13T10:11:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4106e-05 2024-09-13T10:11:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403179+0000", 2024-09-13T10:11:33.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41119127100000002 2024-09-13T10:11:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403233+0000", 2024-09-13T10:11:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3894999999999998e-05 2024-09-13T10:11:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.383 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1601 4.10 4.31420bd0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.008020+0000", 2024-09-13T10:11:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.309355735, 2024-09-13T10:11:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39533983, 2024-09-13T10:11:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:11:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.384 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1601 2024-09-13T10:11:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.008020+0000", 2024-09-13T10:11:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.008020+0000", 2024-09-13T10:11:33.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.008023+0000", 2024-09-13T10:11:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.807e-06 2024-09-13T10:11:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009268+0000", 2024-09-13T10:11:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001245257 2024-09-13T10:11:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009298+0000", 2024-09-13T10:11:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0171999999999998e-05 2024-09-13T10:11:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009313+0000", 2024-09-13T10:11:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.471e-05 2024-09-13T10:11:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009469+0000", 2024-09-13T10:11:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015595600000000001 2024-09-13T10:11:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009481+0000", 2024-09-13T10:11:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2454e-05 2024-09-13T10:11:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403334+0000", 2024-09-13T10:11:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39385227499999997 2024-09-13T10:11:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403360+0000", 2024-09-13T10:11:33.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6199000000000001e-05 2024-09-13T10:11:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1602 4.10 4.31420bd0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.009344+0000", 2024-09-13T10:11:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.308031903, 2024-09-13T10:11:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39420750999999998, 2024-09-13T10:11:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:11:33.389 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1602 2024-09-13T10:11:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009344+0000", 2024-09-13T10:11:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009344+0000", 2024-09-13T10:11:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009345+0000", 2024-09-13T10:11:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1930000000000001e-06 2024-09-13T10:11:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009351+0000", 2024-09-13T10:11:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1399999999999997e-06 2024-09-13T10:11:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009361+0000", 2024-09-13T10:11:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9569999999999997e-06 2024-09-13T10:11:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009370+0000", 2024-09-13T10:11:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.619e-06 2024-09-13T10:11:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009570+0000", 2024-09-13T10:11:33.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019990599999999999 2024-09-13T10:11:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009583+0000", 2024-09-13T10:11:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2886e-05 2024-09-13T10:11:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403522+0000", 2024-09-13T10:11:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39393976000000003 2024-09-13T10:11:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403551+0000", 2024-09-13T10:11:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9048999999999999e-05 2024-09-13T10:11:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1603 4.10 4.31420bd0 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.009383+0000", 2024-09-13T10:11:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.307993011000001, 2024-09-13T10:11:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39425412900000001, 2024-09-13T10:11:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.394 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:11:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1603 2024-09-13T10:11:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009383+0000", 2024-09-13T10:11:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009383+0000", 2024-09-13T10:11:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009384+0000", 2024-09-13T10:11:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8700000000000005e-07 2024-09-13T10:11:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009389+0000", 2024-09-13T10:11:33.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1669999999999998e-06 2024-09-13T10:11:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009398+0000", 2024-09-13T10:11:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6260000000000007e-06 2024-09-13T10:11:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009407+0000", 2024-09-13T10:11:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1960000000000008e-06 2024-09-13T10:11:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009623+0000", 2024-09-13T10:11:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000216842 2024-09-13T10:11:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009632+0000", 2024-09-13T10:11:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3529999999999995e-06 2024-09-13T10:11:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403609+0000", 2024-09-13T10:11:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39397721600000002 2024-09-13T10:11:33.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403637+0000", 2024-09-13T10:11:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7942000000000001e-05 2024-09-13T10:11:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:1618 4.19 e48/41 4:998d9bf1:::smithi19530714-33:head v 48'30, mlcod=45'28)", 2024-09-13T10:11:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.067938+0000", 2024-09-13T10:11:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.249437616000002, 2024-09-13T10:11:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040350539999999997, 2024-09-13T10:11:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.067938+0000", 2024-09-13T10:11:33.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.067938+0000", 2024-09-13T10:11:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.067939+0000", 2024-09-13T10:11:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.33e-07 2024-09-13T10:11:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.069175+0000", 2024-09-13T10:11:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001235667 2024-09-13T10:11:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.069203+0000", 2024-09-13T10:11:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7742999999999998e-05 2024-09-13T10:11:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.069219+0000", 2024-09-13T10:11:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6195999999999999e-05 2024-09-13T10:11:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.069355+0000", 2024-09-13T10:11:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013634300000000001 2024-09-13T10:11:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.069396+0000", 2024-09-13T10:11:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1099000000000001e-05 2024-09-13T10:11:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.071866+0000", 2024-09-13T10:11:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024698720000000001 2024-09-13T10:11:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.071973+0000", 2024-09-13T10:11:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010720100000000001 2024-09-13T10:11:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:1619 4.19 e48/41 4:998d9bf1:::smithi19530714-33:head v 48'31, mlcod=47'29)", 2024-09-13T10:11:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.076123+0000", 2024-09-13T10:11:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.241253227000001, 2024-09-13T10:11:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057688829999999998, 2024-09-13T10:11:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.076123+0000", 2024-09-13T10:11:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.076123+0000", 2024-09-13T10:11:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.076125+0000", 2024-09-13T10:11:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.633e-06 2024-09-13T10:11:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077122+0000", 2024-09-13T10:11:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00099714000000000009 2024-09-13T10:11:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077157+0000", 2024-09-13T10:11:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4869000000000001e-05 2024-09-13T10:11:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077177+0000", 2024-09-13T10:11:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9245999999999998e-05 2024-09-13T10:11:33.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077335+0000", 2024-09-13T10:11:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015893999999999999 2024-09-13T10:11:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077389+0000", 2024-09-13T10:11:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3871e-05 2024-09-13T10:11:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.081809+0000", 2024-09-13T10:11:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044200189999999999 2024-09-13T10:11:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.081892+0000", 2024-09-13T10:11:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2164999999999997e-05 2024-09-13T10:11:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.409 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:1620 4.19 e48/41 4:998d9bf1:::smithi19530714-33:head v 48'32, mlcod=47'29)", 2024-09-13T10:11:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.077802+0000", 2024-09-13T10:11:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.239573597, 2024-09-13T10:11:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049265200000000002, 2024-09-13T10:11:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077802+0000", 2024-09-13T10:11:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077802+0000", 2024-09-13T10:11:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077804+0000", 2024-09-13T10:11:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.933e-06 2024-09-13T10:11:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077820+0000", 2024-09-13T10:11:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5899000000000001e-05 2024-09-13T10:11:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077841+0000", 2024-09-13T10:11:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0409999999999999e-05 2024-09-13T10:11:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.077858+0000", 2024-09-13T10:11:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7903000000000001e-05 2024-09-13T10:11:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.079658+0000", 2024-09-13T10:11:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00179999 2024-09-13T10:11:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.079710+0000", 2024-09-13T10:11:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1969000000000003e-05 2024-09-13T10:11:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.082655+0000", 2024-09-13T10:11:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029449340000000002 2024-09-13T10:11:33.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.082729+0000", 2024-09-13T10:11:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3481999999999999e-05 2024-09-13T10:11:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:2416 4.19 e63/41 4:998d9bf1:::smithi19530714-33:head v 63'47, mlcod=57'45)", 2024-09-13T10:11:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:22.238492+0000", 2024-09-13T10:11:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.078883934, 2024-09-13T10:11:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027270770000000001, 2024-09-13T10:11:33.414 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.238492+0000", 2024-09-13T10:11:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.238492+0000", 2024-09-13T10:11:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.238493+0000", 2024-09-13T10:11:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.23e-07 2024-09-13T10:11:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.238502+0000", 2024-09-13T10:11:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2900000000000008e-06 2024-09-13T10:11:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.238526+0000", 2024-09-13T10:11:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3465e-05 2024-09-13T10:11:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.238544+0000", 2024-09-13T10:11:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8396000000000001e-05 2024-09-13T10:11:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.238688+0000", 2024-09-13T10:11:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000144058 2024-09-13T10:11:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.238740+0000", 2024-09-13T10:11:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1714e-05 2024-09-13T10:11:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.241145+0000", 2024-09-13T10:11:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024049990000000001 2024-09-13T10:11:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.241219+0000", 2024-09-13T10:11:33.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4332000000000006e-05 2024-09-13T10:11:33.419 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.419 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.419 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.419 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.419 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.419 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:33.420 INFO:tasks.rados.rados.0.smithi195.stdout:1292: expect (ObjNum 160 snap 0 seq_num 160) 2024-09-13T10:11:33.420 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:33.421 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:33.418+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:33.421 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-13T10:11:33.451 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:33.515 INFO:tasks.rados.rados.0.smithi195.stdout:1296: expect (ObjNum 180 snap 0 seq_num 180) 2024-09-13T10:11:33.516 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:33.521 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:33.516+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:33.549 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:33.549 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:33.549 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:11:33.549 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:33.563 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-13T10:11:33.622 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:33.618+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:33.643 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1723027 2024-09-13T10:11:33.666 INFO:tasks.rados.rados.0.smithi195.stdout:1290: finishing set_chunk to oid 20 2024-09-13T10:11:33.666 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 76 (ObjNum 232 snap 0 seq_num 232) dirty exists 2024-09-13T10:11:33.666 INFO:tasks.rados.rados.0.smithi195.stdout:1298: expect (ObjNum 227 snap 0 seq_num 227) 2024-09-13T10:11:33.666 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:33.705 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-13T10:11:33.723 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:33.720+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:33.751 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-13T10:11:33.774 INFO:tasks.rados.rados.0.smithi195.stdout:1302: got (22) Invalid argument 2024-09-13T10:11:33.775 INFO:tasks.rados.rados.0.smithi195.stdout:1297: expect (ObjNum 215 snap 0 seq_num 215) 2024-09-13T10:11:33.775 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:33.824 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:33.820+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:33.839 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:33.885 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:33.885 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:11:33.885 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:11:33.885 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:11:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4365.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:30.119900+0000", 2024-09-13T10:11:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 63.761505829999997, 2024-09-13T10:11:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020673093999999999, 2024-09-13T10:11:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4365", 2024-09-13T10:11:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.73:0/186187721", 2024-09-13T10:11:33.886 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1 2024-09-13T10:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119900+0000", 2024-09-13T10:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119900+0000", 2024-09-13T10:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119903+0000", 2024-09-13T10:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0929999999999999e-06 2024-09-13T10:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119919+0000", 2024-09-13T10:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6082999999999998e-05 2024-09-13T10:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119954+0000", 2024-09-13T10:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4944000000000003e-05 2024-09-13T10:11:33.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.119983+0000", 2024-09-13T10:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8932e-05 2024-09-13T10:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.120157+0000", 2024-09-13T10:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017421 2024-09-13T10:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-13T10:11:33.889 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.120180+0000", 2024-09-13T10:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3039000000000001e-05 2024-09-13T10:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.121820+0000", 2024-09-13T10:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016397479999999999 2024-09-13T10:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.121842+0000", 2024-09-13T10:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2612e-05 2024-09-13T10:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.890 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.125236+0000", 2024-09-13T10:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003394181 2024-09-13T10:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.125597+0000", 2024-09-13T10:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036106199999999999 2024-09-13T10:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.125908+0000", 2024-09-13T10:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031034200000000002 2024-09-13T10:11:33.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:33.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.139886+0000", 2024-09-13T10:11:33.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013977824 2024-09-13T10:11:33.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.892 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.892 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140020+0000", 2024-09-13T10:11:33.892 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013476199999999999 2024-09-13T10:11:33.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.892 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140357+0000", 2024-09-13T10:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033711 2024-09-13T10:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140464+0000", 2024-09-13T10:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106028 2024-09-13T10:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:30.140573+0000", 2024-09-13T10:11:33.893 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010912399999999999 2024-09-13T10:11:33.894 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.894 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.894 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.894 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.894 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:203 3.1 3:b033e801:::smithi19530714-4 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:33.894 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:11:33.894 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.889941788000002, 2024-09-13T10:11:33.894 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035547764000000003, 2024-09-13T10:11:33.894 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.894 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.895 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.895 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.895 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.895 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.895 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 203 2024-09-13T10:11:33.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.895 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.895 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.895 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.895 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:11:33.895 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:11:33.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.896 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.896 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.896 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991467+0000", 2024-09-13T10:11:33.896 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5030000000000002e-06 2024-09-13T10:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991483+0000", 2024-09-13T10:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5503e-05 2024-09-13T10:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991522+0000", 2024-09-13T10:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9399e-05 2024-09-13T10:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.897 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991544+0000", 2024-09-13T10:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2408999999999998e-05 2024-09-13T10:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991646+0000", 2024-09-13T10:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101798 2024-09-13T10:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991818+0000", 2024-09-13T10:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017204300000000001 2024-09-13T10:11:33.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.001618+0000", 2024-09-13T10:11:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097993490000000006 2024-09-13T10:11:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026785+0000", 2024-09-13T10:11:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025167446 2024-09-13T10:11:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.899 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026856+0000", 2024-09-13T10:11:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0575999999999996e-05 2024-09-13T10:11:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026915+0000", 2024-09-13T10:11:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9258000000000001e-05 2024-09-13T10:11:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026964+0000", 2024-09-13T10:11:33.900 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000003e-05 2024-09-13T10:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027011+0000", 2024-09-13T10:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7200000000000002e-05 2024-09-13T10:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:204 3.1 3:babcb876:::smithi19530714-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:33.901 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.889817272000002, 2024-09-13T10:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.22268644700000001, 2024-09-13T10:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 204 2024-09-13T10:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.902 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:11:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:11:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991589+0000", 2024-09-13T10:11:33.903 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3859999999999999e-06 2024-09-13T10:11:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991598+0000", 2024-09-13T10:11:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7669999999999992e-06 2024-09-13T10:11:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991616+0000", 2024-09-13T10:11:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7589e-05 2024-09-13T10:11:33.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991631+0000", 2024-09-13T10:11:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.47e-05 2024-09-13T10:11:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992050+0000", 2024-09-13T10:11:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041913399999999999 2024-09-13T10:11:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.905 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992229+0000", 2024-09-13T10:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017956499999999999 2024-09-13T10:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.203594+0000", 2024-09-13T10:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.211364303 2024-09-13T10:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.208526+0000", 2024-09-13T10:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049325020000000001 2024-09-13T10:11:33.906 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214019+0000", 2024-09-13T10:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054933270000000001 2024-09-13T10:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214108+0000", 2024-09-13T10:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8517000000000006e-05 2024-09-13T10:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.907 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214202+0000", 2024-09-13T10:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3763000000000003e-05 2024-09-13T10:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214275+0000", 2024-09-13T10:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2893999999999997e-05 2024-09-13T10:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:205 3.1 3:aec11a92:::smithi19530714-6 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:33.908 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:11:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.889741618000002, 2024-09-13T10:11:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056950259000000003, 2024-09-13T10:11:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 205 2024-09-13T10:11:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.909 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.910 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991665+0000", 2024-09-13T10:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1400000000000001e-06 2024-09-13T10:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991673+0000", 2024-09-13T10:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5730000000000008e-06 2024-09-13T10:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991689+0000", 2024-09-13T10:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5956000000000001e-05 2024-09-13T10:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.911 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991705+0000", 2024-09-13T10:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5539000000000001e-05 2024-09-13T10:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992393+0000", 2024-09-13T10:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068836600000000004 2024-09-13T10:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992509+0000", 2024-09-13T10:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115563 2024-09-13T10:11:33.912 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.011894+0000", 2024-09-13T10:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019385594999999999 2024-09-13T10:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027029+0000", 2024-09-13T10:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015134664000000001 2024-09-13T10:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.031277+0000", 2024-09-13T10:11:33.913 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004247804 2024-09-13T10:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048542+0000", 2024-09-13T10:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017265401999999999 2024-09-13T10:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048579+0000", 2024-09-13T10:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6238000000000001e-05 2024-09-13T10:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.914 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048614+0000", 2024-09-13T10:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5419000000000001e-05 2024-09-13T10:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:209 3.1 3:9798d58d:::smithi19530714-10 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.889613402999998, 2024-09-13T10:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.069824676000000002, 2024-09-13T10:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.915 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 209 2024-09-13T10:11:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:11:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991793+0000", 2024-09-13T10:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.057e-06 2024-09-13T10:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991803+0000", 2024-09-13T10:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7860000000000008e-06 2024-09-13T10:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991817+0000", 2024-09-13T10:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4066000000000001e-05 2024-09-13T10:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991828+0000", 2024-09-13T10:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1097000000000001e-05 2024-09-13T10:11:33.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992666+0000", 2024-09-13T10:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083755799999999997 2024-09-13T10:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992792+0000", 2024-09-13T10:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000126334 2024-09-13T10:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027675+0000", 2024-09-13T10:11:33.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034883299 2024-09-13T10:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.035648+0000", 2024-09-13T10:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007972972 2024-09-13T10:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.045764+0000", 2024-09-13T10:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010115832 2024-09-13T10:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061460+0000", 2024-09-13T10:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015695563999999999 2024-09-13T10:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061544+0000", 2024-09-13T10:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4866999999999999e-05 2024-09-13T10:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061617+0000", 2024-09-13T10:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2243999999999995e-05 2024-09-13T10:11:33.921 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:213 3.1 3:accf4e56:::smithi19530714-14 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.88883946, 2024-09-13T10:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.060593045999999998, 2024-09-13T10:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.922 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 213 2024-09-13T10:11:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:11:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992568+0000", 2024-09-13T10:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3640000000000002e-06 2024-09-13T10:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992580+0000", 2024-09-13T10:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1796e-05 2024-09-13T10:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992662+0000", 2024-09-13T10:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1814e-05 2024-09-13T10:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992683+0000", 2024-09-13T10:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0746000000000001e-05 2024-09-13T10:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.993024+0000", 2024-09-13T10:11:33.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034141299999999999 2024-09-13T10:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.993182+0000", 2024-09-13T10:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015817699999999999 2024-09-13T10:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.019852+0000", 2024-09-13T10:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026669470000000001 2024-09-13T10:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.032551+0000", 2024-09-13T10:11:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012699729999999999 2024-09-13T10:11:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.034351+0000", 2024-09-13T10:11:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017998269999999999 2024-09-13T10:11:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053106+0000", 2024-09-13T10:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018755236000000002 2024-09-13T10:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053131+0000", 2024-09-13T10:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4729e-05 2024-09-13T10:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053159+0000", 2024-09-13T10:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7744000000000001e-05 2024-09-13T10:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.928 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:219 3.1 3:87bfc7b1:::smithi19530714-20 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.848390387000002, 2024-09-13T10:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.037494069999999997, 2024-09-13T10:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.929 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 219 2024-09-13T10:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:11:33.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033019+0000", 2024-09-13T10:11:33.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.726e-06 2024-09-13T10:11:33.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033036+0000", 2024-09-13T10:11:33.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7329999999999998e-05 2024-09-13T10:11:33.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033060+0000", 2024-09-13T10:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4286000000000001e-05 2024-09-13T10:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033079+0000", 2024-09-13T10:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8872000000000002e-05 2024-09-13T10:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033155+0000", 2024-09-13T10:11:33.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6118999999999994e-05 2024-09-13T10:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033297+0000", 2024-09-13T10:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014154700000000001 2024-09-13T10:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.041666+0000", 2024-09-13T10:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083696180000000005 2024-09-13T10:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046621+0000", 2024-09-13T10:11:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049545759999999996 2024-09-13T10:11:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.052333+0000", 2024-09-13T10:11:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057120160000000003 2024-09-13T10:11:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070340+0000", 2024-09-13T10:11:33.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018006859 2024-09-13T10:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070447+0000", 2024-09-13T10:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106637 2024-09-13T10:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070509+0000", 2024-09-13T10:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2483999999999993e-05 2024-09-13T10:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:222 3.1 3:bdac4e61:::smithi19530714-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.834702413000002, 2024-09-13T10:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041434034000000002, 2024-09-13T10:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 222 2024-09-13T10:11:33.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046706+0000", 2024-09-13T10:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0089999999999999e-06 2024-09-13T10:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046719+0000", 2024-09-13T10:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2565000000000001e-05 2024-09-13T10:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046740+0000", 2024-09-13T10:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0979e-05 2024-09-13T10:11:33.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046752+0000", 2024-09-13T10:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2917e-05 2024-09-13T10:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046803+0000", 2024-09-13T10:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0417000000000002e-05 2024-09-13T10:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046908+0000", 2024-09-13T10:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105039 2024-09-13T10:11:33.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.066097+0000", 2024-09-13T10:11:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019189277000000001 2024-09-13T10:11:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070969+0000", 2024-09-13T10:11:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048721429999999998 2024-09-13T10:11:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.087852+0000", 2024-09-13T10:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016882489000000001 2024-09-13T10:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.087942+0000", 2024-09-13T10:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9738e-05 2024-09-13T10:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088076+0000", 2024-09-13T10:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013400700000000001 2024-09-13T10:11:33.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088137+0000", 2024-09-13T10:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1453999999999998e-05 2024-09-13T10:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:223 3.1 3:8759f4e0:::smithi19530714-24 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.822025543999999, 2024-09-13T10:11:33.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053691092000000003, 2024-09-13T10:11:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 223 2024-09-13T10:11:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059383+0000", 2024-09-13T10:11:33.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.847e-06 2024-09-13T10:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059399+0000", 2024-09-13T10:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6739e-05 2024-09-13T10:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059433+0000", 2024-09-13T10:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3946000000000003e-05 2024-09-13T10:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059456+0000", 2024-09-13T10:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2163000000000001e-05 2024-09-13T10:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059621+0000", 2024-09-13T10:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016546700000000001 2024-09-13T10:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.060003+0000", 2024-09-13T10:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000381982 2024-09-13T10:11:33.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.085339+0000", 2024-09-13T10:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025335610000000001 2024-09-13T10:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.089896+0000", 2024-09-13T10:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045575930000000004 2024-09-13T10:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.102515+0000", 2024-09-13T10:11:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012618664999999999 2024-09-13T10:11:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113009+0000", 2024-09-13T10:11:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010493784000000001 2024-09-13T10:11:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113044+0000", 2024-09-13T10:11:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5543999999999997e-05 2024-09-13T10:11:33.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113071+0000", 2024-09-13T10:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6752000000000001e-05 2024-09-13T10:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:231 3.1 3:9d247594:::smithi19530714-32 low_tier:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.065612+0000", 2024-09-13T10:11:33.949 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.815792885999997, 2024-09-13T10:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031259105000000002, 2024-09-13T10:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 231 2024-09-13T10:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065612+0000", 2024-09-13T10:11:33.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065612+0000", 2024-09-13T10:11:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065614+0000", 2024-09-13T10:11:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7370000000000001e-06 2024-09-13T10:11:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065622+0000", 2024-09-13T10:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1110000000000008e-06 2024-09-13T10:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065640+0000", 2024-09-13T10:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7499999999999998e-05 2024-09-13T10:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065655+0000", 2024-09-13T10:11:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5021999999999999e-05 2024-09-13T10:11:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069879+0000", 2024-09-13T10:11:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042246360000000004 2024-09-13T10:11:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070080+0000", 2024-09-13T10:11:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020099199999999999 2024-09-13T10:11:33.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.075971+0000", 2024-09-13T10:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058906030000000003 2024-09-13T10:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088156+0000", 2024-09-13T10:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012184868999999999 2024-09-13T10:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.095237+0000", 2024-09-13T10:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0070814279999999999 2024-09-13T10:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.096766+0000", 2024-09-13T10:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001529007 2024-09-13T10:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.096823+0000", 2024-09-13T10:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6789999999999997e-05 2024-09-13T10:11:33.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.096872+0000", 2024-09-13T10:11:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8409999999999999e-05 2024-09-13T10:11:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:234 3.1 3:bdce9caa:::smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:11:33.956 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.812306282999998, 2024-09-13T10:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043032263000000001, 2024-09-13T10:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 234 2024-09-13T10:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069102+0000", 2024-09-13T10:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.543e-06 2024-09-13T10:11:33.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069112+0000", 2024-09-13T10:11:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0091e-05 2024-09-13T10:11:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069133+0000", 2024-09-13T10:11:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1188000000000001e-05 2024-09-13T10:11:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069143+0000", 2024-09-13T10:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0456e-05 2024-09-13T10:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070588+0000", 2024-09-13T10:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001445046 2024-09-13T10:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070801+0000", 2024-09-13T10:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000212151 2024-09-13T10:11:33.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.082362+0000", 2024-09-13T10:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561781 2024-09-13T10:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088161+0000", 2024-09-13T10:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057986119999999999 2024-09-13T10:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.101972+0000", 2024-09-13T10:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013811498 2024-09-13T10:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112028+0000", 2024-09-13T10:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010055530999999999 2024-09-13T10:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112082+0000", 2024-09-13T10:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3974999999999997e-05 2024-09-13T10:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112131+0000", 2024-09-13T10:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9391000000000002e-05 2024-09-13T10:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:243 3.1 3:a4dd4282:::smithi19530714-44 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.784098577999998, 2024-09-13T10:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036759694000000002, 2024-09-13T10:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.963 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 243 2024-09-13T10:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097308+0000", 2024-09-13T10:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2099999999999995e-07 2024-09-13T10:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097314+0000", 2024-09-13T10:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9989999999999997e-06 2024-09-13T10:11:33.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097326+0000", 2024-09-13T10:11:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1787000000000001e-05 2024-09-13T10:11:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097332+0000", 2024-09-13T10:11:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6950000000000003e-06 2024-09-13T10:11:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097378+0000", 2024-09-13T10:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5744000000000001e-05 2024-09-13T10:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097489+0000", 2024-09-13T10:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111352 2024-09-13T10:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.118571+0000", 2024-09-13T10:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021081249999999999 2024-09-13T10:11:33.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.128494+0000", 2024-09-13T10:11:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099237270000000002 2024-09-13T10:11:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.128775+0000", 2024-09-13T10:11:33.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028027299999999999 2024-09-13T10:11:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.133963+0000", 2024-09-13T10:11:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005188191 2024-09-13T10:11:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134015+0000", 2024-09-13T10:11:33.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2438000000000002e-05 2024-09-13T10:11:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134066+0000", 2024-09-13T10:11:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1316999999999997e-05 2024-09-13T10:11:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.970 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:248 3.1 3:b0d90ce0:::smithi19530714-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 18] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.114744+0000", 2024-09-13T10:11:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.766661677000002, 2024-09-13T10:11:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028886754000000001, 2024-09-13T10:11:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.971 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 248 2024-09-13T10:11:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114744+0000", 2024-09-13T10:11:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114744+0000", 2024-09-13T10:11:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114745+0000", 2024-09-13T10:11:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.55e-06 2024-09-13T10:11:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114752+0000", 2024-09-13T10:11:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1670000000000002e-06 2024-09-13T10:11:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114768+0000", 2024-09-13T10:11:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5471000000000001e-05 2024-09-13T10:11:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114775+0000", 2024-09-13T10:11:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3799999999999996e-06 2024-09-13T10:11:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114826+0000", 2024-09-13T10:11:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0909999999999999e-05 2024-09-13T10:11:33.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114981+0000", 2024-09-13T10:11:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015455699999999999 2024-09-13T10:11:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.125745+0000", 2024-09-13T10:11:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010764404 2024-09-13T10:11:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.129548+0000", 2024-09-13T10:11:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038027019999999998 2024-09-13T10:11:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.140602+0000", 2024-09-13T10:11:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011054082 2024-09-13T10:11:33.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.143523+0000", 2024-09-13T10:11:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002920962 2024-09-13T10:11:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.143586+0000", 2024-09-13T10:11:33.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2866000000000004e-05 2024-09-13T10:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.143630+0000", 2024-09-13T10:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4703000000000002e-05 2024-09-13T10:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:280 4.f 4:fe60987c:::smithi19530714-31:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:33.979 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.236614+0000", 2024-09-13T10:11:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.644791789000003, 2024-09-13T10:11:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021117787999999998, 2024-09-13T10:11:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 280 2024-09-13T10:11:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.980 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236614+0000", 2024-09-13T10:11:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236614+0000", 2024-09-13T10:11:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236616+0000", 2024-09-13T10:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4700000000000001e-06 2024-09-13T10:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236627+0000", 2024-09-13T10:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1226000000000001e-05 2024-09-13T10:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236654+0000", 2024-09-13T10:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6876000000000002e-05 2024-09-13T10:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236675+0000", 2024-09-13T10:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1199000000000002e-05 2024-09-13T10:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236772+0000", 2024-09-13T10:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6359000000000004e-05 2024-09-13T10:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236873+0000", 2024-09-13T10:11:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010113199999999999 2024-09-13T10:11:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2,7", 2024-09-13T10:11:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.254958+0000", 2024-09-13T10:11:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018085127999999999 2024-09-13T10:11:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.256855+0000", 2024-09-13T10:11:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001896888 2024-09-13T10:11:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257456+0000", 2024-09-13T10:11:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060094800000000002 2024-09-13T10:11:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257633+0000", 2024-09-13T10:11:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017736499999999999 2024-09-13T10:11:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257684+0000", 2024-09-13T10:11:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0812000000000001e-05 2024-09-13T10:11:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257731+0000", 2024-09-13T10:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7385000000000001e-05 2024-09-13T10:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:292 4.f 4:f6a0e8ce:::smithi19530714-43:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.257863+0000", 2024-09-13T10:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.623542456999999, 2024-09-13T10:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022747474, 2024-09-13T10:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.987 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 292 2024-09-13T10:11:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257863+0000", 2024-09-13T10:11:33.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257863+0000", 2024-09-13T10:11:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257865+0000", 2024-09-13T10:11:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.883e-06 2024-09-13T10:11:33.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257874+0000", 2024-09-13T10:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0699999999999996e-06 2024-09-13T10:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257894+0000", 2024-09-13T10:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0349000000000001e-05 2024-09-13T10:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257925+0000", 2024-09-13T10:11:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1092e-05 2024-09-13T10:11:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258040+0000", 2024-09-13T10:11:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114474 2024-09-13T10:11:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258126+0000", 2024-09-13T10:11:33.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6479999999999999e-05 2024-09-13T10:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2,7", 2024-09-13T10:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.273691+0000", 2024-09-13T10:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015564626999999999 2024-09-13T10:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.275091+0000", 2024-09-13T10:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001400079 2024-09-13T10:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.276990+0000", 2024-09-13T10:11:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018994179999999999 2024-09-13T10:11:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.280333+0000", 2024-09-13T10:11:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033424320000000002 2024-09-13T10:11:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:33.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.280486+0000", 2024-09-13T10:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015310500000000001 2024-09-13T10:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.280610+0000", 2024-09-13T10:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012446500000000001 2024-09-13T10:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:300 4.f 4:f7ad6ef6:::smithi19530714-1:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:33.994 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.264825+0000", 2024-09-13T10:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.616580155000001, 2024-09-13T10:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018261785999999999, 2024-09-13T10:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 300 2024-09-13T10:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264825+0000", 2024-09-13T10:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264825+0000", 2024-09-13T10:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264827+0000", 2024-09-13T10:11:33.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.204e-06 2024-09-13T10:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264838+0000", 2024-09-13T10:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0212e-05 2024-09-13T10:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264859+0000", 2024-09-13T10:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1613000000000001e-05 2024-09-13T10:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264873+0000", 2024-09-13T10:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3893000000000001e-05 2024-09-13T10:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264941+0000", 2024-09-13T10:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8355000000000005e-05 2024-09-13T10:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.265012+0000", 2024-09-13T10:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0759999999999993e-05 2024-09-13T10:11:33.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2,7", 2024-09-13T10:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.276117+0000", 2024-09-13T10:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011104283 2024-09-13T10:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.278041+0000", 2024-09-13T10:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001924427 2024-09-13T10:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:33.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.278343+0000", 2024-09-13T10:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030249900000000002 2024-09-13T10:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.283022+0000", 2024-09-13T10:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046781840000000002 2024-09-13T10:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.283053+0000", 2024-09-13T10:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1696999999999998e-05 2024-09-13T10:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.283087+0000", 2024-09-13T10:11:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3658999999999997e-05 2024-09-13T10:11:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.1.22:23 3.1 3:969e51ee:::93c498e92137985868c9dcdff823d39ad7f943e0cf15bbbe4a7d3098e4bc2da6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.128192+0000", 2024-09-13T10:11:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.753213725000002, 2024-09-13T10:11:34.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016280833000000001, 2024-09-13T10:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128192+0000", 2024-09-13T10:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128192+0000", 2024-09-13T10:11:34.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128193+0000", 2024-09-13T10:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0139999999999999e-06 2024-09-13T10:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128359+0000", 2024-09-13T10:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000166455 2024-09-13T10:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128372+0000", 2024-09-13T10:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3029e-05 2024-09-13T10:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128382+0000", 2024-09-13T10:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.55e-06 2024-09-13T10:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.137523+0000", 2024-09-13T10:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0091413320000000003 2024-09-13T10:11:34.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.137721+0000", 2024-09-13T10:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000197777 2024-09-13T10:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.138223+0000", 2024-09-13T10:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050221499999999997 2024-09-13T10:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.144219+0000", 2024-09-13T10:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059962310000000003 2024-09-13T10:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.144284+0000", 2024-09-13T10:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4734999999999996e-05 2024-09-13T10:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.144319+0000", 2024-09-13T10:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5271999999999997e-05 2024-09-13T10:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.144403+0000", 2024-09-13T10:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3792e-05 2024-09-13T10:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.144472+0000", 2024-09-13T10:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9431000000000007e-05 2024-09-13T10:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.007 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.1.22:26 3.1 3:81602715:::0cd1941c9b5b3a2b092b213a045ec3831dc9f094ae77b0454d92fa84833c395e:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.128673+0000", 2024-09-13T10:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.752731955999998, 2024-09-13T10:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017705240000000001, 2024-09-13T10:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128673+0000", 2024-09-13T10:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:34.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128673+0000", 2024-09-13T10:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.128676+0000", 2024-09-13T10:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9129999999999999e-06 2024-09-13T10:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:34.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.129361+0000", 2024-09-13T10:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068506999999999997 2024-09-13T10:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.129403+0000", 2024-09-13T10:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1776000000000001e-05 2024-09-13T10:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.129420+0000", 2024-09-13T10:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7125e-05 2024-09-13T10:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.141718+0000", 2024-09-13T10:11:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012297439 2024-09-13T10:11:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.141913+0000", 2024-09-13T10:11:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019493000000000001 2024-09-13T10:11:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.142410+0000", 2024-09-13T10:11:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049744199999999996 2024-09-13T10:11:34.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.145111+0000", 2024-09-13T10:11:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027010329999999998 2024-09-13T10:11:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.145797+0000", 2024-09-13T10:11:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068605300000000005 2024-09-13T10:11:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.146186+0000", 2024-09-13T10:11:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038835200000000001 2024-09-13T10:11:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.146290+0000", 2024-09-13T10:11:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010416000000000001 2024-09-13T10:11:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.146379+0000", 2024-09-13T10:11:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8947000000000001e-05 2024-09-13T10:11:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:34.013 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1272 4.f 4:f7ad6ef6:::smithi19530714-1:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e45)", 2024-09-13T10:11:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:11:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 34.005870403000003, 2024-09-13T10:11:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.050041173000000001, 2024-09-13T10:11:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:34.014 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1272 2024-09-13T10:11:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:11:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:11:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:34.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875537+0000", 2024-09-13T10:11:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2689999999999998e-06 2024-09-13T10:11:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875548+0000", 2024-09-13T10:11:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0291e-05 2024-09-13T10:11:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:34.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875567+0000", 2024-09-13T10:11:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9284999999999999e-05 2024-09-13T10:11:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875578+0000", 2024-09-13T10:11:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1256e-05 2024-09-13T10:11:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875763+0000", 2024-09-13T10:11:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000185396 2024-09-13T10:11:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875908+0000", 2024-09-13T10:11:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014412299999999999 2024-09-13T10:11:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.925576+0000", 2024-09-13T10:11:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049668552999999997 2024-09-13T10:11:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2719 4.f 4:f7ad6ef6:::smithi19530714-1:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:34.018 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:29.386743+0000", 2024-09-13T10:11:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.4946623199999998, 2024-09-13T10:11:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047467252000000001, 2024-09-13T10:11:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2719 2024-09-13T10:11:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386743+0000", 2024-09-13T10:11:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386743+0000", 2024-09-13T10:11:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386746+0000", 2024-09-13T10:11:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2729999999999998e-06 2024-09-13T10:11:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386758+0000", 2024-09-13T10:11:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1703e-05 2024-09-13T10:11:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386789+0000", 2024-09-13T10:11:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0815000000000002e-05 2024-09-13T10:11:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386806+0000", 2024-09-13T10:11:34.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7595999999999999e-05 2024-09-13T10:11:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386996+0000", 2024-09-13T10:11:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000189521 2024-09-13T10:11:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.387151+0000", 2024-09-13T10:11:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015501600000000001 2024-09-13T10:11:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:34.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.434210+0000", 2024-09-13T10:11:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047059327999999997 2024-09-13T10:11:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:34.023 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:34.023 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:34.024 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2395497 2024-09-13T10:11:34.025 INFO:tasks.rados.rados.0.smithi195.stdout:1304: finishing write tid 1 to smithi19530714-2 2024-09-13T10:11:34.025 INFO:tasks.rados.rados.0.smithi195.stdout:1291: expect (ObjNum 217 snap 0 seq_num 217) 2024-09-13T10:11:34.025 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:34.025 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T10:11:34.026 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:34.020+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:34.126 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:34.124+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:34.156 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:34.157 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:34.157 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:11:34.157 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:34.171 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T10:11:34.227 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:34.222+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:34.252 INFO:tasks.rados.rados.0.smithi195.stdout:1300: expect (ObjNum 181 snap 0 seq_num 181) 2024-09-13T10:11:34.284 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:11:34.289 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:34.289 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:34.289 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:11:34.289 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:11:34.289 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:34.303 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T10:11:34.304 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:34.328 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:34.324+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:34.426 INFO:tasks.rados.rados.0.smithi195.stdout:1304: finishing write tid 2 to smithi19530714-2 2024-09-13T10:11:34.427 INFO:tasks.rados.rados.0.smithi195.stdout:1273: done (15 left) 2024-09-13T10:11:34.427 INFO:tasks.rados.rados.0.smithi195.stdout:1290: done (14 left) 2024-09-13T10:11:34.427 INFO:tasks.rados.rados.0.smithi195.stdout:1291: done (13 left) 2024-09-13T10:11:34.427 INFO:tasks.rados.rados.0.smithi195.stdout:1292: done (12 left) 2024-09-13T10:11:34.427 INFO:tasks.rados.rados.0.smithi195.stdout:1294: done (11 left) 2024-09-13T10:11:34.427 INFO:tasks.rados.rados.0.smithi195.stdout:1295: done (10 left) 2024-09-13T10:11:34.427 INFO:tasks.rados.rados.0.smithi195.stdout:1296: done (9 left) 2024-09-13T10:11:34.427 INFO:tasks.rados.rados.0.smithi195.stdout:1297: done (8 left) 2024-09-13T10:11:34.427 INFO:tasks.rados.rados.0.smithi195.stdout:1298: done (7 left) 2024-09-13T10:11:34.427 INFO:tasks.rados.rados.0.smithi195.stdout:1299: done (6 left) 2024-09-13T10:11:34.427 INFO:tasks.rados.rados.0.smithi195.stdout:1300: done (5 left) 2024-09-13T10:11:34.427 INFO:tasks.rados.rados.0.smithi195.stdout:1301: done (4 left) 2024-09-13T10:11:34.427 INFO:tasks.rados.rados.0.smithi195.stdout:1302: done (3 left) 2024-09-13T10:11:34.428 INFO:tasks.rados.rados.0.smithi195.stdout:1303: done (2 left) 2024-09-13T10:11:34.428 INFO:tasks.rados.rados.0.smithi195.stdout:1305: done (1 left) 2024-09-13T10:11:34.428 INFO:tasks.rados.rados.0.smithi195.stdout:1306: read oid 23 snap -1 2024-09-13T10:11:34.428 INFO:tasks.rados.rados.0.smithi195.stdout:1306: expect (ObjNum 206 snap 0 seq_num 206) 2024-09-13T10:11:34.428 INFO:tasks.rados.rados.0.smithi195.stdout:1307: tier_promote oid 26 2024-09-13T10:11:34.428 INFO:tasks.rados.rados.0.smithi195.stdout:1304: finishing write tid 3 to smithi19530714-2 2024-09-13T10:11:34.428 INFO:tasks.rados.rados.0.smithi195.stdout:1304: finishing write tid 4 to smithi19530714-2 2024-09-13T10:11:34.428 INFO:tasks.rados.rados.0.smithi195.stdout:1304: oid 2 updating version 0 to 118 2024-09-13T10:11:34.428 INFO:tasks.rados.rados.0.smithi195.stdout:1304: oid 2 updating version 118 to 119 2024-09-13T10:11:34.428 INFO:tasks.rados.rados.0.smithi195.stdout:1304: oid 2 updating version 119 to 120 2024-09-13T10:11:34.429 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 120 (ObjNum 249 snap 0 seq_num 249) dirty exists 2024-09-13T10:11:34.429 INFO:tasks.rados.rados.0.smithi195.stdout:1304: left oid 2 (ObjNum 249 snap 0 seq_num 249) 2024-09-13T10:11:34.429 INFO:tasks.rados.rados.0.smithi195.stdout:1304: done (2 left) 2024-09-13T10:11:34.429 INFO:tasks.rados.rados.0.smithi195.stdout:1308: tier_promote oid 40 2024-09-13T10:11:34.429 INFO:tasks.rados.rados.0.smithi195.stdout:1309: tier_evict oid 30 2024-09-13T10:11:34.429 INFO:tasks.rados.rados.0.smithi195.stdout:1309: tier_evict oid 30 snap -1 2024-09-13T10:11:34.429 INFO:tasks.rados.rados.0.smithi195.stdout:1310: delete oid 4 current snap is 0 2024-09-13T10:11:34.430 INFO:tasks.rados.rados.0.smithi195.stdout:1309: got (22) Invalid argument 2024-09-13T10:11:34.430 INFO:tasks.rados.rados.0.smithi195.stdout:1306: expect (ObjNum 206 snap 0 seq_num 206) 2024-09-13T10:11:34.432 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:34.432 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:11:34.432 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:11:34.432 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:11:34.432 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.432 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:40 4.1 4:85f1e656:::smithi19530714-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:34.432 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.449537+0000", 2024-09-13T10:11:34.432 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 43.976969924999999, 2024-09-13T10:11:34.432 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011651083, 2024-09-13T10:11:34.432 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.432 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.433 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:34.433 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.433 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.433 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.433 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 40 2024-09-13T10:11:34.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.433 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.433 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.433 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.433 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449537+0000", 2024-09-13T10:11:34.433 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.433 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.433 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.434 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449537+0000", 2024-09-13T10:11:34.434 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.434 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.434 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.434 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449538+0000", 2024-09-13T10:11:34.434 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0070000000000001e-06 2024-09-13T10:11:34.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.434 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.434 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.434 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449543+0000", 2024-09-13T10:11:34.434 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.3560000000000003e-06 2024-09-13T10:11:34.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.435 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.435 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.435 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449554+0000", 2024-09-13T10:11:34.435 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1263e-05 2024-09-13T10:11:34.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.435 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.435 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.435 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.449566+0000", 2024-09-13T10:11:34.435 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2383e-05 2024-09-13T10:11:34.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.435 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.435 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.435 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.456239+0000", 2024-09-13T10:11:34.436 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0066730080000000002 2024-09-13T10:11:34.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.436 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.436 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:34.436 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.456350+0000", 2024-09-13T10:11:34.436 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000111214 2024-09-13T10:11:34.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.436 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.436 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.436 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.460834+0000", 2024-09-13T10:11:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044834080000000004 2024-09-13T10:11:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.460956+0000", 2024-09-13T10:11:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000121934 2024-09-13T10:11:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.461188+0000", 2024-09-13T10:11:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023251000000000001 2024-09-13T10:11:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.437 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:259 4.1 4:85f1e656:::smithi19530714-10:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:11:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 43.211340505999999, 2024-09-13T10:11:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014693513, 2024-09-13T10:11:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.438 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 259 2024-09-13T10:11:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:11:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:11:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.440 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.440 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.440 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215171+0000", 2024-09-13T10:11:34.440 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.3329999999999998e-06 2024-09-13T10:11:34.440 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.440 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.440 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.440 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215190+0000", 2024-09-13T10:11:34.440 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8672999999999999e-05 2024-09-13T10:11:34.440 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.440 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.440 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.440 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215229+0000", 2024-09-13T10:11:34.441 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9076000000000003e-05 2024-09-13T10:11:34.441 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.441 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.441 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.441 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215287+0000", 2024-09-13T10:11:34.441 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8142e-05 2024-09-13T10:11:34.441 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.441 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.441 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.441 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215435+0000", 2024-09-13T10:11:34.441 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014800600000000001 2024-09-13T10:11:34.441 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.441 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.442 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.442 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215593+0000", 2024-09-13T10:11:34.442 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015864300000000001 2024-09-13T10:11:34.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.442 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.442 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:34.442 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226037+0000", 2024-09-13T10:11:34.442 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010443948 2024-09-13T10:11:34.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.442 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.442 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:34.442 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228312+0000", 2024-09-13T10:11:34.442 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022743009999999998 2024-09-13T10:11:34.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.443 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.443 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.443 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228708+0000", 2024-09-13T10:11:34.443 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039651999999999998 2024-09-13T10:11:34.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.443 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.443 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.443 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229625+0000", 2024-09-13T10:11:34.443 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00091692300000000004 2024-09-13T10:11:34.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.443 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.443 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:34.443 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229750+0000", 2024-09-13T10:11:34.444 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012532099999999999 2024-09-13T10:11:34.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.444 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.444 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.444 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229860+0000", 2024-09-13T10:11:34.444 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000109627 2024-09-13T10:11:34.444 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.444 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.444 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.444 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.444 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:260 4.3 4:c426503f:::smithi19530714-11:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:34.444 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:11:34.445 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 43.211172546999997, 2024-09-13T10:11:34.445 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041961871999999997, 2024-09-13T10:11:34.445 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.445 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.445 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:34.445 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.445 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.445 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.445 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 260 2024-09-13T10:11:34.445 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.445 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.445 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.445 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.445 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:11:34.446 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.446 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.446 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.446 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.446 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:11:34.446 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.446 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.446 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.446 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.446 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215336+0000", 2024-09-13T10:11:34.446 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9069999999999999e-06 2024-09-13T10:11:34.446 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.446 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.446 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.447 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215345+0000", 2024-09-13T10:11:34.447 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.6659999999999997e-06 2024-09-13T10:11:34.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.447 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.447 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.447 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215363+0000", 2024-09-13T10:11:34.447 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8153000000000001e-05 2024-09-13T10:11:34.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.447 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.447 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.447 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215388+0000", 2024-09-13T10:11:34.447 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4556e-05 2024-09-13T10:11:34.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.448 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.448 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.448 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215542+0000", 2024-09-13T10:11:34.448 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000154637 2024-09-13T10:11:34.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.448 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.448 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.448 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215825+0000", 2024-09-13T10:11:34.448 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028208600000000002 2024-09-13T10:11:34.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.448 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.448 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:34.448 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254572+0000", 2024-09-13T10:11:34.449 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.038747464000000002 2024-09-13T10:11:34.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.449 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.449 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:34.449 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.255694+0000", 2024-09-13T10:11:34.449 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011222929999999999 2024-09-13T10:11:34.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.449 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.449 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.449 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.256931+0000", 2024-09-13T10:11:34.449 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0012370980000000001 2024-09-13T10:11:34.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.449 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.449 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.450 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257080+0000", 2024-09-13T10:11:34.450 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014874800000000001 2024-09-13T10:11:34.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.450 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.450 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:34.450 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257211+0000", 2024-09-13T10:11:34.450 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000130674 2024-09-13T10:11:34.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.450 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.450 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.450 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257296+0000", 2024-09-13T10:11:34.450 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5589999999999999e-05 2024-09-13T10:11:34.450 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.451 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.451 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.451 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.451 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.451 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:269 4.3 4:c5b5ff75:::smithi19530714-20:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:34.451 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:11:34.451 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 43.210668308999999, 2024-09-13T10:11:34.451 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014988243, 2024-09-13T10:11:34.451 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.451 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.452 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:34.452 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.452 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.452 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.452 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 269 2024-09-13T10:11:34.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.452 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.452 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.452 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.452 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:11:34.452 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.452 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:11:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215841+0000", 2024-09-13T10:11:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0829999999999998e-06 2024-09-13T10:11:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215854+0000", 2024-09-13T10:11:34.453 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2976000000000001e-05 2024-09-13T10:11:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215875+0000", 2024-09-13T10:11:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1100000000000001e-05 2024-09-13T10:11:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215892+0000", 2024-09-13T10:11:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7388999999999999e-05 2024-09-13T10:11:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.454 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216353+0000", 2024-09-13T10:11:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00046100900000000002 2024-09-13T10:11:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216611+0000", 2024-09-13T10:11:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025716299999999998 2024-09-13T10:11:34.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.227177+0000", 2024-09-13T10:11:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010566676000000001 2024-09-13T10:11:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229574+0000", 2024-09-13T10:11:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023964469999999999 2024-09-13T10:11:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.456 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230619+0000", 2024-09-13T10:11:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0010455950000000001 2024-09-13T10:11:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230697+0000", 2024-09-13T10:11:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8021000000000004e-05 2024-09-13T10:11:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230767+0000", 2024-09-13T10:11:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9544999999999999e-05 2024-09-13T10:11:34.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230827+0000", 2024-09-13T10:11:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.0238999999999997e-05 2024-09-13T10:11:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:288 4.3 4:cceea2a7:::smithi19530714-39:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.249445+0000", 2024-09-13T10:11:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 43.177062018000001, 2024-09-13T10:11:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012024603, 2024-09-13T10:11:34.458 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 288 2024-09-13T10:11:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249445+0000", 2024-09-13T10:11:34.459 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249445+0000", 2024-09-13T10:11:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249448+0000", 2024-09-13T10:11:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0350000000000002e-06 2024-09-13T10:11:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.460 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249460+0000", 2024-09-13T10:11:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2214e-05 2024-09-13T10:11:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249490+0000", 2024-09-13T10:11:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0130000000000001e-05 2024-09-13T10:11:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249510+0000", 2024-09-13T10:11:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9757000000000001e-05 2024-09-13T10:11:34.461 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249684+0000", 2024-09-13T10:11:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000174086 2024-09-13T10:11:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249913+0000", 2024-09-13T10:11:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022833400000000001 2024-09-13T10:11:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.462 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:34.463 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257665+0000", 2024-09-13T10:11:34.463 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0077526330000000001 2024-09-13T10:11:34.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.463 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.463 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.463 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.259621+0000", 2024-09-13T10:11:34.463 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019558610000000001 2024-09-13T10:11:34.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.463 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.463 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.463 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260048+0000", 2024-09-13T10:11:34.463 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00042668799999999999 2024-09-13T10:11:34.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.464 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.464 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:34.464 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.261264+0000", 2024-09-13T10:11:34.464 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001216445 2024-09-13T10:11:34.464 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.464 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.464 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:34.464 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.261374+0000", 2024-09-13T10:11:34.464 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110117 2024-09-13T10:11:34.464 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.464 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.464 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.465 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.261470+0000", 2024-09-13T10:11:34.465 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5303000000000006e-05 2024-09-13T10:11:34.465 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.465 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.465 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.465 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.465 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.465 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:310 4.3 4:c426503f:::smithi19530714-11:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:34.465 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:11:34.465 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 43.153372236000003, 2024-09-13T10:11:34.465 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013782279999999999, 2024-09-13T10:11:34.465 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.465 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.466 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:34.466 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.466 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.466 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.466 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 310 2024-09-13T10:11:34.466 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.466 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.466 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.466 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.466 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:11:34.466 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.466 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.467 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.467 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.467 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:11:34.467 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.467 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.467 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.467 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.467 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273138+0000", 2024-09-13T10:11:34.467 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8459999999999999e-06 2024-09-13T10:11:34.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.468 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.468 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.468 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273150+0000", 2024-09-13T10:11:34.468 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2133999999999999e-05 2024-09-13T10:11:34.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.468 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.468 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.468 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273175+0000", 2024-09-13T10:11:34.469 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5321999999999999e-05 2024-09-13T10:11:34.469 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.469 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.469 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.469 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273192+0000", 2024-09-13T10:11:34.469 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.681e-05 2024-09-13T10:11:34.469 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.469 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.469 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.469 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273314+0000", 2024-09-13T10:11:34.469 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012228 2024-09-13T10:11:34.469 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273474+0000", 2024-09-13T10:11:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016019799999999999 2024-09-13T10:11:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.284136+0000", 2024-09-13T10:11:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010661761000000001 2024-09-13T10:11:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.470 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286085+0000", 2024-09-13T10:11:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019484929999999999 2024-09-13T10:11:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286111+0000", 2024-09-13T10:11:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5834e-05 2024-09-13T10:11:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286721+0000", 2024-09-13T10:11:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00061038599999999998 2024-09-13T10:11:34.471 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286838+0000", 2024-09-13T10:11:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117064 2024-09-13T10:11:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286917+0000", 2024-09-13T10:11:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9152000000000004e-05 2024-09-13T10:11:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.472 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.473 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.473 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.473 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:391 4.1 4:8e413aa3:::smithi19530714-42:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:34.473 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.321223+0000", 2024-09-13T10:11:34.473 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 43.105284228999999, 2024-09-13T10:11:34.473 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011654998999999999, 2024-09-13T10:11:34.473 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.473 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.473 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:34.473 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.473 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.473 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.473 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 391 2024-09-13T10:11:34.474 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.474 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.474 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.474 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.474 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321223+0000", 2024-09-13T10:11:34.474 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.474 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.474 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.474 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.474 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321223+0000", 2024-09-13T10:11:34.474 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.474 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.474 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.475 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.475 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321225+0000", 2024-09-13T10:11:34.475 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0360000000000001e-06 2024-09-13T10:11:34.475 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.475 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.475 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.475 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321236+0000", 2024-09-13T10:11:34.475 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1419e-05 2024-09-13T10:11:34.475 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.475 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.475 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.475 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321262+0000", 2024-09-13T10:11:34.475 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6035000000000001e-05 2024-09-13T10:11:34.476 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.476 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.476 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.476 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321280+0000", 2024-09-13T10:11:34.476 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8022000000000001e-05 2024-09-13T10:11:34.476 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.476 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.476 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.476 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321388+0000", 2024-09-13T10:11:34.476 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010752900000000001 2024-09-13T10:11:34.476 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.476 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.476 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.477 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.321518+0000", 2024-09-13T10:11:34.477 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000129901 2024-09-13T10:11:34.477 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.477 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.477 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:34.477 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.329954+0000", 2024-09-13T10:11:34.477 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0084364639999999994 2024-09-13T10:11:34.477 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.477 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.477 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.477 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.331173+0000", 2024-09-13T10:11:34.477 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001218968 2024-09-13T10:11:34.478 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.478 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.478 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:34.478 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.331209+0000", 2024-09-13T10:11:34.478 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6018e-05 2024-09-13T10:11:34.478 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.478 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.478 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.478 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.332764+0000", 2024-09-13T10:11:34.478 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001554466 2024-09-13T10:11:34.478 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.478 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.479 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:34.479 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.332838+0000", 2024-09-13T10:11:34.479 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.3779e-05 2024-09-13T10:11:34.479 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.479 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.479 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.479 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.332878+0000", 2024-09-13T10:11:34.479 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0361999999999997e-05 2024-09-13T10:11:34.479 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.479 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.479 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.479 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.479 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.480 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1006 4.3 4:c5b5ff75:::smithi19530714-20:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:34.480 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:11:34.480 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 38.772458987999997, 2024-09-13T10:11:34.480 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.047764635, 2024-09-13T10:11:34.480 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.480 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.480 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:34.480 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.480 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.480 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.480 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1006 2024-09-13T10:11:34.481 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.481 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.481 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.481 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.481 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:11:34.481 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.481 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.481 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.481 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.482 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:11:34.482 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.482 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.482 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.482 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.482 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654049+0000", 2024-09-13T10:11:34.482 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5600000000000005e-07 2024-09-13T10:11:34.482 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.482 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.482 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.482 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654058+0000", 2024-09-13T10:11:34.482 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8200000000000003e-06 2024-09-13T10:11:34.483 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.483 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.483 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.483 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654067+0000", 2024-09-13T10:11:34.483 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.4599999999999992e-06 2024-09-13T10:11:34.483 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.483 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.483 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.483 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654075+0000", 2024-09-13T10:11:34.483 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.1070000000000005e-06 2024-09-13T10:11:34.483 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.483 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.483 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.484 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654188+0000", 2024-09-13T10:11:34.484 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000112734 2024-09-13T10:11:34.484 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.484 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.484 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.484 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654395+0000", 2024-09-13T10:11:34.484 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000207235 2024-09-13T10:11:34.484 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.484 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.484 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.484 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.701813+0000", 2024-09-13T10:11:34.484 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.047417523000000003 2024-09-13T10:11:34.485 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.485 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.485 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.485 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.485 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.485 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1013 4.3 4:c426503f:::smithi19530714-11:head [write 477332~766101] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:34.485 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:11:34.485 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 38.765137734, 2024-09-13T10:11:34.485 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027363838000000001, 2024-09-13T10:11:34.485 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.485 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.485 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:34.485 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.486 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.486 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.486 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1013 2024-09-13T10:11:34.486 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.486 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.486 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.486 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.486 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:11:34.486 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.486 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.486 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.486 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.487 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:11:34.487 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.487 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.487 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.487 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.487 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661374+0000", 2024-09-13T10:11:34.487 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.0930000000000002e-06 2024-09-13T10:11:34.487 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.487 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.487 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.487 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665828+0000", 2024-09-13T10:11:34.487 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044532119999999998 2024-09-13T10:11:34.487 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.488 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.488 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.488 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665864+0000", 2024-09-13T10:11:34.488 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5979000000000002e-05 2024-09-13T10:11:34.488 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.488 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.488 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.488 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665881+0000", 2024-09-13T10:11:34.488 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7836e-05 2024-09-13T10:11:34.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.489 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.489 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.489 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.678976+0000", 2024-09-13T10:11:34.489 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.01309448 2024-09-13T10:11:34.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.489 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.489 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.489 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.679188+0000", 2024-09-13T10:11:34.489 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000211945 2024-09-13T10:11:34.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.489 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.489 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:34.490 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.679468+0000", 2024-09-13T10:11:34.490 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028034000000000002 2024-09-13T10:11:34.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.490 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.490 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:34.490 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688233+0000", 2024-09-13T10:11:34.490 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0087645369999999993 2024-09-13T10:11:34.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.490 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.490 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.490 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688388+0000", 2024-09-13T10:11:34.490 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015488000000000001 2024-09-13T10:11:34.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.491 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.491 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.491 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688505+0000", 2024-09-13T10:11:34.491 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117114 2024-09-13T10:11:34.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.491 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.491 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:34.491 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688612+0000", 2024-09-13T10:11:34.491 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010741100000000001 2024-09-13T10:11:34.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.491 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.491 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.492 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688733+0000", 2024-09-13T10:11:34.492 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000121011 2024-09-13T10:11:34.492 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.492 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.492 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.492 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.492 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.492 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1014 4.3 4:c426503f:::smithi19530714-11:head [write 1681102~474010] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:34.492 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:11:34.492 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 38.759864348000001, 2024-09-13T10:11:34.492 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022694374, 2024-09-13T10:11:34.492 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.492 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.493 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:34.493 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.493 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.493 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.493 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1014 2024-09-13T10:11:34.493 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.493 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.493 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.493 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.493 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:11:34.493 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.493 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.493 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.494 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.494 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:11:34.494 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.494 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.494 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.494 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.494 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666645+0000", 2024-09-13T10:11:34.494 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2970000000000002e-06 2024-09-13T10:11:34.494 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.494 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.494 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.494 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667326+0000", 2024-09-13T10:11:34.494 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00068073199999999997 2024-09-13T10:11:34.495 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.495 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.495 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.495 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667346+0000", 2024-09-13T10:11:34.495 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0188999999999999e-05 2024-09-13T10:11:34.495 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.495 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.495 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.495 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667361+0000", 2024-09-13T10:11:34.495 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5299999999999999e-05 2024-09-13T10:11:34.495 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.495 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.495 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.496 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.683468+0000", 2024-09-13T10:11:34.496 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016106944000000002 2024-09-13T10:11:34.496 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.496 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.496 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.496 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.683714+0000", 2024-09-13T10:11:34.496 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024595699999999999 2024-09-13T10:11:34.496 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.496 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.496 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:34.496 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.684132+0000", 2024-09-13T10:11:34.496 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00041775800000000001 2024-09-13T10:11:34.497 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.497 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.497 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:34.497 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688771+0000", 2024-09-13T10:11:34.497 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0046394540000000003 2024-09-13T10:11:34.497 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.497 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.497 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.497 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688920+0000", 2024-09-13T10:11:34.497 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014886000000000001 2024-09-13T10:11:34.497 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.497 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.497 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.498 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689085+0000", 2024-09-13T10:11:34.498 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000164539 2024-09-13T10:11:34.498 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.498 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.498 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:34.498 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689230+0000", 2024-09-13T10:11:34.498 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000145394 2024-09-13T10:11:34.498 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.498 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.498 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.498 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689337+0000", 2024-09-13T10:11:34.498 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010695 2024-09-13T10:11:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1015 4.3 4:c426503f:::smithi19530714-11:head [write 2687912~589420] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:11:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 38.753547671, 2024-09-13T10:11:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.018489873, 2024-09-13T10:11:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:34.499 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1015 2024-09-13T10:11:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:11:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.500 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:11:34.501 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.501 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.501 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.501 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672962+0000", 2024-09-13T10:11:34.501 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.277e-06 2024-09-13T10:11:34.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.501 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.501 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.501 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673758+0000", 2024-09-13T10:11:34.501 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00079599100000000002 2024-09-13T10:11:34.501 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.502 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.502 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.502 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673787+0000", 2024-09-13T10:11:34.502 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9280000000000001e-05 2024-09-13T10:11:34.502 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.502 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.502 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.502 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673801+0000", 2024-09-13T10:11:34.502 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4493e-05 2024-09-13T10:11:34.502 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.502 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.502 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.503 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.685885+0000", 2024-09-13T10:11:34.503 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012083692 2024-09-13T10:11:34.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.503 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.503 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.503 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.686057+0000", 2024-09-13T10:11:34.503 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017133900000000001 2024-09-13T10:11:34.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.503 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.503 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:34.503 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.686396+0000", 2024-09-13T10:11:34.503 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000339809 2024-09-13T10:11:34.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.504 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.504 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.504 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691040+0000", 2024-09-13T10:11:34.504 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.004643215 2024-09-13T10:11:34.504 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.504 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.504 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:34.504 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691082+0000", 2024-09-13T10:11:34.504 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.2499000000000001e-05 2024-09-13T10:11:34.504 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.504 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.504 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.504 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691179+0000", 2024-09-13T10:11:34.505 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7044000000000001e-05 2024-09-13T10:11:34.505 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.505 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.505 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:34.505 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691324+0000", 2024-09-13T10:11:34.505 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014532299999999999 2024-09-13T10:11:34.505 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.505 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.505 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.505 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691449+0000", 2024-09-13T10:11:34.505 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012491100000000001 2024-09-13T10:11:34.505 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.505 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.506 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.506 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.506 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.506 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1016 4.3 4:c426503f:::smithi19530714-11:head [setxattr _header (56) in=7b,truncate 3277332] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:34.506 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:11:34.506 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 38.752678713999998, 2024-09-13T10:11:34.506 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.019820562999999999, 2024-09-13T10:11:34.506 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.506 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.506 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:34.506 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.506 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.506 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.507 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1016 2024-09-13T10:11:34.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.507 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.507 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.507 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.507 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:11:34.507 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.507 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.507 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.507 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:11:34.507 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.508 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.508 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.508 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673830+0000", 2024-09-13T10:11:34.508 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1170000000000001e-06 2024-09-13T10:11:34.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.508 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.508 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.508 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673835+0000", 2024-09-13T10:11:34.508 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5559999999999998e-06 2024-09-13T10:11:34.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.508 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.508 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.509 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673845+0000", 2024-09-13T10:11:34.509 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6330000000000008e-06 2024-09-13T10:11:34.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.509 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.509 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.509 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673856+0000", 2024-09-13T10:11:34.509 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0963e-05 2024-09-13T10:11:34.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.509 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.509 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.509 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689480+0000", 2024-09-13T10:11:34.509 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015624605 2024-09-13T10:11:34.510 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.510 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.510 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.510 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689739+0000", 2024-09-13T10:11:34.510 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025878700000000001 2024-09-13T10:11:34.510 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.510 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.510 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:34.510 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.690154+0000", 2024-09-13T10:11:34.510 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00041533800000000001 2024-09-13T10:11:34.510 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.510 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.510 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.511 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691858+0000", 2024-09-13T10:11:34.511 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001703811 2024-09-13T10:11:34.511 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.511 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.511 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:34.511 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691880+0000", 2024-09-13T10:11:34.511 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1997e-05 2024-09-13T10:11:34.511 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.511 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.511 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.511 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693384+0000", 2024-09-13T10:11:34.511 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0015040190000000001 2024-09-13T10:11:34.512 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.512 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.512 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:34.512 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693541+0000", 2024-09-13T10:11:34.512 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015723899999999999 2024-09-13T10:11:34.512 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.512 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.512 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.512 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693649+0000", 2024-09-13T10:11:34.512 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010749800000000001 2024-09-13T10:11:34.512 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.512 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.512 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.513 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.513 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.513 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1017 4.3 4:c426503f:::smithi19530714-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:34.513 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:11:34.513 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 38.752638701999999, 2024-09-13T10:11:34.513 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020166488999999999, 2024-09-13T10:11:34.513 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.513 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.513 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:34.513 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.513 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.513 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.514 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1017 2024-09-13T10:11:34.514 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.514 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.514 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.514 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.514 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:11:34.514 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.514 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.514 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.514 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.514 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:11:34.514 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.514 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.515 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.515 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.515 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673869+0000", 2024-09-13T10:11:34.515 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9699999999999995e-07 2024-09-13T10:11:34.515 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.515 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.515 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.515 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673873+0000", 2024-09-13T10:11:34.515 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.4970000000000001e-06 2024-09-13T10:11:34.515 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.515 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.515 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.516 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673885+0000", 2024-09-13T10:11:34.516 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2398999999999999e-05 2024-09-13T10:11:34.516 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.516 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.516 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.516 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673893+0000", 2024-09-13T10:11:34.516 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7530000000000001e-06 2024-09-13T10:11:34.516 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.516 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.516 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.516 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691638+0000", 2024-09-13T10:11:34.516 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017745007 2024-09-13T10:11:34.516 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.517 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.517 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:34.517 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691789+0000", 2024-09-13T10:11:34.517 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015133400000000001 2024-09-13T10:11:34.517 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.517 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.517 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.517 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693731+0000", 2024-09-13T10:11:34.517 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019422459999999999 2024-09-13T10:11:34.517 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.517 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.517 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.518 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693870+0000", 2024-09-13T10:11:34.518 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013820299999999999 2024-09-13T10:11:34.518 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.518 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.518 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.518 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.694035+0000", 2024-09-13T10:11:34.518 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000165253 2024-09-13T10:11:34.518 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.518 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.519 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.519 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.519 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.519 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1405 4.3 4:c5b5ff75:::smithi19530714-20:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T10:11:34.519 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:01.492787+0000", 2024-09-13T10:11:34.519 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 32.933720057999999, 2024-09-13T10:11:34.519 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021225698000000001, 2024-09-13T10:11:34.519 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.519 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.520 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:34.520 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.520 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.520 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.520 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1405 2024-09-13T10:11:34.520 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.520 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.520 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.520 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.520 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492787+0000", 2024-09-13T10:11:34.520 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.520 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.521 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.521 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.521 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492787+0000", 2024-09-13T10:11:34.521 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.521 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.521 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.521 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.521 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492791+0000", 2024-09-13T10:11:34.521 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.89e-06 2024-09-13T10:11:34.521 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.521 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.521 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.521 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492805+0000", 2024-09-13T10:11:34.522 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4083000000000001e-05 2024-09-13T10:11:34.522 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.522 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.522 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.522 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492846+0000", 2024-09-13T10:11:34.522 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0849000000000001e-05 2024-09-13T10:11:34.522 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.522 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.522 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.522 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492867+0000", 2024-09-13T10:11:34.522 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1058999999999999e-05 2024-09-13T10:11:34.522 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.523 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.523 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.523 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492973+0000", 2024-09-13T10:11:34.523 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000106084 2024-09-13T10:11:34.523 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.523 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.523 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.523 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.493105+0000", 2024-09-13T10:11:34.523 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013215899999999999 2024-09-13T10:11:34.523 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.523 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.523 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.523 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.514013+0000", 2024-09-13T10:11:34.524 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020907574000000002 2024-09-13T10:11:34.524 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.524 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.524 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.524 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.524 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.524 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2658 4.3 4:c5b5ff75:::smithi19530714-20:head [setxattr _header (56) in=7b,truncate 3899224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:34.524 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:28.049623+0000", 2024-09-13T10:11:34.524 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 6.376883791, 2024-09-13T10:11:34.524 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012241349, 2024-09-13T10:11:34.524 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.524 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.525 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:34.525 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.525 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.525 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.525 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2658 2024-09-13T10:11:34.525 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.525 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.525 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.525 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.525 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049623+0000", 2024-09-13T10:11:34.525 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.525 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.526 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.526 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.526 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049623+0000", 2024-09-13T10:11:34.526 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.526 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.526 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.526 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.526 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049625+0000", 2024-09-13T10:11:34.526 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9439999999999999e-06 2024-09-13T10:11:34.526 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.526 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.526 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.526 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049635+0000", 2024-09-13T10:11:34.527 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5589999999999994e-06 2024-09-13T10:11:34.527 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.527 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.527 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.527 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049652+0000", 2024-09-13T10:11:34.527 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.751e-05 2024-09-13T10:11:34.527 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.527 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.527 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.527 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049668+0000", 2024-09-13T10:11:34.527 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5214999999999999e-05 2024-09-13T10:11:34.527 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.528 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.528 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.528 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.052042+0000", 2024-09-13T10:11:34.528 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023748350000000001 2024-09-13T10:11:34.528 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.528 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.528 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.528 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.052248+0000", 2024-09-13T10:11:34.528 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000205651 2024-09-13T10:11:34.528 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.528 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.528 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:34.528 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.052639+0000", 2024-09-13T10:11:34.529 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039102000000000001 2024-09-13T10:11:34.529 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.529 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.529 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.529 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.054795+0000", 2024-09-13T10:11:34.529 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002155587 2024-09-13T10:11:34.529 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.529 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.529 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.529 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.061492+0000", 2024-09-13T10:11:34.529 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0066973409999999999 2024-09-13T10:11:34.529 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.530 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.530 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:34.530 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.061539+0000", 2024-09-13T10:11:34.530 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.7515000000000002e-05 2024-09-13T10:11:34.530 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.530 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.530 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:34.530 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.061691+0000", 2024-09-13T10:11:34.530 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015110300000000001 2024-09-13T10:11:34.530 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.530 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.530 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.530 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.061865+0000", 2024-09-13T10:11:34.531 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000174069 2024-09-13T10:11:34.531 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.531 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.531 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.531 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.531 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.531 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2659 4.3 4:c5b5ff75:::smithi19530714-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:34.531 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:28.049693+0000", 2024-09-13T10:11:34.531 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 6.376814134, 2024-09-13T10:11:34.531 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012500902, 2024-09-13T10:11:34.531 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.531 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.531 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:34.531 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.532 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.532 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.532 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2659 2024-09-13T10:11:34.532 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.532 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.532 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.532 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.532 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049693+0000", 2024-09-13T10:11:34.532 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.532 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.533 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.533 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.533 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049693+0000", 2024-09-13T10:11:34.533 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.533 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.533 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.533 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.533 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049694+0000", 2024-09-13T10:11:34.533 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3740000000000001e-06 2024-09-13T10:11:34.533 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.533 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.534 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.534 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049701+0000", 2024-09-13T10:11:34.534 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.5699999999999998e-06 2024-09-13T10:11:34.534 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.534 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.534 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.534 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049717+0000", 2024-09-13T10:11:34.534 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5795e-05 2024-09-13T10:11:34.534 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.534 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.534 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.534 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049729+0000", 2024-09-13T10:11:34.535 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2153e-05 2024-09-13T10:11:34.535 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.535 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.535 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.535 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.053546+0000", 2024-09-13T10:11:34.535 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0038173769999999998 2024-09-13T10:11:34.535 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.535 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.535 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:34.535 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.053767+0000", 2024-09-13T10:11:34.535 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022096099999999999 2024-09-13T10:11:34.535 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.535 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.536 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.536 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.061937+0000", 2024-09-13T10:11:34.536 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0081695359999999998 2024-09-13T10:11:34.536 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.536 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.536 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.536 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.062061+0000", 2024-09-13T10:11:34.536 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012419800000000001 2024-09-13T10:11:34.536 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.536 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.536 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.536 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.062194+0000", 2024-09-13T10:11:34.536 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013293800000000001 2024-09-13T10:11:34.537 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.537 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.537 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.537 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.537 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.537 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2816 4.3 4:c5b5ff75:::smithi19530714-20:head [assert-version v76,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:34.537 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.891848+0000", 2024-09-13T10:11:34.537 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 1.5346590330000001, 2024-09-13T10:11:34.537 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020571445000000001, 2024-09-13T10:11:34.537 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.537 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.537 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:34.538 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.538 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.538 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.538 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2816 2024-09-13T10:11:34.538 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.538 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.538 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.538 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.538 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891848+0000", 2024-09-13T10:11:34.538 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.538 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.538 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.538 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.539 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891848+0000", 2024-09-13T10:11:34.539 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.539 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.539 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.539 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.539 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891852+0000", 2024-09-13T10:11:34.539 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1729999999999997e-06 2024-09-13T10:11:34.539 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.539 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.539 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.539 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891869+0000", 2024-09-13T10:11:34.539 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6639000000000001e-05 2024-09-13T10:11:34.539 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.540 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.540 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.540 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891910+0000", 2024-09-13T10:11:34.540 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1125999999999999e-05 2024-09-13T10:11:34.540 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.540 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.540 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.540 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891934+0000", 2024-09-13T10:11:34.540 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4039000000000001e-05 2024-09-13T10:11:34.540 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.540 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.540 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.540 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892099+0000", 2024-09-13T10:11:34.541 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016462000000000001 2024-09-13T10:11:34.541 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.541 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.541 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.541 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892368+0000", 2024-09-13T10:11:34.541 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026922699999999998 2024-09-13T10:11:34.541 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.541 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.541 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:34.541 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.907670+0000", 2024-09-13T10:11:34.541 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015302299 2024-09-13T10:11:34.541 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.541 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.541 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.542 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.910850+0000", 2024-09-13T10:11:34.542 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003179383 2024-09-13T10:11:34.542 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.542 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.542 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:34.542 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.910901+0000", 2024-09-13T10:11:34.542 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.1548000000000002e-05 2024-09-13T10:11:34.542 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.542 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.542 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:34.542 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.912089+0000", 2024-09-13T10:11:34.542 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011878170000000001 2024-09-13T10:11:34.542 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.543 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.543 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:34.543 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.912267+0000", 2024-09-13T10:11:34.543 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000178393 2024-09-13T10:11:34.543 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.543 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.543 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.543 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.912420+0000", 2024-09-13T10:11:34.543 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015218100000000001 2024-09-13T10:11:34.543 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.543 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.543 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.543 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.544 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.544 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2817 4.3 4:c426503f:::smithi19530714-11:head [read 0~3647119,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:34.544 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.892018+0000", 2024-09-13T10:11:34.544 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 1.5344895629999999, 2024-09-13T10:11:34.544 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013334828, 2024-09-13T10:11:34.544 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.544 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.544 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:34.544 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.544 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.544 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.544 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2817 2024-09-13T10:11:34.544 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.545 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.545 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.545 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.545 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892018+0000", 2024-09-13T10:11:34.545 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.545 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.545 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.545 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.545 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892018+0000", 2024-09-13T10:11:34.545 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.545 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.545 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.545 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.546 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892019+0000", 2024-09-13T10:11:34.546 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.747e-06 2024-09-13T10:11:34.546 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.546 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.546 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.546 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892031+0000", 2024-09-13T10:11:34.546 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.149e-05 2024-09-13T10:11:34.546 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.546 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.546 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.546 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892053+0000", 2024-09-13T10:11:34.546 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2069000000000002e-05 2024-09-13T10:11:34.546 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.546 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.547 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.547 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892073+0000", 2024-09-13T10:11:34.547 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0591999999999999e-05 2024-09-13T10:11:34.547 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.547 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.547 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.547 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892825+0000", 2024-09-13T10:11:34.547 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00075120299999999998 2024-09-13T10:11:34.547 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.547 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.547 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.547 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893031+0000", 2024-09-13T10:11:34.547 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000206579 2024-09-13T10:11:34.548 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.548 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.548 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.548 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.905352+0000", 2024-09-13T10:11:34.548 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012321148000000001 2024-09-13T10:11:34.548 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.548 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.548 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.548 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.548 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.548 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2818 4.3 4:c426503f:::smithi19530714-11:head [sparse-read 0~2823261] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:34.548 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.892118+0000", 2024-09-13T10:11:34.548 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 1.5343887729999999, 2024-09-13T10:11:34.548 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014982577, 2024-09-13T10:11:34.549 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.549 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.549 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:34.549 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.549 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.549 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.549 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2818 2024-09-13T10:11:34.549 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.549 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.549 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.549 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.549 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892118+0000", 2024-09-13T10:11:34.550 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.550 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.550 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.550 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.550 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892118+0000", 2024-09-13T10:11:34.550 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.550 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.550 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.550 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.550 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892120+0000", 2024-09-13T10:11:34.550 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5260000000000001e-06 2024-09-13T10:11:34.550 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.550 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.550 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.551 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892128+0000", 2024-09-13T10:11:34.551 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7640000000000005e-06 2024-09-13T10:11:34.551 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.551 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.551 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.551 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892172+0000", 2024-09-13T10:11:34.551 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4561000000000002e-05 2024-09-13T10:11:34.551 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.551 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.551 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.551 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892231+0000", 2024-09-13T10:11:34.551 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9194999999999998e-05 2024-09-13T10:11:34.551 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.551 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.551 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.552 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.903452+0000", 2024-09-13T10:11:34.552 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011221024 2024-09-13T10:11:34.552 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.552 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.552 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.552 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.903711+0000", 2024-09-13T10:11:34.552 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025891100000000002 2024-09-13T10:11:34.552 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.552 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.552 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.552 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.907101+0000", 2024-09-13T10:11:34.552 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0033895959999999999 2024-09-13T10:11:34.552 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.553 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.553 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.553 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.553 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.553 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2819 4.3 4:c426503f:::smithi19530714-11:head [sparse-read 1467121~1802604] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:34.553 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.892259+0000", 2024-09-13T10:11:34.553 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 1.5342477649999999, 2024-09-13T10:11:34.553 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.018443480000000002, 2024-09-13T10:11:34.553 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:34.553 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:34.553 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:34.553 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:34.553 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:34.553 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:34.554 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2819 2024-09-13T10:11:34.554 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.554 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:34.554 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.554 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:34.554 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892259+0000", 2024-09-13T10:11:34.554 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.554 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.554 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.554 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:34.554 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892259+0000", 2024-09-13T10:11:34.554 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:34.554 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.555 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.555 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:34.555 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892261+0000", 2024-09-13T10:11:34.555 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5200000000000001e-06 2024-09-13T10:11:34.555 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.555 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.555 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:34.555 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892287+0000", 2024-09-13T10:11:34.555 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5976e-05 2024-09-13T10:11:34.555 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.555 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.555 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:34.556 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892306+0000", 2024-09-13T10:11:34.556 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9208999999999999e-05 2024-09-13T10:11:34.556 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.556 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.556 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:34.556 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892322+0000", 2024-09-13T10:11:34.556 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6426000000000002e-05 2024-09-13T10:11:34.556 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.556 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.556 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:34.556 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.908963+0000", 2024-09-13T10:11:34.556 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016640922999999998 2024-09-13T10:11:34.556 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.556 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.556 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:34.557 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.909202+0000", 2024-09-13T10:11:34.557 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023879500000000001 2024-09-13T10:11:34.557 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:34.557 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:34.557 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:34.557 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.910703+0000", 2024-09-13T10:11:34.557 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0015006310000000001 2024-09-13T10:11:34.557 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.557 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.557 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.557 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:34.557 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:34.557 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:34.558 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:34.427+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:34.559 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T10:11:34.560 INFO:tasks.rados.rados.0.smithi195.stdout:1308: got (2) No such file or directory 2024-09-13T10:11:34.560 INFO:tasks.rados.rados.0.smithi195.stdout:1308: got expected ENOENT (src dne) 2024-09-13T10:11:34.560 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 25 (ObjNum 129 snap 0 seq_num 644488928) clean dne 2024-09-13T10:11:34.560 INFO:tasks.rados.rados.0.smithi195.stdout:1307: got (0) Success 2024-09-13T10:11:34.560 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 31 (ObjNum 244 snap 0 seq_num 244) dirty exists 2024-09-13T10:11:34.560 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:34.561 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:34.561 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:34.561 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:34.562 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:34.562 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:34.562 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:34.562 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:34.563 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:34.563 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:34.563 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:34.564 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:34.564 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:34.558+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:34.564 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:34.565 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:34.587 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:34.587 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:34.604 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:34.605 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:34.627 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:34.627 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:34.645 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:34.646 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:34.661 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:34.658+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:34.667 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:34.668 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:34.687 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:34.687 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:34.710 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:34.710 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:34.725 INFO:tasks.rados.rados.0.smithi195.stdout:1306: done (4 left) 2024-09-13T10:11:34.725 INFO:tasks.rados.rados.0.smithi195.stdout:1307: done (3 left) 2024-09-13T10:11:34.725 INFO:tasks.rados.rados.0.smithi195.stdout:1308: done (2 left) 2024-09-13T10:11:34.725 INFO:tasks.rados.rados.0.smithi195.stdout:1309: done (1 left) 2024-09-13T10:11:34.725 INFO:tasks.rados.rados.0.smithi195.stdout:1310: done (0 left) 2024-09-13T10:11:34.725 INFO:tasks.rados.rados.0.smithi195.stdout:1311: tier_promote oid 6 2024-09-13T10:11:34.726 INFO:tasks.rados.rados.0.smithi195.stdout:1312: delete oid 18 current snap is 0 2024-09-13T10:11:34.728 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:34.728 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:34.730 INFO:tasks.rados.rados.0.smithi195.stdout:1312: done (1 left) 2024-09-13T10:11:34.730 INFO:tasks.rados.rados.0.smithi195.stdout:1313: read oid 40 snap -1 2024-09-13T10:11:34.730 INFO:tasks.rados.rados.0.smithi195.stdout:1313: expect deleted 2024-09-13T10:11:34.731 INFO:tasks.rados.rados.0.smithi195.stdout:1314: write oid 24 current snap is 0 2024-09-13T10:11:34.731 INFO:tasks.rados.rados.0.smithi195.stdout:1314: seq_num 252 ranges {635364=402397,1763227=472137} 2024-09-13T10:11:34.734 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:34.734 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:34.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:34.732+0000 7f5af6a0c640 0 -- 172.21.15.73:0/635486525 >> v2:172.21.15.195:6805/848686366 conn(0x7f5af0222410 msgr2=0x7f5af0224800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-13T10:11:34.744 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T10:11:34.748 INFO:tasks.rados.rados.0.smithi195.stdout:1314: writing smithi19530714-24 from 635364 to 1037761 tid 1 2024-09-13T10:11:34.762 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:34.756+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:34.768 INFO:tasks.rados.rados.0.smithi195.stdout:1314: writing smithi19530714-24 from 1763227 to 2235364 tid 2 2024-09-13T10:11:34.768 INFO:tasks.rados.rados.0.smithi195.stdout:1315: read oid 21 snap -1 2024-09-13T10:11:34.768 INFO:tasks.rados.rados.0.smithi195.stdout:1315: expect (ObjNum 235 snap 0 seq_num 235) 2024-09-13T10:11:34.768 INFO:tasks.rados.rados.0.smithi195.stdout:1316: write (excl) oid 33 current snap is 0 2024-09-13T10:11:34.768 INFO:tasks.rados.rados.0.smithi195.stdout:1316: seq_num 253 ranges {652171=477646,1821709=430462} 2024-09-13T10:11:34.784 INFO:tasks.rados.rados.0.smithi195.stdout:1316: writing smithi19530714-33 from 652171 to 1129817 tid 1 2024-09-13T10:11:34.797 INFO:tasks.rados.rados.0.smithi195.stdout:1316: writing smithi19530714-33 from 1821709 to 2252171 tid 2 2024-09-13T10:11:34.797 INFO:tasks.rados.rados.0.smithi195.stdout:1317: read oid 32 snap -1 2024-09-13T10:11:34.797 INFO:tasks.rados.rados.0.smithi195.stdout:1317: expect (ObjNum 225 snap 0 seq_num 225) 2024-09-13T10:11:34.797 INFO:tasks.rados.rados.0.smithi195.stdout:1318: tier_flush oid 35 2024-09-13T10:11:34.798 INFO:tasks.rados.rados.0.smithi195.stdout:1318: tier_flush oid 35 snap -1 2024-09-13T10:11:34.798 INFO:tasks.rados.rados.0.smithi195.stdout:1311: got (0) Success 2024-09-13T10:11:34.798 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 66 (ObjNum 229 snap 0 seq_num 229) dirty exists 2024-09-13T10:11:34.798 INFO:tasks.rados.rados.0.smithi195.stdout:1314: finishing write tid 1 to smithi19530714-24 2024-09-13T10:11:34.798 INFO:tasks.rados.rados.0.smithi195.stdout:1314: finishing write tid 2 to smithi19530714-24 2024-09-13T10:11:34.798 INFO:tasks.rados.rados.0.smithi195.stdout:1314: finishing write tid 3 to smithi19530714-24 2024-09-13T10:11:34.798 INFO:tasks.rados.rados.0.smithi195.stdout:1314: finishing write tid 4 to smithi19530714-24 2024-09-13T10:11:34.798 INFO:tasks.rados.rados.0.smithi195.stdout:1314: oid 24 updating version 0 to 121 2024-09-13T10:11:34.798 INFO:tasks.rados.rados.0.smithi195.stdout:1314: oid 24 updating version 121 to 123 2024-09-13T10:11:34.798 INFO:tasks.rados.rados.0.smithi195.stdout:1314: oid 24 version 123 is already newer than 122 2024-09-13T10:11:34.798 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 123 (ObjNum 252 snap 0 seq_num 252) dirty exists 2024-09-13T10:11:34.798 INFO:tasks.rados.rados.0.smithi195.stdout:1314: left oid 24 (ObjNum 252 snap 0 seq_num 252) 2024-09-13T10:11:34.798 INFO:tasks.rados.rados.0.smithi195.stdout:1315: expect (ObjNum 235 snap 0 seq_num 235) 2024-09-13T10:11:34.826 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2451244 2024-09-13T10:11:34.863 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:34.859+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:34.872 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:34.872 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:34.875 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 347501 2024-09-13T10:11:34.886 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T10:11:34.957 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1908877 2024-09-13T10:11:34.965 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:34.965 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:34.972 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:34.971+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:35.005 INFO:tasks.rados.rados.0.smithi195.stdout:1316: finishing write tid 1 to smithi19530714-33 2024-09-13T10:11:35.005 INFO:tasks.rados.rados.0.smithi195.stdout:1311: done (6 left) 2024-09-13T10:11:35.005 INFO:tasks.rados.rados.0.smithi195.stdout:1313: done (5 left) 2024-09-13T10:11:35.005 INFO:tasks.rados.rados.0.smithi195.stdout:1314: done (4 left) 2024-09-13T10:11:35.005 INFO:tasks.rados.rados.0.smithi195.stdout:1315: done (3 left) 2024-09-13T10:11:35.005 INFO:tasks.rados.rados.0.smithi195.stdout:1319: read oid 5 snap -1 2024-09-13T10:11:35.005 INFO:tasks.rados.rados.0.smithi195.stdout:1319: expect (ObjNum 231 snap 0 seq_num 231) 2024-09-13T10:11:35.005 INFO:tasks.rados.rados.0.smithi195.stdout:1320: read oid 49 snap -1 2024-09-13T10:11:35.005 INFO:tasks.rados.rados.0.smithi195.stdout:1320: expect (ObjNum 181 snap 0 seq_num 181) 2024-09-13T10:11:35.005 INFO:tasks.rados.rados.0.smithi195.stdout:1321: write oid 23 current snap is 0 2024-09-13T10:11:35.005 INFO:tasks.rados.rados.0.smithi195.stdout:1316: finishing write tid 2 to smithi19530714-33 2024-09-13T10:11:35.006 INFO:tasks.rados.rados.0.smithi195.stdout:1316: finishing write tid 3 to smithi19530714-33 2024-09-13T10:11:35.006 INFO:tasks.rados.rados.0.smithi195.stdout:1316: finishing write tid 4 to smithi19530714-33 2024-09-13T10:11:35.006 INFO:tasks.rados.rados.0.smithi195.stdout:1316: oid 33 updating version 0 to 49 2024-09-13T10:11:35.006 INFO:tasks.rados.rados.0.smithi195.stdout:1316: oid 33 updating version 49 to 50 2024-09-13T10:11:35.006 INFO:tasks.rados.rados.0.smithi195.stdout:1316: oid 33 updating version 50 to 51 2024-09-13T10:11:35.006 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 51 (ObjNum 253 snap 0 seq_num 253) dirty exists 2024-09-13T10:11:35.006 INFO:tasks.rados.rados.0.smithi195.stdout:1316: left oid 33 (ObjNum 253 snap 0 seq_num 253) 2024-09-13T10:11:35.006 INFO:tasks.rados.rados.0.smithi195.stdout:1317: expect (ObjNum 225 snap 0 seq_num 225) 2024-09-13T10:11:35.006 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:35.050 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:35.051 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:35.066 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-13T10:11:35.067 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:35.060+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:35.135 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2944586 2024-09-13T10:11:35.157 INFO:tasks.rados.rados.0.smithi195.stdout:1318: got (0) Success 2024-09-13T10:11:35.157 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 0 (ObjNum 208 snap 0 seq_num 208) dirty exists 2024-09-13T10:11:35.157 INFO:tasks.rados.rados.0.smithi195.stdout:1320: expect (ObjNum 181 snap 0 seq_num 181) 2024-09-13T10:11:35.167 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:35.164+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:35.219 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:35.232 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:35.232 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:35.246 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-13T10:11:35.268 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:35.264+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:35.336 INFO:tasks.rados.rados.0.smithi195.stdout:1319: expect (ObjNum 231 snap 0 seq_num 231) 2024-09-13T10:11:35.369 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:35.369 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:35.369 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:35.367+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:35.385 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-13T10:11:35.404 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:35.471 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:35.468+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:35.538 INFO:tasks.rados.rados.0.smithi195.stdout:1321: seq_num 254 ranges {668978=552895,1763838=505140} 2024-09-13T10:11:35.551 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:35.552 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:35.558 INFO:tasks.rados.rados.0.smithi195.stdout:1321: writing smithi19530714-23 from 668978 to 1221873 tid 1 2024-09-13T10:11:35.566 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-13T10:11:35.571 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:35.566+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:35.576 INFO:tasks.rados.rados.0.smithi195.stdout:1321: writing smithi19530714-23 from 1763838 to 2268978 tid 2 2024-09-13T10:11:35.576 INFO:tasks.rados.rados.0.smithi195.stdout:1316: done (5 left) 2024-09-13T10:11:35.577 INFO:tasks.rados.rados.0.smithi195.stdout:1317: done (4 left) 2024-09-13T10:11:35.577 INFO:tasks.rados.rados.0.smithi195.stdout:1318: done (3 left) 2024-09-13T10:11:35.577 INFO:tasks.rados.rados.0.smithi195.stdout:1319: done (2 left) 2024-09-13T10:11:35.577 INFO:tasks.rados.rados.0.smithi195.stdout:1320: done (1 left) 2024-09-13T10:11:35.577 INFO:tasks.rados.rados.0.smithi195.stdout:1322: read oid 19 snap -1 2024-09-13T10:11:35.577 INFO:tasks.rados.rados.0.smithi195.stdout:1322: expect (ObjNum 160 snap 0 seq_num 160) 2024-09-13T10:11:35.577 INFO:tasks.rados.rados.0.smithi195.stdout:1323: read oid 32 snap -1 2024-09-13T10:11:35.577 INFO:tasks.rados.rados.0.smithi195.stdout:1323: expect (ObjNum 225 snap 0 seq_num 225) 2024-09-13T10:11:35.577 INFO:tasks.rados.rados.0.smithi195.stdout:1324: set_chunk oid 4 target oid 2024-09-13T10:11:35.577 INFO:tasks.rados.rados.0.smithi195.stdout:1324: set_chunk oid 4 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:35.577 INFO:tasks.rados.rados.0.smithi195.stdout:1325: delete oid 30 current snap is 0 2024-09-13T10:11:35.577 INFO:tasks.rados.rados.0.smithi195.stdout:1321: finishing write tid 1 to smithi19530714-23 2024-09-13T10:11:35.578 INFO:tasks.rados.rados.0.smithi195.stdout:1324: finishing set_chunk to oid 4 2024-09-13T10:11:35.578 INFO:tasks.rados.rados.0.smithi195.stdout:1324: got expected ENOENT (src dne) 2024-09-13T10:11:35.581 INFO:tasks.rados.rados.0.smithi195.stdout:1324: done (4 left) 2024-09-13T10:11:35.581 INFO:tasks.rados.rados.0.smithi195.stdout:1325: done (3 left) 2024-09-13T10:11:35.581 INFO:tasks.rados.rados.0.smithi195.stdout:1326: delete oid 8 current snap is 0 2024-09-13T10:11:35.582 INFO:tasks.rados.rados.0.smithi195.stdout:1321: finishing write tid 2 to smithi19530714-23 2024-09-13T10:11:35.584 INFO:tasks.rados.rados.0.smithi195.stdout:1321: finishing write tid 3 to smithi19530714-23 2024-09-13T10:11:35.585 INFO:tasks.rados.rados.0.smithi195.stdout:1321: finishing write tid 4 to smithi19530714-23 2024-09-13T10:11:35.585 INFO:tasks.rados.rados.0.smithi195.stdout:1321: oid 23 updating version 0 to 36 2024-09-13T10:11:35.585 INFO:tasks.rados.rados.0.smithi195.stdout:1321: oid 23 updating version 36 to 37 2024-09-13T10:11:35.585 INFO:tasks.rados.rados.0.smithi195.stdout:1321: oid 23 updating version 37 to 38 2024-09-13T10:11:35.585 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 38 (ObjNum 254 snap 0 seq_num 254) dirty exists 2024-09-13T10:11:35.585 INFO:tasks.rados.rados.0.smithi195.stdout:1321: left oid 23 (ObjNum 254 snap 0 seq_num 254) 2024-09-13T10:11:35.587 INFO:tasks.rados.rados.0.smithi195.stdout:1322: expect (ObjNum 160 snap 0 seq_num 160) 2024-09-13T10:11:35.587 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:35.658 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:35.672 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:35.668+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:35.691 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:35.691 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:35.705 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-13T10:11:35.731 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T10:11:35.779 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:35.776+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:35.791 INFO:tasks.rados.rados.0.smithi195.stdout:1323: expect (ObjNum 225 snap 0 seq_num 225) 2024-09-13T10:11:35.797 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 626148 2024-09-13T10:11:35.827 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:35.827 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:35.841 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-13T10:11:35.842 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:35.873 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:35.868+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:35.943 INFO:tasks.rados.rados.0.smithi195.stdout:1321: done (3 left) 2024-09-13T10:11:35.943 INFO:tasks.rados.rados.0.smithi195.stdout:1322: done (2 left) 2024-09-13T10:11:35.943 INFO:tasks.rados.rados.0.smithi195.stdout:1323: done (1 left) 2024-09-13T10:11:35.943 INFO:tasks.rados.rados.0.smithi195.stdout:1326: done (0 left) 2024-09-13T10:11:35.943 INFO:tasks.rados.rados.0.smithi195.stdout:1327: set_chunk oid 28 target oid 2024-09-13T10:11:35.957 INFO:tasks.rados.rados.0.smithi195.stdout:1327: set_chunk oid 28 offset: 1645568 length: 588800 target oid 70f543ceac13557aac6181fdd52e54fc5ed9e711a82b2c0209d8a5106e22ca9c offset: 0 snap -1 2024-09-13T10:11:35.957 INFO:tasks.rados.rados.0.smithi195.stdout:1328: delete oid 8 current snap is 0 2024-09-13T10:11:35.960 INFO:tasks.rados.rados.0.smithi195.stdout:1328: done (1 left) 2024-09-13T10:11:35.960 INFO:tasks.rados.rados.0.smithi195.stdout:1329: set_chunk oid 42 target oid 2024-09-13T10:11:35.971 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:35.971 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:35.973 INFO:tasks.rados.rados.0.smithi195.stdout:1329: set_chunk oid 42 offset: 1644544 length: 771584 target oid d060d39c62fa63798f1f012cd3aa4bc7e14eb41c2fba351989f2bc6c583fc9c0 offset: 0 snap -1 2024-09-13T10:11:35.973 INFO:tasks.rados.rados.0.smithi195.stdout:1330: tier_evict oid 24 2024-09-13T10:11:35.973 INFO:tasks.rados.rados.0.smithi195.stdout:1330: tier_evict oid 24 snap -1 2024-09-13T10:11:35.974 INFO:tasks.rados.rados.0.smithi195.stdout:1327: finishing set_chunk to oid 28 2024-09-13T10:11:35.974 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 96 (ObjNum 180 snap 0 seq_num 180) dirty exists 2024-09-13T10:11:35.974 INFO:tasks.rados.rados.0.smithi195.stdout:1327: done (2 left) 2024-09-13T10:11:35.974 INFO:tasks.rados.rados.0.smithi195.stdout:1331: read oid 34 snap -1 2024-09-13T10:11:35.974 INFO:tasks.rados.rados.0.smithi195.stdout:1331: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:35.974 INFO:tasks.rados.rados.0.smithi195.stdout:1332: tier_promote oid 18 2024-09-13T10:11:35.974 INFO:tasks.rados.rados.0.smithi195.stdout:1333: read oid 41 snap -1 2024-09-13T10:11:35.974 INFO:tasks.rados.rados.0.smithi195.stdout:1333: expect (ObjNum 236 snap 0 seq_num 236) 2024-09-13T10:11:35.974 INFO:tasks.rados.rados.0.smithi195.stdout:1334: write (excl) oid 16 current snap is 0 2024-09-13T10:11:35.974 INFO:tasks.rados.rados.0.smithi195.stdout:1334: seq_num 258 ranges {736206=453891,1881413=454793} 2024-09-13T10:11:35.976 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:35.971+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:35.987 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T10:11:35.996 INFO:tasks.rados.rados.0.smithi195.stdout:1334: writing smithi19530714-16 from 736206 to 1190097 tid 1 2024-09-13T10:11:36.015 INFO:tasks.rados.rados.0.smithi195.stdout:1334: writing smithi19530714-16 from 1881413 to 2336206 tid 2 2024-09-13T10:11:36.015 INFO:tasks.rados.rados.0.smithi195.stdout:1335: set_chunk oid 17 target oid 2024-09-13T10:11:36.015 INFO:tasks.rados.rados.0.smithi195.stdout:1335: set_chunk oid 17 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:36.015 INFO:tasks.rados.rados.0.smithi195.stdout:1336: read oid 22 snap -1 2024-09-13T10:11:36.015 INFO:tasks.rados.rados.0.smithi195.stdout:1336: expect (ObjNum 139 snap 0 seq_num 139) 2024-09-13T10:11:36.016 INFO:tasks.rados.rados.0.smithi195.stdout:1337: read oid 44 snap -1 2024-09-13T10:11:36.016 INFO:tasks.rados.rados.0.smithi195.stdout:1337: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:11:36.016 INFO:tasks.rados.rados.0.smithi195.stdout:1338: tier_promote oid 39 2024-09-13T10:11:36.016 INFO:tasks.rados.rados.0.smithi195.stdout:1329: finishing set_chunk to oid 42 2024-09-13T10:11:36.016 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 42 set_chunk returned error code -95 offset: 1644544 length: 771584 2024-09-13T10:11:36.016 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 0 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-09-13T10:11:36.016 INFO:tasks.rados.rados.0.smithi195.stdout:1330: got (22) Invalid argument 2024-09-13T10:11:36.016 INFO:tasks.rados.rados.0.smithi195.stdout:1332: got (2) No such file or directory 2024-09-13T10:11:36.016 INFO:tasks.rados.rados.0.smithi195.stdout:1332: got expected ENOENT (src dne) 2024-09-13T10:11:36.016 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 89 (ObjNum 129 snap 0 seq_num 644434360) clean dne 2024-09-13T10:11:36.016 INFO:tasks.rados.rados.0.smithi195.stdout:1331: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:36.053 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:36.077 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:36.075+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:36.112 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:36.112 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:36.126 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T10:11:36.140 INFO:tasks.rados.rados.0.smithi195.stdout:1333: expect (ObjNum 236 snap 0 seq_num 236) 2024-09-13T10:11:36.178 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:36.175+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:36.206 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:36.249 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:36.249 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:36.261 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T10:11:36.280 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:36.276+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:36.286 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:36.367 INFO:tasks.rados.rados.0.smithi195.stdout:1334: finishing write tid 1 to smithi19530714-16 2024-09-13T10:11:36.367 INFO:tasks.rados.rados.0.smithi195.stdout:1329: done (9 left) 2024-09-13T10:11:36.367 INFO:tasks.rados.rados.0.smithi195.stdout:1330: done (8 left) 2024-09-13T10:11:36.367 INFO:tasks.rados.rados.0.smithi195.stdout:1331: done (7 left) 2024-09-13T10:11:36.367 INFO:tasks.rados.rados.0.smithi195.stdout:1332: done (6 left) 2024-09-13T10:11:36.367 INFO:tasks.rados.rados.0.smithi195.stdout:1333: done (5 left) 2024-09-13T10:11:36.368 INFO:tasks.rados.rados.0.smithi195.stdout:1339: tier_evict oid 26 2024-09-13T10:11:36.368 INFO:tasks.rados.rados.0.smithi195.stdout:1339: tier_evict oid 26 snap -1 2024-09-13T10:11:36.368 INFO:tasks.rados.rados.0.smithi195.stdout:1335: finishing set_chunk to oid 17 2024-09-13T10:11:36.368 INFO:tasks.rados.rados.0.smithi195.stdout:1335: got expected ENOENT (src dne) 2024-09-13T10:11:36.368 INFO:tasks.rados.rados.0.smithi195.stdout:1338: got (0) Success 2024-09-13T10:11:36.368 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 27 (ObjNum 241 snap 0 seq_num 241) dirty exists 2024-09-13T10:11:36.368 INFO:tasks.rados.rados.0.smithi195.stdout:1334: finishing write tid 2 to smithi19530714-16 2024-09-13T10:11:36.368 INFO:tasks.rados.rados.0.smithi195.stdout:1334: finishing write tid 3 to smithi19530714-16 2024-09-13T10:11:36.368 INFO:tasks.rados.rados.0.smithi195.stdout:1334: finishing write tid 4 to smithi19530714-16 2024-09-13T10:11:36.368 INFO:tasks.rados.rados.0.smithi195.stdout:1334: oid 16 updating version 0 to 37 2024-09-13T10:11:36.368 INFO:tasks.rados.rados.0.smithi195.stdout:1334: oid 16 version 37 is already newer than 35 2024-09-13T10:11:36.368 INFO:tasks.rados.rados.0.smithi195.stdout:1334: oid 16 version 37 is already newer than 36 2024-09-13T10:11:36.368 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 37 (ObjNum 258 snap 0 seq_num 258) dirty exists 2024-09-13T10:11:36.369 INFO:tasks.rados.rados.0.smithi195.stdout:1334: left oid 16 (ObjNum 258 snap 0 seq_num 258) 2024-09-13T10:11:36.369 INFO:tasks.rados.rados.0.smithi195.stdout:1336: expect (ObjNum 139 snap 0 seq_num 139) 2024-09-13T10:11:36.371 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:36.371 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:36.380 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:36.376+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:36.383 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T10:11:36.409 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:36.481 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:36.479+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:36.510 INFO:tasks.rados.rados.0.smithi195.stdout:1337: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:11:36.525 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:36.526 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:36.539 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T10:11:36.541 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2837928 2024-09-13T10:11:36.582 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:36.579+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:36.588 INFO:tasks.rados.rados.0.smithi195.stdout:1339: got (22) Invalid argument 2024-09-13T10:11:36.588 INFO:tasks.rados.rados.0.smithi195.stdout:1334: done (5 left) 2024-09-13T10:11:36.588 INFO:tasks.rados.rados.0.smithi195.stdout:1335: done (4 left) 2024-09-13T10:11:36.588 INFO:tasks.rados.rados.0.smithi195.stdout:1336: done (3 left) 2024-09-13T10:11:36.588 INFO:tasks.rados.rados.0.smithi195.stdout:1337: done (2 left) 2024-09-13T10:11:36.588 INFO:tasks.rados.rados.0.smithi195.stdout:1338: done (1 left) 2024-09-13T10:11:36.588 INFO:tasks.rados.rados.0.smithi195.stdout:1339: done (0 left) 2024-09-13T10:11:36.588 INFO:tasks.rados.rados.0.smithi195.stdout:1340: tier_evict oid 24 2024-09-13T10:11:36.589 INFO:tasks.rados.rados.0.smithi195.stdout:1340: tier_evict oid 24 snap -1 2024-09-13T10:11:36.589 INFO:tasks.rados.rados.0.smithi195.stdout:1341: tier_flush oid 13 2024-09-13T10:11:36.589 INFO:tasks.rados.rados.0.smithi195.stdout:1341: tier_flush oid 13 snap -1 2024-09-13T10:11:36.589 INFO:tasks.rados.rados.0.smithi195.stdout:1342: write oid 9 current snap is 0 2024-09-13T10:11:36.589 INFO:tasks.rados.rados.0.smithi195.stdout:1342: seq_num 259 ranges {753013=645493,2056248=296765} 2024-09-13T10:11:36.598 INFO:tasks.rados.rados.0.smithi195.stdout:1342: writing smithi19530714-9 from 753013 to 1398506 tid 1 2024-09-13T10:11:36.604 INFO:tasks.rados.rados.0.smithi195.stdout:1342: writing smithi19530714-9 from 2056248 to 2353013 tid 2 2024-09-13T10:11:36.605 INFO:tasks.rados.rados.0.smithi195.stdout:1343: set_chunk oid 20 target oid 2024-09-13T10:11:36.627 INFO:tasks.rados.rados.0.smithi195.stdout:1343: set_chunk oid 20 offset: 2245120 length: 1408000 target oid e0196c2489f7e8f0eed3c059568550cf2c0d4a8d186405443d9bd115288dd7cd offset: 0 snap -1 2024-09-13T10:11:36.627 INFO:tasks.rados.rados.0.smithi195.stdout:1344: tier_flush oid 2 2024-09-13T10:11:36.627 INFO:tasks.rados.rados.0.smithi195.stdout:1344: tier_flush oid 2 snap -1 2024-09-13T10:11:36.628 INFO:tasks.rados.rados.0.smithi195.stdout:1340: got (22) Invalid argument 2024-09-13T10:11:36.628 INFO:tasks.rados.rados.0.smithi195.stdout:1340: done (4 left) 2024-09-13T10:11:36.628 INFO:tasks.rados.rados.0.smithi195.stdout:1345: write oid 47 current snap is 0 2024-09-13T10:11:36.628 INFO:tasks.rados.rados.0.smithi195.stdout:1345: seq_num 260 ranges {769820=720742,2114730=255090} 2024-09-13T10:11:36.640 INFO:tasks.rados.rados.0.smithi195.stdout:1345: writing smithi19530714-47 from 769820 to 1490562 tid 1 2024-09-13T10:11:36.652 INFO:tasks.rados.rados.0.smithi195.stdout:1345: writing smithi19530714-47 from 2114730 to 2369820 tid 2 2024-09-13T10:11:36.652 INFO:tasks.rados.rados.0.smithi195.stdout:1346: read oid 21 snap -1 2024-09-13T10:11:36.652 INFO:tasks.rados.rados.0.smithi195.stdout:1346: expect (ObjNum 235 snap 0 seq_num 235) 2024-09-13T10:11:36.652 INFO:tasks.rados.rados.0.smithi195.stdout:1347: read oid 42 snap -1 2024-09-13T10:11:36.652 INFO:tasks.rados.rados.0.smithi195.stdout:1347: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-13T10:11:36.652 INFO:tasks.rados.rados.0.smithi195.stdout:1348: set_chunk oid 23 target oid 2024-09-13T10:11:36.661 INFO:tasks.rados.rados.0.smithi195.stdout:1348: set_chunk oid 23 offset: 1538048 length: 354816 target oid a96dfc68d386dc3231712e7da9ac76df20e215d6a20065dd316139636f8e8d43 offset: 0 snap -1 2024-09-13T10:11:36.662 INFO:tasks.rados.rados.0.smithi195.stdout:1349: read oid 44 snap -1 2024-09-13T10:11:36.662 INFO:tasks.rados.rados.0.smithi195.stdout:1349: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:11:36.662 INFO:tasks.rados.rados.0.smithi195.stdout:1350: write (excl) oid 15 current snap is 0 2024-09-13T10:11:36.662 INFO:tasks.rados.rados.0.smithi195.stdout:1342: finishing write tid 1 to smithi19530714-9 2024-09-13T10:11:36.662 INFO:tasks.rados.rados.0.smithi195.stdout:1342: finishing write tid 2 to smithi19530714-9 2024-09-13T10:11:36.662 INFO:tasks.rados.rados.0.smithi195.stdout:1342: finishing write tid 3 to smithi19530714-9 2024-09-13T10:11:36.663 INFO:tasks.rados.rados.0.smithi195.stdout:1342: finishing write tid 4 to smithi19530714-9 2024-09-13T10:11:36.663 INFO:tasks.rados.rados.0.smithi195.stdout:1342: oid 9 updating version 0 to 122 2024-09-13T10:11:36.663 INFO:tasks.rados.rados.0.smithi195.stdout:1342: oid 9 updating version 122 to 123 2024-09-13T10:11:36.663 INFO:tasks.rados.rados.0.smithi195.stdout:1342: oid 9 version 123 is already newer than 121 2024-09-13T10:11:36.663 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 123 (ObjNum 259 snap 0 seq_num 259) dirty exists 2024-09-13T10:11:36.663 INFO:tasks.rados.rados.0.smithi195.stdout:1342: left oid 9 (ObjNum 259 snap 0 seq_num 259) 2024-09-13T10:11:36.663 INFO:tasks.rados.rados.0.smithi195.stdout:1343: finishing set_chunk to oid 20 2024-09-13T10:11:36.663 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 20 set_chunk returned error code -95 offset: 2245120 length: 1408000 2024-09-13T10:11:36.663 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 0 (ObjNum 232 snap 0 seq_num 232) dirty exists 2024-09-13T10:11:36.663 INFO:tasks.rados.rados.0.smithi195.stdout:1345: finishing write tid 1 to smithi19530714-47 2024-09-13T10:11:36.663 INFO:tasks.rados.rados.0.smithi195.stdout:1341: got (0) Success 2024-09-13T10:11:36.663 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 0 (ObjNum 228 snap 0 seq_num 228) dirty exists 2024-09-13T10:11:36.664 INFO:tasks.rados.rados.0.smithi195.stdout:1345: finishing write tid 2 to smithi19530714-47 2024-09-13T10:11:36.664 INFO:tasks.rados.rados.0.smithi195.stdout:1345: finishing write tid 3 to smithi19530714-47 2024-09-13T10:11:36.664 INFO:tasks.rados.rados.0.smithi195.stdout:1345: finishing write tid 4 to smithi19530714-47 2024-09-13T10:11:36.664 INFO:tasks.rados.rados.0.smithi195.stdout:1345: oid 47 updating version 0 to 53 2024-09-13T10:11:36.664 INFO:tasks.rados.rados.0.smithi195.stdout:1345: oid 47 version 53 is already newer than 52 2024-09-13T10:11:36.664 INFO:tasks.rados.rados.0.smithi195.stdout:1345: oid 47 updating version 53 to 54 2024-09-13T10:11:36.664 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 54 (ObjNum 260 snap 0 seq_num 260) dirty exists 2024-09-13T10:11:36.664 INFO:tasks.rados.rados.0.smithi195.stdout:1345: left oid 47 (ObjNum 260 snap 0 seq_num 260) 2024-09-13T10:11:36.664 INFO:tasks.rados.rados.0.smithi195.stdout:1350: seq_num 261 ranges {786627=795991,2173212=213415} 2024-09-13T10:11:36.674 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:36.674 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:36.675 INFO:tasks.rados.rados.0.smithi195.stdout:1350: writing smithi19530714-15 from 786627 to 1582618 tid 1 2024-09-13T10:11:36.682 INFO:tasks.rados.rados.0.smithi195.stdout:1350: writing smithi19530714-15 from 2173212 to 2386627 tid 2 2024-09-13T10:11:36.683 INFO:tasks.rados.rados.0.smithi195.stdout:1341: done (9 left) 2024-09-13T10:11:36.683 INFO:tasks.rados.rados.0.smithi195.stdout:1342: done (8 left) 2024-09-13T10:11:36.683 INFO:tasks.rados.rados.0.smithi195.stdout:1343: done (7 left) 2024-09-13T10:11:36.683 INFO:tasks.rados.rados.0.smithi195.stdout:1345: done (6 left) 2024-09-13T10:11:36.683 INFO:tasks.rados.rados.0.smithi195.stdout:1351: write (excl) oid 24 current snap is 0 2024-09-13T10:11:36.683 INFO:tasks.rados.rados.0.smithi195.stdout:1351: seq_num 262 ranges {403434=471240,1315341=400756} 2024-09-13T10:11:36.683 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:36.679+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:36.688 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T10:11:36.689 INFO:tasks.rados.rados.0.smithi195.stdout:1351: writing smithi19530714-24 from 403434 to 874674 tid 1 2024-09-13T10:11:36.696 INFO:tasks.rados.rados.0.smithi195.stdout:1351: writing smithi19530714-24 from 1315341 to 1716097 tid 2 2024-09-13T10:11:36.696 INFO:tasks.rados.rados.0.smithi195.stdout:1352: read oid 14 snap -1 2024-09-13T10:11:36.696 INFO:tasks.rados.rados.0.smithi195.stdout:1352: expect (ObjNum 243 snap 0 seq_num 243) 2024-09-13T10:11:36.696 INFO:tasks.rados.rados.0.smithi195.stdout:1353: write oid 43 current snap is 0 2024-09-13T10:11:36.696 INFO:tasks.rados.rados.0.smithi195.stdout:1353: seq_num 263 ranges {420241=546489,1373823=544414} 2024-09-13T10:11:36.703 INFO:tasks.rados.rados.0.smithi195.stdout:1353: writing smithi19530714-43 from 420241 to 966730 tid 1 2024-09-13T10:11:36.709 INFO:tasks.rados.rados.0.smithi195.stdout:1353: writing smithi19530714-43 from 1373823 to 1918237 tid 2 2024-09-13T10:11:36.709 INFO:tasks.rados.rados.0.smithi195.stdout:1354: read oid 22 snap -1 2024-09-13T10:11:36.709 INFO:tasks.rados.rados.0.smithi195.stdout:1354: expect (ObjNum 139 snap 0 seq_num 139) 2024-09-13T10:11:36.709 INFO:tasks.rados.rados.0.smithi195.stdout:1355: write oid 48 current snap is 0 2024-09-13T10:11:36.710 INFO:tasks.rados.rados.0.smithi195.stdout:1347: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-13T10:11:36.741 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:36.783 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:36.778+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:36.794 INFO:tasks.rados.rados.0.smithi195.stdout:1348: finishing set_chunk to oid 23 2024-09-13T10:11:36.794 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 38 (ObjNum 254 snap 0 seq_num 254) dirty exists 2024-09-13T10:11:36.794 INFO:tasks.rados.rados.0.smithi195.stdout:1346: expect (ObjNum 235 snap 0 seq_num 235) 2024-09-13T10:11:36.804 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:36.804 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:36.817 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-13T10:11:36.901 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3946987 2024-09-13T10:11:36.901 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:36.898+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:36.958 INFO:tasks.rados.rados.0.smithi195.stdout:1349: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:11:36.985 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:36.980+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:36.989 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:36.989 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:36.995 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:11:37.004 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-13T10:11:37.030 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-13T10:11:37.032 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:11:37.032 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:11:37.032 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:11:37.032 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T10:11:37.033 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 0.000000 2024-09-13T10:11:37.033 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 0 2024-09-13T10:11:37.035 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2049510 2024-09-13T10:11:37.048 INFO:tasks.rados.rados.0.smithi195.stdout:1344: got (0) Success 2024-09-13T10:11:37.049 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 0 (ObjNum 249 snap 0 seq_num 249) dirty exists 2024-09-13T10:11:37.049 INFO:tasks.rados.rados.0.smithi195.stdout:1350: finishing write tid 1 to smithi19530714-15 2024-09-13T10:11:37.049 INFO:tasks.rados.rados.0.smithi195.stdout:1350: finishing write tid 2 to smithi19530714-15 2024-09-13T10:11:37.049 INFO:tasks.rados.rados.0.smithi195.stdout:1350: finishing write tid 3 to smithi19530714-15 2024-09-13T10:11:37.049 INFO:tasks.rados.rados.0.smithi195.stdout:1355: seq_num 264 ranges {437048=621738,1832305=404425} 2024-09-13T10:11:37.056 INFO:tasks.rados.rados.0.smithi195.stdout:1355: writing smithi19530714-48 from 437048 to 1058786 tid 1 2024-09-13T10:11:37.065 INFO:tasks.rados.rados.0.smithi195.stdout:1355: writing smithi19530714-48 from 1832305 to 2236730 tid 2 2024-09-13T10:11:37.066 INFO:tasks.rados.rados.0.smithi195.stdout:1344: done (10 left) 2024-09-13T10:11:37.066 INFO:tasks.rados.rados.0.smithi195.stdout:1346: done (9 left) 2024-09-13T10:11:37.066 INFO:tasks.rados.rados.0.smithi195.stdout:1347: done (8 left) 2024-09-13T10:11:37.066 INFO:tasks.rados.rados.0.smithi195.stdout:1348: done (7 left) 2024-09-13T10:11:37.066 INFO:tasks.rados.rados.0.smithi195.stdout:1349: done (6 left) 2024-09-13T10:11:37.066 INFO:tasks.rados.rados.0.smithi195.stdout:1356: read oid 4 snap -1 2024-09-13T10:11:37.066 INFO:tasks.rados.rados.0.smithi195.stdout:1356: expect deleted 2024-09-13T10:11:37.066 INFO:tasks.rados.rados.0.smithi195.stdout:1357: read oid 16 snap -1 2024-09-13T10:11:37.066 INFO:tasks.rados.rados.0.smithi195.stdout:1357: expect (ObjNum 258 snap 0 seq_num 258) 2024-09-13T10:11:37.066 INFO:tasks.rados.rados.0.smithi195.stdout:1358: read oid 38 snap -1 2024-09-13T10:11:37.066 INFO:tasks.rados.rados.0.smithi195.stdout:1358: expect (ObjNum 193 snap 0 seq_num 193) 2024-09-13T10:11:37.066 INFO:tasks.rados.rados.0.smithi195.stdout:1359: read oid 44 snap -1 2024-09-13T10:11:37.066 INFO:tasks.rados.rados.0.smithi195.stdout:1359: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:11:37.066 INFO:tasks.rados.rados.0.smithi195.stdout:1360: write (excl) oid 35 current snap is 0 2024-09-13T10:11:37.067 INFO:tasks.rados.rados.0.smithi195.stdout:1350: finishing write tid 4 to smithi19530714-15 2024-09-13T10:11:37.067 INFO:tasks.rados.rados.0.smithi195.stdout:1350: oid 15 updating version 0 to 83 2024-09-13T10:11:37.067 INFO:tasks.rados.rados.0.smithi195.stdout:1350: oid 15 updating version 83 to 84 2024-09-13T10:11:37.067 INFO:tasks.rados.rados.0.smithi195.stdout:1350: oid 15 updating version 84 to 85 2024-09-13T10:11:37.067 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 85 (ObjNum 261 snap 0 seq_num 261) dirty exists 2024-09-13T10:11:37.067 INFO:tasks.rados.rados.0.smithi195.stdout:1350: left oid 15 (ObjNum 261 snap 0 seq_num 261) 2024-09-13T10:11:37.067 INFO:tasks.rados.rados.0.smithi195.stdout:1351: finishing write tid 1 to smithi19530714-24 2024-09-13T10:11:37.067 INFO:tasks.rados.rados.0.smithi195.stdout:1351: finishing write tid 2 to smithi19530714-24 2024-09-13T10:11:37.067 INFO:tasks.rados.rados.0.smithi195.stdout:1351: finishing write tid 3 to smithi19530714-24 2024-09-13T10:11:37.067 INFO:tasks.rados.rados.0.smithi195.stdout:1351: finishing write tid 4 to smithi19530714-24 2024-09-13T10:11:37.067 INFO:tasks.rados.rados.0.smithi195.stdout:1351: oid 24 updating version 0 to 127 2024-09-13T10:11:37.067 INFO:tasks.rados.rados.0.smithi195.stdout:1351: oid 24 version 127 is already newer than 126 2024-09-13T10:11:37.067 INFO:tasks.rados.rados.0.smithi195.stdout:1351: oid 24 version 127 is already newer than 125 2024-09-13T10:11:37.068 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 127 (ObjNum 262 snap 0 seq_num 262) dirty exists 2024-09-13T10:11:37.068 INFO:tasks.rados.rados.0.smithi195.stdout:1351: left oid 24 (ObjNum 262 snap 0 seq_num 262) 2024-09-13T10:11:37.068 INFO:tasks.rados.rados.0.smithi195.stdout:1352: expect (ObjNum 243 snap 0 seq_num 243) 2024-09-13T10:11:37.081 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1368016 2024-09-13T10:11:37.099 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1083182 2024-09-13T10:11:37.106 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:37.104+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:37.132 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1180283 2024-09-13T10:11:37.152 INFO:tasks.rados.rados.0.smithi195.stdout:1353: finishing write tid 1 to smithi19530714-43 2024-09-13T10:11:37.152 INFO:tasks.rados.rados.0.smithi195.stdout:1353: finishing write tid 2 to smithi19530714-43 2024-09-13T10:11:37.153 INFO:tasks.rados.rados.0.smithi195.stdout:1353: finishing write tid 3 to smithi19530714-43 2024-09-13T10:11:37.153 INFO:tasks.rados.rados.0.smithi195.stdout:1353: finishing write tid 4 to smithi19530714-43 2024-09-13T10:11:37.153 INFO:tasks.rados.rados.0.smithi195.stdout:1353: oid 43 updating version 0 to 77 2024-09-13T10:11:37.153 INFO:tasks.rados.rados.0.smithi195.stdout:1353: oid 43 version 77 is already newer than 76 2024-09-13T10:11:37.153 INFO:tasks.rados.rados.0.smithi195.stdout:1353: oid 43 updating version 77 to 78 2024-09-13T10:11:37.153 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 78 (ObjNum 263 snap 0 seq_num 263) dirty exists 2024-09-13T10:11:37.153 INFO:tasks.rados.rados.0.smithi195.stdout:1353: left oid 43 (ObjNum 263 snap 0 seq_num 263) 2024-09-13T10:11:37.153 INFO:tasks.rados.rados.0.smithi195.stdout:1354: expect (ObjNum 139 snap 0 seq_num 139) 2024-09-13T10:11:37.153 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:37.154 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:37.154 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:37.162 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:37.162 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:37.174 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:37.174 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:37.175 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-13T10:11:37.195 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:37.195 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:37.215 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:37.216 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:37.220 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:37.218+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:37.232 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:37.232 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:37.252 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:37.253 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:37.269 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1408722 2024-09-13T10:11:37.273 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:37.273 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:37.286 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:37.283+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:37.291 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:37.291 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:37.293 INFO:tasks.rados.rados.0.smithi195.stdout:1355: finishing write tid 1 to smithi19530714-48 2024-09-13T10:11:37.293 INFO:tasks.rados.rados.0.smithi195.stdout:1360: seq_num 265 ranges {453855=696987,1890787=548083} 2024-09-13T10:11:37.300 INFO:tasks.rados.rados.0.smithi195.stdout:1360: writing smithi19530714-35 from 453855 to 1150842 tid 1 2024-09-13T10:11:37.308 INFO:tasks.rados.rados.0.smithi195.stdout:1360: writing smithi19530714-35 from 1890787 to 2438870 tid 2 2024-09-13T10:11:37.309 INFO:tasks.rados.rados.0.smithi195.stdout:1350: done (10 left) 2024-09-13T10:11:37.309 INFO:tasks.rados.rados.0.smithi195.stdout:1351: done (9 left) 2024-09-13T10:11:37.309 INFO:tasks.rados.rados.0.smithi195.stdout:1352: done (8 left) 2024-09-13T10:11:37.309 INFO:tasks.rados.rados.0.smithi195.stdout:1353: done (7 left) 2024-09-13T10:11:37.309 INFO:tasks.rados.rados.0.smithi195.stdout:1354: done (6 left) 2024-09-13T10:11:37.309 INFO:tasks.rados.rados.0.smithi195.stdout:1361: set_chunk oid 39 target oid 2024-09-13T10:11:37.309 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:37.310 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:37.318 INFO:tasks.rados.rados.0.smithi195.stdout:1361: set_chunk oid 39 offset: 1205248 length: 112128 target oid 076cc38ba7b79f2600ab98e5f57f7aa689cc50ce0fe9df6003b39ca070255c5c offset: 0 snap -1 2024-09-13T10:11:37.318 INFO:tasks.rados.rados.0.smithi195.stdout:1362: read oid 49 snap -1 2024-09-13T10:11:37.318 INFO:tasks.rados.rados.0.smithi195.stdout:1362: expect (ObjNum 181 snap 0 seq_num 181) 2024-09-13T10:11:37.318 INFO:tasks.rados.rados.0.smithi195.stdout:1363: read oid 14 snap -1 2024-09-13T10:11:37.318 INFO:tasks.rados.rados.0.smithi195.stdout:1363: expect (ObjNum 243 snap 0 seq_num 243) 2024-09-13T10:11:37.318 INFO:tasks.rados.rados.0.smithi195.stdout:1364: set_chunk oid 8 target oid 2024-09-13T10:11:37.318 INFO:tasks.rados.rados.0.smithi195.stdout:1364: set_chunk oid 8 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:37.318 INFO:tasks.rados.rados.0.smithi195.stdout:1365: read oid 17 snap -1 2024-09-13T10:11:37.318 INFO:tasks.rados.rados.0.smithi195.stdout:1365: expect deleted 2024-09-13T10:11:37.319 INFO:tasks.rados.rados.0.smithi195.stdout:1366: read oid 13 snap -1 2024-09-13T10:11:37.319 INFO:tasks.rados.rados.0.smithi195.stdout:1366: expect (ObjNum 228 snap 0 seq_num 228) 2024-09-13T10:11:37.319 INFO:tasks.rados.rados.0.smithi195.stdout:1367: set_chunk oid 36 target oid 2024-09-13T10:11:37.336 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:37.336 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:37.349 INFO:tasks.rados.rados.0.smithi195.stdout:1367: set_chunk oid 36 offset: 313856 length: 1610752 target oid 0e1630eac2db719bece9f724a38472c04c221dcc7fd6c97ddf0e27f4300cdbd7 offset: 0 snap -1 2024-09-13T10:11:37.349 INFO:tasks.rados.rados.0.smithi195.stdout:1368: write (excl) oid 18 current snap is 0 2024-09-13T10:11:37.349 INFO:tasks.rados.rados.0.smithi195.stdout:1368: seq_num 266 ranges {470662=772236,1832916=408094} 2024-09-13T10:11:37.356 INFO:tasks.rados.rados.0.smithi195.stdout:1368: writing smithi19530714-18 from 470662 to 1242898 tid 1 2024-09-13T10:11:37.359 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:37.360 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:37.362 INFO:tasks.rados.rados.0.smithi195.stdout:1368: writing smithi19530714-18 from 1832916 to 2241010 tid 2 2024-09-13T10:11:37.362 INFO:tasks.rados.rados.0.smithi195.stdout:1356: done (13 left) 2024-09-13T10:11:37.362 INFO:tasks.rados.rados.0.smithi195.stdout:1369: read oid 12 snap -1 2024-09-13T10:11:37.362 INFO:tasks.rados.rados.0.smithi195.stdout:1369: expect deleted 2024-09-13T10:11:37.363 INFO:tasks.rados.rados.0.smithi195.stdout:1370: read oid 21 snap -1 2024-09-13T10:11:37.363 INFO:tasks.rados.rados.0.smithi195.stdout:1370: expect (ObjNum 235 snap 0 seq_num 235) 2024-09-13T10:11:37.363 INFO:tasks.rados.rados.0.smithi195.stdout:1371: write (excl) oid 11 current snap is 0 2024-09-13T10:11:37.363 INFO:tasks.rados.rados.0.smithi195.stdout:1371: seq_num 267 ranges {487469=563838,1607751=551752} 2024-09-13T10:11:37.363 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:37.363 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:37.368 INFO:tasks.rados.rados.0.smithi195.stdout:1371: writing smithi19530714-11 from 487469 to 1051307 tid 1 2024-09-13T10:11:37.375 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-13T10:11:37.376 INFO:tasks.rados.rados.0.smithi195.stdout:1371: writing smithi19530714-11 from 1607751 to 2159503 tid 2 2024-09-13T10:11:37.376 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:37.376 INFO:tasks.rados.rados.0.smithi195.stdout:1359: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:37.383 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:37.387 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:37.383+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:37.421 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:37.443 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:37.463 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2197595 2024-09-13T10:11:37.463 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:37.463 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:37.476 INFO:tasks.rados.rados.0.smithi195.stdout:1355: finishing write tid 2 to smithi19530714-48 2024-09-13T10:11:37.476 INFO:tasks.rados.rados.0.smithi195.stdout:1357: expect (ObjNum 258 snap 0 seq_num 258) 2024-09-13T10:11:37.476 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:37.488 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:37.487+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:37.494 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:37.494 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:37.494 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:11:37.494 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:37.508 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-13T10:11:37.568 INFO:tasks.rados.rados.0.smithi195.stdout:1358: expect (ObjNum 193 snap 0 seq_num 193) 2024-09-13T10:11:37.568 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:37.589 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:37.586+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:37.623 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:11:37.623 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:11:37.623 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:37.636 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-13T10:11:37.690 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:37.684+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:37.696 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:37.699 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.7 primary-affinity to 0 (802) 2024-09-13T10:11:37.757 INFO:tasks.rados.rados.0.smithi195.stdout:1355: finishing write tid 3 to smithi19530714-48 2024-09-13T10:11:37.758 INFO:tasks.rados.rados.0.smithi195.stdout:1355: finishing write tid 4 to smithi19530714-48 2024-09-13T10:11:37.758 INFO:tasks.rados.rados.0.smithi195.stdout:1355: oid 48 updating version 0 to 87 2024-09-13T10:11:37.758 INFO:tasks.rados.rados.0.smithi195.stdout:1355: oid 48 updating version 87 to 88 2024-09-13T10:11:37.758 INFO:tasks.rados.rados.0.smithi195.stdout:1355: oid 48 version 88 is already newer than 86 2024-09-13T10:11:37.758 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 88 (ObjNum 264 snap 0 seq_num 264) dirty exists 2024-09-13T10:11:37.758 INFO:tasks.rados.rados.0.smithi195.stdout:1355: left oid 48 (ObjNum 264 snap 0 seq_num 264) 2024-09-13T10:11:37.758 INFO:tasks.rados.rados.0.smithi195.stdout:1360: finishing write tid 1 to smithi19530714-35 2024-09-13T10:11:37.758 INFO:tasks.rados.rados.0.smithi195.stdout:1355: done (15 left) 2024-09-13T10:11:37.758 INFO:tasks.rados.rados.0.smithi195.stdout:1357: done (14 left) 2024-09-13T10:11:37.758 INFO:tasks.rados.rados.0.smithi195.stdout:1358: done (13 left) 2024-09-13T10:11:37.758 INFO:tasks.rados.rados.0.smithi195.stdout:1359: done (12 left) 2024-09-13T10:11:37.759 INFO:tasks.rados.rados.0.smithi195.stdout:1372: write oid 45 current snap is 0 2024-09-13T10:11:37.759 INFO:tasks.rados.rados.0.smithi195.stdout:1360: finishing write tid 2 to smithi19530714-35 2024-09-13T10:11:37.759 INFO:tasks.rados.rados.0.smithi195.stdout:1360: finishing write tid 3 to smithi19530714-35 2024-09-13T10:11:37.759 INFO:tasks.rados.rados.0.smithi195.stdout:1360: finishing write tid 4 to smithi19530714-35 2024-09-13T10:11:37.759 INFO:tasks.rados.rados.0.smithi195.stdout:1360: oid 35 updating version 0 to 72 2024-09-13T10:11:37.759 INFO:tasks.rados.rados.0.smithi195.stdout:1360: oid 35 updating version 72 to 73 2024-09-13T10:11:37.759 INFO:tasks.rados.rados.0.smithi195.stdout:1360: oid 35 version 73 is already newer than 71 2024-09-13T10:11:37.759 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 73 (ObjNum 265 snap 0 seq_num 265) dirty exists 2024-09-13T10:11:37.759 INFO:tasks.rados.rados.0.smithi195.stdout:1360: left oid 35 (ObjNum 265 snap 0 seq_num 265) 2024-09-13T10:11:37.759 INFO:tasks.rados.rados.0.smithi195.stdout:1364: finishing set_chunk to oid 8 2024-09-13T10:11:37.759 INFO:tasks.rados.rados.0.smithi195.stdout:1364: got expected ENOENT (src dne) 2024-09-13T10:11:37.759 INFO:tasks.rados.rados.0.smithi195.stdout:1361: finishing set_chunk to oid 39 2024-09-13T10:11:37.760 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 27 (ObjNum 241 snap 0 seq_num 241) dirty exists 2024-09-13T10:11:37.760 INFO:tasks.rados.rados.0.smithi195.stdout:1362: expect (ObjNum 181 snap 0 seq_num 181) 2024-09-13T10:11:37.791 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:37.787+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:37.802 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:37.802 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:11:37.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:11:37.802 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:11:37.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.802 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:161 4.d e38/31 4:b19b932a:::smithi19530714-41:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:37.802 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:11:37.802 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.943898533000002, 2024-09-13T10:11:37.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056705289999999997, 2024-09-13T10:11:37.802 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.802 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.803 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:37.803 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:11:37.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:11:37.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:11:37.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0999999999999998e-07 2024-09-13T10:11:37.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856051+0000", 2024-09-13T10:11:37.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020382130000000001 2024-09-13T10:11:37.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856079+0000", 2024-09-13T10:11:37.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8345999999999999e-05 2024-09-13T10:11:37.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856093+0000", 2024-09-13T10:11:37.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4353e-05 2024-09-13T10:11:37.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856232+0000", 2024-09-13T10:11:37.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013922199999999999 2024-09-13T10:11:37.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856267+0000", 2024-09-13T10:11:37.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4823000000000002e-05 2024-09-13T10:11:37.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:37.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.859505+0000", 2024-09-13T10:11:37.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032373269999999999 2024-09-13T10:11:37.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.859682+0000", 2024-09-13T10:11:37.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017753499999999999 2024-09-13T10:11:37.806 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.806 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.807 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.807 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:162 4.d e38/31 4:b19b932a:::smithi19530714-41:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:37.807 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.861006+0000", 2024-09-13T10:11:37.807 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.936904257000002, 2024-09-13T10:11:37.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003007591, 2024-09-13T10:11:37.807 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.807 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.807 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:37.807 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861006+0000", 2024-09-13T10:11:37.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861006+0000", 2024-09-13T10:11:37.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861008+0000", 2024-09-13T10:11:37.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9860000000000001e-06 2024-09-13T10:11:37.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861797+0000", 2024-09-13T10:11:37.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00078932099999999995 2024-09-13T10:11:37.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861821+0000", 2024-09-13T10:11:37.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4165e-05 2024-09-13T10:11:37.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.809 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.809 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861841+0000", 2024-09-13T10:11:37.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9383000000000001e-05 2024-09-13T10:11:37.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861960+0000", 2024-09-13T10:11:37.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000118997 2024-09-13T10:11:37.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.861996+0000", 2024-09-13T10:11:37.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6099000000000001e-05 2024-09-13T10:11:37.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:37.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.863918+0000", 2024-09-13T10:11:37.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019220489999999999 2024-09-13T10:11:37.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.864013+0000", 2024-09-13T10:11:37.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5591e-05 2024-09-13T10:11:37.811 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.811 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.811 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.811 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:181 4.d e38/31 4:ba7d2bc5:::smithi19530714-46:head v 38'4, mlcod=38'2)", 2024-09-13T10:11:37.811 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.926727+0000", 2024-09-13T10:11:37.812 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.871183524000003, 2024-09-13T10:11:37.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0042825909999999997, 2024-09-13T10:11:37.812 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.812 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.812 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:37.812 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.926727+0000", 2024-09-13T10:11:37.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.926727+0000", 2024-09-13T10:11:37.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.926729+0000", 2024-09-13T10:11:37.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.244e-06 2024-09-13T10:11:37.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.927979+0000", 2024-09-13T10:11:37.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0012504230000000001 2024-09-13T10:11:37.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.928010+0000", 2024-09-13T10:11:37.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1115000000000003e-05 2024-09-13T10:11:37.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.928030+0000", 2024-09-13T10:11:37.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0063000000000001e-05 2024-09-13T10:11:37.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.928185+0000", 2024-09-13T10:11:37.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015451899999999999 2024-09-13T10:11:37.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.928248+0000", 2024-09-13T10:11:37.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3391000000000003e-05 2024-09-13T10:11:37.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:37.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.930963+0000", 2024-09-13T10:11:37.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027147170000000002 2024-09-13T10:11:37.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.931009+0000", 2024-09-13T10:11:37.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6118999999999997e-05 2024-09-13T10:11:37.816 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.816 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.816 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.816 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:182 4.d e38/31 4:ba7d2bc5:::smithi19530714-46:head v 38'5, mlcod=38'3)", 2024-09-13T10:11:37.816 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.933744+0000", 2024-09-13T10:11:37.816 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.864165694999997, 2024-09-13T10:11:37.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0031909019999999998, 2024-09-13T10:11:37.816 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.816 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.817 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:37.817 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.933744+0000", 2024-09-13T10:11:37.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.933744+0000", 2024-09-13T10:11:37.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.933746+0000", 2024-09-13T10:11:37.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.903e-06 2024-09-13T10:11:37.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.934786+0000", 2024-09-13T10:11:37.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001039683 2024-09-13T10:11:37.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.934812+0000", 2024-09-13T10:11:37.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5902e-05 2024-09-13T10:11:37.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.934826+0000", 2024-09-13T10:11:37.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3696000000000001e-05 2024-09-13T10:11:37.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.934896+0000", 2024-09-13T10:11:37.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0117999999999995e-05 2024-09-13T10:11:37.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.934915+0000", 2024-09-13T10:11:37.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9505e-05 2024-09-13T10:11:37.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:37.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.936894+0000", 2024-09-13T10:11:37.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019791990000000001 2024-09-13T10:11:37.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.821 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.936935+0000", 2024-09-13T10:11:37.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0896000000000003e-05 2024-09-13T10:11:37.821 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.821 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.821 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.821 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:786 4.d e39/31 4:b19b932a:::smithi19530714-41:head v 39'15, mlcod=39'13)", 2024-09-13T10:11:37.821 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.620217+0000", 2024-09-13T10:11:37.821 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 46.177692720000003, 2024-09-13T10:11:37.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033766159999999998, 2024-09-13T10:11:37.821 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.821 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.822 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:37.822 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.822 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.822 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.822 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.620217+0000", 2024-09-13T10:11:37.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.620217+0000", 2024-09-13T10:11:37.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.620220+0000", 2024-09-13T10:11:37.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6230000000000001e-06 2024-09-13T10:11:37.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.621490+0000", 2024-09-13T10:11:37.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0012697019999999999 2024-09-13T10:11:37.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.621531+0000", 2024-09-13T10:11:37.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1205999999999998e-05 2024-09-13T10:11:37.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.621552+0000", 2024-09-13T10:11:37.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0948999999999999e-05 2024-09-13T10:11:37.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.621651+0000", 2024-09-13T10:11:37.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9482999999999996e-05 2024-09-13T10:11:37.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.621676+0000", 2024-09-13T10:11:37.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4570999999999999e-05 2024-09-13T10:11:37.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:37.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.623532+0000", 2024-09-13T10:11:37.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001856218 2024-09-13T10:11:37.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.623594+0000", 2024-09-13T10:11:37.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1864000000000003e-05 2024-09-13T10:11:37.827 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.827 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.827 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.828 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1337 4.d 4:b19b932a:::smithi19530714-41:head [cache-evict] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e46)", 2024-09-13T10:11:37.828 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:00.391725+0000", 2024-09-13T10:11:37.828 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 37.406184697999997, 2024-09-13T10:11:37.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0036187670000000002, 2024-09-13T10:11:37.828 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.828 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.828 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:37.828 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:37.828 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:37.828 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:37.828 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1337 2024-09-13T10:11:37.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.829 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.391725+0000", 2024-09-13T10:11:37.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.391725+0000", 2024-09-13T10:11:37.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.391728+0000", 2024-09-13T10:11:37.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7829999999999999e-06 2024-09-13T10:11:37.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.391741+0000", 2024-09-13T10:11:37.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3166000000000001e-05 2024-09-13T10:11:37.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.830 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.830 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.391772+0000", 2024-09-13T10:11:37.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1069999999999999e-05 2024-09-13T10:11:37.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.391790+0000", 2024-09-13T10:11:37.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7062e-05 2024-09-13T10:11:37.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.391910+0000", 2024-09-13T10:11:37.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000120703 2024-09-13T10:11:37.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op must wait for map", 2024-09-13T10:11:37.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.391918+0000", 2024-09-13T10:11:37.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2570000000000007e-06 2024-09-13T10:11:37.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.395125+0000", 2024-09-13T10:11:37.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032068610000000001 2024-09-13T10:11:37.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.395250+0000", 2024-09-13T10:11:37.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012494700000000001 2024-09-13T10:11:37.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:00.395344+0000", 2024-09-13T10:11:37.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3918000000000004e-05 2024-09-13T10:11:37.833 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.833 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.833 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.833 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1367 4.d 4:ba7d2bc5:::smithi19530714-46:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T10:11:37.833 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:01.023318+0000", 2024-09-13T10:11:37.833 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 36.774591979, 2024-09-13T10:11:37.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034398089999999998, 2024-09-13T10:11:37.833 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.833 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.833 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:37.834 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:37.834 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:37.834 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:37.834 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1367 2024-09-13T10:11:37.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.834 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.023318+0000", 2024-09-13T10:11:37.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.023318+0000", 2024-09-13T10:11:37.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.023321+0000", 2024-09-13T10:11:37.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9170000000000002e-06 2024-09-13T10:11:37.835 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.835 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.835 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.835 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.023334+0000", 2024-09-13T10:11:37.835 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3263e-05 2024-09-13T10:11:37.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.023365+0000", 2024-09-13T10:11:37.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0945999999999998e-05 2024-09-13T10:11:37.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.836 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.023383+0000", 2024-09-13T10:11:37.836 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7546000000000001e-05 2024-09-13T10:11:37.836 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.836 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.836 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.023464+0000", 2024-09-13T10:11:37.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1623000000000001e-05 2024-09-13T10:11:37.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.023596+0000", 2024-09-13T10:11:37.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013198100000000001 2024-09-13T10:11:37.837 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.837 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.837 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.837 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.026758+0000", 2024-09-13T10:11:37.837 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0031615329999999998 2024-09-13T10:11:37.838 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.838 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.838 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.838 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.838 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.838 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1665 4.d 4:b19b932a:::smithi19530714-41:head [write 769787~704463,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-13T10:11:37.838 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.752922+0000", 2024-09-13T10:11:37.838 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 31.044988113999999, 2024-09-13T10:11:37.838 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014152090000000001, 2024-09-13T10:11:37.838 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.838 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.838 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:37.838 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:37.839 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:37.839 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:37.839 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1665 2024-09-13T10:11:37.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.839 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.752922+0000", 2024-09-13T10:11:37.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.839 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.839 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.839 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.839 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.752922+0000", 2024-09-13T10:11:37.839 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.752927+0000", 2024-09-13T10:11:37.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7430000000000002e-06 2024-09-13T10:11:37.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.840 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.753904+0000", 2024-09-13T10:11:37.840 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00097707799999999993 2024-09-13T10:11:37.840 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.840 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.840 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.753959+0000", 2024-09-13T10:11:37.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5099000000000002e-05 2024-09-13T10:11:37.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.753987+0000", 2024-09-13T10:11:37.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7736999999999999e-05 2024-09-13T10:11:37.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.841 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.841 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.841 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.754289+0000", 2024-09-13T10:11:37.841 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000301926 2024-09-13T10:11:37.841 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.755253+0000", 2024-09-13T10:11:37.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00096452500000000004 2024-09-13T10:11:37.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:37.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.756279+0000", 2024-09-13T10:11:37.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010256919999999999 2024-09-13T10:11:37.842 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.842 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.842 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:37.842 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.765950+0000", 2024-09-13T10:11:37.842 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096707410000000001 2024-09-13T10:11:37.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:37.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766011+0000", 2024-09-13T10:11:37.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1826000000000001e-05 2024-09-13T10:11:37.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.843 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:37.843 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766431+0000", 2024-09-13T10:11:37.843 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041941400000000001 2024-09-13T10:11:37.843 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.843 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:37.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.767009+0000", 2024-09-13T10:11:37.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00057826500000000001 2024-09-13T10:11:37.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.844 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.844 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.767074+0000", 2024-09-13T10:11:37.844 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5043999999999996e-05 2024-09-13T10:11:37.844 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.844 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.844 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.844 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.844 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.845 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1666 4.d 4:b19b932a:::smithi19530714-41:head [write 2267961~101826] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-13T10:11:37.845 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.762495+0000", 2024-09-13T10:11:37.845 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 31.035415582999999, 2024-09-13T10:11:37.845 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061889800000000002, 2024-09-13T10:11:37.845 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.845 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.845 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:37.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:37.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:37.845 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:37.845 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1666 2024-09-13T10:11:37.845 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.845 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.845 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762495+0000", 2024-09-13T10:11:37.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762495+0000", 2024-09-13T10:11:37.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.846 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.846 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.846 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.846 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762542+0000", 2024-09-13T10:11:37.846 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7382999999999997e-05 2024-09-13T10:11:37.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762850+0000", 2024-09-13T10:11:37.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030803999999999998 2024-09-13T10:11:37.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.847 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762888+0000", 2024-09-13T10:11:37.847 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7645999999999998e-05 2024-09-13T10:11:37.847 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.847 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.847 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762919+0000", 2024-09-13T10:11:37.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0879e-05 2024-09-13T10:11:37.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763057+0000", 2024-09-13T10:11:37.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000138672 2024-09-13T10:11:37.848 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.848 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.848 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.848 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763221+0000", 2024-09-13T10:11:37.848 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016337200000000001 2024-09-13T10:11:37.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:37.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763457+0000", 2024-09-13T10:11:37.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023669399999999999 2024-09-13T10:11:37.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:37.849 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766139+0000", 2024-09-13T10:11:37.849 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026813819999999999 2024-09-13T10:11:37.849 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.849 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.849 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:37.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.767138+0000", 2024-09-13T10:11:37.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0009992200000000001 2024-09-13T10:11:37.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:37.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768189+0000", 2024-09-13T10:11:37.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001051614 2024-09-13T10:11:37.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.850 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.850 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:37.850 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768616+0000", 2024-09-13T10:11:37.850 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042670000000000002 2024-09-13T10:11:37.850 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.851 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.851 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768684+0000", 2024-09-13T10:11:37.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7378000000000004e-05 2024-09-13T10:11:37.851 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.851 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.851 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.851 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.851 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.851 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1667 4.d 4:b19b932a:::smithi19530714-41:head [setxattr _header (56) in=7b,truncate 2369787] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-13T10:11:37.851 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.762978+0000", 2024-09-13T10:11:37.851 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 31.034932019999999, 2024-09-13T10:11:37.851 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0077634310000000003, 2024-09-13T10:11:37.851 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.852 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.852 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:37.852 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:37.852 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:37.852 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:37.852 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1667 2024-09-13T10:11:37.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.852 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.852 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.852 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762978+0000", 2024-09-13T10:11:37.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.852 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762978+0000", 2024-09-13T10:11:37.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762980+0000", 2024-09-13T10:11:37.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.654e-06 2024-09-13T10:11:37.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.853 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.853 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.853 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762990+0000", 2024-09-13T10:11:37.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9629999999999993e-06 2024-09-13T10:11:37.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763008+0000", 2024-09-13T10:11:37.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8122e-05 2024-09-13T10:11:37.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763026+0000", 2024-09-13T10:11:37.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7747000000000001e-05 2024-09-13T10:11:37.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.764724+0000", 2024-09-13T10:11:37.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001698726 2024-09-13T10:11:37.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.764905+0000", 2024-09-13T10:11:37.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018053200000000001 2024-09-13T10:11:37.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:37.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.765167+0000", 2024-09-13T10:11:37.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026248100000000001 2024-09-13T10:11:37.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:37.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.767098+0000", 2024-09-13T10:11:37.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019303949999999999 2024-09-13T10:11:37.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:37.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768108+0000", 2024-09-13T10:11:37.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010101820000000001 2024-09-13T10:11:37.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:37.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768766+0000", 2024-09-13T10:11:37.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00065837299999999997 2024-09-13T10:11:37.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:37.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770530+0000", 2024-09-13T10:11:37.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00176384 2024-09-13T10:11:37.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770742+0000", 2024-09-13T10:11:37.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000211416 2024-09-13T10:11:37.857 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.857 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.857 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.858 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1668 4.d 4:b19b932a:::smithi19530714-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-09-13T10:11:37.858 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.763054+0000", 2024-09-13T10:11:37.858 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 31.034855921999998, 2024-09-13T10:11:37.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0080519860000000006, 2024-09-13T10:11:37.858 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.858 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.858 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:37.858 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:37.858 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:37.858 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:37.858 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1668 2024-09-13T10:11:37.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.858 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763054+0000", 2024-09-13T10:11:37.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763054+0000", 2024-09-13T10:11:37.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763056+0000", 2024-09-13T10:11:37.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.404e-06 2024-09-13T10:11:37.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763062+0000", 2024-09-13T10:11:37.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1260000000000003e-06 2024-09-13T10:11:37.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.860 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763080+0000", 2024-09-13T10:11:37.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7864e-05 2024-09-13T10:11:37.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763093+0000", 2024-09-13T10:11:37.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3702e-05 2024-09-13T10:11:37.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766050+0000", 2024-09-13T10:11:37.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029562400000000002 2024-09-13T10:11:37.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:37.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766127+0000", 2024-09-13T10:11:37.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7469e-05 2024-09-13T10:11:37.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770802+0000", 2024-09-13T10:11:37.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0046747309999999997 2024-09-13T10:11:37.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770954+0000", 2024-09-13T10:11:37.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015184000000000001 2024-09-13T10:11:37.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.771106+0000", 2024-09-13T10:11:37.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015260999999999999 2024-09-13T10:11:37.863 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.863 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.863 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.863 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop_reply(client.4607.0:1665 4.d e49/44 ondisk, result = 0)", 2024-09-13T10:11:37.863 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.763155+0000", 2024-09-13T10:11:37.863 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 31.034754797000002, 2024-09-13T10:11:37.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002865163, 2024-09-13T10:11:37.863 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.863 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.863 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:37.864 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763155+0000", 2024-09-13T10:11:37.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763155+0000", 2024-09-13T10:11:37.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763157+0000", 2024-09-13T10:11:37.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4300000000000001e-06 2024-09-13T10:11:37.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763165+0000", 2024-09-13T10:11:37.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1200000000000002e-06 2024-09-13T10:11:37.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763184+0000", 2024-09-13T10:11:37.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9253e-05 2024-09-13T10:11:37.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763198+0000", 2024-09-13T10:11:37.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3533000000000001e-05 2024-09-13T10:11:37.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.765999+0000", 2024-09-13T10:11:37.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0028012850000000001 2024-09-13T10:11:37.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766006+0000", 2024-09-13T10:11:37.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2189999999999998e-06 2024-09-13T10:11:37.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766021+0000", 2024-09-13T10:11:37.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4323e-05 2024-09-13T10:11:37.867 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.867 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.867 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.867 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1920 4.d 4:b19b932a:::smithi19530714-41:head [read 0~2369787,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:37.867 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:11.758355+0000", 2024-09-13T10:11:37.868 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 26.039554711000001, 2024-09-13T10:11:37.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0052833139999999999, 2024-09-13T10:11:37.868 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.868 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.868 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:37.868 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:37.868 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:37.868 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:37.868 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1920 2024-09-13T10:11:37.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.868 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758355+0000", 2024-09-13T10:11:37.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758355+0000", 2024-09-13T10:11:37.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758358+0000", 2024-09-13T10:11:37.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7130000000000001e-06 2024-09-13T10:11:37.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758369+0000", 2024-09-13T10:11:37.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0974e-05 2024-09-13T10:11:37.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758405+0000", 2024-09-13T10:11:37.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6272000000000001e-05 2024-09-13T10:11:37.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758422+0000", 2024-09-13T10:11:37.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6322999999999999e-05 2024-09-13T10:11:37.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758592+0000", 2024-09-13T10:11:37.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000170147 2024-09-13T10:11:37.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758876+0000", 2024-09-13T10:11:37.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028450099999999999 2024-09-13T10:11:37.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.763639+0000", 2024-09-13T10:11:37.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047623839999999997 2024-09-13T10:11:37.872 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.872 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.872 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.872 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1921 4.d 4:b19b932a:::smithi19530714-41:head [read 1405629~964158,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:37.872 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:11.758460+0000", 2024-09-13T10:11:37.872 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 26.039450565999999, 2024-09-13T10:11:37.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068984509999999999, 2024-09-13T10:11:37.872 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.872 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.873 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:37.873 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:37.873 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:37.873 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:37.873 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1921 2024-09-13T10:11:37.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.873 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.873 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.873 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758460+0000", 2024-09-13T10:11:37.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758460+0000", 2024-09-13T10:11:37.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758461+0000", 2024-09-13T10:11:37.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-13T10:11:37.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758469+0000", 2024-09-13T10:11:37.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8029999999999997e-06 2024-09-13T10:11:37.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758490+0000", 2024-09-13T10:11:37.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1365999999999998e-05 2024-09-13T10:11:37.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758503+0000", 2024-09-13T10:11:37.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3200000000000001e-05 2024-09-13T10:11:37.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.763779+0000", 2024-09-13T10:11:37.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0052761079999999998 2024-09-13T10:11:37.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.764101+0000", 2024-09-13T10:11:37.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032119700000000001 2024-09-13T10:11:37.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.765358+0000", 2024-09-13T10:11:37.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001257467 2024-09-13T10:11:37.876 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.876 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.876 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.877 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1922 4.d 4:b19b932a:::smithi19530714-41:head [sparse-read 1834913~534874] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:37.877 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:11.758578+0000", 2024-09-13T10:11:37.877 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 26.039331942, 2024-09-13T10:11:37.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007449954, 2024-09-13T10:11:37.877 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.877 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.877 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:37.877 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:37.877 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:37.877 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:37.877 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1922 2024-09-13T10:11:37.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.878 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758578+0000", 2024-09-13T10:11:37.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.878 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758578+0000", 2024-09-13T10:11:37.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.878 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758580+0000", 2024-09-13T10:11:37.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.79e-06 2024-09-13T10:11:37.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758587+0000", 2024-09-13T10:11:37.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5599999999999999e-06 2024-09-13T10:11:37.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758620+0000", 2024-09-13T10:11:37.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3649000000000003e-05 2024-09-13T10:11:37.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758628+0000", 2024-09-13T10:11:37.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2400000000000007e-06 2024-09-13T10:11:37.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.765429+0000", 2024-09-13T10:11:37.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068003600000000001 2024-09-13T10:11:37.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.765651+0000", 2024-09-13T10:11:37.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022221900000000001 2024-09-13T10:11:37.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.766028+0000", 2024-09-13T10:11:37.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00037713599999999998 2024-09-13T10:11:37.881 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.881 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.881 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.881 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2711 4.d 4:b19b932a:::smithi19530714-41:head [write 766452~565707] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:37.881 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:29.366064+0000", 2024-09-13T10:11:37.881 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.4318456449999992, 2024-09-13T10:11:37.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0046776860000000003, 2024-09-13T10:11:37.882 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.882 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.882 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:37.882 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:37.882 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:37.882 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:37.882 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2711 2024-09-13T10:11:37.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.882 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.366064+0000", 2024-09-13T10:11:37.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.366064+0000", 2024-09-13T10:11:37.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.366067+0000", 2024-09-13T10:11:37.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6120000000000001e-06 2024-09-13T10:11:37.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.366466+0000", 2024-09-13T10:11:37.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00039925200000000001 2024-09-13T10:11:37.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.366491+0000", 2024-09-13T10:11:37.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4292000000000001e-05 2024-09-13T10:11:37.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.366507+0000", 2024-09-13T10:11:37.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6807999999999999e-05 2024-09-13T10:11:37.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.366577+0000", 2024-09-13T10:11:37.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9614000000000001e-05 2024-09-13T10:11:37.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.366655+0000", 2024-09-13T10:11:37.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8333999999999999e-05 2024-09-13T10:11:37.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:37.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.366763+0000", 2024-09-13T10:11:37.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107837 2024-09-13T10:11:37.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:37.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.368651+0000", 2024-09-13T10:11:37.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0018882149999999999 2024-09-13T10:11:37.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:37.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.369441+0000", 2024-09-13T10:11:37.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00078916299999999995 2024-09-13T10:11:37.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:37.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.370675+0000", 2024-09-13T10:11:37.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001234639 2024-09-13T10:11:37.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:37.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.370708+0000", 2024-09-13T10:11:37.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2678999999999997e-05 2024-09-13T10:11:37.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.370742+0000", 2024-09-13T10:11:37.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4241e-05 2024-09-13T10:11:37.888 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.888 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.888 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.888 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2712 4.d 4:b19b932a:::smithi19530714-41:head [write 1860221~469412] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:37.888 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:29.375231+0000", 2024-09-13T10:11:37.888 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.4226786580000006, 2024-09-13T10:11:37.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.005573349, 2024-09-13T10:11:37.888 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.888 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.888 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:37.888 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:37.889 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:37.889 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:37.889 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2712 2024-09-13T10:11:37.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.889 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.889 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375231+0000", 2024-09-13T10:11:37.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.889 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375231+0000", 2024-09-13T10:11:37.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375233+0000", 2024-09-13T10:11:37.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.517e-06 2024-09-13T10:11:37.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375610+0000", 2024-09-13T10:11:37.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00037666799999999998 2024-09-13T10:11:37.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375628+0000", 2024-09-13T10:11:37.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7918e-05 2024-09-13T10:11:37.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375639+0000", 2024-09-13T10:11:37.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1345000000000001e-05 2024-09-13T10:11:37.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.891 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375690+0000", 2024-09-13T10:11:37.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0603000000000003e-05 2024-09-13T10:11:37.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375755+0000", 2024-09-13T10:11:37.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5331000000000002e-05 2024-09-13T10:11:37.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:37.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375864+0000", 2024-09-13T10:11:37.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000108695 2024-09-13T10:11:37.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:37.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.377367+0000", 2024-09-13T10:11:37.893 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015037920000000001 2024-09-13T10:11:37.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:37.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.378220+0000", 2024-09-13T10:11:37.893 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00085309600000000002 2024-09-13T10:11:37.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:37.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.380433+0000", 2024-09-13T10:11:37.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0022124760000000001 2024-09-13T10:11:37.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.894 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:37.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.380726+0000", 2024-09-13T10:11:37.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029259799999999998 2024-09-13T10:11:37.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.894 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.380805+0000", 2024-09-13T10:11:37.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9309999999999998e-05 2024-09-13T10:11:37.894 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.894 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.894 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.895 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2713 4.d 4:b19b932a:::smithi19530714-41:head [write 2981238~594195] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:37.895 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:29.386348+0000", 2024-09-13T10:11:37.895 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.4115618829999992, 2024-09-13T10:11:37.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0042511449999999996, 2024-09-13T10:11:37.895 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.895 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.895 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:37.895 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:37.895 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:37.895 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:37.895 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2713 2024-09-13T10:11:37.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.896 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386348+0000", 2024-09-13T10:11:37.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386348+0000", 2024-09-13T10:11:37.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386350+0000", 2024-09-13T10:11:37.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3599999999999999e-06 2024-09-13T10:11:37.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386677+0000", 2024-09-13T10:11:37.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032784099999999999 2024-09-13T10:11:37.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386692+0000", 2024-09-13T10:11:37.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4745e-05 2024-09-13T10:11:37.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.898 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386699+0000", 2024-09-13T10:11:37.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2450000000000001e-06 2024-09-13T10:11:37.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.898 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386764+0000", 2024-09-13T10:11:37.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4992000000000004e-05 2024-09-13T10:11:37.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386873+0000", 2024-09-13T10:11:37.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000108663 2024-09-13T10:11:37.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:37.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.387109+0000", 2024-09-13T10:11:37.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000235951 2024-09-13T10:11:37.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.899 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:37.899 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.389783+0000", 2024-09-13T10:11:37.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026742900000000002 2024-09-13T10:11:37.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:37.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.389814+0000", 2024-09-13T10:11:37.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0402000000000001e-05 2024-09-13T10:11:37.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:37.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.390459+0000", 2024-09-13T10:11:37.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064565200000000003 2024-09-13T10:11:37.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:37.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.390554+0000", 2024-09-13T10:11:37.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4594999999999998e-05 2024-09-13T10:11:37.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.390599+0000", 2024-09-13T10:11:37.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5408999999999998e-05 2024-09-13T10:11:37.901 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.901 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.901 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.901 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2714 4.d 4:b19b932a:::smithi19530714-41:head [setxattr _header (56) in=7b,truncate 3966452] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:37.901 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:29.386719+0000", 2024-09-13T10:11:37.902 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.4111911609999996, 2024-09-13T10:11:37.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0048978509999999999, 2024-09-13T10:11:37.902 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.902 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.902 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:37.902 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:37.902 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:37.902 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:37.902 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2714 2024-09-13T10:11:37.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.902 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386719+0000", 2024-09-13T10:11:37.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386719+0000", 2024-09-13T10:11:37.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386720+0000", 2024-09-13T10:11:37.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4300000000000003e-07 2024-09-13T10:11:37.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386723+0000", 2024-09-13T10:11:37.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8899999999999999e-06 2024-09-13T10:11:37.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386735+0000", 2024-09-13T10:11:37.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2052000000000001e-05 2024-09-13T10:11:37.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386740+0000", 2024-09-13T10:11:37.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9629999999999997e-06 2024-09-13T10:11:37.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.388114+0000", 2024-09-13T10:11:37.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001374073 2024-09-13T10:11:37.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.388324+0000", 2024-09-13T10:11:37.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020999500000000001 2024-09-13T10:11:37.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:37.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.388765+0000", 2024-09-13T10:11:37.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044132600000000002 2024-09-13T10:11:37.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:37.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.390618+0000", 2024-09-13T10:11:37.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001852993 2024-09-13T10:11:37.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.906 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:37.906 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.390672+0000", 2024-09-13T10:11:37.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3949999999999997e-05 2024-09-13T10:11:37.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:37.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.391515+0000", 2024-09-13T10:11:37.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00084309200000000004 2024-09-13T10:11:37.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:37.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.391563+0000", 2024-09-13T10:11:37.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8133000000000001e-05 2024-09-13T10:11:37.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.391617+0000", 2024-09-13T10:11:37.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3740999999999999e-05 2024-09-13T10:11:37.908 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.908 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.908 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.908 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2715 4.d 4:b19b932a:::smithi19530714-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:37.908 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:29.386766+0000", 2024-09-13T10:11:37.908 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 8.4111445459999992, 2024-09-13T10:11:37.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0051525249999999998, 2024-09-13T10:11:37.908 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:37.908 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:37.908 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:37.909 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:37.909 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:37.909 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:37.909 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2715 2024-09-13T10:11:37.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.909 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:37.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:37.909 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386766+0000", 2024-09-13T10:11:37.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:37.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386766+0000", 2024-09-13T10:11:37.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:37.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:37.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386766+0000", 2024-09-13T10:11:37.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5200000000000002e-07 2024-09-13T10:11:37.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:37.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386768+0000", 2024-09-13T10:11:37.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2500000000000001e-06 2024-09-13T10:11:37.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:37.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386774+0000", 2024-09-13T10:11:37.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2050000000000001e-06 2024-09-13T10:11:37.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:37.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386778+0000", 2024-09-13T10:11:37.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.318e-06 2024-09-13T10:11:37.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.389475+0000", 2024-09-13T10:11:37.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026974130000000001 2024-09-13T10:11:37.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:37.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.389685+0000", 2024-09-13T10:11:37.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020930800000000001 2024-09-13T10:11:37.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:37.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.391672+0000", 2024-09-13T10:11:37.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001987716 2024-09-13T10:11:37.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:37.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.391777+0000", 2024-09-13T10:11:37.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010505699999999999 2024-09-13T10:11:37.913 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:37.913 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:37.913 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:37.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.391918+0000", 2024-09-13T10:11:37.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014080599999999999 2024-09-13T10:11:37.913 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.913 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.913 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.913 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:37.913 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:37.913 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:37.915 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T10:11:37.916 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 896817 2024-09-13T10:11:37.916 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1016499 2024-09-13T10:11:37.916 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:37.910+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:37.933 INFO:tasks.rados.rados.0.smithi195.stdout:1366: expect (ObjNum 228 snap 0 seq_num 228) 2024-09-13T10:11:37.934 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:38.010 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 539258 2024-09-13T10:11:38.016 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:38.012+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:38.063 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T10:11:38.116 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:38.112+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:38.150 INFO:tasks.rados.rados.0.smithi195.stdout:1363: expect (ObjNum 243 snap 0 seq_num 243) 2024-09-13T10:11:38.174 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:38.192 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:11:38.192 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:11:38.192 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:38.207 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T10:11:38.207 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:38.217 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:38.212+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:38.239 INFO:tasks.rados.rados.0.smithi195.stdout:1367: finishing set_chunk to oid 36 2024-09-13T10:11:38.240 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 26 (ObjNum 190 snap 0 seq_num 190) dirty exists 2024-09-13T10:11:38.240 INFO:tasks.rados.rados.0.smithi195.stdout:1368: finishing write tid 1 to smithi19530714-18 2024-09-13T10:11:38.240 INFO:tasks.rados.rados.0.smithi195.stdout:1368: finishing write tid 2 to smithi19530714-18 2024-09-13T10:11:38.240 INFO:tasks.rados.rados.0.smithi195.stdout:1368: finishing write tid 3 to smithi19530714-18 2024-09-13T10:11:38.240 INFO:tasks.rados.rados.0.smithi195.stdout:1368: finishing write tid 4 to smithi19530714-18 2024-09-13T10:11:38.240 INFO:tasks.rados.rados.0.smithi195.stdout:1368: oid 18 updating version 0 to 91 2024-09-13T10:11:38.240 INFO:tasks.rados.rados.0.smithi195.stdout:1368: oid 18 updating version 91 to 92 2024-09-13T10:11:38.240 INFO:tasks.rados.rados.0.smithi195.stdout:1368: oid 18 updating version 92 to 93 2024-09-13T10:11:38.240 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 93 (ObjNum 266 snap 0 seq_num 266) dirty exists 2024-09-13T10:11:38.240 INFO:tasks.rados.rados.0.smithi195.stdout:1368: left oid 18 (ObjNum 266 snap 0 seq_num 266) 2024-09-13T10:11:38.240 INFO:tasks.rados.rados.0.smithi195.stdout:1371: finishing write tid 1 to smithi19530714-11 2024-09-13T10:11:38.240 INFO:tasks.rados.rados.0.smithi195.stdout:1371: finishing write tid 2 to smithi19530714-11 2024-09-13T10:11:38.241 INFO:tasks.rados.rados.0.smithi195.stdout:1371: finishing write tid 3 to smithi19530714-11 2024-09-13T10:11:38.241 INFO:tasks.rados.rados.0.smithi195.stdout:1371: finishing write tid 4 to smithi19530714-11 2024-09-13T10:11:38.241 INFO:tasks.rados.rados.0.smithi195.stdout:1371: oid 11 updating version 0 to 81 2024-09-13T10:11:38.242 INFO:tasks.rados.rados.0.smithi195.stdout:1371: oid 11 version 81 is already newer than 79 2024-09-13T10:11:38.242 INFO:tasks.rados.rados.0.smithi195.stdout:1371: oid 11 version 81 is already newer than 80 2024-09-13T10:11:38.242 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 81 (ObjNum 267 snap 0 seq_num 267) dirty exists 2024-09-13T10:11:38.242 INFO:tasks.rados.rados.0.smithi195.stdout:1371: left oid 11 (ObjNum 267 snap 0 seq_num 267) 2024-09-13T10:11:38.242 INFO:tasks.rados.rados.0.smithi195.stdout:1370: expect (ObjNum 235 snap 0 seq_num 235) 2024-09-13T10:11:38.304 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:38.318 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:38.312+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:38.333 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:38.333 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:11:38.333 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:11:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:11:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:202 3.2 e38/30 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:11:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.327659924999999, 2024-09-13T10:11:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016741669000000001, 2024-09-13T10:11:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.334 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:11:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:11:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.335 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002368+0000", 2024-09-13T10:11:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-13T10:11:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010047+0000", 2024-09-13T10:11:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007678748 2024-09-13T10:11:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010081+0000", 2024-09-13T10:11:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4011999999999999e-05 2024-09-13T10:11:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010098+0000", 2024-09-13T10:11:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7125999999999998e-05 2024-09-13T10:11:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010253+0000", 2024-09-13T10:11:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000154881 2024-09-13T10:11:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.338 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.338 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.338 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010294+0000", 2024-09-13T10:11:38.338 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1746000000000003e-05 2024-09-13T10:11:38.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.338 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.338 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.338 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.018922+0000", 2024-09-13T10:11:38.338 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0086279619999999994 2024-09-13T10:11:38.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.338 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.338 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.338 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.019109+0000", 2024-09-13T10:11:38.338 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000186057 2024-09-13T10:11:38.339 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.339 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.339 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.339 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.339 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.339 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:206 3.2 e38/30 3:4cbdbe9e:::smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:38.339 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:11:38.339 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.319275193999999, 2024-09-13T10:11:38.339 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011224615, 2024-09-13T10:11:38.339 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.339 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.340 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.340 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.340 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.340 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.340 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:11:38.340 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.340 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.340 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.340 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:11:38.340 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.341 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.341 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.341 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010754+0000", 2024-09-13T10:11:38.341 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9999999999999999e-06 2024-09-13T10:11:38.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.341 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.341 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.341 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016750+0000", 2024-09-13T10:11:38.341 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059966480000000003 2024-09-13T10:11:38.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.341 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.341 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.342 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016782+0000", 2024-09-13T10:11:38.342 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2218999999999998e-05 2024-09-13T10:11:38.342 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.342 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.342 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.342 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016799+0000", 2024-09-13T10:11:38.342 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6685999999999999e-05 2024-09-13T10:11:38.342 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.342 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.342 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.342 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016923+0000", 2024-09-13T10:11:38.342 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000124122 2024-09-13T10:11:38.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.343 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.343 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.343 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016962+0000", 2024-09-13T10:11:38.343 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8748999999999998e-05 2024-09-13T10:11:38.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.343 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.343 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.343 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021869+0000", 2024-09-13T10:11:38.343 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049073249999999997 2024-09-13T10:11:38.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.343 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.343 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.344 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021976+0000", 2024-09-13T10:11:38.344 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106866 2024-09-13T10:11:38.344 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.344 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.344 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.344 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.344 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.344 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:214 3.2 e38/30 3:68ba95ba:::smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'3, mlcod=0'0)", 2024-09-13T10:11:38.344 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:11:38.344 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.309572537999998, 2024-09-13T10:11:38.344 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011518149, 2024-09-13T10:11:38.345 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.345 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.345 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.345 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.345 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.345 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.345 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:11:38.345 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.345 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.345 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.345 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:11:38.346 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.346 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.346 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.346 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020456+0000", 2024-09-13T10:11:38.346 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0430000000000002e-06 2024-09-13T10:11:38.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.346 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.346 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.346 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026998+0000", 2024-09-13T10:11:38.346 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065420169999999998 2024-09-13T10:11:38.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.346 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.346 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.347 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027030+0000", 2024-09-13T10:11:38.347 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1976e-05 2024-09-13T10:11:38.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.347 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.347 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.347 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027048+0000", 2024-09-13T10:11:38.347 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7615e-05 2024-09-13T10:11:38.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.347 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.347 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.347 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027206+0000", 2024-09-13T10:11:38.347 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015788900000000001 2024-09-13T10:11:38.348 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.348 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.348 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.348 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027283+0000", 2024-09-13T10:11:38.348 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6942999999999998e-05 2024-09-13T10:11:38.348 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.348 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.348 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.348 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.031808+0000", 2024-09-13T10:11:38.348 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045256940000000002 2024-09-13T10:11:38.348 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.348 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.349 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.349 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.031972+0000", 2024-09-13T10:11:38.349 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016397199999999999 2024-09-13T10:11:38.349 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.349 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.349 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.349 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.349 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.349 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:210 3.2 e38/30 3:46ab5247:::smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:11:38.350 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:11:38.350 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.30170682, 2024-09-13T10:11:38.350 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015681245, 2024-09-13T10:11:38.350 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.350 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.350 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.350 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.350 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.350 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:11:38.350 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.350 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.351 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:11:38.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.351 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.351 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028322+0000", 2024-09-13T10:11:38.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0200000000000001e-06 2024-09-13T10:11:38.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.351 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.351 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037903+0000", 2024-09-13T10:11:38.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095811639999999997 2024-09-13T10:11:38.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.352 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.352 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037945+0000", 2024-09-13T10:11:38.352 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1891000000000003e-05 2024-09-13T10:11:38.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.352 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.352 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.352 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037971+0000", 2024-09-13T10:11:38.352 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5829e-05 2024-09-13T10:11:38.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.352 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.353 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.353 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038125+0000", 2024-09-13T10:11:38.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000153852 2024-09-13T10:11:38.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.353 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.353 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.353 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038193+0000", 2024-09-13T10:11:38.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7995000000000002e-05 2024-09-13T10:11:38.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.354 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.354 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.043904+0000", 2024-09-13T10:11:38.354 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0057110340000000002 2024-09-13T10:11:38.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.354 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.354 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.044001+0000", 2024-09-13T10:11:38.355 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7460000000000005e-05 2024-09-13T10:11:38.355 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.355 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.355 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.355 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.355 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:218 3.2 e38/30 3:5915c927:::smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'5, mlcod=38'2)", 2024-09-13T10:11:38.355 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:11:38.356 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.288344234, 2024-09-13T10:11:38.356 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012988942, 2024-09-13T10:11:38.356 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.356 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.356 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.356 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.356 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.357 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:11:38.357 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.357 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.357 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.357 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:11:38.357 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.358 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041685+0000", 2024-09-13T10:11:38.358 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7769999999999998e-06 2024-09-13T10:11:38.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.359 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046575+0000", 2024-09-13T10:11:38.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048899520000000004 2024-09-13T10:11:38.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.359 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.359 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046633+0000", 2024-09-13T10:11:38.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7561e-05 2024-09-13T10:11:38.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.360 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.360 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046657+0000", 2024-09-13T10:11:38.360 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4223000000000002e-05 2024-09-13T10:11:38.360 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.360 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.360 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.360 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046819+0000", 2024-09-13T10:11:38.360 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016167699999999999 2024-09-13T10:11:38.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.361 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.361 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046882+0000", 2024-09-13T10:11:38.361 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3467999999999996e-05 2024-09-13T10:11:38.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.362 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.362 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054529+0000", 2024-09-13T10:11:38.362 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0076467289999999997 2024-09-13T10:11:38.362 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.362 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.362 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.362 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054671+0000", 2024-09-13T10:11:38.362 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014255499999999999 2024-09-13T10:11:38.362 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.363 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.363 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.363 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.363 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.363 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:217 3.2 e38/30 3:554c406d:::smithi19530714-18 low_tier:head v 38'6, mlcod=38'2)", 2024-09-13T10:11:38.363 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:11:38.363 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.283297443999999, 2024-09-13T10:11:38.364 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013473855999999999, 2024-09-13T10:11:38.364 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.364 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.364 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.364 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.364 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.364 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.365 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:11:38.365 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.365 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.365 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.365 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:11:38.365 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.365 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.365 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.365 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046731+0000", 2024-09-13T10:11:38.365 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.813e-06 2024-09-13T10:11:38.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.365 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.366 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.366 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054994+0000", 2024-09-13T10:11:38.366 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0082631279999999998 2024-09-13T10:11:38.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.366 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.366 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.366 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055024+0000", 2024-09-13T10:11:38.366 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9839e-05 2024-09-13T10:11:38.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.366 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.366 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.366 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055044+0000", 2024-09-13T10:11:38.366 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9507000000000001e-05 2024-09-13T10:11:38.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.367 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.367 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.367 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055186+0000", 2024-09-13T10:11:38.367 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014243099999999999 2024-09-13T10:11:38.367 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.367 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.367 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.367 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055240+0000", 2024-09-13T10:11:38.367 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4444999999999998e-05 2024-09-13T10:11:38.367 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.367 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.367 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.367 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060135+0000", 2024-09-13T10:11:38.368 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048942949999999999 2024-09-13T10:11:38.368 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.368 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.368 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.368 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060203+0000", 2024-09-13T10:11:38.368 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8398000000000005e-05 2024-09-13T10:11:38.368 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.368 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.368 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.368 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.369 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.369 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:220 3.2 e38/30 3:753d5785:::smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'7, mlcod=38'3)", 2024-09-13T10:11:38.369 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:11:38.369 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.274916175999998, 2024-09-13T10:11:38.369 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.009287455, 2024-09-13T10:11:38.369 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.369 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.369 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.369 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.369 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.369 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.370 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:11:38.370 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.370 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.370 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.370 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:11:38.370 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.370 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.370 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.370 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055112+0000", 2024-09-13T10:11:38.370 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6589999999999999e-06 2024-09-13T10:11:38.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.371 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.371 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.371 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057327+0000", 2024-09-13T10:11:38.371 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0022145229999999999 2024-09-13T10:11:38.371 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.371 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.371 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.371 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057354+0000", 2024-09-13T10:11:38.371 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7642000000000001e-05 2024-09-13T10:11:38.371 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.371 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.371 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.371 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057373+0000", 2024-09-13T10:11:38.372 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8768999999999999e-05 2024-09-13T10:11:38.372 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.372 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.372 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.372 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057511+0000", 2024-09-13T10:11:38.372 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013793900000000001 2024-09-13T10:11:38.372 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.372 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.372 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.372 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057550+0000", 2024-09-13T10:11:38.372 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9301999999999997e-05 2024-09-13T10:11:38.372 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.373 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.373 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.373 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.064324+0000", 2024-09-13T10:11:38.373 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0067737429999999996 2024-09-13T10:11:38.373 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.373 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.373 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.373 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.064398+0000", 2024-09-13T10:11:38.374 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3878e-05 2024-09-13T10:11:38.374 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.374 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.374 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.374 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.374 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.374 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:227 3.2 e38/30 3:6ac7800c:::smithi19530714-28 low_tier:head v 38'8, mlcod=38'6)", 2024-09-13T10:11:38.374 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:11:38.374 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.255325526, 2024-09-13T10:11:38.374 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018812816, 2024-09-13T10:11:38.374 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.375 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.375 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.375 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.375 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.375 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.375 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:11:38.375 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.375 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.376 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.376 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:11:38.376 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.376 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.376 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.376 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.376 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074708+0000", 2024-09-13T10:11:38.376 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.968e-06 2024-09-13T10:11:38.376 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.376 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.377 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.377 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085262+0000", 2024-09-13T10:11:38.377 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010553842000000001 2024-09-13T10:11:38.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.377 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.377 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.377 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085289+0000", 2024-09-13T10:11:38.377 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7288999999999999e-05 2024-09-13T10:11:38.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.377 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.377 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.377 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085305+0000", 2024-09-13T10:11:38.377 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5925000000000001e-05 2024-09-13T10:11:38.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.378 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.378 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.378 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085418+0000", 2024-09-13T10:11:38.378 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112584 2024-09-13T10:11:38.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.378 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.378 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.378 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085451+0000", 2024-09-13T10:11:38.378 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3605000000000001e-05 2024-09-13T10:11:38.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.378 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.378 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.379 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093417+0000", 2024-09-13T10:11:38.379 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0079654249999999999 2024-09-13T10:11:38.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.379 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.379 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.379 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093514+0000", 2024-09-13T10:11:38.379 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7177999999999997e-05 2024-09-13T10:11:38.379 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.379 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.379 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.380 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.380 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:230 3.2 e38/30 3:5d108080:::smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'9, mlcod=38'6)", 2024-09-13T10:11:38.380 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:11:38.380 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.241042495999999, 2024-09-13T10:11:38.380 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01953365, 2024-09-13T10:11:38.380 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.380 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.380 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.380 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.380 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.380 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.380 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:11:38.380 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.381 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.381 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:11:38.381 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.381 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.381 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088987+0000", 2024-09-13T10:11:38.382 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4600000000000002e-06 2024-09-13T10:11:38.382 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.382 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.382 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.382 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095842+0000", 2024-09-13T10:11:38.382 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0068548760000000002 2024-09-13T10:11:38.382 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.382 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.382 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.383 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095872+0000", 2024-09-13T10:11:38.383 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0476000000000001e-05 2024-09-13T10:11:38.383 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.383 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.383 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.383 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095890+0000", 2024-09-13T10:11:38.383 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7554000000000002e-05 2024-09-13T10:11:38.383 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.383 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.383 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.383 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.096006+0000", 2024-09-13T10:11:38.384 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011597199999999999 2024-09-13T10:11:38.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.384 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.384 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.384 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.096058+0000", 2024-09-13T10:11:38.384 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2274000000000001e-05 2024-09-13T10:11:38.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.384 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.384 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.384 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.108451+0000", 2024-09-13T10:11:38.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012392815 2024-09-13T10:11:38.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.385 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.385 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.108518+0000", 2024-09-13T10:11:38.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7223000000000003e-05 2024-09-13T10:11:38.385 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.385 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.385 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.385 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:232 3.2 e38/30 3:6538713c:::smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'10, mlcod=38'6)", 2024-09-13T10:11:38.385 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:11:38.386 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.234091614999997, 2024-09-13T10:11:38.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015307497, 2024-09-13T10:11:38.386 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.386 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.386 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.386 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:11:38.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.387 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:11:38.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.387 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095937+0000", 2024-09-13T10:11:38.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.367e-06 2024-09-13T10:11:38.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.388 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099841+0000", 2024-09-13T10:11:38.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0039043960000000001 2024-09-13T10:11:38.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.388 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.388 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.388 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099865+0000", 2024-09-13T10:11:38.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4538000000000001e-05 2024-09-13T10:11:38.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.388 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.388 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.388 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099881+0000", 2024-09-13T10:11:38.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5809000000000001e-05 2024-09-13T10:11:38.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.388 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099995+0000", 2024-09-13T10:11:38.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113843 2024-09-13T10:11:38.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.100043+0000", 2024-09-13T10:11:38.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8291999999999998e-05 2024-09-13T10:11:38.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111200+0000", 2024-09-13T10:11:38.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011156291 2024-09-13T10:11:38.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111243+0000", 2024-09-13T10:11:38.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2960999999999997e-05 2024-09-13T10:11:38.390 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.390 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.390 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.390 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:236 3.2 e38/30 3:73c2893b:::smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'11, mlcod=38'7)", 2024-09-13T10:11:38.391 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:11:38.391 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.224591496999999, 2024-09-13T10:11:38.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0098707670000000008, 2024-09-13T10:11:38.391 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.391 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.391 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.391 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:11:38.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.392 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:11:38.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.392 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.392 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.392 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105438+0000", 2024-09-13T10:11:38.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3010000000000001e-06 2024-09-13T10:11:38.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.392 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.392 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.392 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112262+0000", 2024-09-13T10:11:38.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.006824793 2024-09-13T10:11:38.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112279+0000", 2024-09-13T10:11:38.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6829000000000001e-05 2024-09-13T10:11:38.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112293+0000", 2024-09-13T10:11:38.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3801000000000001e-05 2024-09-13T10:11:38.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112381+0000", 2024-09-13T10:11:38.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7825999999999996e-05 2024-09-13T10:11:38.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112411+0000", 2024-09-13T10:11:38.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0434e-05 2024-09-13T10:11:38.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.115246+0000", 2024-09-13T10:11:38.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028352360000000001 2024-09-13T10:11:38.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.115306+0000", 2024-09-13T10:11:38.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9546999999999998e-05 2024-09-13T10:11:38.395 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.395 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.395 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.396 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:240 3.2 e38/30 3:7eee2f4a:::smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'12, mlcod=38'7)", 2024-09-13T10:11:38.396 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:11:38.396 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.217681710000001, 2024-09-13T10:11:38.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011743134000000001, 2024-09-13T10:11:38.396 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.396 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.396 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.396 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.397 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.397 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.397 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:11:38.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.397 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.397 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.397 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:11:38.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.397 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.397 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112346+0000", 2024-09-13T10:11:38.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.037e-06 2024-09-13T10:11:38.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120152+0000", 2024-09-13T10:11:38.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0078054700000000001 2024-09-13T10:11:38.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120173+0000", 2024-09-13T10:11:38.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1498999999999999e-05 2024-09-13T10:11:38.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120188+0000", 2024-09-13T10:11:38.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5159e-05 2024-09-13T10:11:38.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120313+0000", 2024-09-13T10:11:38.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012440199999999999 2024-09-13T10:11:38.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120358+0000", 2024-09-13T10:11:38.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-13T10:11:38.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.123945+0000", 2024-09-13T10:11:38.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035865490000000001 2024-09-13T10:11:38.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.401 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.401 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.124088+0000", 2024-09-13T10:11:38.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014318999999999999 2024-09-13T10:11:38.401 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.401 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.401 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.401 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:242 3.2 e38/30 3:501d22a1:::smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'13, mlcod=38'7)", 2024-09-13T10:11:38.401 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:11:38.401 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.209788701999997, 2024-09-13T10:11:38.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0088394330000000007, 2024-09-13T10:11:38.402 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.402 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.402 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.402 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:11:38.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:11:38.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120240+0000", 2024-09-13T10:11:38.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.731e-06 2024-09-13T10:11:38.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125395+0000", 2024-09-13T10:11:38.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0051547490000000001 2024-09-13T10:11:38.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125412+0000", 2024-09-13T10:11:38.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7833e-05 2024-09-13T10:11:38.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125426+0000", 2024-09-13T10:11:38.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3237999999999999e-05 2024-09-13T10:11:38.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125541+0000", 2024-09-13T10:11:38.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011566299999999999 2024-09-13T10:11:38.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125578+0000", 2024-09-13T10:11:38.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7197000000000003e-05 2024-09-13T10:11:38.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.128885+0000", 2024-09-13T10:11:38.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0033067119999999998 2024-09-13T10:11:38.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.129077+0000", 2024-09-13T10:11:38.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019231000000000001 2024-09-13T10:11:38.407 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.407 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.407 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.407 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:244 3.2 e38/30 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'14, mlcod=38'10)", 2024-09-13T10:11:38.407 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:11:38.407 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.203769334, 2024-09-13T10:11:38.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01536599, 2024-09-13T10:11:38.407 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.407 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.407 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.408 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:11:38.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:11:38.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126259+0000", 2024-09-13T10:11:38.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.528e-06 2024-09-13T10:11:38.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134336+0000", 2024-09-13T10:11:38.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080767410000000001 2024-09-13T10:11:38.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134356+0000", 2024-09-13T10:11:38.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9936999999999999e-05 2024-09-13T10:11:38.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134367+0000", 2024-09-13T10:11:38.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1352000000000001e-05 2024-09-13T10:11:38.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134532+0000", 2024-09-13T10:11:38.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001651 2024-09-13T10:11:38.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134578+0000", 2024-09-13T10:11:38.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5442000000000003e-05 2024-09-13T10:11:38.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.141521+0000", 2024-09-13T10:11:38.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069432720000000003 2024-09-13T10:11:38.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.141623+0000", 2024-09-13T10:11:38.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102618 2024-09-13T10:11:38.411 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.411 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.412 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.412 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:250 3.2 e38/30 3:78d40eb9:::smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'15, mlcod=38'10)", 2024-09-13T10:11:38.412 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:11:38.412 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.195572863000002, 2024-09-13T10:11:38.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095447210000000008, 2024-09-13T10:11:38.412 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.412 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.412 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.412 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:11:38.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:11:38.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134455+0000", 2024-09-13T10:11:38.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2699999999999999e-07 2024-09-13T10:11:38.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139352+0000", 2024-09-13T10:11:38.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048977810000000004 2024-09-13T10:11:38.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139374+0000", 2024-09-13T10:11:38.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1339e-05 2024-09-13T10:11:38.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139384+0000", 2024-09-13T10:11:38.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0183999999999999e-05 2024-09-13T10:11:38.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139477+0000", 2024-09-13T10:11:38.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2590999999999995e-05 2024-09-13T10:11:38.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139515+0000", 2024-09-13T10:11:38.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8624999999999997e-05 2024-09-13T10:11:38.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.143939+0000", 2024-09-13T10:11:38.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0044241189999999998 2024-09-13T10:11:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.143999+0000", 2024-09-13T10:11:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9354999999999997e-05 2024-09-13T10:11:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:245 3.2 e38/30 3:7837e849:::smithi19530714-46 low_tier:head v 38'16, mlcod=38'10)", 2024-09-13T10:11:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:11:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.190616509000002, 2024-09-13T10:11:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012687423999999999, 2024-09-13T10:11:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:11:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:11:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139411+0000", 2024-09-13T10:11:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6300000000000005e-07 2024-09-13T10:11:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144635+0000", 2024-09-13T10:11:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0052242169999999997 2024-09-13T10:11:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144651+0000", 2024-09-13T10:11:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6371e-05 2024-09-13T10:11:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144662+0000", 2024-09-13T10:11:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0003000000000001e-05 2024-09-13T10:11:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144741+0000", 2024-09-13T10:11:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9614e-05 2024-09-13T10:11:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144768+0000", 2024-09-13T10:11:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6826999999999999e-05 2024-09-13T10:11:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152015+0000", 2024-09-13T10:11:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0072470879999999996 2024-09-13T10:11:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152098+0000", 2024-09-13T10:11:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2640999999999997e-05 2024-09-13T10:11:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(osd.7.22:1 3.2 e38/30 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'17, mlcod=38'12)", 2024-09-13T10:11:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.144693+0000", 2024-09-13T10:11:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.185333737000001, 2024-09-13T10:11:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0075754180000000004, 2024-09-13T10:11:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144693+0000", 2024-09-13T10:11:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144693+0000", 2024-09-13T10:11:38.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144694+0000", 2024-09-13T10:11:38.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6500000000000008e-07 2024-09-13T10:11:38.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144699+0000", 2024-09-13T10:11:38.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6820000000000002e-06 2024-09-13T10:11:38.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144704+0000", 2024-09-13T10:11:38.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7869999999999998e-06 2024-09-13T10:11:38.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144710+0000", 2024-09-13T10:11:38.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3079999999999998e-06 2024-09-13T10:11:38.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.146857+0000", 2024-09-13T10:11:38.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021475209999999999 2024-09-13T10:11:38.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.146883+0000", 2024-09-13T10:11:38.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5947000000000001e-05 2024-09-13T10:11:38.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152225+0000", 2024-09-13T10:11:38.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053421450000000004 2024-09-13T10:11:38.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152268+0000", 2024-09-13T10:11:38.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3062999999999997e-05 2024-09-13T10:11:38.426 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.426 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.426 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.427 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(osd.6.0:104 4.5 e42/41 4:a7549961:::smithi19530714-27:head v 42'40, mlcod=42'38)", 2024-09-13T10:11:38.427 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:56.665769+0000", 2024-09-13T10:11:38.427 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 41.664258171999997, 2024-09-13T10:11:38.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065613520000000003, 2024-09-13T10:11:38.427 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.427 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.427 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.427 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.665769+0000", 2024-09-13T10:11:38.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.665769+0000", 2024-09-13T10:11:38.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.665771+0000", 2024-09-13T10:11:38.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.317e-06 2024-09-13T10:11:38.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.669348+0000", 2024-09-13T10:11:38.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035773689999999999 2024-09-13T10:11:38.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.669365+0000", 2024-09-13T10:11:38.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6623999999999999e-05 2024-09-13T10:11:38.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.669373+0000", 2024-09-13T10:11:38.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1820000000000006e-06 2024-09-13T10:11:38.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.669438+0000", 2024-09-13T10:11:38.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5265999999999995e-05 2024-09-13T10:11:38.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.669462+0000", 2024-09-13T10:11:38.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3555000000000001e-05 2024-09-13T10:11:38.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.672241+0000", 2024-09-13T10:11:38.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0027788650000000002 2024-09-13T10:11:38.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.672330+0000", 2024-09-13T10:11:38.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9173999999999997e-05 2024-09-13T10:11:38.431 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.431 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.431 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.431 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:2811 3.2 e71/30 3:5a2483ca:::890eeb33aa7613d6935d7cb9ed4860e522183188e5dc23a51d7045443a8d23dd:head v 71'264, mlcod=70'262)", 2024-09-13T10:11:38.431 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:32.583079+0000", 2024-09-13T10:11:38.432 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.7469473119999996, 2024-09-13T10:11:38.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0094005459999999992, 2024-09-13T10:11:38.432 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.432 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.432 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.432 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.583079+0000", 2024-09-13T10:11:38.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.583079+0000", 2024-09-13T10:11:38.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.583082+0000", 2024-09-13T10:11:38.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3070000000000001e-06 2024-09-13T10:11:38.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.587892+0000", 2024-09-13T10:11:38.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048105889999999997 2024-09-13T10:11:38.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.587924+0000", 2024-09-13T10:11:38.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1217999999999998e-05 2024-09-13T10:11:38.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.587938+0000", 2024-09-13T10:11:38.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4341999999999999e-05 2024-09-13T10:11:38.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.588076+0000", 2024-09-13T10:11:38.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013836600000000001 2024-09-13T10:11:38.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.588116+0000", 2024-09-13T10:11:38.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9480999999999997e-05 2024-09-13T10:11:38.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.592358+0000", 2024-09-13T10:11:38.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042423740000000001 2024-09-13T10:11:38.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.592480+0000", 2024-09-13T10:11:38.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012186899999999999 2024-09-13T10:11:38.436 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.436 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.436 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.436 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:2815 3.2 e72/30 3:5bb07d66:::1fa676dc3d1e506ea93eaeb504791933fdde520bb2cba670c5c248da82aeece8:head v 72'266, mlcod=71'264)", 2024-09-13T10:11:38.436 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:32.884401+0000", 2024-09-13T10:11:38.436 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.4456257289999996, 2024-09-13T10:11:38.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0064160989999999998, 2024-09-13T10:11:38.437 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.437 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.437 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.437 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.884401+0000", 2024-09-13T10:11:38.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.884401+0000", 2024-09-13T10:11:38.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.884402+0000", 2024-09-13T10:11:38.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5300000000000004e-07 2024-09-13T10:11:38.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.887370+0000", 2024-09-13T10:11:38.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029684210000000002 2024-09-13T10:11:38.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.887397+0000", 2024-09-13T10:11:38.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6889e-05 2024-09-13T10:11:38.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.887420+0000", 2024-09-13T10:11:38.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3451999999999998e-05 2024-09-13T10:11:38.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.887547+0000", 2024-09-13T10:11:38.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000126756 2024-09-13T10:11:38.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.887604+0000", 2024-09-13T10:11:38.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6793000000000003e-05 2024-09-13T10:11:38.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.890701+0000", 2024-09-13T10:11:38.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003097355 2024-09-13T10:11:38.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.890817+0000", 2024-09-13T10:11:38.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011588 2024-09-13T10:11:38.442 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.442 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.442 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.442 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.442 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.442 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:38.443 INFO:tasks.rados.rados.0.smithi195.stdout:1372: seq_num 268 ranges {504276=639087,1666233=695410} 2024-09-13T10:11:38.444 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T10:11:38.444 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:38.438+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:38.452 INFO:tasks.rados.rados.0.smithi195.stdout:1372: writing smithi19530714-45 from 504276 to 1143363 tid 1 2024-09-13T10:11:38.472 INFO:tasks.rados.rados.0.smithi195.stdout:1372: writing smithi19530714-45 from 1666233 to 2361643 tid 2 2024-09-13T10:11:38.472 INFO:tasks.rados.rados.0.smithi195.stdout:1360: done (12 left) 2024-09-13T10:11:38.472 INFO:tasks.rados.rados.0.smithi195.stdout:1361: done (11 left) 2024-09-13T10:11:38.472 INFO:tasks.rados.rados.0.smithi195.stdout:1362: done (10 left) 2024-09-13T10:11:38.472 INFO:tasks.rados.rados.0.smithi195.stdout:1363: done (9 left) 2024-09-13T10:11:38.473 INFO:tasks.rados.rados.0.smithi195.stdout:1364: done (8 left) 2024-09-13T10:11:38.473 INFO:tasks.rados.rados.0.smithi195.stdout:1365: done (7 left) 2024-09-13T10:11:38.473 INFO:tasks.rados.rados.0.smithi195.stdout:1366: done (6 left) 2024-09-13T10:11:38.473 INFO:tasks.rados.rados.0.smithi195.stdout:1367: done (5 left) 2024-09-13T10:11:38.473 INFO:tasks.rados.rados.0.smithi195.stdout:1368: done (4 left) 2024-09-13T10:11:38.473 INFO:tasks.rados.rados.0.smithi195.stdout:1369: done (3 left) 2024-09-13T10:11:38.473 INFO:tasks.rados.rados.0.smithi195.stdout:1370: done (2 left) 2024-09-13T10:11:38.473 INFO:tasks.rados.rados.0.smithi195.stdout:1371: done (1 left) 2024-09-13T10:11:38.473 INFO:tasks.rados.rados.0.smithi195.stdout:1373: set_chunk oid 22 target oid 2024-09-13T10:11:38.494 INFO:tasks.rados.rados.0.smithi195.stdout:1373: set_chunk oid 22 offset: 392192 length: 1146368 target oid 8ed3798f9ae0090888391becf03802d76887ccbfb9186018a16c1e249d281afc offset: 0 snap -1 2024-09-13T10:11:38.494 INFO:tasks.rados.rados.0.smithi195.stdout:1374: read oid 34 snap -1 2024-09-13T10:11:38.494 INFO:tasks.rados.rados.0.smithi195.stdout:1374: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:38.495 INFO:tasks.rados.rados.0.smithi195.stdout:1375: write oid 36 current snap is 0 2024-09-13T10:11:38.495 INFO:tasks.rados.rados.0.smithi195.stdout:1372: finishing write tid 1 to smithi19530714-45 2024-09-13T10:11:38.495 INFO:tasks.rados.rados.0.smithi195.stdout:1372: finishing write tid 2 to smithi19530714-45 2024-09-13T10:11:38.495 INFO:tasks.rados.rados.0.smithi195.stdout:1375: seq_num 269 ranges {521083=714336,1724715=555421} 2024-09-13T10:11:38.514 INFO:tasks.rados.rados.0.smithi195.stdout:1375: writing smithi19530714-36 from 521083 to 1235419 tid 1 2024-09-13T10:11:38.529 INFO:tasks.rados.rados.0.smithi195.stdout:1375: writing smithi19530714-36 from 1724715 to 2280136 tid 2 2024-09-13T10:11:38.529 INFO:tasks.rados.rados.0.smithi195.stdout:1376: read oid 13 snap -1 2024-09-13T10:11:38.529 INFO:tasks.rados.rados.0.smithi195.stdout:1376: expect (ObjNum 228 snap 0 seq_num 228) 2024-09-13T10:11:38.529 INFO:tasks.rados.rados.0.smithi195.stdout:1377: read oid 46 snap -1 2024-09-13T10:11:38.529 INFO:tasks.rados.rados.0.smithi195.stdout:1377: expect deleted 2024-09-13T10:11:38.529 INFO:tasks.rados.rados.0.smithi195.stdout:1378: write (excl) oid 32 current snap is 0 2024-09-13T10:11:38.529 INFO:tasks.rados.rados.0.smithi195.stdout:1378: seq_num 270 ranges {537890=789585,1783197=699079} 2024-09-13T10:11:38.543 INFO:tasks.rados.rados.0.smithi195.stdout:1378: writing smithi19530714-32 from 537890 to 1327475 tid 1 2024-09-13T10:11:38.544 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:38.539+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:38.553 INFO:tasks.rados.rados.0.smithi195.stdout:1378: writing smithi19530714-32 from 1783197 to 2482276 tid 2 2024-09-13T10:11:38.553 INFO:tasks.rados.rados.0.smithi195.stdout:1379: read oid 48 snap -1 2024-09-13T10:11:38.553 INFO:tasks.rados.rados.0.smithi195.stdout:1379: expect (ObjNum 264 snap 0 seq_num 264) 2024-09-13T10:11:38.553 INFO:tasks.rados.rados.0.smithi195.stdout:1380: set_chunk oid 28 target oid 2024-09-13T10:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:38.582 INFO:tasks.rados.rados.0.smithi195.stdout:1380: set_chunk oid 28 offset: 870912 length: 1417728 target oid 37fab00bed791e76d422b4729d813dc1f49bba63932352cb886998cef789e7b3 offset: 0 snap -1 2024-09-13T10:11:38.582 INFO:tasks.rados.rados.0.smithi195.stdout:1381: set_chunk oid 20 target oid 2024-09-13T10:11:38.591 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T10:11:38.610 INFO:tasks.rados.rados.0.smithi195.stdout:1381: set_chunk oid 20 offset: 585728 length: 2168832 target oid 2de456ef195b1e3657c1aba04023dfbeca0a460056bb00695a713e375ac556e2 offset: 0 snap -1 2024-09-13T10:11:38.610 INFO:tasks.rados.rados.0.smithi195.stdout:1382: set_chunk oid 19 target oid 2024-09-13T10:11:38.635 INFO:tasks.rados.rados.0.smithi195.stdout:1382: set_chunk oid 19 offset: 536064 length: 1571840 target oid 6103ef44e0364747b02bee2da71c28fbc7346b99c73fc085e3f4048beddb5e06 offset: 0 snap -1 2024-09-13T10:11:38.635 INFO:tasks.rados.rados.0.smithi195.stdout:1383: write (excl) oid 7 current snap is 0 2024-09-13T10:11:38.635 INFO:tasks.rados.rados.0.smithi195.stdout:1372: finishing write tid 3 to smithi19530714-45 2024-09-13T10:11:38.635 INFO:tasks.rados.rados.0.smithi195.stdout:1372: finishing write tid 4 to smithi19530714-45 2024-09-13T10:11:38.635 INFO:tasks.rados.rados.0.smithi195.stdout:1372: oid 45 updating version 0 to 128 2024-09-13T10:11:38.635 INFO:tasks.rados.rados.0.smithi195.stdout:1372: oid 45 updating version 128 to 129 2024-09-13T10:11:38.635 INFO:tasks.rados.rados.0.smithi195.stdout:1372: oid 45 updating version 129 to 130 2024-09-13T10:11:38.635 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 130 (ObjNum 268 snap 0 seq_num 268) dirty exists 2024-09-13T10:11:38.635 INFO:tasks.rados.rados.0.smithi195.stdout:1372: left oid 45 (ObjNum 268 snap 0 seq_num 268) 2024-09-13T10:11:38.635 INFO:tasks.rados.rados.0.smithi195.stdout:1383: seq_num 271 ranges {554697=464834,1725326=559090} 2024-09-13T10:11:38.649 INFO:tasks.rados.rados.0.smithi195.stdout:1383: writing smithi19530714-7 from 554697 to 1019531 tid 1 2024-09-13T10:11:38.665 INFO:tasks.rados.rados.0.smithi195.stdout:1383: writing smithi19530714-7 from 1725326 to 2284416 tid 2 2024-09-13T10:11:38.666 INFO:tasks.rados.rados.0.smithi195.stdout:1372: done (11 left) 2024-09-13T10:11:38.666 INFO:tasks.rados.rados.0.smithi195.stdout:1384: write oid 31 current snap is 0 2024-09-13T10:11:38.666 INFO:tasks.rados.rados.0.smithi195.stdout:1384: seq_num 272 ranges {571504=540083,1783808=702748} 2024-09-13T10:11:38.677 INFO:tasks.rados.rados.0.smithi195.stdout:1384: writing smithi19530714-31 from 571504 to 1111587 tid 1 2024-09-13T10:11:38.678 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:38.675+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:38.690 INFO:tasks.rados.rados.0.smithi195.stdout:1384: writing smithi19530714-31 from 1783808 to 2486556 tid 2 2024-09-13T10:11:38.690 INFO:tasks.rados.rados.0.smithi195.stdout:1385: write oid 26 current snap is 0 2024-09-13T10:11:38.690 INFO:tasks.rados.rados.0.smithi195.stdout:1385: seq_num 273 ranges {588311=615332,1842290=562759} 2024-09-13T10:11:38.699 INFO:tasks.rados.rados.0.smithi195.stdout:1385: writing smithi19530714-26 from 588311 to 1203643 tid 1 2024-09-13T10:11:38.708 INFO:tasks.rados.rados.0.smithi195.stdout:1385: writing smithi19530714-26 from 1842290 to 2405049 tid 2 2024-09-13T10:11:38.708 INFO:tasks.rados.rados.0.smithi195.stdout:1386: read oid 11 snap -1 2024-09-13T10:11:38.708 INFO:tasks.rados.rados.0.smithi195.stdout:1386: expect (ObjNum 267 snap 0 seq_num 267) 2024-09-13T10:11:38.708 INFO:tasks.rados.rados.0.smithi195.stdout:1387: write oid 50 current snap is 0 2024-09-13T10:11:38.708 INFO:tasks.rados.rados.0.smithi195.stdout:1373: finishing set_chunk to oid 22 2024-09-13T10:11:38.709 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 22 set_chunk returned error code -95 offset: 392192 length: 1146368 2024-09-13T10:11:38.709 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 0 (ObjNum 139 snap 0 seq_num 139) dirty exists 2024-09-13T10:11:38.709 INFO:tasks.rados.rados.0.smithi195.stdout:1374: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:38.709 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:38.745 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:38.742+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:38.759 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:11:38.760 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:11:38.760 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:38.775 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T10:11:38.846 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:38.842+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:38.886 INFO:tasks.rados.rados.0.smithi195.stdout:1375: finishing write tid 1 to smithi19530714-36 2024-09-13T10:11:38.887 INFO:tasks.rados.rados.0.smithi195.stdout:1375: finishing write tid 2 to smithi19530714-36 2024-09-13T10:11:38.887 INFO:tasks.rados.rados.0.smithi195.stdout:1375: finishing write tid 3 to smithi19530714-36 2024-09-13T10:11:38.887 INFO:tasks.rados.rados.0.smithi195.stdout:1375: finishing write tid 4 to smithi19530714-36 2024-09-13T10:11:38.887 INFO:tasks.rados.rados.0.smithi195.stdout:1375: oid 36 updating version 0 to 35 2024-09-13T10:11:38.887 INFO:tasks.rados.rados.0.smithi195.stdout:1375: oid 36 updating version 35 to 36 2024-09-13T10:11:38.887 INFO:tasks.rados.rados.0.smithi195.stdout:1375: oid 36 version 36 is already newer than 34 2024-09-13T10:11:38.887 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 36 (ObjNum 269 snap 0 seq_num 269) dirty exists 2024-09-13T10:11:38.887 INFO:tasks.rados.rados.0.smithi195.stdout:1375: left oid 36 (ObjNum 269 snap 0 seq_num 269) 2024-09-13T10:11:38.887 INFO:tasks.rados.rados.0.smithi195.stdout:1376: expect (ObjNum 228 snap 0 seq_num 228) 2024-09-13T10:11:38.887 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:38.941 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:38.941 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:11:38.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:11:38.941 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:11:38.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.941 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:203 3.1 e38/30 3:b033e801:::smithi19530714-4 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:38.941 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.002504+0000", 2024-09-13T10:11:38.941 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.935519304000003, 2024-09-13T10:11:38.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016371101999999998, 2024-09-13T10:11:38.942 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.942 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.942 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.942 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002504+0000", 2024-09-13T10:11:38.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002504+0000", 2024-09-13T10:11:38.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002507+0000", 2024-09-13T10:11:38.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2699999999999999e-06 2024-09-13T10:11:38.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007079+0000", 2024-09-13T10:11:38.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045722200000000001 2024-09-13T10:11:38.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007114+0000", 2024-09-13T10:11:38.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5389000000000003e-05 2024-09-13T10:11:38.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007132+0000", 2024-09-13T10:11:38.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7489000000000001e-05 2024-09-13T10:11:38.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007266+0000", 2024-09-13T10:11:38.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013401900000000001 2024-09-13T10:11:38.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007299+0000", 2024-09-13T10:11:38.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3136000000000003e-05 2024-09-13T10:11:38.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.945 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.018741+0000", 2024-09-13T10:11:38.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011441957000000001 2024-09-13T10:11:38.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.945 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.018875+0000", 2024-09-13T10:11:38.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013462200000000001 2024-09-13T10:11:38.946 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.946 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.946 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.946 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:211 3.0 e38/30 3:0596651d:::smithi19530714-12 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:38.946 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.007453+0000", 2024-09-13T10:11:38.946 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.93057048, 2024-09-13T10:11:38.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018632359000000001, 2024-09-13T10:11:38.946 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.946 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.947 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.947 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007453+0000", 2024-09-13T10:11:38.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007453+0000", 2024-09-13T10:11:38.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007455+0000", 2024-09-13T10:11:38.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4030000000000001e-06 2024-09-13T10:11:38.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020746+0000", 2024-09-13T10:11:38.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013290219000000001 2024-09-13T10:11:38.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020788+0000", 2024-09-13T10:11:38.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2342000000000002e-05 2024-09-13T10:11:38.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020868+0000", 2024-09-13T10:11:38.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9519999999999998e-05 2024-09-13T10:11:38.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021080+0000", 2024-09-13T10:11:38.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021249600000000001 2024-09-13T10:11:38.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021113+0000", 2024-09-13T10:11:38.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2786000000000001e-05 2024-09-13T10:11:38.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.025974+0000", 2024-09-13T10:11:38.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048614929999999997 2024-09-13T10:11:38.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.950 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026085+0000", 2024-09-13T10:11:38.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001111 2024-09-13T10:11:38.951 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.951 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.951 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.951 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:205 3.1 e38/30 3:aec11a92:::smithi19530714-6 low_tier:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:38.951 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.014358+0000", 2024-09-13T10:11:38.951 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.923665856, 2024-09-13T10:11:38.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033491923999999999, 2024-09-13T10:11:38.951 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.952 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.952 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.952 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.014358+0000", 2024-09-13T10:11:38.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.014358+0000", 2024-09-13T10:11:38.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.014360+0000", 2024-09-13T10:11:38.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.057e-06 2024-09-13T10:11:38.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.955 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026358+0000", 2024-09-13T10:11:38.955 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011998688 2024-09-13T10:11:38.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.955 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.955 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026392+0000", 2024-09-13T10:11:38.955 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3575999999999998e-05 2024-09-13T10:11:38.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027340+0000", 2024-09-13T10:11:38.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00094826100000000005 2024-09-13T10:11:38.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027449+0000", 2024-09-13T10:11:38.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108797 2024-09-13T10:11:38.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027500+0000", 2024-09-13T10:11:38.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0757999999999998e-05 2024-09-13T10:11:38.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047748+0000", 2024-09-13T10:11:38.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020247884000000001 2024-09-13T10:11:38.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047850+0000", 2024-09-13T10:11:38.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101903 2024-09-13T10:11:38.958 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.958 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.958 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.958 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:212 3.0 e38/30 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:38.958 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.021018+0000", 2024-09-13T10:11:38.958 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.917005048, 2024-09-13T10:11:38.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044503290000000001, 2024-09-13T10:11:38.958 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.959 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.959 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.959 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021018+0000", 2024-09-13T10:11:38.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021018+0000", 2024-09-13T10:11:38.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021019+0000", 2024-09-13T10:11:38.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7400000000000002e-07 2024-09-13T10:11:38.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038659+0000", 2024-09-13T10:11:38.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017639555000000001 2024-09-13T10:11:38.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038678+0000", 2024-09-13T10:11:38.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8912000000000001e-05 2024-09-13T10:11:38.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038693+0000", 2024-09-13T10:11:38.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5393e-05 2024-09-13T10:11:38.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038829+0000", 2024-09-13T10:11:38.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000135912 2024-09-13T10:11:38.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.963 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.963 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038887+0000", 2024-09-13T10:11:38.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7858000000000001e-05 2024-09-13T10:11:38.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.963 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.963 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065395+0000", 2024-09-13T10:11:38.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02650773 2024-09-13T10:11:38.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065522+0000", 2024-09-13T10:11:38.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012705600000000001 2024-09-13T10:11:38.964 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.964 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.964 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.964 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:213 3.1 e38/30 3:accf4e56:::smithi19530714-14 low_tier:head v 38'3, mlcod=0'0)", 2024-09-13T10:11:38.964 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.027392+0000", 2024-09-13T10:11:38.964 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.910631539000001, 2024-09-13T10:11:38.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025423449000000001, 2024-09-13T10:11:38.965 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.965 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.965 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.965 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027392+0000", 2024-09-13T10:11:38.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027392+0000", 2024-09-13T10:11:38.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027394+0000", 2024-09-13T10:11:38.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7370000000000001e-06 2024-09-13T10:11:38.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032354+0000", 2024-09-13T10:11:38.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049601589999999996 2024-09-13T10:11:38.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032413+0000", 2024-09-13T10:11:38.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9095000000000003e-05 2024-09-13T10:11:38.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032426+0000", 2024-09-13T10:11:38.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3465999999999999e-05 2024-09-13T10:11:38.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032805+0000", 2024-09-13T10:11:38.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000378704 2024-09-13T10:11:38.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032862+0000", 2024-09-13T10:11:38.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6878e-05 2024-09-13T10:11:38.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.052688+0000", 2024-09-13T10:11:38.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019826308000000001 2024-09-13T10:11:38.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.052815+0000", 2024-09-13T10:11:38.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127102 2024-09-13T10:11:38.969 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.970 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.970 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.970 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:209 3.1 e38/30 3:9798d58d:::smithi19530714-10 low_tier:head v 38'4, mlcod=0'0)", 2024-09-13T10:11:38.970 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.032594+0000", 2024-09-13T10:11:38.970 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.905429104, 2024-09-13T10:11:38.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028120108000000001, 2024-09-13T10:11:38.970 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.970 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.971 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.971 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032594+0000", 2024-09-13T10:11:38.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032594+0000", 2024-09-13T10:11:38.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032596+0000", 2024-09-13T10:11:38.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2330000000000001e-06 2024-09-13T10:11:38.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.035217+0000", 2024-09-13T10:11:38.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026212679999999999 2024-09-13T10:11:38.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.035250+0000", 2024-09-13T10:11:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3065000000000003e-05 2024-09-13T10:11:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.035272+0000", 2024-09-13T10:11:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2163000000000001e-05 2024-09-13T10:11:38.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038562+0000", 2024-09-13T10:11:38.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003290322 2024-09-13T10:11:38.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038642+0000", 2024-09-13T10:11:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9457999999999997e-05 2024-09-13T10:11:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060605+0000", 2024-09-13T10:11:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021962751999999999 2024-09-13T10:11:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060715+0000", 2024-09-13T10:11:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010984699999999999 2024-09-13T10:11:38.975 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:207 3.0 e38/30 3:332cd44b:::smithi19530714-8 low_tier:head v 38'3, mlcod=0'0)", 2024-09-13T10:11:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.038735+0000", 2024-09-13T10:11:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.899288525000003, 2024-09-13T10:11:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.026872200999999998, 2024-09-13T10:11:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.976 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038735+0000", 2024-09-13T10:11:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038735+0000", 2024-09-13T10:11:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038737+0000", 2024-09-13T10:11:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5430000000000001e-06 2024-09-13T10:11:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046860+0000", 2024-09-13T10:11:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081236629999999997 2024-09-13T10:11:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046895+0000", 2024-09-13T10:11:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4989e-05 2024-09-13T10:11:38.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046915+0000", 2024-09-13T10:11:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0026000000000001e-05 2024-09-13T10:11:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047057+0000", 2024-09-13T10:11:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014151800000000001 2024-09-13T10:11:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047111+0000", 2024-09-13T10:11:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3885000000000003e-05 2024-09-13T10:11:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065539+0000", 2024-09-13T10:11:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018428225999999999 2024-09-13T10:11:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065607+0000", 2024-09-13T10:11:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8350999999999997e-05 2024-09-13T10:11:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.980 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:216 3.0 e38/30 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:11:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.046965+0000", 2024-09-13T10:11:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.891058061999999, 2024-09-13T10:11:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.022018617000000001, 2024-09-13T10:11:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046965+0000", 2024-09-13T10:11:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046965+0000", 2024-09-13T10:11:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046967+0000", 2024-09-13T10:11:38.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3689999999999999e-06 2024-09-13T10:11:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054950+0000", 2024-09-13T10:11:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0079834910000000005 2024-09-13T10:11:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054967+0000", 2024-09-13T10:11:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6382e-05 2024-09-13T10:11:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054981+0000", 2024-09-13T10:11:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3974999999999999e-05 2024-09-13T10:11:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055057+0000", 2024-09-13T10:11:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5964000000000007e-05 2024-09-13T10:11:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055100+0000", 2024-09-13T10:11:38.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3522000000000001e-05 2024-09-13T10:11:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.068892+0000", 2024-09-13T10:11:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01379175 2024-09-13T10:11:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.068984+0000", 2024-09-13T10:11:38.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2163999999999994e-05 2024-09-13T10:11:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:219 3.1 e38/30 3:87bfc7b1:::smithi19530714-20 low_tier:head v 38'5, mlcod=0'0)", 2024-09-13T10:11:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.048853+0000", 2024-09-13T10:11:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.889170589999999, 2024-09-13T10:11:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017128925999999999, 2024-09-13T10:11:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.986 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.048853+0000", 2024-09-13T10:11:38.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.048853+0000", 2024-09-13T10:11:38.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.048855+0000", 2024-09-13T10:11:38.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.877e-06 2024-09-13T10:11:38.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.988 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.988 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053357+0000", 2024-09-13T10:11:38.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045020149999999998 2024-09-13T10:11:38.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.988 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.988 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053378+0000", 2024-09-13T10:11:38.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1639000000000001e-05 2024-09-13T10:11:38.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053396+0000", 2024-09-13T10:11:38.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7135000000000001e-05 2024-09-13T10:11:38.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053526+0000", 2024-09-13T10:11:38.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013083999999999999 2024-09-13T10:11:38.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053590+0000", 2024-09-13T10:11:38.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3355000000000006e-05 2024-09-13T10:11:38.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065909+0000", 2024-09-13T10:11:38.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012319583 2024-09-13T10:11:38.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065982+0000", 2024-09-13T10:11:38.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2482000000000002e-05 2024-09-13T10:11:38.990 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.990 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.991 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.991 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:221 3.0 e38/30 3:18ae852b:::smithi19530714-22 low_tier:head v 38'5, mlcod=0'0)", 2024-09-13T10:11:38.991 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.058895+0000", 2024-09-13T10:11:38.991 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.879128375000001, 2024-09-13T10:11:38.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019544733000000002, 2024-09-13T10:11:38.991 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.991 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.991 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.991 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.058895+0000", 2024-09-13T10:11:38.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.992 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.058895+0000", 2024-09-13T10:11:38.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.992 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.058897+0000", 2024-09-13T10:11:38.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0420000000000001e-06 2024-09-13T10:11:38.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.061589+0000", 2024-09-13T10:11:38.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026919869999999999 2024-09-13T10:11:38.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.061614+0000", 2024-09-13T10:11:38.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4984999999999999e-05 2024-09-13T10:11:38.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.061629+0000", 2024-09-13T10:11:38.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4501999999999999e-05 2024-09-13T10:11:38.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065668+0000", 2024-09-13T10:11:38.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040393260000000002 2024-09-13T10:11:38.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065711+0000", 2024-09-13T10:11:38.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2673999999999998e-05 2024-09-13T10:11:38.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:38.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.078336+0000", 2024-09-13T10:11:38.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012625113 2024-09-13T10:11:38.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:38.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.078440+0000", 2024-09-13T10:11:38.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000104104 2024-09-13T10:11:38.995 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.995 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:38.995 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:38.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.996 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:231 3.1 e38/30 3:9d247594:::smithi19530714-32 low_tier:head v 38'7, mlcod=38'4)", 2024-09-13T10:11:38.996 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.076614+0000", 2024-09-13T10:11:38.996 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.861409412, 2024-09-13T10:11:38.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017222986999999999, 2024-09-13T10:11:38.996 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:38.996 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:38.996 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:38.996 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:38.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:38.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.076614+0000", 2024-09-13T10:11:38.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:38.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.076614+0000", 2024-09-13T10:11:38.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:38.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:38.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.076616+0000", 2024-09-13T10:11:38.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5069999999999999e-06 2024-09-13T10:11:38.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:38.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082649+0000", 2024-09-13T10:11:38.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0060329499999999996 2024-09-13T10:11:38.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:38.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082674+0000", 2024-09-13T10:11:38.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5624000000000001e-05 2024-09-13T10:11:38.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:38.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082690+0000", 2024-09-13T10:11:38.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6152999999999999e-05 2024-09-13T10:11:38.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:38.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082812+0000", 2024-09-13T10:11:38.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012209499999999999 2024-09-13T10:11:38.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:38.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:38.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:38.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082863+0000", 2024-09-13T10:11:39.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0781000000000001e-05 2024-09-13T10:11:39.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.000 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.000 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093763+0000", 2024-09-13T10:11:39.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010899382000000001 2024-09-13T10:11:39.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.000 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.000 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093837+0000", 2024-09-13T10:11:39.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4494999999999997e-05 2024-09-13T10:11:39.000 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.000 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.001 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.001 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:234 3.1 e38/30 3:bdce9caa:::smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'8, mlcod=38'4)", 2024-09-13T10:11:39.001 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.086591+0000", 2024-09-13T10:11:39.001 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.851432299000003, 2024-09-13T10:11:39.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025699538000000001, 2024-09-13T10:11:39.001 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.001 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.002 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.002 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.086591+0000", 2024-09-13T10:11:39.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.086591+0000", 2024-09-13T10:11:39.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.086623+0000", 2024-09-13T10:11:39.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1335999999999999e-05 2024-09-13T10:11:39.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094695+0000", 2024-09-13T10:11:39.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080726919999999994 2024-09-13T10:11:39.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094716+0000", 2024-09-13T10:11:39.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1015000000000001e-05 2024-09-13T10:11:39.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094732+0000", 2024-09-13T10:11:39.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5532999999999998e-05 2024-09-13T10:11:39.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094807+0000", 2024-09-13T10:11:39.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4837000000000002e-05 2024-09-13T10:11:39.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094836+0000", 2024-09-13T10:11:39.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8912999999999999e-05 2024-09-13T10:11:39.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112239+0000", 2024-09-13T10:11:39.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017402995000000001 2024-09-13T10:11:39.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112291+0000", 2024-09-13T10:11:39.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2216999999999998e-05 2024-09-13T10:11:39.006 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.006 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.006 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.006 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:233 3.0 e38/30 3:24ead787:::smithi19530714-34 low_tier:head v 38'9, mlcod=38'4)", 2024-09-13T10:11:39.007 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.090071+0000", 2024-09-13T10:11:39.007 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.847952292999999, 2024-09-13T10:11:39.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.023396641999999999, 2024-09-13T10:11:39.007 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.007 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.007 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.007 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.007 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090071+0000", 2024-09-13T10:11:39.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090071+0000", 2024-09-13T10:11:39.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090073+0000", 2024-09-13T10:11:39.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0470000000000001e-06 2024-09-13T10:11:39.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097736+0000", 2024-09-13T10:11:39.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007662416 2024-09-13T10:11:39.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097761+0000", 2024-09-13T10:11:39.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4919e-05 2024-09-13T10:11:39.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097773+0000", 2024-09-13T10:11:39.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2182e-05 2024-09-13T10:11:39.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097877+0000", 2024-09-13T10:11:39.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010438900000000001 2024-09-13T10:11:39.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097910+0000", 2024-09-13T10:11:39.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2808999999999997e-05 2024-09-13T10:11:39.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113429+0000", 2024-09-13T10:11:39.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015518557000000001 2024-09-13T10:11:39.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113468+0000", 2024-09-13T10:11:39.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9323000000000003e-05 2024-09-13T10:11:39.011 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.011 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.011 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.011 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:223 3.1 e38/30 3:8759f4e0:::smithi19530714-24 low_tier:head v 38'9, mlcod=38'4)", 2024-09-13T10:11:39.011 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.094775+0000", 2024-09-13T10:11:39.012 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.843248776000003, 2024-09-13T10:11:39.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018476023000000001, 2024-09-13T10:11:39.012 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.012 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.012 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.012 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.012 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094775+0000", 2024-09-13T10:11:39.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094775+0000", 2024-09-13T10:11:39.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094776+0000", 2024-09-13T10:11:39.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.466e-06 2024-09-13T10:11:39.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097100+0000", 2024-09-13T10:11:39.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002323435 2024-09-13T10:11:39.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097123+0000", 2024-09-13T10:11:39.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3756000000000001e-05 2024-09-13T10:11:39.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097139+0000", 2024-09-13T10:11:39.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5424e-05 2024-09-13T10:11:39.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.098642+0000", 2024-09-13T10:11:39.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015034359999999999 2024-09-13T10:11:39.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.098683+0000", 2024-09-13T10:11:39.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0756000000000001e-05 2024-09-13T10:11:39.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113193+0000", 2024-09-13T10:11:39.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014509831000000001 2024-09-13T10:11:39.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113251+0000", 2024-09-13T10:11:39.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7918999999999999e-05 2024-09-13T10:11:39.016 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.016 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.016 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.017 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:241 3.0 e38/30 3:388ca573:::smithi19530714-42 low_tier:head v 38'12, mlcod=38'9)", 2024-09-13T10:11:39.017 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.137896+0000", 2024-09-13T10:11:39.017 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.800127123000003, 2024-09-13T10:11:39.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015462800000000001, 2024-09-13T10:11:39.017 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.017 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.017 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.017 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.017 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.017 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.137896+0000", 2024-09-13T10:11:39.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.137896+0000", 2024-09-13T10:11:39.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.137897+0000", 2024-09-13T10:11:39.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5099999999999998e-07 2024-09-13T10:11:39.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.142234+0000", 2024-09-13T10:11:39.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0043362160000000004 2024-09-13T10:11:39.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.142248+0000", 2024-09-13T10:11:39.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4854e-05 2024-09-13T10:11:39.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.142257+0000", 2024-09-13T10:11:39.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5820000000000002e-06 2024-09-13T10:11:39.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.142334+0000", 2024-09-13T10:11:39.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6518000000000002e-05 2024-09-13T10:11:39.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.142367+0000", 2024-09-13T10:11:39.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3667000000000001e-05 2024-09-13T10:11:39.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152035+0000", 2024-09-13T10:11:39.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0096680550000000001 2024-09-13T10:11:39.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.153359+0000", 2024-09-13T10:11:39.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001323957 2024-09-13T10:11:39.021 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.021 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.022 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.022 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1959 4.13 4.e5457733 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:39.022 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.866379+0000", 2024-09-13T10:11:39.022 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.071644411000001, 2024-09-13T10:11:39.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.83266194900000001, 2024-09-13T10:11:39.022 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.022 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.022 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:11:39.022 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.022 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:39.023 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:39.023 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1959 2024-09-13T10:11:39.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.023 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.866379+0000", 2024-09-13T10:11:39.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.866379+0000", 2024-09-13T10:11:39.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.866381+0000", 2024-09-13T10:11:39.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.125e-06 2024-09-13T10:11:39.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867454+0000", 2024-09-13T10:11:39.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001072649 2024-09-13T10:11:39.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867475+0000", 2024-09-13T10:11:39.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1299999999999999e-05 2024-09-13T10:11:39.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867486+0000", 2024-09-13T10:11:39.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1167e-05 2024-09-13T10:11:39.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867581+0000", 2024-09-13T10:11:39.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4561999999999993e-05 2024-09-13T10:11:39.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:11:39.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867592+0000", 2024-09-13T10:11:39.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1031e-05 2024-09-13T10:11:39.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699018+0000", 2024-09-13T10:11:39.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.83142604099999995 2024-09-13T10:11:39.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699041+0000", 2024-09-13T10:11:39.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3074e-05 2024-09-13T10:11:39.027 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.027 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.027 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.027 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1960 4.13 4.e5457733 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:39.027 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.874165+0000", 2024-09-13T10:11:39.027 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.063858771, 2024-09-13T10:11:39.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82492870799999996, 2024-09-13T10:11:39.027 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.028 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.028 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:11:39.028 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.028 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:39.028 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:39.028 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1960 2024-09-13T10:11:39.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.028 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874165+0000", 2024-09-13T10:11:39.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874165+0000", 2024-09-13T10:11:39.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874166+0000", 2024-09-13T10:11:39.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.325e-06 2024-09-13T10:11:39.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874706+0000", 2024-09-13T10:11:39.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00053996899999999995 2024-09-13T10:11:39.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874719+0000", 2024-09-13T10:11:39.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2866e-05 2024-09-13T10:11:39.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874726+0000", 2024-09-13T10:11:39.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7560000000000002e-06 2024-09-13T10:11:39.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874801+0000", 2024-09-13T10:11:39.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5612999999999996e-05 2024-09-13T10:11:39.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:11:39.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874810+0000", 2024-09-13T10:11:39.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2619999999999998e-06 2024-09-13T10:11:39.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699082+0000", 2024-09-13T10:11:39.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82427217699999999 2024-09-13T10:11:39.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699093+0000", 2024-09-13T10:11:39.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1739999999999999e-05 2024-09-13T10:11:39.032 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.032 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.032 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.033 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1961 4.13 4.e5457733 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:39.033 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.877465+0000", 2024-09-13T10:11:39.033 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.060558520000001, 2024-09-13T10:11:39.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82170111099999998, 2024-09-13T10:11:39.033 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.033 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.033 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:11:39.033 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.033 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:39.033 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:39.034 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1961 2024-09-13T10:11:39.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.034 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877465+0000", 2024-09-13T10:11:39.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877465+0000", 2024-09-13T10:11:39.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877466+0000", 2024-09-13T10:11:39.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0100000000000004e-07 2024-09-13T10:11:39.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877640+0000", 2024-09-13T10:11:39.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000173795 2024-09-13T10:11:39.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877650+0000", 2024-09-13T10:11:39.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0465000000000001e-05 2024-09-13T10:11:39.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877656+0000", 2024-09-13T10:11:39.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6849999999999998e-06 2024-09-13T10:11:39.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877714+0000", 2024-09-13T10:11:39.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8097999999999998e-05 2024-09-13T10:11:39.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:11:39.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877726+0000", 2024-09-13T10:11:39.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1857e-05 2024-09-13T10:11:39.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699146+0000", 2024-09-13T10:11:39.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82142019700000002 2024-09-13T10:11:39.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699166+0000", 2024-09-13T10:11:39.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0213000000000001e-05 2024-09-13T10:11:39.038 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.038 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.038 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.038 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1962 4.13 4.e5457733 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:39.038 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.877669+0000", 2024-09-13T10:11:39.038 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.060354385, 2024-09-13T10:11:39.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82156359899999998, 2024-09-13T10:11:39.038 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.038 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.038 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:11:39.039 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.039 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:39.039 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:39.039 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1962 2024-09-13T10:11:39.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.039 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877669+0000", 2024-09-13T10:11:39.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877669+0000", 2024-09-13T10:11:39.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877670+0000", 2024-09-13T10:11:39.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9899999999999999e-07 2024-09-13T10:11:39.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877675+0000", 2024-09-13T10:11:39.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5510000000000001e-06 2024-09-13T10:11:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877684+0000", 2024-09-13T10:11:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5829999999999996e-06 2024-09-13T10:11:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877694+0000", 2024-09-13T10:11:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0207000000000001e-05 2024-09-13T10:11:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.041 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877784+0000", 2024-09-13T10:11:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9990999999999999e-05 2024-09-13T10:11:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:11:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877798+0000", 2024-09-13T10:11:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3444e-05 2024-09-13T10:11:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699222+0000", 2024-09-13T10:11:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82142437999999995 2024-09-13T10:11:39.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699233+0000", 2024-09-13T10:11:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0644e-05 2024-09-13T10:11:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1963 4.13 4.e5457733 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.877721+0000", 2024-09-13T10:11:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.060302127, 2024-09-13T10:11:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82158741499999999, 2024-09-13T10:11:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:11:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1963 2024-09-13T10:11:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877721+0000", 2024-09-13T10:11:39.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877721+0000", 2024-09-13T10:11:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877722+0000", 2024-09-13T10:11:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4099999999999998e-07 2024-09-13T10:11:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877728+0000", 2024-09-13T10:11:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1129999999999997e-06 2024-09-13T10:11:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877737+0000", 2024-09-13T10:11:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3969999999999994e-06 2024-09-13T10:11:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877743+0000", 2024-09-13T10:11:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7950000000000002e-06 2024-09-13T10:11:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877826+0000", 2024-09-13T10:11:39.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2213999999999996e-05 2024-09-13T10:11:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:11:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877831+0000", 2024-09-13T10:11:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.604e-06 2024-09-13T10:11:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699295+0000", 2024-09-13T10:11:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82146349299999999 2024-09-13T10:11:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699309+0000", 2024-09-13T10:11:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4058e-05 2024-09-13T10:11:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.051 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.052 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.052 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:39.054 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-13T10:11:39.055 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:39.047+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:39.059 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T10:11:39.118 INFO:tasks.rados.rados.0.smithi195.stdout:1378: finishing write tid 1 to smithi19530714-32 2024-09-13T10:11:39.119 INFO:tasks.rados.rados.0.smithi195.stdout:1379: expect (ObjNum 264 snap 0 seq_num 264) 2024-09-13T10:11:39.154 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:39.150+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:39.157 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:39.214 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1584909 2024-09-13T10:11:39.231 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:39.231 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:11:39.232 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:39.235 INFO:tasks.rados.rados.0.smithi195.stdout:1378: finishing write tid 2 to smithi19530714-32 2024-09-13T10:11:39.235 INFO:tasks.rados.rados.0.smithi195.stdout:1378: finishing write tid 3 to smithi19530714-32 2024-09-13T10:11:39.235 INFO:tasks.rados.rados.0.smithi195.stdout:1378: finishing write tid 4 to smithi19530714-32 2024-09-13T10:11:39.235 INFO:tasks.rados.rados.0.smithi195.stdout:1378: oid 32 updating version 0 to 67 2024-09-13T10:11:39.235 INFO:tasks.rados.rados.0.smithi195.stdout:1378: oid 32 version 67 is already newer than 66 2024-09-13T10:11:39.235 INFO:tasks.rados.rados.0.smithi195.stdout:1378: oid 32 version 67 is already newer than 65 2024-09-13T10:11:39.235 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 67 (ObjNum 270 snap 0 seq_num 270) dirty exists 2024-09-13T10:11:39.235 INFO:tasks.rados.rados.0.smithi195.stdout:1378: left oid 32 (ObjNum 270 snap 0 seq_num 270) 2024-09-13T10:11:39.235 INFO:tasks.rados.rados.0.smithi195.stdout:1380: finishing set_chunk to oid 28 2024-09-13T10:11:39.235 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 28 set_chunk returned error code -95 offset: 870912 length: 1417728 2024-09-13T10:11:39.236 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 0 (ObjNum 180 snap 0 seq_num 180) dirty exists 2024-09-13T10:11:39.236 INFO:tasks.rados.rados.0.smithi195.stdout:1381: finishing set_chunk to oid 20 2024-09-13T10:11:39.236 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 20 set_chunk returned error code -95 offset: 585728 length: 2168832 2024-09-13T10:11:39.236 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 0 (ObjNum 232 snap 0 seq_num 232) dirty exists 2024-09-13T10:11:39.236 INFO:tasks.rados.rados.0.smithi195.stdout:1382: finishing set_chunk to oid 19 2024-09-13T10:11:39.236 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 19 set_chunk returned error code -95 offset: 536064 length: 1571840 2024-09-13T10:11:39.236 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 0 (ObjNum 160 snap 0 seq_num 160) dirty exists 2024-09-13T10:11:39.236 INFO:tasks.rados.rados.0.smithi195.stdout:1383: finishing write tid 1 to smithi19530714-7 2024-09-13T10:11:39.236 INFO:tasks.rados.rados.0.smithi195.stdout:1383: finishing write tid 2 to smithi19530714-7 2024-09-13T10:11:39.236 INFO:tasks.rados.rados.0.smithi195.stdout:1383: finishing write tid 3 to smithi19530714-7 2024-09-13T10:11:39.236 INFO:tasks.rados.rados.0.smithi195.stdout:1383: finishing write tid 4 to smithi19530714-7 2024-09-13T10:11:39.236 INFO:tasks.rados.rados.0.smithi195.stdout:1383: oid 7 updating version 0 to 94 2024-09-13T10:11:39.237 INFO:tasks.rados.rados.0.smithi195.stdout:1383: oid 7 updating version 94 to 96 2024-09-13T10:11:39.237 INFO:tasks.rados.rados.0.smithi195.stdout:1383: oid 7 version 96 is already newer than 95 2024-09-13T10:11:39.237 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 96 (ObjNum 271 snap 0 seq_num 271) dirty exists 2024-09-13T10:11:39.237 INFO:tasks.rados.rados.0.smithi195.stdout:1383: left oid 7 (ObjNum 271 snap 0 seq_num 271) 2024-09-13T10:11:39.237 INFO:tasks.rados.rados.0.smithi195.stdout:1384: finishing write tid 1 to smithi19530714-31 2024-09-13T10:11:39.237 INFO:tasks.rados.rados.0.smithi195.stdout:1384: finishing write tid 2 to smithi19530714-31 2024-09-13T10:11:39.237 INFO:tasks.rados.rados.0.smithi195.stdout:1384: finishing write tid 3 to smithi19530714-31 2024-09-13T10:11:39.237 INFO:tasks.rados.rados.0.smithi195.stdout:1384: finishing write tid 4 to smithi19530714-31 2024-09-13T10:11:39.237 INFO:tasks.rados.rados.0.smithi195.stdout:1384: oid 31 updating version 0 to 81 2024-09-13T10:11:39.237 INFO:tasks.rados.rados.0.smithi195.stdout:1384: oid 31 version 81 is already newer than 80 2024-09-13T10:11:39.237 INFO:tasks.rados.rados.0.smithi195.stdout:1384: oid 31 version 81 is already newer than 79 2024-09-13T10:11:39.237 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 81 (ObjNum 272 snap 0 seq_num 272) dirty exists 2024-09-13T10:11:39.238 INFO:tasks.rados.rados.0.smithi195.stdout:1384: left oid 31 (ObjNum 272 snap 0 seq_num 272) 2024-09-13T10:11:39.238 INFO:tasks.rados.rados.0.smithi195.stdout:1385: finishing write tid 1 to smithi19530714-26 2024-09-13T10:11:39.238 INFO:tasks.rados.rados.0.smithi195.stdout:1385: finishing write tid 2 to smithi19530714-26 2024-09-13T10:11:39.238 INFO:tasks.rados.rados.0.smithi195.stdout:1385: finishing write tid 3 to smithi19530714-26 2024-09-13T10:11:39.238 INFO:tasks.rados.rados.0.smithi195.stdout:1385: finishing write tid 4 to smithi19530714-26 2024-09-13T10:11:39.238 INFO:tasks.rados.rados.0.smithi195.stdout:1385: oid 26 updating version 0 to 38 2024-09-13T10:11:39.238 INFO:tasks.rados.rados.0.smithi195.stdout:1385: oid 26 updating version 38 to 39 2024-09-13T10:11:39.238 INFO:tasks.rados.rados.0.smithi195.stdout:1385: oid 26 version 39 is already newer than 37 2024-09-13T10:11:39.238 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 39 (ObjNum 273 snap 0 seq_num 273) dirty exists 2024-09-13T10:11:39.238 INFO:tasks.rados.rados.0.smithi195.stdout:1385: left oid 26 (ObjNum 273 snap 0 seq_num 273) 2024-09-13T10:11:39.238 INFO:tasks.rados.rados.0.smithi195.stdout:1386: expect (ObjNum 267 snap 0 seq_num 267) 2024-09-13T10:11:39.238 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:39.247 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-13T10:11:39.294 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:39.291+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:39.297 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:39.356 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:39.354+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:39.359 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:39.362 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:39.362 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:39.362 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:11:39.362 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:11:39.362 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:39.376 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-13T10:11:39.421 INFO:tasks.rados.rados.0.smithi195.stdout:1387: seq_num 274 ranges {605118=406934,1617125=706417} 2024-09-13T10:11:39.437 INFO:tasks.rados.rados.0.smithi195.stdout:1387: writing smithi19530714-50 from 605118 to 1012052 tid 1 2024-09-13T10:11:39.457 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:39.454+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:39.460 INFO:tasks.rados.rados.0.smithi195.stdout:1387: writing smithi19530714-50 from 1617125 to 2323542 tid 2 2024-09-13T10:11:39.460 INFO:tasks.rados.rados.0.smithi195.stdout:1373: done (14 left) 2024-09-13T10:11:39.460 INFO:tasks.rados.rados.0.smithi195.stdout:1374: done (13 left) 2024-09-13T10:11:39.460 INFO:tasks.rados.rados.0.smithi195.stdout:1375: done (12 left) 2024-09-13T10:11:39.460 INFO:tasks.rados.rados.0.smithi195.stdout:1376: done (11 left) 2024-09-13T10:11:39.460 INFO:tasks.rados.rados.0.smithi195.stdout:1377: done (10 left) 2024-09-13T10:11:39.460 INFO:tasks.rados.rados.0.smithi195.stdout:1378: done (9 left) 2024-09-13T10:11:39.460 INFO:tasks.rados.rados.0.smithi195.stdout:1379: done (8 left) 2024-09-13T10:11:39.461 INFO:tasks.rados.rados.0.smithi195.stdout:1380: done (7 left) 2024-09-13T10:11:39.461 INFO:tasks.rados.rados.0.smithi195.stdout:1381: done (6 left) 2024-09-13T10:11:39.461 INFO:tasks.rados.rados.0.smithi195.stdout:1382: done (5 left) 2024-09-13T10:11:39.461 INFO:tasks.rados.rados.0.smithi195.stdout:1383: done (4 left) 2024-09-13T10:11:39.461 INFO:tasks.rados.rados.0.smithi195.stdout:1384: done (3 left) 2024-09-13T10:11:39.461 INFO:tasks.rados.rados.0.smithi195.stdout:1385: done (2 left) 2024-09-13T10:11:39.461 INFO:tasks.rados.rados.0.smithi195.stdout:1386: done (1 left) 2024-09-13T10:11:39.461 INFO:tasks.rados.rados.0.smithi195.stdout:1388: read oid 26 snap -1 2024-09-13T10:11:39.461 INFO:tasks.rados.rados.0.smithi195.stdout:1388: expect (ObjNum 273 snap 0 seq_num 273) 2024-09-13T10:11:39.461 INFO:tasks.rados.rados.0.smithi195.stdout:1389: read oid 41 snap -1 2024-09-13T10:11:39.461 INFO:tasks.rados.rados.0.smithi195.stdout:1389: expect (ObjNum 236 snap 0 seq_num 236) 2024-09-13T10:11:39.461 INFO:tasks.rados.rados.0.smithi195.stdout:1390: read oid 43 snap -1 2024-09-13T10:11:39.461 INFO:tasks.rados.rados.0.smithi195.stdout:1390: expect (ObjNum 263 snap 0 seq_num 263) 2024-09-13T10:11:39.462 INFO:tasks.rados.rados.0.smithi195.stdout:1391: read oid 47 snap -1 2024-09-13T10:11:39.462 INFO:tasks.rados.rados.0.smithi195.stdout:1391: expect (ObjNum 260 snap 0 seq_num 260) 2024-09-13T10:11:39.462 INFO:tasks.rados.rados.0.smithi195.stdout:1392: read oid 27 snap -1 2024-09-13T10:11:39.462 INFO:tasks.rados.rados.0.smithi195.stdout:1392: expect (ObjNum 221 snap 0 seq_num 221) 2024-09-13T10:11:39.462 INFO:tasks.rados.rados.0.smithi195.stdout:1393: write oid 16 current snap is 0 2024-09-13T10:11:39.462 INFO:tasks.rados.rados.0.smithi195.stdout:1393: seq_num 275 ranges {621925=482183,1559254=566428} 2024-09-13T10:11:39.480 INFO:tasks.rados.rados.0.smithi195.stdout:1393: writing smithi19530714-16 from 621925 to 1104108 tid 1 2024-09-13T10:11:39.482 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:11:39.492 INFO:tasks.rados.rados.0.smithi195.stdout:1393: writing smithi19530714-16 from 1559254 to 2125682 tid 2 2024-09-13T10:11:39.492 INFO:tasks.rados.rados.0.smithi195.stdout:1394: write (excl) oid 7 current snap is 0 2024-09-13T10:11:39.492 INFO:tasks.rados.rados.0.smithi195.stdout:1394: seq_num 276 ranges {638732=557432,1617736=710086} 2024-09-13T10:11:39.502 INFO:tasks.rados.rados.0.smithi195.stdout:1394: writing smithi19530714-7 from 638732 to 1196164 tid 1 2024-09-13T10:11:39.512 INFO:tasks.rados.rados.0.smithi195.stdout:1394: writing smithi19530714-7 from 1617736 to 2327822 tid 2 2024-09-13T10:11:39.512 INFO:tasks.rados.rados.0.smithi195.stdout:1395: read oid 15 snap -1 2024-09-13T10:11:39.512 INFO:tasks.rados.rados.0.smithi195.stdout:1395: expect (ObjNum 261 snap 0 seq_num 261) 2024-09-13T10:11:39.512 INFO:tasks.rados.rados.0.smithi195.stdout:1396: write oid 18 current snap is 0 2024-09-13T10:11:39.512 INFO:tasks.rados.rados.0.smithi195.stdout:1396: seq_num 277 ranges {655539=632681,2076218=570097} 2024-09-13T10:11:39.521 INFO:tasks.rados.rados.0.smithi195.stdout:1396: writing smithi19530714-18 from 655539 to 1288220 tid 1 2024-09-13T10:11:39.529 INFO:tasks.rados.rados.0.smithi195.stdout:1396: writing smithi19530714-18 from 2076218 to 2646315 tid 2 2024-09-13T10:11:39.529 INFO:tasks.rados.rados.0.smithi195.stdout:1397: write oid 17 current snap is 0 2024-09-13T10:11:39.530 INFO:tasks.rados.rados.0.smithi195.stdout:1397: seq_num 278 ranges {672346=707930,2134700=537646} 2024-09-13T10:11:39.538 INFO:tasks.rados.rados.0.smithi195.stdout:1397: writing smithi19530714-17 from 672346 to 1380276 tid 1 2024-09-13T10:11:39.545 INFO:tasks.rados.rados.0.smithi195.stdout:1397: writing smithi19530714-17 from 2134700 to 2672346 tid 2 2024-09-13T10:11:39.545 INFO:tasks.rados.rados.0.smithi195.stdout:1398: read oid 24 snap -1 2024-09-13T10:11:39.545 INFO:tasks.rados.rados.0.smithi195.stdout:1398: expect (ObjNum 262 snap 0 seq_num 262) 2024-09-13T10:11:39.545 INFO:tasks.rados.rados.0.smithi195.stdout:1399: read oid 11 snap -1 2024-09-13T10:11:39.545 INFO:tasks.rados.rados.0.smithi195.stdout:1399: expect (ObjNum 267 snap 0 seq_num 267) 2024-09-13T10:11:39.545 INFO:tasks.rados.rados.0.smithi195.stdout:1400: set_chunk oid 28 target oid 2024-09-13T10:11:39.553 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:39.553 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:11:39.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:11:39.553 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:11:39.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.553 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:39.553 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:11:39.553 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.169328758000006, 2024-09-13T10:11:39.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034624456999999997, 2024-09-13T10:11:39.554 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.554 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.554 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.554 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.554 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:39.554 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:39.554 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 70 2024-09-13T10:11:39.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.554 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:11:39.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:11:39.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381723+0000", 2024-09-13T10:11:39.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.540000000000001e-07 2024-09-13T10:11:39.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381731+0000", 2024-09-13T10:11:39.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.486e-06 2024-09-13T10:11:39.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381742+0000", 2024-09-13T10:11:39.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1269e-05 2024-09-13T10:11:39.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381750+0000", 2024-09-13T10:11:39.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8830000000000006e-06 2024-09-13T10:11:39.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401255+0000", 2024-09-13T10:11:39.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019505110999999999 2024-09-13T10:11:39.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401388+0000", 2024-09-13T10:11:39.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013253200000000001 2024-09-13T10:11:39.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:39.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401609+0000", 2024-09-13T10:11:39.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000220981 2024-09-13T10:11:39.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407538+0000", 2024-09-13T10:11:39.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059291220000000002 2024-09-13T10:11:39.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407626+0000", 2024-09-13T10:11:39.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8361000000000003e-05 2024-09-13T10:11:39.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:39.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416154+0000", 2024-09-13T10:11:39.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0085281000000000003 2024-09-13T10:11:39.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416234+0000", 2024-09-13T10:11:39.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9594999999999999e-05 2024-09-13T10:11:39.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416347+0000", 2024-09-13T10:11:39.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113063 2024-09-13T10:11:39.560 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.560 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.560 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.560 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:71 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:39.560 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:11:39.560 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.169288175999995, 2024-09-13T10:11:39.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035004, 2024-09-13T10:11:39.560 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.561 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.561 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.561 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.561 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:39.561 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:39.561 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 71 2024-09-13T10:11:39.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.561 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:11:39.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:11:39.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381764+0000", 2024-09-13T10:11:39.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4600000000000004e-07 2024-09-13T10:11:39.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381769+0000", 2024-09-13T10:11:39.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.48e-06 2024-09-13T10:11:39.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381780+0000", 2024-09-13T10:11:39.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0893000000000001e-05 2024-09-13T10:11:39.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381788+0000", 2024-09-13T10:11:39.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6389999999999992e-06 2024-09-13T10:11:39.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407896+0000", 2024-09-13T10:11:39.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.026107807 2024-09-13T10:11:39.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.408029+0000", 2024-09-13T10:11:39.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013383899999999999 2024-09-13T10:11:39.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:39.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.408250+0000", 2024-09-13T10:11:39.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022105399999999999 2024-09-13T10:11:39.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416130+0000", 2024-09-13T10:11:39.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0078798500000000007 2024-09-13T10:11:39.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416488+0000", 2024-09-13T10:11:39.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00035819299999999998 2024-09-13T10:11:39.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:39.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416595+0000", 2024-09-13T10:11:39.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001064 2024-09-13T10:11:39.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416662+0000", 2024-09-13T10:11:39.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6775999999999999e-05 2024-09-13T10:11:39.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416767+0000", 2024-09-13T10:11:39.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000105423 2024-09-13T10:11:39.566 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.567 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.567 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.567 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:72 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:39.567 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:11:39.567 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.169251141999993, 2024-09-13T10:11:39.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039399762999999997, 2024-09-13T10:11:39.567 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.567 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.567 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.567 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.568 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:39.568 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:39.568 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 72 2024-09-13T10:11:39.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.568 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:11:39.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:11:39.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381801+0000", 2024-09-13T10:11:39.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7700000000000004e-07 2024-09-13T10:11:39.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381898+0000", 2024-09-13T10:11:39.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7066999999999997e-05 2024-09-13T10:11:39.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381908+0000", 2024-09-13T10:11:39.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0336000000000001e-05 2024-09-13T10:11:39.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381917+0000", 2024-09-13T10:11:39.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5169999999999999e-06 2024-09-13T10:11:39.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.411900+0000", 2024-09-13T10:11:39.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029983381999999999 2024-09-13T10:11:39.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.412028+0000", 2024-09-13T10:11:39.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127916 2024-09-13T10:11:39.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:39.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.412248+0000", 2024-09-13T10:11:39.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022014699999999999 2024-09-13T10:11:39.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416576+0000", 2024-09-13T10:11:39.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004327549 2024-09-13T10:11:39.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416860+0000", 2024-09-13T10:11:39.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028391000000000001 2024-09-13T10:11:39.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:39.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421014+0000", 2024-09-13T10:11:39.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041539330000000003 2024-09-13T10:11:39.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421093+0000", 2024-09-13T10:11:39.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9097999999999994e-05 2024-09-13T10:11:39.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421200+0000", 2024-09-13T10:11:39.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000107231 2024-09-13T10:11:39.573 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.574 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.574 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.574 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:73 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:39.574 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:11:39.574 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.16911743, 2024-09-13T10:11:39.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044034528000000003, 2024-09-13T10:11:39.574 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.574 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.574 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.574 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.574 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:39.575 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:39.575 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 73 2024-09-13T10:11:39.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.575 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:11:39.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:11:39.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381935+0000", 2024-09-13T10:11:39.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0999999999999997e-07 2024-09-13T10:11:39.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.576 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.576 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381940+0000", 2024-09-13T10:11:39.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7030000000000003e-06 2024-09-13T10:11:39.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381949+0000", 2024-09-13T10:11:39.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5359999999999992e-06 2024-09-13T10:11:39.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381961+0000", 2024-09-13T10:11:39.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2539e-05 2024-09-13T10:11:39.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416942+0000", 2024-09-13T10:11:39.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034980270000000001 2024-09-13T10:11:39.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.417074+0000", 2024-09-13T10:11:39.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013246199999999999 2024-09-13T10:11:39.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:39.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.417298+0000", 2024-09-13T10:11:39.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000223823 2024-09-13T10:11:39.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425459+0000", 2024-09-13T10:11:39.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081613750000000002 2024-09-13T10:11:39.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425575+0000", 2024-09-13T10:11:39.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115513 2024-09-13T10:11:39.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:39.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425774+0000", 2024-09-13T10:11:39.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019959399999999999 2024-09-13T10:11:39.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425854+0000", 2024-09-13T10:11:39.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9532999999999999e-05 2024-09-13T10:11:39.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425968+0000", 2024-09-13T10:11:39.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011437 2024-09-13T10:11:39.580 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.580 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.581 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.581 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:74 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:39.581 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:11:39.581 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.169074555999998, 2024-09-13T10:11:39.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052701496, 2024-09-13T10:11:39.581 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.581 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.581 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:39.582 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:39.582 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 74 2024-09-13T10:11:39.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.582 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:11:39.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:11:39.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:11:39.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7999999999999995e-07 2024-09-13T10:11:39.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381983+0000", 2024-09-13T10:11:39.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4430000000000002e-06 2024-09-13T10:11:39.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381991+0000", 2024-09-13T10:11:39.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8960000000000003e-06 2024-09-13T10:11:39.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381998+0000", 2024-09-13T10:11:39.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7729999999999999e-06 2024-09-13T10:11:39.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421291+0000", 2024-09-13T10:11:39.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039292753 2024-09-13T10:11:39.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421424+0000", 2024-09-13T10:11:39.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013291900000000001 2024-09-13T10:11:39.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:39.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421646+0000", 2024-09-13T10:11:39.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000221843 2024-09-13T10:11:39.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425667+0000", 2024-09-13T10:11:39.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040206779999999998 2024-09-13T10:11:39.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425755+0000", 2024-09-13T10:11:39.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8848e-05 2024-09-13T10:11:39.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:39.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434472+0000", 2024-09-13T10:11:39.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0087164879999999997 2024-09-13T10:11:39.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434544+0000", 2024-09-13T10:11:39.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2273000000000004e-05 2024-09-13T10:11:39.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434678+0000", 2024-09-13T10:11:39.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013390200000000001 2024-09-13T10:11:39.587 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.587 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.587 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.588 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:75 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:39.588 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:11:39.588 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.169037691, 2024-09-13T10:11:39.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052864585999999998, 2024-09-13T10:11:39.588 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.588 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.588 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.588 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.588 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:39.588 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:39.588 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 75 2024-09-13T10:11:39.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.589 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:11:39.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:11:39.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:11:39.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8299999999999996e-07 2024-09-13T10:11:39.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382120+0000", 2024-09-13T10:11:39.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010627400000000001 2024-09-13T10:11:39.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382131+0000", 2024-09-13T10:11:39.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0796e-05 2024-09-13T10:11:39.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382140+0000", 2024-09-13T10:11:39.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5560000000000008e-06 2024-09-13T10:11:39.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426014+0000", 2024-09-13T10:11:39.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.043873843000000003 2024-09-13T10:11:39.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426205+0000", 2024-09-13T10:11:39.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019154600000000001 2024-09-13T10:11:39.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:39.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426429+0000", 2024-09-13T10:11:39.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022344699999999999 2024-09-13T10:11:39.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.592 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.592 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.592 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434352+0000", 2024-09-13T10:11:39.592 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007923355 2024-09-13T10:11:39.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434452+0000", 2024-09-13T10:11:39.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9961000000000001e-05 2024-09-13T10:11:39.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:39.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434700+0000", 2024-09-13T10:11:39.593 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024790899999999999 2024-09-13T10:11:39.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.593 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.593 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.593 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434774+0000", 2024-09-13T10:11:39.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4457999999999998e-05 2024-09-13T10:11:39.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.594 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.594 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434878+0000", 2024-09-13T10:11:39.594 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010375800000000001 2024-09-13T10:11:39.594 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.594 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.594 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.594 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:76 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:39.594 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:11:39.594 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.168870710999997, 2024-09-13T10:11:39.595 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053104138000000002, 2024-09-13T10:11:39.595 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.595 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.595 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.595 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.595 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:39.595 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:39.595 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 76 2024-09-13T10:11:39.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.595 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.595 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.595 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.595 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:11:39.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:11:39.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.596 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.596 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:11:39.596 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4600000000000003e-07 2024-09-13T10:11:39.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.596 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382188+0000", 2024-09-13T10:11:39.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4869999999999999e-06 2024-09-13T10:11:39.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.597 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382197+0000", 2024-09-13T10:11:39.597 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5519999999999993e-06 2024-09-13T10:11:39.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.597 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382207+0000", 2024-09-13T10:11:39.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7240000000000004e-06 2024-09-13T10:11:39.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430285+0000", 2024-09-13T10:11:39.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.048078093000000002 2024-09-13T10:11:39.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.598 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.598 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.598 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430429+0000", 2024-09-13T10:11:39.598 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000143981 2024-09-13T10:11:39.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:39.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430717+0000", 2024-09-13T10:11:39.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028802 2024-09-13T10:11:39.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435015+0000", 2024-09-13T10:11:39.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042973150000000003 2024-09-13T10:11:39.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.599 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.599 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435083+0000", 2024-09-13T10:11:39.599 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8448999999999995e-05 2024-09-13T10:11:39.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:39.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435109+0000", 2024-09-13T10:11:39.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6064999999999999e-05 2024-09-13T10:11:39.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.600 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435176+0000", 2024-09-13T10:11:39.600 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7362999999999999e-05 2024-09-13T10:11:39.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.600 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.601 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435285+0000", 2024-09-13T10:11:39.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108243 2024-09-13T10:11:39.601 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.601 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.601 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.601 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.601 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:96 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:39.601 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:11:39.601 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.027239281000007, 2024-09-13T10:11:39.601 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032701419000000002, 2024-09-13T10:11:39.601 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.601 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.602 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.602 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.602 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:39.602 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:39.602 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 96 2024-09-13T10:11:39.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.602 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.602 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:11:39.602 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.602 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.602 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:11:39.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523813+0000", 2024-09-13T10:11:39.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.364e-06 2024-09-13T10:11:39.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.603 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.603 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.603 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523913+0000", 2024-09-13T10:11:39.603 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9827000000000005e-05 2024-09-13T10:11:39.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523927+0000", 2024-09-13T10:11:39.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4029000000000001e-05 2024-09-13T10:11:39.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.604 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.604 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523936+0000", 2024-09-13T10:11:39.604 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1030000000000008e-06 2024-09-13T10:11:39.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.544707+0000", 2024-09-13T10:11:39.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0207706 2024-09-13T10:11:39.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.544834+0000", 2024-09-13T10:11:39.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012683200000000001 2024-09-13T10:11:39.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.605 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:39.605 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.545057+0000", 2024-09-13T10:11:39.605 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022338100000000001 2024-09-13T10:11:39.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549279+0000", 2024-09-13T10:11:39.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042223080000000001 2024-09-13T10:11:39.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.606 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549556+0000", 2024-09-13T10:11:39.606 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000276307 2024-09-13T10:11:39.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.606 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:39.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556364+0000", 2024-09-13T10:11:39.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0068082510000000004 2024-09-13T10:11:39.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556445+0000", 2024-09-13T10:11:39.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1081000000000005e-05 2024-09-13T10:11:39.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.607 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.607 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.607 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556513+0000", 2024-09-13T10:11:39.607 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8336000000000004e-05 2024-09-13T10:11:39.607 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.608 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.608 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.608 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.608 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.608 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:97 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:39.608 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:11:39.608 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.027096396000005, 2024-09-13T10:11:39.608 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035410807000000002, 2024-09-13T10:11:39.608 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.608 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.608 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.608 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.608 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:39.608 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:39.609 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 97 2024-09-13T10:11:39.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.609 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:11:39.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.609 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.609 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.609 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:11:39.609 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523956+0000", 2024-09-13T10:11:39.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8599999999999997e-07 2024-09-13T10:11:39.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523962+0000", 2024-09-13T10:11:39.610 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1240000000000002e-06 2024-09-13T10:11:39.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.610 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.610 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523971+0000", 2024-09-13T10:11:39.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7320000000000004e-06 2024-09-13T10:11:39.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523978+0000", 2024-09-13T10:11:39.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0899999999999999e-06 2024-09-13T10:11:39.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.611 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549637+0000", 2024-09-13T10:11:39.611 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025658876000000001 2024-09-13T10:11:39.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.611 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.611 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549757+0000", 2024-09-13T10:11:39.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000120837 2024-09-13T10:11:39.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:39.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549971+0000", 2024-09-13T10:11:39.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021356000000000001 2024-09-13T10:11:39.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.612 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.612 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.612 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556335+0000", 2024-09-13T10:11:39.612 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0063639329999999996 2024-09-13T10:11:39.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556725+0000", 2024-09-13T10:11:39.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000390221 2024-09-13T10:11:39.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:39.613 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559219+0000", 2024-09-13T10:11:39.613 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0024943500000000002 2024-09-13T10:11:39.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.613 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.613 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559296+0000", 2024-09-13T10:11:39.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7065999999999997e-05 2024-09-13T10:11:39.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.614 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.614 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559366+0000", 2024-09-13T10:11:39.614 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9132000000000002e-05 2024-09-13T10:11:39.614 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.614 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.614 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.614 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.614 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:98 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:39.614 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:11:39.615 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.027061787999997, 2024-09-13T10:11:39.615 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035839604999999997, 2024-09-13T10:11:39.615 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.615 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.615 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.615 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.615 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:39.615 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:39.615 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 98 2024-09-13T10:11:39.615 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.615 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.615 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.615 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.615 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:11:39.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:11:39.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523990+0000", 2024-09-13T10:11:39.616 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6300000000000005e-07 2024-09-13T10:11:39.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.616 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.616 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523995+0000", 2024-09-13T10:11:39.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2669999999999997e-06 2024-09-13T10:11:39.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.617 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.617 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524007+0000", 2024-09-13T10:11:39.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2002e-05 2024-09-13T10:11:39.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.617 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.617 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524015+0000", 2024-09-13T10:11:39.617 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4869999999999996e-06 2024-09-13T10:11:39.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.553854+0000", 2024-09-13T10:11:39.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029839633000000001 2024-09-13T10:11:39.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.553985+0000", 2024-09-13T10:11:39.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013032899999999999 2024-09-13T10:11:39.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.618 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:39.618 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.554212+0000", 2024-09-13T10:11:39.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022723400000000001 2024-09-13T10:11:39.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559523+0000", 2024-09-13T10:11:39.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053113509999999997 2024-09-13T10:11:39.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559662+0000", 2024-09-13T10:11:39.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013833500000000001 2024-09-13T10:11:39.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:39.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559698+0000", 2024-09-13T10:11:39.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5982000000000002e-05 2024-09-13T10:11:39.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559764+0000", 2024-09-13T10:11:39.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6019000000000006e-05 2024-09-13T10:11:39.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559829+0000", 2024-09-13T10:11:39.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5302999999999995e-05 2024-09-13T10:11:39.620 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.620 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.620 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.621 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:99 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:39.621 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:11:39.621 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.027024093999998, 2024-09-13T10:11:39.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.037637084000000001, 2024-09-13T10:11:39.621 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.621 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.621 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:39.621 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:39.622 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 99 2024-09-13T10:11:39.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.622 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:11:39.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:11:39.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524028+0000", 2024-09-13T10:11:39.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7999999999999995e-07 2024-09-13T10:11:39.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524109+0000", 2024-09-13T10:11:39.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0834999999999994e-05 2024-09-13T10:11:39.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524119+0000", 2024-09-13T10:11:39.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0149999999999999e-05 2024-09-13T10:11:39.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524127+0000", 2024-09-13T10:11:39.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8660000000000006e-06 2024-09-13T10:11:39.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556868+0000", 2024-09-13T10:11:39.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.03274146 2024-09-13T10:11:39.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.557001+0000", 2024-09-13T10:11:39.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000133249 2024-09-13T10:11:39.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:39.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.557224+0000", 2024-09-13T10:11:39.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000222914 2024-09-13T10:11:39.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561387+0000", 2024-09-13T10:11:39.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041627019999999999 2024-09-13T10:11:39.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:39.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561419+0000", 2024-09-13T10:11:39.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1924999999999997e-05 2024-09-13T10:11:39.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561476+0000", 2024-09-13T10:11:39.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7346999999999998e-05 2024-09-13T10:11:39.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561552+0000", 2024-09-13T10:11:39.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5847999999999996e-05 2024-09-13T10:11:39.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561664+0000", 2024-09-13T10:11:39.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112108 2024-09-13T10:11:39.627 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.627 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.627 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.628 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:100 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:39.628 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:11:39.628 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.026847943999996, 2024-09-13T10:11:39.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.038825510000000001, 2024-09-13T10:11:39.628 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.628 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.628 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.628 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.628 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:39.628 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:39.628 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 100 2024-09-13T10:11:39.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.628 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:11:39.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:11:39.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524204+0000", 2024-09-13T10:11:39.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.54e-07 2024-09-13T10:11:39.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524211+0000", 2024-09-13T10:11:39.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6919999999999996e-06 2024-09-13T10:11:39.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524221+0000", 2024-09-13T10:11:39.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0223e-05 2024-09-13T10:11:39.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524229+0000", 2024-09-13T10:11:39.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9859999999999994e-06 2024-09-13T10:11:39.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559920+0000", 2024-09-13T10:11:39.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035690854000000001 2024-09-13T10:11:39.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.560052+0000", 2024-09-13T10:11:39.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013199899999999999 2024-09-13T10:11:39.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:39.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.560356+0000", 2024-09-13T10:11:39.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030450200000000001 2024-09-13T10:11:39.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:39.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562115+0000", 2024-09-13T10:11:39.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0017586430000000001 2024-09-13T10:11:39.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562512+0000", 2024-09-13T10:11:39.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00039728099999999998 2024-09-13T10:11:39.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562828+0000", 2024-09-13T10:11:39.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00031528799999999999 2024-09-13T10:11:39.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562913+0000", 2024-09-13T10:11:39.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5628000000000001e-05 2024-09-13T10:11:39.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.563029+0000", 2024-09-13T10:11:39.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011556 2024-09-13T10:11:39.634 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.634 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.634 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.634 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:122 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:39.634 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:11:39.634 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.910163463000004, 2024-09-13T10:11:39.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.027900708, 2024-09-13T10:11:39.635 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.635 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.635 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.635 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.635 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:39.635 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:39.635 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 122 2024-09-13T10:11:39.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.635 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:11:39.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:11:39.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640889+0000", 2024-09-13T10:11:39.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4139999999999999e-06 2024-09-13T10:11:39.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640900+0000", 2024-09-13T10:11:39.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0559e-05 2024-09-13T10:11:39.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640916+0000", 2024-09-13T10:11:39.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6399e-05 2024-09-13T10:11:39.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640931+0000", 2024-09-13T10:11:39.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4659e-05 2024-09-13T10:11:39.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663602+0000", 2024-09-13T10:11:39.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.022671674999999999 2024-09-13T10:11:39.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663734+0000", 2024-09-13T10:11:39.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013189899999999999 2024-09-13T10:11:39.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:39.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663946+0000", 2024-09-13T10:11:39.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021198699999999999 2024-09-13T10:11:39.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668523+0000", 2024-09-13T10:11:39.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045765789999999999 2024-09-13T10:11:39.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668638+0000", 2024-09-13T10:11:39.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504 2024-09-13T10:11:39.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:39.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668657+0000", 2024-09-13T10:11:39.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9298999999999999e-05 2024-09-13T10:11:39.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668722+0000", 2024-09-13T10:11:39.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5126e-05 2024-09-13T10:11:39.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668788+0000", 2024-09-13T10:11:39.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6072e-05 2024-09-13T10:11:39.640 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.641 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.641 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.641 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:123 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:39.641 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:11:39.641 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.910097586000006, 2024-09-13T10:11:39.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028273795000000001, 2024-09-13T10:11:39.641 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.641 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.641 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.641 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:39.642 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:39.642 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 123 2024-09-13T10:11:39.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.642 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:11:39.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:11:39.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640955+0000", 2024-09-13T10:11:39.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.356e-06 2024-09-13T10:11:39.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641083+0000", 2024-09-13T10:11:39.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012763600000000001 2024-09-13T10:11:39.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641102+0000", 2024-09-13T10:11:39.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9055e-05 2024-09-13T10:11:39.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641117+0000", 2024-09-13T10:11:39.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5665999999999999e-05 2024-09-13T10:11:39.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.665747+0000", 2024-09-13T10:11:39.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.024629796999999998 2024-09-13T10:11:39.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.665876+0000", 2024-09-13T10:11:39.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012925600000000001 2024-09-13T10:11:39.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:39.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.666105+0000", 2024-09-13T10:11:39.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000228543 2024-09-13T10:11:39.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668924+0000", 2024-09-13T10:11:39.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028194309999999998 2024-09-13T10:11:39.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669070+0000", 2024-09-13T10:11:39.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014560199999999999 2024-09-13T10:11:39.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:39.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669104+0000", 2024-09-13T10:11:39.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3654e-05 2024-09-13T10:11:39.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669173+0000", 2024-09-13T10:11:39.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9819000000000004e-05 2024-09-13T10:11:39.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669227+0000", 2024-09-13T10:11:39.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3980000000000002e-05 2024-09-13T10:11:39.647 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.647 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.647 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.648 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:124 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:39.648 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:11:39.648 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.909904545000003, 2024-09-13T10:11:39.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030852801999999999, 2024-09-13T10:11:39.648 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.648 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.648 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.648 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.648 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:39.648 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:39.648 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 124 2024-09-13T10:11:39.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.648 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:11:39.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:11:39.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641148+0000", 2024-09-13T10:11:39.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4529999999999999e-06 2024-09-13T10:11:39.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.650 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641158+0000", 2024-09-13T10:11:39.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0003999999999999e-05 2024-09-13T10:11:39.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.650 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641199+0000", 2024-09-13T10:11:39.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0426999999999998e-05 2024-09-13T10:11:39.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641214+0000", 2024-09-13T10:11:39.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5716e-05 2024-09-13T10:11:39.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669267+0000", 2024-09-13T10:11:39.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028053112000000002 2024-09-13T10:11:39.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669394+0000", 2024-09-13T10:11:39.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012656000000000001 2024-09-13T10:11:39.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:39.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669614+0000", 2024-09-13T10:11:39.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021970400000000001 2024-09-13T10:11:39.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671267+0000", 2024-09-13T10:11:39.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001652976 2024-09-13T10:11:39.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671383+0000", 2024-09-13T10:11:39.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000116113 2024-09-13T10:11:39.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:39.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671828+0000", 2024-09-13T10:11:39.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000444871 2024-09-13T10:11:39.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671935+0000", 2024-09-13T10:11:39.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010703 2024-09-13T10:11:39.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671999+0000", 2024-09-13T10:11:39.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4836000000000001e-05 2024-09-13T10:11:39.654 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.654 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.654 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.654 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:201 3.3 e38/30 3:d4fb61be:::smithi19530714-2 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:39.654 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:11:39.654 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 48.537453651999996, 2024-09-13T10:11:39.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032127035999999998, 2024-09-13T10:11:39.655 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.655 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.655 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.655 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:11:39.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:11:39.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:11:39.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2600000000000001e-07 2024-09-13T10:11:39.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030449+0000", 2024-09-13T10:11:39.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016850477999999999 2024-09-13T10:11:39.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030499+0000", 2024-09-13T10:11:39.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0212e-05 2024-09-13T10:11:39.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030524+0000", 2024-09-13T10:11:39.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5222e-05 2024-09-13T10:11:39.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030739+0000", 2024-09-13T10:11:39.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000214396 2024-09-13T10:11:39.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030818+0000", 2024-09-13T10:11:39.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8941000000000002e-05 2024-09-13T10:11:39.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.045512+0000", 2024-09-13T10:11:39.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014694248 2024-09-13T10:11:39.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.045725+0000", 2024-09-13T10:11:39.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021261300000000001 2024-09-13T10:11:39.659 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.659 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.659 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.659 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:226 3.3 e38/30 3:fe5bfa3a:::smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'5, mlcod=38'3)", 2024-09-13T10:11:39.659 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:11:39.659 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 48.470593409000003, 2024-09-13T10:11:39.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030943633000000002, 2024-09-13T10:11:39.659 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.659 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.659 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.660 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:11:39.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:11:39.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080461+0000", 2024-09-13T10:11:39.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7049999999999999e-06 2024-09-13T10:11:39.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097571+0000", 2024-09-13T10:11:39.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017110658000000001 2024-09-13T10:11:39.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097615+0000", 2024-09-13T10:11:39.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3404e-05 2024-09-13T10:11:39.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097635+0000", 2024-09-13T10:11:39.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0279e-05 2024-09-13T10:11:39.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097995+0000", 2024-09-13T10:11:39.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00036038699999999999 2024-09-13T10:11:39.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.098029+0000", 2024-09-13T10:11:39.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4162000000000003e-05 2024-09-13T10:11:39.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111330+0000", 2024-09-13T10:11:39.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013300268000000001 2024-09-13T10:11:39.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111401+0000", 2024-09-13T10:11:39.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1769999999999999e-05 2024-09-13T10:11:39.663 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.663 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.663 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.664 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:237 3.3 e38/30 3:de40747e:::smithi19530714-38 low_tier:head v 38'7, mlcod=38'3)", 2024-09-13T10:11:39.664 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:11:39.664 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 48.436360616999998, 2024-09-13T10:11:39.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.037747504000000001, 2024-09-13T10:11:39.664 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.664 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.664 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.664 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:11:39.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:11:39.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:11:39.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7299999999999995e-07 2024-09-13T10:11:39.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145311+0000", 2024-09-13T10:11:39.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030619621 2024-09-13T10:11:39.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145336+0000", 2024-09-13T10:11:39.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5009999999999999e-05 2024-09-13T10:11:39.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145349+0000", 2024-09-13T10:11:39.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2724999999999999e-05 2024-09-13T10:11:39.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145445+0000", 2024-09-13T10:11:39.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6108999999999997e-05 2024-09-13T10:11:39.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145470+0000", 2024-09-13T10:11:39.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5514e-05 2024-09-13T10:11:39.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152389+0000", 2024-09-13T10:11:39.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069190739999999999 2024-09-13T10:11:39.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152438+0000", 2024-09-13T10:11:39.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8778e-05 2024-09-13T10:11:39.668 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.668 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.668 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.668 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:286 4.a 4:5ad8883b:::smithi19530714-37:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:39.668 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:11:39.668 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 48.302763990000003, 2024-09-13T10:11:39.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032980435000000002, 2024-09-13T10:11:39.669 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.669 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.669 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:39.672 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.673 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:39.673 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:39.673 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 286 2024-09-13T10:11:39.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.673 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:11:39.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:11:39.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248290+0000", 2024-09-13T10:11:39.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0170000000000001e-06 2024-09-13T10:11:39.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248302+0000", 2024-09-13T10:11:39.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1602e-05 2024-09-13T10:11:39.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248328+0000", 2024-09-13T10:11:39.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.622e-05 2024-09-13T10:11:39.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248344+0000", 2024-09-13T10:11:39.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6192999999999999e-05 2024-09-13T10:11:39.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248466+0000", 2024-09-13T10:11:39.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012186899999999999 2024-09-13T10:11:39.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248602+0000", 2024-09-13T10:11:39.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013623500000000001 2024-09-13T10:11:39.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 6,13", 2024-09-13T10:11:39.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274091+0000", 2024-09-13T10:11:39.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025489050999999999 2024-09-13T10:11:39.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275976+0000", 2024-09-13T10:11:39.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018847289999999999 2024-09-13T10:11:39.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:39.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276801+0000", 2024-09-13T10:11:39.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00082477899999999999 2024-09-13T10:11:39.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:39.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281064+0000", 2024-09-13T10:11:39.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042625950000000001 2024-09-13T10:11:39.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:39.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281173+0000", 2024-09-13T10:11:39.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000109391 2024-09-13T10:11:39.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281268+0000", 2024-09-13T10:11:39.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4753999999999994e-05 2024-09-13T10:11:39.678 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.678 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.678 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.679 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1631 4.a 4:5ad8883b:::smithi19530714-37:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:39.679 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:06.099753+0000", 2024-09-13T10:11:39.679 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.451298416999997, 2024-09-13T10:11:39.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033125612999999998, 2024-09-13T10:11:39.679 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:39.679 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:39.679 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:11:39.679 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:39.679 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:39.679 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:39.679 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1631 2024-09-13T10:11:39.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.679 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:39.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:39.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099753+0000", 2024-09-13T10:11:39.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:39.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099753+0000", 2024-09-13T10:11:39.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:39.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:39.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099754+0000", 2024-09-13T10:11:39.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2699999999999999e-06 2024-09-13T10:11:39.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:39.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099758+0000", 2024-09-13T10:11:39.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6540000000000001e-06 2024-09-13T10:11:39.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:39.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099768+0000", 2024-09-13T10:11:39.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0412e-05 2024-09-13T10:11:39.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:39.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099777+0000", 2024-09-13T10:11:39.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4600000000000003e-06 2024-09-13T10:11:39.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:39.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099830+0000", 2024-09-13T10:11:39.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3192000000000002e-05 2024-09-13T10:11:39.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:39.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099889+0000", 2024-09-13T10:11:39.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9576e-05 2024-09-13T10:11:39.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:39.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:39.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:39.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.132878+0000", 2024-09-13T10:11:39.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032989048999999999 2024-09-13T10:11:39.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.683 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:39.683 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:39.683 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:39.685 INFO:tasks.rados.rados.0.smithi195.stdout:1400: set_chunk oid 28 offset: 1460736 length: 596480 target oid 5d8cdb7a7816cf686e0ea88619018ffa8074d204228cead6cc70f4c00e944693 offset: 0 snap -1 2024-09-13T10:11:39.685 INFO:tasks.rados.rados.0.smithi195.stdout:1401: read oid 3 snap -1 2024-09-13T10:11:39.685 INFO:tasks.rados.rados.0.smithi195.stdout:1401: expect (ObjNum 237 snap 0 seq_num 237) 2024-09-13T10:11:39.685 INFO:tasks.rados.rados.0.smithi195.stdout:1402: set_chunk oid 20 target oid 2024-09-13T10:11:39.685 INFO:tasks.rados.rados.0.smithi195.stdout:1402: set_chunk oid 20 offset: 1479168 length: 38912 target oid 027973d089e9ed6de4388fc918085e2ab276e4503b484a34e40e378d2d68c3d6 offset: 0 snap -1 2024-09-13T10:11:39.685 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:39.685 INFO:tasks.rados.rados.0.smithi195.stdout:1391: expect (ObjNum 260 snap 0 seq_num 260) 2024-09-13T10:11:39.686 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:39.682+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:39.686 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-13T10:11:39.687 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:39.687 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:39.687 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:39.688 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:39.688 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:39.688 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:39.692 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:39.692 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:39.704 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 72003 2024-09-13T10:11:39.708 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:39.709 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:39.729 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:39.729 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:39.750 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:39.750 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:39.751 INFO:tasks.rados.rados.0.smithi195.stdout:1389: expect (ObjNum 236 snap 0 seq_num 236) 2024-09-13T10:11:39.770 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:39.771 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:39.785 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:39.782+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:39.791 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:39.791 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:39.812 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:39.812 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:39.817 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:39.817 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:39.818 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:11:39.818 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:39.832 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-13T10:11:39.833 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:39.834 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:39.835 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:39.854 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:39.854 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:39.872 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:39.873 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:39.887 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:39.880+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:39.893 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:39.894 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:39.915 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:39.915 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:39.936 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:39.937 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:39.961 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:39.962 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:39.976 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-13T10:11:40.004 INFO:tasks.rados.rados.0.smithi195.stdout:1388: expect (ObjNum 273 snap 0 seq_num 273) 2024-09-13T10:11:40.018 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:40.015+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:40.047 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-13T10:11:40.065 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2066268 2024-09-13T10:11:40.076 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:40.088 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:40.086+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:40.121 INFO:tasks.rados.rados.0.smithi195.stdout:1390: expect (ObjNum 263 snap 0 seq_num 263) 2024-09-13T10:11:40.121 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 393006 2024-09-13T10:11:40.143 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:40.143 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:40.157 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T10:11:40.166 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:40.189 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:40.186+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:40.257 INFO:tasks.rados.rados.0.smithi195.stdout:1392: expect (ObjNum 221 snap 0 seq_num 221) 2024-09-13T10:11:40.276 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1730122 2024-09-13T10:11:40.289 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:40.289 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:40.289 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:40.284+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:40.303 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T10:11:40.335 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1431939 2024-09-13T10:11:40.391 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:40.388+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:40.462 INFO:tasks.rados.rados.0.smithi195.stdout:1394: finishing write tid 1 to smithi19530714-7 2024-09-13T10:11:40.462 INFO:tasks.rados.rados.0.smithi195.stdout:1394: finishing write tid 2 to smithi19530714-7 2024-09-13T10:11:40.462 INFO:tasks.rados.rados.0.smithi195.stdout:1395: expect (ObjNum 261 snap 0 seq_num 261) 2024-09-13T10:11:40.469 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:40.469 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:40.483 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T10:11:40.491 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:40.487+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:40.527 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 877663 2024-09-13T10:11:40.548 INFO:tasks.rados.rados.0.smithi195.stdout:1394: finishing write tid 3 to smithi19530714-7 2024-09-13T10:11:40.548 INFO:tasks.rados.rados.0.smithi195.stdout:1394: finishing write tid 4 to smithi19530714-7 2024-09-13T10:11:40.548 INFO:tasks.rados.rados.0.smithi195.stdout:1394: oid 7 updating version 0 to 97 2024-09-13T10:11:40.548 INFO:tasks.rados.rados.0.smithi195.stdout:1394: oid 7 updating version 97 to 98 2024-09-13T10:11:40.548 INFO:tasks.rados.rados.0.smithi195.stdout:1394: oid 7 updating version 98 to 99 2024-09-13T10:11:40.548 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 99 (ObjNum 276 snap 0 seq_num 276) dirty exists 2024-09-13T10:11:40.548 INFO:tasks.rados.rados.0.smithi195.stdout:1394: left oid 7 (ObjNum 276 snap 0 seq_num 276) 2024-09-13T10:11:40.548 INFO:tasks.rados.rados.0.smithi195.stdout:1396: finishing write tid 1 to smithi19530714-18 2024-09-13T10:11:40.548 INFO:tasks.rados.rados.0.smithi195.stdout:1396: finishing write tid 2 to smithi19530714-18 2024-09-13T10:11:40.548 INFO:tasks.rados.rados.0.smithi195.stdout:1396: finishing write tid 3 to smithi19530714-18 2024-09-13T10:11:40.548 INFO:tasks.rados.rados.0.smithi195.stdout:1396: finishing write tid 4 to smithi19530714-18 2024-09-13T10:11:40.549 INFO:tasks.rados.rados.0.smithi195.stdout:1396: oid 18 updating version 0 to 100 2024-09-13T10:11:40.549 INFO:tasks.rados.rados.0.smithi195.stdout:1396: oid 18 updating version 100 to 101 2024-09-13T10:11:40.549 INFO:tasks.rados.rados.0.smithi195.stdout:1396: oid 18 updating version 101 to 102 2024-09-13T10:11:40.549 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 102 (ObjNum 277 snap 0 seq_num 277) dirty exists 2024-09-13T10:11:40.549 INFO:tasks.rados.rados.0.smithi195.stdout:1396: left oid 18 (ObjNum 277 snap 0 seq_num 277) 2024-09-13T10:11:40.549 INFO:tasks.rados.rados.0.smithi195.stdout:1397: finishing write tid 1 to smithi19530714-17 2024-09-13T10:11:40.549 INFO:tasks.rados.rados.0.smithi195.stdout:1397: finishing write tid 2 to smithi19530714-17 2024-09-13T10:11:40.549 INFO:tasks.rados.rados.0.smithi195.stdout:1397: finishing write tid 3 to smithi19530714-17 2024-09-13T10:11:40.549 INFO:tasks.rados.rados.0.smithi195.stdout:1397: finishing write tid 4 to smithi19530714-17 2024-09-13T10:11:40.549 INFO:tasks.rados.rados.0.smithi195.stdout:1397: oid 17 updating version 0 to 68 2024-09-13T10:11:40.549 INFO:tasks.rados.rados.0.smithi195.stdout:1397: oid 17 updating version 68 to 69 2024-09-13T10:11:40.549 INFO:tasks.rados.rados.0.smithi195.stdout:1397: oid 17 updating version 69 to 70 2024-09-13T10:11:40.549 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 70 (ObjNum 278 snap 0 seq_num 278) dirty exists 2024-09-13T10:11:40.549 INFO:tasks.rados.rados.0.smithi195.stdout:1397: left oid 17 (ObjNum 278 snap 0 seq_num 278) 2024-09-13T10:11:40.550 INFO:tasks.rados.rados.0.smithi195.stdout:1399: expect (ObjNum 267 snap 0 seq_num 267) 2024-09-13T10:11:40.550 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:40.592 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:40.590+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:40.644 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1728879 2024-09-13T10:11:40.649 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:40.649 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:40.664 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T10:11:40.664 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 253420 2024-09-13T10:11:40.693 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:40.691+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:40.724 INFO:tasks.rados.rados.0.smithi195.stdout:1398: expect (ObjNum 262 snap 0 seq_num 262) 2024-09-13T10:11:40.737 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1144145 2024-09-13T10:11:40.772 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 767592 2024-09-13T10:11:40.789 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:40.789 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:40.793 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:40.788+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:40.803 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T10:11:40.813 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:40.861 INFO:tasks.rados.rados.0.smithi195.stdout:1400: finishing set_chunk to oid 28 2024-09-13T10:11:40.861 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 28 set_chunk returned error code -95 offset: 1460736 length: 596480 2024-09-13T10:11:40.862 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 0 (ObjNum 180 snap 0 seq_num 180) dirty exists 2024-09-13T10:11:40.862 INFO:tasks.rados.rados.0.smithi195.stdout:1402: finishing set_chunk to oid 20 2024-09-13T10:11:40.862 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 76 (ObjNum 232 snap 0 seq_num 232) dirty exists 2024-09-13T10:11:40.862 INFO:tasks.rados.rados.0.smithi195.stdout:1401: expect (ObjNum 237 snap 0 seq_num 237) 2024-09-13T10:11:40.894 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:40.891+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:40.909 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3607748 2024-09-13T10:11:40.921 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:40.965 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:40.966 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:40.980 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T10:11:40.995 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:40.990+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:41.070 INFO:tasks.rados.rados.0.smithi195.stdout:1387: finishing write tid 1 to smithi19530714-50 2024-09-13T10:11:41.070 INFO:tasks.rados.rados.0.smithi195.stdout:1387: finishing write tid 2 to smithi19530714-50 2024-09-13T10:11:41.070 INFO:tasks.rados.rados.0.smithi195.stdout:1387: finishing write tid 3 to smithi19530714-50 2024-09-13T10:11:41.071 INFO:tasks.rados.rados.0.smithi195.stdout:1387: finishing write tid 4 to smithi19530714-50 2024-09-13T10:11:41.071 INFO:tasks.rados.rados.0.smithi195.stdout:1387: oid 50 updating version 0 to 39 2024-09-13T10:11:41.071 INFO:tasks.rados.rados.0.smithi195.stdout:1387: oid 50 version 39 is already newer than 38 2024-09-13T10:11:41.071 INFO:tasks.rados.rados.0.smithi195.stdout:1387: oid 50 updating version 39 to 40 2024-09-13T10:11:41.071 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 40 (ObjNum 274 snap 0 seq_num 274) dirty exists 2024-09-13T10:11:41.071 INFO:tasks.rados.rados.0.smithi195.stdout:1387: left oid 50 (ObjNum 274 snap 0 seq_num 274) 2024-09-13T10:11:41.071 INFO:tasks.rados.rados.0.smithi195.stdout:1393: finishing write tid 1 to smithi19530714-16 2024-09-13T10:11:41.071 INFO:tasks.rados.rados.0.smithi195.stdout:1393: finishing write tid 2 to smithi19530714-16 2024-09-13T10:11:41.071 INFO:tasks.rados.rados.0.smithi195.stdout:1393: finishing write tid 3 to smithi19530714-16 2024-09-13T10:11:41.071 INFO:tasks.rados.rados.0.smithi195.stdout:1387: done (15 left) 2024-09-13T10:11:41.071 INFO:tasks.rados.rados.0.smithi195.stdout:1388: done (14 left) 2024-09-13T10:11:41.071 INFO:tasks.rados.rados.0.smithi195.stdout:1389: done (13 left) 2024-09-13T10:11:41.072 INFO:tasks.rados.rados.0.smithi195.stdout:1390: done (12 left) 2024-09-13T10:11:41.072 INFO:tasks.rados.rados.0.smithi195.stdout:1391: done (11 left) 2024-09-13T10:11:41.072 INFO:tasks.rados.rados.0.smithi195.stdout:1392: done (10 left) 2024-09-13T10:11:41.072 INFO:tasks.rados.rados.0.smithi195.stdout:1394: done (9 left) 2024-09-13T10:11:41.072 INFO:tasks.rados.rados.0.smithi195.stdout:1395: done (8 left) 2024-09-13T10:11:41.072 INFO:tasks.rados.rados.0.smithi195.stdout:1396: done (7 left) 2024-09-13T10:11:41.072 INFO:tasks.rados.rados.0.smithi195.stdout:1397: done (6 left) 2024-09-13T10:11:41.072 INFO:tasks.rados.rados.0.smithi195.stdout:1398: done (5 left) 2024-09-13T10:11:41.072 INFO:tasks.rados.rados.0.smithi195.stdout:1399: done (4 left) 2024-09-13T10:11:41.072 INFO:tasks.rados.rados.0.smithi195.stdout:1400: done (3 left) 2024-09-13T10:11:41.072 INFO:tasks.rados.rados.0.smithi195.stdout:1401: done (2 left) 2024-09-13T10:11:41.072 INFO:tasks.rados.rados.0.smithi195.stdout:1402: done (1 left) 2024-09-13T10:11:41.072 INFO:tasks.rados.rados.0.smithi195.stdout:1403: write oid 38 current snap is 0 2024-09-13T10:11:41.073 INFO:tasks.rados.rados.0.smithi195.stdout:1403: seq_num 279 ranges {689153=783179,2076829=457413} 2024-09-13T10:11:41.094 INFO:tasks.rados.rados.0.smithi195.stdout:1403: writing smithi19530714-38 from 689153 to 1472332 tid 1 2024-09-13T10:11:41.095 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:41.091+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:41.108 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:41.108 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:41.112 INFO:tasks.rados.rados.0.smithi195.stdout:1403: writing smithi19530714-38 from 2076829 to 2534242 tid 2 2024-09-13T10:11:41.112 INFO:tasks.rados.rados.0.smithi195.stdout:1404: read oid 36 snap -1 2024-09-13T10:11:41.113 INFO:tasks.rados.rados.0.smithi195.stdout:1404: expect (ObjNum 269 snap 0 seq_num 269) 2024-09-13T10:11:41.113 INFO:tasks.rados.rados.0.smithi195.stdout:1405: write (excl) oid 2 current snap is 0 2024-09-13T10:11:41.113 INFO:tasks.rados.rados.0.smithi195.stdout:1393: finishing write tid 4 to smithi19530714-16 2024-09-13T10:11:41.113 INFO:tasks.rados.rados.0.smithi195.stdout:1393: oid 16 updating version 0 to 42 2024-09-13T10:11:41.113 INFO:tasks.rados.rados.0.smithi195.stdout:1393: oid 16 updating version 42 to 43 2024-09-13T10:11:41.113 INFO:tasks.rados.rados.0.smithi195.stdout:1393: oid 16 version 43 is already newer than 41 2024-09-13T10:11:41.113 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 43 (ObjNum 275 snap 0 seq_num 275) dirty exists 2024-09-13T10:11:41.113 INFO:tasks.rados.rados.0.smithi195.stdout:1393: left oid 16 (ObjNum 275 snap 0 seq_num 275) 2024-09-13T10:11:41.113 INFO:tasks.rados.rados.0.smithi195.stdout:1403: finishing write tid 1 to smithi19530714-38 2024-09-13T10:11:41.113 INFO:tasks.rados.rados.0.smithi195.stdout:1405: seq_num 280 ranges {705960=458428,1735311=717424} 2024-09-13T10:11:41.122 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-13T10:11:41.130 INFO:tasks.rados.rados.0.smithi195.stdout:1405: writing smithi19530714-2 from 705960 to 1164388 tid 1 2024-09-13T10:11:41.147 INFO:tasks.rados.rados.0.smithi195.stdout:1405: writing smithi19530714-2 from 1735311 to 2452735 tid 2 2024-09-13T10:11:41.147 INFO:tasks.rados.rados.0.smithi195.stdout:1393: done (3 left) 2024-09-13T10:11:41.147 INFO:tasks.rados.rados.0.smithi195.stdout:1406: read oid 20 snap -1 2024-09-13T10:11:41.147 INFO:tasks.rados.rados.0.smithi195.stdout:1406: expect (ObjNum 232 snap 0 seq_num 232) 2024-09-13T10:11:41.147 INFO:tasks.rados.rados.0.smithi195.stdout:1407: tier_promote oid 5 2024-09-13T10:11:41.148 INFO:tasks.rados.rados.0.smithi195.stdout:1403: finishing write tid 2 to smithi19530714-38 2024-09-13T10:11:41.148 INFO:tasks.rados.rados.0.smithi195.stdout:1404: expect (ObjNum 269 snap 0 seq_num 269) 2024-09-13T10:11:41.188 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:41.196 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:41.192+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:41.231 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 793288 2024-09-13T10:11:41.241 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:41.241 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:41.255 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-13T10:11:41.260 INFO:tasks.rados.rados.0.smithi195.stdout:1403: finishing write tid 3 to smithi19530714-38 2024-09-13T10:11:41.260 INFO:tasks.rados.rados.0.smithi195.stdout:1403: finishing write tid 4 to smithi19530714-38 2024-09-13T10:11:41.260 INFO:tasks.rados.rados.0.smithi195.stdout:1403: oid 38 updating version 0 to 132 2024-09-13T10:11:41.260 INFO:tasks.rados.rados.0.smithi195.stdout:1403: oid 38 version 132 is already newer than 131 2024-09-13T10:11:41.260 INFO:tasks.rados.rados.0.smithi195.stdout:1403: oid 38 updating version 132 to 133 2024-09-13T10:11:41.261 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 133 (ObjNum 279 snap 0 seq_num 279) dirty exists 2024-09-13T10:11:41.261 INFO:tasks.rados.rados.0.smithi195.stdout:1403: left oid 38 (ObjNum 279 snap 0 seq_num 279) 2024-09-13T10:11:41.261 INFO:tasks.rados.rados.0.smithi195.stdout:1405: finishing write tid 1 to smithi19530714-2 2024-09-13T10:11:41.261 INFO:tasks.rados.rados.0.smithi195.stdout:1405: finishing write tid 2 to smithi19530714-2 2024-09-13T10:11:41.261 INFO:tasks.rados.rados.0.smithi195.stdout:1405: finishing write tid 3 to smithi19530714-2 2024-09-13T10:11:41.261 INFO:tasks.rados.rados.0.smithi195.stdout:1405: finishing write tid 4 to smithi19530714-2 2024-09-13T10:11:41.261 INFO:tasks.rados.rados.0.smithi195.stdout:1405: oid 2 updating version 0 to 126 2024-09-13T10:11:41.261 INFO:tasks.rados.rados.0.smithi195.stdout:1405: oid 2 version 126 is already newer than 125 2024-09-13T10:11:41.261 INFO:tasks.rados.rados.0.smithi195.stdout:1405: oid 2 updating version 126 to 127 2024-09-13T10:11:41.261 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 127 (ObjNum 280 snap 0 seq_num 280) dirty exists 2024-09-13T10:11:41.261 INFO:tasks.rados.rados.0.smithi195.stdout:1405: left oid 2 (ObjNum 280 snap 0 seq_num 280) 2024-09-13T10:11:41.261 INFO:tasks.rados.rados.0.smithi195.stdout:1406: expect (ObjNum 232 snap 0 seq_num 232) 2024-09-13T10:11:41.263 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1283752 2024-09-13T10:11:41.298 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:41.294+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:41.304 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2546082 2024-09-13T10:11:41.356 INFO:tasks.rados.rados.0.smithi195.stdout:1407: got (0) Success 2024-09-13T10:11:41.356 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 16 (ObjNum 231 snap 0 seq_num 231) dirty exists 2024-09-13T10:11:41.356 INFO:tasks.rados.rados.0.smithi195.stdout:1403: done (4 left) 2024-09-13T10:11:41.356 INFO:tasks.rados.rados.0.smithi195.stdout:1404: done (3 left) 2024-09-13T10:11:41.356 INFO:tasks.rados.rados.0.smithi195.stdout:1405: done (2 left) 2024-09-13T10:11:41.356 INFO:tasks.rados.rados.0.smithi195.stdout:1406: done (1 left) 2024-09-13T10:11:41.356 INFO:tasks.rados.rados.0.smithi195.stdout:1407: done (0 left) 2024-09-13T10:11:41.357 INFO:tasks.rados.rados.0.smithi195.stdout:1408: read oid 30 snap -1 2024-09-13T10:11:41.357 INFO:tasks.rados.rados.0.smithi195.stdout:1408: expect deleted 2024-09-13T10:11:41.357 INFO:tasks.rados.rados.0.smithi195.stdout:1409: write oid 46 current snap is 0 2024-09-13T10:11:41.357 INFO:tasks.rados.rados.0.smithi195.stdout:1409: seq_num 281 ranges {722767=533677,1793793=461082} 2024-09-13T10:11:41.374 INFO:tasks.rados.rados.0.smithi195.stdout:1409: writing smithi19530714-46 from 722767 to 1256444 tid 1 2024-09-13T10:11:41.390 INFO:tasks.rados.rados.0.smithi195.stdout:1409: writing smithi19530714-46 from 1793793 to 2254875 tid 2 2024-09-13T10:11:41.390 INFO:tasks.rados.rados.0.smithi195.stdout:1410: read oid 27 snap -1 2024-09-13T10:11:41.390 INFO:tasks.rados.rados.0.smithi195.stdout:1410: expect (ObjNum 221 snap 0 seq_num 221) 2024-09-13T10:11:41.390 INFO:tasks.rados.rados.0.smithi195.stdout:1411: tier_evict oid 36 2024-09-13T10:11:41.390 INFO:tasks.rados.rados.0.smithi195.stdout:1411: tier_evict oid 36 snap -1 2024-09-13T10:11:41.390 INFO:tasks.rados.rados.0.smithi195.stdout:1412: read oid 40 snap -1 2024-09-13T10:11:41.390 INFO:tasks.rados.rados.0.smithi195.stdout:1412: expect deleted 2024-09-13T10:11:41.390 INFO:tasks.rados.rados.0.smithi195.stdout:1409: finishing write tid 1 to smithi19530714-46 2024-09-13T10:11:41.390 INFO:tasks.rados.rados.0.smithi195.stdout:1408: done (4 left) 2024-09-13T10:11:41.391 INFO:tasks.rados.rados.0.smithi195.stdout:1413: delete oid 20 current snap is 0 2024-09-13T10:11:41.391 INFO:tasks.rados.rados.0.smithi195.stdout:1411: got (22) Invalid argument 2024-09-13T10:11:41.394 INFO:tasks.rados.rados.0.smithi195.stdout:1411: done (4 left) 2024-09-13T10:11:41.394 INFO:tasks.rados.rados.0.smithi195.stdout:1412: done (3 left) 2024-09-13T10:11:41.394 INFO:tasks.rados.rados.0.smithi195.stdout:1413: done (2 left) 2024-09-13T10:11:41.394 INFO:tasks.rados.rados.0.smithi195.stdout:1414: write oid 39 current snap is 0 2024-09-13T10:11:41.394 INFO:tasks.rados.rados.0.smithi195.stdout:1414: seq_num 283 ranges {756381=400528,1910757=464751} 2024-09-13T10:11:41.399 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:41.395+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:41.400 INFO:tasks.rados.rados.0.smithi195.stdout:1414: writing smithi19530714-39 from 756381 to 1156909 tid 1 2024-09-13T10:11:41.409 INFO:tasks.rados.rados.0.smithi195.stdout:1414: writing smithi19530714-39 from 1910757 to 2375508 tid 2 2024-09-13T10:11:41.409 INFO:tasks.rados.rados.0.smithi195.stdout:1415: read oid 28 snap -1 2024-09-13T10:11:41.409 INFO:tasks.rados.rados.0.smithi195.stdout:1415: expect (ObjNum 180 snap 0 seq_num 180) 2024-09-13T10:11:41.409 INFO:tasks.rados.rados.0.smithi195.stdout:1416: set_chunk oid 44 target oid 2024-09-13T10:11:41.421 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:41.421 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:41.436 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-13T10:11:41.446 INFO:tasks.rados.rados.0.smithi195.stdout:1416: set_chunk oid 44 offset: 95232 length: 2359808 target oid f8ac1c2b27cd80e9e9d7eded0cb6d15fa71f15944afb624453acc59c8d569d97 offset: 0 snap -1 2024-09-13T10:11:41.446 INFO:tasks.rados.rados.0.smithi195.stdout:1417: write (excl) oid 4 current snap is 0 2024-09-13T10:11:41.446 INFO:tasks.rados.rados.0.smithi195.stdout:1409: finishing write tid 2 to smithi19530714-46 2024-09-13T10:11:41.446 INFO:tasks.rados.rados.0.smithi195.stdout:1409: finishing write tid 3 to smithi19530714-46 2024-09-13T10:11:41.446 INFO:tasks.rados.rados.0.smithi195.stdout:1409: finishing write tid 4 to smithi19530714-46 2024-09-13T10:11:41.446 INFO:tasks.rados.rados.0.smithi195.stdout:1409: oid 46 updating version 0 to 39 2024-09-13T10:11:41.446 INFO:tasks.rados.rados.0.smithi195.stdout:1409: oid 46 version 39 is already newer than 38 2024-09-13T10:11:41.446 INFO:tasks.rados.rados.0.smithi195.stdout:1409: oid 46 version 39 is already newer than 37 2024-09-13T10:11:41.446 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 39 (ObjNum 281 snap 0 seq_num 281) dirty exists 2024-09-13T10:11:41.446 INFO:tasks.rados.rados.0.smithi195.stdout:1409: left oid 46 (ObjNum 281 snap 0 seq_num 281) 2024-09-13T10:11:41.447 INFO:tasks.rados.rados.0.smithi195.stdout:1417: seq_num 284 ranges {773188=475777,1969239=724762} 2024-09-13T10:11:41.460 INFO:tasks.rados.rados.0.smithi195.stdout:1417: writing smithi19530714-4 from 773188 to 1248965 tid 1 2024-09-13T10:11:41.476 INFO:tasks.rados.rados.0.smithi195.stdout:1417: writing smithi19530714-4 from 1969239 to 2694001 tid 2 2024-09-13T10:11:41.476 INFO:tasks.rados.rados.0.smithi195.stdout:1409: done (5 left) 2024-09-13T10:11:41.476 INFO:tasks.rados.rados.0.smithi195.stdout:1418: read oid 23 snap -1 2024-09-13T10:11:41.476 INFO:tasks.rados.rados.0.smithi195.stdout:1418: expect (ObjNum 254 snap 0 seq_num 254) 2024-09-13T10:11:41.476 INFO:tasks.rados.rados.0.smithi195.stdout:1419: read oid 33 snap -1 2024-09-13T10:11:41.477 INFO:tasks.rados.rados.0.smithi195.stdout:1419: expect (ObjNum 253 snap 0 seq_num 253) 2024-09-13T10:11:41.477 INFO:tasks.rados.rados.0.smithi195.stdout:1420: set_chunk oid 46 target oid 2024-09-13T10:11:41.493 INFO:tasks.rados.rados.0.smithi195.stdout:1420: set_chunk oid 46 offset: 1357312 length: 1160704 target oid 68b8da6fd9d35331f6480334c21b910c7a9693208d49e22b3a9750d8e0e676e9 offset: 0 snap -1 2024-09-13T10:11:41.493 INFO:tasks.rados.rados.0.smithi195.stdout:1421: delete oid 1 current snap is 0 2024-09-13T10:11:41.493 INFO:tasks.rados.rados.0.smithi195.stdout:1410: expect (ObjNum 221 snap 0 seq_num 221) 2024-09-13T10:11:41.493 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 314805 2024-09-13T10:11:41.499 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:41.496+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:41.557 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:41.557 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:41.572 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-13T10:11:41.601 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:41.595+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:41.650 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2245341 2024-09-13T10:11:41.701 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:41.696+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:41.733 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:41.733 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:41.747 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-13T10:11:41.772 INFO:tasks.rados.rados.0.smithi195.stdout:1414: finishing write tid 1 to smithi19530714-39 2024-09-13T10:11:41.772 INFO:tasks.rados.rados.0.smithi195.stdout:1414: finishing write tid 2 to smithi19530714-39 2024-09-13T10:11:41.772 INFO:tasks.rados.rados.0.smithi195.stdout:1414: finishing write tid 3 to smithi19530714-39 2024-09-13T10:11:41.772 INFO:tasks.rados.rados.0.smithi195.stdout:1414: finishing write tid 4 to smithi19530714-39 2024-09-13T10:11:41.772 INFO:tasks.rados.rados.0.smithi195.stdout:1414: oid 39 updating version 0 to 32 2024-09-13T10:11:41.772 INFO:tasks.rados.rados.0.smithi195.stdout:1414: oid 39 version 32 is already newer than 31 2024-09-13T10:11:41.772 INFO:tasks.rados.rados.0.smithi195.stdout:1414: oid 39 version 32 is already newer than 30 2024-09-13T10:11:41.773 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 32 (ObjNum 283 snap 0 seq_num 283) dirty exists 2024-09-13T10:11:41.773 INFO:tasks.rados.rados.0.smithi195.stdout:1414: left oid 39 (ObjNum 283 snap 0 seq_num 283) 2024-09-13T10:11:41.773 INFO:tasks.rados.rados.0.smithi195.stdout:1415: expect (ObjNum 180 snap 0 seq_num 180) 2024-09-13T10:11:41.773 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:41.839 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:41.835+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:41.903 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:41.898+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:41.916 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:41.916 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:41.930 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-13T10:11:41.933 INFO:tasks.rados.rados.0.smithi195.stdout:1416: finishing set_chunk to oid 44 2024-09-13T10:11:41.933 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 57 (ObjNum 205 snap 0 seq_num 205) dirty exists 2024-09-13T10:11:41.933 INFO:tasks.rados.rados.0.smithi195.stdout:1417: finishing write tid 1 to smithi19530714-4 2024-09-13T10:11:41.933 INFO:tasks.rados.rados.0.smithi195.stdout:1418: expect (ObjNum 254 snap 0 seq_num 254) 2024-09-13T10:11:41.955 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:11:42.004 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:42.002+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:42.027 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2010044 2024-09-13T10:11:42.030 INFO:tasks.rados.rados.0.smithi195.stdout:1417: finishing write tid 2 to smithi19530714-4 2024-09-13T10:11:42.030 INFO:tasks.rados.rados.0.smithi195.stdout:1417: finishing write tid 3 to smithi19530714-4 2024-09-13T10:11:42.030 INFO:tasks.rados.rados.0.smithi195.stdout:1417: finishing write tid 4 to smithi19530714-4 2024-09-13T10:11:42.030 INFO:tasks.rados.rados.0.smithi195.stdout:1417: oid 4 updating version 0 to 105 2024-09-13T10:11:42.030 INFO:tasks.rados.rados.0.smithi195.stdout:1417: oid 4 version 105 is already newer than 103 2024-09-13T10:11:42.030 INFO:tasks.rados.rados.0.smithi195.stdout:1417: oid 4 version 105 is already newer than 104 2024-09-13T10:11:42.030 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 105 (ObjNum 284 snap 0 seq_num 284) dirty exists 2024-09-13T10:11:42.031 INFO:tasks.rados.rados.0.smithi195.stdout:1417: left oid 4 (ObjNum 284 snap 0 seq_num 284) 2024-09-13T10:11:42.031 INFO:tasks.rados.rados.0.smithi195.stdout:1420: finishing set_chunk to oid 46 2024-09-13T10:11:42.031 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 39 (ObjNum 281 snap 0 seq_num 281) dirty exists 2024-09-13T10:11:42.031 INFO:tasks.rados.rados.0.smithi195.stdout:1410: done (8 left) 2024-09-13T10:11:42.031 INFO:tasks.rados.rados.0.smithi195.stdout:1414: done (7 left) 2024-09-13T10:11:42.031 INFO:tasks.rados.rados.0.smithi195.stdout:1415: done (6 left) 2024-09-13T10:11:42.031 INFO:tasks.rados.rados.0.smithi195.stdout:1416: done (5 left) 2024-09-13T10:11:42.031 INFO:tasks.rados.rados.0.smithi195.stdout:1417: done (4 left) 2024-09-13T10:11:42.031 INFO:tasks.rados.rados.0.smithi195.stdout:1418: done (3 left) 2024-09-13T10:11:42.031 INFO:tasks.rados.rados.0.smithi195.stdout:1420: done (2 left) 2024-09-13T10:11:42.032 INFO:tasks.rados.rados.0.smithi195.stdout:1421: done (1 left) 2024-09-13T10:11:42.032 INFO:tasks.rados.rados.0.smithi195.stdout:1422: write oid 23 current snap is 0 2024-09-13T10:11:42.032 INFO:tasks.rados.rados.0.smithi195.stdout:1422: seq_num 286 ranges {406802=626275,1686203=728431} 2024-09-13T10:11:42.046 INFO:tasks.rados.rados.0.smithi195.stdout:1422: writing smithi19530714-23 from 406802 to 1033077 tid 1 2024-09-13T10:11:42.063 INFO:tasks.rados.rados.0.smithi195.stdout:1422: writing smithi19530714-23 from 1686203 to 2414634 tid 2 2024-09-13T10:11:42.064 INFO:tasks.rados.rados.0.smithi195.stdout:1423: read oid 13 snap -1 2024-09-13T10:11:42.064 INFO:tasks.rados.rados.0.smithi195.stdout:1423: expect (ObjNum 228 snap 0 seq_num 228) 2024-09-13T10:11:42.064 INFO:tasks.rados.rados.0.smithi195.stdout:1424: write oid 12 current snap is 0 2024-09-13T10:11:42.064 INFO:tasks.rados.rados.0.smithi195.stdout:1424: seq_num 287 ranges {423609=701524,1628332=472089,2648987=174622} 2024-09-13T10:11:42.073 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T10:11:42.081 INFO:tasks.rados.rados.0.smithi195.stdout:1424: writing smithi19530714-12 from 423609 to 1125133 tid 1 2024-09-13T10:11:42.092 INFO:tasks.rados.rados.0.smithi195.stdout:1424: writing smithi19530714-12 from 1628332 to 2100421 tid 2 2024-09-13T10:11:42.095 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:42.095 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:42.099 INFO:tasks.rados.rados.0.smithi195.stdout:1424: writing smithi19530714-12 from 2648987 to 2823609 tid 3 2024-09-13T10:11:42.100 INFO:tasks.rados.rados.0.smithi195.stdout:1425: set_chunk oid 50 target oid 2024-09-13T10:11:42.110 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T10:11:42.111 INFO:tasks.rados.rados.0.smithi195.stdout:1425: set_chunk oid 50 offset: 492032 length: 396288 target oid a02e398036fd2de9105e7f184602cab1bafb18d27ecb6eed222d7eab1daae2ce offset: 0 snap -1 2024-09-13T10:11:42.111 INFO:tasks.rados.rados.0.smithi195.stdout:1426: set_chunk oid 2 target oid 2024-09-13T10:11:42.118 INFO:tasks.rados.rados.0.smithi195.stdout:1426: set_chunk oid 2 offset: 1756672 length: 320000 target oid 5f4f087865e56fb4d42a29a55333bbda836fff2766f32873c120a89e03460ee8 offset: 0 snap -1 2024-09-13T10:11:42.118 INFO:tasks.rados.rados.0.smithi195.stdout:1427: write (excl) oid 39 current snap is 0 2024-09-13T10:11:42.118 INFO:tasks.rados.rados.0.smithi195.stdout:1422: finishing write tid 1 to smithi19530714-23 2024-09-13T10:11:42.118 INFO:tasks.rados.rados.0.smithi195.stdout:1422: finishing write tid 2 to smithi19530714-23 2024-09-13T10:11:42.118 INFO:tasks.rados.rados.0.smithi195.stdout:1427: seq_num 288 ranges {440416=776773,1686814=732100} 2024-09-13T10:11:42.118 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:42.115+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:42.119 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:42.119 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:42.133 INFO:tasks.rados.rados.0.smithi195.stdout:1427: writing smithi19530714-39 from 440416 to 1217189 tid 1 2024-09-13T10:11:42.139 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:42.139 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:42.143 INFO:tasks.rados.rados.0.smithi195.stdout:1427: writing smithi19530714-39 from 1686814 to 2418914 tid 2 2024-09-13T10:11:42.143 INFO:tasks.rados.rados.0.smithi195.stdout:1428: read oid 4 snap -1 2024-09-13T10:11:42.143 INFO:tasks.rados.rados.0.smithi195.stdout:1428: expect (ObjNum 284 snap 0 seq_num 284) 2024-09-13T10:11:42.143 INFO:tasks.rados.rados.0.smithi195.stdout:1429: read oid 15 snap -1 2024-09-13T10:11:42.143 INFO:tasks.rados.rados.0.smithi195.stdout:1429: expect (ObjNum 261 snap 0 seq_num 261) 2024-09-13T10:11:42.143 INFO:tasks.rados.rados.0.smithi195.stdout:1430: write (excl) oid 17 current snap is 0 2024-09-13T10:11:42.143 INFO:tasks.rados.rados.0.smithi195.stdout:1430: seq_num 289 ranges {457223=568375,1461649=475758,2420186=437037} 2024-09-13T10:11:42.154 INFO:tasks.rados.rados.0.smithi195.stdout:1430: writing smithi19530714-17 from 457223 to 1025598 tid 1 2024-09-13T10:11:42.159 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:42.159 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:42.162 INFO:tasks.rados.rados.0.smithi195.stdout:1430: writing smithi19530714-17 from 1461649 to 1937407 tid 2 2024-09-13T10:11:42.170 INFO:tasks.rados.rados.0.smithi195.stdout:1430: writing smithi19530714-17 from 2420186 to 2857223 tid 3 2024-09-13T10:11:42.170 INFO:tasks.rados.rados.0.smithi195.stdout:1431: set_chunk oid 31 target oid 2024-09-13T10:11:42.178 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:42.179 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:42.183 INFO:tasks.rados.rados.0.smithi195.stdout:1431: set_chunk oid 31 offset: 1630720 length: 729088 target oid 62fa729b5c263c7e2390f0dd11f2cf2f2baf59afb62f8f4c38eafd9ff9a4c617 offset: 0 snap -1 2024-09-13T10:11:42.183 INFO:tasks.rados.rados.0.smithi195.stdout:1432: write oid 38 current snap is 0 2024-09-13T10:11:42.183 INFO:tasks.rados.rados.0.smithi195.stdout:1432: seq_num 290 ranges {474030=643624,1520131=735769} 2024-09-13T10:11:42.193 INFO:tasks.rados.rados.0.smithi195.stdout:1432: writing smithi19530714-38 from 474030 to 1117654 tid 1 2024-09-13T10:11:42.195 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:42.195 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:42.203 INFO:tasks.rados.rados.0.smithi195.stdout:1432: writing smithi19530714-38 from 1520131 to 2255900 tid 2 2024-09-13T10:11:42.203 INFO:tasks.rados.rados.0.smithi195.stdout:1433: read oid 49 snap -1 2024-09-13T10:11:42.203 INFO:tasks.rados.rados.0.smithi195.stdout:1433: expect (ObjNum 181 snap 0 seq_num 181) 2024-09-13T10:11:42.203 INFO:tasks.rados.rados.0.smithi195.stdout:1434: write oid 46 current snap is 0 2024-09-13T10:11:42.203 INFO:tasks.rados.rados.0.smithi195.stdout:1434: seq_num 291 ranges {490837=718873,1862260=479427,2875032=15805} 2024-09-13T10:11:42.206 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:42.202+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:42.212 INFO:tasks.rados.rados.0.smithi195.stdout:1434: writing smithi19530714-46 from 490837 to 1209710 tid 1 2024-09-13T10:11:42.216 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:42.217 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:42.221 INFO:tasks.rados.rados.0.smithi195.stdout:1434: writing smithi19530714-46 from 1862260 to 2341687 tid 2 2024-09-13T10:11:42.226 INFO:tasks.rados.rados.0.smithi195.stdout:1434: writing smithi19530714-46 from 2875032 to 2890837 tid 3 2024-09-13T10:11:42.226 INFO:tasks.rados.rados.0.smithi195.stdout:1435: write oid 26 current snap is 0 2024-09-13T10:11:42.226 INFO:tasks.rados.rados.0.smithi195.stdout:1435: seq_num 292 ranges {507644=794122,1920742=623085} 2024-09-13T10:11:42.234 INFO:tasks.rados.rados.0.smithi195.stdout:1435: writing smithi19530714-26 from 507644 to 1301766 tid 1 2024-09-13T10:11:42.242 INFO:tasks.rados.rados.0.smithi195.stdout:1435: writing smithi19530714-26 from 1920742 to 2543827 tid 2 2024-09-13T10:11:42.242 INFO:tasks.rados.rados.0.smithi195.stdout:1436: read oid 42 snap -1 2024-09-13T10:11:42.242 INFO:tasks.rados.rados.0.smithi195.stdout:1436: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-13T10:11:42.242 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:42.242 INFO:tasks.rados.rados.0.smithi195.stdout:1422: finishing write tid 3 to smithi19530714-23 2024-09-13T10:11:42.242 INFO:tasks.rados.rados.0.smithi195.stdout:1422: finishing write tid 4 to smithi19530714-23 2024-09-13T10:11:42.242 INFO:tasks.rados.rados.0.smithi195.stdout:1422: oid 23 updating version 0 to 42 2024-09-13T10:11:42.242 INFO:tasks.rados.rados.0.smithi195.stdout:1422: oid 23 version 42 is already newer than 40 2024-09-13T10:11:42.242 INFO:tasks.rados.rados.0.smithi195.stdout:1422: oid 23 version 42 is already newer than 41 2024-09-13T10:11:42.242 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 42 (ObjNum 286 snap 0 seq_num 286) dirty exists 2024-09-13T10:11:42.243 INFO:tasks.rados.rados.0.smithi195.stdout:1422: left oid 23 (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:11:42.243 INFO:tasks.rados.rados.0.smithi195.stdout:1423: expect (ObjNum 228 snap 0 seq_num 228) 2024-09-13T10:11:42.250 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:42.250 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:42.250 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:11:42.250 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:42.251 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:42.252 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:42.264 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T10:11:42.287 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:42.287 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:42.304 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:42.304 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:42.308 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:42.306+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:42.325 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:42.325 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:42.340 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2882572 2024-09-13T10:11:42.342 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:42.343 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:42.357 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:42.363 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:42.363 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:42.380 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:42.380 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:42.400 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:42.401 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:42.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:42.395+0000 7f644a7fc640 0 -- 172.21.15.73:0/2284663451 >> v2:172.21.15.181:6804/1173379175 conn(0x7f644c21f930 msgr2=0x7f644c221d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-13T10:11:42.407 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:42.402+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:42.432 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:42.433 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:42.433 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:11:42.433 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:11:42.433 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:42.447 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T10:11:42.508 INFO:tasks.rados.rados.0.smithi195.stdout:1424: finishing write tid 1 to smithi19530714-12 2024-09-13T10:11:42.508 INFO:tasks.rados.rados.0.smithi195.stdout:1424: finishing write tid 2 to smithi19530714-12 2024-09-13T10:11:42.508 INFO:tasks.rados.rados.0.smithi195.stdout:1424: finishing write tid 3 to smithi19530714-12 2024-09-13T10:11:42.509 INFO:tasks.rados.rados.0.smithi195.stdout:1424: finishing write tid 4 to smithi19530714-12 2024-09-13T10:11:42.509 INFO:tasks.rados.rados.0.smithi195.stdout:1424: finishing write tid 5 to smithi19530714-12 2024-09-13T10:11:42.509 INFO:tasks.rados.rados.0.smithi195.stdout:1424: oid 12 updating version 0 to 90 2024-09-13T10:11:42.509 INFO:tasks.rados.rados.0.smithi195.stdout:1424: oid 12 version 90 is already newer than 89 2024-09-13T10:11:42.509 INFO:tasks.rados.rados.0.smithi195.stdout:1424: oid 12 updating version 90 to 92 2024-09-13T10:11:42.509 INFO:tasks.rados.rados.0.smithi195.stdout:1424: oid 12 version 92 is already newer than 91 2024-09-13T10:11:42.509 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 92 (ObjNum 287 snap 0 seq_num 287) dirty exists 2024-09-13T10:11:42.509 INFO:tasks.rados.rados.0.smithi195.stdout:1424: left oid 12 (ObjNum 287 snap 0 seq_num 287) 2024-09-13T10:11:42.509 INFO:tasks.rados.rados.0.smithi195.stdout:1426: finishing set_chunk to oid 2 2024-09-13T10:11:42.509 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 127 (ObjNum 280 snap 0 seq_num 280) dirty exists 2024-09-13T10:11:42.509 INFO:tasks.rados.rados.0.smithi195.stdout:1427: finishing write tid 1 to smithi19530714-39 2024-09-13T10:11:42.510 INFO:tasks.rados.rados.0.smithi195.stdout:1425: finishing set_chunk to oid 50 2024-09-13T10:11:42.510 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 40 (ObjNum 274 snap 0 seq_num 274) dirty exists 2024-09-13T10:11:42.510 INFO:tasks.rados.rados.0.smithi195.stdout:1427: finishing write tid 2 to smithi19530714-39 2024-09-13T10:11:42.510 INFO:tasks.rados.rados.0.smithi195.stdout:1427: finishing write tid 3 to smithi19530714-39 2024-09-13T10:11:42.510 INFO:tasks.rados.rados.0.smithi195.stdout:1427: finishing write tid 4 to smithi19530714-39 2024-09-13T10:11:42.510 INFO:tasks.rados.rados.0.smithi195.stdout:1427: oid 39 updating version 0 to 35 2024-09-13T10:11:42.510 INFO:tasks.rados.rados.0.smithi195.stdout:1427: oid 39 version 35 is already newer than 34 2024-09-13T10:11:42.510 INFO:tasks.rados.rados.0.smithi195.stdout:1427: oid 39 version 35 is already newer than 33 2024-09-13T10:11:42.510 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 35 (ObjNum 288 snap 0 seq_num 288) dirty exists 2024-09-13T10:11:42.510 INFO:tasks.rados.rados.0.smithi195.stdout:1427: left oid 39 (ObjNum 288 snap 0 seq_num 288) 2024-09-13T10:11:42.511 INFO:tasks.rados.rados.0.smithi195.stdout:1429: expect (ObjNum 261 snap 0 seq_num 261) 2024-09-13T10:11:42.511 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:42.511 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:42.506+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:42.589 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1392488 2024-09-13T10:11:42.607 INFO:tasks.rados.rados.0.smithi195.stdout:1428: expect (ObjNum 284 snap 0 seq_num 284) 2024-09-13T10:11:42.609 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:42.603+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:42.613 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:42.614 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:11:42.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:11:42.614 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:11:42.614 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.614 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:259 4.1 4:85f1e656:::smithi19530714-10:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:42.614 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:11:42.614 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 51.394872632999999, 2024-09-13T10:11:42.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014693513, 2024-09-13T10:11:42.614 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.614 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.614 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:42.615 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.615 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.615 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.615 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 259 2024-09-13T10:11:42.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.615 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.615 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.615 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.616 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:11:42.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.616 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.616 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:11:42.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215171+0000", 2024-09-13T10:11:42.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.3329999999999998e-06 2024-09-13T10:11:42.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.617 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215190+0000", 2024-09-13T10:11:42.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8672999999999999e-05 2024-09-13T10:11:42.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.617 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.618 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215229+0000", 2024-09-13T10:11:42.618 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9076000000000003e-05 2024-09-13T10:11:42.618 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.618 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.618 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.618 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215287+0000", 2024-09-13T10:11:42.618 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8142e-05 2024-09-13T10:11:42.618 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.618 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.618 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.619 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215435+0000", 2024-09-13T10:11:42.619 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014800600000000001 2024-09-13T10:11:42.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.619 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.619 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.619 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215593+0000", 2024-09-13T10:11:42.619 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015864300000000001 2024-09-13T10:11:42.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.619 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.619 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:42.619 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226037+0000", 2024-09-13T10:11:42.620 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010443948 2024-09-13T10:11:42.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.620 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.620 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:42.620 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228312+0000", 2024-09-13T10:11:42.620 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022743009999999998 2024-09-13T10:11:42.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.620 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.620 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.620 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228708+0000", 2024-09-13T10:11:42.620 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039651999999999998 2024-09-13T10:11:42.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.621 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.621 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.621 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229625+0000", 2024-09-13T10:11:42.621 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00091692300000000004 2024-09-13T10:11:42.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.621 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.621 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:42.621 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229750+0000", 2024-09-13T10:11:42.621 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012532099999999999 2024-09-13T10:11:42.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.622 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.622 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.622 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229860+0000", 2024-09-13T10:11:42.622 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000109627 2024-09-13T10:11:42.622 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.622 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.622 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.622 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.622 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:260 4.3 4:c426503f:::smithi19530714-11:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:42.622 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:11:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 51.394704674000003, 2024-09-13T10:11:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041961871999999997, 2024-09-13T10:11:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 260 2024-09-13T10:11:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.623 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:11:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:11:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215336+0000", 2024-09-13T10:11:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9069999999999999e-06 2024-09-13T10:11:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215345+0000", 2024-09-13T10:11:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.6659999999999997e-06 2024-09-13T10:11:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.625 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215363+0000", 2024-09-13T10:11:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8153000000000001e-05 2024-09-13T10:11:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215388+0000", 2024-09-13T10:11:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4556e-05 2024-09-13T10:11:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.626 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215542+0000", 2024-09-13T10:11:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000154637 2024-09-13T10:11:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215825+0000", 2024-09-13T10:11:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028208600000000002 2024-09-13T10:11:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:42.627 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254572+0000", 2024-09-13T10:11:42.628 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.038747464000000002 2024-09-13T10:11:42.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.628 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.628 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:42.628 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.255694+0000", 2024-09-13T10:11:42.628 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011222929999999999 2024-09-13T10:11:42.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.628 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.628 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.629 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.256931+0000", 2024-09-13T10:11:42.629 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0012370980000000001 2024-09-13T10:11:42.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.629 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.629 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257080+0000", 2024-09-13T10:11:42.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014874800000000001 2024-09-13T10:11:42.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.630 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:42.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257211+0000", 2024-09-13T10:11:42.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000130674 2024-09-13T10:11:42.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.630 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257296+0000", 2024-09-13T10:11:42.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5589999999999999e-05 2024-09-13T10:11:42.631 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.631 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.631 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.631 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:269 4.3 4:c5b5ff75:::smithi19530714-20:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:42.631 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:11:42.631 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 51.394200435999998, 2024-09-13T10:11:42.631 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014988243, 2024-09-13T10:11:42.631 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.631 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.632 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:42.632 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.632 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.632 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.632 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 269 2024-09-13T10:11:42.632 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.632 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.632 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.632 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.632 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:11:42.632 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.633 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.633 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:11:42.633 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.633 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.633 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215841+0000", 2024-09-13T10:11:42.633 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0829999999999998e-06 2024-09-13T10:11:42.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.634 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.634 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.634 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215854+0000", 2024-09-13T10:11:42.634 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2976000000000001e-05 2024-09-13T10:11:42.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.634 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.634 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.634 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215875+0000", 2024-09-13T10:11:42.634 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1100000000000001e-05 2024-09-13T10:11:42.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.635 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.635 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215892+0000", 2024-09-13T10:11:42.635 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7388999999999999e-05 2024-09-13T10:11:42.635 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.635 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.635 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216353+0000", 2024-09-13T10:11:42.635 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00046100900000000002 2024-09-13T10:11:42.635 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.636 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.636 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216611+0000", 2024-09-13T10:11:42.636 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025716299999999998 2024-09-13T10:11:42.636 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.636 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.636 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:42.636 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.227177+0000", 2024-09-13T10:11:42.636 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010566676000000001 2024-09-13T10:11:42.636 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.636 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.636 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:42.637 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229574+0000", 2024-09-13T10:11:42.637 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023964469999999999 2024-09-13T10:11:42.637 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.637 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.637 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.637 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230619+0000", 2024-09-13T10:11:42.637 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0010455950000000001 2024-09-13T10:11:42.637 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.637 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.637 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.638 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230697+0000", 2024-09-13T10:11:42.638 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8021000000000004e-05 2024-09-13T10:11:42.638 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.638 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.638 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:42.638 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230767+0000", 2024-09-13T10:11:42.638 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9544999999999999e-05 2024-09-13T10:11:42.638 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.638 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.639 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.639 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230827+0000", 2024-09-13T10:11:42.639 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.0238999999999997e-05 2024-09-13T10:11:42.639 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.639 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.639 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.639 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.639 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.639 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:288 4.3 4:cceea2a7:::smithi19530714-39:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:42.639 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.249445+0000", 2024-09-13T10:11:42.639 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 51.360594145, 2024-09-13T10:11:42.639 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012024603, 2024-09-13T10:11:42.640 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.640 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.640 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:42.640 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.640 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.640 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.640 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 288 2024-09-13T10:11:42.640 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.640 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.640 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.640 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.641 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249445+0000", 2024-09-13T10:11:42.641 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.641 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.641 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.641 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249445+0000", 2024-09-13T10:11:42.641 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.641 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.641 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.642 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249448+0000", 2024-09-13T10:11:42.642 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0350000000000002e-06 2024-09-13T10:11:42.642 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.642 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.642 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.642 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249460+0000", 2024-09-13T10:11:42.642 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2214e-05 2024-09-13T10:11:42.642 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.642 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.642 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.642 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249490+0000", 2024-09-13T10:11:42.643 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0130000000000001e-05 2024-09-13T10:11:42.643 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.643 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.643 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.643 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249510+0000", 2024-09-13T10:11:42.643 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9757000000000001e-05 2024-09-13T10:11:42.643 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.643 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.643 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.643 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249684+0000", 2024-09-13T10:11:42.643 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000174086 2024-09-13T10:11:42.643 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.644 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.644 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.644 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.249913+0000", 2024-09-13T10:11:42.644 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022833400000000001 2024-09-13T10:11:42.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.644 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.644 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:42.644 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257665+0000", 2024-09-13T10:11:42.644 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0077526330000000001 2024-09-13T10:11:42.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.644 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.645 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.645 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.259621+0000", 2024-09-13T10:11:42.645 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019558610000000001 2024-09-13T10:11:42.645 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.645 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.645 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.645 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.260048+0000", 2024-09-13T10:11:42.645 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00042668799999999999 2024-09-13T10:11:42.645 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.645 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.645 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:42.646 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.261264+0000", 2024-09-13T10:11:42.646 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001216445 2024-09-13T10:11:42.646 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.646 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.646 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:42.646 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.261374+0000", 2024-09-13T10:11:42.646 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110117 2024-09-13T10:11:42.646 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.646 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.646 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.646 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.261470+0000", 2024-09-13T10:11:42.647 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5303000000000006e-05 2024-09-13T10:11:42.647 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.647 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.647 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.647 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.647 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.647 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:310 4.3 4:c426503f:::smithi19530714-11:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:42.647 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:11:42.647 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 51.336904363000002, 2024-09-13T10:11:42.647 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013782279999999999, 2024-09-13T10:11:42.647 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.647 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.648 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:42.648 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.648 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.648 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.648 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 310 2024-09-13T10:11:42.648 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.648 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.648 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.648 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.648 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:11:42.648 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.649 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.649 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.649 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.649 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:11:42.649 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.649 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.649 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.649 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.649 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273138+0000", 2024-09-13T10:11:42.649 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8459999999999999e-06 2024-09-13T10:11:42.649 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.649 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.650 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.650 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273150+0000", 2024-09-13T10:11:42.650 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2133999999999999e-05 2024-09-13T10:11:42.650 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.650 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.650 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.650 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273175+0000", 2024-09-13T10:11:42.650 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5321999999999999e-05 2024-09-13T10:11:42.650 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.650 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.650 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.651 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273192+0000", 2024-09-13T10:11:42.651 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.681e-05 2024-09-13T10:11:42.651 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.651 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.651 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.651 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273314+0000", 2024-09-13T10:11:42.651 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012228 2024-09-13T10:11:42.651 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.651 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.651 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.651 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273474+0000", 2024-09-13T10:11:42.652 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016019799999999999 2024-09-13T10:11:42.652 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.652 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.652 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:42.652 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.284136+0000", 2024-09-13T10:11:42.652 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010661761000000001 2024-09-13T10:11:42.652 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.652 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.652 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.652 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286085+0000", 2024-09-13T10:11:42.652 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019484929999999999 2024-09-13T10:11:42.653 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.653 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.653 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:42.653 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286111+0000", 2024-09-13T10:11:42.653 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5834e-05 2024-09-13T10:11:42.653 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.653 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.653 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.653 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286721+0000", 2024-09-13T10:11:42.653 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00061038599999999998 2024-09-13T10:11:42.653 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.654 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.654 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:42.654 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286838+0000", 2024-09-13T10:11:42.654 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117064 2024-09-13T10:11:42.654 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.654 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.654 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.654 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286917+0000", 2024-09-13T10:11:42.654 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9152000000000004e-05 2024-09-13T10:11:42.654 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.654 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.655 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.655 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.655 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1006 4.3 4:c5b5ff75:::smithi19530714-20:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:42.655 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:11:42.655 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 46.955991115000003, 2024-09-13T10:11:42.655 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.047764635, 2024-09-13T10:11:42.655 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.655 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.656 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:42.656 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.656 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.656 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.656 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1006 2024-09-13T10:11:42.656 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.656 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.656 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.656 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.656 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:11:42.656 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.657 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.657 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.657 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.657 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:11:42.657 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.657 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.657 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.657 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.657 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654049+0000", 2024-09-13T10:11:42.657 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5600000000000005e-07 2024-09-13T10:11:42.657 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.658 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.658 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.658 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654058+0000", 2024-09-13T10:11:42.658 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8200000000000003e-06 2024-09-13T10:11:42.658 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.658 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.658 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.658 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654067+0000", 2024-09-13T10:11:42.658 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.4599999999999992e-06 2024-09-13T10:11:42.658 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.658 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.659 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.659 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654075+0000", 2024-09-13T10:11:42.659 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.1070000000000005e-06 2024-09-13T10:11:42.659 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.659 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.659 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.659 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654188+0000", 2024-09-13T10:11:42.659 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000112734 2024-09-13T10:11:42.659 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.659 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.659 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.659 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654395+0000", 2024-09-13T10:11:42.660 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000207235 2024-09-13T10:11:42.660 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.660 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.660 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.660 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.701813+0000", 2024-09-13T10:11:42.660 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.047417523000000003 2024-09-13T10:11:42.660 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.660 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.660 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.660 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.660 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.661 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1013 4.3 4:c426503f:::smithi19530714-11:head [write 477332~766101] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:42.661 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:11:42.661 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 46.948669860999999, 2024-09-13T10:11:42.661 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027363838000000001, 2024-09-13T10:11:42.661 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.661 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.661 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:42.661 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.661 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.661 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.661 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1013 2024-09-13T10:11:42.662 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.662 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.662 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.662 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.662 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:11:42.662 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.662 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.662 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.662 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.662 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:11:42.662 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.662 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.663 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.663 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.663 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661374+0000", 2024-09-13T10:11:42.663 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.0930000000000002e-06 2024-09-13T10:11:42.663 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.663 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.663 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.663 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665828+0000", 2024-09-13T10:11:42.663 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044532119999999998 2024-09-13T10:11:42.663 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.663 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.663 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.664 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665864+0000", 2024-09-13T10:11:42.664 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5979000000000002e-05 2024-09-13T10:11:42.664 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.664 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.664 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.664 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665881+0000", 2024-09-13T10:11:42.664 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7836e-05 2024-09-13T10:11:42.664 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.664 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.664 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.664 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.678976+0000", 2024-09-13T10:11:42.665 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.01309448 2024-09-13T10:11:42.665 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.665 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.665 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.665 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.679188+0000", 2024-09-13T10:11:42.665 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000211945 2024-09-13T10:11:42.665 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.665 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.665 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:42.665 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.679468+0000", 2024-09-13T10:11:42.665 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028034000000000002 2024-09-13T10:11:42.666 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.666 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.666 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:42.666 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688233+0000", 2024-09-13T10:11:42.666 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0087645369999999993 2024-09-13T10:11:42.666 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.666 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.666 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.666 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688388+0000", 2024-09-13T10:11:42.666 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015488000000000001 2024-09-13T10:11:42.666 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.666 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.667 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.667 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688505+0000", 2024-09-13T10:11:42.667 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117114 2024-09-13T10:11:42.667 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.667 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.667 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:42.667 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688612+0000", 2024-09-13T10:11:42.667 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010741100000000001 2024-09-13T10:11:42.667 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.667 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.667 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.668 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688733+0000", 2024-09-13T10:11:42.668 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000121011 2024-09-13T10:11:42.668 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.668 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.668 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.668 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.668 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.668 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1014 4.3 4:c426503f:::smithi19530714-11:head [write 1681102~474010] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:42.668 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:11:42.668 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 46.943396475, 2024-09-13T10:11:42.668 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022694374, 2024-09-13T10:11:42.669 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.669 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.669 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:42.669 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.669 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.669 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.669 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1014 2024-09-13T10:11:42.669 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.669 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.670 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.670 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.670 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:11:42.670 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.670 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.670 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.670 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.670 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:11:42.670 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.670 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.671 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.671 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.671 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666645+0000", 2024-09-13T10:11:42.671 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2970000000000002e-06 2024-09-13T10:11:42.671 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.671 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.671 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.671 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667326+0000", 2024-09-13T10:11:42.671 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00068073199999999997 2024-09-13T10:11:42.671 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.671 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.672 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.672 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667346+0000", 2024-09-13T10:11:42.672 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0188999999999999e-05 2024-09-13T10:11:42.672 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.672 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.672 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.672 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667361+0000", 2024-09-13T10:11:42.672 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5299999999999999e-05 2024-09-13T10:11:42.672 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.672 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.673 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.673 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.683468+0000", 2024-09-13T10:11:42.673 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016106944000000002 2024-09-13T10:11:42.673 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.673 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.673 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.673 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.683714+0000", 2024-09-13T10:11:42.673 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024595699999999999 2024-09-13T10:11:42.673 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.673 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.673 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:42.673 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.684132+0000", 2024-09-13T10:11:42.674 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00041775800000000001 2024-09-13T10:11:42.674 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.674 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.674 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:42.674 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688771+0000", 2024-09-13T10:11:42.674 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0046394540000000003 2024-09-13T10:11:42.674 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.674 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.674 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.674 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688920+0000", 2024-09-13T10:11:42.674 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014886000000000001 2024-09-13T10:11:42.675 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.675 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.675 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.675 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689085+0000", 2024-09-13T10:11:42.675 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000164539 2024-09-13T10:11:42.675 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.675 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.675 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:42.675 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689230+0000", 2024-09-13T10:11:42.675 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000145394 2024-09-13T10:11:42.675 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.676 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.676 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.676 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689337+0000", 2024-09-13T10:11:42.676 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010695 2024-09-13T10:11:42.676 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.676 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.676 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.676 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.676 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.676 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1015 4.3 4:c426503f:::smithi19530714-11:head [write 2687912~589420] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:42.677 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:11:42.677 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 46.937079797999999, 2024-09-13T10:11:42.677 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.018489873, 2024-09-13T10:11:42.677 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.677 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.677 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:42.677 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.677 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.677 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.677 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1015 2024-09-13T10:11:42.677 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.678 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.678 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.678 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.678 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:11:42.678 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.678 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.678 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.678 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.678 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:11:42.678 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.678 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.679 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.679 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.679 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672962+0000", 2024-09-13T10:11:42.679 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.277e-06 2024-09-13T10:11:42.679 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.679 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.679 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.679 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673758+0000", 2024-09-13T10:11:42.679 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00079599100000000002 2024-09-13T10:11:42.679 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.679 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.679 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.680 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673787+0000", 2024-09-13T10:11:42.680 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9280000000000001e-05 2024-09-13T10:11:42.680 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.680 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.680 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.680 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673801+0000", 2024-09-13T10:11:42.680 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4493e-05 2024-09-13T10:11:42.680 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.680 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.680 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.680 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.685885+0000", 2024-09-13T10:11:42.681 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012083692 2024-09-13T10:11:42.681 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.681 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.681 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.681 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.686057+0000", 2024-09-13T10:11:42.681 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017133900000000001 2024-09-13T10:11:42.681 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.681 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.681 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:42.681 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.686396+0000", 2024-09-13T10:11:42.681 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000339809 2024-09-13T10:11:42.682 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.682 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.682 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.682 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691040+0000", 2024-09-13T10:11:42.682 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.004643215 2024-09-13T10:11:42.682 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.682 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.682 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:42.682 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691082+0000", 2024-09-13T10:11:42.682 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.2499000000000001e-05 2024-09-13T10:11:42.682 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.683 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.683 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.683 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691179+0000", 2024-09-13T10:11:42.683 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7044000000000001e-05 2024-09-13T10:11:42.683 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.683 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.683 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:42.683 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691324+0000", 2024-09-13T10:11:42.683 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014532299999999999 2024-09-13T10:11:42.683 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.683 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.684 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.684 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691449+0000", 2024-09-13T10:11:42.684 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012491100000000001 2024-09-13T10:11:42.684 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.684 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.684 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.684 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.684 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.684 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1016 4.3 4:c426503f:::smithi19530714-11:head [setxattr _header (56) in=7b,truncate 3277332] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:42.684 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:11:42.685 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 46.936210840999998, 2024-09-13T10:11:42.685 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.019820562999999999, 2024-09-13T10:11:42.685 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.685 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.685 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:42.685 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.685 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.685 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.685 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1016 2024-09-13T10:11:42.685 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.686 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.686 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.686 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.686 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:11:42.686 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.686 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.686 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.686 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.686 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:11:42.686 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.686 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.687 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.687 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.687 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673830+0000", 2024-09-13T10:11:42.687 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1170000000000001e-06 2024-09-13T10:11:42.687 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.687 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.687 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.687 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673835+0000", 2024-09-13T10:11:42.687 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5559999999999998e-06 2024-09-13T10:11:42.687 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.687 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.688 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.688 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673845+0000", 2024-09-13T10:11:42.688 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6330000000000008e-06 2024-09-13T10:11:42.688 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.688 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.688 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.688 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673856+0000", 2024-09-13T10:11:42.688 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0963e-05 2024-09-13T10:11:42.688 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.688 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.688 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.689 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689480+0000", 2024-09-13T10:11:42.689 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015624605 2024-09-13T10:11:42.689 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.689 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.689 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.689 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689739+0000", 2024-09-13T10:11:42.689 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025878700000000001 2024-09-13T10:11:42.689 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.689 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.689 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:42.689 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.690154+0000", 2024-09-13T10:11:42.690 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00041533800000000001 2024-09-13T10:11:42.690 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.690 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.690 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.690 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691858+0000", 2024-09-13T10:11:42.690 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001703811 2024-09-13T10:11:42.690 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.690 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.690 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:42.690 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691880+0000", 2024-09-13T10:11:42.690 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1997e-05 2024-09-13T10:11:42.691 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.691 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.691 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.691 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693384+0000", 2024-09-13T10:11:42.691 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0015040190000000001 2024-09-13T10:11:42.691 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.691 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.691 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:42.691 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693541+0000", 2024-09-13T10:11:42.691 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015723899999999999 2024-09-13T10:11:42.691 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.692 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.692 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.692 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693649+0000", 2024-09-13T10:11:42.692 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010749800000000001 2024-09-13T10:11:42.692 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.692 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.692 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.692 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.692 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.692 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1017 4.3 4:c426503f:::smithi19530714-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:42.692 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:11:42.693 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 46.936170828999998, 2024-09-13T10:11:42.693 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020166488999999999, 2024-09-13T10:11:42.693 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.693 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.693 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:42.693 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.693 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.693 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.693 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1017 2024-09-13T10:11:42.694 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.694 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.694 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.694 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.694 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:11:42.694 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.694 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.694 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.694 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.694 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:11:42.694 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.695 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.695 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.695 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.695 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673869+0000", 2024-09-13T10:11:42.695 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9699999999999995e-07 2024-09-13T10:11:42.695 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.695 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.695 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.695 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673873+0000", 2024-09-13T10:11:42.695 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.4970000000000001e-06 2024-09-13T10:11:42.695 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.695 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.696 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.696 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673885+0000", 2024-09-13T10:11:42.696 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2398999999999999e-05 2024-09-13T10:11:42.696 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.696 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.696 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.696 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673893+0000", 2024-09-13T10:11:42.696 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7530000000000001e-06 2024-09-13T10:11:42.696 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.696 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.697 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.697 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691638+0000", 2024-09-13T10:11:42.697 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017745007 2024-09-13T10:11:42.697 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.697 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.697 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:42.697 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691789+0000", 2024-09-13T10:11:42.697 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015133400000000001 2024-09-13T10:11:42.697 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.697 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.697 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.698 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693731+0000", 2024-09-13T10:11:42.698 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019422459999999999 2024-09-13T10:11:42.698 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.698 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.698 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.698 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693870+0000", 2024-09-13T10:11:42.698 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013820299999999999 2024-09-13T10:11:42.698 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.698 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.698 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.698 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.694035+0000", 2024-09-13T10:11:42.698 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000165253 2024-09-13T10:11:42.699 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.699 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.699 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.699 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.699 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.699 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1405 4.3 4:c5b5ff75:::smithi19530714-20:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T10:11:42.699 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:01.492787+0000", 2024-09-13T10:11:42.699 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 41.117252184999998, 2024-09-13T10:11:42.699 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021225698000000001, 2024-09-13T10:11:42.700 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.700 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.700 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:42.700 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.700 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.700 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.700 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1405 2024-09-13T10:11:42.700 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.700 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.700 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.700 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.701 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492787+0000", 2024-09-13T10:11:42.701 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.701 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.701 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.701 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.701 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492787+0000", 2024-09-13T10:11:42.701 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.701 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.701 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.701 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.702 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492791+0000", 2024-09-13T10:11:42.702 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.89e-06 2024-09-13T10:11:42.702 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.702 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.702 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.702 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492805+0000", 2024-09-13T10:11:42.702 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4083000000000001e-05 2024-09-13T10:11:42.702 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.702 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.702 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.702 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492846+0000", 2024-09-13T10:11:42.703 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0849000000000001e-05 2024-09-13T10:11:42.703 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.703 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.703 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.703 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492867+0000", 2024-09-13T10:11:42.703 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1058999999999999e-05 2024-09-13T10:11:42.703 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.703 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.703 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.703 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492973+0000", 2024-09-13T10:11:42.703 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000106084 2024-09-13T10:11:42.703 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.704 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.704 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.704 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.493105+0000", 2024-09-13T10:11:42.704 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013215899999999999 2024-09-13T10:11:42.704 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.704 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.704 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.704 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.514013+0000", 2024-09-13T10:11:42.704 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020907574000000002 2024-09-13T10:11:42.704 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.704 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.705 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.705 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.705 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.705 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2658 4.3 4:c5b5ff75:::smithi19530714-20:head [setxattr _header (56) in=7b,truncate 3899224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:42.705 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:28.049623+0000", 2024-09-13T10:11:42.705 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 14.560415918, 2024-09-13T10:11:42.705 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012241349, 2024-09-13T10:11:42.705 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.705 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.705 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:42.705 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.706 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.706 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.706 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2658 2024-09-13T10:11:42.706 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.706 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.706 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.706 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.706 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049623+0000", 2024-09-13T10:11:42.706 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.706 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.707 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.707 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.707 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049623+0000", 2024-09-13T10:11:42.707 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.707 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.707 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.707 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.707 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049625+0000", 2024-09-13T10:11:42.707 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9439999999999999e-06 2024-09-13T10:11:42.707 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.707 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.707 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.708 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049635+0000", 2024-09-13T10:11:42.708 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5589999999999994e-06 2024-09-13T10:11:42.708 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.708 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.708 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.708 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049652+0000", 2024-09-13T10:11:42.708 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.751e-05 2024-09-13T10:11:42.708 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.708 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.708 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.708 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049668+0000", 2024-09-13T10:11:42.709 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5214999999999999e-05 2024-09-13T10:11:42.709 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.709 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.709 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.709 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.052042+0000", 2024-09-13T10:11:42.709 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023748350000000001 2024-09-13T10:11:42.709 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.709 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.709 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.709 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.052248+0000", 2024-09-13T10:11:42.709 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000205651 2024-09-13T10:11:42.710 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.710 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.710 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:42.710 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.052639+0000", 2024-09-13T10:11:42.710 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039102000000000001 2024-09-13T10:11:42.710 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.710 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.710 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.710 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.054795+0000", 2024-09-13T10:11:42.710 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002155587 2024-09-13T10:11:42.710 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.710 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.061492+0000", 2024-09-13T10:11:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0066973409999999999 2024-09-13T10:11:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.061539+0000", 2024-09-13T10:11:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.7515000000000002e-05 2024-09-13T10:11:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.711 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.061691+0000", 2024-09-13T10:11:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015110300000000001 2024-09-13T10:11:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.061865+0000", 2024-09-13T10:11:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000174069 2024-09-13T10:11:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.712 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2659 4.3 4:c5b5ff75:::smithi19530714-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:28.049693+0000", 2024-09-13T10:11:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 14.560346260999999, 2024-09-13T10:11:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012500902, 2024-09-13T10:11:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.713 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2659 2024-09-13T10:11:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049693+0000", 2024-09-13T10:11:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049693+0000", 2024-09-13T10:11:42.714 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049694+0000", 2024-09-13T10:11:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3740000000000001e-06 2024-09-13T10:11:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049701+0000", 2024-09-13T10:11:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.5699999999999998e-06 2024-09-13T10:11:42.715 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049717+0000", 2024-09-13T10:11:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5795e-05 2024-09-13T10:11:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.049729+0000", 2024-09-13T10:11:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2153e-05 2024-09-13T10:11:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.716 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.053546+0000", 2024-09-13T10:11:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0038173769999999998 2024-09-13T10:11:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.053767+0000", 2024-09-13T10:11:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022096099999999999 2024-09-13T10:11:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.717 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.061937+0000", 2024-09-13T10:11:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0081695359999999998 2024-09-13T10:11:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.062061+0000", 2024-09-13T10:11:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012419800000000001 2024-09-13T10:11:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.718 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:28.062194+0000", 2024-09-13T10:11:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013293800000000001 2024-09-13T10:11:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2816 4.3 4:c5b5ff75:::smithi19530714-20:head [assert-version v76,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.891848+0000", 2024-09-13T10:11:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.7181911599999999, 2024-09-13T10:11:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020571445000000001, 2024-09-13T10:11:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.719 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2816 2024-09-13T10:11:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891848+0000", 2024-09-13T10:11:42.720 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891848+0000", 2024-09-13T10:11:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891852+0000", 2024-09-13T10:11:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1729999999999997e-06 2024-09-13T10:11:42.721 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891869+0000", 2024-09-13T10:11:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6639000000000001e-05 2024-09-13T10:11:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891910+0000", 2024-09-13T10:11:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1125999999999999e-05 2024-09-13T10:11:42.722 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891934+0000", 2024-09-13T10:11:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4039000000000001e-05 2024-09-13T10:11:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892099+0000", 2024-09-13T10:11:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016462000000000001 2024-09-13T10:11:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.723 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.724 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.724 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892368+0000", 2024-09-13T10:11:42.724 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026922699999999998 2024-09-13T10:11:42.724 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.724 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.724 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:42.724 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.907670+0000", 2024-09-13T10:11:42.724 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015302299 2024-09-13T10:11:42.724 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.724 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.724 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.725 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.910850+0000", 2024-09-13T10:11:42.725 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003179383 2024-09-13T10:11:42.725 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.725 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.725 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:42.725 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.910901+0000", 2024-09-13T10:11:42.725 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.1548000000000002e-05 2024-09-13T10:11:42.725 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.725 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.725 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:42.726 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.912089+0000", 2024-09-13T10:11:42.726 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011878170000000001 2024-09-13T10:11:42.726 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.726 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.726 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:42.726 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.912267+0000", 2024-09-13T10:11:42.726 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000178393 2024-09-13T10:11:42.726 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.726 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.726 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.727 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.912420+0000", 2024-09-13T10:11:42.727 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015218100000000001 2024-09-13T10:11:42.727 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.727 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.727 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.727 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.727 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.727 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2817 4.3 4:c426503f:::smithi19530714-11:head [read 0~3647119,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:42.727 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.892018+0000", 2024-09-13T10:11:42.727 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.7180216900000005, 2024-09-13T10:11:42.727 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013334828, 2024-09-13T10:11:42.728 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.728 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.728 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:42.728 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.728 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.728 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.728 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2817 2024-09-13T10:11:42.728 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.728 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.728 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.728 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.729 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892018+0000", 2024-09-13T10:11:42.729 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.729 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.729 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.729 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.729 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892018+0000", 2024-09-13T10:11:42.729 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.729 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.729 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.729 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.729 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892019+0000", 2024-09-13T10:11:42.730 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.747e-06 2024-09-13T10:11:42.730 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.730 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.730 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.730 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892031+0000", 2024-09-13T10:11:42.730 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.149e-05 2024-09-13T10:11:42.730 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.730 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.730 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.730 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892053+0000", 2024-09-13T10:11:42.730 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2069000000000002e-05 2024-09-13T10:11:42.731 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.731 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.731 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.731 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892073+0000", 2024-09-13T10:11:42.731 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0591999999999999e-05 2024-09-13T10:11:42.731 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.731 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.731 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.731 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892825+0000", 2024-09-13T10:11:42.731 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00075120299999999998 2024-09-13T10:11:42.731 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.731 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.732 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.732 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893031+0000", 2024-09-13T10:11:42.732 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000206579 2024-09-13T10:11:42.732 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.732 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.732 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.732 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.905352+0000", 2024-09-13T10:11:42.732 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012321148000000001 2024-09-13T10:11:42.732 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.732 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.732 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.733 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.733 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.733 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2818 4.3 4:c426503f:::smithi19530714-11:head [sparse-read 0~2823261] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:42.733 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.892118+0000", 2024-09-13T10:11:42.733 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.7179208999999993, 2024-09-13T10:11:42.733 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014982577, 2024-09-13T10:11:42.733 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.733 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.733 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:42.733 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.733 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.734 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.734 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2818 2024-09-13T10:11:42.734 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.734 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.734 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.734 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.734 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892118+0000", 2024-09-13T10:11:42.734 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.734 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.734 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.734 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.735 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892118+0000", 2024-09-13T10:11:42.735 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.735 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.735 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.735 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.735 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892120+0000", 2024-09-13T10:11:42.735 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5260000000000001e-06 2024-09-13T10:11:42.735 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.735 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.735 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.735 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892128+0000", 2024-09-13T10:11:42.736 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7640000000000005e-06 2024-09-13T10:11:42.736 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.736 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.736 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.736 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892172+0000", 2024-09-13T10:11:42.736 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4561000000000002e-05 2024-09-13T10:11:42.736 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.736 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.736 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.736 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892231+0000", 2024-09-13T10:11:42.736 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9194999999999998e-05 2024-09-13T10:11:42.737 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.737 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.737 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.737 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.903452+0000", 2024-09-13T10:11:42.737 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011221024 2024-09-13T10:11:42.737 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.737 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.737 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.737 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.903711+0000", 2024-09-13T10:11:42.737 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025891100000000002 2024-09-13T10:11:42.737 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.738 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.738 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.738 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.907101+0000", 2024-09-13T10:11:42.738 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0033895959999999999 2024-09-13T10:11:42.738 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.738 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.738 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.738 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.738 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.738 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2819 4.3 4:c426503f:::smithi19530714-11:head [sparse-read 1467121~1802604] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:42.738 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.892259+0000", 2024-09-13T10:11:42.739 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.7177798919999994, 2024-09-13T10:11:42.739 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.018443480000000002, 2024-09-13T10:11:42.739 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.739 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.739 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:42.739 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.739 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.739 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.739 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2819 2024-09-13T10:11:42.739 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.739 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.740 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.740 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.740 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892259+0000", 2024-09-13T10:11:42.740 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.740 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.740 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.740 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.740 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892259+0000", 2024-09-13T10:11:42.740 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.740 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.740 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.741 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.741 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892261+0000", 2024-09-13T10:11:42.741 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5200000000000001e-06 2024-09-13T10:11:42.741 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.741 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.741 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.741 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892287+0000", 2024-09-13T10:11:42.741 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5976e-05 2024-09-13T10:11:42.741 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.741 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.741 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.741 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892306+0000", 2024-09-13T10:11:42.742 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9208999999999999e-05 2024-09-13T10:11:42.742 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.742 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.742 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.742 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892322+0000", 2024-09-13T10:11:42.742 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6426000000000002e-05 2024-09-13T10:11:42.742 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.742 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.742 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.742 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.908963+0000", 2024-09-13T10:11:42.742 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016640922999999998 2024-09-13T10:11:42.743 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.743 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.743 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.743 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.909202+0000", 2024-09-13T10:11:42.744 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023879500000000001 2024-09-13T10:11:42.745 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.745 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.745 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.745 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.910703+0000", 2024-09-13T10:11:42.745 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0015006310000000001 2024-09-13T10:11:42.745 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.746 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.746 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.746 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.746 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.746 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3077 4.3 4:c426503f:::smithi19530714-11:head [read 0~2487469,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T10:11:42.746 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:38.708637+0000", 2024-09-13T10:11:42.746 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 3.9014026070000001, 2024-09-13T10:11:42.746 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011733501, 2024-09-13T10:11:42.747 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.747 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.747 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:42.747 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.747 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.747 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.747 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3077 2024-09-13T10:11:42.747 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.747 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.747 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.747 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.747 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708637+0000", 2024-09-13T10:11:42.748 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.748 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.748 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.748 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.748 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708637+0000", 2024-09-13T10:11:42.748 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.748 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.749 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.749 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.749 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708638+0000", 2024-09-13T10:11:42.749 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2640000000000001e-06 2024-09-13T10:11:42.749 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.749 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.749 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.749 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708644+0000", 2024-09-13T10:11:42.750 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4359999999999999e-06 2024-09-13T10:11:42.750 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.750 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.750 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.750 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708655+0000", 2024-09-13T10:11:42.750 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.11e-05 2024-09-13T10:11:42.750 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.750 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.751 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.751 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708664+0000", 2024-09-13T10:11:42.751 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.653e-06 2024-09-13T10:11:42.751 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.751 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.751 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.751 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.717974+0000", 2024-09-13T10:11:42.751 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0093095160000000003 2024-09-13T10:11:42.751 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.751 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.752 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.752 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.718139+0000", 2024-09-13T10:11:42.752 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016553100000000001 2024-09-13T10:11:42.752 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.752 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.752 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.752 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.720370+0000", 2024-09-13T10:11:42.752 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022310009999999998 2024-09-13T10:11:42.752 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.752 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.752 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.753 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.753 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.753 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3078 4.3 4:c426503f:::smithi19530714-11:head [read 0~2487469,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T10:11:42.753 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:38.708725+0000", 2024-09-13T10:11:42.753 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 3.901314293, 2024-09-13T10:11:42.753 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013854750000000001, 2024-09-13T10:11:42.753 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:42.753 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:42.753 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:42.753 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:42.753 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:42.754 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:42.754 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3078 2024-09-13T10:11:42.754 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.754 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:42.754 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.754 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:42.754 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708725+0000", 2024-09-13T10:11:42.754 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.754 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.754 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.754 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:42.754 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708725+0000", 2024-09-13T10:11:42.755 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:42.755 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.755 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.755 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:42.755 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708726+0000", 2024-09-13T10:11:42.755 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.08e-06 2024-09-13T10:11:42.755 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.755 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.755 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:42.756 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708731+0000", 2024-09-13T10:11:42.756 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.3029999999999999e-06 2024-09-13T10:11:42.756 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.756 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.756 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:42.757 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708741+0000", 2024-09-13T10:11:42.757 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0077e-05 2024-09-13T10:11:42.757 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.757 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.757 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:42.757 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708750+0000", 2024-09-13T10:11:42.757 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5560000000000008e-06 2024-09-13T10:11:42.758 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.758 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.758 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:42.758 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.720444+0000", 2024-09-13T10:11:42.758 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011694334000000001 2024-09-13T10:11:42.758 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.758 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.758 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:42.758 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.720692+0000", 2024-09-13T10:11:42.759 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024811599999999999 2024-09-13T10:11:42.759 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:42.759 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:42.759 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:42.759 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.722580+0000", 2024-09-13T10:11:42.759 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0018872839999999999 2024-09-13T10:11:42.759 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.759 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.760 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.760 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:42.760 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:42.760 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:42.761 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:11:42.761 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:11:42.761 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:11:42.761 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2024-09-13T10:11:42.762 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:11:42.764 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:42.765 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:42.765 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1801652 2024-09-13T10:11:42.765 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:42.765 INFO:tasks.rados.rados.0.smithi195.stdout:1430: finishing write tid 1 to smithi19530714-17 2024-09-13T10:11:42.766 INFO:tasks.rados.rados.0.smithi195.stdout:1430: finishing write tid 2 to smithi19530714-17 2024-09-13T10:11:42.766 INFO:tasks.rados.rados.0.smithi195.stdout:1430: finishing write tid 3 to smithi19530714-17 2024-09-13T10:11:42.766 INFO:tasks.rados.rados.0.smithi195.stdout:1430: finishing write tid 4 to smithi19530714-17 2024-09-13T10:11:42.766 INFO:tasks.rados.rados.0.smithi195.stdout:1430: finishing write tid 5 to smithi19530714-17 2024-09-13T10:11:42.766 INFO:tasks.rados.rados.0.smithi195.stdout:1430: oid 17 updating version 0 to 74 2024-09-13T10:11:42.766 INFO:tasks.rados.rados.0.smithi195.stdout:1430: oid 17 version 74 is already newer than 71 2024-09-13T10:11:42.766 INFO:tasks.rados.rados.0.smithi195.stdout:1430: oid 17 version 74 is already newer than 73 2024-09-13T10:11:42.766 INFO:tasks.rados.rados.0.smithi195.stdout:1430: oid 17 version 74 is already newer than 72 2024-09-13T10:11:42.767 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 74 (ObjNum 289 snap 0 seq_num 289) dirty exists 2024-09-13T10:11:42.767 INFO:tasks.rados.rados.0.smithi195.stdout:1430: left oid 17 (ObjNum 289 snap 0 seq_num 289) 2024-09-13T10:11:42.767 INFO:tasks.rados.rados.0.smithi195.stdout:1431: finishing set_chunk to oid 31 2024-09-13T10:11:42.767 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 81 (ObjNum 272 snap 0 seq_num 272) dirty exists 2024-09-13T10:11:42.767 INFO:tasks.rados.rados.0.smithi195.stdout:1432: finishing write tid 1 to smithi19530714-38 2024-09-13T10:11:42.767 INFO:tasks.rados.rados.0.smithi195.stdout:1432: finishing write tid 2 to smithi19530714-38 2024-09-13T10:11:42.767 INFO:tasks.rados.rados.0.smithi195.stdout:1432: finishing write tid 3 to smithi19530714-38 2024-09-13T10:11:42.767 INFO:tasks.rados.rados.0.smithi195.stdout:1432: finishing write tid 4 to smithi19530714-38 2024-09-13T10:11:42.768 INFO:tasks.rados.rados.0.smithi195.stdout:1432: oid 38 updating version 0 to 135 2024-09-13T10:11:42.768 INFO:tasks.rados.rados.0.smithi195.stdout:1432: oid 38 updating version 135 to 136 2024-09-13T10:11:42.768 INFO:tasks.rados.rados.0.smithi195.stdout:1432: oid 38 version 136 is already newer than 134 2024-09-13T10:11:42.768 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 136 (ObjNum 290 snap 0 seq_num 290) dirty exists 2024-09-13T10:11:42.768 INFO:tasks.rados.rados.0.smithi195.stdout:1432: left oid 38 (ObjNum 290 snap 0 seq_num 290) 2024-09-13T10:11:42.768 INFO:tasks.rados.rados.0.smithi195.stdout:1433: expect (ObjNum 181 snap 0 seq_num 181) 2024-09-13T10:11:42.768 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:42.768 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:42.769 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:42.770 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T10:11:42.770 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:42.758+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:42.771 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:42.819 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 478960 2024-09-13T10:11:42.863 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:42.858+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:42.879 INFO:tasks.rados.rados.0.smithi195.stdout:1434: finishing write tid 1 to smithi19530714-46 2024-09-13T10:11:42.879 INFO:tasks.rados.rados.0.smithi195.stdout:1434: finishing write tid 2 to smithi19530714-46 2024-09-13T10:11:42.880 INFO:tasks.rados.rados.0.smithi195.stdout:1434: finishing write tid 3 to smithi19530714-46 2024-09-13T10:11:42.880 INFO:tasks.rados.rados.0.smithi195.stdout:1434: finishing write tid 4 to smithi19530714-46 2024-09-13T10:11:42.880 INFO:tasks.rados.rados.0.smithi195.stdout:1434: finishing write tid 5 to smithi19530714-46 2024-09-13T10:11:42.880 INFO:tasks.rados.rados.0.smithi195.stdout:1434: oid 46 updating version 0 to 41 2024-09-13T10:11:42.880 INFO:tasks.rados.rados.0.smithi195.stdout:1434: oid 46 updating version 41 to 42 2024-09-13T10:11:42.880 INFO:tasks.rados.rados.0.smithi195.stdout:1434: oid 46 updating version 42 to 44 2024-09-13T10:11:42.880 INFO:tasks.rados.rados.0.smithi195.stdout:1434: oid 46 version 44 is already newer than 43 2024-09-13T10:11:42.880 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 44 (ObjNum 291 snap 0 seq_num 291) dirty exists 2024-09-13T10:11:42.880 INFO:tasks.rados.rados.0.smithi195.stdout:1434: left oid 46 (ObjNum 291 snap 0 seq_num 291) 2024-09-13T10:11:42.880 INFO:tasks.rados.rados.0.smithi195.stdout:1436: expect (ObjNum 41 snap 0 seq_num 41) 2024-09-13T10:11:42.880 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:42.942 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:42.942 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:42.942 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:11:42.942 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:42.955 INFO:tasks.rados.rados.0.smithi195.stdout:1422: done (15 left) 2024-09-13T10:11:42.956 INFO:tasks.rados.rados.0.smithi195.stdout:1423: done (14 left) 2024-09-13T10:11:42.956 INFO:tasks.rados.rados.0.smithi195.stdout:1424: done (13 left) 2024-09-13T10:11:42.956 INFO:tasks.rados.rados.0.smithi195.stdout:1425: done (12 left) 2024-09-13T10:11:42.956 INFO:tasks.rados.rados.0.smithi195.stdout:1426: done (11 left) 2024-09-13T10:11:42.956 INFO:tasks.rados.rados.0.smithi195.stdout:1427: done (10 left) 2024-09-13T10:11:42.957 INFO:tasks.rados.rados.0.smithi195.stdout:1428: done (9 left) 2024-09-13T10:11:42.957 INFO:tasks.rados.rados.0.smithi195.stdout:1429: done (8 left) 2024-09-13T10:11:42.957 INFO:tasks.rados.rados.0.smithi195.stdout:1430: done (7 left) 2024-09-13T10:11:42.957 INFO:tasks.rados.rados.0.smithi195.stdout:1431: done (6 left) 2024-09-13T10:11:42.957 INFO:tasks.rados.rados.0.smithi195.stdout:1432: done (5 left) 2024-09-13T10:11:42.958 INFO:tasks.rados.rados.0.smithi195.stdout:1433: done (4 left) 2024-09-13T10:11:42.958 INFO:tasks.rados.rados.0.smithi195.stdout:1434: done (3 left) 2024-09-13T10:11:42.958 INFO:tasks.rados.rados.0.smithi195.stdout:1436: done (2 left) 2024-09-13T10:11:42.958 INFO:tasks.rados.rados.0.smithi195.stdout:1437: write (excl) oid 15 current snap is 0 2024-09-13T10:11:42.958 INFO:tasks.rados.rados.0.smithi195.stdout:1437: seq_num 293 ranges {524451=469371,1579224=483096,2529878=394573} 2024-09-13T10:11:42.960 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T10:11:42.964 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:42.959+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:42.971 INFO:tasks.rados.rados.0.smithi195.stdout:1437: writing smithi19530714-15 from 524451 to 993822 tid 1 2024-09-13T10:11:42.989 INFO:tasks.rados.rados.0.smithi195.stdout:1437: writing smithi19530714-15 from 1579224 to 2062320 tid 2 2024-09-13T10:11:43.003 INFO:tasks.rados.rados.0.smithi195.stdout:1437: writing smithi19530714-15 from 2529878 to 2924451 tid 3 2024-09-13T10:11:43.003 INFO:tasks.rados.rados.0.smithi195.stdout:1438: read oid 40 snap -1 2024-09-13T10:11:43.003 INFO:tasks.rados.rados.0.smithi195.stdout:1438: expect deleted 2024-09-13T10:11:43.004 INFO:tasks.rados.rados.0.smithi195.stdout:1439: set_chunk oid 4 target oid 2024-09-13T10:11:43.033 INFO:tasks.rados.rados.0.smithi195.stdout:1439: set_chunk oid 4 offset: 201216 length: 2331648 target oid 71f8cb809b47e1dc3dd3aea8719375a9e681992461b1c1cb1fc81e6c9519fc6b offset: 0 snap -1 2024-09-13T10:11:43.033 INFO:tasks.rados.rados.0.smithi195.stdout:1440: delete oid 41 current snap is 0 2024-09-13T10:11:43.034 INFO:tasks.rados.rados.0.smithi195.stdout:1437: finishing write tid 1 to smithi19530714-15 2024-09-13T10:11:43.034 INFO:tasks.rados.rados.0.smithi195.stdout:1437: finishing write tid 2 to smithi19530714-15 2024-09-13T10:11:43.034 INFO:tasks.rados.rados.0.smithi195.stdout:1437: finishing write tid 3 to smithi19530714-15 2024-09-13T10:11:43.034 INFO:tasks.rados.rados.0.smithi195.stdout:1437: finishing write tid 4 to smithi19530714-15 2024-09-13T10:11:43.034 INFO:tasks.rados.rados.0.smithi195.stdout:1437: finishing write tid 5 to smithi19530714-15 2024-09-13T10:11:43.034 INFO:tasks.rados.rados.0.smithi195.stdout:1437: oid 15 updating version 0 to 94 2024-09-13T10:11:43.034 INFO:tasks.rados.rados.0.smithi195.stdout:1437: oid 15 updating version 94 to 95 2024-09-13T10:11:43.034 INFO:tasks.rados.rados.0.smithi195.stdout:1437: oid 15 updating version 95 to 96 2024-09-13T10:11:43.035 INFO:tasks.rados.rados.0.smithi195.stdout:1437: oid 15 version 96 is already newer than 93 2024-09-13T10:11:43.035 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 96 (ObjNum 293 snap 0 seq_num 293) dirty exists 2024-09-13T10:11:43.035 INFO:tasks.rados.rados.0.smithi195.stdout:1437: left oid 15 (ObjNum 293 snap 0 seq_num 293) 2024-09-13T10:11:43.037 INFO:tasks.rados.rados.0.smithi195.stdout:1437: done (5 left) 2024-09-13T10:11:43.038 INFO:tasks.rados.rados.0.smithi195.stdout:1438: done (4 left) 2024-09-13T10:11:43.038 INFO:tasks.rados.rados.0.smithi195.stdout:1440: done (3 left) 2024-09-13T10:11:43.038 INFO:tasks.rados.rados.0.smithi195.stdout:1441: read oid 18 snap -1 2024-09-13T10:11:43.038 INFO:tasks.rados.rados.0.smithi195.stdout:1441: expect (ObjNum 277 snap 0 seq_num 277) 2024-09-13T10:11:43.038 INFO:tasks.rados.rados.0.smithi195.stdout:1442: read oid 5 snap -1 2024-09-13T10:11:43.038 INFO:tasks.rados.rados.0.smithi195.stdout:1442: expect (ObjNum 231 snap 0 seq_num 231) 2024-09-13T10:11:43.038 INFO:tasks.rados.rados.0.smithi195.stdout:1443: write (excl) oid 46 current snap is 0 2024-09-13T10:11:43.038 INFO:tasks.rados.rados.0.smithi195.stdout:1443: seq_num 295 ranges {558065=619869,1579835=486765,2468371=489694} 2024-09-13T10:11:43.057 INFO:tasks.rados.rados.0.smithi195.stdout:1443: writing smithi19530714-46 from 558065 to 1177934 tid 1 2024-09-13T10:11:43.065 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:43.062+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:43.067 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-13T10:11:43.070 INFO:tasks.rados.rados.0.smithi195.stdout:1443: writing smithi19530714-46 from 1579835 to 2066600 tid 2 2024-09-13T10:11:43.083 INFO:tasks.rados.rados.0.smithi195.stdout:1443: writing smithi19530714-46 from 2468371 to 2958065 tid 3 2024-09-13T10:11:43.083 INFO:tasks.rados.rados.0.smithi195.stdout:1444: read oid 16 snap -1 2024-09-13T10:11:43.083 INFO:tasks.rados.rados.0.smithi195.stdout:1444: expect (ObjNum 275 snap 0 seq_num 275) 2024-09-13T10:11:43.083 INFO:tasks.rados.rados.0.smithi195.stdout:1445: read oid 21 snap -1 2024-09-13T10:11:43.083 INFO:tasks.rados.rados.0.smithi195.stdout:1445: expect (ObjNum 235 snap 0 seq_num 235) 2024-09-13T10:11:43.083 INFO:tasks.rados.rados.0.smithi195.stdout:1446: read oid 7 snap -1 2024-09-13T10:11:43.083 INFO:tasks.rados.rados.0.smithi195.stdout:1446: expect (ObjNum 276 snap 0 seq_num 276) 2024-09-13T10:11:43.083 INFO:tasks.rados.rados.0.smithi195.stdout:1447: read oid 6 snap -1 2024-09-13T10:11:43.084 INFO:tasks.rados.rados.0.smithi195.stdout:1447: expect (ObjNum 229 snap 0 seq_num 229) 2024-09-13T10:11:43.084 INFO:tasks.rados.rados.0.smithi195.stdout:1448: set_chunk oid 48 target oid 2024-09-13T10:11:43.109 INFO:tasks.rados.rados.0.smithi195.stdout:1448: set_chunk oid 48 offset: 1024000 length: 836096 target oid ef837f0829b5a1afe0615c879ede0c421fb7e6ae6ca62c3d6092fd809eb805e9 offset: 0 snap -1 2024-09-13T10:11:43.109 INFO:tasks.rados.rados.0.smithi195.stdout:1449: read oid 17 snap -1 2024-09-13T10:11:43.110 INFO:tasks.rados.rados.0.smithi195.stdout:1449: expect (ObjNum 289 snap 0 seq_num 289) 2024-09-13T10:11:43.110 INFO:tasks.rados.rados.0.smithi195.stdout:1450: write (excl) oid 10 current snap is 0 2024-09-13T10:11:43.110 INFO:tasks.rados.rados.0.smithi195.stdout:1450: seq_num 296 ranges {574872=695118,2038317=630423} 2024-09-13T10:11:43.118 INFO:tasks.rados.rados.0.smithi195.stdout:1450: writing smithi19530714-10 from 574872 to 1269990 tid 1 2024-09-13T10:11:43.128 INFO:tasks.rados.rados.0.smithi195.stdout:1450: writing smithi19530714-10 from 2038317 to 2668740 tid 2 2024-09-13T10:11:43.128 INFO:tasks.rados.rados.0.smithi195.stdout:1451: read oid 47 snap -1 2024-09-13T10:11:43.128 INFO:tasks.rados.rados.0.smithi195.stdout:1451: expect (ObjNum 260 snap 0 seq_num 260) 2024-09-13T10:11:43.128 INFO:tasks.rados.rados.0.smithi195.stdout:1452: read oid 35 snap -1 2024-09-13T10:11:43.129 INFO:tasks.rados.rados.0.smithi195.stdout:1452: expect (ObjNum 265 snap 0 seq_num 265) 2024-09-13T10:11:43.129 INFO:tasks.rados.rados.0.smithi195.stdout:1453: write (excl) oid 42 current snap is 0 2024-09-13T10:11:43.129 INFO:tasks.rados.rados.0.smithi195.stdout:1453: seq_num 297 ranges {591679=486720,1813152=490434} 2024-09-13T10:11:43.135 INFO:tasks.rados.rados.0.smithi195.stdout:1453: writing smithi19530714-42 from 591679 to 1078399 tid 1 2024-09-13T10:11:43.142 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:11:43.143 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:11:43.144 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:43.144 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:43.144 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:11:43.144 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:11:43.144 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:43.145 INFO:tasks.rados.rados.0.smithi195.stdout:1453: writing smithi19530714-42 from 1813152 to 2303586 tid 2 2024-09-13T10:11:43.145 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:43.145 INFO:tasks.rados.rados.0.smithi195.stdout:1441: expect (ObjNum 277 snap 0 seq_num 277) 2024-09-13T10:11:43.158 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T10:11:43.160 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-13T10:11:43.067694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":222167771,"num_objects":197,"num_object_clones":0,"num_object_copies":591,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":15,"num_objects_unfound":0,"num_objects_dirty":191,"num_whiteouts":0,"num_read":3447,"num_read_kb":2182504,"num_write":2410,"num_write_kb":530616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":17577428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":22,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102,"ondisk_log_size":102,"up":117,"acting":117,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":962728,"kb_used_data":489804,"kb_used_omap":24,"kb_used_meta":465383,"kb_avail":1508986712,"statfs":{"total":1546188226560,"available":1545202393088,"internally_reserved":0,"allocated":501559296,"data_stored":494195900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":476552271},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15328242,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-6,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":68,"num_read_kb":22799,"num_write":104,"num_write_kb":29800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":5326268,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":-4,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.038810"},"pg_stats":[{"pgid":"4.19","version":"74'54","reported_seq":225,"reported_epoch":80,"state":"remapped+peering","last_fresh":"2024-09-13T10:11:42.074065+0000","last_change":"2024-09-13T10:11:42.074065+0000","last_active":"2024-09-13T10:11:42.074065+0000","last_peered":"2024-09-13T10:11:36.648699+0000","last_clean":"2024-09-13T10:11:36.648699+0000","last_became_active":"2024-09-13T10:10:54.324793+0000","last_became_peered":"2024-09-13T10:10:54.324793+0000","last_unstale":"2024-09-13T10:11:42.074065+0000","last_undegraded":"2024-09-13T10:11:42.074065+0000","last_fullsized":"2024-09-13T10:11:42.074065+0000","mapping_epoch":80,"log_start":"74'51","ondisk_log_start":"74'51","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'28","last_scrub_stamp":"2024-09-13T10:11:00.388132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.388132+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":51,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4621991,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":70312,"num_write":31,"num_write_kb":7941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[5,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"79'18","reported_seq":171,"reported_epoch":80,"state":"peering","last_fresh":"2024-09-13T10:11:42.068603+0000","last_change":"2024-09-13T10:11:42.068603+0000","last_active":"2024-09-13T10:11:42.068603+0000","last_peered":"2024-09-13T10:11:41.192283+0000","last_clean":"2024-09-13T10:11:27.365225+0000","last_became_active":"2024-09-13T10:11:41.148211+0000","last_became_peered":"2024-09-13T10:11:41.148211+0000","last_unstale":"2024-09-13T10:11:42.068603+0000","last_undegraded":"2024-09-13T10:11:42.068603+0000","last_fullsized":"2024-09-13T10:11:42.068603+0000","mapping_epoch":80,"log_start":"68'15","ondisk_log_start":"68'15","created":41,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'5","last_scrub_stamp":"2024-09-13T10:11:01.783496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.783496+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":15,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3882417,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":45,"num_read_kb":36360,"num_write":14,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2436277,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,13],"acting":[10,8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.17","version":"79'105","reported_seq":352,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-13T10:11:42.147353+0000","last_change":"2024-09-13T10:11:34.723291+0000","last_active":"2024-09-13T10:11:42.147353+0000","last_peered":"2024-09-13T10:11:42.147353+0000","last_clean":"2024-09-13T10:11:42.147353+0000","last_became_active":"2024-09-13T10:11:34.720812+0000","last_became_peered":"2024-09-13T10:11:34.720812+0000","last_unstale":"2024-09-13T10:11:42.147353+0000","last_undegraded":"2024-09-13T10:11:42.147353+0000","last_fullsized":"2024-09-13T10:11:42.147353+0000","mapping_epoch":73,"log_start":"77'102","ondisk_log_start":"77'102","created":41,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'34","last_scrub_stamp":"2024-09-13T10:11:00.442413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.442413+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":102,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:24.850546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8067459,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":131,"num_read_kb":106812,"num_write":98,"num_write_kb":26695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3911454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"69'66","reported_seq":263,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T10:11:37.703843+0000","last_change":"2024-09-13T10:11:25.825915+0000","last_active":"2024-09-13T10:11:37.703843+0000","last_peered":"2024-09-13T10:11:37.703843+0000","last_clean":"2024-09-13T10:11:37.703843+0000","last_became_active":"2024-09-13T10:11:25.825236+0000","last_became_peered":"2024-09-13T10:11:25.825236+0000","last_unstale":"2024-09-13T10:11:37.703843+0000","last_undegraded":"2024-09-13T10:11:37.703843+0000","last_fullsized":"2024-09-13T10:11:37.703843+0000","mapping_epoch":66,"log_start":"69'63","ondisk_log_start":"69'63","created":41,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":5,"last_scrub":"46'44","last_scrub_stamp":"2024-09-13T10:11:00.456457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.456457+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":60,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:10.951998+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6672379,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":61,"num_read_kb":69493,"num_write":24,"num_write_kb":6198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3338234,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13,5],"acting":[14,13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.15","version":"74'38","reported_seq":224,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T10:11:37.823337+0000","last_change":"2024-09-13T10:11:23.706313+0000","last_active":"2024-09-13T10:11:37.823337+0000","last_peered":"2024-09-13T10:11:37.823337+0000","last_clean":"2024-09-13T10:11:37.823337+0000","last_became_active":"2024-09-13T10:11:23.705370+0000","last_became_peered":"2024-09-13T10:11:23.705370+0000","last_unstale":"2024-09-13T10:11:37.823337+0000","last_undegraded":"2024-09-13T10:11:37.823337+0000","last_fullsized":"2024-09-13T10:11:37.823337+0000","mapping_epoch":64,"log_start":"45'35","ondisk_log_start":"45'35","created":41,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'36","last_scrub_stamp":"2024-09-13T10:11:05.611551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.611551+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":33,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:22.429696+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":37703,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1702851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,13],"acting":[6,7,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"39'40","reported_seq":139,"reported_epoch":80,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:11:42.073336+0000","last_change":"2024-09-13T10:11:15.854375+0000","last_active":"2024-09-13T10:11:42.073336+0000","last_peered":"2024-09-13T10:11:42.073336+0000","last_clean":"2024-09-13T10:10:51.933477+0000","last_became_active":"2024-09-13T10:11:15.850712+0000","last_became_peered":"2024-09-13T10:11:15.850712+0000","last_unstale":"2024-09-13T10:11:42.073336+0000","last_undegraded":"2024-09-13T10:11:42.073336+0000","last_fullsized":"2024-09-13T10:11:42.073336+0000","mapping_epoch":58,"log_start":"39'37","ondisk_log_start":"39'37","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'40","last_scrub_stamp":"2024-09-13T10:11:04.595221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.595221+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":37,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7831963,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":112,"num_write":16,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[5,6,10],"avail_no_missing":["5","6","10"],"object_location_counts":[{"shards":"5,6,10","objects":3}],"blocked_by":[],"up_primary":11,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"80'35","reported_seq":229,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-13T10:11:42.150237+0000","last_change":"2024-09-13T10:11:15.761185+0000","last_active":"2024-09-13T10:11:42.150237+0000","last_peered":"2024-09-13T10:11:42.150237+0000","last_clean":"2024-09-13T10:11:42.150237+0000","last_became_active":"2024-09-13T10:11:15.757364+0000","last_became_peered":"2024-09-13T10:11:15.757364+0000","last_unstale":"2024-09-13T10:11:42.150237+0000","last_undegraded":"2024-09-13T10:11:42.150237+0000","last_fullsized":"2024-09-13T10:11:42.150237+0000","mapping_epoch":58,"log_start":"79'32","ondisk_log_start":"79'32","created":41,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'17","last_scrub_stamp":"2024-09-13T10:11:03.220846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:03.220846+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":30,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:14.471504+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2840416,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":33963,"num_write":21,"num_write_kb":5429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1493592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"76'36","reported_seq":240,"reported_epoch":76,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:11:38.542661+0000","last_change":"2024-09-13T10:11:11.763857+0000","last_active":"2024-09-13T10:11:38.542661+0000","last_peered":"2024-09-13T10:11:38.542661+0000","last_clean":"2024-09-13T10:11:08.694521+0000","last_became_active":"2024-09-13T10:11:11.753624+0000","last_became_peered":"2024-09-13T10:11:11.753624+0000","last_unstale":"2024-09-13T10:11:38.542661+0000","last_undegraded":"2024-09-13T10:11:38.542661+0000","last_fullsized":"2024-09-13T10:11:38.542661+0000","mapping_epoch":54,"log_start":"75'33","ondisk_log_start":"75'33","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'17","last_scrub_stamp":"2024-09-13T10:11:02.582829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:02.582829+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":33,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4621991,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":105676,"num_write":18,"num_write_kb":4915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,9],"acting":[6,8,14],"avail_no_missing":["6","8","14"],"object_location_counts":[{"shards":"6,8,14","objects":2}],"blocked_by":[],"up_primary":13,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"66'15","reported_seq":190,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T10:11:40.067263+0000","last_change":"2024-09-13T10:11:40.065958+0000","last_active":"2024-09-13T10:11:40.067263+0000","last_peered":"2024-09-13T10:11:40.067263+0000","last_clean":"2024-09-13T10:11:40.067263+0000","last_became_active":"2024-09-13T10:11:40.064608+0000","last_became_peered":"2024-09-13T10:11:40.064608+0000","last_unstale":"2024-09-13T10:11:40.067263+0000","last_undegraded":"2024-09-13T10:11:40.067263+0000","last_fullsized":"2024-09-13T10:11:40.067263+0000","mapping_epoch":77,"log_start":"39'10","ondisk_log_start":"39'10","created":41,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":5,"last_scrub":"40'13","last_scrub_stamp":"2024-09-13T10:11:04.242723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.242723+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":10,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:19.279924+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2689087,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":44,"num_read_kb":33385,"num_write":2,"num_write_kb":1060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1805029,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,12],"acting":[13,4,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.10","version":"78'43","reported_seq":259,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-13T10:11:41.066637+0000","last_change":"2024-09-13T10:11:40.068589+0000","last_active":"2024-09-13T10:11:41.066637+0000","last_peered":"2024-09-13T10:11:41.066637+0000","last_clean":"2024-09-13T10:11:41.066637+0000","last_became_active":"2024-09-13T10:11:40.063380+0000","last_became_peered":"2024-09-13T10:11:40.063380+0000","last_unstale":"2024-09-13T10:11:41.066637+0000","last_undegraded":"2024-09-13T10:11:41.066637+0000","last_fullsized":"2024-09-13T10:11:41.066637+0000","mapping_epoch":77,"log_start":"78'38","ondisk_log_start":"78'38","created":41,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'16","last_scrub_stamp":"2024-09-13T10:11:00.706874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.706874+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":36,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:13.764566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5227043,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":82895,"num_write":33,"num_write_kb":8964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2889991,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,14,3],"acting":[9,14,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":61,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-13T10:11:42.067093+0000","last_change":"2024-09-13T10:11:12.744453+0000","last_active":"2024-09-13T10:11:42.067093+0000","last_peered":"2024-09-13T10:11:42.067093+0000","last_clean":"2024-09-13T10:11:42.067093+0000","last_became_active":"2024-09-13T10:11:12.744090+0000","last_became_peered":"2024-09-13T10:11:12.744090+0000","last_unstale":"2024-09-13T10:11:42.067093+0000","last_undegraded":"2024-09-13T10:11:42.067093+0000","last_fullsized":"2024-09-13T10:11:42.067093+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:57.001325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,12],"acting":[3,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"39'12","reported_seq":154,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T10:11:37.823228+0000","last_change":"2024-09-13T10:11:01.537080+0000","last_active":"2024-09-13T10:11:37.823228+0000","last_peered":"2024-09-13T10:11:37.823228+0000","last_clean":"2024-09-13T10:11:37.823228+0000","last_became_active":"2024-09-13T10:10:54.318519+0000","last_became_peered":"2024-09-13T10:10:54.318519+0000","last_unstale":"2024-09-13T10:11:37.823228+0000","last_undegraded":"2024-09-13T10:11:37.823228+0000","last_fullsized":"2024-09-13T10:11:37.823228+0000","mapping_epoch":31,"log_start":"38'9","ondisk_log_start":"38'9","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'12","last_scrub_stamp":"2024-09-13T10:11:01.536963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.536963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":9,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:21.111444+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,14],"acting":[6,8,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":125,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-13T10:11:42.065677+0000","last_change":"2024-09-13T10:10:32.932079+0000","last_active":"2024-09-13T10:11:42.065677+0000","last_peered":"2024-09-13T10:11:42.065677+0000","last_clean":"2024-09-13T10:11:42.065677+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:11:42.065677+0000","last_undegraded":"2024-09-13T10:11:42.065677+0000","last_fullsized":"2024-09-13T10:11:42.065677+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:57.533673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"75'81","reported_seq":271,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T10:11:38.720268+0000","last_change":"2024-09-13T10:11:01.219554+0000","last_active":"2024-09-13T10:11:38.720268+0000","last_peered":"2024-09-13T10:11:38.720268+0000","last_clean":"2024-09-13T10:11:38.720268+0000","last_became_active":"2024-09-13T10:10:54.314545+0000","last_became_peered":"2024-09-13T10:10:54.314545+0000","last_unstale":"2024-09-13T10:11:38.720268+0000","last_undegraded":"2024-09-13T10:11:38.720268+0000","last_fullsized":"2024-09-13T10:11:38.720268+0000","mapping_epoch":31,"log_start":"72'78","ondisk_log_start":"72'78","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'27","last_scrub_stamp":"2024-09-13T10:11:01.219486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.219486+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":77,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:24.831741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6386693,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":70819,"num_write":70,"num_write_kb":20315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,8],"acting":[2,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":125,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-13T10:11:42.216403+0000","last_change":"2024-09-13T10:10:32.177488+0000","last_active":"2024-09-13T10:11:42.216403+0000","last_peered":"2024-09-13T10:11:42.216403+0000","last_clean":"2024-09-13T10:11:42.216403+0000","last_became_active":"2024-09-13T10:10:30.128658+0000","last_became_peered":"2024-09-13T10:10:30.128658+0000","last_unstale":"2024-09-13T10:11:42.216403+0000","last_undegraded":"2024-09-13T10:11:42.216403+0000","last_fullsized":"2024-09-13T10:11:42.216403+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:50.278429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.0","version":"73'26","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-13T10:11:42.073345+0000","last_change":"2024-09-13T10:11:05.681812+0000","last_active":"2024-09-13T10:11:42.073345+0000","last_peered":"2024-09-13T10:11:42.073345+0000","last_clean":"2024-09-13T10:11:42.073345+0000","last_became_active":"2024-09-13T10:10:54.320014+0000","last_became_peered":"2024-09-13T10:10:54.320014+0000","last_unstale":"2024-09-13T10:11:42.073345+0000","last_undegraded":"2024-09-13T10:11:42.073345+0000","last_fullsized":"2024-09-13T10:11:42.073345+0000","mapping_epoch":31,"log_start":"60'23","ondisk_log_start":"60'23","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'14","last_scrub_stamp":"2024-09-13T10:11:05.681653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.681653+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":22,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:09.761839+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":43456,"num_write":11,"num_write_kb":3373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T10:11:37.703759+0000","last_change":"2024-09-13T10:10:32.701581+0000","last_active":"2024-09-13T10:11:37.703759+0000","last_peered":"2024-09-13T10:11:37.703759+0000","last_clean":"2024-09-13T10:11:37.703759+0000","last_became_active":"2024-09-13T10:10:30.126588+0000","last_became_peered":"2024-09-13T10:10:30.126588+0000","last_unstale":"2024-09-13T10:11:37.703759+0000","last_undegraded":"2024-09-13T10:11:37.703759+0000","last_fullsized":"2024-09-13T10:11:37.703759+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:40.095103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089379399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.1","version":"65'23","reported_seq":186,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T10:11:38.106589+0000","last_change":"2024-09-13T10:11:02.190704+0000","last_active":"2024-09-13T10:11:38.106589+0000","last_peered":"2024-09-13T10:11:38.106589+0000","last_clean":"2024-09-13T10:11:38.106589+0000","last_became_active":"2024-09-13T10:10:54.317600+0000","last_became_peered":"2024-09-13T10:10:54.317600+0000","last_unstale":"2024-09-13T10:11:38.106589+0000","last_undegraded":"2024-09-13T10:11:38.106589+0000","last_fullsized":"2024-09-13T10:11:38.106589+0000","mapping_epoch":31,"log_start":"61'19","ondisk_log_start":"61'19","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'18","last_scrub_stamp":"2024-09-13T10:11:02.190506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:02.190506+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":19,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:19.006753+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3479049,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":58405,"num_write":5,"num_write_kb":1502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,12],"acting":[2,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"77'218","reported_seq":337,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T10:11:40.057433+0000","last_change":"2024-09-13T10:10:43.231291+0000","last_active":"2024-09-13T10:11:40.057433+0000","last_peered":"2024-09-13T10:11:40.057433+0000","last_clean":"2024-09-13T10:11:40.057433+0000","last_became_active":"2024-09-13T10:10:43.230866+0000","last_became_peered":"2024-09-13T10:10:43.230866+0000","last_unstale":"2024-09-13T10:11:40.057433+0000","last_undegraded":"2024-09-13T10:11:40.057433+0000","last_fullsized":"2024-09-13T10:11:40.057433+0000","mapping_epoch":30,"log_start":"76'215","ondisk_log_start":"76'215","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":230,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:54.564637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21643527,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":203,"num_read_kb":2853,"num_write":289,"num_write_kb":57268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,4,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":208,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-13T10:11:42.067086+0000","last_change":"2024-09-13T10:11:12.699917+0000","last_active":"2024-09-13T10:11:42.067086+0000","last_peered":"2024-09-13T10:11:42.067086+0000","last_clean":"2024-09-13T10:11:42.067086+0000","last_became_active":"2024-09-13T10:11:12.699043+0000","last_became_peered":"2024-09-13T10:11:12.699043+0000","last_unstale":"2024-09-13T10:11:42.067086+0000","last_undegraded":"2024-09-13T10:11:42.067086+0000","last_fullsized":"2024-09-13T10:11:42.067086+0000","mapping_epoch":55,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,7],"acting":[3,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T10:11:40.055143+0000","last_change":"2024-09-13T10:11:38.808286+0000","last_active":"2024-09-13T10:11:40.055143+0000","last_peered":"2024-09-13T10:11:40.055143+0000","last_clean":"2024-09-13T10:11:40.055143+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:11:40.055143+0000","last_undegraded":"2024-09-13T10:11:40.055143+0000","last_fullsized":"2024-09-13T10:11:40.055143+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T10:11:38.808119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:38.808119+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:43.158000+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.6","version":"75'73","reported_seq":264,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T10:11:40.055624+0000","last_change":"2024-09-13T10:11:39.048972+0000","last_active":"2024-09-13T10:11:40.055624+0000","last_peered":"2024-09-13T10:11:40.055624+0000","last_clean":"2024-09-13T10:11:40.055624+0000","last_became_active":"2024-09-13T10:11:39.047906+0000","last_became_peered":"2024-09-13T10:11:39.047906+0000","last_unstale":"2024-09-13T10:11:40.055624+0000","last_undegraded":"2024-09-13T10:11:40.055624+0000","last_fullsized":"2024-09-13T10:11:40.055624+0000","mapping_epoch":76,"log_start":"74'70","ondisk_log_start":"74'70","created":31,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'44","last_scrub_stamp":"2024-09-13T10:11:05.396838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.396838+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":70,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:20.834115+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6302658,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":35092,"num_write":33,"num_write_kb":9547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,1],"acting":[13,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"80'272","reported_seq":392,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-13T10:11:42.183270+0000","last_change":"2024-09-13T10:10:43.220235+0000","last_active":"2024-09-13T10:11:42.183270+0000","last_peered":"2024-09-13T10:11:42.183270+0000","last_clean":"2024-09-13T10:11:42.183270+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:11:42.183270+0000","last_undegraded":"2024-09-13T10:11:42.183270+0000","last_fullsized":"2024-09-13T10:11:42.183270+0000","mapping_epoch":30,"log_start":"79'269","ondisk_log_start":"79'269","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":281,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:55.166722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30146426,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":244,"num_read_kb":6284,"num_write":364,"num_write_kb":77852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":129,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T10:11:38.106584+0000","last_change":"2024-09-13T10:11:30.291380+0000","last_active":"2024-09-13T10:11:38.106584+0000","last_peered":"2024-09-13T10:11:38.106584+0000","last_clean":"2024-09-13T10:11:38.106584+0000","last_became_active":"2024-09-13T10:10:30.127232+0000","last_became_peered":"2024-09-13T10:10:30.127232+0000","last_unstale":"2024-09-13T10:11:38.106584+0000","last_undegraded":"2024-09-13T10:11:38.106584+0000","last_fullsized":"2024-09-13T10:11:38.106584+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-13T10:11:30.291157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:30.291157+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:31.417661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0034622780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"80'74","reported_seq":275,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-13T10:11:42.177086+0000","last_change":"2024-09-13T10:11:05.436449+0000","last_active":"2024-09-13T10:11:42.177086+0000","last_peered":"2024-09-13T10:11:42.177086+0000","last_clean":"2024-09-13T10:11:42.177086+0000","last_became_active":"2024-09-13T10:10:54.314465+0000","last_became_peered":"2024-09-13T10:10:54.314465+0000","last_unstale":"2024-09-13T10:11:42.177086+0000","last_undegraded":"2024-09-13T10:11:42.177086+0000","last_fullsized":"2024-09-13T10:11:42.177086+0000","mapping_epoch":31,"log_start":"80'71","ondisk_log_start":"80'71","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'40","last_scrub_stamp":"2024-09-13T10:11:05.436272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.436272+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":68,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:27.246008+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5395113,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":39840,"num_write":40,"num_write_kb":12093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,10],"acting":[1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"80'250","reported_seq":368,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-13T10:11:42.109479+0000","last_change":"2024-09-13T10:10:43.220955+0000","last_active":"2024-09-13T10:11:42.109479+0000","last_peered":"2024-09-13T10:11:42.109479+0000","last_clean":"2024-09-13T10:11:42.109479+0000","last_became_active":"2024-09-13T10:10:43.220074+0000","last_became_peered":"2024-09-13T10:10:43.220074+0000","last_unstale":"2024-09-13T10:11:42.109479+0000","last_undegraded":"2024-09-13T10:11:42.109479+0000","last_fullsized":"2024-09-13T10:11:42.109479+0000","mapping_epoch":30,"log_start":"79'247","ondisk_log_start":"79'247","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":259,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:54.562129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19834273,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":223,"num_read_kb":4260,"num_write":326,"num_write_kb":66185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-13T10:11:42.216777+0000","last_change":"2024-09-13T10:11:39.046876+0000","last_active":"2024-09-13T10:11:42.216777+0000","last_peered":"2024-09-13T10:11:42.216777+0000","last_clean":"2024-09-13T10:11:42.216777+0000","last_became_active":"2024-09-13T10:11:39.046090+0000","last_became_peered":"2024-09-13T10:11:39.046090+0000","last_unstale":"2024-09-13T10:11:42.216777+0000","last_undegraded":"2024-09-13T10:11:42.216777+0000","last_fullsized":"2024-09-13T10:11:42.216777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,7,9],"acting":[15,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.4","version":"76'130","reported_seq":446,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T10:11:38.481645+0000","last_change":"2024-09-13T10:11:03.633738+0000","last_active":"2024-09-13T10:11:38.481645+0000","last_peered":"2024-09-13T10:11:38.481645+0000","last_clean":"2024-09-13T10:11:38.481645+0000","last_became_active":"2024-09-13T10:10:54.316518+0000","last_became_peered":"2024-09-13T10:10:54.316518+0000","last_unstale":"2024-09-13T10:11:38.481645+0000","last_undegraded":"2024-09-13T10:11:38.481645+0000","last_fullsized":"2024-09-13T10:11:38.481645+0000","mapping_epoch":31,"log_start":"74'127","ondisk_log_start":"74'127","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'66","last_scrub_stamp":"2024-09-13T10:11:03.633499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:03.633499+0000","objects_scrubbed":5,"log_size":3,"log_dups_size":127,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:22.405104+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17832293,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":292,"num_read_kb":272043,"num_write":72,"num_write_kb":20251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10],"acting":[6,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"76'303","reported_seq":418,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T10:11:38.525574+0000","last_change":"2024-09-13T10:10:43.230767+0000","last_active":"2024-09-13T10:11:38.525574+0000","last_peered":"2024-09-13T10:11:38.525574+0000","last_clean":"2024-09-13T10:11:38.525574+0000","last_became_active":"2024-09-13T10:10:43.229961+0000","last_became_peered":"2024-09-13T10:10:43.229961+0000","last_unstale":"2024-09-13T10:11:38.525574+0000","last_undegraded":"2024-09-13T10:11:38.525574+0000","last_fullsized":"2024-09-13T10:11:38.525574+0000","mapping_epoch":30,"log_start":"75'300","ondisk_log_start":"75'300","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":309,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:53.856674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12504786,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":281,"num_read_kb":3428,"num_write":379,"num_write_kb":71067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T10:11:37.823024+0000","last_change":"2024-09-13T10:10:32.149042+0000","last_active":"2024-09-13T10:11:37.823024+0000","last_peered":"2024-09-13T10:11:37.823024+0000","last_clean":"2024-09-13T10:11:37.823024+0000","last_became_active":"2024-09-13T10:10:30.272007+0000","last_became_peered":"2024-09-13T10:10:30.272007+0000","last_unstale":"2024-09-13T10:11:37.823024+0000","last_undegraded":"2024-09-13T10:11:37.823024+0000","last_fullsized":"2024-09-13T10:11:37.823024+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:49.552306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000987604,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"74'124","reported_seq":399,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T10:11:37.823290+0000","last_change":"2024-09-13T10:11:00.541038+0000","last_active":"2024-09-13T10:11:37.823290+0000","last_peered":"2024-09-13T10:11:37.823290+0000","last_clean":"2024-09-13T10:11:37.823290+0000","last_became_active":"2024-09-13T10:10:54.315196+0000","last_became_peered":"2024-09-13T10:10:54.315196+0000","last_unstale":"2024-09-13T10:11:37.823290+0000","last_undegraded":"2024-09-13T10:11:37.823290+0000","last_fullsized":"2024-09-13T10:11:37.823290+0000","mapping_epoch":31,"log_start":"72'120","ondisk_log_start":"72'120","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'48","last_scrub_stamp":"2024-09-13T10:11:00.540905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.540905+0000","objects_scrubbed":5,"log_size":4,"log_dups_size":119,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:23.486842+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14538121,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":189,"num_read_kb":142474,"num_write":87,"num_write_kb":24793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,0],"acting":[6,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"39'5","reported_seq":146,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-13T10:11:42.065487+0000","last_change":"2024-09-13T10:11:00.786261+0000","last_active":"2024-09-13T10:11:42.065487+0000","last_peered":"2024-09-13T10:11:42.065487+0000","last_clean":"2024-09-13T10:11:42.065487+0000","last_became_active":"2024-09-13T10:10:54.314935+0000","last_became_peered":"2024-09-13T10:10:54.314935+0000","last_unstale":"2024-09-13T10:11:42.065487+0000","last_undegraded":"2024-09-13T10:11:42.065487+0000","last_fullsized":"2024-09-13T10:11:42.065487+0000","mapping_epoch":31,"log_start":"38'2","ondisk_log_start":"38'2","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'5","last_scrub_stamp":"2024-09-13T10:11:00.786110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.786110+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:21.650090+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,0],"acting":[3,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"79'42","reported_seq":218,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-13T10:11:42.073566+0000","last_change":"2024-09-13T10:11:04.446852+0000","last_active":"2024-09-13T10:11:42.073566+0000","last_peered":"2024-09-13T10:11:42.073566+0000","last_clean":"2024-09-13T10:11:42.073566+0000","last_became_active":"2024-09-13T10:10:54.316057+0000","last_became_peered":"2024-09-13T10:10:54.316057+0000","last_unstale":"2024-09-13T10:11:42.073566+0000","last_undegraded":"2024-09-13T10:11:42.073566+0000","last_fullsized":"2024-09-13T10:11:42.073566+0000","mapping_epoch":31,"log_start":"74'39","ondisk_log_start":"74'39","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'29","last_scrub_stamp":"2024-09-13T10:11:04.446717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.446717+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":36,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:07.066416+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2806802,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":48,"num_read_kb":31213,"num_write":13,"num_write_kb":3657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,9],"acting":[11,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.a","version":"69'22","reported_seq":166,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T10:11:37.822925+0000","last_change":"2024-09-13T10:11:12.708135+0000","last_active":"2024-09-13T10:11:37.822925+0000","last_peered":"2024-09-13T10:11:37.822925+0000","last_clean":"2024-09-13T10:11:37.822925+0000","last_became_active":"2024-09-13T10:11:12.707306+0000","last_became_peered":"2024-09-13T10:11:12.707306+0000","last_unstale":"2024-09-13T10:11:37.822925+0000","last_undegraded":"2024-09-13T10:11:37.822925+0000","last_fullsized":"2024-09-13T10:11:37.822925+0000","mapping_epoch":55,"log_start":"54'19","ondisk_log_start":"54'19","created":31,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":19,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:06.716561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":54207,"num_write":21,"num_write_kb":7244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,13],"acting":[6,8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"74'65","reported_seq":281,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T10:11:38.538533+0000","last_change":"2024-09-13T10:10:44.227804+0000","last_active":"2024-09-13T10:11:38.538533+0000","last_peered":"2024-09-13T10:11:38.538533+0000","last_clean":"2024-09-13T10:11:38.538533+0000","last_became_active":"2024-09-13T10:10:44.226956+0000","last_became_peered":"2024-09-13T10:10:44.226956+0000","last_unstale":"2024-09-13T10:11:38.538533+0000","last_undegraded":"2024-09-13T10:11:38.538533+0000","last_fullsized":"2024-09-13T10:11:38.538533+0000","mapping_epoch":31,"log_start":"67'62","ondisk_log_start":"67'62","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":62,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:51.165568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9764867,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":250,"num_read_kb":219322,"num_write":60,"num_write_kb":16720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,4],"acting":[14,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.c","version":"75'88","reported_seq":347,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-13T10:11:38.559973+0000","last_change":"2024-09-13T10:10:59.373229+0000","last_active":"2024-09-13T10:11:38.559973+0000","last_peered":"2024-09-13T10:11:38.559973+0000","last_clean":"2024-09-13T10:11:38.559973+0000","last_became_active":"2024-09-13T10:10:59.366341+0000","last_became_peered":"2024-09-13T10:10:59.366341+0000","last_unstale":"2024-09-13T10:11:38.559973+0000","last_undegraded":"2024-09-13T10:11:38.559973+0000","last_fullsized":"2024-09-13T10:11:38.559973+0000","mapping_epoch":44,"log_start":"74'85","ondisk_log_start":"74'85","created":31,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":85,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:54.217342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10605316,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":335,"num_read_kb":314604,"num_write":87,"num_write_kb":23978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,12,3],"acting":[6,12,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"80'44","reported_seq":218,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-13T10:11:42.230940+0000","last_change":"2024-09-13T10:10:59.365358+0000","last_active":"2024-09-13T10:11:42.230940+0000","last_peered":"2024-09-13T10:11:42.230940+0000","last_clean":"2024-09-13T10:11:42.230940+0000","last_became_active":"2024-09-13T10:10:59.364443+0000","last_became_peered":"2024-09-13T10:10:59.364443+0000","last_unstale":"2024-09-13T10:11:42.230940+0000","last_undegraded":"2024-09-13T10:11:42.230940+0000","last_fullsized":"2024-09-13T10:11:42.230940+0000","mapping_epoch":44,"log_start":"80'41","ondisk_log_start":"80'41","created":31,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":41,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:47.900264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6857289,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":103115,"num_write":41,"num_write_kb":11456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,12,7],"acting":[15,12,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.e","version":"74'21","reported_seq":124,"reported_epoch":76,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:11:38.106704+0000","last_change":"2024-09-13T10:11:20.876764+0000","last_active":"2024-09-13T10:11:38.106704+0000","last_peered":"2024-09-13T10:11:38.106704+0000","last_clean":"2024-09-13T10:11:03.155929+0000","last_became_active":"2024-09-13T10:11:20.874865+0000","last_became_peered":"2024-09-13T10:11:20.874865+0000","last_unstale":"2024-09-13T10:11:38.106704+0000","last_undegraded":"2024-09-13T10:11:38.106704+0000","last_fullsized":"2024-09-13T10:11:38.106704+0000","mapping_epoch":61,"log_start":"67'18","ondisk_log_start":"67'18","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":18,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":23639,"num_write":20,"num_write_kb":5797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,15],"acting":[2,8,4],"avail_no_missing":["2","3","4","8","15"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"76'81","reported_seq":273,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-13T10:11:40.055304+0000","last_change":"2024-09-13T10:10:44.224347+0000","last_active":"2024-09-13T10:11:40.055304+0000","last_peered":"2024-09-13T10:11:40.055304+0000","last_clean":"2024-09-13T10:11:40.055304+0000","last_became_active":"2024-09-13T10:10:44.223700+0000","last_became_peered":"2024-09-13T10:10:44.223700+0000","last_unstale":"2024-09-13T10:11:40.055304+0000","last_undegraded":"2024-09-13T10:11:40.055304+0000","last_fullsized":"2024-09-13T10:11:40.055304+0000","mapping_epoch":31,"log_start":"74'78","ondisk_log_start":"74'78","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":78,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:10.889950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7025425,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":169,"num_read_kb":180666,"num_write":87,"num_write_kb":24842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,7],"acting":[13,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":26,"stat_sum":{"num_bytes":137448372,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":15,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":2410,"num_read_kb":2165606,"num_write":905,"num_write_kb":255570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":17577428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":22,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":259846144,"data_stored":259113973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84,"ondisk_log_size":84,"up":78,"acting":78,"num_store_stats":16},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":84129012,"num_objects":145,"num_object_clones":0,"num_object_copies":435,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":951,"num_read_kb":16825,"num_write":1358,"num_write_kb":272372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":231940096,"data_stored":231155776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":12}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247825,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80880,"kb_used_data":49696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94290960,"statfs":{"total":96636764160,"available":96553943040,"internally_reserved":0,"allocated":50888704,"data_stored":50382165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247827,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40216,"kb_used_data":13192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331624,"statfs":{"total":96636764160,"available":96595582976,"internally_reserved":0,"allocated":13508608,"data_stored":13128110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247827,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63176,"kb_used_data":31992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308664,"statfs":{"total":96636764160,"available":96572071936,"internally_reserved":0,"allocated":32759808,"data_stored":32295394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313235,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34672,"kb_used_data":7648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337168,"statfs":{"total":96636764160,"available":96601260032,"internally_reserved":0,"allocated":7831552,"data_stored":7455061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,5,6,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313234,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65932,"kb_used_data":38908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94305908,"statfs":{"total":96636764160,"available":96569249792,"internally_reserved":0,"allocated":39841792,"data_stored":39403428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,3,4,6,7,8,9,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247825,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53572,"kb_used_data":22380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94318268,"statfs":{"total":96636764160,"available":96581906432,"internally_reserved":0,"allocated":22917120,"data_stored":22437104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280530,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57044,"kb_used_data":24192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94314796,"statfs":{"total":96636764160,"available":96578351104,"internally_reserved":0,"allocated":24772608,"data_stored":24290409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280530,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":91164,"kb_used_data":59980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94280676,"statfs":{"total":96636764160,"available":96543412224,"internally_reserved":0,"allocated":61419520,"data_stored":60876533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280531,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86704,"kb_used_data":59680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94285136,"statfs":{"total":96636764160,"available":96547979264,"internally_reserved":0,"allocated":61112320,"data_stored":60583892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280529,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52948,"kb_used_data":25924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94318892,"statfs":{"total":96636764160,"available":96582545408,"internally_reserved":0,"allocated":26546176,"data_stored":26142910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280530,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70232,"kb_used_data":43208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94301608,"statfs":{"total":96636764160,"available":96564846592,"internally_reserved":0,"allocated":44244992,"data_stored":43779048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280530,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69980,"kb_used_data":41244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94301860,"statfs":{"total":96636764160,"available":96565104640,"internally_reserved":0,"allocated":42233856,"data_stored":41678101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280530,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40672,"kb_used_data":9488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331168,"statfs":{"total":96636764160,"available":96595116032,"internally_reserved":0,"allocated":9715712,"data_stored":9333506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,5,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280530,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":4460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":4567040,"data_stored":4192379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247825,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50588,"kb_used_data":19404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321252,"statfs":{"total":96636764160,"available":96584962048,"internally_reserved":0,"allocated":19869696,"data_stored":19465576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247824,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73464,"kb_used_data":38408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94298376,"statfs":{"total":96636764160,"available":96561537024,"internally_reserved":0,"allocated":39329792,"data_stored":38752284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9228288,"data_stored":9177810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12275712,"data_stored":12220744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":41488384,"data_stored":41347574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":51109888,"data_stored":50972632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":30228480,"data_stored":30146426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":19992576,"data_stored":19913398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":12275712,"data_stored":12220744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":21708800,"data_stored":21643527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":33632256,"data_stored":33512921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14561280,"data_stored":14479350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12881920,"data_stored":12815309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":18632704,"data_stored":18592395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":18432000,"data_stored":18403897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9097216,"data_stored":9076566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25866240,"data_stored":25833496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":37806080,"data_stored":37604614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12517376,"data_stored":12507560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":11317248,"data_stored":11290928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":10485760,"data_stored":10477819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":29052928,"data_stored":28922663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8810496,"data_stored":8798812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":12652544,"data_stored":12627410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":17252352,"data_stored":17242192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":16818176,"data_stored":16783277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":3661824,"data_stored":3657685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:11:43.163 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pgp_num 26 2024-09-13T10:11:43.181 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2204178 2024-09-13T10:11:43.208 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:43.206+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:43.267 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:43.263+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:43.282 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:43.283 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:11:43.283 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:11:43.283 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:11:43.283 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.283 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:251 4.5 4:a71ca386:::smithi19530714-2:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.283 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:11:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 52.161939828999998, 2024-09-13T10:11:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041813326999999997, 2024-09-13T10:11:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 251 2024-09-13T10:11:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.284 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:11:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:11:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117990+0000", 2024-09-13T10:11:43.285 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.203e-06 2024-09-13T10:11:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117997+0000", 2024-09-13T10:11:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9750000000000001e-06 2024-09-13T10:11:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118009+0000", 2024-09-13T10:11:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2478e-05 2024-09-13T10:11:43.286 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.287 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.287 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.287 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118017+0000", 2024-09-13T10:11:43.287 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6909999999999997e-06 2024-09-13T10:11:43.287 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.287 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.287 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.287 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118106+0000", 2024-09-13T10:11:43.288 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.9331000000000002e-05 2024-09-13T10:11:43.288 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.288 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.288 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.288 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118184+0000", 2024-09-13T10:11:43.288 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7448999999999997e-05 2024-09-13T10:11:43.288 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.288 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.288 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,10", 2024-09-13T10:11:43.288 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158604+0000", 2024-09-13T10:11:43.289 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.040419954000000001 2024-09-13T10:11:43.289 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.289 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.289 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.289 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159306+0000", 2024-09-13T10:11:43.289 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00070229299999999999 2024-09-13T10:11:43.290 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.290 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.290 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.290 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159539+0000", 2024-09-13T10:11:43.290 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023353600000000001 2024-09-13T10:11:43.290 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.290 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.290 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.290 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159696+0000", 2024-09-13T10:11:43.291 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015616 2024-09-13T10:11:43.291 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.291 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.291 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.291 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159754+0000", 2024-09-13T10:11:43.291 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8513e-05 2024-09-13T10:11:43.291 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.291 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.291 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159802+0000", 2024-09-13T10:11:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.7744000000000002e-05 2024-09-13T10:11:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:270 4.5 4:a59e88f7:::smithi19530714-21:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:11:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 52.053893703, 2024-09-13T10:11:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041472594000000002, 2024-09-13T10:11:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.292 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 270 2024-09-13T10:11:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:11:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.293 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.294 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.294 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.294 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:11:43.294 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.294 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.294 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.294 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.294 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226037+0000", 2024-09-13T10:11:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6560000000000001e-06 2024-09-13T10:11:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226048+0000", 2024-09-13T10:11:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0893000000000001e-05 2024-09-13T10:11:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226079+0000", 2024-09-13T10:11:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0412999999999998e-05 2024-09-13T10:11:43.295 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226106+0000", 2024-09-13T10:11:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7160000000000001e-05 2024-09-13T10:11:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226234+0000", 2024-09-13T10:11:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000128393 2024-09-13T10:11:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.296 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226388+0000", 2024-09-13T10:11:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015424300000000001 2024-09-13T10:11:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,10", 2024-09-13T10:11:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262647+0000", 2024-09-13T10:11:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036258460999999999 2024-09-13T10:11:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264690+0000", 2024-09-13T10:11:43.297 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020429889999999998 2024-09-13T10:11:43.298 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.298 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.298 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.298 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.266720+0000", 2024-09-13T10:11:43.298 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002029669 2024-09-13T10:11:43.298 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.298 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.298 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.299 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267350+0000", 2024-09-13T10:11:43.299 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00063076699999999996 2024-09-13T10:11:43.299 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.299 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.299 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.299 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267409+0000", 2024-09-13T10:11:43.299 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8366000000000003e-05 2024-09-13T10:11:43.299 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.299 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.299 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.300 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267507+0000", 2024-09-13T10:11:43.300 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.8584000000000003e-05 2024-09-13T10:11:43.300 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.300 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.300 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.300 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.300 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.301 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:275 4.2 4:4ac406d9:::smithi19530714-26:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.301 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:11:43.301 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 52.050146640000001, 2024-09-13T10:11:43.301 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024603505000000001, 2024-09-13T10:11:43.301 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.301 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.301 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.301 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.301 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.301 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.301 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 275 2024-09-13T10:11:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:11:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:11:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.302 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.303 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.303 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.303 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229783+0000", 2024-09-13T10:11:43.303 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.44e-06 2024-09-13T10:11:43.303 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.303 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.303 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.303 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229793+0000", 2024-09-13T10:11:43.303 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0163000000000001e-05 2024-09-13T10:11:43.303 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.303 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229814+0000", 2024-09-13T10:11:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0564000000000002e-05 2024-09-13T10:11:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229831+0000", 2024-09-13T10:11:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7142e-05 2024-09-13T10:11:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.304 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229949+0000", 2024-09-13T10:11:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000118014 2024-09-13T10:11:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230081+0000", 2024-09-13T10:11:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013181299999999999 2024-09-13T10:11:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,14", 2024-09-13T10:11:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.236541+0000", 2024-09-13T10:11:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0064597680000000003 2024-09-13T10:11:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.305 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.306 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.238773+0000", 2024-09-13T10:11:43.306 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022326680000000002 2024-09-13T10:11:43.306 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.306 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.306 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.306 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.241978+0000", 2024-09-13T10:11:43.306 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00320455 2024-09-13T10:11:43.306 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254159+0000", 2024-09-13T10:11:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012181149 2024-09-13T10:11:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254259+0000", 2024-09-13T10:11:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000100184 2024-09-13T10:11:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.307 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254385+0000", 2024-09-13T10:11:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012605 2024-09-13T10:11:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:270 4.5 4:a59e88f7:::smithi19530714-21:head [assert-version v9,set-chunk] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:11:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 52.046492209999997, 2024-09-13T10:11:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033985986000000003, 2024-09-13T10:11:43.308 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "waiting for ondisk", 2024-09-13T10:11:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 270 2024-09-13T10:11:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:11:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.309 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:11:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233438+0000", 2024-09-13T10:11:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.187e-06 2024-09-13T10:11:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.310 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.311 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233449+0000", 2024-09-13T10:11:43.311 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0926e-05 2024-09-13T10:11:43.311 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.311 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.311 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.311 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233477+0000", 2024-09-13T10:11:43.311 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7515999999999999e-05 2024-09-13T10:11:43.311 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.311 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.311 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.311 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233499+0000", 2024-09-13T10:11:43.312 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1719999999999999e-05 2024-09-13T10:11:43.312 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.312 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.312 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.312 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233670+0000", 2024-09-13T10:11:43.312 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000171146 2024-09-13T10:11:43.312 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.312 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.312 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for blocked object", 2024-09-13T10:11:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233823+0000", 2024-09-13T10:11:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015309300000000001 2024-09-13T10:11:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.263191+0000", 2024-09-13T10:11:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029368157999999998 2024-09-13T10:11:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for ondisk", 2024-09-13T10:11:43.313 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.263301+0000", 2024-09-13T10:11:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110214 2024-09-13T10:11:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267422+0000", 2024-09-13T10:11:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0041210259999999999 2024-09-13T10:11:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:277 4.4 4:20c7201f:::smithi19530714-28:head [assert-version v9,set-chunk] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.314 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:11:43.315 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 52.045886125000003, 2024-09-13T10:11:43.315 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.042430906999999997, 2024-09-13T10:11:43.315 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.315 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.315 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.316 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.316 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.316 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.316 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 277 2024-09-13T10:11:43.316 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.316 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.317 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.317 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.317 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:11:43.317 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.317 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.318 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.318 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.318 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:11:43.318 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.318 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.319 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.319 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.319 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234043+0000", 2024-09-13T10:11:43.319 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.13e-06 2024-09-13T10:11:43.319 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.320 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.320 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.320 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234050+0000", 2024-09-13T10:11:43.320 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.263e-06 2024-09-13T10:11:43.320 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.321 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.321 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.321 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234065+0000", 2024-09-13T10:11:43.321 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5207e-05 2024-09-13T10:11:43.321 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.321 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.322 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.322 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234078+0000", 2024-09-13T10:11:43.322 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3643e-05 2024-09-13T10:11:43.322 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.322 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.323 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.323 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.235383+0000", 2024-09-13T10:11:43.323 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0013048269999999999 2024-09-13T10:11:43.323 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.324 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.324 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.324 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.235604+0000", 2024-09-13T10:11:43.324 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022116900000000001 2024-09-13T10:11:43.324 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.325 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.325 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:43.325 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.271455+0000", 2024-09-13T10:11:43.325 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035850173999999999 2024-09-13T10:11:43.326 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.326 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.326 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.326 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273083+0000", 2024-09-13T10:11:43.327 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0016287649999999999 2024-09-13T10:11:43.327 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.327 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.327 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.328 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.275756+0000", 2024-09-13T10:11:43.328 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0026721140000000002 2024-09-13T10:11:43.328 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.328 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.328 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.329 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276255+0000", 2024-09-13T10:11:43.329 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00049961600000000004 2024-09-13T10:11:43.329 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.329 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.330 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.330 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276401+0000", 2024-09-13T10:11:43.330 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014584099999999999 2024-09-13T10:11:43.330 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.330 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.330 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.331 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276473+0000", 2024-09-13T10:11:43.331 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2157999999999996e-05 2024-09-13T10:11:43.331 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.331 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.332 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.332 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.332 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.332 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:324 4.4 4:26b855f3:::smithi19530714-25:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.332 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:11:43.333 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 51.999223878999999, 2024-09-13T10:11:43.333 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033888489000000001, 2024-09-13T10:11:43.333 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.333 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.333 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.334 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.334 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.334 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.334 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 324 2024-09-13T10:11:43.334 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.335 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.335 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.335 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.335 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:11:43.335 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.336 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.336 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.336 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.336 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:11:43.336 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.337 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.337 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.337 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.337 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280706+0000", 2024-09-13T10:11:43.337 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6479999999999999e-06 2024-09-13T10:11:43.338 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.338 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.338 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.338 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280715+0000", 2024-09-13T10:11:43.338 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1500000000000005e-06 2024-09-13T10:11:43.339 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.339 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.339 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.339 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280733+0000", 2024-09-13T10:11:43.339 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8168e-05 2024-09-13T10:11:43.339 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.340 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.340 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.340 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280747+0000", 2024-09-13T10:11:43.340 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3661e-05 2024-09-13T10:11:43.340 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.341 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.341 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.341 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281013+0000", 2024-09-13T10:11:43.341 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026565099999999999 2024-09-13T10:11:43.341 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.342 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.342 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.342 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281124+0000", 2024-09-13T10:11:43.342 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000111715 2024-09-13T10:11:43.342 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.342 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.343 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:43.343 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286502+0000", 2024-09-13T10:11:43.343 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0053775259999999997 2024-09-13T10:11:43.343 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.343 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.344 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.344 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.289334+0000", 2024-09-13T10:11:43.344 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0028316779999999998 2024-09-13T10:11:43.344 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.344 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.345 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.345 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.290331+0000", 2024-09-13T10:11:43.345 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0009974929999999999 2024-09-13T10:11:43.345 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.345 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.346 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.346 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314453+0000", 2024-09-13T10:11:43.346 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024121417999999999 2024-09-13T10:11:43.346 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.346 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.347 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.347 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314531+0000", 2024-09-13T10:11:43.347 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8841e-05 2024-09-13T10:11:43.347 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.347 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.348 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.348 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314593+0000", 2024-09-13T10:11:43.348 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.1539999999999997e-05 2024-09-13T10:11:43.348 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.348 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.349 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.349 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.349 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.349 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:327 4.4 4:20c7201f:::smithi19530714-28:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.349 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:11:43.350 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 51.998756049999997, 2024-09-13T10:11:43.350 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035855512999999999, 2024-09-13T10:11:43.350 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.350 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.350 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.351 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.351 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.351 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.351 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 327 2024-09-13T10:11:43.351 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.352 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.352 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.352 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.352 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:11:43.352 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.353 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.353 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.353 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.353 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:11:43.353 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.354 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.354 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.356 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.356 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281173+0000", 2024-09-13T10:11:43.356 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0020000000000001e-06 2024-09-13T10:11:43.357 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.357 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.357 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.357 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281180+0000", 2024-09-13T10:11:43.357 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.8260000000000004e-06 2024-09-13T10:11:43.358 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.358 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.358 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.358 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281192+0000", 2024-09-13T10:11:43.358 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1725e-05 2024-09-13T10:11:43.359 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.359 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.359 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.359 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281202+0000", 2024-09-13T10:11:43.359 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0492000000000001e-05 2024-09-13T10:11:43.359 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.360 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.360 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.360 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281396+0000", 2024-09-13T10:11:43.360 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019404200000000001 2024-09-13T10:11:43.360 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.361 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.361 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.361 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281533+0000", 2024-09-13T10:11:43.361 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000136267 2024-09-13T10:11:43.361 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.362 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.362 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:43.362 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.313834+0000", 2024-09-13T10:11:43.362 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032301491000000002 2024-09-13T10:11:43.362 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.363 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.363 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.363 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314993+0000", 2024-09-13T10:11:43.363 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001158585 2024-09-13T10:11:43.363 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.363 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.364 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.364 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.315097+0000", 2024-09-13T10:11:43.364 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000104685 2024-09-13T10:11:43.364 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.364 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.364 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.364 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.316808+0000", 2024-09-13T10:11:43.365 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0017110529999999999 2024-09-13T10:11:43.365 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.365 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.365 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.365 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.316946+0000", 2024-09-13T10:11:43.365 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000137737 2024-09-13T10:11:43.365 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.365 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.365 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.365 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.317028+0000", 2024-09-13T10:11:43.365 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.1607999999999995e-05 2024-09-13T10:11:43.365 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.366 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.366 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.366 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.366 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.366 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:335 4.2 4:4a24ecac:::smithi19530714-36:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.366 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:11:43.366 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 51.997437304000002, 2024-09-13T10:11:43.366 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032260373000000002, 2024-09-13T10:11:43.366 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.366 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.366 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.366 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.366 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.367 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.367 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 335 2024-09-13T10:11:43.367 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.367 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.367 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.367 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.367 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:11:43.367 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.367 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.367 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.367 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.367 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:11:43.367 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.368 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.368 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.368 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.368 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282494+0000", 2024-09-13T10:11:43.368 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5390000000000002e-06 2024-09-13T10:11:43.368 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.368 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.368 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.368 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282506+0000", 2024-09-13T10:11:43.368 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2646000000000001e-05 2024-09-13T10:11:43.368 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.368 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.368 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.369 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282539+0000", 2024-09-13T10:11:43.369 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3176000000000002e-05 2024-09-13T10:11:43.369 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.369 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.369 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.369 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282558+0000", 2024-09-13T10:11:43.369 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9065000000000001e-05 2024-09-13T10:11:43.369 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.369 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.369 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.369 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282697+0000", 2024-09-13T10:11:43.369 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013857999999999999 2024-09-13T10:11:43.370 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.370 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.370 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.370 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282874+0000", 2024-09-13T10:11:43.370 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017656900000000001 2024-09-13T10:11:43.370 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.370 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.370 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,14", 2024-09-13T10:11:43.370 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.287911+0000", 2024-09-13T10:11:43.370 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0050369220000000001 2024-09-13T10:11:43.370 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.370 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.370 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.371 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.290491+0000", 2024-09-13T10:11:43.371 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025802500000000001 2024-09-13T10:11:43.371 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.371 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.371 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.371 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.291374+0000", 2024-09-13T10:11:43.371 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00088367800000000005 2024-09-13T10:11:43.371 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.371 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.371 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.371 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314557+0000", 2024-09-13T10:11:43.371 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023182094 2024-09-13T10:11:43.371 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.372 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.372 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.372 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314664+0000", 2024-09-13T10:11:43.372 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010751099999999999 2024-09-13T10:11:43.372 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.372 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.372 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.372 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314751+0000", 2024-09-13T10:11:43.372 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7342999999999994e-05 2024-09-13T10:11:43.372 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.372 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.372 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.372 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.372 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.373 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:337 4.4 4:24275569:::smithi19530714-38:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.373 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:11:43.373 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 51.996780753000003, 2024-09-13T10:11:43.373 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.031660147, 2024-09-13T10:11:43.373 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.373 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.373 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.373 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.373 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.373 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.373 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 337 2024-09-13T10:11:43.373 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.373 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.374 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.374 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.374 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:11:43.374 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.374 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.374 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.374 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.374 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:11:43.374 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.374 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.374 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.374 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.374 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283149+0000", 2024-09-13T10:11:43.374 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.655e-06 2024-09-13T10:11:43.375 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.375 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.375 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.375 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283159+0000", 2024-09-13T10:11:43.375 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.9529999999999994e-06 2024-09-13T10:11:43.375 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.375 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.375 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.375 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283177+0000", 2024-09-13T10:11:43.375 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8168e-05 2024-09-13T10:11:43.375 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.375 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.375 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.375 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283193+0000", 2024-09-13T10:11:43.376 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5563e-05 2024-09-13T10:11:43.376 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.376 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.376 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.376 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283317+0000", 2024-09-13T10:11:43.376 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012421700000000001 2024-09-13T10:11:43.376 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.376 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.376 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.376 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283478+0000", 2024-09-13T10:11:43.376 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000160378 2024-09-13T10:11:43.376 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.377 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.377 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:43.377 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.289593+0000", 2024-09-13T10:11:43.377 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0061152200000000002 2024-09-13T10:11:43.377 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.377 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.377 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.377 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.291636+0000", 2024-09-13T10:11:43.377 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020435430000000001 2024-09-13T10:11:43.377 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.377 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.377 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.377 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294186+0000", 2024-09-13T10:11:43.377 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025494760000000002 2024-09-13T10:11:43.378 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.378 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.378 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.378 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314672+0000", 2024-09-13T10:11:43.378 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020486392999999999 2024-09-13T10:11:43.378 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.378 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.378 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.378 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314741+0000", 2024-09-13T10:11:43.378 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9003000000000003e-05 2024-09-13T10:11:43.378 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.378 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.378 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.379 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314808+0000", 2024-09-13T10:11:43.379 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.6577999999999999e-05 2024-09-13T10:11:43.379 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.379 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.379 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.379 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.379 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.379 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1865 4.12 4:4ac406d9:::smithi19530714-26:head [read 0~2050454,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:43.379 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.728406+0000", 2024-09-13T10:11:43.379 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 31.551522177999999, 2024-09-13T10:11:43.379 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029384652000000001, 2024-09-13T10:11:43.379 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.379 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.380 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:43.380 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.380 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.380 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.380 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1865 2024-09-13T10:11:43.380 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.380 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.380 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.380 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.380 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728406+0000", 2024-09-13T10:11:43.380 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.380 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.380 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.381 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.381 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728406+0000", 2024-09-13T10:11:43.381 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.381 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.381 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.381 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728410+0000", 2024-09-13T10:11:43.381 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9129999999999996e-06 2024-09-13T10:11:43.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.381 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.381 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.381 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728423+0000", 2024-09-13T10:11:43.381 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2425999999999999e-05 2024-09-13T10:11:43.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.382 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.382 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.382 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728453+0000", 2024-09-13T10:11:43.382 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0673999999999998e-05 2024-09-13T10:11:43.382 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.382 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.382 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.382 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728478+0000", 2024-09-13T10:11:43.382 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5099000000000001e-05 2024-09-13T10:11:43.382 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.382 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.382 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.382 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730729+0000", 2024-09-13T10:11:43.383 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002251009 2024-09-13T10:11:43.383 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.383 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.383 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:11:43.383 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730758+0000", 2024-09-13T10:11:43.383 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8623e-05 2024-09-13T10:11:43.383 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.383 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.383 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.383 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.754517+0000", 2024-09-13T10:11:43.383 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023759199000000002 2024-09-13T10:11:43.383 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.383 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.383 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.384 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.754761+0000", 2024-09-13T10:11:43.384 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000243923 2024-09-13T10:11:43.384 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.384 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.384 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.384 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.757791+0000", 2024-09-13T10:11:43.384 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003029786 2024-09-13T10:11:43.384 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.384 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.384 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.384 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.384 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.384 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1866 4.12 4:4ac406d9:::smithi19530714-26:head [sparse-read 0~1216897] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:43.385 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.728515+0000", 2024-09-13T10:11:43.385 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 31.551413123, 2024-09-13T10:11:43.385 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.031206358999999999, 2024-09-13T10:11:43.385 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.385 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.385 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:43.385 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.385 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.385 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.385 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1866 2024-09-13T10:11:43.385 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.385 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.385 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.386 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.386 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728515+0000", 2024-09-13T10:11:43.386 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.386 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.386 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.386 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.386 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728515+0000", 2024-09-13T10:11:43.386 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.386 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.386 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.386 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.386 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728516+0000", 2024-09-13T10:11:43.386 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1429999999999999e-06 2024-09-13T10:11:43.386 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.387 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.387 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.387 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728522+0000", 2024-09-13T10:11:43.387 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5740000000000003e-06 2024-09-13T10:11:43.387 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.387 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.387 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.387 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728534+0000", 2024-09-13T10:11:43.387 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1539000000000001e-05 2024-09-13T10:11:43.387 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.387 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.387 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.387 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728543+0000", 2024-09-13T10:11:43.387 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6059999999999992e-06 2024-09-13T10:11:43.388 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.388 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.388 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.388 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730840+0000", 2024-09-13T10:11:43.388 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022966789999999998 2024-09-13T10:11:43.388 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.388 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.388 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:11:43.388 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730859+0000", 2024-09-13T10:11:43.388 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9613e-05 2024-09-13T10:11:43.388 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.388 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.388 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.389 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.758405+0000", 2024-09-13T10:11:43.389 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027545656000000002 2024-09-13T10:11:43.389 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.389 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.389 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.389 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.758574+0000", 2024-09-13T10:11:43.389 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000168808 2024-09-13T10:11:43.389 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.389 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.389 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.389 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759722+0000", 2024-09-13T10:11:43.389 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011477410000000001 2024-09-13T10:11:43.389 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.389 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.390 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.390 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.390 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.390 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1867 4.12 4:4ac406d9:::smithi19530714-26:head [read 116628~1933826,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:43.390 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.728557+0000", 2024-09-13T10:11:43.390 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 31.551371640999999, 2024-09-13T10:11:43.390 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033656772000000001, 2024-09-13T10:11:43.390 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.390 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.390 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:43.390 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.390 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.390 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.390 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1867 2024-09-13T10:11:43.391 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.391 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.391 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.391 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.391 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728557+0000", 2024-09-13T10:11:43.391 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.391 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.391 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.391 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.391 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728557+0000", 2024-09-13T10:11:43.391 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.391 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.391 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.391 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.392 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728558+0000", 2024-09-13T10:11:43.392 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8400000000000003e-07 2024-09-13T10:11:43.392 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.392 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.392 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.392 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728562+0000", 2024-09-13T10:11:43.392 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.8029999999999996e-06 2024-09-13T10:11:43.392 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.392 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.392 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.392 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728589+0000", 2024-09-13T10:11:43.392 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6536000000000001e-05 2024-09-13T10:11:43.392 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.393 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.393 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.393 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728603+0000", 2024-09-13T10:11:43.393 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4483e-05 2024-09-13T10:11:43.393 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.393 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.393 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.393 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730933+0000", 2024-09-13T10:11:43.393 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023295939999999999 2024-09-13T10:11:43.393 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.393 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.393 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:11:43.393 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730952+0000", 2024-09-13T10:11:43.394 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8614000000000002e-05 2024-09-13T10:11:43.394 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.394 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.394 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.394 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759881+0000", 2024-09-13T10:11:43.394 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.02892962 2024-09-13T10:11:43.394 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.394 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.394 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.394 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.760061+0000", 2024-09-13T10:11:43.394 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00018033 2024-09-13T10:11:43.394 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.394 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.395 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.395 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.762214+0000", 2024-09-13T10:11:43.395 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002152008 2024-09-13T10:11:43.395 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.395 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.395 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.395 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.395 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.395 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1932 4.5 4:a3098556:::smithi19530714-22:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:43.395 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.759351+0000", 2024-09-13T10:11:43.395 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 31.520577228000001, 2024-09-13T10:11:43.395 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.043979880999999998, 2024-09-13T10:11:43.396 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.396 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.396 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:43.396 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.396 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.396 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.396 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1932 2024-09-13T10:11:43.396 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.396 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.396 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.396 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.396 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759351+0000", 2024-09-13T10:11:43.396 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.397 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.397 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.397 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.397 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759351+0000", 2024-09-13T10:11:43.397 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.397 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.397 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.397 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.397 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759352+0000", 2024-09-13T10:11:43.397 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.28e-06 2024-09-13T10:11:43.397 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.397 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.397 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.397 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759358+0000", 2024-09-13T10:11:43.398 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8699999999999997e-06 2024-09-13T10:11:43.398 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.398 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.398 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.398 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759376+0000", 2024-09-13T10:11:43.398 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7532000000000001e-05 2024-09-13T10:11:43.398 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.398 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.398 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.398 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759391+0000", 2024-09-13T10:11:43.398 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5590000000000002e-05 2024-09-13T10:11:43.398 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.399 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.399 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.399 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759607+0000", 2024-09-13T10:11:43.399 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021536999999999999 2024-09-13T10:11:43.399 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.399 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.399 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.399 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759831+0000", 2024-09-13T10:11:43.399 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000224243 2024-09-13T10:11:43.399 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.399 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.399 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.399 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.803331+0000", 2024-09-13T10:11:43.399 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.043499995999999999 2024-09-13T10:11:43.400 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.400 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.400 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.400 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.400 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.400 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2189 4.4 4:26b855f3:::smithi19530714-25:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:11:43.400 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:17.462885+0000", 2024-09-13T10:11:43.400 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 25.817043521999999, 2024-09-13T10:11:43.400 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041082475, 2024-09-13T10:11:43.400 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.400 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.400 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:43.400 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.401 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.401 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.401 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2189 2024-09-13T10:11:43.401 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.401 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.401 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.401 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.401 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462885+0000", 2024-09-13T10:11:43.401 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.401 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.401 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.401 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.401 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462885+0000", 2024-09-13T10:11:43.401 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.402 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.402 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.402 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.402 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462888+0000", 2024-09-13T10:11:43.402 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.74e-06 2024-09-13T10:11:43.402 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.402 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.402 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.402 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462898+0000", 2024-09-13T10:11:43.402 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0618999999999999e-05 2024-09-13T10:11:43.402 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.402 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.402 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.403 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462928+0000", 2024-09-13T10:11:43.403 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9685000000000001e-05 2024-09-13T10:11:43.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.403 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.403 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.403 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462944+0000", 2024-09-13T10:11:43.403 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6175999999999999e-05 2024-09-13T10:11:43.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.403 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.403 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.403 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.463063+0000", 2024-09-13T10:11:43.403 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000119393 2024-09-13T10:11:43.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.403 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.404 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.404 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.463204+0000", 2024-09-13T10:11:43.404 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014020600000000001 2024-09-13T10:11:43.404 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.404 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.404 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.404 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.503967+0000", 2024-09-13T10:11:43.404 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.040763656000000002 2024-09-13T10:11:43.404 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.404 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.404 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.404 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.404 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.405 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2832 4.4 4:26b855f3:::smithi19530714-25:head [read 0~3613505,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:43.405 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.892960+0000", 2024-09-13T10:11:43.405 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.386968255999999, 2024-09-13T10:11:43.405 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.026786087, 2024-09-13T10:11:43.405 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.405 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.405 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:43.405 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.405 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.405 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.405 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2832 2024-09-13T10:11:43.405 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.405 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.406 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.406 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.406 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892960+0000", 2024-09-13T10:11:43.406 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.406 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.406 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.406 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.406 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892960+0000", 2024-09-13T10:11:43.406 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.406 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.406 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.406 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.406 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892962+0000", 2024-09-13T10:11:43.407 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.573e-06 2024-09-13T10:11:43.407 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.407 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.407 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.407 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892969+0000", 2024-09-13T10:11:43.407 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.277e-06 2024-09-13T10:11:43.407 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.407 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.407 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.407 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892984+0000", 2024-09-13T10:11:43.407 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5013e-05 2024-09-13T10:11:43.407 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.407 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.407 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.408 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892997+0000", 2024-09-13T10:11:43.408 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2692e-05 2024-09-13T10:11:43.408 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.408 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.408 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.408 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.917535+0000", 2024-09-13T10:11:43.408 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024538806 2024-09-13T10:11:43.408 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.408 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.408 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.408 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.917788+0000", 2024-09-13T10:11:43.408 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025253100000000001 2024-09-13T10:11:43.409 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.409 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.409 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.409 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.919746+0000", 2024-09-13T10:11:43.409 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019581949999999998 2024-09-13T10:11:43.409 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.409 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.409 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.409 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.409 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.409 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2833 4.4 4:26b855f3:::smithi19530714-25:head [read 2395497~1218008,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:43.409 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.893015+0000", 2024-09-13T10:11:43.409 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.386913451, 2024-09-13T10:11:43.410 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027607530000000002, 2024-09-13T10:11:43.410 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.410 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.410 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:43.410 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.410 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.410 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.410 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2833 2024-09-13T10:11:43.410 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.410 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.410 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.410 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.410 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893015+0000", 2024-09-13T10:11:43.411 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.411 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.411 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.411 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.411 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893015+0000", 2024-09-13T10:11:43.411 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.411 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.411 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.411 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.411 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893016+0000", 2024-09-13T10:11:43.411 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.156e-06 2024-09-13T10:11:43.411 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.411 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.412 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.412 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893024+0000", 2024-09-13T10:11:43.412 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5900000000000002e-06 2024-09-13T10:11:43.412 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.412 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.412 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.412 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893039+0000", 2024-09-13T10:11:43.412 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4949999999999999e-05 2024-09-13T10:11:43.412 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.412 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.412 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.412 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893052+0000", 2024-09-13T10:11:43.412 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3362e-05 2024-09-13T10:11:43.413 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.413 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.413 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.413 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.919791+0000", 2024-09-13T10:11:43.413 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.026739237999999999 2024-09-13T10:11:43.413 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.413 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.413 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.413 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.919930+0000", 2024-09-13T10:11:43.413 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000139202 2024-09-13T10:11:43.413 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.413 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.413 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.414 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.920622+0000", 2024-09-13T10:11:43.414 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00069203199999999998 2024-09-13T10:11:43.414 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.414 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.414 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.414 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.414 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.414 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2840 4.4 4:24da8222:::smithi19530714-49:head [sparse-read 2104672~451177,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:43.414 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.893098+0000", 2024-09-13T10:11:43.414 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.386829884000001, 2024-09-13T10:11:43.414 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029549518, 2024-09-13T10:11:43.414 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.414 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.415 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:43.415 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.415 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.415 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.415 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2840 2024-09-13T10:11:43.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.415 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.415 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.415 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.415 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893098+0000", 2024-09-13T10:11:43.415 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.415 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.416 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.416 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893098+0000", 2024-09-13T10:11:43.416 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.416 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.416 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.416 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893107+0000", 2024-09-13T10:11:43.416 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7430000000000008e-06 2024-09-13T10:11:43.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.416 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.416 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.416 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893117+0000", 2024-09-13T10:11:43.416 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7000000000000003e-06 2024-09-13T10:11:43.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.417 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.417 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.417 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893136+0000", 2024-09-13T10:11:43.417 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8705000000000001e-05 2024-09-13T10:11:43.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.417 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.417 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.417 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893151+0000", 2024-09-13T10:11:43.417 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5183e-05 2024-09-13T10:11:43.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.417 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.417 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.417 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.920721+0000", 2024-09-13T10:11:43.418 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027569917999999999 2024-09-13T10:11:43.418 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.418 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.418 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.418 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.920886+0000", 2024-09-13T10:11:43.418 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016508699999999999 2024-09-13T10:11:43.418 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.418 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.418 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.418 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.922648+0000", 2024-09-13T10:11:43.418 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0017621819999999999 2024-09-13T10:11:43.418 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.418 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.418 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.419 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.419 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2841 4.4 4:24da8222:::smithi19530714-49:head [read 0~3042067,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:43.419 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.893173+0000", 2024-09-13T10:11:43.419 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.386755463, 2024-09-13T10:11:43.419 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.037712765000000002, 2024-09-13T10:11:43.419 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.419 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.419 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:43.419 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.419 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.419 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.419 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2841 2024-09-13T10:11:43.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.420 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.420 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.420 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.420 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893173+0000", 2024-09-13T10:11:43.420 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.420 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.420 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.420 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.420 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893173+0000", 2024-09-13T10:11:43.420 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.420 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.420 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.420 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.420 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893174+0000", 2024-09-13T10:11:43.421 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4959999999999999e-06 2024-09-13T10:11:43.421 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.421 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.421 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.421 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893182+0000", 2024-09-13T10:11:43.421 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7060000000000004e-06 2024-09-13T10:11:43.421 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.421 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.421 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.421 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893201+0000", 2024-09-13T10:11:43.421 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8655999999999999e-05 2024-09-13T10:11:43.421 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.421 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.421 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.422 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893215+0000", 2024-09-13T10:11:43.422 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4513e-05 2024-09-13T10:11:43.422 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.422 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.422 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.422 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.922869+0000", 2024-09-13T10:11:43.422 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029653845000000002 2024-09-13T10:11:43.422 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.422 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.422 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.422 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.923431+0000", 2024-09-13T10:11:43.422 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00056212899999999997 2024-09-13T10:11:43.422 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.422 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.423 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.423 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.930886+0000", 2024-09-13T10:11:43.423 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0074544199999999998 2024-09-13T10:11:43.423 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.423 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.423 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.423 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.423 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.423 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2842 4.4 4:24da8222:::smithi19530714-49:head [sparse-read 0~1861544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:43.423 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.893233+0000", 2024-09-13T10:11:43.423 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.386694983, 2024-09-13T10:11:43.423 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.038772649999999999, 2024-09-13T10:11:43.423 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.424 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.424 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:43.424 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.424 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.424 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.424 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2842 2024-09-13T10:11:43.424 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.424 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.424 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.424 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.424 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893233+0000", 2024-09-13T10:11:43.424 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.424 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.424 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.425 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.425 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893233+0000", 2024-09-13T10:11:43.425 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.425 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.425 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.425 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893235+0000", 2024-09-13T10:11:43.425 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.217e-06 2024-09-13T10:11:43.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.425 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.425 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.425 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893240+0000", 2024-09-13T10:11:43.425 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.6999999999999996e-06 2024-09-13T10:11:43.426 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.426 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.426 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.426 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893254+0000", 2024-09-13T10:11:43.426 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.401e-05 2024-09-13T10:11:43.426 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.426 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.426 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.426 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893267+0000", 2024-09-13T10:11:43.426 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3135e-05 2024-09-13T10:11:43.426 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.426 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.426 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.426 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.930989+0000", 2024-09-13T10:11:43.427 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.037721452000000003 2024-09-13T10:11:43.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.427 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.427 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.427 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.931214+0000", 2024-09-13T10:11:43.427 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000225529 2024-09-13T10:11:43.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.427 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.427 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.427 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.932006+0000", 2024-09-13T10:11:43.427 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00079160700000000005 2024-09-13T10:11:43.427 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.427 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.427 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.428 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.428 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3003 4.c 4:3e6706e5:::smithi19530714-14:head [read 0~2084101,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-13T10:11:43.428 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:37.319096+0000", 2024-09-13T10:11:43.428 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 5.9608326720000004, 2024-09-13T10:11:43.428 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024544567, 2024-09-13T10:11:43.428 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.428 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.428 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:43.428 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.428 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.428 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.428 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3003 2024-09-13T10:11:43.429 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.429 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.429 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.429 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.429 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.319096+0000", 2024-09-13T10:11:43.429 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.429 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.429 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.429 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.429 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.319096+0000", 2024-09-13T10:11:43.429 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.429 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.429 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.429 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.430 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.319097+0000", 2024-09-13T10:11:43.430 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5999999999999999e-06 2024-09-13T10:11:43.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.430 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.430 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.430 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.319107+0000", 2024-09-13T10:11:43.430 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5429999999999999e-06 2024-09-13T10:11:43.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.430 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.430 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.430 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.319124+0000", 2024-09-13T10:11:43.430 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7436e-05 2024-09-13T10:11:43.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.430 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.431 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.431 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.319138+0000", 2024-09-13T10:11:43.431 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4209999999999999e-05 2024-09-13T10:11:43.431 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.431 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.431 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.431 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.340175+0000", 2024-09-13T10:11:43.431 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021036837999999999 2024-09-13T10:11:43.431 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.431 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.431 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.431 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.340370+0000", 2024-09-13T10:11:43.431 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019481000000000001 2024-09-13T10:11:43.431 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.432 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.432 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.432 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.343640+0000", 2024-09-13T10:11:43.432 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00327013 2024-09-13T10:11:43.432 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.432 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.432 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.432 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.432 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.432 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:43.433 INFO:tasks.rados.rados.0.smithi195.stdout:1439: finishing set_chunk to oid 4 2024-09-13T10:11:43.433 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 105 (ObjNum 284 snap 0 seq_num 284) dirty exists 2024-09-13T10:11:43.433 INFO:tasks.rados.rados.0.smithi195.stdout:1443: finishing write tid 1 to smithi19530714-46 2024-09-13T10:11:43.433 INFO:tasks.rados.rados.0.smithi195.stdout:1443: finishing write tid 2 to smithi19530714-46 2024-09-13T10:11:43.433 INFO:tasks.rados.rados.0.smithi195.stdout:1442: expect (ObjNum 231 snap 0 seq_num 231) 2024-09-13T10:11:43.434 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-13T10:11:43.435 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:43.427+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:43.465 INFO:tasks.rados.rados.0.smithi195.stdout:1443: finishing write tid 3 to smithi19530714-46 2024-09-13T10:11:43.465 INFO:tasks.rados.rados.0.smithi195.stdout:1443: finishing write tid 4 to smithi19530714-46 2024-09-13T10:11:43.465 INFO:tasks.rados.rados.0.smithi195.stdout:1443: finishing write tid 5 to smithi19530714-46 2024-09-13T10:11:43.465 INFO:tasks.rados.rados.0.smithi195.stdout:1443: oid 46 updating version 0 to 47 2024-09-13T10:11:43.466 INFO:tasks.rados.rados.0.smithi195.stdout:1443: oid 46 updating version 47 to 48 2024-09-13T10:11:43.466 INFO:tasks.rados.rados.0.smithi195.stdout:1443: oid 46 version 48 is already newer than 46 2024-09-13T10:11:43.466 INFO:tasks.rados.rados.0.smithi195.stdout:1443: oid 46 updating version 48 to 49 2024-09-13T10:11:43.466 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 49 (ObjNum 295 snap 0 seq_num 295) dirty exists 2024-09-13T10:11:43.466 INFO:tasks.rados.rados.0.smithi195.stdout:1443: left oid 46 (ObjNum 295 snap 0 seq_num 295) 2024-09-13T10:11:43.466 INFO:tasks.rados.rados.0.smithi195.stdout:1445: expect (ObjNum 235 snap 0 seq_num 235) 2024-09-13T10:11:43.466 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:43.535 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:43.531+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:43.606 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:43.606 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:43.606 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:11:43.606 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:43.620 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-13T10:11:43.636 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:43.631+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:43.658 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3903447 2024-09-13T10:11:43.659 INFO:tasks.rados.rados.0.smithi195.stdout:1446: expect (ObjNum 276 snap 0 seq_num 276) 2024-09-13T10:11:43.665 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1396464 2024-09-13T10:11:43.736 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:43.735+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:43.747 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:43.747 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:43.747 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:11:43.747 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:11:43.747 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:43.765 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-13T10:11:43.774 INFO:tasks.rados.rados.0.smithi195.stdout:1435: finishing write tid 1 to smithi19530714-26 2024-09-13T10:11:43.774 INFO:tasks.rados.rados.0.smithi195.stdout:1435: finishing write tid 2 to smithi19530714-26 2024-09-13T10:11:43.774 INFO:tasks.rados.rados.0.smithi195.stdout:1435: finishing write tid 3 to smithi19530714-26 2024-09-13T10:11:43.774 INFO:tasks.rados.rados.0.smithi195.stdout:1435: finishing write tid 4 to smithi19530714-26 2024-09-13T10:11:43.774 INFO:tasks.rados.rados.0.smithi195.stdout:1435: oid 26 updating version 0 to 42 2024-09-13T10:11:43.774 INFO:tasks.rados.rados.0.smithi195.stdout:1435: oid 26 version 42 is already newer than 41 2024-09-13T10:11:43.774 INFO:tasks.rados.rados.0.smithi195.stdout:1435: oid 26 version 42 is already newer than 40 2024-09-13T10:11:43.774 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 42 (ObjNum 292 snap 0 seq_num 292) dirty exists 2024-09-13T10:11:43.775 INFO:tasks.rados.rados.0.smithi195.stdout:1435: left oid 26 (ObjNum 292 snap 0 seq_num 292) 2024-09-13T10:11:43.775 INFO:tasks.rados.rados.0.smithi195.stdout:1444: expect (ObjNum 275 snap 0 seq_num 275) 2024-09-13T10:11:43.775 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 264340 2024-09-13T10:11:43.817 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:43.837 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:43.831+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:43.892 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:43.892 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:11:43.892 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:11:43.892 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:11:43.892 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.892 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:201 3.3 3:d4fb61be:::smithi19530714-2 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.892 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:11:43.892 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 52.894822196, 2024-09-13T10:11:43.892 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.052937043000000003, 2024-09-13T10:11:43.892 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.892 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.893 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.893 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.893 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.893 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.893 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 201 2024-09-13T10:11:43.893 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.893 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.893 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.893 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.893 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:11:43.893 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.893 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.894 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.894 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.894 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:11:43.894 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.894 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.894 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.894 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.894 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993192+0000", 2024-09-13T10:11:43.894 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8559999999999996e-06 2024-09-13T10:11:43.894 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.894 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.894 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.895 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993203+0000", 2024-09-13T10:11:43.895 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1113e-05 2024-09-13T10:11:43.895 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.895 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.895 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.895 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993225+0000", 2024-09-13T10:11:43.895 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2793e-05 2024-09-13T10:11:43.895 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.895 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.895 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.895 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993241+0000", 2024-09-13T10:11:43.895 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5806000000000001e-05 2024-09-13T10:11:43.895 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.896 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.896 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.896 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993401+0000", 2024-09-13T10:11:43.896 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000160149 2024-09-13T10:11:43.896 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.896 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.896 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.896 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994047+0000", 2024-09-13T10:11:43.896 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00064594100000000003 2024-09-13T10:11:43.896 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.896 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.896 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:43.896 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.011243+0000", 2024-09-13T10:11:43.897 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017196026999999999 2024-09-13T10:11:43.897 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.897 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.897 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.897 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029096+0000", 2024-09-13T10:11:43.897 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017852115000000002 2024-09-13T10:11:43.897 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.897 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.897 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.897 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029271+0000", 2024-09-13T10:11:43.897 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017589299999999999 2024-09-13T10:11:43.897 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.898 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.898 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.898 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.045867+0000", 2024-09-13T10:11:43.898 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016595625999999999 2024-09-13T10:11:43.898 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.898 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.898 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.898 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.046017+0000", 2024-09-13T10:11:43.898 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014974199999999999 2024-09-13T10:11:43.898 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.898 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.898 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.899 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.046120+0000", 2024-09-13T10:11:43.899 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000102982 2024-09-13T10:11:43.899 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.899 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.899 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.899 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.899 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.899 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:208 3.3 3:dbb0d8aa:::smithi19530714-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.900 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:11:43.900 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 52.894726386000002, 2024-09-13T10:11:43.900 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.071216475000000001, 2024-09-13T10:11:43.900 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.900 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.900 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.900 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.900 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.900 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.900 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 208 2024-09-13T10:11:43.901 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.901 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.901 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.901 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.901 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:11:43.901 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.901 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.902 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.902 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.902 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:11:43.902 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.902 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.902 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.902 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.902 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993280+0000", 2024-09-13T10:11:43.902 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1999999999999999e-06 2024-09-13T10:11:43.902 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.902 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.903 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.903 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993289+0000", 2024-09-13T10:11:43.903 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8200000000000003e-06 2024-09-13T10:11:43.903 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.903 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.903 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.903 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994000+0000", 2024-09-13T10:11:43.903 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00071192499999999997 2024-09-13T10:11:43.903 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.904 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.904 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.904 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994024+0000", 2024-09-13T10:11:43.904 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3578999999999999e-05 2024-09-13T10:11:43.904 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.905 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.905 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.905 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994468+0000", 2024-09-13T10:11:43.905 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00044410599999999999 2024-09-13T10:11:43.905 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.905 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.905 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.906 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994842+0000", 2024-09-13T10:11:43.906 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037380300000000003 2024-09-13T10:11:43.906 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.906 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.906 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:43.906 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031572+0000", 2024-09-13T10:11:43.906 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036730479000000003 2024-09-13T10:11:43.906 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.906 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.906 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.906 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.039007+0000", 2024-09-13T10:11:43.907 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.007434871 2024-09-13T10:11:43.907 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.907 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.907 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.907 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.041001+0000", 2024-09-13T10:11:43.907 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001994104 2024-09-13T10:11:43.907 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.907 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.908 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.908 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064330+0000", 2024-09-13T10:11:43.908 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023328912 2024-09-13T10:11:43.908 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.908 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.908 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.908 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064401+0000", 2024-09-13T10:11:43.909 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.1123000000000003e-05 2024-09-13T10:11:43.909 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.909 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.909 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.909 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064495+0000", 2024-09-13T10:11:43.909 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.3553000000000004e-05 2024-09-13T10:11:43.910 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.910 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.910 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.910 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.910 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.910 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:215 3.3 3:ed6c0a17:::smithi19530714-16 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.910 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:11:43.910 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 52.893946131999996, 2024-09-13T10:11:43.910 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.055806517999999999, 2024-09-13T10:11:43.910 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.910 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.911 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.911 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.911 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.911 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.911 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 215 2024-09-13T10:11:43.911 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.911 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.911 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.911 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.911 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:11:43.911 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.911 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.912 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.912 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.912 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:11:43.912 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.912 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.912 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.912 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.912 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994060+0000", 2024-09-13T10:11:43.912 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5030000000000001e-06 2024-09-13T10:11:43.912 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.912 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.913 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.913 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994070+0000", 2024-09-13T10:11:43.913 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0046e-05 2024-09-13T10:11:43.913 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.913 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.913 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.913 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994087+0000", 2024-09-13T10:11:43.913 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6283999999999999e-05 2024-09-13T10:11:43.913 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.913 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.913 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.913 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994096+0000", 2024-09-13T10:11:43.913 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.969e-06 2024-09-13T10:11:43.914 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.914 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.914 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.914 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.995226+0000", 2024-09-13T10:11:43.914 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011299540000000001 2024-09-13T10:11:43.914 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.914 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.914 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.914 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.995548+0000", 2024-09-13T10:11:43.914 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00032266199999999998 2024-09-13T10:11:43.914 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.915 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.915 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:43.915 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.018922+0000", 2024-09-13T10:11:43.915 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023373485999999999 2024-09-13T10:11:43.915 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.915 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.915 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.915 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029400+0000", 2024-09-13T10:11:43.915 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010478721 2024-09-13T10:11:43.915 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.915 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.915 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.916 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038932+0000", 2024-09-13T10:11:43.916 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0095316859999999993 2024-09-13T10:11:43.916 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.916 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.916 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.916 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049620+0000", 2024-09-13T10:11:43.916 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010687688000000001 2024-09-13T10:11:43.916 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.916 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.917 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.917 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049754+0000", 2024-09-13T10:11:43.917 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013457700000000001 2024-09-13T10:11:43.917 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.917 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.917 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.917 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049865+0000", 2024-09-13T10:11:43.917 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110942 2024-09-13T10:11:43.917 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.917 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.917 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.918 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.918 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.918 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:224 3.3 3:f941658c:::smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.918 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:11:43.918 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 52.837438095000003, 2024-09-13T10:11:43.918 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.025014027000000001, 2024-09-13T10:11:43.918 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.918 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.918 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.918 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.918 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.918 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.918 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 224 2024-09-13T10:11:43.919 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.919 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.919 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.919 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.919 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:11:43.919 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.919 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.919 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.919 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.919 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:11:43.919 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.919 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.920 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.920 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.920 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050570+0000", 2024-09-13T10:11:43.920 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9799999999999998e-06 2024-09-13T10:11:43.920 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.920 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.920 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.920 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050582+0000", 2024-09-13T10:11:43.920 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2170999999999999e-05 2024-09-13T10:11:43.920 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.920 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.920 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.921 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050605+0000", 2024-09-13T10:11:43.921 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3543000000000001e-05 2024-09-13T10:11:43.921 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.921 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.921 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.921 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050619+0000", 2024-09-13T10:11:43.921 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3478999999999999e-05 2024-09-13T10:11:43.921 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.921 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.921 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.921 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050725+0000", 2024-09-13T10:11:43.921 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000105585 2024-09-13T10:11:43.921 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.922 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.922 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.922 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.051086+0000", 2024-09-13T10:11:43.922 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036162100000000001 2024-09-13T10:11:43.922 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.922 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.922 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:43.922 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.058628+0000", 2024-09-13T10:11:43.922 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075415569999999999 2024-09-13T10:11:43.922 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.922 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.922 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.922 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065344+0000", 2024-09-13T10:11:43.923 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0067161740000000001 2024-09-13T10:11:43.923 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.923 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.923 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.923 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.066222+0000", 2024-09-13T10:11:43.923 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00087765599999999999 2024-09-13T10:11:43.923 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.923 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.923 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.923 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075372+0000", 2024-09-13T10:11:43.923 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0091499609999999999 2024-09-13T10:11:43.923 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.923 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.924 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.924 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075489+0000", 2024-09-13T10:11:43.924 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011718300000000001 2024-09-13T10:11:43.924 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.924 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.924 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.924 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075581+0000", 2024-09-13T10:11:43.924 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.2116999999999999e-05 2024-09-13T10:11:43.924 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.924 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.924 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.924 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.924 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.925 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:226 3.3 3:fe5bfa3a:::smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.925 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:11:43.925 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 52.827905086000001, 2024-09-13T10:11:43.925 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051515848000000003, 2024-09-13T10:11:43.925 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.925 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.925 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.925 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.925 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.925 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.925 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 226 2024-09-13T10:11:43.925 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.926 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.926 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.926 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.926 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:11:43.926 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.926 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.926 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.926 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.926 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:11:43.926 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.926 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.926 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.926 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.927 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060102+0000", 2024-09-13T10:11:43.927 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0480000000000001e-06 2024-09-13T10:11:43.927 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.927 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.927 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.927 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060112+0000", 2024-09-13T10:11:43.927 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6649999999999998e-06 2024-09-13T10:11:43.927 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.927 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.927 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.928 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060132+0000", 2024-09-13T10:11:43.928 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0262e-05 2024-09-13T10:11:43.928 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.928 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.928 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.928 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060143+0000", 2024-09-13T10:11:43.928 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1420000000000001e-05 2024-09-13T10:11:43.928 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.928 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.928 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.928 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064563+0000", 2024-09-13T10:11:43.928 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044199119999999998 2024-09-13T10:11:43.929 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.929 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.929 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.929 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064882+0000", 2024-09-13T10:11:43.929 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000318735 2024-09-13T10:11:43.929 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.929 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.929 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:43.929 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.078858+0000", 2024-09-13T10:11:43.929 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013976455 2024-09-13T10:11:43.929 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.929 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.930 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.930 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088977+0000", 2024-09-13T10:11:43.930 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010118544 2024-09-13T10:11:43.930 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.930 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.930 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.930 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.090451+0000", 2024-09-13T10:11:43.930 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001474026 2024-09-13T10:11:43.930 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.930 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.930 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.930 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111539+0000", 2024-09-13T10:11:43.931 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.02108786 2024-09-13T10:11:43.931 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.931 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.931 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.931 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111580+0000", 2024-09-13T10:11:43.931 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1245000000000002e-05 2024-09-13T10:11:43.931 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.932 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.932 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.932 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111616+0000", 2024-09-13T10:11:43.932 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5676000000000002e-05 2024-09-13T10:11:43.932 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.932 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.932 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.932 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.932 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.933 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:235 3.3 3:e217e96e:::smithi19530714-36 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.933 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:11:43.933 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 52.817127456999998, 2024-09-13T10:11:43.933 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051184682000000002, 2024-09-13T10:11:43.933 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.933 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.933 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.933 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.933 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.933 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.934 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 235 2024-09-13T10:11:43.934 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.934 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.934 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.934 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.934 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:11:43.934 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.936 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.936 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.936 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.936 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:11:43.936 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.936 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.936 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.936 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.936 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070881+0000", 2024-09-13T10:11:43.936 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0579999999999999e-06 2024-09-13T10:11:43.937 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.937 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.937 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.937 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070897+0000", 2024-09-13T10:11:43.937 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5967000000000001e-05 2024-09-13T10:11:43.937 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.937 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.937 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.938 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070942+0000", 2024-09-13T10:11:43.938 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4947000000000002e-05 2024-09-13T10:11:43.938 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.938 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.938 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.938 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070963+0000", 2024-09-13T10:11:43.938 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0098e-05 2024-09-13T10:11:43.938 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.938 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.939 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.939 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.071099+0000", 2024-09-13T10:11:43.939 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013697199999999999 2024-09-13T10:11:43.939 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.939 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.939 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.939 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.071330+0000", 2024-09-13T10:11:43.939 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000230881 2024-09-13T10:11:43.940 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.940 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.940 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:43.940 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.084668+0000", 2024-09-13T10:11:43.940 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013337663 2024-09-13T10:11:43.940 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.940 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.941 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.941 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.092656+0000", 2024-09-13T10:11:43.941 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0079878880000000003 2024-09-13T10:11:43.941 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.941 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.941 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.941 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.100238+0000", 2024-09-13T10:11:43.941 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075819449999999997 2024-09-13T10:11:43.942 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.942 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.942 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.942 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121877+0000", 2024-09-13T10:11:43.942 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021639104999999999 2024-09-13T10:11:43.942 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.942 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.942 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.942 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121978+0000", 2024-09-13T10:11:43.943 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000101363 2024-09-13T10:11:43.943 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.943 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.943 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.943 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.122062+0000", 2024-09-13T10:11:43.943 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.3795000000000007e-05 2024-09-13T10:11:43.943 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.943 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.943 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.943 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.944 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.944 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:237 3.3 3:de40747e:::smithi19530714-38 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.944 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:11:43.944 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 52.808938253000001, 2024-09-13T10:11:43.944 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.073341331999999995, 2024-09-13T10:11:43.945 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.945 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.945 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.945 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.945 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.945 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.946 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 237 2024-09-13T10:11:43.946 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.946 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.946 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.946 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.946 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:11:43.947 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.947 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.947 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.947 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.947 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:11:43.947 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.947 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.948 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.948 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.948 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079069+0000", 2024-09-13T10:11:43.948 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6979999999999998e-06 2024-09-13T10:11:43.948 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.948 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.948 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.949 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079083+0000", 2024-09-13T10:11:43.949 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.384e-05 2024-09-13T10:11:43.949 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.949 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.949 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.949 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079105+0000", 2024-09-13T10:11:43.950 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1736e-05 2024-09-13T10:11:43.950 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.950 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.950 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.950 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079123+0000", 2024-09-13T10:11:43.951 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7889000000000001e-05 2024-09-13T10:11:43.951 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.951 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.951 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.951 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088459+0000", 2024-09-13T10:11:43.951 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0093359700000000007 2024-09-13T10:11:43.952 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.952 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.952 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.952 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088726+0000", 2024-09-13T10:11:43.952 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026732299999999997 2024-09-13T10:11:43.953 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.953 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.953 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:43.953 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.097522+0000", 2024-09-13T10:11:43.953 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0087962630000000003 2024-09-13T10:11:43.953 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.954 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.954 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.954 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.103972+0000", 2024-09-13T10:11:43.954 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0064494840000000001 2024-09-13T10:11:43.954 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.954 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.954 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.954 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.104160+0000", 2024-09-13T10:11:43.954 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00018782200000000001 2024-09-13T10:11:43.955 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.955 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.955 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.955 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152294+0000", 2024-09-13T10:11:43.955 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048134458999999997 2024-09-13T10:11:43.955 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.955 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.955 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.955 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152373+0000", 2024-09-13T10:11:43.955 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8574999999999999e-05 2024-09-13T10:11:43.955 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.956 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.956 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.956 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152408+0000", 2024-09-13T10:11:43.956 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5272999999999999e-05 2024-09-13T10:11:43.956 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.956 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.956 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.956 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.956 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.956 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:238 3.3 3:c43dd60b:::smithi19530714-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.956 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:11:43.957 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 52.799352605999999, 2024-09-13T10:11:43.957 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.069426360000000006, 2024-09-13T10:11:43.957 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.957 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.957 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.957 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.957 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.957 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.957 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 238 2024-09-13T10:11:43.957 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.958 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.958 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.958 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.958 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:11:43.958 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.958 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.958 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.958 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.958 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:11:43.958 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.959 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.959 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.959 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.959 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088655+0000", 2024-09-13T10:11:43.959 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2359999999999999e-06 2024-09-13T10:11:43.959 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.959 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.959 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.959 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088663+0000", 2024-09-13T10:11:43.959 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7800000000000006e-06 2024-09-13T10:11:43.959 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.959 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.960 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.960 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088688+0000", 2024-09-13T10:11:43.960 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4507000000000001e-05 2024-09-13T10:11:43.960 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.960 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.960 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.960 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088717+0000", 2024-09-13T10:11:43.960 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9204e-05 2024-09-13T10:11:43.960 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.961 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.961 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.961 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.089038+0000", 2024-09-13T10:11:43.961 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00032087199999999999 2024-09-13T10:11:43.961 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.961 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.961 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.961 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.089435+0000", 2024-09-13T10:11:43.961 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039756600000000002 2024-09-13T10:11:43.961 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.962 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.962 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:43.962 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.101672+0000", 2024-09-13T10:11:43.962 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012236528 2024-09-13T10:11:43.962 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.962 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.962 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.962 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.105256+0000", 2024-09-13T10:11:43.962 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0035837159999999998 2024-09-13T10:11:43.962 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.962 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.963 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.963 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111504+0000", 2024-09-13T10:11:43.963 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.006248577 2024-09-13T10:11:43.963 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.963 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.963 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.963 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.157950+0000", 2024-09-13T10:11:43.963 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.046446031999999998 2024-09-13T10:11:43.963 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.963 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.963 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.963 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158027+0000", 2024-09-13T10:11:43.964 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6402999999999993e-05 2024-09-13T10:11:43.964 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.964 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.964 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.964 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158079+0000", 2024-09-13T10:11:43.964 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.1938999999999999e-05 2024-09-13T10:11:43.964 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.964 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.964 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.964 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.965 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.965 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:246 3.3 3:f1be28e7:::smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.965 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:11:43.965 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 52.774641858999999, 2024-09-13T10:11:43.965 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051630810999999999, 2024-09-13T10:11:43.965 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.965 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.965 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.965 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.965 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.965 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.966 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 246 2024-09-13T10:11:43.966 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.966 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.966 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.966 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.966 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:11:43.966 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.966 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.966 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.967 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.967 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:11:43.967 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.967 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.967 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.967 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.967 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113365+0000", 2024-09-13T10:11:43.967 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.762e-06 2024-09-13T10:11:43.967 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.967 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.967 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.967 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113372+0000", 2024-09-13T10:11:43.967 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5020000000000003e-06 2024-09-13T10:11:43.968 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.968 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.968 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.968 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113390+0000", 2024-09-13T10:11:43.968 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8056000000000001e-05 2024-09-13T10:11:43.968 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.969 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.969 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.969 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113400+0000", 2024-09-13T10:11:43.969 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5119999999999997e-06 2024-09-13T10:11:43.969 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.969 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.969 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.969 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113456+0000", 2024-09-13T10:11:43.969 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.6221999999999998e-05 2024-09-13T10:11:43.969 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.969 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.970 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.970 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113576+0000", 2024-09-13T10:11:43.970 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000119466 2024-09-13T10:11:43.970 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.970 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.970 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:43.970 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.129127+0000", 2024-09-13T10:11:43.970 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015551003000000001 2024-09-13T10:11:43.970 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.970 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.970 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.970 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142482+0000", 2024-09-13T10:11:43.970 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013355104 2024-09-13T10:11:43.971 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.971 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.971 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.971 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142518+0000", 2024-09-13T10:11:43.971 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6532000000000002e-05 2024-09-13T10:11:43.971 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.971 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.971 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.971 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164919+0000", 2024-09-13T10:11:43.971 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022401038000000002 2024-09-13T10:11:43.971 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.971 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.972 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.972 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164962+0000", 2024-09-13T10:11:43.972 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.2329000000000001e-05 2024-09-13T10:11:43.972 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.972 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.972 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.973 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164994+0000", 2024-09-13T10:11:43.973 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2285e-05 2024-09-13T10:11:43.973 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.973 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.973 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.973 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.974 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.974 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:249 3.3 3:f565152a:::smithi19530714-50 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.974 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:11:43.974 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 52.773039382, 2024-09-13T10:11:43.974 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051233251, 2024-09-13T10:11:43.974 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.974 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.974 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.974 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:43.974 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:43.975 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:43.975 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 249 2024-09-13T10:11:43.975 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.975 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.975 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.975 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.975 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:11:43.975 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.975 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.976 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.976 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.976 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:11:43.976 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.976 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.976 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.976 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.976 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114967+0000", 2024-09-13T10:11:43.977 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.313e-06 2024-09-13T10:11:43.977 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.977 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.977 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.977 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114972+0000", 2024-09-13T10:11:43.977 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5620000000000003e-06 2024-09-13T10:11:43.977 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.977 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.977 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.977 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114997+0000", 2024-09-13T10:11:43.978 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4845999999999999e-05 2024-09-13T10:11:43.978 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.978 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.978 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.978 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115007+0000", 2024-09-13T10:11:43.978 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7200000000000001e-06 2024-09-13T10:11:43.978 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.978 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.979 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.979 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115076+0000", 2024-09-13T10:11:43.979 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9473000000000004e-05 2024-09-13T10:11:43.979 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.979 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.979 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.980 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115221+0000", 2024-09-13T10:11:43.980 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000144604 2024-09-13T10:11:43.980 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.980 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.980 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:43.980 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.136862+0000", 2024-09-13T10:11:43.981 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021641365999999999 2024-09-13T10:11:43.981 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.981 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.981 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.981 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.145775+0000", 2024-09-13T10:11:43.982 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0089130649999999995 2024-09-13T10:11:43.982 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.982 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.982 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.982 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.150578+0000", 2024-09-13T10:11:43.982 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0048025999999999998 2024-09-13T10:11:43.983 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.983 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.983 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.983 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166133+0000", 2024-09-13T10:11:43.983 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015554853 2024-09-13T10:11:43.983 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.983 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.983 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.983 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166167+0000", 2024-09-13T10:11:43.984 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3973999999999997e-05 2024-09-13T10:11:43.984 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.984 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.984 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.984 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166199+0000", 2024-09-13T10:11:43.984 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1875000000000002e-05 2024-09-13T10:11:43.984 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.984 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.984 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.984 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.984 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.985 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.6.23:1 3.3 3:d4fb61be:::smithi19530714-2 low_tier:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:43.985 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:11:43.985 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 52.767876332999997, 2024-09-13T10:11:43.985 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.038168746000000003, 2024-09-13T10:11:43.985 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.985 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.985 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.985 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.985 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.985 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.985 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:11:43.985 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.986 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.986 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.986 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.986 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:11:43.986 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.986 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.986 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.986 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.986 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120131+0000", 2024-09-13T10:11:43.986 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5620000000000002e-06 2024-09-13T10:11:43.986 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.987 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.987 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.987 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120140+0000", 2024-09-13T10:11:43.987 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1889999999999996e-06 2024-09-13T10:11:43.987 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.987 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.987 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.987 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120161+0000", 2024-09-13T10:11:43.987 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0485000000000001e-05 2024-09-13T10:11:43.987 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.987 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.988 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.988 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120200+0000", 2024-09-13T10:11:43.988 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9648999999999999e-05 2024-09-13T10:11:43.988 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.988 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.988 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.988 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120285+0000", 2024-09-13T10:11:43.988 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.4488999999999997e-05 2024-09-13T10:11:43.988 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.988 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.988 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.989 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120399+0000", 2024-09-13T10:11:43.989 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000113857 2024-09-13T10:11:43.989 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.989 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.989 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:43.989 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120640+0000", 2024-09-13T10:11:43.989 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024112600000000001 2024-09-13T10:11:43.989 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.990 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.990 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.990 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121715+0000", 2024-09-13T10:11:43.990 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001074885 2024-09-13T10:11:43.990 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.990 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.990 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.990 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.122870+0000", 2024-09-13T10:11:43.990 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00115504 2024-09-13T10:11:43.990 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.990 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.990 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.991 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158160+0000", 2024-09-13T10:11:43.991 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035289760000000003 2024-09-13T10:11:43.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.991 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.991 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.991 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158211+0000", 2024-09-13T10:11:43.991 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.1005999999999999e-05 2024-09-13T10:11:43.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.991 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.991 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.991 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158297+0000", 2024-09-13T10:11:43.991 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.6698000000000002e-05 2024-09-13T10:11:43.992 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.992 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.992 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.992 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.992 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.992 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.8.23:15 3.3 3:e24de40a:::8acc92edede4f51df2e7506dbd189cab8b5b01cc17028d04a4fc401eaf5ce1ea:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:43.992 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:06.115725+0000", 2024-09-13T10:11:43.992 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 37.772279466000001, 2024-09-13T10:11:43.992 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012080047999999999, 2024-09-13T10:11:43.992 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.992 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.992 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.993 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.993 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.993 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.993 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.115725+0000", 2024-09-13T10:11:43.993 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.993 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.993 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.993 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:43.993 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.115725+0000", 2024-09-13T10:11:43.993 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.993 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.993 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.994 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:43.994 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.115727+0000", 2024-09-13T10:11:43.994 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9939999999999999e-06 2024-09-13T10:11:43.994 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.994 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.994 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:43.994 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116297+0000", 2024-09-13T10:11:43.994 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00056970499999999999 2024-09-13T10:11:43.994 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.994 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.994 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:43.994 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116330+0000", 2024-09-13T10:11:43.994 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2727e-05 2024-09-13T10:11:43.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.995 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.995 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:43.995 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116352+0000", 2024-09-13T10:11:43.995 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2569e-05 2024-09-13T10:11:43.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.995 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.995 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:43.995 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.119140+0000", 2024-09-13T10:11:43.995 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0027870780000000001 2024-09-13T10:11:43.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.995 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.995 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:43.996 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.119363+0000", 2024-09-13T10:11:43.996 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022384299999999999 2024-09-13T10:11:43.996 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.996 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.996 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:43.996 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.119746+0000", 2024-09-13T10:11:43.996 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000382533 2024-09-13T10:11:43.996 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.996 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.996 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:43.996 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.127251+0000", 2024-09-13T10:11:43.996 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075055720000000003 2024-09-13T10:11:43.997 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.997 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.997 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.997 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.127366+0000", 2024-09-13T10:11:43.997 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000114695 2024-09-13T10:11:43.997 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.997 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.997 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:43.997 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.127516+0000", 2024-09-13T10:11:43.997 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014997 2024-09-13T10:11:43.997 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.997 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.997 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:43.998 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.127700+0000", 2024-09-13T10:11:43.998 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000184267 2024-09-13T10:11:43.998 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.998 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.998 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:43.998 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.127805+0000", 2024-09-13T10:11:43.998 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000105095 2024-09-13T10:11:43.998 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.998 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:43.998 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:43.998 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.998 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.998 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.8.23:23 3.3 3:e316fc5a:::252a0f0dcd13ac7cc8768c369d40341dfd148109baa17147b0e0aef9e7d09876:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:43.999 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:06.116669+0000", 2024-09-13T10:11:43.999 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 37.77133628, 2024-09-13T10:11:43.999 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012469427, 2024-09-13T10:11:43.999 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:43.999 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:43.999 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:43.999 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:43.999 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:43.999 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:43.999 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116669+0000", 2024-09-13T10:11:43.999 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:43.999 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:43.999 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.000 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.000 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116669+0000", 2024-09-13T10:11:44.000 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.000 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.000 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.000 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.000 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116670+0000", 2024-09-13T10:11:44.000 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.756e-06 2024-09-13T10:11:44.000 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.000 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.000 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.000 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116850+0000", 2024-09-13T10:11:44.001 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017943700000000001 2024-09-13T10:11:44.001 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.001 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.001 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.001 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116878+0000", 2024-09-13T10:11:44.001 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8119e-05 2024-09-13T10:11:44.001 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.001 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.002 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.002 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.116901+0000", 2024-09-13T10:11:44.002 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2963999999999999e-05 2024-09-13T10:11:44.002 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.002 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.002 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.002 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.125063+0000", 2024-09-13T10:11:44.002 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.008162117 2024-09-13T10:11:44.002 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.002 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.002 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.002 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.125376+0000", 2024-09-13T10:11:44.003 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000312627 2024-09-13T10:11:44.003 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.003 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.003 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:44.003 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.125950+0000", 2024-09-13T10:11:44.003 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00057483899999999995 2024-09-13T10:11:44.003 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.003 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.003 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.004 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.128105+0000", 2024-09-13T10:11:44.004 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0021549630000000002 2024-09-13T10:11:44.004 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.004 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.004 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.004 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.128661+0000", 2024-09-13T10:11:44.004 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00055533700000000004 2024-09-13T10:11:44.004 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.004 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.005 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.005 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.128932+0000", 2024-09-13T10:11:44.005 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00027158199999999998 2024-09-13T10:11:44.005 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.005 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.005 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.005 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.129040+0000", 2024-09-13T10:11:44.005 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000107871 2024-09-13T10:11:44.005 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.006 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.006 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.006 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:06.129138+0000", 2024-09-13T10:11:44.006 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7814999999999998e-05 2024-09-13T10:11:44.006 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.006 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.006 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.007 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.007 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.14.23:42 3.3 3:ce7ec0f0:::302c6a3bdf6d249958d305a37c5742c0de5d7cad722f2dabd19f570d2f795527:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:44.007 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:10.780924+0000", 2024-09-13T10:11:44.007 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 33.107080633000002, 2024-09-13T10:11:44.007 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013436228, 2024-09-13T10:11:44.007 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.007 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.007 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.007 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.007 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.007 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.008 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.780924+0000", 2024-09-13T10:11:44.008 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.008 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.008 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.008 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.008 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.780924+0000", 2024-09-13T10:11:44.008 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.008 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.009 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.009 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.009 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.780927+0000", 2024-09-13T10:11:44.009 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3860000000000001e-06 2024-09-13T10:11:44.009 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.009 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.009 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.009 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781100+0000", 2024-09-13T10:11:44.009 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000172967 2024-09-13T10:11:44.010 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.010 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.010 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.010 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781123+0000", 2024-09-13T10:11:44.010 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.302e-05 2024-09-13T10:11:44.010 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.010 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.010 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.011 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781141+0000", 2024-09-13T10:11:44.011 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8692999999999999e-05 2024-09-13T10:11:44.011 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.011 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.011 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.011 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.785827+0000", 2024-09-13T10:11:44.011 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0046856850000000002 2024-09-13T10:11:44.011 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.012 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.012 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.012 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.786033+0000", 2024-09-13T10:11:44.012 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00020591000000000001 2024-09-13T10:11:44.012 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.012 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.012 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:44.012 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.786402+0000", 2024-09-13T10:11:44.013 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036895 2024-09-13T10:11:44.013 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.013 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.013 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.013 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.793957+0000", 2024-09-13T10:11:44.013 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075548940000000004 2024-09-13T10:11:44.013 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.013 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.014 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.014 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794103+0000", 2024-09-13T10:11:44.014 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014615799999999999 2024-09-13T10:11:44.014 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.014 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.014 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.014 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794217+0000", 2024-09-13T10:11:44.014 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011437799999999999 2024-09-13T10:11:44.014 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.014 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.014 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.014 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794297+0000", 2024-09-13T10:11:44.014 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9431999999999994e-05 2024-09-13T10:11:44.015 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.015 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.015 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.015 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794360+0000", 2024-09-13T10:11:44.015 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.3755000000000002e-05 2024-09-13T10:11:44.015 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.015 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.015 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.015 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.015 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.015 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.14.23:44 3.3 3:e2816a99:::d4bb749789c90c884ffd20ac62f804c7a6b80502d14432fa627e1239249ee2da:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:44.015 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:10.781168+0000", 2024-09-13T10:11:44.015 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 33.106836723999997, 2024-09-13T10:11:44.016 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013562833, 2024-09-13T10:11:44.016 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.016 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.016 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.016 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.016 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.016 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.016 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781168+0000", 2024-09-13T10:11:44.017 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.017 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.017 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.017 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.017 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781168+0000", 2024-09-13T10:11:44.017 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.017 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.017 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.017 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.017 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781170+0000", 2024-09-13T10:11:44.017 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5400000000000001e-06 2024-09-13T10:11:44.017 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.018 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.018 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.018 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781470+0000", 2024-09-13T10:11:44.018 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00030077100000000002 2024-09-13T10:11:44.018 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.018 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.018 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.018 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781508+0000", 2024-09-13T10:11:44.018 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.7517e-05 2024-09-13T10:11:44.018 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.019 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.019 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.019 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781527+0000", 2024-09-13T10:11:44.019 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8766e-05 2024-09-13T10:11:44.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.019 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.019 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.019 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.788166+0000", 2024-09-13T10:11:44.019 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0066389450000000003 2024-09-13T10:11:44.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.019 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.019 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.019 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.788515+0000", 2024-09-13T10:11:44.020 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00034976099999999997 2024-09-13T10:11:44.020 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.020 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.020 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:44.020 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.789180+0000", 2024-09-13T10:11:44.020 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00066460200000000003 2024-09-13T10:11:44.020 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.020 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.020 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.020 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794399+0000", 2024-09-13T10:11:44.020 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0052185340000000004 2024-09-13T10:11:44.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.021 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.021 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.021 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794487+0000", 2024-09-13T10:11:44.021 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8660999999999997e-05 2024-09-13T10:11:44.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.021 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.021 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.021 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794601+0000", 2024-09-13T10:11:44.021 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000114188 2024-09-13T10:11:44.022 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.022 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.022 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.022 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794678+0000", 2024-09-13T10:11:44.022 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6058999999999998e-05 2024-09-13T10:11:44.022 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.022 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.022 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.022 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794731+0000", 2024-09-13T10:11:44.022 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.3489000000000002e-05 2024-09-13T10:11:44.022 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.022 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.022 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.023 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.14.23:47 3.3 3:ffec9690:::b0d2dba9f365561c17cd4cce8c8a8f4d8f931d05ca998f81ae8f5caa986fcf1e:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:44.023 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:10.781559+0000", 2024-09-13T10:11:44.023 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 33.106445426999997, 2024-09-13T10:11:44.023 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015576100000000001, 2024-09-13T10:11:44.023 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.023 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.023 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.023 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.023 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.023 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781559+0000", 2024-09-13T10:11:44.024 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.024 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.024 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.024 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781559+0000", 2024-09-13T10:11:44.024 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.024 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.024 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.024 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781561+0000", 2024-09-13T10:11:44.024 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5460000000000001e-06 2024-09-13T10:11:44.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.025 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.025 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.025 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781708+0000", 2024-09-13T10:11:44.025 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000147291 2024-09-13T10:11:44.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.025 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.025 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.025 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781735+0000", 2024-09-13T10:11:44.025 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7175999999999999e-05 2024-09-13T10:11:44.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.025 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.025 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.026 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781755+0000", 2024-09-13T10:11:44.026 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9653e-05 2024-09-13T10:11:44.026 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.026 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.026 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.026 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.790861+0000", 2024-09-13T10:11:44.026 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0091056920000000003 2024-09-13T10:11:44.026 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.026 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.026 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.026 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.791072+0000", 2024-09-13T10:11:44.026 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000211103 2024-09-13T10:11:44.027 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.027 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.027 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:44.027 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.791511+0000", 2024-09-13T10:11:44.027 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00043908899999999998 2024-09-13T10:11:44.027 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.027 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.027 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.027 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794857+0000", 2024-09-13T10:11:44.028 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0033462940000000001 2024-09-13T10:11:44.028 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.028 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.028 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.028 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794945+0000", 2024-09-13T10:11:44.028 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7626000000000004e-05 2024-09-13T10:11:44.028 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.028 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.028 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.028 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.797001+0000", 2024-09-13T10:11:44.028 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020557280000000002 2024-09-13T10:11:44.028 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.029 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.029 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.029 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.797080+0000", 2024-09-13T10:11:44.029 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9169e-05 2024-09-13T10:11:44.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.029 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.029 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.029 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.797136+0000", 2024-09-13T10:11:44.029 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5733000000000003e-05 2024-09-13T10:11:44.029 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.029 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.029 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.030 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.030 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.6.23:158 3.3 3:f5f18922:::5fd497cf663f539dfd95b3997c5a776f4928ee513a2033637dac7a26a4ef0dd6:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:11:44.030 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:18.185966+0000", 2024-09-13T10:11:44.030 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 25.702038742999999, 2024-09-13T10:11:44.030 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012329359999999999, 2024-09-13T10:11:44.030 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.030 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.030 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.030 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.030 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.030 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.030 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.185966+0000", 2024-09-13T10:11:44.031 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.031 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.031 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.031 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.185966+0000", 2024-09-13T10:11:44.031 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.031 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.031 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.031 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.185967+0000", 2024-09-13T10:11:44.031 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2200000000000003e-07 2024-09-13T10:11:44.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.031 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.032 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.032 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186094+0000", 2024-09-13T10:11:44.032 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012747100000000001 2024-09-13T10:11:44.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.032 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.032 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.032 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186104+0000", 2024-09-13T10:11:44.032 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.2560000000000008e-06 2024-09-13T10:11:44.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.032 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.032 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.033 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186108+0000", 2024-09-13T10:11:44.033 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.583e-06 2024-09-13T10:11:44.033 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.033 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.033 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.033 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.192945+0000", 2024-09-13T10:11:44.033 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0068368170000000002 2024-09-13T10:11:44.033 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.033 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.033 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.033 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.193177+0000", 2024-09-13T10:11:44.033 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000232458 2024-09-13T10:11:44.033 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.034 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.034 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:44.034 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.193568+0000", 2024-09-13T10:11:44.034 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039028799999999997 2024-09-13T10:11:44.034 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.034 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.034 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.034 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.196406+0000", 2024-09-13T10:11:44.034 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0028379529999999998 2024-09-13T10:11:44.034 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.034 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.035 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.035 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.196839+0000", 2024-09-13T10:11:44.035 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00043299899999999999 2024-09-13T10:11:44.035 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.035 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.035 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.035 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.198197+0000", 2024-09-13T10:11:44.035 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0013580560000000001 2024-09-13T10:11:44.035 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.035 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.035 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.035 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.198250+0000", 2024-09-13T10:11:44.036 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.3563999999999997e-05 2024-09-13T10:11:44.036 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.036 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.036 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.036 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.198296+0000", 2024-09-13T10:11:44.036 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.5192999999999999e-05 2024-09-13T10:11:44.036 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.036 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.036 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.036 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.036 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.036 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.6.23:159 3.3 3:d330ea58:::b1e0a8f1f87dd65498385354f65ab6aa4dbeb3cb948fa25b5d7b0975437eef68:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:11:44.037 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:18.186115+0000", 2024-09-13T10:11:44.037 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 25.701889426000001, 2024-09-13T10:11:44.037 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.01270717, 2024-09-13T10:11:44.037 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.037 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.037 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.037 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.037 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.037 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.037 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186115+0000", 2024-09-13T10:11:44.037 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.038 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.038 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.038 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186115+0000", 2024-09-13T10:11:44.038 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.038 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.038 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.038 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186116+0000", 2024-09-13T10:11:44.038 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5599999999999995e-07 2024-09-13T10:11:44.039 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.039 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.039 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.039 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186174+0000", 2024-09-13T10:11:44.039 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8397999999999999e-05 2024-09-13T10:11:44.039 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.039 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.039 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.039 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186182+0000", 2024-09-13T10:11:44.039 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.729e-06 2024-09-13T10:11:44.039 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.039 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.040 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.040 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186189+0000", 2024-09-13T10:11:44.040 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4629999999999998e-06 2024-09-13T10:11:44.040 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.040 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.040 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.040 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.195175+0000", 2024-09-13T10:11:44.041 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0089862550000000003 2024-09-13T10:11:44.041 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.041 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.041 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.041 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.195287+0000", 2024-09-13T10:11:44.041 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011233700000000001 2024-09-13T10:11:44.041 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.041 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.041 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:44.041 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.195457+0000", 2024-09-13T10:11:44.041 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016981799999999999 2024-09-13T10:11:44.041 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.042 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.042 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.042 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.196713+0000", 2024-09-13T10:11:44.042 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001256463 2024-09-13T10:11:44.042 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.042 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.042 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.042 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.197341+0000", 2024-09-13T10:11:44.043 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00062791400000000003 2024-09-13T10:11:44.043 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.043 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.043 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.043 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.198733+0000", 2024-09-13T10:11:44.043 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001391403 2024-09-13T10:11:44.043 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.043 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.043 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.043 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.198781+0000", 2024-09-13T10:11:44.044 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.8680000000000001e-05 2024-09-13T10:11:44.044 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.044 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.044 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.044 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.198823+0000", 2024-09-13T10:11:44.044 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1153999999999999e-05 2024-09-13T10:11:44.044 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.044 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.044 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.044 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.044 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.045 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.13.21:65 3.3 3:febf9037:::86029bdee449ecbe1f721f6a717c52067baf9f2cd6735940d23b17d31604c51c:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:44.045 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:29.419053+0000", 2024-09-13T10:11:44.045 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 14.468952292000001, 2024-09-13T10:11:44.045 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013463819, 2024-09-13T10:11:44.045 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.045 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.045 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.045 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.045 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.045 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.045 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.419053+0000", 2024-09-13T10:11:44.045 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.045 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.046 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.046 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.046 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.419053+0000", 2024-09-13T10:11:44.046 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.046 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.046 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.046 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.046 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.419054+0000", 2024-09-13T10:11:44.046 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.657e-06 2024-09-13T10:11:44.046 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.046 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.046 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.046 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.419978+0000", 2024-09-13T10:11:44.046 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00092341999999999999 2024-09-13T10:11:44.047 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.047 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.047 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.047 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.419998+0000", 2024-09-13T10:11:44.047 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0809e-05 2024-09-13T10:11:44.047 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.047 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.047 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.048 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.420011+0000", 2024-09-13T10:11:44.048 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2163e-05 2024-09-13T10:11:44.048 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.048 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.048 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.049 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.426538+0000", 2024-09-13T10:11:44.049 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0065269639999999997 2024-09-13T10:11:44.049 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.049 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.049 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.049 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.426917+0000", 2024-09-13T10:11:44.050 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037961499999999998 2024-09-13T10:11:44.050 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.050 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.050 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:44.050 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.427518+0000", 2024-09-13T10:11:44.050 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00060034900000000002 2024-09-13T10:11:44.050 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.051 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.051 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.051 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.432102+0000", 2024-09-13T10:11:44.051 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0045840079999999997 2024-09-13T10:11:44.051 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.051 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.051 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.051 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.432144+0000", 2024-09-13T10:11:44.052 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1922000000000003e-05 2024-09-13T10:11:44.052 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.052 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.052 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.052 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.432268+0000", 2024-09-13T10:11:44.052 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012485900000000001 2024-09-13T10:11:44.052 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.052 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.052 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.052 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.432417+0000", 2024-09-13T10:11:44.052 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014855399999999999 2024-09-13T10:11:44.052 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.053 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.053 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.053 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.432516+0000", 2024-09-13T10:11:44.053 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.9499000000000004e-05 2024-09-13T10:11:44.053 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.053 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.053 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.053 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.053 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.053 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.13.21:66 3.3 3:ec59e195:::89902e50bfc3ea21737900836f42c22669bb4be539f50b5d1ee5d1c721c9a0f8:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:44.053 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:29.420033+0000", 2024-09-13T10:11:44.053 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 14.467971594, 2024-09-13T10:11:44.053 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013349420000000001, 2024-09-13T10:11:44.054 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.054 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.054 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.054 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.054 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.054 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.054 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.420033+0000", 2024-09-13T10:11:44.055 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.055 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.055 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.055 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.055 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.420033+0000", 2024-09-13T10:11:44.055 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.055 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.055 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.055 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.055 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.420035+0000", 2024-09-13T10:11:44.055 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4160000000000001e-06 2024-09-13T10:11:44.055 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.055 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.056 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.056 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.420254+0000", 2024-09-13T10:11:44.056 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021894500000000001 2024-09-13T10:11:44.056 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.056 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.056 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.056 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.420316+0000", 2024-09-13T10:11:44.056 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.2757999999999998e-05 2024-09-13T10:11:44.056 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.056 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.056 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.057 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.420335+0000", 2024-09-13T10:11:44.057 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9072e-05 2024-09-13T10:11:44.057 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.057 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.057 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.057 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.430124+0000", 2024-09-13T10:11:44.057 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0097888520000000007 2024-09-13T10:11:44.057 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.057 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.057 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.057 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.430481+0000", 2024-09-13T10:11:44.057 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00035628899999999998 2024-09-13T10:11:44.057 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.058 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.058 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:11:44.058 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.430877+0000", 2024-09-13T10:11:44.058 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039646100000000002 2024-09-13T10:11:44.058 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.058 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.058 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.058 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.432557+0000", 2024-09-13T10:11:44.059 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00168023 2024-09-13T10:11:44.059 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.059 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.059 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.059 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.433129+0000", 2024-09-13T10:11:44.059 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00057135699999999998 2024-09-13T10:11:44.059 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.059 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.059 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.059 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.433245+0000", 2024-09-13T10:11:44.059 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011629900000000001 2024-09-13T10:11:44.059 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.059 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.060 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.060 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.433321+0000", 2024-09-13T10:11:44.060 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6054000000000001e-05 2024-09-13T10:11:44.060 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.060 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.060 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.060 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.433383+0000", 2024-09-13T10:11:44.060 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.1686999999999994e-05 2024-09-13T10:11:44.061 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.061 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.061 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.061 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.061 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.061 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:44.063 INFO:tasks.rados.rados.0.smithi195.stdout:1447: expect (ObjNum 229 snap 0 seq_num 229) 2024-09-13T10:11:44.064 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-13T10:11:44.065 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:44.058+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:44.069 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:44.071 INFO:teuthology.orchestra.run.smithi073.stderr:set pool 4 pgp_num to 26 2024-09-13T10:11:44.159 INFO:tasks.rados.rados.0.smithi195.stdout:1448: finishing set_chunk to oid 48 2024-09-13T10:11:44.159 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 88 (ObjNum 264 snap 0 seq_num 264) dirty exists 2024-09-13T10:11:44.159 INFO:tasks.rados.rados.0.smithi195.stdout:1449: expect (ObjNum 289 snap 0 seq_num 289) 2024-09-13T10:11:44.159 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:44.164 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:44.158+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:44.196 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:44.241 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:44.241 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:44.241 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:11:44.241 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:44.258 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-13T10:11:44.268 INFO:tasks.rados.rados.0.smithi195.stdout:1450: finishing write tid 1 to smithi19530714-10 2024-09-13T10:11:44.269 INFO:tasks.rados.rados.0.smithi195.stdout:1450: finishing write tid 2 to smithi19530714-10 2024-09-13T10:11:44.269 INFO:tasks.rados.rados.0.smithi195.stdout:1450: finishing write tid 3 to smithi19530714-10 2024-09-13T10:11:44.269 INFO:tasks.rados.rados.0.smithi195.stdout:1450: finishing write tid 4 to smithi19530714-10 2024-09-13T10:11:44.269 INFO:tasks.rados.rados.0.smithi195.stdout:1450: oid 10 updating version 0 to 26 2024-09-13T10:11:44.269 INFO:tasks.rados.rados.0.smithi195.stdout:1450: oid 10 version 26 is already newer than 25 2024-09-13T10:11:44.269 INFO:tasks.rados.rados.0.smithi195.stdout:1450: oid 10 version 26 is already newer than 24 2024-09-13T10:11:44.269 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 26 (ObjNum 296 snap 0 seq_num 296) dirty exists 2024-09-13T10:11:44.269 INFO:tasks.rados.rados.0.smithi195.stdout:1450: left oid 10 (ObjNum 296 snap 0 seq_num 296) 2024-09-13T10:11:44.270 INFO:tasks.rados.rados.0.smithi195.stdout:1419: expect (ObjNum 253 snap 0 seq_num 253) 2024-09-13T10:11:44.272 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1373852 2024-09-13T10:11:44.285 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:44.300 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:44.298+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:44.311 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1151249 2024-09-13T10:11:44.325 INFO:tasks.rados.rados.0.smithi195.stdout:1452: expect (ObjNum 265 snap 0 seq_num 265) 2024-09-13T10:11:44.365 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:44.359+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:44.377 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:44.423 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:44.423 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:44.423 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:11:44.423 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:11:44.423 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:44.437 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-13T10:11:44.438 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1358617 2024-09-13T10:11:44.466 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:44.463+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:44.476 INFO:tasks.rados.rados.0.smithi195.stdout:1453: finishing write tid 1 to smithi19530714-42 2024-09-13T10:11:44.476 INFO:tasks.rados.rados.0.smithi195.stdout:1451: expect (ObjNum 260 snap 0 seq_num 260) 2024-09-13T10:11:44.562 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:44.562 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:11:44.562 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:11:44.562 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:11:44.562 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.563 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:202 3.2 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.563 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.991310+0000", 2024-09-13T10:11:44.563 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.566744345000004, 2024-09-13T10:11:44.563 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.040945251000000002, 2024-09-13T10:11:44.563 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.563 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.563 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.563 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.563 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.563 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.563 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 202 2024-09-13T10:11:44.563 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.563 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.564 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.564 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.564 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991310+0000", 2024-09-13T10:11:44.564 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.564 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.564 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.564 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.564 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991310+0000", 2024-09-13T10:11:44.564 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.564 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.564 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.564 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.565 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991314+0000", 2024-09-13T10:11:44.565 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.7100000000000001e-06 2024-09-13T10:11:44.565 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.565 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.565 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.565 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991328+0000", 2024-09-13T10:11:44.565 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.413e-05 2024-09-13T10:11:44.565 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.565 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.565 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.565 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991361+0000", 2024-09-13T10:11:44.565 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2512999999999999e-05 2024-09-13T10:11:44.566 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.566 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.566 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.566 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991377+0000", 2024-09-13T10:11:44.566 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6478999999999999e-05 2024-09-13T10:11:44.566 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.566 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.566 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.566 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991546+0000", 2024-09-13T10:11:44.566 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016856600000000001 2024-09-13T10:11:44.566 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.566 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.567 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.567 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991850+0000", 2024-09-13T10:11:44.567 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00030478800000000001 2024-09-13T10:11:44.567 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.567 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.567 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.567 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.001416+0000", 2024-09-13T10:11:44.567 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0095659279999999996 2024-09-13T10:11:44.567 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.567 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.567 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.567 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031502+0000", 2024-09-13T10:11:44.568 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.030085526000000001 2024-09-13T10:11:44.568 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.568 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.568 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.568 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031585+0000", 2024-09-13T10:11:44.568 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.3175000000000002e-05 2024-09-13T10:11:44.568 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.568 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.568 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.568 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031775+0000", 2024-09-13T10:11:44.568 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000190161 2024-09-13T10:11:44.568 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.569 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.569 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.569 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031982+0000", 2024-09-13T10:11:44.569 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000207245 2024-09-13T10:11:44.569 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.569 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.569 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.569 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032255+0000", 2024-09-13T10:11:44.569 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00027303000000000001 2024-09-13T10:11:44.569 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.569 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.569 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.570 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.570 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.570 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:206 3.2 3:4cbdbe9e:::smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.570 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.991649+0000", 2024-09-13T10:11:44.570 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.566406016999998, 2024-09-13T10:11:44.570 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041234240999999998, 2024-09-13T10:11:44.570 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.570 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.570 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.570 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.570 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.570 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.570 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 206 2024-09-13T10:11:44.571 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.571 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.571 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.571 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.571 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991649+0000", 2024-09-13T10:11:44.571 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.571 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.571 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.571 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.571 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991649+0000", 2024-09-13T10:11:44.571 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.571 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.571 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.572 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.572 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991650+0000", 2024-09-13T10:11:44.572 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.483e-06 2024-09-13T10:11:44.572 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.572 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.572 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.572 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991657+0000", 2024-09-13T10:11:44.572 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.0899999999999999e-06 2024-09-13T10:11:44.572 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.572 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.572 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.572 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991672+0000", 2024-09-13T10:11:44.572 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4514e-05 2024-09-13T10:11:44.573 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.573 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.573 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.573 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991681+0000", 2024-09-13T10:11:44.573 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6360000000000006e-06 2024-09-13T10:11:44.573 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.573 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.573 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.573 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992159+0000", 2024-09-13T10:11:44.573 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00047820099999999998 2024-09-13T10:11:44.573 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.573 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.574 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.574 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992411+0000", 2024-09-13T10:11:44.574 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025199700000000001 2024-09-13T10:11:44.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.574 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.574 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.574 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.009775+0000", 2024-09-13T10:11:44.574 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017363841000000001 2024-09-13T10:11:44.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.574 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.574 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.574 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032446+0000", 2024-09-13T10:11:44.574 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022670490000000001 2024-09-13T10:11:44.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.575 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.575 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032482+0000", 2024-09-13T10:11:44.575 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5902000000000003e-05 2024-09-13T10:11:44.575 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.575 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.575 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032612+0000", 2024-09-13T10:11:44.575 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013022700000000001 2024-09-13T10:11:44.575 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.575 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.575 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032767+0000", 2024-09-13T10:11:44.576 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015477000000000001 2024-09-13T10:11:44.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.576 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.576 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.576 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032883+0000", 2024-09-13T10:11:44.576 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011608999999999999 2024-09-13T10:11:44.576 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.576 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.576 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.576 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.576 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:210 3.2 3:46ab5247:::smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.577 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.992337+0000", 2024-09-13T10:11:44.577 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.565717874000001, 2024-09-13T10:11:44.577 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.057985696000000003, 2024-09-13T10:11:44.577 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.577 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.577 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.577 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.577 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.577 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.577 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 210 2024-09-13T10:11:44.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.577 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.578 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.578 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.578 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992337+0000", 2024-09-13T10:11:44.578 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.578 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.578 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.578 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.578 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992337+0000", 2024-09-13T10:11:44.578 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.578 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.578 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.578 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.578 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992339+0000", 2024-09-13T10:11:44.579 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2400000000000002e-06 2024-09-13T10:11:44.579 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.579 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.579 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.579 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992350+0000", 2024-09-13T10:11:44.579 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1259e-05 2024-09-13T10:11:44.579 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.579 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.579 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.579 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992368+0000", 2024-09-13T10:11:44.579 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8133000000000001e-05 2024-09-13T10:11:44.579 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.580 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.580 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.580 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992382+0000", 2024-09-13T10:11:44.580 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3837e-05 2024-09-13T10:11:44.580 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.580 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.580 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.580 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992752+0000", 2024-09-13T10:11:44.580 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037008000000000001 2024-09-13T10:11:44.580 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.580 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.580 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.581 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992980+0000", 2024-09-13T10:11:44.581 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022799100000000001 2024-09-13T10:11:44.581 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.581 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.581 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027651+0000", 2024-09-13T10:11:44.581 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.034671027 2024-09-13T10:11:44.581 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.581 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.581 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.034868+0000", 2024-09-13T10:11:44.581 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0072169920000000002 2024-09-13T10:11:44.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.582 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.582 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038033+0000", 2024-09-13T10:11:44.582 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003164966 2024-09-13T10:11:44.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.582 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.582 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050187+0000", 2024-09-13T10:11:44.582 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012153815 2024-09-13T10:11:44.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.582 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.583 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050266+0000", 2024-09-13T10:11:44.583 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9359999999999999e-05 2024-09-13T10:11:44.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.583 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.583 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050322+0000", 2024-09-13T10:11:44.583 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5995999999999997e-05 2024-09-13T10:11:44.583 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.583 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.583 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.583 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:214 3.2 3:68ba95ba:::smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.584 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.992479+0000", 2024-09-13T10:11:44.584 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.565575801000001, 2024-09-13T10:11:44.584 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.045987867000000002, 2024-09-13T10:11:44.584 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.584 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.584 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.584 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.584 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.584 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.584 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 214 2024-09-13T10:11:44.584 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.584 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.585 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.585 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.585 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992479+0000", 2024-09-13T10:11:44.585 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.585 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.585 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.585 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992479+0000", 2024-09-13T10:11:44.585 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.585 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.585 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.585 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992481+0000", 2024-09-13T10:11:44.585 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2960000000000001e-06 2024-09-13T10:11:44.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.586 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.586 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992493+0000", 2024-09-13T10:11:44.586 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2077e-05 2024-09-13T10:11:44.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.586 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.586 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992519+0000", 2024-09-13T10:11:44.586 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6282e-05 2024-09-13T10:11:44.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.586 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.587 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992538+0000", 2024-09-13T10:11:44.587 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9083e-05 2024-09-13T10:11:44.587 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.587 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.587 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.587 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993393+0000", 2024-09-13T10:11:44.587 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00085445200000000001 2024-09-13T10:11:44.587 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.587 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.587 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.588 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993682+0000", 2024-09-13T10:11:44.588 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000288895 2024-09-13T10:11:44.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.588 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.588 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.588 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.019700+0000", 2024-09-13T10:11:44.588 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.02601794 2024-09-13T10:11:44.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.588 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.588 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.588 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032498+0000", 2024-09-13T10:11:44.588 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012798535999999999 2024-09-13T10:11:44.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.589 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.589 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033641+0000", 2024-09-13T10:11:44.589 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001142569 2024-09-13T10:11:44.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.589 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.589 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038259+0000", 2024-09-13T10:11:44.589 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0046180939999999997 2024-09-13T10:11:44.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.590 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.590 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038379+0000", 2024-09-13T10:11:44.590 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000119914 2024-09-13T10:11:44.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.590 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.590 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.590 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038467+0000", 2024-09-13T10:11:44.590 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7729e-05 2024-09-13T10:11:44.590 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.590 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.590 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.590 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.590 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:217 3.2 3:554c406d:::smithi19530714-18 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.591 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.027515+0000", 2024-09-13T10:11:44.591 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.530539967000003, 2024-09-13T10:11:44.591 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032787180999999999, 2024-09-13T10:11:44.591 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.591 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.591 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.591 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.591 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.591 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.592 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 217 2024-09-13T10:11:44.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.592 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.592 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.592 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027515+0000", 2024-09-13T10:11:44.592 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.592 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.592 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027515+0000", 2024-09-13T10:11:44.592 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.593 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.593 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.593 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027518+0000", 2024-09-13T10:11:44.593 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.4429999999999999e-06 2024-09-13T10:11:44.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.593 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.593 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.593 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027532+0000", 2024-09-13T10:11:44.593 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3517e-05 2024-09-13T10:11:44.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.593 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.593 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.594 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027563+0000", 2024-09-13T10:11:44.594 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1158999999999998e-05 2024-09-13T10:11:44.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.594 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.594 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.594 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027585+0000", 2024-09-13T10:11:44.594 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.228e-05 2024-09-13T10:11:44.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.594 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.594 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.594 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033014+0000", 2024-09-13T10:11:44.594 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0054288649999999997 2024-09-13T10:11:44.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.595 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.595 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.595 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033269+0000", 2024-09-13T10:11:44.595 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000255054 2024-09-13T10:11:44.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.595 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.595 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.595 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.044618+0000", 2024-09-13T10:11:44.595 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011349391 2024-09-13T10:11:44.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.595 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.595 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.595 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.054627+0000", 2024-09-13T10:11:44.596 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010008992 2024-09-13T10:11:44.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.596 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.596 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.058047+0000", 2024-09-13T10:11:44.596 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0034193000000000001 2024-09-13T10:11:44.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.596 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.596 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060139+0000", 2024-09-13T10:11:44.596 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020925240000000001 2024-09-13T10:11:44.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.596 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.597 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060222+0000", 2024-09-13T10:11:44.597 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.2917999999999995e-05 2024-09-13T10:11:44.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.597 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.597 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.597 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060302+0000", 2024-09-13T10:11:44.597 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9738000000000001e-05 2024-09-13T10:11:44.597 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.597 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.597 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.597 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.597 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:218 3.2 3:5915c927:::smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.598 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.032754+0000", 2024-09-13T10:11:44.598 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.525300698999999, 2024-09-13T10:11:44.598 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023630241, 2024-09-13T10:11:44.598 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.598 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.598 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.598 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.598 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.598 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.598 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 218 2024-09-13T10:11:44.598 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.598 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.599 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.599 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.599 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032754+0000", 2024-09-13T10:11:44.599 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.599 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.599 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.599 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032754+0000", 2024-09-13T10:11:44.599 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.599 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.599 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.599 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032757+0000", 2024-09-13T10:11:44.600 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6330000000000002e-06 2024-09-13T10:11:44.600 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.600 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.600 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.600 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032772+0000", 2024-09-13T10:11:44.600 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4636e-05 2024-09-13T10:11:44.600 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.600 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.600 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.600 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032796+0000", 2024-09-13T10:11:44.600 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3963000000000001e-05 2024-09-13T10:11:44.600 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.600 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.601 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.601 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032815+0000", 2024-09-13T10:11:44.601 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9159000000000001e-05 2024-09-13T10:11:44.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.601 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.601 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.601 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033742+0000", 2024-09-13T10:11:44.601 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00092672700000000002 2024-09-13T10:11:44.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.601 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.601 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.601 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.034124+0000", 2024-09-13T10:11:44.601 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00038220399999999997 2024-09-13T10:11:44.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.602 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.602 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.602 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.040963+0000", 2024-09-13T10:11:44.602 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0068392230000000002 2024-09-13T10:11:44.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.602 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.602 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.602 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050362+0000", 2024-09-13T10:11:44.602 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0093986060000000003 2024-09-13T10:11:44.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.602 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.603 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.603 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.054585+0000", 2024-09-13T10:11:44.603 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0042225429999999996 2024-09-13T10:11:44.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.603 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.603 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.603 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.056250+0000", 2024-09-13T10:11:44.603 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001665251 2024-09-13T10:11:44.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.603 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.603 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.603 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.056320+0000", 2024-09-13T10:11:44.603 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9913999999999995e-05 2024-09-13T10:11:44.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.604 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.604 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.056384+0000", 2024-09-13T10:11:44.604 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4381999999999994e-05 2024-09-13T10:11:44.604 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.604 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.604 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.604 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:220 3.2 3:753d5785:::smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.604 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.035716+0000", 2024-09-13T10:11:44.604 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.522338206000001, 2024-09-13T10:11:44.604 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.030171203000000001, 2024-09-13T10:11:44.605 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.605 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.605 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.605 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.605 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.605 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.605 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 220 2024-09-13T10:11:44.605 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.605 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.605 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.605 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.605 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035716+0000", 2024-09-13T10:11:44.605 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.606 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.606 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.606 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035716+0000", 2024-09-13T10:11:44.606 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.606 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.606 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.606 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035720+0000", 2024-09-13T10:11:44.606 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5310000000000002e-06 2024-09-13T10:11:44.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.606 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.606 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.607 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035735+0000", 2024-09-13T10:11:44.607 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5517000000000001e-05 2024-09-13T10:11:44.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.607 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.607 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.607 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035765+0000", 2024-09-13T10:11:44.607 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9771e-05 2024-09-13T10:11:44.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.607 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.607 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.607 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035786+0000", 2024-09-13T10:11:44.607 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0928e-05 2024-09-13T10:11:44.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.608 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.608 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035945+0000", 2024-09-13T10:11:44.608 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015871900000000001 2024-09-13T10:11:44.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.608 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.608 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.036337+0000", 2024-09-13T10:11:44.608 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039231299999999998 2024-09-13T10:11:44.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.609 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.609 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.051063+0000", 2024-09-13T10:11:44.609 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014725607 2024-09-13T10:11:44.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.609 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.609 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.609 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.055802+0000", 2024-09-13T10:11:44.609 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0047387929999999998 2024-09-13T10:11:44.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.609 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.609 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.609 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064353+0000", 2024-09-13T10:11:44.610 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0085516629999999993 2024-09-13T10:11:44.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.610 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.610 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.610 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065750+0000", 2024-09-13T10:11:44.610 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001396748 2024-09-13T10:11:44.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.610 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.610 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.610 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065811+0000", 2024-09-13T10:11:44.610 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.0770000000000003e-05 2024-09-13T10:11:44.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.610 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.611 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.611 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065888+0000", 2024-09-13T10:11:44.611 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6842999999999996e-05 2024-09-13T10:11:44.611 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.611 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.611 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.611 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.611 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.611 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:227 3.2 3:6ac7800c:::smithi19530714-28 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.611 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.060091+0000", 2024-09-13T10:11:44.611 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.497963141, 2024-09-13T10:11:44.611 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.034993362, 2024-09-13T10:11:44.611 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.612 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.612 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.612 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.612 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.612 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.612 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 227 2024-09-13T10:11:44.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.612 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.612 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.612 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.612 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060091+0000", 2024-09-13T10:11:44.612 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.613 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.613 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060091+0000", 2024-09-13T10:11:44.613 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.613 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.613 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060094+0000", 2024-09-13T10:11:44.613 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5330000000000001e-06 2024-09-13T10:11:44.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.613 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.613 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060103+0000", 2024-09-13T10:11:44.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7020000000000007e-06 2024-09-13T10:11:44.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.614 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.614 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.614 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060121+0000", 2024-09-13T10:11:44.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.791e-05 2024-09-13T10:11:44.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.614 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.614 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.614 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060135+0000", 2024-09-13T10:11:44.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4059999999999999e-05 2024-09-13T10:11:44.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.614 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.615 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.615 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060346+0000", 2024-09-13T10:11:44.615 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021175600000000001 2024-09-13T10:11:44.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.615 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.615 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.615 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060559+0000", 2024-09-13T10:11:44.615 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021239499999999999 2024-09-13T10:11:44.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.615 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.615 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.615 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.073003+0000", 2024-09-13T10:11:44.615 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012444255 2024-09-13T10:11:44.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.616 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.616 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.081388+0000", 2024-09-13T10:11:44.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0083852119999999995 2024-09-13T10:11:44.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.616 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.616 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094289+0000", 2024-09-13T10:11:44.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012900271 2024-09-13T10:11:44.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.616 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094913+0000", 2024-09-13T10:11:44.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00062429200000000003 2024-09-13T10:11:44.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.617 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094972+0000", 2024-09-13T10:11:44.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9457999999999999e-05 2024-09-13T10:11:44.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.617 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095085+0000", 2024-09-13T10:11:44.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000112518 2024-09-13T10:11:44.617 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.617 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.618 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.618 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.618 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.618 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:230 3.2 3:5d108080:::smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.618 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.065002+0000", 2024-09-13T10:11:44.618 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.493052697000003, 2024-09-13T10:11:44.618 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.049477431000000002, 2024-09-13T10:11:44.618 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.618 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.618 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.618 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.618 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.618 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.619 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 230 2024-09-13T10:11:44.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.619 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.619 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.619 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.619 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065002+0000", 2024-09-13T10:11:44.619 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.619 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.619 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.619 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065002+0000", 2024-09-13T10:11:44.619 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.620 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.620 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.620 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065004+0000", 2024-09-13T10:11:44.620 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4269999999999998e-06 2024-09-13T10:11:44.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.620 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.620 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.620 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065014+0000", 2024-09-13T10:11:44.620 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.9799999999999993e-06 2024-09-13T10:11:44.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.620 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.620 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.621 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065038+0000", 2024-09-13T10:11:44.621 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4229000000000001e-05 2024-09-13T10:11:44.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.621 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.621 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.621 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065054+0000", 2024-09-13T10:11:44.621 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5764999999999999e-05 2024-09-13T10:11:44.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.621 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.621 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.621 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065171+0000", 2024-09-13T10:11:44.621 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117032 2024-09-13T10:11:44.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.622 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.622 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.622 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065513+0000", 2024-09-13T10:11:44.622 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00034131 2024-09-13T10:11:44.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.622 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.622 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.622 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.087042+0000", 2024-09-13T10:11:44.622 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021529534999999999 2024-09-13T10:11:44.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.622 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.622 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.622 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.097903+0000", 2024-09-13T10:11:44.623 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010861282999999999 2024-09-13T10:11:44.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.623 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.623 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.623 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.099922+0000", 2024-09-13T10:11:44.623 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020186539999999999 2024-09-13T10:11:44.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.623 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.623 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.623 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114391+0000", 2024-09-13T10:11:44.623 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014468880999999999 2024-09-13T10:11:44.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.623 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.624 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.624 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114434+0000", 2024-09-13T10:11:44.624 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.3511999999999999e-05 2024-09-13T10:11:44.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.624 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.624 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114479+0000", 2024-09-13T10:11:44.624 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4823000000000001e-05 2024-09-13T10:11:44.624 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.624 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.624 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.624 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:232 3.2 3:6538713c:::smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.625 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.065891+0000", 2024-09-13T10:11:44.625 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.492163826000002, 2024-09-13T10:11:44.625 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048804489999999999, 2024-09-13T10:11:44.625 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.625 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.625 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.625 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.625 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.625 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.625 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 232 2024-09-13T10:11:44.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.625 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.625 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.626 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.626 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065891+0000", 2024-09-13T10:11:44.626 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.626 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.626 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065891+0000", 2024-09-13T10:11:44.626 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.626 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.626 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065892+0000", 2024-09-13T10:11:44.626 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6249999999999999e-06 2024-09-13T10:11:44.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.627 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.627 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.627 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065902+0000", 2024-09-13T10:11:44.627 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.4150000000000005e-06 2024-09-13T10:11:44.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.627 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.627 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.627 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065920+0000", 2024-09-13T10:11:44.627 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7907e-05 2024-09-13T10:11:44.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.627 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.627 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.628 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065932+0000", 2024-09-13T10:11:44.628 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2469999999999999e-05 2024-09-13T10:11:44.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.628 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.628 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.628 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.066072+0000", 2024-09-13T10:11:44.628 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013935999999999999 2024-09-13T10:11:44.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.628 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.628 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.628 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.066435+0000", 2024-09-13T10:11:44.628 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036323300000000002 2024-09-13T10:11:44.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.629 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.629 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091133+0000", 2024-09-13T10:11:44.629 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024698494000000001 2024-09-13T10:11:44.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.629 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.629 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.101407+0000", 2024-09-13T10:11:44.629 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010273835 2024-09-13T10:11:44.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.629 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114342+0000", 2024-09-13T10:11:44.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012934826 2024-09-13T10:11:44.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.630 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114548+0000", 2024-09-13T10:11:44.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000206243 2024-09-13T10:11:44.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.630 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114630+0000", 2024-09-13T10:11:44.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.2127999999999997e-05 2024-09-13T10:11:44.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.631 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.631 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114695+0000", 2024-09-13T10:11:44.631 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4954000000000002e-05 2024-09-13T10:11:44.631 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.631 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.631 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.631 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:236 3.2 3:73c2893b:::smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.631 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.076316+0000", 2024-09-13T10:11:44.631 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.481738704999998, 2024-09-13T10:11:44.631 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041217193999999999, 2024-09-13T10:11:44.631 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.632 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.632 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.632 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.632 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.632 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.632 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 236 2024-09-13T10:11:44.632 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.632 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.632 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.632 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.632 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076316+0000", 2024-09-13T10:11:44.632 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.632 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.633 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.633 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076316+0000", 2024-09-13T10:11:44.633 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.633 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.633 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076319+0000", 2024-09-13T10:11:44.633 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1750000000000001e-06 2024-09-13T10:11:44.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.633 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.634 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076333+0000", 2024-09-13T10:11:44.634 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3862e-05 2024-09-13T10:11:44.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.634 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.634 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.634 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076359+0000", 2024-09-13T10:11:44.634 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5641999999999999e-05 2024-09-13T10:11:44.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.634 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.634 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.634 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076378+0000", 2024-09-13T10:11:44.634 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9406999999999999e-05 2024-09-13T10:11:44.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.635 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.635 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.078831+0000", 2024-09-13T10:11:44.635 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0024532590000000002 2024-09-13T10:11:44.635 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.635 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.635 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079137+0000", 2024-09-13T10:11:44.635 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00030614500000000002 2024-09-13T10:11:44.635 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.635 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.635 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.102393+0000", 2024-09-13T10:11:44.635 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023255175999999999 2024-09-13T10:11:44.636 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.636 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.636 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.636 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114728+0000", 2024-09-13T10:11:44.636 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012335376 2024-09-13T10:11:44.636 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.636 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.636 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.636 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114735+0000", 2024-09-13T10:11:44.636 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2969999999999998e-06 2024-09-13T10:11:44.636 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.636 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.636 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.637 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117465+0000", 2024-09-13T10:11:44.637 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0027298019999999999 2024-09-13T10:11:44.637 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.637 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.637 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.637 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117501+0000", 2024-09-13T10:11:44.637 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5837999999999998e-05 2024-09-13T10:11:44.637 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.637 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.637 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.637 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117533+0000", 2024-09-13T10:11:44.637 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2215000000000002e-05 2024-09-13T10:11:44.637 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.638 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.638 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.638 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.638 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.638 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:240 3.2 3:7eee2f4a:::smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.638 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.091046+0000", 2024-09-13T10:11:44.638 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.467008864, 2024-09-13T10:11:44.638 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048376970999999998, 2024-09-13T10:11:44.638 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.638 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.638 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.638 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.639 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.639 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.639 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 240 2024-09-13T10:11:44.639 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.639 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.639 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.639 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.639 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091046+0000", 2024-09-13T10:11:44.639 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.639 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.639 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.639 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.639 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091046+0000", 2024-09-13T10:11:44.640 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.640 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.640 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.640 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.640 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091048+0000", 2024-09-13T10:11:44.640 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6299999999999998e-06 2024-09-13T10:11:44.640 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.640 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.640 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.640 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091060+0000", 2024-09-13T10:11:44.640 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1229999999999999e-05 2024-09-13T10:11:44.640 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.640 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.641 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.641 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091075+0000", 2024-09-13T10:11:44.641 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5313000000000001e-05 2024-09-13T10:11:44.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.641 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.641 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.641 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091085+0000", 2024-09-13T10:11:44.641 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0284e-05 2024-09-13T10:11:44.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.641 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.641 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.641 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094402+0000", 2024-09-13T10:11:44.642 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00331708 2024-09-13T10:11:44.642 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.642 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.642 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.642 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094649+0000", 2024-09-13T10:11:44.642 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024695300000000001 2024-09-13T10:11:44.642 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.642 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.642 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.642 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.107510+0000", 2024-09-13T10:11:44.642 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012860778999999999 2024-09-13T10:11:44.642 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.642 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.642 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.643 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114739+0000", 2024-09-13T10:11:44.643 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0072289700000000004 2024-09-13T10:11:44.643 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.643 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.643 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.643 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118693+0000", 2024-09-13T10:11:44.643 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0039540560000000001 2024-09-13T10:11:44.643 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.643 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.643 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.643 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139282+0000", 2024-09-13T10:11:44.643 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020589423999999999 2024-09-13T10:11:44.643 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.644 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.644 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.644 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139357+0000", 2024-09-13T10:11:44.644 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.4276999999999994e-05 2024-09-13T10:11:44.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.644 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.644 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.644 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139423+0000", 2024-09-13T10:11:44.644 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.5975000000000004e-05 2024-09-13T10:11:44.644 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.644 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.644 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.645 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.645 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:242 3.2 3:501d22a1:::smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.645 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.095846+0000", 2024-09-13T10:11:44.645 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.462208660000002, 2024-09-13T10:11:44.645 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.043813831999999997, 2024-09-13T10:11:44.645 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.645 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.645 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.645 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.645 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.645 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.645 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 242 2024-09-13T10:11:44.645 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.646 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.646 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.646 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.646 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095846+0000", 2024-09-13T10:11:44.646 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.646 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.646 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.646 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.646 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095846+0000", 2024-09-13T10:11:44.646 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.646 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.646 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.646 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.647 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095848+0000", 2024-09-13T10:11:44.647 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3760000000000002e-06 2024-09-13T10:11:44.647 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.647 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.647 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.647 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095859+0000", 2024-09-13T10:11:44.647 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0429e-05 2024-09-13T10:11:44.647 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.647 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.647 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.647 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095876+0000", 2024-09-13T10:11:44.647 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7504e-05 2024-09-13T10:11:44.648 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.648 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.648 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.648 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095886+0000", 2024-09-13T10:11:44.648 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0182e-05 2024-09-13T10:11:44.648 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.648 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.648 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.648 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.096010+0000", 2024-09-13T10:11:44.648 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012336700000000001 2024-09-13T10:11:44.648 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.648 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.648 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.649 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.096252+0000", 2024-09-13T10:11:44.649 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024189200000000001 2024-09-13T10:11:44.649 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.649 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.649 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.649 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111492+0000", 2024-09-13T10:11:44.649 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015239920000000001 2024-09-13T10:11:44.649 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.649 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.649 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.649 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115691+0000", 2024-09-13T10:11:44.649 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0041992410000000003 2024-09-13T10:11:44.649 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.650 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.650 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.650 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139492+0000", 2024-09-13T10:11:44.650 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023801495999999998 2024-09-13T10:11:44.650 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.650 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.650 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.650 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139560+0000", 2024-09-13T10:11:44.650 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.8163000000000005e-05 2024-09-13T10:11:44.650 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.650 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.650 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.650 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139619+0000", 2024-09-13T10:11:44.651 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8678999999999998e-05 2024-09-13T10:11:44.651 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.651 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.651 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.651 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139660+0000", 2024-09-13T10:11:44.651 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0583000000000001e-05 2024-09-13T10:11:44.651 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.651 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.651 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.651 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.651 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.651 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:244 3.2 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.651 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.112158+0000", 2024-09-13T10:11:44.652 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.445896197000003, 2024-09-13T10:11:44.652 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.030058202999999999, 2024-09-13T10:11:44.652 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.652 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.652 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.652 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.652 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.652 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.652 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 244 2024-09-13T10:11:44.652 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.652 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.652 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.652 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.653 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112158+0000", 2024-09-13T10:11:44.653 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.653 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.653 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.653 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.653 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112158+0000", 2024-09-13T10:11:44.653 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.653 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.653 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.653 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.653 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112161+0000", 2024-09-13T10:11:44.654 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.384e-06 2024-09-13T10:11:44.654 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.654 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.654 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.654 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112168+0000", 2024-09-13T10:11:44.654 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.7780000000000002e-06 2024-09-13T10:11:44.654 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.654 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.654 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.654 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112183+0000", 2024-09-13T10:11:44.654 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5917999999999999e-05 2024-09-13T10:11:44.654 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.654 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.654 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.655 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112195+0000", 2024-09-13T10:11:44.655 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1578e-05 2024-09-13T10:11:44.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.655 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.655 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.655 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114766+0000", 2024-09-13T10:11:44.655 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025710659999999999 2024-09-13T10:11:44.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.655 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.655 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.655 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114995+0000", 2024-09-13T10:11:44.655 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022844 2024-09-13T10:11:44.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.656 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.656 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.656 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.125677+0000", 2024-09-13T10:11:44.656 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010682677999999999 2024-09-13T10:11:44.656 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.656 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.656 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.656 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.140355+0000", 2024-09-13T10:11:44.656 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014677773999999999 2024-09-13T10:11:44.656 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.656 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.656 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.657 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.141368+0000", 2024-09-13T10:11:44.657 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0010134040000000001 2024-09-13T10:11:44.657 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.657 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.657 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.657 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142112+0000", 2024-09-13T10:11:44.657 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00074347599999999997 2024-09-13T10:11:44.657 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.657 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.657 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.657 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142172+0000", 2024-09-13T10:11:44.657 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9749e-05 2024-09-13T10:11:44.657 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.658 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.658 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.658 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142217+0000", 2024-09-13T10:11:44.658 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4957999999999999e-05 2024-09-13T10:11:44.658 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.658 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.658 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.658 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.658 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.658 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:245 3.2 3:7837e849:::smithi19530714-46 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.658 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.112398+0000", 2024-09-13T10:11:44.658 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.445656139999997, 2024-09-13T10:11:44.658 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.039535731999999997, 2024-09-13T10:11:44.659 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.659 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.659 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.659 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.659 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.659 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.659 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 245 2024-09-13T10:11:44.659 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.659 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.659 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.659 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.659 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112398+0000", 2024-09-13T10:11:44.660 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.660 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.660 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.660 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.660 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112398+0000", 2024-09-13T10:11:44.660 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.660 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.660 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.660 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.660 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112399+0000", 2024-09-13T10:11:44.660 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0470000000000001e-06 2024-09-13T10:11:44.660 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.660 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112405+0000", 2024-09-13T10:11:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.6899999999999997e-06 2024-09-13T10:11:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112417+0000", 2024-09-13T10:11:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2334000000000001e-05 2024-09-13T10:11:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.661 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112427+0000", 2024-09-13T10:11:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.0929999999999992e-06 2024-09-13T10:11:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115182+0000", 2024-09-13T10:11:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0027552739999999998 2024-09-13T10:11:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115305+0000", 2024-09-13T10:11:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012364200000000001 2024-09-13T10:11:44.662 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.663 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.663 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.663 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.136435+0000", 2024-09-13T10:11:44.663 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021129479999999999 2024-09-13T10:11:44.663 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.663 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.663 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.663 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143376+0000", 2024-09-13T10:11:44.663 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0069408600000000001 2024-09-13T10:11:44.663 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.663 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.663 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.663 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.150416+0000", 2024-09-13T10:11:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0070403669999999996 2024-09-13T10:11:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.151872+0000", 2024-09-13T10:11:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001456286 2024-09-13T10:11:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.151906+0000", 2024-09-13T10:11:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3167999999999998e-05 2024-09-13T10:11:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.664 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.151934+0000", 2024-09-13T10:11:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8490999999999999e-05 2024-09-13T10:11:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:250 3.2 3:78d40eb9:::smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.115099+0000", 2024-09-13T10:11:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.442955896999997, 2024-09-13T10:11:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.028750412999999999, 2024-09-13T10:11:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.665 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.666 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.666 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.666 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.666 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.666 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 250 2024-09-13T10:11:44.666 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.666 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.666 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.666 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.666 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115099+0000", 2024-09-13T10:11:44.666 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.666 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.666 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.666 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115099+0000", 2024-09-13T10:11:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115101+0000", 2024-09-13T10:11:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1789999999999998e-06 2024-09-13T10:11:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115132+0000", 2024-09-13T10:11:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1163e-05 2024-09-13T10:11:44.667 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115150+0000", 2024-09-13T10:11:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7864999999999999e-05 2024-09-13T10:11:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115159+0000", 2024-09-13T10:11:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1719999999999996e-06 2024-09-13T10:11:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.668 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115440+0000", 2024-09-13T10:11:44.669 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028068199999999998 2024-09-13T10:11:44.669 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.669 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.669 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.669 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115579+0000", 2024-09-13T10:11:44.669 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013910800000000001 2024-09-13T10:11:44.669 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.669 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.669 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.669 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.132849+0000", 2024-09-13T10:11:44.669 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017270127999999999 2024-09-13T10:11:44.669 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.669 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.670 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.670 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142332+0000", 2024-09-13T10:11:44.670 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0094831859999999994 2024-09-13T10:11:44.670 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.670 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.670 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.670 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143034+0000", 2024-09-13T10:11:44.670 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00070226499999999999 2024-09-13T10:11:44.670 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.670 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.670 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.670 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143793+0000", 2024-09-13T10:11:44.671 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00075876700000000004 2024-09-13T10:11:44.671 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.671 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.671 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.671 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143825+0000", 2024-09-13T10:11:44.671 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2017999999999997e-05 2024-09-13T10:11:44.671 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.671 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.671 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.671 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143849+0000", 2024-09-13T10:11:44.671 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3879999999999998e-05 2024-09-13T10:11:44.671 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.671 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.672 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.672 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.672 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.672 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.7.22:1 3.2 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.672 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.130657+0000", 2024-09-13T10:11:44.672 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.427397501999998, 2024-09-13T10:11:44.672 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021425745, 2024-09-13T10:11:44.672 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.672 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.672 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.672 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.673 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.673 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.673 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130657+0000", 2024-09-13T10:11:44.673 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.673 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.673 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.673 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.673 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130657+0000", 2024-09-13T10:11:44.673 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.673 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.673 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.673 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.673 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130660+0000", 2024-09-13T10:11:44.674 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.095e-06 2024-09-13T10:11:44.674 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.674 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.674 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.674 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130671+0000", 2024-09-13T10:11:44.674 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0455000000000001e-05 2024-09-13T10:11:44.674 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.674 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.674 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.674 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130684+0000", 2024-09-13T10:11:44.674 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2933e-05 2024-09-13T10:11:44.674 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.674 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.675 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.675 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.130695+0000", 2024-09-13T10:11:44.675 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1138000000000001e-05 2024-09-13T10:11:44.675 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.675 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.675 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.675 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139699+0000", 2024-09-13T10:11:44.675 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0090041779999999998 2024-09-13T10:11:44.675 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.675 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.675 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.675 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139816+0000", 2024-09-13T10:11:44.675 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117484 2024-09-13T10:11:44.676 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.676 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.676 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.676 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.140032+0000", 2024-09-13T10:11:44.676 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021595699999999999 2024-09-13T10:11:44.676 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.676 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.676 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.676 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143538+0000", 2024-09-13T10:11:44.676 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0035057180000000001 2024-09-13T10:11:44.676 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.676 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.676 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.677 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.150610+0000", 2024-09-13T10:11:44.677 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0070718969999999997 2024-09-13T10:11:44.677 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.677 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.677 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.677 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152002+0000", 2024-09-13T10:11:44.677 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001391834 2024-09-13T10:11:44.677 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.677 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.677 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.677 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152049+0000", 2024-09-13T10:11:44.677 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.7097e-05 2024-09-13T10:11:44.677 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.678 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.678 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.678 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152083+0000", 2024-09-13T10:11:44.678 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3958999999999998e-05 2024-09-13T10:11:44.678 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.678 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.678 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.678 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.678 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.678 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.7.22:3 3.2 3:5915c927:::smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.678 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.216433+0000", 2024-09-13T10:11:44.678 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.341621519, 2024-09-13T10:11:44.678 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.037069469000000001, 2024-09-13T10:11:44.679 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.679 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.679 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.679 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.679 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.679 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.679 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216433+0000", 2024-09-13T10:11:44.679 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.679 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.679 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.679 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.679 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216433+0000", 2024-09-13T10:11:44.679 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.680 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.680 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.680 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.680 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216436+0000", 2024-09-13T10:11:44.680 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7800000000000001e-06 2024-09-13T10:11:44.680 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.680 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.680 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.680 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216473+0000", 2024-09-13T10:11:44.680 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6931999999999998e-05 2024-09-13T10:11:44.680 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.680 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.680 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.681 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216512+0000", 2024-09-13T10:11:44.681 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.8831999999999997e-05 2024-09-13T10:11:44.681 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.681 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.681 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.681 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216539+0000", 2024-09-13T10:11:44.681 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7705999999999999e-05 2024-09-13T10:11:44.681 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.681 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.681 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.681 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216732+0000", 2024-09-13T10:11:44.681 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019224500000000001 2024-09-13T10:11:44.682 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.682 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.682 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.682 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217000+0000", 2024-09-13T10:11:44.682 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026813100000000001 2024-09-13T10:11:44.682 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.682 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.682 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.682 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217638+0000", 2024-09-13T10:11:44.682 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00063874400000000001 2024-09-13T10:11:44.682 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.682 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.682 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.683 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.219940+0000", 2024-09-13T10:11:44.683 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023015890000000002 2024-09-13T10:11:44.683 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.683 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.683 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.683 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.220304+0000", 2024-09-13T10:11:44.683 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036399099999999998 2024-09-13T10:11:44.683 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.683 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.683 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.683 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253232+0000", 2024-09-13T10:11:44.683 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032927640000000001 2024-09-13T10:11:44.683 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.683 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.683 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.684 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253393+0000", 2024-09-13T10:11:44.684 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000161142 2024-09-13T10:11:44.684 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.684 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.684 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.684 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253503+0000", 2024-09-13T10:11:44.684 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000109737 2024-09-13T10:11:44.684 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.684 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.684 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.684 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.684 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.684 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.2.23:2 3.2 3:46ab5247:::smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:44.685 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.217995+0000", 2024-09-13T10:11:44.685 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 53.340060047999998, 2024-09-13T10:11:44.685 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036081690999999999, 2024-09-13T10:11:44.685 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.685 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.685 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:44.685 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.685 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.685 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.685 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217995+0000", 2024-09-13T10:11:44.685 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.685 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.685 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.686 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.686 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217995+0000", 2024-09-13T10:11:44.686 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.686 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.686 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.686 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.686 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217997+0000", 2024-09-13T10:11:44.686 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3429999999999998e-06 2024-09-13T10:11:44.686 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.686 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.686 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.686 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218007+0000", 2024-09-13T10:11:44.686 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.91e-06 2024-09-13T10:11:44.686 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.687 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.687 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.687 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218024+0000", 2024-09-13T10:11:44.687 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7549000000000001e-05 2024-09-13T10:11:44.687 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.687 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.687 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.687 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218038+0000", 2024-09-13T10:11:44.687 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3509999999999999e-05 2024-09-13T10:11:44.687 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.687 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.687 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.687 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218384+0000", 2024-09-13T10:11:44.688 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00034618499999999999 2024-09-13T10:11:44.688 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.688 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.688 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.688 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218605+0000", 2024-09-13T10:11:44.688 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022059400000000001 2024-09-13T10:11:44.688 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.688 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.688 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:11:44.688 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.219124+0000", 2024-09-13T10:11:44.688 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00051934699999999997 2024-09-13T10:11:44.688 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.689 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.689 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.689 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.222285+0000", 2024-09-13T10:11:44.689 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0031605790000000002 2024-09-13T10:11:44.689 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.689 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.689 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:44.689 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.224618+0000", 2024-09-13T10:11:44.689 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023338209999999998 2024-09-13T10:11:44.689 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.689 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.689 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:44.689 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253766+0000", 2024-09-13T10:11:44.690 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029147890999999999 2024-09-13T10:11:44.690 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.690 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.690 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:44.690 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253920+0000", 2024-09-13T10:11:44.690 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015327299999999999 2024-09-13T10:11:44.690 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.690 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.690 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.690 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254076+0000", 2024-09-13T10:11:44.690 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000156689 2024-09-13T10:11:44.690 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.690 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.691 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.691 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.691 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.691 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1854 4.b 4:d8e9927b:::smithi19530714-44:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:44.691 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:10.749110+0000", 2024-09-13T10:11:44.691 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 33.808944957999998, 2024-09-13T10:11:44.691 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.049047407000000001, 2024-09-13T10:11:44.691 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:44.691 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:44.691 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:44.691 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:44.691 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:44.691 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:44.692 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1854 2024-09-13T10:11:44.692 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.692 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:44.692 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.692 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:44.692 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749110+0000", 2024-09-13T10:11:44.692 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.692 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.692 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.692 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:44.692 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749110+0000", 2024-09-13T10:11:44.693 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:44.693 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.693 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.693 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:44.693 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749113+0000", 2024-09-13T10:11:44.693 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0970000000000002e-06 2024-09-13T10:11:44.693 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.693 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.693 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:44.693 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749125+0000", 2024-09-13T10:11:44.693 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2048999999999999e-05 2024-09-13T10:11:44.693 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.693 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.694 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:44.694 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749164+0000", 2024-09-13T10:11:44.694 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9579999999999997e-05 2024-09-13T10:11:44.694 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.694 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.694 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:44.694 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749188+0000", 2024-09-13T10:11:44.694 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4026e-05 2024-09-13T10:11:44.694 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.694 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.694 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:44.694 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749320+0000", 2024-09-13T10:11:44.694 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013178500000000001 2024-09-13T10:11:44.695 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.695 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.695 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:44.695 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749506+0000", 2024-09-13T10:11:44.695 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00018623700000000001 2024-09-13T10:11:44.695 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:44.695 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:44.695 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:44.695 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.798157+0000", 2024-09-13T10:11:44.695 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048650632999999999 2024-09-13T10:11:44.695 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.695 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.695 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.696 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:44.696 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:44.696 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:44.698 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T10:11:44.701 INFO:tasks.rados.rados.0.smithi195.stdout:1453: finishing write tid 2 to smithi19530714-42 2024-09-13T10:11:44.701 INFO:tasks.rados.rados.0.smithi195.stdout:1453: finishing write tid 3 to smithi19530714-42 2024-09-13T10:11:44.701 INFO:tasks.rados.rados.0.smithi195.stdout:1453: finishing write tid 4 to smithi19530714-42 2024-09-13T10:11:44.701 INFO:tasks.rados.rados.0.smithi195.stdout:1453: oid 42 updating version 0 to 18 2024-09-13T10:11:44.701 INFO:tasks.rados.rados.0.smithi195.stdout:1453: oid 42 version 18 is already newer than 17 2024-09-13T10:11:44.701 INFO:tasks.rados.rados.0.smithi195.stdout:1453: oid 42 version 18 is already newer than 16 2024-09-13T10:11:44.701 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 18 (ObjNum 297 snap 0 seq_num 297) dirty exists 2024-09-13T10:11:44.702 INFO:tasks.rados.rados.0.smithi195.stdout:1453: left oid 42 (ObjNum 297 snap 0 seq_num 297) 2024-09-13T10:11:44.702 INFO:tasks.rados.rados.0.smithi195.stdout:1419: done (15 left) 2024-09-13T10:11:44.702 INFO:tasks.rados.rados.0.smithi195.stdout:1435: done (14 left) 2024-09-13T10:11:44.702 INFO:tasks.rados.rados.0.smithi195.stdout:1439: done (13 left) 2024-09-13T10:11:44.702 INFO:tasks.rados.rados.0.smithi195.stdout:1441: done (12 left) 2024-09-13T10:11:44.702 INFO:tasks.rados.rados.0.smithi195.stdout:1442: done (11 left) 2024-09-13T10:11:44.702 INFO:tasks.rados.rados.0.smithi195.stdout:1443: done (10 left) 2024-09-13T10:11:44.702 INFO:tasks.rados.rados.0.smithi195.stdout:1444: done (9 left) 2024-09-13T10:11:44.702 INFO:tasks.rados.rados.0.smithi195.stdout:1445: done (8 left) 2024-09-13T10:11:44.702 INFO:tasks.rados.rados.0.smithi195.stdout:1446: done (7 left) 2024-09-13T10:11:44.702 INFO:tasks.rados.rados.0.smithi195.stdout:1447: done (6 left) 2024-09-13T10:11:44.702 INFO:tasks.rados.rados.0.smithi195.stdout:1448: done (5 left) 2024-09-13T10:11:44.702 INFO:tasks.rados.rados.0.smithi195.stdout:1449: done (4 left) 2024-09-13T10:11:44.703 INFO:tasks.rados.rados.0.smithi195.stdout:1450: done (3 left) 2024-09-13T10:11:44.703 INFO:tasks.rados.rados.0.smithi195.stdout:1451: done (2 left) 2024-09-13T10:11:44.703 INFO:tasks.rados.rados.0.smithi195.stdout:1452: done (1 left) 2024-09-13T10:11:44.703 INFO:tasks.rados.rados.0.smithi195.stdout:1453: done (0 left) 2024-09-13T10:11:44.703 INFO:tasks.rados.rados.0.smithi195.stdout:1454: read oid 50 snap -1 2024-09-13T10:11:44.703 INFO:tasks.rados.rados.0.smithi195.stdout:1454: expect (ObjNum 274 snap 0 seq_num 274) 2024-09-13T10:11:44.703 INFO:tasks.rados.rados.0.smithi195.stdout:1455: tier_evict oid 32 2024-09-13T10:11:44.703 INFO:tasks.rados.rados.0.smithi195.stdout:1455: tier_evict oid 32 snap -1 2024-09-13T10:11:44.703 INFO:tasks.rados.rados.0.smithi195.stdout:1456: read oid 37 snap -1 2024-09-13T10:11:44.703 INFO:tasks.rados.rados.0.smithi195.stdout:1456: expect deleted 2024-09-13T10:11:44.703 INFO:tasks.rados.rados.0.smithi195.stdout:1457: read oid 40 snap -1 2024-09-13T10:11:44.703 INFO:tasks.rados.rados.0.smithi195.stdout:1457: expect deleted 2024-09-13T10:11:44.703 INFO:tasks.rados.rados.0.smithi195.stdout:1458: read oid 8 snap -1 2024-09-13T10:11:44.703 INFO:tasks.rados.rados.0.smithi195.stdout:1458: expect deleted 2024-09-13T10:11:44.704 INFO:tasks.rados.rados.0.smithi195.stdout:1459: tier_promote oid 22 2024-09-13T10:11:44.704 INFO:tasks.rados.rados.0.smithi195.stdout:1460: set_chunk oid 44 target oid 2024-09-13T10:11:44.704 INFO:tasks.rados.rados.0.smithi195.stdout:1460: set_chunk oid 44 offset: 1531392 length: 379392 target oid 488b54b2444da5a91f8f3ac62a4b07161cbc4c7f89775d26d8f8df7bf9ecd617 offset: 0 snap -1 2024-09-13T10:11:44.704 INFO:tasks.rados.rados.0.smithi195.stdout:1461: read oid 3 snap -1 2024-09-13T10:11:44.704 INFO:tasks.rados.rados.0.smithi195.stdout:1461: expect (ObjNum 237 snap 0 seq_num 237) 2024-09-13T10:11:44.704 INFO:tasks.rados.rados.0.smithi195.stdout:1462: read oid 20 snap -1 2024-09-13T10:11:44.704 INFO:tasks.rados.rados.0.smithi195.stdout:1462: expect deleted 2024-09-13T10:11:44.704 INFO:tasks.rados.rados.0.smithi195.stdout:1463: read oid 6 snap -1 2024-09-13T10:11:44.704 INFO:tasks.rados.rados.0.smithi195.stdout:1463: expect (ObjNum 229 snap 0 seq_num 229) 2024-09-13T10:11:44.704 INFO:tasks.rados.rados.0.smithi195.stdout:1464: read oid 12 snap -1 2024-09-13T10:11:44.704 INFO:tasks.rados.rados.0.smithi195.stdout:1464: expect (ObjNum 287 snap 0 seq_num 287) 2024-09-13T10:11:44.704 INFO:tasks.rados.rados.0.smithi195.stdout:1465: read oid 2 snap -1 2024-09-13T10:11:44.705 INFO:tasks.rados.rados.0.smithi195.stdout:1465: expect (ObjNum 280 snap 0 seq_num 280) 2024-09-13T10:11:44.705 INFO:tasks.rados.rados.0.smithi195.stdout:1466: set_chunk oid 13 target oid 2024-09-13T10:11:44.705 INFO:tasks.rados.rados.0.smithi195.stdout:1466: set_chunk oid 13 offset: 1028096 length: 1376768 target oid c22debe59152be99990d8cdb36453b06023f300dfa39698338f5e7bf67373dd8 offset: 0 snap -1 2024-09-13T10:11:44.705 INFO:tasks.rados.rados.0.smithi195.stdout:1467: read oid 43 snap -1 2024-09-13T10:11:44.705 INFO:tasks.rados.rados.0.smithi195.stdout:1467: expect (ObjNum 263 snap 0 seq_num 263) 2024-09-13T10:11:44.705 INFO:tasks.rados.rados.0.smithi195.stdout:1468: write (excl) oid 38 current snap is 0 2024-09-13T10:11:44.705 INFO:tasks.rados.rados.0.smithi195.stdout:1468: seq_num 298 ranges {608486=561969,1871634=634092,2950640=57846} 2024-09-13T10:11:44.705 INFO:tasks.rados.rados.0.smithi195.stdout:1468: writing smithi19530714-38 from 608486 to 1170455 tid 1 2024-09-13T10:11:44.705 INFO:tasks.rados.rados.0.smithi195.stdout:1468: writing smithi19530714-38 from 1871634 to 2505726 tid 2 2024-09-13T10:11:44.705 INFO:tasks.rados.rados.0.smithi195.stdout:1468: writing smithi19530714-38 from 2950640 to 3008486 tid 3 2024-09-13T10:11:44.705 INFO:tasks.rados.rados.0.smithi195.stdout:1469: read oid 33 snap -1 2024-09-13T10:11:44.705 INFO:tasks.rados.rados.0.smithi195.stdout:1469: expect (ObjNum 253 snap 0 seq_num 253) 2024-09-13T10:11:44.705 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:44.706 INFO:tasks.rados.rados.0.smithi195.stdout:1455: got (22) Invalid argument 2024-09-13T10:11:44.706 INFO:tasks.rados.rados.0.smithi195.stdout:1455: done (15 left) 2024-09-13T10:11:44.706 INFO:tasks.rados.rados.0.smithi195.stdout:1456: done (14 left) 2024-09-13T10:11:44.706 INFO:tasks.rados.rados.0.smithi195.stdout:1457: done (13 left) 2024-09-13T10:11:44.706 INFO:tasks.rados.rados.0.smithi195.stdout:1458: done (12 left) 2024-09-13T10:11:44.706 INFO:tasks.rados.rados.0.smithi195.stdout:1470: tier_promote oid 46 2024-09-13T10:11:44.706 INFO:tasks.rados.rados.0.smithi195.stdout:1454: expect (ObjNum 274 snap 0 seq_num 274) 2024-09-13T10:11:44.706 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:44.691+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:44.764 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2357540 2024-09-13T10:11:44.772 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:11:44.798 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:44.794+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:44.837 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:44.837 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:44.837 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:11:44.838 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:44.851 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T10:11:44.876 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1545508 2024-09-13T10:11:44.918 INFO:tasks.rados.rados.0.smithi195.stdout:1460: finishing set_chunk to oid 44 2024-09-13T10:11:44.918 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 44 set_chunk returned error code -95 offset: 1531392 length: 379392 2024-09-13T10:11:44.918 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 0 (ObjNum 205 snap 0 seq_num 205) dirty exists 2024-09-13T10:11:44.918 INFO:tasks.rados.rados.0.smithi195.stdout:1454: done (12 left) 2024-09-13T10:11:44.918 INFO:tasks.rados.rados.0.smithi195.stdout:1460: done (11 left) 2024-09-13T10:11:44.918 INFO:tasks.rados.rados.0.smithi195.stdout:1462: done (10 left) 2024-09-13T10:11:44.919 INFO:tasks.rados.rados.0.smithi195.stdout:1471: set_chunk oid 20 target oid 2024-09-13T10:11:44.919 INFO:tasks.rados.rados.0.smithi195.stdout:1471: set_chunk oid 20 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:44.919 INFO:tasks.rados.rados.0.smithi195.stdout:1472: read oid 31 snap -1 2024-09-13T10:11:44.919 INFO:tasks.rados.rados.0.smithi195.stdout:1472: expect (ObjNum 272 snap 0 seq_num 272) 2024-09-13T10:11:44.919 INFO:tasks.rados.rados.0.smithi195.stdout:1473: read oid 30 snap -1 2024-09-13T10:11:44.919 INFO:tasks.rados.rados.0.smithi195.stdout:1473: expect deleted 2024-09-13T10:11:44.919 INFO:tasks.rados.rados.0.smithi195.stdout:1474: read oid 39 snap -1 2024-09-13T10:11:44.919 INFO:tasks.rados.rados.0.smithi195.stdout:1474: expect (ObjNum 288 snap 0 seq_num 288) 2024-09-13T10:11:44.919 INFO:tasks.rados.rados.0.smithi195.stdout:1475: read oid 24 snap -1 2024-09-13T10:11:44.919 INFO:tasks.rados.rados.0.smithi195.stdout:1475: expect (ObjNum 262 snap 0 seq_num 262) 2024-09-13T10:11:44.919 INFO:tasks.rados.rados.0.smithi195.stdout:1476: read oid 37 snap -1 2024-09-13T10:11:44.919 INFO:tasks.rados.rados.0.smithi195.stdout:1476: expect deleted 2024-09-13T10:11:44.920 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:44.914+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:44.920 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:44.920 INFO:tasks.rados.rados.0.smithi195.stdout:1461: expect (ObjNum 237 snap 0 seq_num 237) 2024-09-13T10:11:44.929 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:44.929 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:44.951 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:44.951 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:44.972 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:44.973 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:44.994 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:44.995 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:45.000 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:44.995+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:45.002 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:45.003 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:45.015 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:45.015 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:45.016 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T10:11:45.017 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1992440 2024-09-13T10:11:45.036 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:45.036 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:45.058 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:45.058 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:45.079 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:45.079 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:45.088 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T10:11:45.098 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:45.099 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:45.101 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:45.095+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:45.119 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:45.120 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:45.137 INFO:tasks.rados.rados.0.smithi195.stdout:1463: expect (ObjNum 229 snap 0 seq_num 229) 2024-09-13T10:11:45.140 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:45.140 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:45.159 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:45.159 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:45.175 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:45.175 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:45.176 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:45.176 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:45.189 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T10:11:45.197 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:45.197 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:45.202 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:45.198+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:45.219 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:45.219 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:45.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:45.231+0000 7f5095ae1640 0 -- 172.21.15.73:0/1553147703 >> v2:172.21.15.195:6805/848686366 conn(0x7f50902227e0 msgr2=0x7f5090224bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:11:45.237 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:45.237 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:45.302 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:45.299+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:45.309 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:45.310 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:45.323 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T10:11:45.392 INFO:tasks.rados.rados.0.smithi195.stdout:1464: expect (ObjNum 287 snap 0 seq_num 287) 2024-09-13T10:11:45.403 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:45.395+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:45.417 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:45.448 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:45.448 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:45.462 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T10:11:45.472 INFO:tasks.rados.rados.0.smithi195.stdout:1459: got (0) Success 2024-09-13T10:11:45.472 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 79 (ObjNum 139 snap 0 seq_num 139) dirty exists 2024-09-13T10:11:45.472 INFO:tasks.rados.rados.0.smithi195.stdout:1465: expect (ObjNum 280 snap 0 seq_num 280) 2024-09-13T10:11:45.503 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:45.498+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:45.508 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2001631 2024-09-13T10:11:45.519 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:45.581 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:45.581 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:45.594 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-13T10:11:45.606 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:45.603+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:45.623 INFO:tasks.rados.rados.0.smithi195.stdout:1466: finishing set_chunk to oid 13 2024-09-13T10:11:45.623 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 13 set_chunk returned error code -95 offset: 1028096 length: 1376768 2024-09-13T10:11:45.623 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 0 (ObjNum 228 snap 0 seq_num 228) dirty exists 2024-09-13T10:11:45.623 INFO:tasks.rados.rados.0.smithi195.stdout:1467: expect (ObjNum 263 snap 0 seq_num 263) 2024-09-13T10:11:45.707 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:45.703+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:45.737 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:45.737 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:45.750 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-13T10:11:45.753 INFO:tasks.rados.rados.0.smithi195.stdout:1468: finishing write tid 1 to smithi19530714-38 2024-09-13T10:11:45.753 INFO:tasks.rados.rados.0.smithi195.stdout:1470: got (0) Success 2024-09-13T10:11:45.753 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 49 (ObjNum 295 snap 0 seq_num 295) dirty exists 2024-09-13T10:11:45.754 INFO:tasks.rados.rados.0.smithi195.stdout:1468: finishing write tid 2 to smithi19530714-38 2024-09-13T10:11:45.754 INFO:tasks.rados.rados.0.smithi195.stdout:1468: finishing write tid 3 to smithi19530714-38 2024-09-13T10:11:45.754 INFO:tasks.rados.rados.0.smithi195.stdout:1468: finishing write tid 4 to smithi19530714-38 2024-09-13T10:11:45.754 INFO:tasks.rados.rados.0.smithi195.stdout:1468: finishing write tid 5 to smithi19530714-38 2024-09-13T10:11:45.754 INFO:tasks.rados.rados.0.smithi195.stdout:1468: oid 38 updating version 0 to 137 2024-09-13T10:11:45.754 INFO:tasks.rados.rados.0.smithi195.stdout:1468: oid 38 updating version 137 to 139 2024-09-13T10:11:45.754 INFO:tasks.rados.rados.0.smithi195.stdout:1468: oid 38 version 139 is already newer than 138 2024-09-13T10:11:45.754 INFO:tasks.rados.rados.0.smithi195.stdout:1468: oid 38 updating version 139 to 140 2024-09-13T10:11:45.754 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 140 (ObjNum 298 snap 0 seq_num 298) dirty exists 2024-09-13T10:11:45.754 INFO:tasks.rados.rados.0.smithi195.stdout:1468: left oid 38 (ObjNum 298 snap 0 seq_num 298) 2024-09-13T10:11:45.754 INFO:tasks.rados.rados.0.smithi195.stdout:1471: finishing set_chunk to oid 20 2024-09-13T10:11:45.754 INFO:tasks.rados.rados.0.smithi195.stdout:1471: got expected ENOENT (src dne) 2024-09-13T10:11:45.754 INFO:tasks.rados.rados.0.smithi195.stdout:1472: expect (ObjNum 272 snap 0 seq_num 272) 2024-09-13T10:11:45.788 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:45.809 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:45.806+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:45.857 INFO:tasks.rados.rados.0.smithi195.stdout:1474: expect (ObjNum 288 snap 0 seq_num 288) 2024-09-13T10:11:45.863 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:45.864 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:45.877 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-13T10:11:45.889 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2727976 2024-09-13T10:11:45.910 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:45.911 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:45.910+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:45.973 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:46.006 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:46.006 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:46.011 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:46.006+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:46.020 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-13T10:11:46.037 INFO:tasks.rados.rados.0.smithi195.stdout:1475: expect (ObjNum 262 snap 0 seq_num 262) 2024-09-13T10:11:46.037 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:46.089 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-13T10:11:46.112 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:46.110+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:46.151 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:46.151 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:46.166 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-13T10:11:46.173 INFO:tasks.rados.rados.0.smithi195.stdout:1469: expect (ObjNum 253 snap 0 seq_num 253) 2024-09-13T10:11:46.173 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:46.196 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 550639 2024-09-13T10:11:46.213 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:46.210+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:46.216 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:46.240 INFO:tasks.rados.rados.0.smithi195.stdout:1459: done (15 left) 2024-09-13T10:11:46.240 INFO:tasks.rados.rados.0.smithi195.stdout:1461: done (14 left) 2024-09-13T10:11:46.240 INFO:tasks.rados.rados.0.smithi195.stdout:1463: done (13 left) 2024-09-13T10:11:46.240 INFO:tasks.rados.rados.0.smithi195.stdout:1464: done (12 left) 2024-09-13T10:11:46.240 INFO:tasks.rados.rados.0.smithi195.stdout:1465: done (11 left) 2024-09-13T10:11:46.240 INFO:tasks.rados.rados.0.smithi195.stdout:1466: done (10 left) 2024-09-13T10:11:46.240 INFO:tasks.rados.rados.0.smithi195.stdout:1467: done (9 left) 2024-09-13T10:11:46.240 INFO:tasks.rados.rados.0.smithi195.stdout:1468: done (8 left) 2024-09-13T10:11:46.240 INFO:tasks.rados.rados.0.smithi195.stdout:1469: done (7 left) 2024-09-13T10:11:46.240 INFO:tasks.rados.rados.0.smithi195.stdout:1470: done (6 left) 2024-09-13T10:11:46.240 INFO:tasks.rados.rados.0.smithi195.stdout:1471: done (5 left) 2024-09-13T10:11:46.240 INFO:tasks.rados.rados.0.smithi195.stdout:1472: done (4 left) 2024-09-13T10:11:46.241 INFO:tasks.rados.rados.0.smithi195.stdout:1473: done (3 left) 2024-09-13T10:11:46.241 INFO:tasks.rados.rados.0.smithi195.stdout:1474: done (2 left) 2024-09-13T10:11:46.241 INFO:tasks.rados.rados.0.smithi195.stdout:1475: done (1 left) 2024-09-13T10:11:46.241 INFO:tasks.rados.rados.0.smithi195.stdout:1476: done (0 left) 2024-09-13T10:11:46.241 INFO:tasks.rados.rados.0.smithi195.stdout:1477: write (excl) oid 17 current snap is 0 2024-09-13T10:11:46.241 INFO:tasks.rados.rados.0.smithi195.stdout:1477: seq_num 299 ranges {625293=637218,1813763=494103,2978063=47230} 2024-09-13T10:11:46.259 INFO:tasks.rados.rados.0.smithi195.stdout:1477: writing smithi19530714-17 from 625293 to 1262511 tid 1 2024-09-13T10:11:46.273 INFO:tasks.rados.rados.0.smithi195.stdout:1477: writing smithi19530714-17 from 1813763 to 2307866 tid 2 2024-09-13T10:11:46.281 INFO:tasks.rados.rados.0.smithi195.stdout:1477: writing smithi19530714-17 from 2978063 to 3025293 tid 3 2024-09-13T10:11:46.281 INFO:tasks.rados.rados.0.smithi195.stdout:1478: read oid 24 snap -1 2024-09-13T10:11:46.281 INFO:tasks.rados.rados.0.smithi195.stdout:1478: expect (ObjNum 262 snap 0 seq_num 262) 2024-09-13T10:11:46.281 INFO:tasks.rados.rados.0.smithi195.stdout:1479: write (excl) oid 35 current snap is 0 2024-09-13T10:11:46.282 INFO:tasks.rados.rados.0.smithi195.stdout:1477: finishing write tid 1 to smithi19530714-17 2024-09-13T10:11:46.282 INFO:tasks.rados.rados.0.smithi195.stdout:1477: finishing write tid 2 to smithi19530714-17 2024-09-13T10:11:46.282 INFO:tasks.rados.rados.0.smithi195.stdout:1479: seq_num 300 ranges {642100=712467,1872245=637761} 2024-09-13T10:11:46.285 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:46.285 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:46.294 INFO:tasks.rados.rados.0.smithi195.stdout:1479: writing smithi19530714-35 from 642100 to 1354567 tid 1 2024-09-13T10:11:46.302 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-13T10:11:46.307 INFO:tasks.rados.rados.0.smithi195.stdout:1479: writing smithi19530714-35 from 1872245 to 2510006 tid 2 2024-09-13T10:11:46.307 INFO:tasks.rados.rados.0.smithi195.stdout:1480: tier_promote oid 4 2024-09-13T10:11:46.307 INFO:tasks.rados.rados.0.smithi195.stdout:1477: finishing write tid 3 to smithi19530714-17 2024-09-13T10:11:46.307 INFO:tasks.rados.rados.0.smithi195.stdout:1477: finishing write tid 4 to smithi19530714-17 2024-09-13T10:11:46.307 INFO:tasks.rados.rados.0.smithi195.stdout:1477: finishing write tid 5 to smithi19530714-17 2024-09-13T10:11:46.307 INFO:tasks.rados.rados.0.smithi195.stdout:1477: oid 17 updating version 0 to 75 2024-09-13T10:11:46.307 INFO:tasks.rados.rados.0.smithi195.stdout:1477: oid 17 updating version 75 to 77 2024-09-13T10:11:46.308 INFO:tasks.rados.rados.0.smithi195.stdout:1477: oid 17 version 77 is already newer than 76 2024-09-13T10:11:46.308 INFO:tasks.rados.rados.0.smithi195.stdout:1477: oid 17 updating version 77 to 78 2024-09-13T10:11:46.308 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 78 (ObjNum 299 snap 0 seq_num 299) dirty exists 2024-09-13T10:11:46.308 INFO:tasks.rados.rados.0.smithi195.stdout:1477: left oid 17 (ObjNum 299 snap 0 seq_num 299) 2024-09-13T10:11:46.308 INFO:tasks.rados.rados.0.smithi195.stdout:1478: expect (ObjNum 262 snap 0 seq_num 262) 2024-09-13T10:11:46.308 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:46.314 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:46.310+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:46.408 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:46.415 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:46.410+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:46.425 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:46.425 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:46.438 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T10:11:46.457 INFO:tasks.rados.rados.0.smithi195.stdout:1479: finishing write tid 1 to smithi19530714-35 2024-09-13T10:11:46.457 INFO:tasks.rados.rados.0.smithi195.stdout:1479: finishing write tid 2 to smithi19530714-35 2024-09-13T10:11:46.457 INFO:tasks.rados.rados.0.smithi195.stdout:1479: finishing write tid 3 to smithi19530714-35 2024-09-13T10:11:46.457 INFO:tasks.rados.rados.0.smithi195.stdout:1479: finishing write tid 4 to smithi19530714-35 2024-09-13T10:11:46.457 INFO:tasks.rados.rados.0.smithi195.stdout:1479: oid 35 updating version 0 to 76 2024-09-13T10:11:46.457 INFO:tasks.rados.rados.0.smithi195.stdout:1479: oid 35 version 76 is already newer than 74 2024-09-13T10:11:46.457 INFO:tasks.rados.rados.0.smithi195.stdout:1479: oid 35 version 76 is already newer than 75 2024-09-13T10:11:46.458 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 76 (ObjNum 300 snap 0 seq_num 300) dirty exists 2024-09-13T10:11:46.458 INFO:tasks.rados.rados.0.smithi195.stdout:1479: left oid 35 (ObjNum 300 snap 0 seq_num 300) 2024-09-13T10:11:46.458 INFO:tasks.rados.rados.0.smithi195.stdout:1480: got (0) Success 2024-09-13T10:11:46.458 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 105 (ObjNum 284 snap 0 seq_num 284) dirty exists 2024-09-13T10:11:46.458 INFO:tasks.rados.rados.0.smithi195.stdout:1477: done (3 left) 2024-09-13T10:11:46.458 INFO:tasks.rados.rados.0.smithi195.stdout:1478: done (2 left) 2024-09-13T10:11:46.458 INFO:tasks.rados.rados.0.smithi195.stdout:1479: done (1 left) 2024-09-13T10:11:46.458 INFO:tasks.rados.rados.0.smithi195.stdout:1480: done (0 left) 2024-09-13T10:11:46.458 INFO:tasks.rados.rados.0.smithi195.stdout:1481: set_chunk oid 8 target oid 2024-09-13T10:11:46.458 INFO:tasks.rados.rados.0.smithi195.stdout:1481: set_chunk oid 8 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:46.458 INFO:tasks.rados.rados.0.smithi195.stdout:1482: write (excl) oid 15 current snap is 0 2024-09-13T10:11:46.458 INFO:tasks.rados.rados.0.smithi195.stdout:1482: seq_num 301 ranges {658907=787716,1930727=497772,3032909=25998} 2024-09-13T10:11:46.469 INFO:tasks.rados.rados.0.smithi195.stdout:1482: writing smithi19530714-15 from 658907 to 1446623 tid 1 2024-09-13T10:11:46.476 INFO:tasks.rados.rados.0.smithi195.stdout:1482: writing smithi19530714-15 from 1930727 to 2428499 tid 2 2024-09-13T10:11:46.481 INFO:tasks.rados.rados.0.smithi195.stdout:1482: writing smithi19530714-15 from 3032909 to 3058907 tid 3 2024-09-13T10:11:46.481 INFO:tasks.rados.rados.0.smithi195.stdout:1481: finishing set_chunk to oid 8 2024-09-13T10:11:46.482 INFO:tasks.rados.rados.0.smithi195.stdout:1481: got expected ENOENT (src dne) 2024-09-13T10:11:46.482 INFO:tasks.rados.rados.0.smithi195.stdout:1482: finishing write tid 1 to smithi19530714-15 2024-09-13T10:11:46.482 INFO:tasks.rados.rados.0.smithi195.stdout:1483: read oid 21 snap -1 2024-09-13T10:11:46.482 INFO:tasks.rados.rados.0.smithi195.stdout:1483: expect (ObjNum 235 snap 0 seq_num 235) 2024-09-13T10:11:46.482 INFO:tasks.rados.rados.0.smithi195.stdout:1481: done (2 left) 2024-09-13T10:11:46.482 INFO:tasks.rados.rados.0.smithi195.stdout:1484: write (excl) oid 11 current snap is 0 2024-09-13T10:11:46.482 INFO:tasks.rados.rados.0.smithi195.stdout:1484: seq_num 302 ranges {675714=462965,1589209=641430,2943979=131735} 2024-09-13T10:11:46.490 INFO:tasks.rados.rados.0.smithi195.stdout:1484: writing smithi19530714-11 from 675714 to 1138679 tid 1 2024-09-13T10:11:46.498 INFO:tasks.rados.rados.0.smithi195.stdout:1484: writing smithi19530714-11 from 1589209 to 2230639 tid 2 2024-09-13T10:11:46.505 INFO:tasks.rados.rados.0.smithi195.stdout:1484: writing smithi19530714-11 from 2943979 to 3075714 tid 3 2024-09-13T10:11:46.505 INFO:tasks.rados.rados.0.smithi195.stdout:1485: write oid 50 current snap is 0 2024-09-13T10:11:46.505 INFO:tasks.rados.rados.0.smithi195.stdout:1482: finishing write tid 2 to smithi19530714-15 2024-09-13T10:11:46.505 INFO:tasks.rados.rados.0.smithi195.stdout:1482: finishing write tid 3 to smithi19530714-15 2024-09-13T10:11:46.505 INFO:tasks.rados.rados.0.smithi195.stdout:1482: finishing write tid 4 to smithi19530714-15 2024-09-13T10:11:46.505 INFO:tasks.rados.rados.0.smithi195.stdout:1482: finishing write tid 5 to smithi19530714-15 2024-09-13T10:11:46.505 INFO:tasks.rados.rados.0.smithi195.stdout:1482: oid 15 updating version 0 to 100 2024-09-13T10:11:46.506 INFO:tasks.rados.rados.0.smithi195.stdout:1482: oid 15 version 100 is already newer than 98 2024-09-13T10:11:46.506 INFO:tasks.rados.rados.0.smithi195.stdout:1482: oid 15 version 100 is already newer than 99 2024-09-13T10:11:46.506 INFO:tasks.rados.rados.0.smithi195.stdout:1482: oid 15 updating version 100 to 101 2024-09-13T10:11:46.506 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 101 (ObjNum 301 snap 0 seq_num 301) dirty exists 2024-09-13T10:11:46.506 INFO:tasks.rados.rados.0.smithi195.stdout:1482: left oid 15 (ObjNum 301 snap 0 seq_num 301) 2024-09-13T10:11:46.506 INFO:tasks.rados.rados.0.smithi195.stdout:1483: expect (ObjNum 235 snap 0 seq_num 235) 2024-09-13T10:11:46.516 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:46.511+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:46.571 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:46.571 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:46.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T10:11:46.617 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:46.611+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:46.686 INFO:tasks.rados.rados.0.smithi195.stdout:1484: finishing write tid 1 to smithi19530714-11 2024-09-13T10:11:46.686 INFO:tasks.rados.rados.0.smithi195.stdout:1484: finishing write tid 2 to smithi19530714-11 2024-09-13T10:11:46.686 INFO:tasks.rados.rados.0.smithi195.stdout:1484: finishing write tid 3 to smithi19530714-11 2024-09-13T10:11:46.686 INFO:tasks.rados.rados.0.smithi195.stdout:1484: finishing write tid 4 to smithi19530714-11 2024-09-13T10:11:46.686 INFO:tasks.rados.rados.0.smithi195.stdout:1484: finishing write tid 5 to smithi19530714-11 2024-09-13T10:11:46.687 INFO:tasks.rados.rados.0.smithi195.stdout:1484: oid 11 updating version 0 to 87 2024-09-13T10:11:46.687 INFO:tasks.rados.rados.0.smithi195.stdout:1484: oid 11 version 87 is already newer than 86 2024-09-13T10:11:46.687 INFO:tasks.rados.rados.0.smithi195.stdout:1484: oid 11 version 87 is already newer than 84 2024-09-13T10:11:46.687 INFO:tasks.rados.rados.0.smithi195.stdout:1484: oid 11 version 87 is already newer than 85 2024-09-13T10:11:46.687 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 87 (ObjNum 302 snap 0 seq_num 302) dirty exists 2024-09-13T10:11:46.687 INFO:tasks.rados.rados.0.smithi195.stdout:1484: left oid 11 (ObjNum 302 snap 0 seq_num 302) 2024-09-13T10:11:46.687 INFO:tasks.rados.rados.0.smithi195.stdout:1485: seq_num 303 ranges {692521=538214,1931338=785088} 2024-09-13T10:11:46.697 INFO:tasks.rados.rados.0.smithi195.stdout:1485: writing smithi19530714-50 from 692521 to 1230735 tid 1 2024-09-13T10:11:46.707 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:46.707 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:46.710 INFO:tasks.rados.rados.0.smithi195.stdout:1485: writing smithi19530714-50 from 1931338 to 2716426 tid 2 2024-09-13T10:11:46.710 INFO:tasks.rados.rados.0.smithi195.stdout:1482: done (3 left) 2024-09-13T10:11:46.711 INFO:tasks.rados.rados.0.smithi195.stdout:1483: done (2 left) 2024-09-13T10:11:46.711 INFO:tasks.rados.rados.0.smithi195.stdout:1484: done (1 left) 2024-09-13T10:11:46.711 INFO:tasks.rados.rados.0.smithi195.stdout:1486: read oid 22 snap -1 2024-09-13T10:11:46.711 INFO:tasks.rados.rados.0.smithi195.stdout:1486: expect (ObjNum 139 snap 0 seq_num 139) 2024-09-13T10:11:46.711 INFO:tasks.rados.rados.0.smithi195.stdout:1487: set_chunk oid 2 target oid 2024-09-13T10:11:46.718 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:46.714+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:46.721 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T10:11:46.730 INFO:tasks.rados.rados.0.smithi195.stdout:1487: set_chunk oid 2 offset: 853504 length: 634880 target oid 8d079475f7f96345b17fa24bf736674786cf3ddeb9f51af52790eec64f2bec49 offset: 0 snap -1 2024-09-13T10:11:46.730 INFO:tasks.rados.rados.0.smithi195.stdout:1488: set_chunk oid 19 target oid 2024-09-13T10:11:46.746 INFO:tasks.rados.rados.0.smithi195.stdout:1488: set_chunk oid 19 offset: 683008 length: 761856 target oid 5d7514268155e8ad01913331644ecb5e6accca6a1534c5d2b51479dbfbae8a41 offset: 0 snap -1 2024-09-13T10:11:46.746 INFO:tasks.rados.rados.0.smithi195.stdout:1489: set_chunk oid 18 target oid 2024-09-13T10:11:46.767 INFO:tasks.rados.rados.0.smithi195.stdout:1489: set_chunk oid 18 offset: 1286656 length: 1355264 target oid 75e0b79c3a248162d173515782ce04b534b69906c7834d185f017f9e5815e675 offset: 0 snap -1 2024-09-13T10:11:46.767 INFO:tasks.rados.rados.0.smithi195.stdout:1490: read oid 14 snap -1 2024-09-13T10:11:46.768 INFO:tasks.rados.rados.0.smithi195.stdout:1490: expect (ObjNum 243 snap 0 seq_num 243) 2024-09-13T10:11:46.768 INFO:tasks.rados.rados.0.smithi195.stdout:1491: read oid 37 snap -1 2024-09-13T10:11:46.768 INFO:tasks.rados.rados.0.smithi195.stdout:1491: expect deleted 2024-09-13T10:11:46.768 INFO:tasks.rados.rados.0.smithi195.stdout:1492: read oid 13 snap -1 2024-09-13T10:11:46.768 INFO:tasks.rados.rados.0.smithi195.stdout:1492: expect (ObjNum 228 snap 0 seq_num 228) 2024-09-13T10:11:46.768 INFO:tasks.rados.rados.0.smithi195.stdout:1493: tier_promote oid 49 2024-09-13T10:11:46.768 INFO:tasks.rados.rados.0.smithi195.stdout:1485: finishing write tid 1 to smithi19530714-50 2024-09-13T10:11:46.768 INFO:tasks.rados.rados.0.smithi195.stdout:1494: set_chunk oid 33 target oid 2024-09-13T10:11:46.782 INFO:tasks.rados.rados.0.smithi195.stdout:1494: set_chunk oid 33 offset: 795648 length: 794112 target oid 680493d497451f2bd1e815e68adff6d70b9eb1f9834871dee319b6d8ce69674f offset: 0 snap -1 2024-09-13T10:11:46.783 INFO:tasks.rados.rados.0.smithi195.stdout:1495: read oid 45 snap -1 2024-09-13T10:11:46.783 INFO:tasks.rados.rados.0.smithi195.stdout:1495: expect (ObjNum 268 snap 0 seq_num 268) 2024-09-13T10:11:46.783 INFO:tasks.rados.rados.0.smithi195.stdout:1496: delete oid 39 current snap is 0 2024-09-13T10:11:46.783 INFO:tasks.rados.rados.0.smithi195.stdout:1485: finishing write tid 2 to smithi19530714-50 2024-09-13T10:11:46.783 INFO:tasks.rados.rados.0.smithi195.stdout:1485: finishing write tid 3 to smithi19530714-50 2024-09-13T10:11:46.783 INFO:tasks.rados.rados.0.smithi195.stdout:1485: finishing write tid 4 to smithi19530714-50 2024-09-13T10:11:46.783 INFO:tasks.rados.rados.0.smithi195.stdout:1485: oid 50 updating version 0 to 46 2024-09-13T10:11:46.783 INFO:tasks.rados.rados.0.smithi195.stdout:1485: oid 50 version 46 is already newer than 45 2024-09-13T10:11:46.783 INFO:tasks.rados.rados.0.smithi195.stdout:1485: oid 50 updating version 46 to 47 2024-09-13T10:11:46.783 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 47 (ObjNum 303 snap 0 seq_num 303) dirty exists 2024-09-13T10:11:46.783 INFO:tasks.rados.rados.0.smithi195.stdout:1485: left oid 50 (ObjNum 303 snap 0 seq_num 303) 2024-09-13T10:11:46.783 INFO:tasks.rados.rados.0.smithi195.stdout:1486: expect (ObjNum 139 snap 0 seq_num 139) 2024-09-13T10:11:46.819 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:46.815+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:46.842 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:46.842 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:46.856 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T10:11:46.901 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1422394 2024-09-13T10:11:46.920 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:46.914+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:46.979 INFO:tasks.rados.rados.0.smithi195.stdout:1487: finishing set_chunk to oid 2 2024-09-13T10:11:46.979 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 2 set_chunk returned error code -95 offset: 853504 length: 634880 2024-09-13T10:11:46.979 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 0 (ObjNum 280 snap 0 seq_num 280) dirty exists 2024-09-13T10:11:46.979 INFO:tasks.rados.rados.0.smithi195.stdout:1488: finishing set_chunk to oid 19 2024-09-13T10:11:46.979 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 19 set_chunk returned error code -95 offset: 683008 length: 761856 2024-09-13T10:11:46.979 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 0 (ObjNum 160 snap 0 seq_num 160) dirty exists 2024-09-13T10:11:46.979 INFO:tasks.rados.rados.0.smithi195.stdout:1492: expect (ObjNum 228 snap 0 seq_num 228) 2024-09-13T10:11:47.021 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:47.021 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:47.021 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:47.018+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:47.035 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T10:11:47.122 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:47.115+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:47.157 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:47.158 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:47.172 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T10:11:47.198 INFO:tasks.rados.rados.0.smithi195.stdout:1489: finishing set_chunk to oid 18 2024-09-13T10:11:47.198 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 102 (ObjNum 277 snap 0 seq_num 277) dirty exists 2024-09-13T10:11:47.198 INFO:tasks.rados.rados.0.smithi195.stdout:1490: expect (ObjNum 243 snap 0 seq_num 243) 2024-09-13T10:11:47.223 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:47.218+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:47.255 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:11:47.259 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:47.293 INFO:tasks.rados.rados.0.smithi195.stdout:1493: got (0) Success 2024-09-13T10:11:47.293 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 100 (ObjNum 181 snap 0 seq_num 181) dirty exists 2024-09-13T10:11:47.293 INFO:tasks.rados.rados.0.smithi195.stdout:1494: finishing set_chunk to oid 33 2024-09-13T10:11:47.293 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 51 (ObjNum 253 snap 0 seq_num 253) dirty exists 2024-09-13T10:11:47.294 INFO:tasks.rados.rados.0.smithi195.stdout:1495: expect (ObjNum 268 snap 0 seq_num 268) 2024-09-13T10:11:47.295 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:47.295 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:47.310 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-13T10:11:47.359 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:47.355+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:47.365 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1798559 2024-09-13T10:11:47.395 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:47.396 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:47.416 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:47.417 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:47.424 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:47.419+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:47.430 INFO:tasks.rados.rados.0.smithi195.stdout:1485: done (11 left) 2024-09-13T10:11:47.431 INFO:tasks.rados.rados.0.smithi195.stdout:1486: done (10 left) 2024-09-13T10:11:47.431 INFO:tasks.rados.rados.0.smithi195.stdout:1487: done (9 left) 2024-09-13T10:11:47.431 INFO:tasks.rados.rados.0.smithi195.stdout:1488: done (8 left) 2024-09-13T10:11:47.431 INFO:tasks.rados.rados.0.smithi195.stdout:1489: done (7 left) 2024-09-13T10:11:47.431 INFO:tasks.rados.rados.0.smithi195.stdout:1490: done (6 left) 2024-09-13T10:11:47.431 INFO:tasks.rados.rados.0.smithi195.stdout:1491: done (5 left) 2024-09-13T10:11:47.431 INFO:tasks.rados.rados.0.smithi195.stdout:1492: done (4 left) 2024-09-13T10:11:47.431 INFO:tasks.rados.rados.0.smithi195.stdout:1493: done (3 left) 2024-09-13T10:11:47.431 INFO:tasks.rados.rados.0.smithi195.stdout:1494: done (2 left) 2024-09-13T10:11:47.431 INFO:tasks.rados.rados.0.smithi195.stdout:1495: done (1 left) 2024-09-13T10:11:47.431 INFO:tasks.rados.rados.0.smithi195.stdout:1496: done (0 left) 2024-09-13T10:11:47.431 INFO:tasks.rados.rados.0.smithi195.stdout:1497: write (excl) oid 41 current snap is 0 2024-09-13T10:11:47.431 INFO:tasks.rados.rados.0.smithi195.stdout:1497: seq_num 305 ranges {726135=405065,1764655=788757,3026248=99887} 2024-09-13T10:11:47.438 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:47.438 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:47.449 INFO:tasks.rados.rados.0.smithi195.stdout:1497: writing smithi19530714-41 from 726135 to 1131200 tid 1 2024-09-13T10:11:47.458 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:47.458 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:47.472 INFO:tasks.rados.rados.0.smithi195.stdout:1497: writing smithi19530714-41 from 1764655 to 2553412 tid 2 2024-09-13T10:11:47.475 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:47.475 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:47.475 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:47.475 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:47.481 INFO:tasks.rados.rados.0.smithi195.stdout:1497: writing smithi19530714-41 from 3026248 to 3126135 tid 3 2024-09-13T10:11:47.482 INFO:tasks.rados.rados.0.smithi195.stdout:1498: read oid 17 snap -1 2024-09-13T10:11:47.482 INFO:tasks.rados.rados.0.smithi195.stdout:1498: expect (ObjNum 299 snap 0 seq_num 299) 2024-09-13T10:11:47.482 INFO:tasks.rados.rados.0.smithi195.stdout:1499: read oid 23 snap -1 2024-09-13T10:11:47.482 INFO:tasks.rados.rados.0.smithi195.stdout:1499: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:11:47.482 INFO:tasks.rados.rados.0.smithi195.stdout:1500: set_chunk oid 19 target oid 2024-09-13T10:11:47.491 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-13T10:11:47.497 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:47.497 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:47.514 INFO:tasks.rados.rados.0.smithi195.stdout:1500: set_chunk oid 19 offset: 308736 length: 2264064 target oid 7feaa84f5417f1ff3792d55c7143e8324c851d4769fc29b9e34d0e3d51ad1177 offset: 0 snap -1 2024-09-13T10:11:47.514 INFO:tasks.rados.rados.0.smithi195.stdout:1501: read oid 24 snap -1 2024-09-13T10:11:47.514 INFO:tasks.rados.rados.0.smithi195.stdout:1501: expect (ObjNum 262 snap 0 seq_num 262) 2024-09-13T10:11:47.514 INFO:tasks.rados.rados.0.smithi195.stdout:1502: read oid 14 snap -1 2024-09-13T10:11:47.514 INFO:tasks.rados.rados.0.smithi195.stdout:1502: expect (ObjNum 243 snap 0 seq_num 243) 2024-09-13T10:11:47.514 INFO:tasks.rados.rados.0.smithi195.stdout:1503: tier_flush oid 10 2024-09-13T10:11:47.514 INFO:tasks.rados.rados.0.smithi195.stdout:1503: tier_flush oid 10 snap -1 2024-09-13T10:11:47.514 INFO:tasks.rados.rados.0.smithi195.stdout:1497: finishing write tid 1 to smithi19530714-41 2024-09-13T10:11:47.514 INFO:tasks.rados.rados.0.smithi195.stdout:1497: finishing write tid 2 to smithi19530714-41 2024-09-13T10:11:47.514 INFO:tasks.rados.rados.0.smithi195.stdout:1497: finishing write tid 3 to smithi19530714-41 2024-09-13T10:11:47.514 INFO:tasks.rados.rados.0.smithi195.stdout:1497: finishing write tid 4 to smithi19530714-41 2024-09-13T10:11:47.514 INFO:tasks.rados.rados.0.smithi195.stdout:1497: finishing write tid 5 to smithi19530714-41 2024-09-13T10:11:47.515 INFO:tasks.rados.rados.0.smithi195.stdout:1497: oid 41 updating version 0 to 51 2024-09-13T10:11:47.515 INFO:tasks.rados.rados.0.smithi195.stdout:1497: oid 41 updating version 51 to 52 2024-09-13T10:11:47.515 INFO:tasks.rados.rados.0.smithi195.stdout:1497: oid 41 updating version 52 to 53 2024-09-13T10:11:47.515 INFO:tasks.rados.rados.0.smithi195.stdout:1497: oid 41 updating version 53 to 54 2024-09-13T10:11:47.515 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 54 (ObjNum 305 snap 0 seq_num 305) dirty exists 2024-09-13T10:11:47.515 INFO:tasks.rados.rados.0.smithi195.stdout:1497: left oid 41 (ObjNum 305 snap 0 seq_num 305) 2024-09-13T10:11:47.515 INFO:tasks.rados.rados.0.smithi195.stdout:1499: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:11:47.522 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:47.522 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:47.525 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:47.522+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:47.544 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:47.544 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:47.544 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1856438 2024-09-13T10:11:47.564 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:47.565 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:47.587 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:47.608 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:47.608 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:47.615 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:47.616 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:47.616 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:11:47.616 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:11:47.616 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:47.616 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:47.626 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:47.619+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:47.628 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:47.628 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:47.629 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-13T10:11:47.645 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:47.645 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:47.666 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:47.666 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:47.670 INFO:tasks.rados.rados.0.smithi195.stdout:1498: expect (ObjNum 299 snap 0 seq_num 299) 2024-09-13T10:11:47.687 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:47.687 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:47.708 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:47.708 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:47.725 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 189961 2024-09-13T10:11:47.726 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:47.721+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:47.769 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:47.769 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:11:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:11:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:11:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:11:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 77.385608641999994, 2024-09-13T10:11:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034624456999999997, 2024-09-13T10:11:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.770 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 70 2024-09-13T10:11:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:11:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:11:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381723+0000", 2024-09-13T10:11:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.540000000000001e-07 2024-09-13T10:11:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381731+0000", 2024-09-13T10:11:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.486e-06 2024-09-13T10:11:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381742+0000", 2024-09-13T10:11:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1269e-05 2024-09-13T10:11:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381750+0000", 2024-09-13T10:11:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8830000000000006e-06 2024-09-13T10:11:47.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401255+0000", 2024-09-13T10:11:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019505110999999999 2024-09-13T10:11:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401388+0000", 2024-09-13T10:11:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013253200000000001 2024-09-13T10:11:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:47.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401609+0000", 2024-09-13T10:11:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000220981 2024-09-13T10:11:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407538+0000", 2024-09-13T10:11:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059291220000000002 2024-09-13T10:11:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407626+0000", 2024-09-13T10:11:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8361000000000003e-05 2024-09-13T10:11:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416154+0000", 2024-09-13T10:11:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0085281000000000003 2024-09-13T10:11:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416234+0000", 2024-09-13T10:11:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9594999999999999e-05 2024-09-13T10:11:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416347+0000", 2024-09-13T10:11:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113063 2024-09-13T10:11:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:71 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:11:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 77.385568059999997, 2024-09-13T10:11:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035004, 2024-09-13T10:11:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:47.777 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 71 2024-09-13T10:11:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:11:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:11:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381764+0000", 2024-09-13T10:11:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4600000000000004e-07 2024-09-13T10:11:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381769+0000", 2024-09-13T10:11:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.48e-06 2024-09-13T10:11:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381780+0000", 2024-09-13T10:11:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0893000000000001e-05 2024-09-13T10:11:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381788+0000", 2024-09-13T10:11:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6389999999999992e-06 2024-09-13T10:11:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407896+0000", 2024-09-13T10:11:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.026107807 2024-09-13T10:11:47.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.408029+0000", 2024-09-13T10:11:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013383899999999999 2024-09-13T10:11:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.408250+0000", 2024-09-13T10:11:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022105399999999999 2024-09-13T10:11:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416130+0000", 2024-09-13T10:11:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0078798500000000007 2024-09-13T10:11:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416488+0000", 2024-09-13T10:11:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00035819299999999998 2024-09-13T10:11:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416595+0000", 2024-09-13T10:11:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001064 2024-09-13T10:11:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416662+0000", 2024-09-13T10:11:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6775999999999999e-05 2024-09-13T10:11:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416767+0000", 2024-09-13T10:11:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000105423 2024-09-13T10:11:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.783 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:72 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:11:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 77.385531025999995, 2024-09-13T10:11:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039399762999999997, 2024-09-13T10:11:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 72 2024-09-13T10:11:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.784 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:11:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:11:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381801+0000", 2024-09-13T10:11:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7700000000000004e-07 2024-09-13T10:11:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381898+0000", 2024-09-13T10:11:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7066999999999997e-05 2024-09-13T10:11:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381908+0000", 2024-09-13T10:11:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0336000000000001e-05 2024-09-13T10:11:47.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381917+0000", 2024-09-13T10:11:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5169999999999999e-06 2024-09-13T10:11:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.411900+0000", 2024-09-13T10:11:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029983381999999999 2024-09-13T10:11:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.787 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.412028+0000", 2024-09-13T10:11:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127916 2024-09-13T10:11:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.412248+0000", 2024-09-13T10:11:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022014699999999999 2024-09-13T10:11:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416576+0000", 2024-09-13T10:11:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004327549 2024-09-13T10:11:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416860+0000", 2024-09-13T10:11:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028391000000000001 2024-09-13T10:11:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421014+0000", 2024-09-13T10:11:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041539330000000003 2024-09-13T10:11:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421093+0000", 2024-09-13T10:11:47.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9097999999999994e-05 2024-09-13T10:11:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421200+0000", 2024-09-13T10:11:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000107231 2024-09-13T10:11:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:73 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:47.790 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:11:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 77.385397314000002, 2024-09-13T10:11:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044034528000000003, 2024-09-13T10:11:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 73 2024-09-13T10:11:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.791 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:11:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:11:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381935+0000", 2024-09-13T10:11:47.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0999999999999997e-07 2024-09-13T10:11:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381940+0000", 2024-09-13T10:11:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7030000000000003e-06 2024-09-13T10:11:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381949+0000", 2024-09-13T10:11:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5359999999999992e-06 2024-09-13T10:11:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381961+0000", 2024-09-13T10:11:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2539e-05 2024-09-13T10:11:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416942+0000", 2024-09-13T10:11:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034980270000000001 2024-09-13T10:11:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.417074+0000", 2024-09-13T10:11:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013246199999999999 2024-09-13T10:11:47.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.417298+0000", 2024-09-13T10:11:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000223823 2024-09-13T10:11:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425459+0000", 2024-09-13T10:11:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081613750000000002 2024-09-13T10:11:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425575+0000", 2024-09-13T10:11:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115513 2024-09-13T10:11:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425774+0000", 2024-09-13T10:11:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019959399999999999 2024-09-13T10:11:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425854+0000", 2024-09-13T10:11:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9532999999999999e-05 2024-09-13T10:11:47.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425968+0000", 2024-09-13T10:11:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011437 2024-09-13T10:11:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.797 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:74 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:11:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 77.38535444, 2024-09-13T10:11:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052701496, 2024-09-13T10:11:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 74 2024-09-13T10:11:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.798 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:11:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:11:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:11:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7999999999999995e-07 2024-09-13T10:11:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381983+0000", 2024-09-13T10:11:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4430000000000002e-06 2024-09-13T10:11:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381991+0000", 2024-09-13T10:11:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8960000000000003e-06 2024-09-13T10:11:47.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381998+0000", 2024-09-13T10:11:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7729999999999999e-06 2024-09-13T10:11:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421291+0000", 2024-09-13T10:11:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039292753 2024-09-13T10:11:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421424+0000", 2024-09-13T10:11:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013291900000000001 2024-09-13T10:11:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421646+0000", 2024-09-13T10:11:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000221843 2024-09-13T10:11:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425667+0000", 2024-09-13T10:11:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040206779999999998 2024-09-13T10:11:47.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425755+0000", 2024-09-13T10:11:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8848e-05 2024-09-13T10:11:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434472+0000", 2024-09-13T10:11:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0087164879999999997 2024-09-13T10:11:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434544+0000", 2024-09-13T10:11:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2273000000000004e-05 2024-09-13T10:11:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434678+0000", 2024-09-13T10:11:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013390200000000001 2024-09-13T10:11:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:75 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:47.805 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:11:47.805 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 77.385317575000002, 2024-09-13T10:11:47.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052864585999999998, 2024-09-13T10:11:47.805 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.805 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.805 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.805 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.805 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:47.805 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:47.805 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 75 2024-09-13T10:11:47.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.805 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:11:47.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:11:47.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:11:47.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8299999999999996e-07 2024-09-13T10:11:47.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382120+0000", 2024-09-13T10:11:47.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010627400000000001 2024-09-13T10:11:47.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382131+0000", 2024-09-13T10:11:47.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0796e-05 2024-09-13T10:11:47.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382140+0000", 2024-09-13T10:11:47.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5560000000000008e-06 2024-09-13T10:11:47.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426014+0000", 2024-09-13T10:11:47.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.043873843000000003 2024-09-13T10:11:47.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426205+0000", 2024-09-13T10:11:47.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019154600000000001 2024-09-13T10:11:47.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:47.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426429+0000", 2024-09-13T10:11:47.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022344699999999999 2024-09-13T10:11:47.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434352+0000", 2024-09-13T10:11:47.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007923355 2024-09-13T10:11:47.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434452+0000", 2024-09-13T10:11:47.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9961000000000001e-05 2024-09-13T10:11:47.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:47.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434700+0000", 2024-09-13T10:11:47.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024790899999999999 2024-09-13T10:11:47.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434774+0000", 2024-09-13T10:11:47.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4457999999999998e-05 2024-09-13T10:11:47.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.811 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434878+0000", 2024-09-13T10:11:47.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010375800000000001 2024-09-13T10:11:47.811 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.811 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.811 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.812 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:76 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:47.812 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:11:47.812 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 77.385150594999999, 2024-09-13T10:11:47.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053104138000000002, 2024-09-13T10:11:47.812 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.812 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.812 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.812 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.812 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:47.812 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:47.813 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 76 2024-09-13T10:11:47.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.813 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.813 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:11:47.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.813 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:11:47.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.814 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:11:47.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4600000000000003e-07 2024-09-13T10:11:47.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.814 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382188+0000", 2024-09-13T10:11:47.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4869999999999999e-06 2024-09-13T10:11:47.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382197+0000", 2024-09-13T10:11:47.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5519999999999993e-06 2024-09-13T10:11:47.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382207+0000", 2024-09-13T10:11:47.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7240000000000004e-06 2024-09-13T10:11:47.816 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.816 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.816 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.816 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430285+0000", 2024-09-13T10:11:47.816 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.048078093000000002 2024-09-13T10:11:47.816 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.816 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.816 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.816 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430429+0000", 2024-09-13T10:11:47.816 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000143981 2024-09-13T10:11:47.816 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.816 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.817 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:47.817 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430717+0000", 2024-09-13T10:11:47.817 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028802 2024-09-13T10:11:47.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.817 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.817 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.817 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435015+0000", 2024-09-13T10:11:47.817 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042973150000000003 2024-09-13T10:11:47.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.817 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.817 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.817 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435083+0000", 2024-09-13T10:11:47.818 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8448999999999995e-05 2024-09-13T10:11:47.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.818 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.818 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:47.818 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435109+0000", 2024-09-13T10:11:47.818 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6064999999999999e-05 2024-09-13T10:11:47.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.818 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.818 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.818 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435176+0000", 2024-09-13T10:11:47.818 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7362999999999999e-05 2024-09-13T10:11:47.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.818 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.819 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.819 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435285+0000", 2024-09-13T10:11:47.819 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108243 2024-09-13T10:11:47.819 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.819 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.819 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.819 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.819 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:96 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:47.819 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:11:47.819 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 77.243519164999995, 2024-09-13T10:11:47.819 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032701419000000002, 2024-09-13T10:11:47.819 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.819 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.820 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.820 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.820 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:47.820 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:47.820 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 96 2024-09-13T10:11:47.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.820 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.820 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.820 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.820 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:11:47.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.821 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.821 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:11:47.821 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.821 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.821 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523813+0000", 2024-09-13T10:11:47.821 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.364e-06 2024-09-13T10:11:47.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.821 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.821 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523913+0000", 2024-09-13T10:11:47.822 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9827000000000005e-05 2024-09-13T10:11:47.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.822 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.822 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523927+0000", 2024-09-13T10:11:47.822 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4029000000000001e-05 2024-09-13T10:11:47.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.822 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.822 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523936+0000", 2024-09-13T10:11:47.822 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1030000000000008e-06 2024-09-13T10:11:47.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.823 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.544707+0000", 2024-09-13T10:11:47.823 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0207706 2024-09-13T10:11:47.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.823 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.544834+0000", 2024-09-13T10:11:47.823 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012683200000000001 2024-09-13T10:11:47.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:47.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.545057+0000", 2024-09-13T10:11:47.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022338100000000001 2024-09-13T10:11:47.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.824 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.824 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549279+0000", 2024-09-13T10:11:47.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042223080000000001 2024-09-13T10:11:47.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.824 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.824 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549556+0000", 2024-09-13T10:11:47.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000276307 2024-09-13T10:11:47.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.825 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:47.825 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556364+0000", 2024-09-13T10:11:47.825 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0068082510000000004 2024-09-13T10:11:47.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.825 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.825 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556445+0000", 2024-09-13T10:11:47.825 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1081000000000005e-05 2024-09-13T10:11:47.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.825 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.826 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556513+0000", 2024-09-13T10:11:47.826 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8336000000000004e-05 2024-09-13T10:11:47.826 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.826 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.826 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.826 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:97 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:47.826 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:11:47.826 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 77.243376280000007, 2024-09-13T10:11:47.826 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035410807000000002, 2024-09-13T10:11:47.826 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.826 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.827 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.827 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.827 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:47.827 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:47.827 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 97 2024-09-13T10:11:47.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.827 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.827 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.827 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:11:47.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.828 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.828 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:11:47.828 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.828 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.828 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523956+0000", 2024-09-13T10:11:47.828 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8599999999999997e-07 2024-09-13T10:11:47.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.828 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.828 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523962+0000", 2024-09-13T10:11:47.829 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1240000000000002e-06 2024-09-13T10:11:47.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.829 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.829 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523971+0000", 2024-09-13T10:11:47.829 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7320000000000004e-06 2024-09-13T10:11:47.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.829 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.829 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523978+0000", 2024-09-13T10:11:47.829 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0899999999999999e-06 2024-09-13T10:11:47.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.830 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.830 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549637+0000", 2024-09-13T10:11:47.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025658876000000001 2024-09-13T10:11:47.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.830 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.830 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549757+0000", 2024-09-13T10:11:47.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000120837 2024-09-13T10:11:47.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.830 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:47.830 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549971+0000", 2024-09-13T10:11:47.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021356000000000001 2024-09-13T10:11:47.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.831 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.831 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556335+0000", 2024-09-13T10:11:47.831 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0063639329999999996 2024-09-13T10:11:47.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.831 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.831 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556725+0000", 2024-09-13T10:11:47.831 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000390221 2024-09-13T10:11:47.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.831 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:47.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559219+0000", 2024-09-13T10:11:47.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0024943500000000002 2024-09-13T10:11:47.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559296+0000", 2024-09-13T10:11:47.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7065999999999997e-05 2024-09-13T10:11:47.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559366+0000", 2024-09-13T10:11:47.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9132000000000002e-05 2024-09-13T10:11:47.833 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.833 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.833 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.833 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.833 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:98 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:47.833 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:11:47.833 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 77.243341672, 2024-09-13T10:11:47.833 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035839604999999997, 2024-09-13T10:11:47.833 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.833 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.833 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.834 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.834 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:47.834 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:47.834 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 98 2024-09-13T10:11:47.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.834 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.834 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.834 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.834 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:11:47.834 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.835 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.835 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:11:47.835 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.835 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.835 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523990+0000", 2024-09-13T10:11:47.835 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6300000000000005e-07 2024-09-13T10:11:47.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.836 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.836 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523995+0000", 2024-09-13T10:11:47.836 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2669999999999997e-06 2024-09-13T10:11:47.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.836 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.836 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.836 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524007+0000", 2024-09-13T10:11:47.836 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2002e-05 2024-09-13T10:11:47.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.836 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.836 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524015+0000", 2024-09-13T10:11:47.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4869999999999996e-06 2024-09-13T10:11:47.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.837 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.553854+0000", 2024-09-13T10:11:47.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029839633000000001 2024-09-13T10:11:47.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.837 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.553985+0000", 2024-09-13T10:11:47.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013032899999999999 2024-09-13T10:11:47.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.838 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:47.838 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.554212+0000", 2024-09-13T10:11:47.838 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022723400000000001 2024-09-13T10:11:47.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.838 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.838 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559523+0000", 2024-09-13T10:11:47.838 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053113509999999997 2024-09-13T10:11:47.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.839 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.839 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559662+0000", 2024-09-13T10:11:47.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013833500000000001 2024-09-13T10:11:47.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.839 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.839 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:47.839 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559698+0000", 2024-09-13T10:11:47.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5982000000000002e-05 2024-09-13T10:11:47.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.839 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.839 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.839 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559764+0000", 2024-09-13T10:11:47.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6019000000000006e-05 2024-09-13T10:11:47.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.840 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.840 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559829+0000", 2024-09-13T10:11:47.840 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5302999999999995e-05 2024-09-13T10:11:47.840 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.840 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.840 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.840 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:99 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:47.840 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:11:47.841 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 77.243303978, 2024-09-13T10:11:47.841 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.037637084000000001, 2024-09-13T10:11:47.841 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.841 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.841 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.841 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.841 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:47.841 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:47.841 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 99 2024-09-13T10:11:47.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.841 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.842 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.842 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.842 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:11:47.842 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.842 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.842 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.842 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:11:47.842 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.842 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.842 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.842 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524028+0000", 2024-09-13T10:11:47.843 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7999999999999995e-07 2024-09-13T10:11:47.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.843 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.843 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524109+0000", 2024-09-13T10:11:47.843 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0834999999999994e-05 2024-09-13T10:11:47.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.843 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.843 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524119+0000", 2024-09-13T10:11:47.843 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0149999999999999e-05 2024-09-13T10:11:47.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524127+0000", 2024-09-13T10:11:47.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8660000000000006e-06 2024-09-13T10:11:47.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556868+0000", 2024-09-13T10:11:47.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.03274146 2024-09-13T10:11:47.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.557001+0000", 2024-09-13T10:11:47.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000133249 2024-09-13T10:11:47.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.845 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:47.845 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.557224+0000", 2024-09-13T10:11:47.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000222914 2024-09-13T10:11:47.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.845 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.845 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561387+0000", 2024-09-13T10:11:47.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041627019999999999 2024-09-13T10:11:47.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.846 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:47.846 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561419+0000", 2024-09-13T10:11:47.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1924999999999997e-05 2024-09-13T10:11:47.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.846 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.846 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.846 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561476+0000", 2024-09-13T10:11:47.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7346999999999998e-05 2024-09-13T10:11:47.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.846 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.846 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.846 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561552+0000", 2024-09-13T10:11:47.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5847999999999996e-05 2024-09-13T10:11:47.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.847 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.847 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561664+0000", 2024-09-13T10:11:47.847 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112108 2024-09-13T10:11:47.847 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.847 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.847 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.847 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:100 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:47.848 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:11:47.848 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 77.243127827999999, 2024-09-13T10:11:47.848 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.038825510000000001, 2024-09-13T10:11:47.848 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.848 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.848 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.848 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.848 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:47.848 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:47.848 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 100 2024-09-13T10:11:47.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.848 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.849 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.849 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:11:47.849 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.849 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.849 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:11:47.849 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.849 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.850 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524204+0000", 2024-09-13T10:11:47.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.54e-07 2024-09-13T10:11:47.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.850 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.850 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524211+0000", 2024-09-13T10:11:47.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6919999999999996e-06 2024-09-13T10:11:47.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.850 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.850 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524221+0000", 2024-09-13T10:11:47.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0223e-05 2024-09-13T10:11:47.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.851 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524229+0000", 2024-09-13T10:11:47.851 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9859999999999994e-06 2024-09-13T10:11:47.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.851 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559920+0000", 2024-09-13T10:11:47.851 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035690854000000001 2024-09-13T10:11:47.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.851 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.560052+0000", 2024-09-13T10:11:47.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013199899999999999 2024-09-13T10:11:47.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.852 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.852 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:47.852 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.560356+0000", 2024-09-13T10:11:47.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030450200000000001 2024-09-13T10:11:47.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.852 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.852 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:47.852 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562115+0000", 2024-09-13T10:11:47.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0017586430000000001 2024-09-13T10:11:47.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562512+0000", 2024-09-13T10:11:47.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00039728099999999998 2024-09-13T10:11:47.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562828+0000", 2024-09-13T10:11:47.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00031528799999999999 2024-09-13T10:11:47.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562913+0000", 2024-09-13T10:11:47.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5628000000000001e-05 2024-09-13T10:11:47.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.854 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.854 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.563029+0000", 2024-09-13T10:11:47.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011556 2024-09-13T10:11:47.854 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.854 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.854 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.854 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:122 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:47.854 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:11:47.855 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 77.126443347000006, 2024-09-13T10:11:47.856 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.027900708, 2024-09-13T10:11:47.856 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.856 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.856 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.856 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.856 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:47.856 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:47.856 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 122 2024-09-13T10:11:47.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.857 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:11:47.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:11:47.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640889+0000", 2024-09-13T10:11:47.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4139999999999999e-06 2024-09-13T10:11:47.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640900+0000", 2024-09-13T10:11:47.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0559e-05 2024-09-13T10:11:47.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640916+0000", 2024-09-13T10:11:47.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6399e-05 2024-09-13T10:11:47.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.859 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640931+0000", 2024-09-13T10:11:47.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4659e-05 2024-09-13T10:11:47.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.859 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663602+0000", 2024-09-13T10:11:47.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.022671674999999999 2024-09-13T10:11:47.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663734+0000", 2024-09-13T10:11:47.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013189899999999999 2024-09-13T10:11:47.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:47.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663946+0000", 2024-09-13T10:11:47.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021198699999999999 2024-09-13T10:11:47.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668523+0000", 2024-09-13T10:11:47.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045765789999999999 2024-09-13T10:11:47.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668638+0000", 2024-09-13T10:11:47.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504 2024-09-13T10:11:47.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:47.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668657+0000", 2024-09-13T10:11:47.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9298999999999999e-05 2024-09-13T10:11:47.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.862 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668722+0000", 2024-09-13T10:11:47.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5126e-05 2024-09-13T10:11:47.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.862 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.862 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668788+0000", 2024-09-13T10:11:47.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6072e-05 2024-09-13T10:11:47.862 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.862 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.862 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.862 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:123 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:47.863 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:11:47.863 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 77.126377469999994, 2024-09-13T10:11:47.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028273795000000001, 2024-09-13T10:11:47.863 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.863 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.863 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.863 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.863 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:47.863 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:47.863 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 123 2024-09-13T10:11:47.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.863 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:11:47.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:11:47.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640955+0000", 2024-09-13T10:11:47.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.356e-06 2024-09-13T10:11:47.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.865 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641083+0000", 2024-09-13T10:11:47.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012763600000000001 2024-09-13T10:11:47.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.865 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641102+0000", 2024-09-13T10:11:47.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9055e-05 2024-09-13T10:11:47.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641117+0000", 2024-09-13T10:11:47.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5665999999999999e-05 2024-09-13T10:11:47.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.665747+0000", 2024-09-13T10:11:47.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.024629796999999998 2024-09-13T10:11:47.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.665876+0000", 2024-09-13T10:11:47.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012925600000000001 2024-09-13T10:11:47.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:47.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.666105+0000", 2024-09-13T10:11:47.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000228543 2024-09-13T10:11:47.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668924+0000", 2024-09-13T10:11:47.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028194309999999998 2024-09-13T10:11:47.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669070+0000", 2024-09-13T10:11:47.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014560199999999999 2024-09-13T10:11:47.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:47.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669104+0000", 2024-09-13T10:11:47.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3654e-05 2024-09-13T10:11:47.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669173+0000", 2024-09-13T10:11:47.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9819000000000004e-05 2024-09-13T10:11:47.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.869 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669227+0000", 2024-09-13T10:11:47.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3980000000000002e-05 2024-09-13T10:11:47.869 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.869 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.869 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.869 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:124 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:11:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:11:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 77.126184429000006, 2024-09-13T10:11:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030852801999999999, 2024-09-13T10:11:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:11:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:11:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 124 2024-09-13T10:11:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.870 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:11:47.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:11:47.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641148+0000", 2024-09-13T10:11:47.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4529999999999999e-06 2024-09-13T10:11:47.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641158+0000", 2024-09-13T10:11:47.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0003999999999999e-05 2024-09-13T10:11:47.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641199+0000", 2024-09-13T10:11:47.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0426999999999998e-05 2024-09-13T10:11:47.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641214+0000", 2024-09-13T10:11:47.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5716e-05 2024-09-13T10:11:47.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669267+0000", 2024-09-13T10:11:47.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028053112000000002 2024-09-13T10:11:47.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669394+0000", 2024-09-13T10:11:47.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012656000000000001 2024-09-13T10:11:47.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:11:47.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669614+0000", 2024-09-13T10:11:47.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021970400000000001 2024-09-13T10:11:47.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.875 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671267+0000", 2024-09-13T10:11:47.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001652976 2024-09-13T10:11:47.875 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.875 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.875 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.875 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671383+0000", 2024-09-13T10:11:47.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000116113 2024-09-13T10:11:47.875 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.875 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.875 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:47.875 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671828+0000", 2024-09-13T10:11:47.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000444871 2024-09-13T10:11:47.875 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671935+0000", 2024-09-13T10:11:47.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010703 2024-09-13T10:11:47.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671999+0000", 2024-09-13T10:11:47.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4836000000000001e-05 2024-09-13T10:11:47.876 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.876 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.876 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.877 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:201 3.3 e38/30 3:d4fb61be:::smithi19530714-2 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:47.877 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:11:47.877 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 56.753733535999999, 2024-09-13T10:11:47.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032127035999999998, 2024-09-13T10:11:47.877 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.877 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.877 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.877 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:11:47.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:11:47.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:11:47.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2600000000000001e-07 2024-09-13T10:11:47.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030449+0000", 2024-09-13T10:11:47.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016850477999999999 2024-09-13T10:11:47.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030499+0000", 2024-09-13T10:11:47.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0212e-05 2024-09-13T10:11:47.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030524+0000", 2024-09-13T10:11:47.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5222e-05 2024-09-13T10:11:47.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030739+0000", 2024-09-13T10:11:47.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000214396 2024-09-13T10:11:47.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030818+0000", 2024-09-13T10:11:47.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8941000000000002e-05 2024-09-13T10:11:47.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.045512+0000", 2024-09-13T10:11:47.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014694248 2024-09-13T10:11:47.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.045725+0000", 2024-09-13T10:11:47.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021261300000000001 2024-09-13T10:11:47.881 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.881 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.881 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.881 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:226 3.3 e38/30 3:fe5bfa3a:::smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'5, mlcod=38'3)", 2024-09-13T10:11:47.881 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:11:47.881 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 56.686873292999998, 2024-09-13T10:11:47.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030943633000000002, 2024-09-13T10:11:47.882 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.882 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.882 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.882 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:11:47.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:11:47.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080461+0000", 2024-09-13T10:11:47.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7049999999999999e-06 2024-09-13T10:11:47.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097571+0000", 2024-09-13T10:11:47.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017110658000000001 2024-09-13T10:11:47.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097615+0000", 2024-09-13T10:11:47.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3404e-05 2024-09-13T10:11:47.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097635+0000", 2024-09-13T10:11:47.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0279e-05 2024-09-13T10:11:47.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097995+0000", 2024-09-13T10:11:47.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00036038699999999999 2024-09-13T10:11:47.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.098029+0000", 2024-09-13T10:11:47.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4162000000000003e-05 2024-09-13T10:11:47.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111330+0000", 2024-09-13T10:11:47.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013300268000000001 2024-09-13T10:11:47.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111401+0000", 2024-09-13T10:11:47.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1769999999999999e-05 2024-09-13T10:11:47.886 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.886 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.886 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.886 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:237 3.3 e38/30 3:de40747e:::smithi19530714-38 low_tier:head v 38'7, mlcod=38'3)", 2024-09-13T10:11:47.886 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:11:47.886 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 56.652640501, 2024-09-13T10:11:47.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.037747504000000001, 2024-09-13T10:11:47.887 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.887 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.887 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.887 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:11:47.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:11:47.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:11:47.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7299999999999995e-07 2024-09-13T10:11:47.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145311+0000", 2024-09-13T10:11:47.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030619621 2024-09-13T10:11:47.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145336+0000", 2024-09-13T10:11:47.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5009999999999999e-05 2024-09-13T10:11:47.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145349+0000", 2024-09-13T10:11:47.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2724999999999999e-05 2024-09-13T10:11:47.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145445+0000", 2024-09-13T10:11:47.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6108999999999997e-05 2024-09-13T10:11:47.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145470+0000", 2024-09-13T10:11:47.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5514e-05 2024-09-13T10:11:47.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152389+0000", 2024-09-13T10:11:47.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069190739999999999 2024-09-13T10:11:47.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152438+0000", 2024-09-13T10:11:47.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8778e-05 2024-09-13T10:11:47.891 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.891 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.891 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.891 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:286 4.a 4:5ad8883b:::smithi19530714-37:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:47.891 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:11:47.891 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 56.519043873999998, 2024-09-13T10:11:47.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032980435000000002, 2024-09-13T10:11:47.891 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.891 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.892 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:47.892 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.892 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:47.892 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:47.892 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 286 2024-09-13T10:11:47.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.892 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:11:47.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:11:47.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248290+0000", 2024-09-13T10:11:47.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0170000000000001e-06 2024-09-13T10:11:47.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248302+0000", 2024-09-13T10:11:47.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1602e-05 2024-09-13T10:11:47.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248328+0000", 2024-09-13T10:11:47.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.622e-05 2024-09-13T10:11:47.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248344+0000", 2024-09-13T10:11:47.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6192999999999999e-05 2024-09-13T10:11:47.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248466+0000", 2024-09-13T10:11:47.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012186899999999999 2024-09-13T10:11:47.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248602+0000", 2024-09-13T10:11:47.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013623500000000001 2024-09-13T10:11:47.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 6,13", 2024-09-13T10:11:47.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274091+0000", 2024-09-13T10:11:47.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025489050999999999 2024-09-13T10:11:47.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275976+0000", 2024-09-13T10:11:47.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018847289999999999 2024-09-13T10:11:47.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:47.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276801+0000", 2024-09-13T10:11:47.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00082477899999999999 2024-09-13T10:11:47.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:47.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281064+0000", 2024-09-13T10:11:47.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042625950000000001 2024-09-13T10:11:47.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:47.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281173+0000", 2024-09-13T10:11:47.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000109391 2024-09-13T10:11:47.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281268+0000", 2024-09-13T10:11:47.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4753999999999994e-05 2024-09-13T10:11:47.897 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.898 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.898 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.898 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1631 4.a 4:5ad8883b:::smithi19530714-37:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:47.898 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:06.099753+0000", 2024-09-13T10:11:47.898 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 41.667578300999999, 2024-09-13T10:11:47.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033125612999999998, 2024-09-13T10:11:47.898 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:47.898 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:47.898 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:11:47.898 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:47.899 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:47.899 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:47.899 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1631 2024-09-13T10:11:47.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.899 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:47.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:47.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099753+0000", 2024-09-13T10:11:47.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.899 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:47.899 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099753+0000", 2024-09-13T10:11:47.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:47.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:47.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099754+0000", 2024-09-13T10:11:47.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2699999999999999e-06 2024-09-13T10:11:47.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.900 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:47.900 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099758+0000", 2024-09-13T10:11:47.900 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6540000000000001e-06 2024-09-13T10:11:47.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:47.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099768+0000", 2024-09-13T10:11:47.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0412e-05 2024-09-13T10:11:47.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:47.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099777+0000", 2024-09-13T10:11:47.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4600000000000003e-06 2024-09-13T10:11:47.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:47.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099830+0000", 2024-09-13T10:11:47.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3192000000000002e-05 2024-09-13T10:11:47.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:47.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099889+0000", 2024-09-13T10:11:47.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9576e-05 2024-09-13T10:11:47.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:47.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:47.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:47.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.132878+0000", 2024-09-13T10:11:47.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032989048999999999 2024-09-13T10:11:47.903 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.903 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.903 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.903 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:47.903 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:47.903 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:47.904 INFO:tasks.rados.rados.0.smithi195.stdout:1500: finishing set_chunk to oid 19 2024-09-13T10:11:47.904 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 19 set_chunk returned error code -95 offset: 308736 length: 2264064 2024-09-13T10:11:47.905 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 0 (ObjNum 160 snap 0 seq_num 160) dirty exists 2024-09-13T10:11:47.905 INFO:tasks.rados.rados.0.smithi195.stdout:1501: expect (ObjNum 262 snap 0 seq_num 262) 2024-09-13T10:11:47.905 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 91662 2024-09-13T10:11:47.905 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:47.901+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:47.906 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-13T10:11:47.934 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1092935 2024-09-13T10:11:47.966 INFO:tasks.rados.rados.0.smithi195.stdout:1502: expect (ObjNum 243 snap 0 seq_num 243) 2024-09-13T10:11:48.005 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:47.999+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:48.061 INFO:tasks.rados.rados.0.smithi195.stdout:1503: got (0) Success 2024-09-13T10:11:48.061 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 0 (ObjNum 296 snap 0 seq_num 296) dirty exists 2024-09-13T10:11:48.062 INFO:tasks.rados.rados.0.smithi195.stdout:1497: done (6 left) 2024-09-13T10:11:48.062 INFO:tasks.rados.rados.0.smithi195.stdout:1498: done (5 left) 2024-09-13T10:11:48.062 INFO:tasks.rados.rados.0.smithi195.stdout:1499: done (4 left) 2024-09-13T10:11:48.062 INFO:tasks.rados.rados.0.smithi195.stdout:1500: done (3 left) 2024-09-13T10:11:48.062 INFO:tasks.rados.rados.0.smithi195.stdout:1501: done (2 left) 2024-09-13T10:11:48.062 INFO:tasks.rados.rados.0.smithi195.stdout:1502: done (1 left) 2024-09-13T10:11:48.062 INFO:tasks.rados.rados.0.smithi195.stdout:1503: done (0 left) 2024-09-13T10:11:48.062 INFO:tasks.rados.rados.0.smithi195.stdout:1504: set_chunk oid 18 target oid 2024-09-13T10:11:48.080 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:48.080 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:48.080 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:11:48.080 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:48.083 INFO:tasks.rados.rados.0.smithi195.stdout:1504: set_chunk oid 18 offset: 289792 length: 1337856 target oid 4ebe85e7e0590db5d8edc57651fe6bbf73cc4452255d346ce0bbc168a62533d5 offset: 0 snap -1 2024-09-13T10:11:48.084 INFO:tasks.rados.rados.0.smithi195.stdout:1505: delete oid 15 current snap is 0 2024-09-13T10:11:48.084 INFO:tasks.rados.rados.0.smithi195.stdout:1504: finishing set_chunk to oid 18 2024-09-13T10:11:48.085 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 18 set_chunk returned error code -95 offset: 289792 length: 1337856 2024-09-13T10:11:48.085 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 0 (ObjNum 277 snap 0 seq_num 277) dirty exists 2024-09-13T10:11:48.089 INFO:tasks.rados.rados.0.smithi195.stdout:1504: done (1 left) 2024-09-13T10:11:48.089 INFO:tasks.rados.rados.0.smithi195.stdout:1505: done (0 left) 2024-09-13T10:11:48.089 INFO:tasks.rados.rados.0.smithi195.stdout:1506: set_chunk oid 5 target oid 2024-09-13T10:11:48.096 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-13T10:11:48.116 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T10:11:48.139 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:48.135+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:48.145 INFO:tasks.rados.rados.0.smithi195.stdout:1506: set_chunk oid 5 offset: 206336 length: 3552768 target oid 4c246a1df6a0d5acaf0a3a85249ff707b21f90a5f3db763dc6d544e5b7cbec73 offset: 0 snap -1 2024-09-13T10:11:48.145 INFO:tasks.rados.rados.0.smithi195.stdout:1507: read oid 30 snap -1 2024-09-13T10:11:48.145 INFO:tasks.rados.rados.0.smithi195.stdout:1507: expect deleted 2024-09-13T10:11:48.145 INFO:tasks.rados.rados.0.smithi195.stdout:1508: read oid 9 snap -1 2024-09-13T10:11:48.145 INFO:tasks.rados.rados.0.smithi195.stdout:1508: expect (ObjNum 259 snap 0 seq_num 259) 2024-09-13T10:11:48.145 INFO:tasks.rados.rados.0.smithi195.stdout:1509: read oid 22 snap -1 2024-09-13T10:11:48.145 INFO:tasks.rados.rados.0.smithi195.stdout:1509: expect (ObjNum 139 snap 0 seq_num 139) 2024-09-13T10:11:48.145 INFO:tasks.rados.rados.0.smithi195.stdout:1510: read oid 18 snap -1 2024-09-13T10:11:48.145 INFO:tasks.rados.rados.0.smithi195.stdout:1510: expect (ObjNum 277 snap 0 seq_num 277) 2024-09-13T10:11:48.146 INFO:tasks.rados.rados.0.smithi195.stdout:1511: delete oid 11 current snap is 0 2024-09-13T10:11:48.146 INFO:tasks.rados.rados.0.smithi195.stdout:1506: finishing set_chunk to oid 5 2024-09-13T10:11:48.146 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 5 set_chunk returned error code -95 offset: 206336 length: 3552768 2024-09-13T10:11:48.146 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 0 (ObjNum 231 snap 0 seq_num 231) dirty exists 2024-09-13T10:11:48.149 INFO:tasks.rados.rados.0.smithi195.stdout:1506: done (5 left) 2024-09-13T10:11:48.149 INFO:tasks.rados.rados.0.smithi195.stdout:1507: done (4 left) 2024-09-13T10:11:48.149 INFO:tasks.rados.rados.0.smithi195.stdout:1511: done (3 left) 2024-09-13T10:11:48.149 INFO:tasks.rados.rados.0.smithi195.stdout:1512: tier_flush oid 8 2024-09-13T10:11:48.149 INFO:tasks.rados.rados.0.smithi195.stdout:1512: tier_flush oid 8 snap -1 2024-09-13T10:11:48.149 INFO:tasks.rados.rados.0.smithi195.stdout:1513: read oid 33 snap -1 2024-09-13T10:11:48.149 INFO:tasks.rados.rados.0.smithi195.stdout:1513: expect (ObjNum 253 snap 0 seq_num 253) 2024-09-13T10:11:48.150 INFO:tasks.rados.rados.0.smithi195.stdout:1514: read oid 34 snap -1 2024-09-13T10:11:48.150 INFO:tasks.rados.rados.0.smithi195.stdout:1514: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:48.150 INFO:tasks.rados.rados.0.smithi195.stdout:1515: write oid 13 current snap is 0 2024-09-13T10:11:48.150 INFO:tasks.rados.rados.0.smithi195.stdout:1515: seq_num 308 ranges {776556=630812,1823748=652437,3108517=68039} 2024-09-13T10:11:48.175 INFO:tasks.rados.rados.0.smithi195.stdout:1515: writing smithi19530714-13 from 776556 to 1407368 tid 1 2024-09-13T10:11:48.191 INFO:tasks.rados.rados.0.smithi195.stdout:1515: writing smithi19530714-13 from 1823748 to 2476185 tid 2 2024-09-13T10:11:48.200 INFO:tasks.rados.rados.0.smithi195.stdout:1515: writing smithi19530714-13 from 3108517 to 3176556 tid 3 2024-09-13T10:11:48.200 INFO:tasks.rados.rados.0.smithi195.stdout:1516: read oid 36 snap -1 2024-09-13T10:11:48.200 INFO:tasks.rados.rados.0.smithi195.stdout:1516: expect (ObjNum 269 snap 0 seq_num 269) 2024-09-13T10:11:48.200 INFO:tasks.rados.rados.0.smithi195.stdout:1517: set_chunk oid 11 target oid 2024-09-13T10:11:48.200 INFO:tasks.rados.rados.0.smithi195.stdout:1517: set_chunk oid 11 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:48.200 INFO:tasks.rados.rados.0.smithi195.stdout:1518: set_chunk oid 31 target oid 2024-09-13T10:11:48.207 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:48.201+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:48.220 INFO:tasks.rados.rados.0.smithi195.stdout:1518: set_chunk oid 31 offset: 1251328 length: 952832 target oid 1b02344f4793b9b27f21bc4c19561a5a6e72c822831f537df06594ca042e0b06 offset: 0 snap -1 2024-09-13T10:11:48.220 INFO:tasks.rados.rados.0.smithi195.stdout:1519: write oid 30 current snap is 0 2024-09-13T10:11:48.220 INFO:tasks.rados.rados.0.smithi195.stdout:1519: seq_num 309 ranges {793363=706061,2282230=796095} 2024-09-13T10:11:48.235 INFO:tasks.rados.rados.0.smithi195.stdout:1519: writing smithi19530714-30 from 793363 to 1499424 tid 1 2024-09-13T10:11:48.254 INFO:tasks.rados.rados.0.smithi195.stdout:1519: writing smithi19530714-30 from 2282230 to 3078325 tid 2 2024-09-13T10:11:48.254 INFO:tasks.rados.rados.0.smithi195.stdout:1520: read oid 23 snap -1 2024-09-13T10:11:48.255 INFO:tasks.rados.rados.0.smithi195.stdout:1520: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:11:48.255 INFO:tasks.rados.rados.0.smithi195.stdout:1521: read oid 16 snap -1 2024-09-13T10:11:48.255 INFO:tasks.rados.rados.0.smithi195.stdout:1521: expect (ObjNum 275 snap 0 seq_num 275) 2024-09-13T10:11:48.255 INFO:tasks.rados.rados.0.smithi195.stdout:1522: read oid 38 snap -1 2024-09-13T10:11:48.255 INFO:tasks.rados.rados.0.smithi195.stdout:1522: expect (ObjNum 298 snap 0 seq_num 298) 2024-09-13T10:11:48.255 INFO:tasks.rados.rados.0.smithi195.stdout:1523: read oid 12 snap -1 2024-09-13T10:11:48.256 INFO:tasks.rados.rados.0.smithi195.stdout:1523: expect (ObjNum 287 snap 0 seq_num 287) 2024-09-13T10:11:48.256 INFO:tasks.rados.rados.0.smithi195.stdout:1512: got (2) No such file or directory 2024-09-13T10:11:48.256 INFO:tasks.rados.rados.0.smithi195.stdout:1512: got expected ENOENT (src dne) 2024-09-13T10:11:48.256 INFO:tasks.rados.rados.0.smithi195.stdout:1512: done (14 left) 2024-09-13T10:11:48.256 INFO:tasks.rados.rados.0.smithi195.stdout:1524: write oid 5 current snap is 0 2024-09-13T10:11:48.256 INFO:tasks.rados.rados.0.smithi195.stdout:1524: seq_num 310 ranges {410170=781310,1940712=656106,3163363=46807} 2024-09-13T10:11:48.261 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:11:48.261 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:11:48.262 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:48.265 INFO:tasks.rados.rados.0.smithi195.stdout:1524: writing smithi19530714-5 from 410170 to 1191480 tid 1 2024-09-13T10:11:48.279 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-13T10:11:48.279 INFO:tasks.rados.rados.0.smithi195.stdout:1524: writing smithi19530714-5 from 1940712 to 2596818 tid 2 2024-09-13T10:11:48.284 INFO:tasks.rados.rados.0.smithi195.stdout:1524: writing smithi19530714-5 from 3163363 to 3210170 tid 3 2024-09-13T10:11:48.284 INFO:tasks.rados.rados.0.smithi195.stdout:1525: read oid 42 snap -1 2024-09-13T10:11:48.284 INFO:tasks.rados.rados.0.smithi195.stdout:1525: expect (ObjNum 297 snap 0 seq_num 297) 2024-09-13T10:11:48.285 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:48.285 INFO:tasks.rados.rados.0.smithi195.stdout:1513: expect (ObjNum 253 snap 0 seq_num 253) 2024-09-13T10:11:48.309 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:48.303+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:48.345 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 210215 2024-09-13T10:11:48.376 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:48.396 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:48.397 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:11:48.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:11:48.397 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:11:48.397 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.397 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:53 4.c 4:3e6706e5:::smithi19530714-14:head [write 618491~694590,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.397 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:50.494212+0000", 2024-09-13T10:11:48.397 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.899463834999999, 2024-09-13T10:11:48.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0086614210000000007, 2024-09-13T10:11:48.397 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.397 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.397 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.397 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.398 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.398 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.398 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53 2024-09-13T10:11:48.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.398 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494212+0000", 2024-09-13T10:11:48.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494212+0000", 2024-09-13T10:11:48.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494213+0000", 2024-09-13T10:11:48.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.84e-06 2024-09-13T10:11:48.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494889+0000", 2024-09-13T10:11:48.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00067531799999999995 2024-09-13T10:11:48.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494907+0000", 2024-09-13T10:11:48.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7884999999999999e-05 2024-09-13T10:11:48.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494917+0000", 2024-09-13T10:11:48.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0043e-05 2024-09-13T10:11:48.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494971+0000", 2024-09-13T10:11:48.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4775000000000003e-05 2024-09-13T10:11:48.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.401 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.401 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.495081+0000", 2024-09-13T10:11:48.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110055 2024-09-13T10:11:48.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.401 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:48.401 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.495184+0000", 2024-09-13T10:11:48.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010221 2024-09-13T10:11:48.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.401 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.401 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.496618+0000", 2024-09-13T10:11:48.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014343590000000001 2024-09-13T10:11:48.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.498806+0000", 2024-09-13T10:11:48.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021880269999999999 2024-09-13T10:11:48.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.502814+0000", 2024-09-13T10:11:48.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040077890000000003 2024-09-13T10:11:48.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.502844+0000", 2024-09-13T10:11:48.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0462000000000001e-05 2024-09-13T10:11:48.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.502873+0000", 2024-09-13T10:11:48.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8657999999999999e-05 2024-09-13T10:11:48.403 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.403 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.403 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.403 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:161 4.d 4:b19b932a:::smithi19530714-41:head [write 672280~791725,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.404 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:50.852563+0000", 2024-09-13T10:11:48.404 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.541111868000002, 2024-09-13T10:11:48.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011425286999999999, 2024-09-13T10:11:48.404 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.404 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.404 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.404 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.404 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.404 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.404 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 161 2024-09-13T10:11:48.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.404 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.852563+0000", 2024-09-13T10:11:48.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.852563+0000", 2024-09-13T10:11:48.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.852567+0000", 2024-09-13T10:11:48.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1269999999999999e-06 2024-09-13T10:11:48.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853363+0000", 2024-09-13T10:11:48.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00079620899999999998 2024-09-13T10:11:48.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853394+0000", 2024-09-13T10:11:48.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1192000000000002e-05 2024-09-13T10:11:48.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853410+0000", 2024-09-13T10:11:48.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5896000000000002e-05 2024-09-13T10:11:48.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.407 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853574+0000", 2024-09-13T10:11:48.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000164358 2024-09-13T10:11:48.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.407 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853920+0000", 2024-09-13T10:11:48.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034612500000000002 2024-09-13T10:11:48.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:48.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.854305+0000", 2024-09-13T10:11:48.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038458900000000001 2024-09-13T10:11:48.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.858018+0000", 2024-09-13T10:11:48.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0037128619999999999 2024-09-13T10:11:48.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.858905+0000", 2024-09-13T10:11:48.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00088737899999999999 2024-09-13T10:11:48.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.863681+0000", 2024-09-13T10:11:48.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004775311 2024-09-13T10:11:48.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.863813+0000", 2024-09-13T10:11:48.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013254900000000001 2024-09-13T10:11:48.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.863989+0000", 2024-09-13T10:11:48.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017568999999999999 2024-09-13T10:11:48.410 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.410 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.410 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.410 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(osd.14.23:4 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.410 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:50.996997+0000", 2024-09-13T10:11:48.410 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.396678461999997, 2024-09-13T10:11:48.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0083262170000000003, 2024-09-13T10:11:48.410 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.410 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.410 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:11:48.410 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.996997+0000", 2024-09-13T10:11:48.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.996997+0000", 2024-09-13T10:11:48.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997000+0000", 2024-09-13T10:11:48.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3560000000000001e-06 2024-09-13T10:11:48.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997015+0000", 2024-09-13T10:11:48.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.523e-05 2024-09-13T10:11:48.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.412 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997048+0000", 2024-09-13T10:11:48.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2162000000000001e-05 2024-09-13T10:11:48.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.412 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997071+0000", 2024-09-13T10:11:48.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2918999999999999e-05 2024-09-13T10:11:48.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.001996+0000", 2024-09-13T10:11:48.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049256559999999996 2024-09-13T10:11:48.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002198+0000", 2024-09-13T10:11:48.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020196600000000001 2024-09-13T10:11:48.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.005323+0000", 2024-09-13T10:11:48.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0031249279999999999 2024-09-13T10:11:48.414 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.414 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.414 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.414 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:261 4.c 4:3d56d753:::smithi19530714-12:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.414 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.215641+0000", 2024-09-13T10:11:48.414 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.178034144000002, 2024-09-13T10:11:48.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010509116000000001, 2024-09-13T10:11:48.415 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.415 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.415 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.415 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.415 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.415 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.415 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 261 2024-09-13T10:11:48.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.415 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215641+0000", 2024-09-13T10:11:48.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215641+0000", 2024-09-13T10:11:48.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215646+0000", 2024-09-13T10:11:48.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9300000000000002e-06 2024-09-13T10:11:48.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215666+0000", 2024-09-13T10:11:48.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0095999999999999e-05 2024-09-13T10:11:48.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215713+0000", 2024-09-13T10:11:48.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7125e-05 2024-09-13T10:11:48.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215739+0000", 2024-09-13T10:11:48.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5488999999999999e-05 2024-09-13T10:11:48.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215918+0000", 2024-09-13T10:11:48.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017948399999999999 2024-09-13T10:11:48.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216165+0000", 2024-09-13T10:11:48.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024625200000000003 2024-09-13T10:11:48.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:48.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.223467+0000", 2024-09-13T10:11:48.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0073023840000000003 2024-09-13T10:11:48.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.225020+0000", 2024-09-13T10:11:48.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015529809999999999 2024-09-13T10:11:48.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.225406+0000", 2024-09-13T10:11:48.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038614699999999999 2024-09-13T10:11:48.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.225903+0000", 2024-09-13T10:11:48.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00049721900000000002 2024-09-13T10:11:48.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.226026+0000", 2024-09-13T10:11:48.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000123163 2024-09-13T10:11:48.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.226150+0000", 2024-09-13T10:11:48.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012384599999999999 2024-09-13T10:11:48.420 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.421 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.421 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.421 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:263 4.c 4:3e6706e5:::smithi19530714-14:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.421 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.215802+0000", 2024-09-13T10:11:48.421 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.177872936, 2024-09-13T10:11:48.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015708573999999999, 2024-09-13T10:11:48.421 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.421 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.421 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.421 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.421 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.421 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.422 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 263 2024-09-13T10:11:48.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.422 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215802+0000", 2024-09-13T10:11:48.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215802+0000", 2024-09-13T10:11:48.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215804+0000", 2024-09-13T10:11:48.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.624e-06 2024-09-13T10:11:48.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215813+0000", 2024-09-13T10:11:48.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4159999999999993e-06 2024-09-13T10:11:48.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215832+0000", 2024-09-13T10:11:48.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8768999999999999e-05 2024-09-13T10:11:48.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215843+0000", 2024-09-13T10:11:48.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0336999999999999e-05 2024-09-13T10:11:48.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216612+0000", 2024-09-13T10:11:48.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00076899399999999995 2024-09-13T10:11:48.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216767+0000", 2024-09-13T10:11:48.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015566399999999999 2024-09-13T10:11:48.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:48.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.227138+0000", 2024-09-13T10:11:48.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010370701 2024-09-13T10:11:48.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.229299+0000", 2024-09-13T10:11:48.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021611870000000002 2024-09-13T10:11:48.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.229896+0000", 2024-09-13T10:11:48.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00059659700000000003 2024-09-13T10:11:48.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231283+0000", 2024-09-13T10:11:48.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001387133 2024-09-13T10:11:48.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231392+0000", 2024-09-13T10:11:48.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010878999999999999 2024-09-13T10:11:48.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231511+0000", 2024-09-13T10:11:48.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119362 2024-09-13T10:11:48.427 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.427 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.427 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.427 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:264 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.427 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.215936+0000", 2024-09-13T10:11:48.428 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.177738957000003, 2024-09-13T10:11:48.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054954390999999998, 2024-09-13T10:11:48.428 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.428 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.428 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.428 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.428 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.428 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.428 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 264 2024-09-13T10:11:48.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.428 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215936+0000", 2024-09-13T10:11:48.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215936+0000", 2024-09-13T10:11:48.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215938+0000", 2024-09-13T10:11:48.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9860000000000001e-06 2024-09-13T10:11:48.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215951+0000", 2024-09-13T10:11:48.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2449999999999999e-05 2024-09-13T10:11:48.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215982+0000", 2024-09-13T10:11:48.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1133000000000002e-05 2024-09-13T10:11:48.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216009+0000", 2024-09-13T10:11:48.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6652000000000002e-05 2024-09-13T10:11:48.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.217072+0000", 2024-09-13T10:11:48.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010634240000000001 2024-09-13T10:11:48.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.217268+0000", 2024-09-13T10:11:48.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019641 2024-09-13T10:11:48.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:48.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.265415+0000", 2024-09-13T10:11:48.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.048146807999999999 2024-09-13T10:11:48.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.267067+0000", 2024-09-13T10:11:48.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00165211 2024-09-13T10:11:48.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.267319+0000", 2024-09-13T10:11:48.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025160500000000001 2024-09-13T10:11:48.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270591+0000", 2024-09-13T10:11:48.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0032720959999999999 2024-09-13T10:11:48.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270753+0000", 2024-09-13T10:11:48.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016215199999999999 2024-09-13T10:11:48.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270891+0000", 2024-09-13T10:11:48.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000137565 2024-09-13T10:11:48.433 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.433 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.434 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.434 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:278 4.c 4:39f17ee9:::smithi19530714-29:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.434 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.231988+0000", 2024-09-13T10:11:48.434 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.161687090999997, 2024-09-13T10:11:48.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011030108, 2024-09-13T10:11:48.434 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.434 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.434 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.434 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.434 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.434 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.434 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 278 2024-09-13T10:11:48.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.435 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231988+0000", 2024-09-13T10:11:48.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231988+0000", 2024-09-13T10:11:48.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231990+0000", 2024-09-13T10:11:48.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0499999999999999e-06 2024-09-13T10:11:48.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232002+0000", 2024-09-13T10:11:48.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2189e-05 2024-09-13T10:11:48.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232029+0000", 2024-09-13T10:11:48.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6673e-05 2024-09-13T10:11:48.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232043+0000", 2024-09-13T10:11:48.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4188999999999999e-05 2024-09-13T10:11:48.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232239+0000", 2024-09-13T10:11:48.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019557 2024-09-13T10:11:48.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232472+0000", 2024-09-13T10:11:48.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00023310900000000001 2024-09-13T10:11:48.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:48.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.237292+0000", 2024-09-13T10:11:48.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048195069999999998 2024-09-13T10:11:48.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.238595+0000", 2024-09-13T10:11:48.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013033260000000001 2024-09-13T10:11:48.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.239056+0000", 2024-09-13T10:11:48.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00046106800000000002 2024-09-13T10:11:48.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.242820+0000", 2024-09-13T10:11:48.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003763897 2024-09-13T10:11:48.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.242905+0000", 2024-09-13T10:11:48.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5580000000000004e-05 2024-09-13T10:11:48.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.243018+0000", 2024-09-13T10:11:48.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011294999999999999 2024-09-13T10:11:48.440 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.440 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.440 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.440 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:290 4.d 4:b19b932a:::smithi19530714-41:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.440 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.255335+0000", 2024-09-13T10:11:48.440 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.138340569, 2024-09-13T10:11:48.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021335703000000001, 2024-09-13T10:11:48.441 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.441 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.441 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.441 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.441 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.441 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.441 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 290 2024-09-13T10:11:48.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.441 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255335+0000", 2024-09-13T10:11:48.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255335+0000", 2024-09-13T10:11:48.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255338+0000", 2024-09-13T10:11:48.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6940000000000001e-06 2024-09-13T10:11:48.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255353+0000", 2024-09-13T10:11:48.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4136e-05 2024-09-13T10:11:48.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255390+0000", 2024-09-13T10:11:48.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7364999999999999e-05 2024-09-13T10:11:48.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255413+0000", 2024-09-13T10:11:48.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2690000000000001e-05 2024-09-13T10:11:48.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255615+0000", 2024-09-13T10:11:48.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000202306 2024-09-13T10:11:48.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255813+0000", 2024-09-13T10:11:48.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019769300000000001 2024-09-13T10:11:48.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:48.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274190+0000", 2024-09-13T10:11:48.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01837714 2024-09-13T10:11:48.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275400+0000", 2024-09-13T10:11:48.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001210275 2024-09-13T10:11:48.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276346+0000", 2024-09-13T10:11:48.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00094586699999999998 2024-09-13T10:11:48.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276497+0000", 2024-09-13T10:11:48.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015150299999999999 2024-09-13T10:11:48.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276597+0000", 2024-09-13T10:11:48.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9092999999999995e-05 2024-09-13T10:11:48.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276670+0000", 2024-09-13T10:11:48.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3941000000000002e-05 2024-09-13T10:11:48.446 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.446 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.447 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.447 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:295 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.447 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.261467+0000", 2024-09-13T10:11:48.447 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.132208429000002, 2024-09-13T10:11:48.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020013564000000001, 2024-09-13T10:11:48.447 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.447 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.447 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.447 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.447 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.447 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.447 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 295 2024-09-13T10:11:48.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.448 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261467+0000", 2024-09-13T10:11:48.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261467+0000", 2024-09-13T10:11:48.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261470+0000", 2024-09-13T10:11:48.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8200000000000001e-06 2024-09-13T10:11:48.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261487+0000", 2024-09-13T10:11:48.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7646e-05 2024-09-13T10:11:48.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261523+0000", 2024-09-13T10:11:48.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5299000000000002e-05 2024-09-13T10:11:48.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261545+0000", 2024-09-13T10:11:48.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2436e-05 2024-09-13T10:11:48.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261717+0000", 2024-09-13T10:11:48.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017231000000000001 2024-09-13T10:11:48.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261980+0000", 2024-09-13T10:11:48.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000262959 2024-09-13T10:11:48.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:48.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277104+0000", 2024-09-13T10:11:48.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015123983000000001 2024-09-13T10:11:48.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.279042+0000", 2024-09-13T10:11:48.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001937338 2024-09-13T10:11:48.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.279396+0000", 2024-09-13T10:11:48.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00035408199999999998 2024-09-13T10:11:48.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281302+0000", 2024-09-13T10:11:48.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001906346 2024-09-13T10:11:48.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281400+0000", 2024-09-13T10:11:48.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7528000000000005e-05 2024-09-13T10:11:48.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281480+0000", 2024-09-13T10:11:48.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0816999999999996e-05 2024-09-13T10:11:48.453 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.453 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.453 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.453 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:297 4.c 4:381e677b:::smithi19530714-48:head [assert-version v15,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.453 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.262493+0000", 2024-09-13T10:11:48.453 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.131182260000003, 2024-09-13T10:11:48.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012458446, 2024-09-13T10:11:48.454 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.454 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.454 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.454 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.454 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.454 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.454 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 297 2024-09-13T10:11:48.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.454 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262493+0000", 2024-09-13T10:11:48.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262493+0000", 2024-09-13T10:11:48.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262496+0000", 2024-09-13T10:11:48.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4729999999999999e-06 2024-09-13T10:11:48.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262510+0000", 2024-09-13T10:11:48.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4205999999999999e-05 2024-09-13T10:11:48.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262545+0000", 2024-09-13T10:11:48.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4789000000000002e-05 2024-09-13T10:11:48.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262566+0000", 2024-09-13T10:11:48.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1789000000000001e-05 2024-09-13T10:11:48.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262748+0000", 2024-09-13T10:11:48.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000181848 2024-09-13T10:11:48.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262986+0000", 2024-09-13T10:11:48.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00023747500000000001 2024-09-13T10:11:48.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:48.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.267794+0000", 2024-09-13T10:11:48.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048086479999999996 2024-09-13T10:11:48.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.269878+0000", 2024-09-13T10:11:48.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0020833900000000001 2024-09-13T10:11:48.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270928+0000", 2024-09-13T10:11:48.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001049901 2024-09-13T10:11:48.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274724+0000", 2024-09-13T10:11:48.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0037964880000000002 2024-09-13T10:11:48.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274862+0000", 2024-09-13T10:11:48.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000138286 2024-09-13T10:11:48.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274952+0000", 2024-09-13T10:11:48.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9153000000000005e-05 2024-09-13T10:11:48.460 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.460 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.460 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.460 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:313 4.c 4:3e6706e5:::smithi19530714-14:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.460 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.275533+0000", 2024-09-13T10:11:48.460 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.118142042999999, 2024-09-13T10:11:48.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019582932000000001, 2024-09-13T10:11:48.460 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.460 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.460 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.461 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.461 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.461 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.461 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 313 2024-09-13T10:11:48.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.461 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.461 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275533+0000", 2024-09-13T10:11:48.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275533+0000", 2024-09-13T10:11:48.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275536+0000", 2024-09-13T10:11:48.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4899999999999999e-06 2024-09-13T10:11:48.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275549+0000", 2024-09-13T10:11:48.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3699e-05 2024-09-13T10:11:48.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275574+0000", 2024-09-13T10:11:48.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4586000000000001e-05 2024-09-13T10:11:48.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275588+0000", 2024-09-13T10:11:48.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3746e-05 2024-09-13T10:11:48.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275705+0000", 2024-09-13T10:11:48.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000117352 2024-09-13T10:11:48.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275898+0000", 2024-09-13T10:11:48.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019293000000000001 2024-09-13T10:11:48.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:48.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286176+0000", 2024-09-13T10:11:48.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010277578000000001 2024-09-13T10:11:48.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.287526+0000", 2024-09-13T10:11:48.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013505349999999999 2024-09-13T10:11:48.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.288025+0000", 2024-09-13T10:11:48.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00049897500000000003 2024-09-13T10:11:48.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294958+0000", 2024-09-13T10:11:48.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069327980000000004 2024-09-13T10:11:48.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.295047+0000", 2024-09-13T10:11:48.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9154999999999996e-05 2024-09-13T10:11:48.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.466 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.466 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.295116+0000", 2024-09-13T10:11:48.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9088e-05 2024-09-13T10:11:48.466 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.466 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.466 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.467 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:314 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.467 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.276956+0000", 2024-09-13T10:11:48.467 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.116719363999998, 2024-09-13T10:11:48.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017940078000000002, 2024-09-13T10:11:48.467 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.467 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.467 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.467 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.467 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.467 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.467 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 314 2024-09-13T10:11:48.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.467 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276956+0000", 2024-09-13T10:11:48.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276956+0000", 2024-09-13T10:11:48.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276959+0000", 2024-09-13T10:11:48.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5730000000000002e-06 2024-09-13T10:11:48.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276971+0000", 2024-09-13T10:11:48.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2680000000000001e-05 2024-09-13T10:11:48.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276997+0000", 2024-09-13T10:11:48.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5579e-05 2024-09-13T10:11:48.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277018+0000", 2024-09-13T10:11:48.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1409000000000001e-05 2024-09-13T10:11:48.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277136+0000", 2024-09-13T10:11:48.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000117353 2024-09-13T10:11:48.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277237+0000", 2024-09-13T10:11:48.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101412 2024-09-13T10:11:48.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:48.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.285332+0000", 2024-09-13T10:11:48.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080951770000000003 2024-09-13T10:11:48.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.287120+0000", 2024-09-13T10:11:48.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001787828 2024-09-13T10:11:48.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.287200+0000", 2024-09-13T10:11:48.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0279999999999997e-05 2024-09-13T10:11:48.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294808+0000", 2024-09-13T10:11:48.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0076072400000000004 2024-09-13T10:11:48.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294853+0000", 2024-09-13T10:11:48.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5438000000000001e-05 2024-09-13T10:11:48.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294896+0000", 2024-09-13T10:11:48.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3108999999999997e-05 2024-09-13T10:11:48.473 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.473 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.473 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.473 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:340 4.d 4:b19b932a:::smithi19530714-41:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.473 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.284107+0000", 2024-09-13T10:11:48.473 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.109568134, 2024-09-13T10:11:48.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010293416, 2024-09-13T10:11:48.474 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.474 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.474 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.474 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.474 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.474 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.474 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 340 2024-09-13T10:11:48.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.474 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284107+0000", 2024-09-13T10:11:48.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284107+0000", 2024-09-13T10:11:48.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284109+0000", 2024-09-13T10:11:48.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.007e-06 2024-09-13T10:11:48.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284120+0000", 2024-09-13T10:11:48.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0346999999999999e-05 2024-09-13T10:11:48.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284144+0000", 2024-09-13T10:11:48.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3998999999999998e-05 2024-09-13T10:11:48.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284157+0000", 2024-09-13T10:11:48.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3045e-05 2024-09-13T10:11:48.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284244+0000", 2024-09-13T10:11:48.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7882999999999999e-05 2024-09-13T10:11:48.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284360+0000", 2024-09-13T10:11:48.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115467 2024-09-13T10:11:48.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:48.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.292368+0000", 2024-09-13T10:11:48.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080080380000000003 2024-09-13T10:11:48.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.293486+0000", 2024-09-13T10:11:48.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001117776 2024-09-13T10:11:48.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294261+0000", 2024-09-13T10:11:48.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00077481100000000003 2024-09-13T10:11:48.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294309+0000", 2024-09-13T10:11:48.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8086e-05 2024-09-13T10:11:48.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294358+0000", 2024-09-13T10:11:48.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8853999999999997e-05 2024-09-13T10:11:48.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294401+0000", 2024-09-13T10:11:48.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3102999999999997e-05 2024-09-13T10:11:48.479 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.479 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.479 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.480 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:345 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.480 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.286309+0000", 2024-09-13T10:11:48.480 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.107365956000002, 2024-09-13T10:11:48.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010478568000000001, 2024-09-13T10:11:48.480 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.480 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.480 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.480 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.480 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.480 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.480 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 345 2024-09-13T10:11:48.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.481 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286309+0000", 2024-09-13T10:11:48.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286309+0000", 2024-09-13T10:11:48.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286311+0000", 2024-09-13T10:11:48.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-13T10:11:48.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286320+0000", 2024-09-13T10:11:48.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8370000000000003e-06 2024-09-13T10:11:48.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286342+0000", 2024-09-13T10:11:48.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1446000000000001e-05 2024-09-13T10:11:48.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286385+0000", 2024-09-13T10:11:48.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3083000000000001e-05 2024-09-13T10:11:48.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286549+0000", 2024-09-13T10:11:48.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016463 2024-09-13T10:11:48.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286739+0000", 2024-09-13T10:11:48.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018965400000000001 2024-09-13T10:11:48.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:48.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294808+0000", 2024-09-13T10:11:48.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080692609999999994 2024-09-13T10:11:48.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.295388+0000", 2024-09-13T10:11:48.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00057994000000000003 2024-09-13T10:11:48.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296321+0000", 2024-09-13T10:11:48.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00093279099999999998 2024-09-13T10:11:48.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296622+0000", 2024-09-13T10:11:48.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030084300000000002 2024-09-13T10:11:48.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296702+0000", 2024-09-13T10:11:48.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0547e-05 2024-09-13T10:11:48.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296788+0000", 2024-09-13T10:11:48.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5612999999999995e-05 2024-09-13T10:11:48.486 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.486 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.486 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.486 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:364 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:48.486 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.297175+0000", 2024-09-13T10:11:48.486 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.09650078, 2024-09-13T10:11:48.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0085499329999999991, 2024-09-13T10:11:48.486 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.487 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.487 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.487 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.487 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.487 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.487 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 364 2024-09-13T10:11:48.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.487 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297175+0000", 2024-09-13T10:11:48.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297175+0000", 2024-09-13T10:11:48.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297176+0000", 2024-09-13T10:11:48.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9199999999999999e-07 2024-09-13T10:11:48.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297181+0000", 2024-09-13T10:11:48.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5149999999999997e-06 2024-09-13T10:11:48.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297191+0000", 2024-09-13T10:11:48.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8260000000000005e-06 2024-09-13T10:11:48.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297198+0000", 2024-09-13T10:11:48.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6950000000000003e-06 2024-09-13T10:11:48.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297244+0000", 2024-09-13T10:11:48.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6397999999999999e-05 2024-09-13T10:11:48.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297291+0000", 2024-09-13T10:11:48.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7342999999999997e-05 2024-09-13T10:11:48.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:48.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.303658+0000", 2024-09-13T10:11:48.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0063669 2024-09-13T10:11:48.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.304437+0000", 2024-09-13T10:11:48.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00077861399999999996 2024-09-13T10:11:48.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.304902+0000", 2024-09-13T10:11:48.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00046505899999999998 2024-09-13T10:11:48.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.305534+0000", 2024-09-13T10:11:48.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00063224199999999998 2024-09-13T10:11:48.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.305643+0000", 2024-09-13T10:11:48.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108384 2024-09-13T10:11:48.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.305725+0000", 2024-09-13T10:11:48.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1965000000000006e-05 2024-09-13T10:11:48.492 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.492 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.492 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.493 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:378 4.c 4:39f17ee9:::smithi19530714-29:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:48.493 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.320710+0000", 2024-09-13T10:11:48.493 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.072965408999998, 2024-09-13T10:11:48.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0079049020000000001, 2024-09-13T10:11:48.493 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.493 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.493 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.493 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.493 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.493 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.493 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 378 2024-09-13T10:11:48.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.493 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320710+0000", 2024-09-13T10:11:48.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320710+0000", 2024-09-13T10:11:48.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320712+0000", 2024-09-13T10:11:48.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4099999999999998e-06 2024-09-13T10:11:48.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320724+0000", 2024-09-13T10:11:48.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.134e-05 2024-09-13T10:11:48.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320746+0000", 2024-09-13T10:11:48.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2342999999999999e-05 2024-09-13T10:11:48.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320758+0000", 2024-09-13T10:11:48.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1557e-05 2024-09-13T10:11:48.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320840+0000", 2024-09-13T10:11:48.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2274000000000005e-05 2024-09-13T10:11:48.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320933+0000", 2024-09-13T10:11:48.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3375999999999995e-05 2024-09-13T10:11:48.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:48.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.326291+0000", 2024-09-13T10:11:48.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053575089999999999 2024-09-13T10:11:48.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.327254+0000", 2024-09-13T10:11:48.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00096337099999999998 2024-09-13T10:11:48.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.327902+0000", 2024-09-13T10:11:48.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00064762100000000003 2024-09-13T10:11:48.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328501+0000", 2024-09-13T10:11:48.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000599232 2024-09-13T10:11:48.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328547+0000", 2024-09-13T10:11:48.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5973000000000002e-05 2024-09-13T10:11:48.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328615+0000", 2024-09-13T10:11:48.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7896000000000002e-05 2024-09-13T10:11:48.499 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.499 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.499 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.499 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:395 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:48.499 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.322105+0000", 2024-09-13T10:11:48.499 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.071570336999997, 2024-09-13T10:11:48.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0084827089999999997, 2024-09-13T10:11:48.499 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.499 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.499 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.499 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.499 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.499 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.500 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 395 2024-09-13T10:11:48.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.500 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322105+0000", 2024-09-13T10:11:48.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322105+0000", 2024-09-13T10:11:48.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322106+0000", 2024-09-13T10:11:48.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2099999999999995e-07 2024-09-13T10:11:48.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322110+0000", 2024-09-13T10:11:48.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.827e-06 2024-09-13T10:11:48.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322120+0000", 2024-09-13T10:11:48.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0227999999999999e-05 2024-09-13T10:11:48.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322160+0000", 2024-09-13T10:11:48.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9941999999999998e-05 2024-09-13T10:11:48.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322232+0000", 2024-09-13T10:11:48.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2451000000000001e-05 2024-09-13T10:11:48.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322314+0000", 2024-09-13T10:11:48.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2089000000000006e-05 2024-09-13T10:11:48.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:48.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328374+0000", 2024-09-13T10:11:48.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0060591350000000002 2024-09-13T10:11:48.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.329439+0000", 2024-09-13T10:11:48.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010654830000000001 2024-09-13T10:11:48.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330219+0000", 2024-09-13T10:11:48.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00078003700000000003 2024-09-13T10:11:48.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330488+0000", 2024-09-13T10:11:48.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000269411 2024-09-13T10:11:48.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330544+0000", 2024-09-13T10:11:48.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5704000000000001e-05 2024-09-13T10:11:48.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330588+0000", 2024-09-13T10:11:48.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3581000000000002e-05 2024-09-13T10:11:48.504 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.505 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.505 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.505 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:595 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:48.505 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.370445+0000", 2024-09-13T10:11:48.505 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.023230302999998, 2024-09-13T10:11:48.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0084586149999999992, 2024-09-13T10:11:48.505 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.505 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.505 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.505 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.505 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.506 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.506 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 595 2024-09-13T10:11:48.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.506 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370445+0000", 2024-09-13T10:11:48.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370445+0000", 2024-09-13T10:11:48.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370447+0000", 2024-09-13T10:11:48.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.66e-06 2024-09-13T10:11:48.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370455+0000", 2024-09-13T10:11:48.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3059999999999998e-06 2024-09-13T10:11:48.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370472+0000", 2024-09-13T10:11:48.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7479999999999999e-05 2024-09-13T10:11:48.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370484+0000", 2024-09-13T10:11:48.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1882000000000001e-05 2024-09-13T10:11:48.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370562+0000", 2024-09-13T10:11:48.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7531e-05 2024-09-13T10:11:48.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370738+0000", 2024-09-13T10:11:48.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017587200000000001 2024-09-13T10:11:48.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:11:48.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.377463+0000", 2024-09-13T10:11:48.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0067247829999999998 2024-09-13T10:11:48.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378047+0000", 2024-09-13T10:11:48.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00058493100000000001 2024-09-13T10:11:48.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378730+0000", 2024-09-13T10:11:48.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068235799999999999 2024-09-13T10:11:48.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378840+0000", 2024-09-13T10:11:48.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110296 2024-09-13T10:11:48.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378875+0000", 2024-09-13T10:11:48.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4552999999999999e-05 2024-09-13T10:11:48.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378904+0000", 2024-09-13T10:11:48.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8963e-05 2024-09-13T10:11:48.511 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.511 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.511 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.511 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:782 4.c 4:39f17ee9:::smithi19530714-29:head [write 1543557~550004] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:48.511 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.601833+0000", 2024-09-13T10:11:48.511 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 56.791842428999999, 2024-09-13T10:11:48.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081663250000000003, 2024-09-13T10:11:48.512 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.512 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.512 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.512 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.512 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.512 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.512 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 782 2024-09-13T10:11:48.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.512 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.601833+0000", 2024-09-13T10:11:48.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.601833+0000", 2024-09-13T10:11:48.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.601835+0000", 2024-09-13T10:11:48.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.576e-06 2024-09-13T10:11:48.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602350+0000", 2024-09-13T10:11:48.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00051449700000000004 2024-09-13T10:11:48.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602379+0000", 2024-09-13T10:11:48.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8598999999999998e-05 2024-09-13T10:11:48.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602394+0000", 2024-09-13T10:11:48.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4952e-05 2024-09-13T10:11:48.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602534+0000", 2024-09-13T10:11:48.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014037899999999999 2024-09-13T10:11:48.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602796+0000", 2024-09-13T10:11:48.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000262415 2024-09-13T10:11:48.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:11:48.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.603214+0000", 2024-09-13T10:11:48.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00041785599999999998 2024-09-13T10:11:48.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.606389+0000", 2024-09-13T10:11:48.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0031751549999999998 2024-09-13T10:11:48.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:11:48.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.608306+0000", 2024-09-13T10:11:48.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0019163540000000001 2024-09-13T10:11:48.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.609577+0000", 2024-09-13T10:11:48.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001271466 2024-09-13T10:11:48.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:48.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.609872+0000", 2024-09-13T10:11:48.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00029501400000000002 2024-09-13T10:11:48.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.609999+0000", 2024-09-13T10:11:48.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127062 2024-09-13T10:11:48.517 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.517 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.518 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.518 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1085 4.c 4:3e6706e5:::smithi19530714-14:head [sparse-read 0~906036] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:48.518 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:56.648267+0000", 2024-09-13T10:11:48.518 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.745407905999997, 2024-09-13T10:11:48.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0083262089999999993, 2024-09-13T10:11:48.518 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:48.518 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:48.518 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:11:48.518 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:11:48.518 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:11:48.518 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.518 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1085 2024-09-13T10:11:48.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.519 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:48.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:48.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648267+0000", 2024-09-13T10:11:48.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:48.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648267+0000", 2024-09-13T10:11:48.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:48.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:48.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648269+0000", 2024-09-13T10:11:48.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1769999999999999e-06 2024-09-13T10:11:48.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:48.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648276+0000", 2024-09-13T10:11:48.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0530000000000001e-06 2024-09-13T10:11:48.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:48.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648292+0000", 2024-09-13T10:11:48.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6209e-05 2024-09-13T10:11:48.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648307+0000", 2024-09-13T10:11:48.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4647000000000001e-05 2024-09-13T10:11:48.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:48.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.655806+0000", 2024-09-13T10:11:48.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0074996500000000001 2024-09-13T10:11:48.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:48.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.656060+0000", 2024-09-13T10:11:48.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000253639 2024-09-13T10:11:48.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:48.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:48.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:48.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.656594+0000", 2024-09-13T10:11:48.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00053383399999999996 2024-09-13T10:11:48.522 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.522 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.522 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.522 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:48.522 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:48.522 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:48.524 INFO:tasks.rados.rados.0.smithi195.stdout:1514: expect (ObjNum 148 snap 0 seq_num 148) 2024-09-13T10:11:48.524 INFO:tasks.rados.rados.0.smithi195.stdout:1510: expect (ObjNum 277 snap 0 seq_num 277) 2024-09-13T10:11:48.524 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:48.524 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T10:11:48.525 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:48.519+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:48.549 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:48.624 INFO:tasks.rados.rados.0.smithi195.stdout:1508: expect (ObjNum 259 snap 0 seq_num 259) 2024-09-13T10:11:48.625 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:48.625 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:48.622+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:48.648 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 286417 2024-09-13T10:11:48.657 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:48.657 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:48.657 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:11:48.657 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:48.671 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T10:11:48.693 INFO:tasks.rados.rados.0.smithi195.stdout:1509: expect (ObjNum 139 snap 0 seq_num 139) 2024-09-13T10:11:48.694 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:48.704 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-13T10:11:48.726 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:48.721+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:48.796 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:48.796 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:48.796 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:11:48.796 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:11:48.796 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:48.810 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T10:11:48.840 INFO:tasks.rados.rados.0.smithi195.stdout:1515: finishing write tid 1 to smithi19530714-13 2024-09-13T10:11:48.840 INFO:tasks.rados.rados.0.smithi195.stdout:1515: finishing write tid 2 to smithi19530714-13 2024-09-13T10:11:48.841 INFO:tasks.rados.rados.0.smithi195.stdout:1517: finishing set_chunk to oid 11 2024-09-13T10:11:48.841 INFO:tasks.rados.rados.0.smithi195.stdout:1517: got expected ENOENT (src dne) 2024-09-13T10:11:48.841 INFO:tasks.rados.rados.0.smithi195.stdout:1515: finishing write tid 3 to smithi19530714-13 2024-09-13T10:11:48.841 INFO:tasks.rados.rados.0.smithi195.stdout:1515: finishing write tid 4 to smithi19530714-13 2024-09-13T10:11:48.841 INFO:tasks.rados.rados.0.smithi195.stdout:1515: finishing write tid 5 to smithi19530714-13 2024-09-13T10:11:48.841 INFO:tasks.rados.rados.0.smithi195.stdout:1515: oid 13 updating version 0 to 70 2024-09-13T10:11:48.841 INFO:tasks.rados.rados.0.smithi195.stdout:1515: oid 13 version 70 is already newer than 67 2024-09-13T10:11:48.841 INFO:tasks.rados.rados.0.smithi195.stdout:1515: oid 13 version 70 is already newer than 68 2024-09-13T10:11:48.841 INFO:tasks.rados.rados.0.smithi195.stdout:1515: oid 13 version 70 is already newer than 69 2024-09-13T10:11:48.841 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 70 (ObjNum 308 snap 0 seq_num 308) dirty exists 2024-09-13T10:11:48.841 INFO:tasks.rados.rados.0.smithi195.stdout:1515: left oid 13 (ObjNum 308 snap 0 seq_num 308) 2024-09-13T10:11:48.841 INFO:tasks.rados.rados.0.smithi195.stdout:1508: done (15 left) 2024-09-13T10:11:48.842 INFO:tasks.rados.rados.0.smithi195.stdout:1509: done (14 left) 2024-09-13T10:11:48.842 INFO:tasks.rados.rados.0.smithi195.stdout:1510: done (13 left) 2024-09-13T10:11:48.842 INFO:tasks.rados.rados.0.smithi195.stdout:1513: done (12 left) 2024-09-13T10:11:48.842 INFO:tasks.rados.rados.0.smithi195.stdout:1514: done (11 left) 2024-09-13T10:11:48.842 INFO:tasks.rados.rados.0.smithi195.stdout:1515: done (10 left) 2024-09-13T10:11:48.842 INFO:tasks.rados.rados.0.smithi195.stdout:1517: done (9 left) 2024-09-13T10:11:48.842 INFO:tasks.rados.rados.0.smithi195.stdout:1526: write oid 45 current snap is 0 2024-09-13T10:11:48.842 INFO:tasks.rados.rados.0.smithi195.stdout:1526: seq_num 311 ranges {426977=456559,1482841=799764,2958080=268897} 2024-09-13T10:11:48.854 INFO:tasks.rados.rados.0.smithi195.stdout:1526: writing smithi19530714-45 from 426977 to 883536 tid 1 2024-09-13T10:11:48.855 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:48.851+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:48.877 INFO:tasks.rados.rados.0.smithi195.stdout:1526: writing smithi19530714-45 from 1482841 to 2282605 tid 2 2024-09-13T10:11:48.891 INFO:tasks.rados.rados.0.smithi195.stdout:1526: writing smithi19530714-45 from 2958080 to 3226977 tid 3 2024-09-13T10:11:48.892 INFO:tasks.rados.rados.0.smithi195.stdout:1527: write (excl) oid 3 current snap is 0 2024-09-13T10:11:48.892 INFO:tasks.rados.rados.0.smithi195.stdout:1527: seq_num 312 ranges {443784=648161,1657676=659775,2818209=425575} 2024-09-13T10:11:48.905 INFO:tasks.rados.rados.0.smithi195.stdout:1527: writing smithi19530714-3 from 443784 to 1091945 tid 1 2024-09-13T10:11:48.917 INFO:tasks.rados.rados.0.smithi195.stdout:1527: writing smithi19530714-3 from 1657676 to 2317451 tid 2 2024-09-13T10:11:48.925 INFO:tasks.rados.rados.0.smithi195.stdout:1527: writing smithi19530714-3 from 2818209 to 3243784 tid 3 2024-09-13T10:11:48.925 INFO:tasks.rados.rados.0.smithi195.stdout:1528: read oid 19 snap -1 2024-09-13T10:11:48.926 INFO:tasks.rados.rados.0.smithi195.stdout:1528: expect (ObjNum 160 snap 0 seq_num 160) 2024-09-13T10:11:48.926 INFO:tasks.rados.rados.0.smithi195.stdout:1529: write oid 4 current snap is 0 2024-09-13T10:11:48.926 INFO:tasks.rados.rados.0.smithi195.stdout:1529: seq_num 313 ranges {460591=723410,1716158=403433,2729279=440140} 2024-09-13T10:11:48.927 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:48.922+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:48.935 INFO:tasks.rados.rados.0.smithi195.stdout:1529: writing smithi19530714-4 from 460591 to 1184001 tid 1 2024-09-13T10:11:48.941 INFO:tasks.rados.rados.0.smithi195.stdout:1529: writing smithi19530714-4 from 1716158 to 2119591 tid 2 2024-09-13T10:11:48.942 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:48.942 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:11:48.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:11:48.942 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:253 4.7 4:e81d9ad0:::smithi19530714-4:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.134475+0000", 2024-09-13T10:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 57.804734496999998, 2024-09-13T10:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013417535, 2024-09-13T10:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 253 2024-09-13T10:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134475+0000", 2024-09-13T10:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134475+0000", 2024-09-13T10:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134477+0000", 2024-09-13T10:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1859999999999999e-06 2024-09-13T10:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134486+0000", 2024-09-13T10:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3139999999999993e-06 2024-09-13T10:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134506+0000", 2024-09-13T10:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9165999999999999e-05 2024-09-13T10:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134515+0000", 2024-09-13T10:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3670000000000003e-06 2024-09-13T10:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134579+0000", 2024-09-13T10:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4200000000000002e-05 2024-09-13T10:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134661+0000", 2024-09-13T10:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1632999999999996e-05 2024-09-13T10:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.144025+0000", 2024-09-13T10:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093644190000000006 2024-09-13T10:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.144574+0000", 2024-09-13T10:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054873699999999999 2024-09-13T10:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.146021+0000", 2024-09-13T10:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001446826 2024-09-13T10:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.147761+0000", 2024-09-13T10:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017406399999999999 2024-09-13T10:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.147835+0000", 2024-09-13T10:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3708999999999995e-05 2024-09-13T10:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.147893+0000", 2024-09-13T10:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7337999999999999e-05 2024-09-13T10:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:256 4.7 4:e8c2cbec:::smithi19530714-7:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.214859+0000", 2024-09-13T10:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 57.724350139000002, 2024-09-13T10:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049512370999999999, 2024-09-13T10:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 256 2024-09-13T10:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214859+0000", 2024-09-13T10:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214859+0000", 2024-09-13T10:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214863+0000", 2024-09-13T10:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.343e-06 2024-09-13T10:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214876+0000", 2024-09-13T10:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.308e-05 2024-09-13T10:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214909+0000", 2024-09-13T10:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2882999999999997e-05 2024-09-13T10:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214925+0000", 2024-09-13T10:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6002e-05 2024-09-13T10:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215088+0000", 2024-09-13T10:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016304199999999999 2024-09-13T10:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215261+0000", 2024-09-13T10:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000173804 2024-09-13T10:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258838+0000", 2024-09-13T10:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043576675000000002 2024-09-13T10:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.260096+0000", 2024-09-13T10:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012580410000000001 2024-09-13T10:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.263959+0000", 2024-09-13T10:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038630840000000001 2024-09-13T10:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264262+0000", 2024-09-13T10:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030279699999999999 2024-09-13T10:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264321+0000", 2024-09-13T10:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-13T10:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264372+0000", 2024-09-13T10:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0513000000000002e-05 2024-09-13T10:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:266 4.7 4:e55883d8:::smithi19530714-17:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.215454+0000", 2024-09-13T10:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 57.723755478000001, 2024-09-13T10:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020880112999999999, 2024-09-13T10:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 266 2024-09-13T10:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215454+0000", 2024-09-13T10:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215454+0000", 2024-09-13T10:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215457+0000", 2024-09-13T10:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6029999999999999e-06 2024-09-13T10:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215471+0000", 2024-09-13T10:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4629e-05 2024-09-13T10:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215503+0000", 2024-09-13T10:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1449000000000003e-05 2024-09-13T10:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215528+0000", 2024-09-13T10:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5273e-05 2024-09-13T10:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215632+0000", 2024-09-13T10:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010387699999999999 2024-09-13T10:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215898+0000", 2024-09-13T10:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026582100000000002 2024-09-13T10:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228364+0000", 2024-09-13T10:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012466672 2024-09-13T10:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.229463+0000", 2024-09-13T10:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001098344 2024-09-13T10:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.230814+0000", 2024-09-13T10:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013513959999999999 2024-09-13T10:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236141+0000", 2024-09-13T10:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053270160000000004 2024-09-13T10:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236223+0000", 2024-09-13T10:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1799999999999996e-05 2024-09-13T10:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236334+0000", 2024-09-13T10:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111233 2024-09-13T10:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:267 4.7 4:ea79a1ec:::smithi19530714-18:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.215583+0000", 2024-09-13T10:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 57.723626506000002, 2024-09-13T10:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.051600035000000002, 2024-09-13T10:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 267 2024-09-13T10:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215583+0000", 2024-09-13T10:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215583+0000", 2024-09-13T10:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215585+0000", 2024-09-13T10:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9099999999999999e-06 2024-09-13T10:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215594+0000", 2024-09-13T10:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8799999999999997e-06 2024-09-13T10:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215613+0000", 2024-09-13T10:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9473000000000001e-05 2024-09-13T10:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215630+0000", 2024-09-13T10:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6466000000000001e-05 2024-09-13T10:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.216360+0000", 2024-09-13T10:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073001200000000004 2024-09-13T10:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.216526+0000", 2024-09-13T10:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000166775 2024-09-13T10:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.260535+0000", 2024-09-13T10:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044008310000000002 2024-09-13T10:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261671+0000", 2024-09-13T10:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011362169999999999 2024-09-13T10:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.266819+0000", 2024-09-13T10:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051479940000000004 2024-09-13T10:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.267087+0000", 2024-09-13T10:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026847299999999998 2024-09-13T10:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.267140+0000", 2024-09-13T10:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2054e-05 2024-09-13T10:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.267183+0000", 2024-09-13T10:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3470999999999998e-05 2024-09-13T10:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:303 4.7 4:e81d9ad0:::smithi19530714-4:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.268636+0000", 2024-09-13T10:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 57.670573890999997, 2024-09-13T10:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018691609000000001, 2024-09-13T10:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 303 2024-09-13T10:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268636+0000", 2024-09-13T10:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268636+0000", 2024-09-13T10:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268638+0000", 2024-09-13T10:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8990000000000001e-06 2024-09-13T10:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268651+0000", 2024-09-13T10:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2556e-05 2024-09-13T10:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268682+0000", 2024-09-13T10:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0979000000000003e-05 2024-09-13T10:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268707+0000", 2024-09-13T10:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4680000000000001e-05 2024-09-13T10:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268787+0000", 2024-09-13T10:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9878999999999999e-05 2024-09-13T10:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268881+0000", 2024-09-13T10:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4242999999999999e-05 2024-09-13T10:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.283577+0000", 2024-09-13T10:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014695952999999999 2024-09-13T10:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.284599+0000", 2024-09-13T10:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001022329 2024-09-13T10:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285985+0000", 2024-09-13T10:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013861430000000001 2024-09-13T10:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.287150+0000", 2024-09-13T10:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001164382 2024-09-13T10:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.287245+0000", 2024-09-13T10:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5643999999999994e-05 2024-09-13T10:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.287327+0000", 2024-09-13T10:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1922000000000006e-05 2024-09-13T10:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:567 4.7 4:ea79a1ec:::smithi19530714-18:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 57.571908739000001, 2024-09-13T10:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012585694999999999, 2024-09-13T10:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 567 2024-09-13T10:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1200000000000003e-07 2024-09-13T10:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367306+0000", 2024-09-13T10:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0309999999999998e-06 2024-09-13T10:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367314+0000", 2024-09-13T10:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8960000000000003e-06 2024-09-13T10:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367321+0000", 2024-09-13T10:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4880000000000004e-06 2024-09-13T10:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367612+0000", 2024-09-13T10:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029163099999999999 2024-09-13T10:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367715+0000", 2024-09-13T10:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102837 2024-09-13T10:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.374040+0000", 2024-09-13T10:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063245539999999996 2024-09-13T10:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.374609+0000", 2024-09-13T10:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056937299999999997 2024-09-13T10:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379774+0000", 2024-09-13T10:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005164737 2024-09-13T10:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379810+0000", 2024-09-13T10:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6587000000000001e-05 2024-09-13T10:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379845+0000", 2024-09-13T10:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.468e-05 2024-09-13T10:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379886+0000", 2024-09-13T10:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1269000000000001e-05 2024-09-13T10:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1039 4.17 4:ea79a1ec:::smithi19530714-18:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.096166+0000", 2024-09-13T10:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.843043481999999, 2024-09-13T10:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.051040927, 2024-09-13T10:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1039 2024-09-13T10:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096166+0000", 2024-09-13T10:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096166+0000", 2024-09-13T10:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096169+0000", 2024-09-13T10:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8100000000000002e-06 2024-09-13T10:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096180+0000", 2024-09-13T10:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1537e-05 2024-09-13T10:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:48.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096212+0000", 2024-09-13T10:11:48.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2104999999999998e-05 2024-09-13T10:11:48.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096268+0000", 2024-09-13T10:11:48.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5127999999999998e-05 2024-09-13T10:11:48.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:48.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096480+0000", 2024-09-13T10:11:48.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021249100000000001 2024-09-13T10:11:48.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:48.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096692+0000", 2024-09-13T10:11:48.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021200400000000001 2024-09-13T10:11:48.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:48.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.147207+0000", 2024-09-13T10:11:48.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.050514851999999999 2024-09-13T10:11:48.988 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:48.988 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1574 4.7 4:e79f1505:::smithi19530714-32:head [write 1799494~402223] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.716930+0000", 2024-09-13T10:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.222279215999997, 2024-09-13T10:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017286329999999999, 2024-09-13T10:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1574 2024-09-13T10:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.716930+0000", 2024-09-13T10:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.716930+0000", 2024-09-13T10:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.716962+0000", 2024-09-13T10:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1279999999999999e-05 2024-09-13T10:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717453+0000", 2024-09-13T10:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049160399999999998 2024-09-13T10:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717480+0000", 2024-09-13T10:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7313000000000001e-05 2024-09-13T10:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717496+0000", 2024-09-13T10:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5435999999999999e-05 2024-09-13T10:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717592+0000", 2024-09-13T10:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5851000000000001e-05 2024-09-13T10:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717732+0000", 2024-09-13T10:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013985000000000001 2024-09-13T10:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717962+0000", 2024-09-13T10:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000230518 2024-09-13T10:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722934+0000", 2024-09-13T10:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049717099999999998 2024-09-13T10:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724179+0000", 2024-09-13T10:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012455389999999999 2024-09-13T10:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.733963+0000", 2024-09-13T10:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097835349999999995 2024-09-13T10:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.734101+0000", 2024-09-13T10:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013852 2024-09-13T10:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.734217+0000", 2024-09-13T10:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011517400000000001 2024-09-13T10:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1575 4.7 4:e79f1505:::smithi19530714-32:head [setxattr _header (56) in=7b,truncate 2201717] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.717530+0000", 2024-09-13T10:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.22167906, 2024-09-13T10:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018674682000000001, 2024-09-13T10:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1575 2024-09-13T10:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717530+0000", 2024-09-13T10:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717530+0000", 2024-09-13T10:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717532+0000", 2024-09-13T10:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.813e-06 2024-09-13T10:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717542+0000", 2024-09-13T10:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4930000000000004e-06 2024-09-13T10:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717559+0000", 2024-09-13T10:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7337e-05 2024-09-13T10:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717575+0000", 2024-09-13T10:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6385999999999999e-05 2024-09-13T10:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.719954+0000", 2024-09-13T10:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023782679999999998 2024-09-13T10:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.720308+0000", 2024-09-13T10:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035397100000000002 2024-09-13T10:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.720827+0000", 2024-09-13T10:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00051909999999999999 2024-09-13T10:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722944+0000", 2024-09-13T10:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021173210000000001 2024-09-13T10:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724274+0000", 2024-09-13T10:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013303099999999999 2024-09-13T10:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.735926+0000", 2024-09-13T10:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011651846 2024-09-13T10:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736061+0000", 2024-09-13T10:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013452000000000001 2024-09-13T10:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736205+0000", 2024-09-13T10:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014431700000000001 2024-09-13T10:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1576 4.7 4:e79f1505:::smithi19530714-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.717602+0000", 2024-09-13T10:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.221607874999997, 2024-09-13T10:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019134776999999999, 2024-09-13T10:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1576 2024-09-13T10:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717602+0000", 2024-09-13T10:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717602+0000", 2024-09-13T10:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717603+0000", 2024-09-13T10:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8899999999999999e-06 2024-09-13T10:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717610+0000", 2024-09-13T10:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8900000000000001e-06 2024-09-13T10:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717631+0000", 2024-09-13T10:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0741999999999999e-05 2024-09-13T10:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717644+0000", 2024-09-13T10:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2680000000000001e-05 2024-09-13T10:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722783+0000", 2024-09-13T10:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005139429 2024-09-13T10:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722912+0000", 2024-09-13T10:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012886399999999999 2024-09-13T10:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736301+0000", 2024-09-13T10:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013389082 2024-09-13T10:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736498+0000", 2024-09-13T10:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000197059 2024-09-13T10:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736736+0000", 2024-09-13T10:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023814100000000001 2024-09-13T10:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1580 4.17 4:e8c2cbec:::smithi19530714-7:head [write 618524~710869,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.724425+0000", 2024-09-13T10:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.214784133999999, 2024-09-13T10:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013470263999999999, 2024-09-13T10:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1580 2024-09-13T10:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724425+0000", 2024-09-13T10:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724425+0000", 2024-09-13T10:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724427+0000", 2024-09-13T10:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9530000000000002e-06 2024-09-13T10:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725799+0000", 2024-09-13T10:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001371466 2024-09-13T10:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725823+0000", 2024-09-13T10:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4332999999999999e-05 2024-09-13T10:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725844+0000", 2024-09-13T10:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0539000000000001e-05 2024-09-13T10:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725963+0000", 2024-09-13T10:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119524 2024-09-13T10:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.726194+0000", 2024-09-13T10:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023045800000000001 2024-09-13T10:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.726619+0000", 2024-09-13T10:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042517199999999997 2024-09-13T10:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.735852+0000", 2024-09-13T10:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0092335560000000004 2024-09-13T10:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.735959+0000", 2024-09-13T10:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106351 2024-09-13T10:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.737757+0000", 2024-09-13T10:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017985119999999999 2024-09-13T10:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.737838+0000", 2024-09-13T10:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0457999999999994e-05 2024-09-13T10:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.737896+0000", 2024-09-13T10:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7942000000000002e-05 2024-09-13T10:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1855 4.17 4:e81d9ad0:::smithi19530714-4:head [write 739541~709000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.758272+0000", 2024-09-13T10:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.18093743, 2024-09-13T10:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01875696, 2024-09-13T10:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1855 2024-09-13T10:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.758272+0000", 2024-09-13T10:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.758272+0000", 2024-09-13T10:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.758275+0000", 2024-09-13T10:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4429999999999999e-06 2024-09-13T10:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.759964+0000", 2024-09-13T10:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001688557 2024-09-13T10:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.759996+0000", 2024-09-13T10:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1705999999999998e-05 2024-09-13T10:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.760013+0000", 2024-09-13T10:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6846000000000001e-05 2024-09-13T10:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.760156+0000", 2024-09-13T10:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014331700000000001 2024-09-13T10:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-13T10:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.760167+0000", 2024-09-13T10:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.062e-05 2024-09-13T10:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.763490+0000", 2024-09-13T10:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033235309999999998 2024-09-13T10:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.763611+0000", 2024-09-13T10:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012099699999999999 2024-09-13T10:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.763830+0000", 2024-09-13T10:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000218519 2024-09-13T10:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773195+0000", 2024-09-13T10:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093654340000000006 2024-09-13T10:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.774701+0000", 2024-09-13T10:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00150588 2024-09-13T10:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.776817+0000", 2024-09-13T10:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021156180000000001 2024-09-13T10:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.776910+0000", 2024-09-13T10:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3194000000000003e-05 2024-09-13T10:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.777029+0000", 2024-09-13T10:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119298 2024-09-13T10:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1856 4.17 4:e81d9ad0:::smithi19530714-4:head [write 2121859~426376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.765194+0000", 2024-09-13T10:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.174015636999997, 2024-09-13T10:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013074073, 2024-09-13T10:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1856 2024-09-13T10:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765194+0000", 2024-09-13T10:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765194+0000", 2024-09-13T10:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765196+0000", 2024-09-13T10:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.317e-06 2024-09-13T10:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765559+0000", 2024-09-13T10:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036326199999999999 2024-09-13T10:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765581+0000", 2024-09-13T10:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1676e-05 2024-09-13T10:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765595+0000", 2024-09-13T10:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4025999999999999e-05 2024-09-13T10:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.767613+0000", 2024-09-13T10:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00201768 2024-09-13T10:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.767751+0000", 2024-09-13T10:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013809 2024-09-13T10:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.767983+0000", 2024-09-13T10:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023206499999999999 2024-09-13T10:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773203+0000", 2024-09-13T10:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052195599999999998 2024-09-13T10:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.777481+0000", 2024-09-13T10:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042782840000000003 2024-09-13T10:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.778120+0000", 2024-09-13T10:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063947299999999999 2024-09-13T10:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.778203+0000", 2024-09-13T10:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2857999999999998e-05 2024-09-13T10:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.778268+0000", 2024-09-13T10:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4782000000000004e-05 2024-09-13T10:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1857 4.17 4:e81d9ad0:::smithi19530714-4:head [setxattr _header (55) in=7b,truncate 2739541] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.765630+0000", 2024-09-13T10:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.173579930000002, 2024-09-13T10:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013989379999999999, 2024-09-13T10:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1857 2024-09-13T10:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765630+0000", 2024-09-13T10:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765630+0000", 2024-09-13T10:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765631+0000", 2024-09-13T10:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.64e-06 2024-09-13T10:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765641+0000", 2024-09-13T10:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0047000000000001e-05 2024-09-13T10:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765658+0000", 2024-09-13T10:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6909e-05 2024-09-13T10:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765673+0000", 2024-09-13T10:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4980000000000001e-05 2024-09-13T10:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.771667+0000", 2024-09-13T10:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059938209999999999 2024-09-13T10:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.771795+0000", 2024-09-13T10:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012816999999999999 2024-09-13T10:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.772028+0000", 2024-09-13T10:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023327800000000001 2024-09-13T10:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773657+0000", 2024-09-13T10:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016287039999999999 2024-09-13T10:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.777579+0000", 2024-09-13T10:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039218589999999998 2024-09-13T10:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779159+0000", 2024-09-13T10:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001580176 2024-09-13T10:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779343+0000", 2024-09-13T10:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000183769 2024-09-13T10:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779619+0000", 2024-09-13T10:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027602699999999998 2024-09-13T10:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1858 4.17 4:e81d9ad0:::smithi19530714-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.765692+0000", 2024-09-13T10:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.173517844000003, 2024-09-13T10:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014311129000000001, 2024-09-13T10:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1858 2024-09-13T10:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765692+0000", 2024-09-13T10:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765692+0000", 2024-09-13T10:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765693+0000", 2024-09-13T10:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-13T10:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765697+0000", 2024-09-13T10:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6770000000000003e-06 2024-09-13T10:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765707+0000", 2024-09-13T10:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0129000000000001e-05 2024-09-13T10:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765715+0000", 2024-09-13T10:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0469999999999994e-06 2024-09-13T10:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773032+0000", 2024-09-13T10:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073166680000000001 2024-09-13T10:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773169+0000", 2024-09-13T10:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136964 2024-09-13T10:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779726+0000", 2024-09-13T10:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065572499999999997 2024-09-13T10:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779851+0000", 2024-09-13T10:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012477500000000001 2024-09-13T10:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.780003+0000", 2024-09-13T10:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015165299999999999 2024-09-13T10:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2113 4.17 4:e81d9ad0:::smithi19530714-4:head [write 1410296~761389] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T10:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:15.709580+0000", 2024-09-13T10:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.229629461999998, 2024-09-13T10:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012946568, 2024-09-13T10:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2113 2024-09-13T10:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.709580+0000", 2024-09-13T10:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.709580+0000", 2024-09-13T10:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.709582+0000", 2024-09-13T10:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4530000000000001e-06 2024-09-13T10:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710399+0000", 2024-09-13T10:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081639799999999997 2024-09-13T10:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710428+0000", 2024-09-13T10:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8736000000000001e-05 2024-09-13T10:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710443+0000", 2024-09-13T10:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5216e-05 2024-09-13T10:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710572+0000", 2024-09-13T10:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012882799999999999 2024-09-13T10:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710808+0000", 2024-09-13T10:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023585500000000001 2024-09-13T10:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.711245+0000", 2024-09-13T10:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043798099999999999 2024-09-13T10:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.716561+0000", 2024-09-13T10:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053156760000000001 2024-09-13T10:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.718008+0000", 2024-09-13T10:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014466380000000001 2024-09-13T10:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.722191+0000", 2024-09-13T10:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041829229999999998 2024-09-13T10:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.722439+0000", 2024-09-13T10:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024823899999999998 2024-09-13T10:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.722527+0000", 2024-09-13T10:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7625000000000002e-05 2024-09-13T10:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2412 4.7 4:e55883d8:::smithi19530714-17:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e63)", 2024-09-13T10:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:22.233147+0000", 2024-09-13T10:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.706062496000001, 2024-09-13T10:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042005107999999999, 2024-09-13T10:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2412 2024-09-13T10:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233147+0000", 2024-09-13T10:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233147+0000", 2024-09-13T10:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233150+0000", 2024-09-13T10:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0029999999999999e-06 2024-09-13T10:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233162+0000", 2024-09-13T10:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2449000000000001e-05 2024-09-13T10:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233195+0000", 2024-09-13T10:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2455e-05 2024-09-13T10:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233213+0000", 2024-09-13T10:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8185999999999998e-05 2024-09-13T10:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233351+0000", 2024-09-13T10:11:49.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000137942 2024-09-13T10:11:49.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233497+0000", 2024-09-13T10:11:49.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014627100000000001 2024-09-13T10:11:49.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.054 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.275152+0000", 2024-09-13T10:11:49.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041654801999999998 2024-09-13T10:11:49.054 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.054 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.054 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.054 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2766 4.17 4:e81d9ad0:::smithi19530714-4:head [sparse-read 1791986~224887,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-13T10:11:49.054 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:32.706469+0000", 2024-09-13T10:11:49.054 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.232740097000001, 2024-09-13T10:11:49.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.186036496, 2024-09-13T10:11:49.054 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.055 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.055 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:49.055 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.055 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.055 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.055 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2766 2024-09-13T10:11:49.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.055 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706469+0000", 2024-09-13T10:11:49.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706469+0000", 2024-09-13T10:11:49.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706472+0000", 2024-09-13T10:11:49.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0800000000000002e-06 2024-09-13T10:11:49.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706483+0000", 2024-09-13T10:11:49.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0386e-05 2024-09-13T10:11:49.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706522+0000", 2024-09-13T10:11:49.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8927999999999997e-05 2024-09-13T10:11:49.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706544+0000", 2024-09-13T10:11:49.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2436e-05 2024-09-13T10:11:49.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706732+0000", 2024-09-13T10:11:49.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000187715 2024-09-13T10:11:49.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:49.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706767+0000", 2024-09-13T10:11:49.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5462e-05 2024-09-13T10:11:49.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.891841+0000", 2024-09-13T10:11:49.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.185074035 2024-09-13T10:11:49.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.892071+0000", 2024-09-13T10:11:49.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000229643 2024-09-13T10:11:49.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.892506+0000", 2024-09-13T10:11:49.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043481100000000001 2024-09-13T10:11:49.059 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.060 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.060 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.060 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2767 4.17 4:e81d9ad0:::smithi19530714-4:head [sparse-read 0~1606736] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-13T10:11:49.060 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:32.706596+0000", 2024-09-13T10:11:49.060 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.232613228999998, 2024-09-13T10:11:49.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18875979000000001, 2024-09-13T10:11:49.060 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.060 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.060 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:49.060 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.060 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.061 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.061 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2767 2024-09-13T10:11:49.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.061 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706596+0000", 2024-09-13T10:11:49.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706596+0000", 2024-09-13T10:11:49.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706598+0000", 2024-09-13T10:11:49.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5659999999999999e-06 2024-09-13T10:11:49.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706606+0000", 2024-09-13T10:11:49.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1899999999999995e-06 2024-09-13T10:11:49.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706624+0000", 2024-09-13T10:11:49.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7546000000000001e-05 2024-09-13T10:11:49.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706639+0000", 2024-09-13T10:11:49.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5126000000000001e-05 2024-09-13T10:11:49.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706840+0000", 2024-09-13T10:11:49.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020117099999999999 2024-09-13T10:11:49.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:49.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706860+0000", 2024-09-13T10:11:49.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0205999999999999e-05 2024-09-13T10:11:49.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.893234+0000", 2024-09-13T10:11:49.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18637414799999999 2024-09-13T10:11:49.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.893485+0000", 2024-09-13T10:11:49.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025044200000000003 2024-09-13T10:11:49.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.895356+0000", 2024-09-13T10:11:49.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018713950000000001 2024-09-13T10:11:49.065 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.065 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.065 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.065 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2768 4.17 4:e81d9ad0:::smithi19530714-4:head [sparse-read 0~1606736] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-13T10:11:49.065 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:32.706665+0000", 2024-09-13T10:11:49.066 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.232544495999999, 2024-09-13T10:11:49.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18904678599999999, 2024-09-13T10:11:49.066 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.066 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.066 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:49.066 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.066 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.066 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.066 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2768 2024-09-13T10:11:49.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.066 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706665+0000", 2024-09-13T10:11:49.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706665+0000", 2024-09-13T10:11:49.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706666+0000", 2024-09-13T10:11:49.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.297e-06 2024-09-13T10:11:49.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706676+0000", 2024-09-13T10:11:49.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6090000000000007e-06 2024-09-13T10:11:49.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706692+0000", 2024-09-13T10:11:49.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5758999999999999e-05 2024-09-13T10:11:49.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706706+0000", 2024-09-13T10:11:49.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.482e-05 2024-09-13T10:11:49.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706925+0000", 2024-09-13T10:11:49.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000218607 2024-09-13T10:11:49.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:49.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706945+0000", 2024-09-13T10:11:49.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9466e-05 2024-09-13T10:11:49.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.895401+0000", 2024-09-13T10:11:49.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18845690700000001 2024-09-13T10:11:49.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.895464+0000", 2024-09-13T10:11:49.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2434999999999994e-05 2024-09-13T10:11:49.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.895712+0000", 2024-09-13T10:11:49.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000247886 2024-09-13T10:11:49.071 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.071 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.071 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.071 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.071 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.071 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:49.073 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T10:11:49.074 INFO:tasks.rados.rados.0.smithi195.stdout:1529: writing smithi19530714-4 from 2729279 to 3169419 tid 3 2024-09-13T10:11:49.074 INFO:tasks.rados.rados.0.smithi195.stdout:1530: write oid 46 current snap is 0 2024-09-13T10:11:49.075 INFO:tasks.rados.rados.0.smithi195.stdout:1530: seq_num 314 ranges {477398=798659,1774640=663444,2873055=404343} 2024-09-13T10:11:49.075 INFO:tasks.rados.rados.0.smithi195.stdout:1530: writing smithi19530714-46 from 477398 to 1276057 tid 1 2024-09-13T10:11:49.075 INFO:tasks.rados.rados.0.smithi195.stdout:1530: writing smithi19530714-46 from 1774640 to 2438084 tid 2 2024-09-13T10:11:49.075 INFO:tasks.rados.rados.0.smithi195.stdout:1530: writing smithi19530714-46 from 2873055 to 3277398 tid 3 2024-09-13T10:11:49.075 INFO:tasks.rados.rados.0.smithi195.stdout:1531: read oid 13 snap -1 2024-09-13T10:11:49.075 INFO:tasks.rados.rados.0.smithi195.stdout:1531: expect (ObjNum 308 snap 0 seq_num 308) 2024-09-13T10:11:49.075 INFO:tasks.rados.rados.0.smithi195.stdout:1532: write oid 20 current snap is 0 2024-09-13T10:11:49.075 INFO:tasks.rados.rados.0.smithi195.stdout:1532: seq_num 315 ranges {494205=473908,1433122=407102,2384125=462684} 2024-09-13T10:11:49.075 INFO:tasks.rados.rados.0.smithi195.stdout:1532: writing smithi19530714-20 from 494205 to 968113 tid 1 2024-09-13T10:11:49.075 INFO:tasks.rados.rados.0.smithi195.stdout:1532: writing smithi19530714-20 from 1433122 to 1840224 tid 2 2024-09-13T10:11:49.075 INFO:tasks.rados.rados.0.smithi195.stdout:1532: writing smithi19530714-20 from 2384125 to 2846809 tid 3 2024-09-13T10:11:49.075 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:49.075 INFO:tasks.rados.rados.0.smithi195.stdout:1516: expect (ObjNum 269 snap 0 seq_num 269) 2024-09-13T10:11:49.075 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:49.076 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:49.067+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:49.098 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:11:49.098 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:11:49.098 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:11:49.098 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-13T10:11:49.099 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T10:11:49.100 INFO:tasks.rados.rados.0.smithi195.stdout:1518: finishing set_chunk to oid 31 2024-09-13T10:11:49.100 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 31 set_chunk returned error code -95 offset: 1251328 length: 952832 2024-09-13T10:11:49.100 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 0 (ObjNum 272 snap 0 seq_num 272) dirty exists 2024-09-13T10:11:49.100 INFO:tasks.rados.rados.0.smithi195.stdout:1519: finishing write tid 1 to smithi19530714-30 2024-09-13T10:11:49.100 INFO:tasks.rados.rados.0.smithi195.stdout:1521: expect (ObjNum 275 snap 0 seq_num 275) 2024-09-13T10:11:49.173 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:49.170+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:49.211 INFO:tasks.rados.rados.0.smithi195.stdout:1520: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:11:49.211 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:49.245 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:49.245 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:49.245 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:11:49.246 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:49.260 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T10:11:49.275 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:49.271+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:49.375 INFO:tasks.rados.rados.0.smithi195.stdout:1519: finishing write tid 2 to smithi19530714-30 2024-09-13T10:11:49.375 INFO:tasks.rados.rados.0.smithi195.stdout:1519: finishing write tid 3 to smithi19530714-30 2024-09-13T10:11:49.376 INFO:tasks.rados.rados.0.smithi195.stdout:1519: finishing write tid 4 to smithi19530714-30 2024-09-13T10:11:49.376 INFO:tasks.rados.rados.0.smithi195.stdout:1519: oid 30 updating version 0 to 39 2024-09-13T10:11:49.376 INFO:tasks.rados.rados.0.smithi195.stdout:1519: oid 30 updating version 39 to 41 2024-09-13T10:11:49.376 INFO:tasks.rados.rados.0.smithi195.stdout:1519: oid 30 version 41 is already newer than 40 2024-09-13T10:11:49.376 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 41 (ObjNum 309 snap 0 seq_num 309) dirty exists 2024-09-13T10:11:49.376 INFO:tasks.rados.rados.0.smithi195.stdout:1519: left oid 30 (ObjNum 309 snap 0 seq_num 309) 2024-09-13T10:11:49.377 INFO:tasks.rados.rados.0.smithi195.stdout:1522: expect (ObjNum 298 snap 0 seq_num 298) 2024-09-13T10:11:49.377 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 380175 2024-09-13T10:11:49.377 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:49.371+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:49.378 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:49.378 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:49.379 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:11:49.379 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:11:49.379 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:49.393 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T10:11:49.420 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:11:49.420 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:49.420 INFO:teuthology.orchestra.run.smithi073.stdout: "epoch": 87, 2024-09-13T10:11:49.421 INFO:teuthology.orchestra.run.smithi073.stdout: "fsid": "c1e91d47-9492-4461-a043-8698fd74ccf3", 2024-09-13T10:11:49.421 INFO:teuthology.orchestra.run.smithi073.stdout: "created": "2024-09-13T10:10:14.019912+0000", 2024-09-13T10:11:49.421 INFO:teuthology.orchestra.run.smithi073.stdout: "modified": "2024-09-13T10:11:48.702008+0000", 2024-09-13T10:11:49.421 INFO:teuthology.orchestra.run.smithi073.stdout: "last_up_change": "2024-09-13T10:10:28.089885+0000", 2024-09-13T10:11:49.421 INFO:teuthology.orchestra.run.smithi073.stdout: "last_in_change": "2024-09-13T10:10:18.489546+0000", 2024-09-13T10:11:49.422 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T10:11:49.422 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_num": 5804032, 2024-09-13T10:11:49.422 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_set": [ 2024-09-13T10:11:49.422 INFO:teuthology.orchestra.run.smithi073.stdout: "nodeep-scrub", 2024-09-13T10:11:49.422 INFO:teuthology.orchestra.run.smithi073.stdout: "pglog_hardlimit", 2024-09-13T10:11:49.423 INFO:teuthology.orchestra.run.smithi073.stdout: "purged_snapdirs", 2024-09-13T10:11:49.423 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_deletes", 2024-09-13T10:11:49.423 INFO:teuthology.orchestra.run.smithi073.stdout: "sortbitwise" 2024-09-13T10:11:49.423 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:49.423 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_version": 6, 2024-09-13T10:11:49.423 INFO:teuthology.orchestra.run.smithi073.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T10:11:49.424 INFO:teuthology.orchestra.run.smithi073.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T10:11:49.424 INFO:teuthology.orchestra.run.smithi073.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T10:11:49.424 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_snapshot": "", 2024-09-13T10:11:49.424 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_max": 4, 2024-09-13T10:11:49.424 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd": 16, 2024-09-13T10:11:49.425 INFO:teuthology.orchestra.run.smithi073.stdout: "require_min_compat_client": "luminous", 2024-09-13T10:11:49.425 INFO:teuthology.orchestra.run.smithi073.stdout: "min_compat_client": "luminous", 2024-09-13T10:11:49.425 INFO:teuthology.orchestra.run.smithi073.stdout: "require_osd_release": "squid", 2024-09-13T10:11:49.425 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_crimson": false, 2024-09-13T10:11:49.425 INFO:teuthology.orchestra.run.smithi073.stdout: "pools": [ 2024-09-13T10:11:49.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.426 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 1, 2024-09-13T10:11:49.426 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "rbd", 2024-09-13T10:11:49.426 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:28.924860+0000", 2024-09-13T10:11:49.426 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-13T10:11:49.427 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T10:11:49.427 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:11:49.427 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:11:49.427 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:11:49.427 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:11:49.427 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:11:49.428 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:11:49.428 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:11:49.428 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:11:49.428 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:11:49.428 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:11:49.429 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:11:49.429 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 8, 2024-09-13T10:11:49.429 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 8, 2024-09-13T10:11:49.429 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 8, 2024-09-13T10:11:49.429 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 8, 2024-09-13T10:11:49.430 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 8, 2024-09-13T10:11:49.430 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:11:49.430 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:11:49.430 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:11:49.430 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:11:49.430 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:11:49.431 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:11:49.431 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:11:49.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.431 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "27", 2024-09-13T10:11:49.431 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:11:49.432 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:11:49.432 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:11:49.432 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:11:49.432 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:11:49.432 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 2, 2024-09-13T10:11:49.433 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 27, 2024-09-13T10:11:49.433 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:11:49.433 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:11:49.433 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:11:49.433 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:11:49.434 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:11:49.434 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:11:49.434 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:11:49.434 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:11:49.434 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:11:49.435 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:11:49.435 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:11:49.435 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:11:49.435 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:11:49.435 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:11:49.435 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:11:49.436 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:11:49.436 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:11:49.436 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:11:49.436 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:11:49.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.437 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:11:49.437 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:11:49.437 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:11:49.437 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:11:49.437 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:11:49.438 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:11:49.438 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:11:49.438 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:11:49.438 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:11:49.438 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:11:49.439 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:11:49.439 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-13T10:11:49.439 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-13T10:11:49.439 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-13T10:11:49.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.440 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:11:49.440 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:11:49.440 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 3.4800000190734863, 2024-09-13T10:11:49.440 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 3.4800000190734863, 2024-09-13T10:11:49.440 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.79000002145767212, 2024-09-13T10:11:49.441 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2.75, 2024-09-13T10:11:49.441 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2.75, 2024-09-13T10:11:49.441 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8372650146484375, 2024-09-13T10:11:49.441 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:11:49.441 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.8399999737739563 2024-09-13T10:11:49.442 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.442 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-13T10:11:49.442 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": ".mgr", 2024-09-13T10:11:49.443 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:28.972519+0000", 2024-09-13T10:11:49.443 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-13T10:11:49.443 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-13T10:11:49.443 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:11:49.443 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:11:49.443 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:11:49.444 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:11:49.444 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:11:49.444 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:11:49.444 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:11:49.444 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:11:49.445 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:11:49.445 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:11:49.445 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:11:49.445 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 1, 2024-09-13T10:11:49.445 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 1, 2024-09-13T10:11:49.446 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 1, 2024-09-13T10:11:49.446 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 1, 2024-09-13T10:11:49.446 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 1, 2024-09-13T10:11:49.446 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:11:49.446 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:11:49.447 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:11:49.447 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:11:49.447 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:11:49.447 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:11:49.447 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:11:49.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.448 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "26", 2024-09-13T10:11:49.448 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:11:49.448 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:11:49.448 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:11:49.449 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:11:49.449 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:11:49.449 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-13T10:11:49.449 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-13T10:11:49.449 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:11:49.450 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:11:49.450 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:11:49.450 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:11:49.450 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:11:49.450 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:11:49.450 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:11:49.451 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:11:49.451 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:11:49.451 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:11:49.451 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:11:49.451 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:11:49.452 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:11:49.452 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:11:49.452 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:11:49.452 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:11:49.452 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:11:49.453 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:11:49.453 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:11:49.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.453 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:11:49.453 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:11:49.453 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:11:49.454 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:11:49.454 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:11:49.454 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:11:49.454 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:11:49.454 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:11:49.455 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:11:49.455 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:11:49.455 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:11:49.455 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-13T10:11:49.455 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_max": 32, 2024-09-13T10:11:49.456 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_min": 1 2024-09-13T10:11:49.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.456 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-13T10:11:49.456 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": {} 2024-09-13T10:11:49.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.457 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:11:49.457 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:11:49.457 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 14.289999961853027, 2024-09-13T10:11:49.457 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 14.289999961853027, 2024-09-13T10:11:49.457 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.20999999344348907, 2024-09-13T10:11:49.457 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 3, 2024-09-13T10:11:49.458 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 3, 2024-09-13T10:11:49.458 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8372650146484375, 2024-09-13T10:11:49.458 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:11:49.458 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.8399999737739563 2024-09-13T10:11:49.458 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.459 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-13T10:11:49.459 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "low_tier", 2024-09-13T10:11:49.459 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:41.320719+0000", 2024-09-13T10:11:49.460 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-13T10:11:49.460 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-13T10:11:49.460 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:11:49.460 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:11:49.460 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:11:49.461 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:11:49.461 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:11:49.461 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:11:49.461 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:11:49.461 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:11:49.462 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:11:49.462 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:11:49.462 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:11:49.462 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 4, 2024-09-13T10:11:49.462 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 4, 2024-09-13T10:11:49.462 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 4, 2024-09-13T10:11:49.463 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 4, 2024-09-13T10:11:49.463 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 4, 2024-09-13T10:11:49.463 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:11:49.463 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:11:49.463 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:11:49.464 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:11:49.464 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:11:49.464 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:11:49.464 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:11:49.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.465 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "32", 2024-09-13T10:11:49.465 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:11:49.465 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:11:49.465 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:11:49.465 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:11:49.466 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:11:49.466 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-13T10:11:49.466 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-13T10:11:49.466 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:11:49.466 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:11:49.467 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:11:49.467 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:11:49.467 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:11:49.467 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:11:49.467 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:11:49.468 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:11:49.468 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:11:49.468 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:11:49.468 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:11:49.468 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:11:49.469 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:11:49.469 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:11:49.469 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:11:49.469 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:11:49.469 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:11:49.469 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:11:49.469 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:11:49.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.470 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:11:49.470 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:11:49.470 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:11:49.470 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:11:49.470 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:11:49.470 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:11:49.470 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:11:49.471 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:11:49.471 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:11:49.471 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:11:49.471 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:11:49.471 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-13T10:11:49.471 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": {}, 2024-09-13T10:11:49.471 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:11:49.472 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:11:49.472 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 3.5199999809265137, 2024-09-13T10:11:49.472 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 3.5199999809265137, 2024-09-13T10:11:49.472 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.63999998569488525, 2024-09-13T10:11:49.472 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2.25, 2024-09-13T10:11:49.472 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2.25, 2024-09-13T10:11:49.472 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8372650146484375, 2024-09-13T10:11:49.472 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:11:49.472 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.8399999737739563 2024-09-13T10:11:49.472 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.473 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 4, 2024-09-13T10:11:49.473 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "unique_pool_0", 2024-09-13T10:11:49.473 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:42.575395+0000", 2024-09-13T10:11:49.473 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-13T10:11:49.473 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-13T10:11:49.473 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:11:49.473 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:11:49.473 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:11:49.473 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:11:49.473 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:11:49.473 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:11:49.474 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:11:49.474 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:11:49.474 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:11:49.474 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:11:49.474 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:11:49.474 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 26, 2024-09-13T10:11:49.474 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 26, 2024-09-13T10:11:49.474 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 26, 2024-09-13T10:11:49.474 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 26, 2024-09-13T10:11:49.474 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 26, 2024-09-13T10:11:49.474 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:11:49.474 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:11:49.474 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:11:49.475 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:11:49.475 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:11:49.475 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:11:49.475 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:11:49.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.475 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "82", 2024-09-13T10:11:49.475 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:11:49.475 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:11:49.475 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "41", 2024-09-13T10:11:49.475 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:11:49.475 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:11:49.475 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-13T10:11:49.475 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-13T10:11:49.476 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:11:49.476 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:11:49.476 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:11:49.476 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:11:49.476 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:11:49.476 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:11:49.476 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:11:49.476 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:11:49.476 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:11:49.476 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:11:49.476 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:11:49.476 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:11:49.477 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:11:49.477 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:11:49.477 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:11:49.477 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:11:49.477 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:11:49.477 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:11:49.477 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:11:49.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.477 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:11:49.477 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:11:49.477 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:11:49.477 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:11:49.478 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:11:49.478 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:11:49.478 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:11:49.478 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:11:49.478 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:11:49.478 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:11:49.478 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:11:49.478 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-13T10:11:49.478 INFO:teuthology.orchestra.run.smithi073.stdout: "dedup_cdc_chunk_size": 131072, 2024-09-13T10:11:49.478 INFO:teuthology.orchestra.run.smithi073.stdout: "dedup_chunk_algorithm": "fastcdc", 2024-09-13T10:11:49.478 INFO:teuthology.orchestra.run.smithi073.stdout: "dedup_tier": 3, 2024-09-13T10:11:49.478 INFO:teuthology.orchestra.run.smithi073.stdout: "fingerprint_algorithm": "sha256" 2024-09-13T10:11:49.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.479 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-13T10:11:49.479 INFO:teuthology.orchestra.run.smithi073.stdout: "rados": {} 2024-09-13T10:11:49.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.479 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:11:49.479 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:11:49.479 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 3.2400000095367432, 2024-09-13T10:11:49.479 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 3.2400000095367432, 2024-09-13T10:11:49.479 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 1.1399999856948853, 2024-09-13T10:11:49.479 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 3.690000057220459, 2024-09-13T10:11:49.479 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 3.690000057220459, 2024-09-13T10:11:49.479 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8372650146484375, 2024-09-13T10:11:49.479 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:11:49.479 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.8399999737739563 2024-09-13T10:11:49.480 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.480 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.480 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:49.480 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:49.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.480 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-13T10:11:49.480 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "f7d92188-c27c-4744-a330-4b7105e0f2d6", 2024-09-13T10:11:49.480 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:49.480 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:49.480 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:49.480 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:49.480 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:49.481 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:49.481 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:49.481 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 79, 2024-09-13T10:11:49.481 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:49.481 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:49.481 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:49.481 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.481 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.481 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6812", 2024-09-13T10:11:49.481 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:11:49.481 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.481 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.482 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:49.482 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.482 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.482 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6813", 2024-09-13T10:11:49.482 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:11:49.482 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.482 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.482 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:49.482 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.483 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.483 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6815", 2024-09-13T10:11:49.483 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:11:49.483 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.483 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.483 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:49.483 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.483 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.483 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6814", 2024-09-13T10:11:49.483 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:11:49.484 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.484 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.484 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6812/1977392321", 2024-09-13T10:11:49.484 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6813/1977392321", 2024-09-13T10:11:49.484 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6815/1977392321", 2024-09-13T10:11:49.484 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6814/1977392321", 2024-09-13T10:11:49.484 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:49.484 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:49.484 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:49.484 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.485 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-13T10:11:49.485 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e900b771-7c96-4b5d-93d7-35a8ea6c026a", 2024-09-13T10:11:49.485 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:49.485 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:49.485 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:49.485 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:49.485 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:49.485 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:49.485 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:49.485 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 72, 2024-09-13T10:11:49.485 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:49.485 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:49.485 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:49.486 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.486 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.486 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6808", 2024-09-13T10:11:49.486 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:11:49.486 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.486 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.486 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:49.486 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.486 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.487 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6809", 2024-09-13T10:11:49.487 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:11:49.487 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.487 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.487 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:49.487 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.487 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.487 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6811", 2024-09-13T10:11:49.487 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:11:49.487 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.487 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.488 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:49.488 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.488 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.488 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6810", 2024-09-13T10:11:49.488 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:11:49.488 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.488 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.488 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6808/1228102864", 2024-09-13T10:11:49.489 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6809/1228102864", 2024-09-13T10:11:49.489 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6811/1228102864", 2024-09-13T10:11:49.489 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6810/1228102864", 2024-09-13T10:11:49.489 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:49.489 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:49.489 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:49.489 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.489 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-13T10:11:49.489 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "28b7aa59-33b2-4e24-8f3d-758f90e25f68", 2024-09-13T10:11:49.489 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:49.490 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:49.490 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:49.490 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:49.490 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:49.490 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:49.490 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:49.490 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 76, 2024-09-13T10:11:49.490 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:49.490 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:49.490 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:49.490 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.491 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.491 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6800", 2024-09-13T10:11:49.491 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:11:49.491 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.491 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.491 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:49.491 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.491 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.492 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6801", 2024-09-13T10:11:49.492 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:11:49.492 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.492 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.492 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:49.492 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.492 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.492 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6803", 2024-09-13T10:11:49.492 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:11:49.492 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.493 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.493 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:49.493 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.493 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.493 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6802", 2024-09-13T10:11:49.496 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:11:49.496 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.496 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.497 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6800/3314820936", 2024-09-13T10:11:49.497 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6801/3314820936", 2024-09-13T10:11:49.497 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6803/3314820936", 2024-09-13T10:11:49.497 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6802/3314820936", 2024-09-13T10:11:49.497 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:49.497 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:49.497 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:49.497 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.497 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-13T10:11:49.497 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "a93d2df5-781e-49e1-a507-1718b6f954ed", 2024-09-13T10:11:49.498 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:49.498 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:49.498 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:49.498 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:49.498 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:49.498 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:49.498 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:49.498 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 82, 2024-09-13T10:11:49.498 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:49.498 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:49.498 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:49.498 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.499 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.499 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6801", 2024-09-13T10:11:49.499 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:11:49.499 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.499 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.499 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:49.499 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.499 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.499 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6802", 2024-09-13T10:11:49.499 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:11:49.500 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.500 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.500 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:49.500 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.500 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.500 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6804", 2024-09-13T10:11:49.500 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:11:49.500 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.500 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.500 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:49.501 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.501 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.501 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6803", 2024-09-13T10:11:49.501 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:11:49.501 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.501 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.501 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6801/2558565460", 2024-09-13T10:11:49.501 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6802/2558565460", 2024-09-13T10:11:49.501 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6804/2558565460", 2024-09-13T10:11:49.501 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6803/2558565460", 2024-09-13T10:11:49.501 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:49.502 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:49.502 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:49.502 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.502 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-13T10:11:49.502 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e69053fd-f6d9-437e-a9d0-8896de6b927f", 2024-09-13T10:11:49.502 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:49.502 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:49.502 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:49.502 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:49.502 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:49.503 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:49.503 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:49.503 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 76, 2024-09-13T10:11:49.503 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:49.503 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:49.503 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:49.503 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.503 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.503 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6800", 2024-09-13T10:11:49.503 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:11:49.503 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.503 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.504 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:49.504 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.504 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.504 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6801", 2024-09-13T10:11:49.504 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:11:49.504 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.504 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.504 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:49.504 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.505 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.505 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6803", 2024-09-13T10:11:49.505 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:11:49.505 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.505 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.505 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:49.505 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.505 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.505 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6802", 2024-09-13T10:11:49.505 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:11:49.506 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.506 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.506 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6800/858330493", 2024-09-13T10:11:49.506 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6801/858330493", 2024-09-13T10:11:49.506 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6803/858330493", 2024-09-13T10:11:49.506 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6802/858330493", 2024-09-13T10:11:49.506 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:49.506 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:49.506 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:49.506 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.507 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-13T10:11:49.507 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "3667ec10-561e-4a23-8197-6357db4af315", 2024-09-13T10:11:49.507 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:49.507 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:49.507 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:49.507 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:49.507 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:49.507 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:49.507 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:49.507 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 84, 2024-09-13T10:11:49.508 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:49.508 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:49.508 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:49.508 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.508 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.508 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6804", 2024-09-13T10:11:49.508 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:11:49.508 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.508 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.508 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:49.509 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.509 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.509 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6805", 2024-09-13T10:11:49.509 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:11:49.509 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.509 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.510 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:49.510 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.510 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.510 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6807", 2024-09-13T10:11:49.510 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:11:49.510 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.510 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.510 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:49.510 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.511 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.511 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6806", 2024-09-13T10:11:49.511 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:11:49.511 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.511 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.511 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6804/984313481", 2024-09-13T10:11:49.511 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6805/984313481", 2024-09-13T10:11:49.511 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6807/984313481", 2024-09-13T10:11:49.511 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6806/984313481", 2024-09-13T10:11:49.511 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:49.511 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:49.511 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:49.512 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.512 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-13T10:11:49.512 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "30d4d63c-f04f-4423-91a6-816a7cbc0287", 2024-09-13T10:11:49.512 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:49.512 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:49.512 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:49.512 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:49.512 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:49.512 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:49.512 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 79, 2024-09-13T10:11:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6812", 2024-09-13T10:11:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:11:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.513 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6813", 2024-09-13T10:11:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:11:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.514 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.515 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6815", 2024-09-13T10:11:49.515 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:11:49.515 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.515 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.515 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:49.515 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.515 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.515 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6814", 2024-09-13T10:11:49.515 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:11:49.515 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.515 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.516 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6812/1505939978", 2024-09-13T10:11:49.516 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6813/1505939978", 2024-09-13T10:11:49.516 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6815/1505939978", 2024-09-13T10:11:49.516 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6814/1505939978", 2024-09-13T10:11:49.516 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:49.516 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:49.516 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:49.516 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.517 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-13T10:11:49.517 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "48812f05-82f1-4263-8ec5-4861a6e91b61", 2024-09-13T10:11:49.517 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:49.517 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:49.517 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:49.517 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0, 2024-09-13T10:11:49.517 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:49.517 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:49.517 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:49.517 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 41, 2024-09-13T10:11:49.517 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:49.517 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:49.517 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:49.518 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.518 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.518 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6809", 2024-09-13T10:11:49.518 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:11:49.518 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.518 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.518 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:49.518 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.519 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.519 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6810", 2024-09-13T10:11:49.519 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:11:49.519 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.519 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.519 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:49.519 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.519 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.519 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6812", 2024-09-13T10:11:49.520 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:11:49.520 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.520 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.520 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:49.520 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.520 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.520 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6811", 2024-09-13T10:11:49.520 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:11:49.520 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.520 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.521 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6809/3679292757", 2024-09-13T10:11:49.521 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6810/3679292757", 2024-09-13T10:11:49.521 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6812/3679292757", 2024-09-13T10:11:49.521 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6811/3679292757", 2024-09-13T10:11:49.521 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:49.521 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:49.521 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:49.521 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.521 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 8, 2024-09-13T10:11:49.521 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "72056e61-d04b-4fd3-94bf-57ee3df4b466", 2024-09-13T10:11:49.522 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:49.522 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:49.522 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:49.522 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0.396240234375, 2024-09-13T10:11:49.522 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:49.522 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:49.522 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:49.522 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 31, 2024-09-13T10:11:49.522 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:49.522 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:49.522 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:49.522 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.523 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.523 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6804", 2024-09-13T10:11:49.523 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:11:49.523 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.523 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.523 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:49.523 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.523 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.523 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6805", 2024-09-13T10:11:49.524 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:11:49.524 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.524 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.524 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:49.524 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.524 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.524 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6807", 2024-09-13T10:11:49.525 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:11:49.525 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.525 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.525 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:49.525 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.525 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.525 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6806", 2024-09-13T10:11:49.525 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:11:49.525 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.525 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.526 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6804/4052106885", 2024-09-13T10:11:49.526 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6805/4052106885", 2024-09-13T10:11:49.526 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6807/4052106885", 2024-09-13T10:11:49.526 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6806/4052106885", 2024-09-13T10:11:49.526 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:49.526 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:49.526 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:49.526 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.526 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 9, 2024-09-13T10:11:49.527 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "20e7f118-7148-4c27-9ec4-a33954709515", 2024-09-13T10:11:49.527 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:49.527 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:49.527 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:49.527 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:49.527 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:49.527 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:49.527 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 21, 2024-09-13T10:11:49.527 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 77, 2024-09-13T10:11:49.527 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:49.527 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:49.528 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:49.528 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.528 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.528 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6800", 2024-09-13T10:11:49.528 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:11:49.528 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.528 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.528 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:49.528 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.528 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.529 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6801", 2024-09-13T10:11:49.529 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:11:49.529 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.529 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.529 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:49.529 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.529 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.529 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6803", 2024-09-13T10:11:49.529 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:11:49.529 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.530 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.530 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:49.530 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.530 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.530 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6802", 2024-09-13T10:11:49.530 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:11:49.530 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.530 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.530 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6800/4266571820", 2024-09-13T10:11:49.530 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6801/4266571820", 2024-09-13T10:11:49.531 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6803/4266571820", 2024-09-13T10:11:49.531 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6802/4266571820", 2024-09-13T10:11:49.531 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:49.531 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:49.531 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:49.531 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.531 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 10, 2024-09-13T10:11:49.531 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "012409e8-51c2-4584-aba7-95996f0d72ad", 2024-09-13T10:11:49.531 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:49.531 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:49.531 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:49.532 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:49.532 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:49.532 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:49.532 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:49.532 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 80, 2024-09-13T10:11:49.532 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:49.532 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:49.532 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:49.532 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.532 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.532 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6808", 2024-09-13T10:11:49.532 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:11:49.533 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.533 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.533 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:49.533 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.533 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.533 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6809", 2024-09-13T10:11:49.533 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:11:49.533 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.533 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.534 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:49.534 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.534 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.534 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6811", 2024-09-13T10:11:49.534 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:11:49.534 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.534 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.534 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:49.534 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.535 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.535 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6810", 2024-09-13T10:11:49.535 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:11:49.535 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.535 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.535 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6808/958255259", 2024-09-13T10:11:49.535 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6809/958255259", 2024-09-13T10:11:49.535 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6811/958255259", 2024-09-13T10:11:49.535 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6810/958255259", 2024-09-13T10:11:49.535 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:49.535 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:49.536 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:49.536 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.536 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 11, 2024-09-13T10:11:49.536 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "88b137b1-7d30-45f7-ac5c-ba026704ef24", 2024-09-13T10:11:49.536 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:49.536 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:49.536 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:49.536 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:49.536 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:49.536 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:49.537 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:49.537 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 85, 2024-09-13T10:11:49.537 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:49.537 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:49.537 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:49.537 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.537 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.537 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6813", 2024-09-13T10:11:49.537 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:11:49.537 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.537 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.538 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:49.538 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.538 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.538 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6814", 2024-09-13T10:11:49.538 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:11:49.538 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.538 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.538 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:49.538 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.538 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.539 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6816", 2024-09-13T10:11:49.539 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:11:49.539 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.539 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.539 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:49.539 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.539 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.539 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6815", 2024-09-13T10:11:49.539 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:11:49.539 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.539 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.540 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6813/281884267", 2024-09-13T10:11:49.540 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6814/281884267", 2024-09-13T10:11:49.540 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6816/281884267", 2024-09-13T10:11:49.540 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6815/281884267", 2024-09-13T10:11:49.540 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:49.540 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:49.540 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:49.540 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.540 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 12, 2024-09-13T10:11:49.541 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "b54732a2-9dff-42e1-b4ef-281cbca3ad7c", 2024-09-13T10:11:49.541 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:49.541 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:49.541 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:49.541 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0, 2024-09-13T10:11:49.541 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:49.541 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:49.541 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:49.541 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 31, 2024-09-13T10:11:49.541 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:49.541 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:49.541 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:49.541 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.542 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.542 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6808", 2024-09-13T10:11:49.542 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:11:49.542 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.542 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.542 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:49.542 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.542 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.542 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6809", 2024-09-13T10:11:49.543 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:11:49.543 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.543 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.543 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:49.543 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.543 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.543 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6811", 2024-09-13T10:11:49.543 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:11:49.543 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.543 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.544 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:49.544 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.544 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.544 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6810", 2024-09-13T10:11:49.544 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:11:49.544 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.544 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.544 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6808/3532043806", 2024-09-13T10:11:49.544 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6809/3532043806", 2024-09-13T10:11:49.545 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6811/3532043806", 2024-09-13T10:11:49.545 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6810/3532043806", 2024-09-13T10:11:49.545 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:49.545 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:49.545 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:49.545 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.545 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 13, 2024-09-13T10:11:49.545 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "20b48864-44dc-4732-8dff-d0d21cc08032", 2024-09-13T10:11:49.545 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:49.545 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:49.545 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:49.546 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:49.546 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:49.546 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:49.546 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 21, 2024-09-13T10:11:49.546 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 80, 2024-09-13T10:11:49.546 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:49.546 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:49.546 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:49.546 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.546 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.546 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6812", 2024-09-13T10:11:49.547 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:11:49.547 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.547 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.547 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:49.547 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.547 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.547 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6813", 2024-09-13T10:11:49.547 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:11:49.547 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.547 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.547 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:49.548 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.548 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.548 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6815", 2024-09-13T10:11:49.548 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:11:49.548 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.548 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.548 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:49.548 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.548 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.548 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6814", 2024-09-13T10:11:49.549 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:11:49.549 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.549 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.549 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6812/4083982131", 2024-09-13T10:11:49.549 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6813/4083982131", 2024-09-13T10:11:49.549 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6815/4083982131", 2024-09-13T10:11:49.549 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6814/4083982131", 2024-09-13T10:11:49.549 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:49.549 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:49.549 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:49.549 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.550 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.550 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 14, 2024-09-13T10:11:49.550 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "9fac3ef4-e9be-4e8b-9a21-0de35b543d65", 2024-09-13T10:11:49.550 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:49.550 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:49.550 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:49.550 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:49.550 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:49.550 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:49.550 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:11:49.550 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 66, 2024-09-13T10:11:49.550 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:49.550 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:49.551 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:49.551 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.551 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.551 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6804", 2024-09-13T10:11:49.551 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:11:49.551 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.551 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.551 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:49.551 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.551 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.551 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6805", 2024-09-13T10:11:49.552 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:11:49.552 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.552 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.552 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:49.552 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.552 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.552 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6807", 2024-09-13T10:11:49.552 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:11:49.552 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.553 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.553 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:49.553 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.553 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.553 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6806", 2024-09-13T10:11:49.553 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:11:49.553 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.553 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.553 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6804/1173379175", 2024-09-13T10:11:49.553 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6805/1173379175", 2024-09-13T10:11:49.554 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6807/1173379175", 2024-09-13T10:11:49.554 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6806/1173379175", 2024-09-13T10:11:49.554 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:49.554 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:49.554 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:49.554 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.554 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 15, 2024-09-13T10:11:49.554 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "16b34f90-3148-4d41-a403-ba0f900991e1", 2024-09-13T10:11:49.554 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:11:49.554 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:11:49.554 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:11:49.555 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:11:49.555 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:11:49.555 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:11:49.555 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:11:49.555 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 76, 2024-09-13T10:11:49.555 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:11:49.555 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:11:49.555 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:11:49.555 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.555 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.555 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6805", 2024-09-13T10:11:49.556 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:11:49.556 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.556 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.556 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:11:49.556 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.556 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.556 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6806", 2024-09-13T10:11:49.556 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:11:49.556 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.556 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.557 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:11:49.557 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.557 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.557 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6808", 2024-09-13T10:11:49.557 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:11:49.557 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.557 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.557 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:11:49.557 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:11:49.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.558 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:11:49.558 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6807", 2024-09-13T10:11:49.558 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:11:49.558 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.558 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.558 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6805/848686366", 2024-09-13T10:11:49.558 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6806/848686366", 2024-09-13T10:11:49.558 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6808/848686366", 2024-09-13T10:11:49.558 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6807/848686366", 2024-09-13T10:11:49.558 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:11:49.558 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:11:49.559 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:11:49.559 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.559 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.559 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:49.559 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_xinfo": [ 2024-09-13T10:11:49.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.559 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-13T10:11:49.559 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:49.559 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:49.559 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:49.559 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:49.559 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:49.560 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.372908+0000", 2024-09-13T10:11:49.560 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:49.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.560 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-13T10:11:49.560 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:49.560 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:49.560 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:49.560 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:49.560 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:49.560 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:45.986851+0000", 2024-09-13T10:11:49.560 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:49.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.561 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-13T10:11:49.561 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:49.561 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:49.561 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:49.561 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:49.561 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:49.561 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:47.777103+0000", 2024-09-13T10:11:49.561 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:49.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.561 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-13T10:11:49.562 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:49.562 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:49.562 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:49.562 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:49.562 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:49.562 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:47.179172+0000", 2024-09-13T10:11:49.562 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:49.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.562 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-13T10:11:49.562 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:49.562 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:49.563 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:49.563 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:49.563 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:49.563 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:10:25.066350+0000", 2024-09-13T10:11:49.563 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:49.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.563 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-13T10:11:49.563 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:49.563 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:49.563 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:49.563 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:49.564 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:49.564 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:46.147756+0000", 2024-09-13T10:11:49.564 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:49.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.564 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-13T10:11:49.564 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:49.564 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:49.564 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:49.564 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:49.564 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:49.564 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:47.809723+0000", 2024-09-13T10:11:49.565 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:49.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.565 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-13T10:11:49.565 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:49.565 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:49.565 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:49.565 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:49.565 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:49.565 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:38.606440+0000", 2024-09-13T10:11:49.565 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:49.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.566 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 8, 2024-09-13T10:11:49.566 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:49.566 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:49.566 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:49.566 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:49.566 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:49.566 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:37.606540+0000", 2024-09-13T10:11:49.567 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:49.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.567 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 9, 2024-09-13T10:11:49.567 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:49.567 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:49.567 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:49.568 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:49.568 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:49.568 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:25.726157+0000", 2024-09-13T10:11:49.568 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:49.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.568 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 10, 2024-09-13T10:11:49.568 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:49.568 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:49.569 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:49.569 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:49.569 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:49.569 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:27.329800+0000", 2024-09-13T10:11:49.569 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:49.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.569 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 11, 2024-09-13T10:11:49.569 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:49.569 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:49.569 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:49.570 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:49.570 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:49.570 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:26.722524+0000", 2024-09-13T10:11:49.570 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:49.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.570 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 12, 2024-09-13T10:11:49.570 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:49.570 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:49.570 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:49.570 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:49.570 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:49.571 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:25.954484+0000", 2024-09-13T10:11:49.571 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:49.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.571 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 13, 2024-09-13T10:11:49.571 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:49.571 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:49.571 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:49.571 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:49.571 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:49.572 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:25.900950+0000", 2024-09-13T10:11:49.572 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:49.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.572 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 14, 2024-09-13T10:11:49.572 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:49.572 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:49.572 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:49.572 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:49.572 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:49.573 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:40.864138+0000", 2024-09-13T10:11:49.573 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:49.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.573 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 15, 2024-09-13T10:11:49.573 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:11:49.573 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:11:49.573 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:11:49.573 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:11:49.573 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:11:49.574 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:48.003986+0000", 2024-09-13T10:11:49.574 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:11:49.574 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.574 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:49.574 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap": [ 2024-09-13T10:11:49.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.574 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.e", 2024-09-13T10:11:49.574 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:49.574 INFO:teuthology.orchestra.run.smithi073.stdout: 8, 2024-09-13T10:11:49.574 INFO:teuthology.orchestra.run.smithi073.stdout: 3, 2024-09-13T10:11:49.574 INFO:teuthology.orchestra.run.smithi073.stdout: 15 2024-09-13T10:11:49.574 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.574 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.575 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:49.575 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_items": [], 2024-09-13T10:11:49.575 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_primaries": [], 2024-09-13T10:11:49.575 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_temp": [ 2024-09-13T10:11:49.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:49.575 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.e", 2024-09-13T10:11:49.575 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:11:49.575 INFO:teuthology.orchestra.run.smithi073.stdout: 2, 2024-09-13T10:11:49.575 INFO:teuthology.orchestra.run.smithi073.stdout: 8, 2024-09-13T10:11:49.575 INFO:teuthology.orchestra.run.smithi073.stdout: 4 2024-09-13T10:11:49.575 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:49.575 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.575 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:11:49.576 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_temp": [], 2024-09-13T10:11:49.576 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklist": {}, 2024-09-13T10:11:49.576 INFO:teuthology.orchestra.run.smithi073.stdout: "range_blocklist": {}, 2024-09-13T10:11:49.576 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profiles": { 2024-09-13T10:11:49.576 INFO:teuthology.orchestra.run.smithi073.stdout: "default": { 2024-09-13T10:11:49.576 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-failure-domain": "osd", 2024-09-13T10:11:49.576 INFO:teuthology.orchestra.run.smithi073.stdout: "k": "2", 2024-09-13T10:11:49.576 INFO:teuthology.orchestra.run.smithi073.stdout: "m": "1", 2024-09-13T10:11:49.576 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin": "jerasure", 2024-09-13T10:11:49.576 INFO:teuthology.orchestra.run.smithi073.stdout: "technique": "reed_sol_van" 2024-09-13T10:11:49.576 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:49.577 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps_queue": [], 2024-09-13T10:11:49.577 INFO:teuthology.orchestra.run.smithi073.stdout: "new_removed_snaps": [], 2024-09-13T10:11:49.577 INFO:teuthology.orchestra.run.smithi073.stdout: "new_purged_snaps": [], 2024-09-13T10:11:49.577 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_node_flags": {}, 2024-09-13T10:11:49.577 INFO:teuthology.orchestra.run.smithi073.stdout: "device_class_flags": {}, 2024-09-13T10:11:49.577 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode": { 2024-09-13T10:11:49.577 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_enabled": false, 2024-09-13T10:11:49.577 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_bucket_count": 0, 2024-09-13T10:11:49.577 INFO:teuthology.orchestra.run.smithi073.stdout: "degraded_stretch_mode": 0, 2024-09-13T10:11:49.577 INFO:teuthology.orchestra.run.smithi073.stdout: "recovering_stretch_mode": 0, 2024-09-13T10:11:49.577 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_bucket": 0 2024-09-13T10:11:49.578 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:49.578 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:49.579 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:49.580 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:49.573+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:49.582 INFO:tasks.thrashosds.thrasher:j is {'epoch': 87, 'fsid': 'c1e91d47-9492-4461-a043-8698fd74ccf3', 'created': '2024-09-13T10:10:14.019912+0000', 'modified': '2024-09-13T10:11:48.702008+0000', 'last_up_change': '2024-09-13T10:10:28.089885+0000', 'last_in_change': '2024-09-13T10:10:18.489546+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T10:10:28.924860+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 27, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.4800000190734863, 'score_stable': 3.4800000190734863, 'optimal_score': 0.7900000214576721, 'raw_score_acting': 2.75, 'raw_score_stable': 2.75, 'primary_affinity_weighted': 0.8372650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8399999737739563}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T10:10:28.972519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 14.289999961853027, 'score_stable': 14.289999961853027, 'optimal_score': 0.20999999344348907, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.8372650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8399999737739563}}, {'pool': 3, 'pool_name': 'low_tier', 'create_time': '2024-09-13T10:10:41.320719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.5199999809265137, 'score_stable': 3.5199999809265137, 'optimal_score': 0.6399999856948853, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8372650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8399999737739563}}, {'pool': 4, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-13T10:10:42.575395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '41', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'dedup_cdc_chunk_size': 131072, 'dedup_chunk_algorithm': 'fastcdc', 'dedup_tier': 3, 'fingerprint_algorithm': 'sha256'}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.240000009536743, 'score_stable': 3.240000009536743, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 3.690000057220459, 'raw_score_stable': 3.690000057220459, 'primary_affinity_weighted': 0.8372650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8399999737739563}}], 'osds': [{'osd': 0, 'uuid': 'f7d92188-c27c-4744-a330-4b7105e0f2d6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 79, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6812', 'nonce': 1977392321}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6813', 'nonce': 1977392321}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6815', 'nonce': 1977392321}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6814', 'nonce': 1977392321}]}, 'public_addr': '172.21.15.73:6812/1977392321', 'cluster_addr': '172.21.15.73:6813/1977392321', 'heartbeat_back_addr': '172.21.15.73:6815/1977392321', 'heartbeat_front_addr': '172.21.15.73:6814/1977392321', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e900b771-7c96-4b5d-93d7-35a8ea6c026a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 72, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6808', 'nonce': 1228102864}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6809', 'nonce': 1228102864}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6811', 'nonce': 1228102864}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6810', 'nonce': 1228102864}]}, 'public_addr': '172.21.15.136:6808/1228102864', 'cluster_addr': '172.21.15.136:6809/1228102864', 'heartbeat_back_addr': '172.21.15.136:6811/1228102864', 'heartbeat_front_addr': '172.21.15.136:6810/1228102864', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '28b7aa59-33b2-4e24-8f3d-758f90e25f68', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6800', 'nonce': 3314820936}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6801', 'nonce': 3314820936}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6803', 'nonce': 3314820936}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6802', 'nonce': 3314820936}]}, 'public_addr': '172.21.15.181:6800/3314820936', 'cluster_addr': '172.21.15.181:6801/3314820936', 'heartbeat_back_addr': '172.21.15.181:6803/3314820936', 'heartbeat_front_addr': '172.21.15.181:6802/3314820936', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a93d2df5-781e-49e1-a507-1718b6f954ed', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6801', 'nonce': 2558565460}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6802', 'nonce': 2558565460}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6804', 'nonce': 2558565460}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6803', 'nonce': 2558565460}]}, 'public_addr': '172.21.15.195:6801/2558565460', 'cluster_addr': '172.21.15.195:6802/2558565460', 'heartbeat_back_addr': '172.21.15.195:6804/2558565460', 'heartbeat_front_addr': '172.21.15.195:6803/2558565460', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e69053fd-f6d9-437e-a9d0-8896de6b927f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6800', 'nonce': 858330493}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6801', 'nonce': 858330493}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6803', 'nonce': 858330493}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6802', 'nonce': 858330493}]}, 'public_addr': '172.21.15.73:6800/858330493', 'cluster_addr': '172.21.15.73:6801/858330493', 'heartbeat_back_addr': '172.21.15.73:6803/858330493', 'heartbeat_front_addr': '172.21.15.73:6802/858330493', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '3667ec10-561e-4a23-8197-6357db4af315', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6804', 'nonce': 984313481}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6805', 'nonce': 984313481}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6807', 'nonce': 984313481}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6806', 'nonce': 984313481}]}, 'public_addr': '172.21.15.136:6804/984313481', 'cluster_addr': '172.21.15.136:6805/984313481', 'heartbeat_back_addr': '172.21.15.136:6807/984313481', 'heartbeat_front_addr': '172.21.15.136:6806/984313481', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '30d4d63c-f04f-4423-91a6-816a7cbc0287', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 79, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6812', 'nonce': 1505939978}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6813', 'nonce': 1505939978}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6815', 'nonce': 1505939978}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6814', 'nonce': 1505939978}]}, 'public_addr': '172.21.15.181:6812/1505939978', 'cluster_addr': '172.21.15.181:6813/1505939978', 'heartbeat_back_addr': '172.21.15.181:6815/1505939978', 'heartbeat_front_addr': '172.21.15.181:6814/1505939978', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '48812f05-82f1-4263-8ec5-4861a6e91b61', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6809', 'nonce': 3679292757}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6810', 'nonce': 3679292757}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6812', 'nonce': 3679292757}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6811', 'nonce': 3679292757}]}, 'public_addr': '172.21.15.195:6809/3679292757', 'cluster_addr': '172.21.15.195:6810/3679292757', 'heartbeat_back_addr': '172.21.15.195:6812/3679292757', 'heartbeat_front_addr': '172.21.15.195:6811/3679292757', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '72056e61-d04b-4fd3-94bf-57ee3df4b466', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.396240234375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6804', 'nonce': 4052106885}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6805', 'nonce': 4052106885}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6807', 'nonce': 4052106885}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6806', 'nonce': 4052106885}]}, 'public_addr': '172.21.15.73:6804/4052106885', 'cluster_addr': '172.21.15.73:6805/4052106885', 'heartbeat_back_addr': '172.21.15.73:6807/4052106885', 'heartbeat_front_addr': '172.21.15.73:6806/4052106885', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '20e7f118-7148-4c27-9ec4-a33954709515', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6800', 'nonce': 4266571820}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6801', 'nonce': 4266571820}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6803', 'nonce': 4266571820}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6802', 'nonce': 4266571820}]}, 'public_addr': '172.21.15.136:6800/4266571820', 'cluster_addr': '172.21.15.136:6801/4266571820', 'heartbeat_back_addr': '172.21.15.136:6803/4266571820', 'heartbeat_front_addr': '172.21.15.136:6802/4266571820', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '012409e8-51c2-4584-aba7-95996f0d72ad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6808', 'nonce': 958255259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6809', 'nonce': 958255259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6811', 'nonce': 958255259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6810', 'nonce': 958255259}]}, 'public_addr': '172.21.15.181:6808/958255259', 'cluster_addr': '172.21.15.181:6809/958255259', 'heartbeat_back_addr': '172.21.15.181:6811/958255259', 'heartbeat_front_addr': '172.21.15.181:6810/958255259', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '88b137b1-7d30-45f7-ac5c-ba026704ef24', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 85, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6813', 'nonce': 281884267}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6814', 'nonce': 281884267}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6816', 'nonce': 281884267}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6815', 'nonce': 281884267}]}, 'public_addr': '172.21.15.195:6813/281884267', 'cluster_addr': '172.21.15.195:6814/281884267', 'heartbeat_back_addr': '172.21.15.195:6816/281884267', 'heartbeat_front_addr': '172.21.15.195:6815/281884267', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': 'b54732a2-9dff-42e1-b4ef-281cbca3ad7c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6808', 'nonce': 3532043806}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6809', 'nonce': 3532043806}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6811', 'nonce': 3532043806}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6810', 'nonce': 3532043806}]}, 'public_addr': '172.21.15.73:6808/3532043806', 'cluster_addr': '172.21.15.73:6809/3532043806', 'heartbeat_back_addr': '172.21.15.73:6811/3532043806', 'heartbeat_front_addr': '172.21.15.73:6810/3532043806', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '20b48864-44dc-4732-8dff-d0d21cc08032', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6812', 'nonce': 4083982131}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6813', 'nonce': 4083982131}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6815', 'nonce': 4083982131}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6814', 'nonce': 4083982131}]}, 'public_addr': '172.21.15.136:6812/4083982131', 'cluster_addr': '172.21.15.136:6813/4083982131', 'heartbeat_back_addr': '172.21.15.136:6815/4083982131', 'heartbeat_front_addr': '172.21.15.136:6814/4083982131', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '9fac3ef4-e9be-4e8b-9a21-0de35b543d65', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6804', 'nonce': 1173379175}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6805', 'nonce': 1173379175}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6807', 'nonce': 1173379175}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6806', 'nonce': 1173379175}]}, 'public_addr': '172.21.15.181:6804/1173379175', 'cluster_addr': '172.21.15.181:6805/1173379175', 'heartbeat_back_addr': '172.21.15.181:6807/1173379175', 'heartbeat_front_addr': '172.21.15.181:6806/1173379175', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '16b34f90-3148-4d41-a403-ba0f900991e1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6805', 'nonce': 848686366}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6806', 'nonce': 848686366}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6808', 'nonce': 848686366}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6807', 'nonce': 848686366}]}, 'public_addr': '172.21.15.195:6805/848686366', 'cluster_addr': '172.21.15.195:6806/848686366', 'heartbeat_back_addr': '172.21.15.195:6808/848686366', 'heartbeat_front_addr': '172.21.15.195:6807/848686366', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.372908+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:45.986851+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:47.777103+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:47.179172+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:10:25.066350+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:46.147756+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:47.809723+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:38.606440+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:37.606540+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:25.726157+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:27.329800+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:26.722524+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:25.954484+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:25.900950+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:40.864138+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:48.003986+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '4.e', 'osds': [8, 3, 15]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '4.e', 'osds': [2, 8, 4]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T10:11:49.582 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:62 4.0 4:0b4c105e:::smithi19530714-16:head [write 1728171~452988] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.721021+0000", 2024-09-13T10:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.795314257000001, 2024-09-13T10:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096364669999999993, 2024-09-13T10:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 62 2024-09-13T10:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721021+0000", 2024-09-13T10:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721021+0000", 2024-09-13T10:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721023+0000", 2024-09-13T10:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0559999999999999e-06 2024-09-13T10:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721744+0000", 2024-09-13T10:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000721092 2024-09-13T10:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721766+0000", 2024-09-13T10:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2391999999999998e-05 2024-09-13T10:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721779+0000", 2024-09-13T10:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2864000000000001e-05 2024-09-13T10:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.724492+0000", 2024-09-13T10:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027128759999999999 2024-09-13T10:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.724746+0000", 2024-09-13T10:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000253914 2024-09-13T10:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.725111+0000", 2024-09-13T10:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036536400000000002 2024-09-13T10:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728978+0000", 2024-09-13T10:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038665969999999998 2024-09-13T10:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.728991+0000", 2024-09-13T10:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3711e-05 2024-09-13T10:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.730564+0000", 2024-09-13T10:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001572564 2024-09-13T10:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.730617+0000", 2024-09-13T10:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2744999999999997e-05 2024-09-13T10:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.730658+0000", 2024-09-13T10:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1291999999999997e-05 2024-09-13T10:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:64 4.0 4:0b4c105e:::smithi19530714-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.721878+0000", 2024-09-13T10:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.794457573999999, 2024-09-13T10:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0094623740000000008, 2024-09-13T10:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 64 2024-09-13T10:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721878+0000", 2024-09-13T10:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721878+0000", 2024-09-13T10:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721880+0000", 2024-09-13T10:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3939999999999999e-06 2024-09-13T10:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721886+0000", 2024-09-13T10:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0390000000000001e-06 2024-09-13T10:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721902+0000", 2024-09-13T10:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6520000000000001e-05 2024-09-13T10:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.721917+0000", 2024-09-13T10:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4959e-05 2024-09-13T10:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.729034+0000", 2024-09-13T10:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0071171100000000003 2024-09-13T10:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.729161+0000", 2024-09-13T10:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012695600000000001 2024-09-13T10:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731172+0000", 2024-09-13T10:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002010651 2024-09-13T10:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731241+0000", 2024-09-13T10:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9646000000000004e-05 2024-09-13T10:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.731341+0000", 2024-09-13T10:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9098999999999994e-05 2024-09-13T10:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:199 4.0 4:0bd0428c:::smithi19530714-50:head [setxattr _header (56) in=7b,truncate 2823543] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.992081+0000", 2024-09-13T10:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.524254794000001, 2024-09-13T10:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011866829000000001, 2024-09-13T10:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 199 2024-09-13T10:11:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992081+0000", 2024-09-13T10:11:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992081+0000", 2024-09-13T10:11:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992082+0000", 2024-09-13T10:11:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.004e-06 2024-09-13T10:11:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992090+0000", 2024-09-13T10:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7440000000000007e-06 2024-09-13T10:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3650000000000004e-06 2024-09-13T10:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992107+0000", 2024-09-13T10:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9230000000000008e-06 2024-09-13T10:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.995938+0000", 2024-09-13T10:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003830773 2024-09-13T10:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.996229+0000", 2024-09-13T10:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029133300000000001 2024-09-13T10:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.996692+0000", 2024-09-13T10:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046326700000000001 2024-09-13T10:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997974+0000", 2024-09-13T10:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001282016 2024-09-13T10:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.999243+0000", 2024-09-13T10:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012682360000000001 2024-09-13T10:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.003767+0000", 2024-09-13T10:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045243970000000003 2024-09-13T10:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.003851+0000", 2024-09-13T10:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4195000000000003e-05 2024-09-13T10:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.003948+0000", 2024-09-13T10:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6576000000000005e-05 2024-09-13T10:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:200 4.0 4:0bd0428c:::smithi19530714-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.992119+0000", 2024-09-13T10:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.524216633999998, 2024-09-13T10:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012149748, 2024-09-13T10:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 200 2024-09-13T10:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992119+0000", 2024-09-13T10:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992119+0000", 2024-09-13T10:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992120+0000", 2024-09-13T10:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9999999999999997e-07 2024-09-13T10:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992123+0000", 2024-09-13T10:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9569999999999998e-06 2024-09-13T10:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992131+0000", 2024-09-13T10:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2024-09-13T10:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992140+0000", 2024-09-13T10:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7900000000000005e-06 2024-09-13T10:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997418+0000", 2024-09-13T10:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052784349999999997 2024-09-13T10:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997546+0000", 2024-09-13T10:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012788900000000001 2024-09-13T10:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.004004+0000", 2024-09-13T10:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064580549999999999 2024-09-13T10:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.004128+0000", 2024-09-13T10:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123628 2024-09-13T10:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.004269+0000", 2024-09-13T10:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000141292 2024-09-13T10:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:265 4.0 4:0b4c105e:::smithi19530714-16:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.215372+0000", 2024-09-13T10:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.300963602000003, 2024-09-13T10:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013030243, 2024-09-13T10:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 265 2024-09-13T10:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215372+0000", 2024-09-13T10:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215372+0000", 2024-09-13T10:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215376+0000", 2024-09-13T10:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.393e-06 2024-09-13T10:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215387+0000", 2024-09-13T10:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1986e-05 2024-09-13T10:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215421+0000", 2024-09-13T10:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3355999999999997e-05 2024-09-13T10:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215438+0000", 2024-09-13T10:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7059e-05 2024-09-13T10:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215529+0000", 2024-09-13T10:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1571999999999997e-05 2024-09-13T10:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215621+0000", 2024-09-13T10:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1094999999999994e-05 2024-09-13T10:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.223688+0000", 2024-09-13T10:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080669859999999999 2024-09-13T10:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.225291+0000", 2024-09-13T10:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001603548 2024-09-13T10:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.225833+0000", 2024-09-13T10:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054185500000000005 2024-09-13T10:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228151+0000", 2024-09-13T10:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023176640000000001 2024-09-13T10:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228300+0000", 2024-09-13T10:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000149152 2024-09-13T10:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228402+0000", 2024-09-13T10:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102577 2024-09-13T10:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:289 4.0 4:03fab808:::smithi19530714-40:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.250617+0000", 2024-09-13T10:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.265718315999997, 2024-09-13T10:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010740764999999999, 2024-09-13T10:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 289 2024-09-13T10:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250617+0000", 2024-09-13T10:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250617+0000", 2024-09-13T10:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250621+0000", 2024-09-13T10:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4029999999999999e-06 2024-09-13T10:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250637+0000", 2024-09-13T10:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5950000000000001e-05 2024-09-13T10:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250673+0000", 2024-09-13T10:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6559e-05 2024-09-13T10:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250699+0000", 2024-09-13T10:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5479000000000001e-05 2024-09-13T10:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250825+0000", 2024-09-13T10:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012585600000000001 2024-09-13T10:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250971+0000", 2024-09-13T10:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000146002 2024-09-13T10:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257715+0000", 2024-09-13T10:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067440360000000001 2024-09-13T10:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258695+0000", 2024-09-13T10:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0009798509999999999 2024-09-13T10:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.260555+0000", 2024-09-13T10:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018600439999999999 2024-09-13T10:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261249+0000", 2024-09-13T10:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00069435899999999997 2024-09-13T10:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261307+0000", 2024-09-13T10:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7630000000000002e-05 2024-09-13T10:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261358+0000", 2024-09-13T10:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1595999999999999e-05 2024-09-13T10:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:299 4.0 4:0bd0428c:::smithi19530714-50:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.264107+0000", 2024-09-13T10:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.252228926999997, 2024-09-13T10:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020961186999999999, 2024-09-13T10:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 299 2024-09-13T10:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264107+0000", 2024-09-13T10:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264107+0000", 2024-09-13T10:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264111+0000", 2024-09-13T10:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0330000000000002e-06 2024-09-13T10:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264121+0000", 2024-09-13T10:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0543999999999999e-05 2024-09-13T10:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264150+0000", 2024-09-13T10:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8581999999999998e-05 2024-09-13T10:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264171+0000", 2024-09-13T10:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-13T10:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264369+0000", 2024-09-13T10:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019837000000000001 2024-09-13T10:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264535+0000", 2024-09-13T10:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000165324 2024-09-13T10:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.276997+0000", 2024-09-13T10:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012461842000000001 2024-09-13T10:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.277946+0000", 2024-09-13T10:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00094919299999999998 2024-09-13T10:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.281540+0000", 2024-09-13T10:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035946910000000001 2024-09-13T10:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.284932+0000", 2024-09-13T10:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033913799999999998 2024-09-13T10:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285010+0000", 2024-09-13T10:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8059000000000006e-05 2024-09-13T10:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285068+0000", 2024-09-13T10:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8072999999999998e-05 2024-09-13T10:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:489 4.0 4:03fab808:::smithi19530714-40:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.350324+0000", 2024-09-13T10:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.166011910000002, 2024-09-13T10:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014394344, 2024-09-13T10:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 489 2024-09-13T10:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350324+0000", 2024-09-13T10:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350324+0000", 2024-09-13T10:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350325+0000", 2024-09-13T10:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6270000000000001e-06 2024-09-13T10:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350333+0000", 2024-09-13T10:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3719999999999999e-06 2024-09-13T10:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350349+0000", 2024-09-13T10:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6004000000000001e-05 2024-09-13T10:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350359+0000", 2024-09-13T10:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.73e-06 2024-09-13T10:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350455+0000", 2024-09-13T10:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6260000000000003e-05 2024-09-13T10:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350526+0000", 2024-09-13T10:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0946999999999997e-05 2024-09-13T10:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.357046+0000", 2024-09-13T10:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065198510000000001 2024-09-13T10:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.357616+0000", 2024-09-13T10:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056994299999999995 2024-09-13T10:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.358085+0000", 2024-09-13T10:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046959399999999999 2024-09-13T10:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.364636+0000", 2024-09-13T10:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065509560000000001 2024-09-13T10:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.364680+0000", 2024-09-13T10:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3859999999999997e-05 2024-09-13T10:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.364718+0000", 2024-09-13T10:11:49.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.82e-05 2024-09-13T10:11:49.641 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.641 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.641 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.641 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3014 4.17 4:ea79a1ec:::smithi19530714-18:head [write 470662~772236,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T10:11:49.641 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:37.357256+0000", 2024-09-13T10:11:49.641 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.159080107999999, 2024-09-13T10:11:49.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010928939, 2024-09-13T10:11:49.642 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.642 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.642 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:49.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.642 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3014 2024-09-13T10:11:49.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.642 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.357256+0000", 2024-09-13T10:11:49.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.357256+0000", 2024-09-13T10:11:49.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.357259+0000", 2024-09-13T10:11:49.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7969999999999998e-06 2024-09-13T10:11:49.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.358967+0000", 2024-09-13T10:11:49.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001707568 2024-09-13T10:11:49.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.359002+0000", 2024-09-13T10:11:49.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5006000000000003e-05 2024-09-13T10:11:49.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.359019+0000", 2024-09-13T10:11:49.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7391999999999999e-05 2024-09-13T10:11:49.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.359145+0000", 2024-09-13T10:11:49.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012542200000000001 2024-09-13T10:11:49.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.359315+0000", 2024-09-13T10:11:49.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000170576 2024-09-13T10:11:49.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0,1", 2024-09-13T10:11:49.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.359578+0000", 2024-09-13T10:11:49.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026278 2024-09-13T10:11:49.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.363296+0000", 2024-09-13T10:11:49.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037180590000000001 2024-09-13T10:11:49.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:49.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.363322+0000", 2024-09-13T10:11:49.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6268999999999999e-05 2024-09-13T10:11:49.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.367999+0000", 2024-09-13T10:11:49.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046769910000000001 2024-09-13T10:11:49.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:49.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.368114+0000", 2024-09-13T10:11:49.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114269 2024-09-13T10:11:49.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.368185+0000", 2024-09-13T10:11:49.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0809999999999995e-05 2024-09-13T10:11:49.648 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.648 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.648 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.648 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3251 4.17 4:e81d9ad0:::smithi19530714-4:head [assert-version v105,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e80)", 2024-09-13T10:11:49.648 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.033800+0000", 2024-09-13T10:11:49.648 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.4825354810000002, 2024-09-13T10:11:49.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010039542, 2024-09-13T10:11:49.649 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.649 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.649 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:49.649 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.649 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.649 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.649 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3251 2024-09-13T10:11:49.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.649 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.033800+0000", 2024-09-13T10:11:49.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.033800+0000", 2024-09-13T10:11:49.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.033803+0000", 2024-09-13T10:11:49.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6240000000000002e-06 2024-09-13T10:11:49.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.033815+0000", 2024-09-13T10:11:49.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2150000000000001e-05 2024-09-13T10:11:49.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.033846+0000", 2024-09-13T10:11:49.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0791999999999999e-05 2024-09-13T10:11:49.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.033862+0000", 2024-09-13T10:11:49.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6436e-05 2024-09-13T10:11:49.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.034025+0000", 2024-09-13T10:11:49.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000162418 2024-09-13T10:11:49.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.034173+0000", 2024-09-13T10:11:49.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014789100000000001 2024-09-13T10:11:49.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0,1", 2024-09-13T10:11:49.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.037464+0000", 2024-09-13T10:11:49.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003291956 2024-09-13T10:11:49.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.042946+0000", 2024-09-13T10:11:49.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054814679999999998 2024-09-13T10:11:49.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.043107+0000", 2024-09-13T10:11:49.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016092100000000001 2024-09-13T10:11:49.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:49.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.043726+0000", 2024-09-13T10:11:49.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000619036 2024-09-13T10:11:49.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:49.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.043789+0000", 2024-09-13T10:11:49.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2634999999999999e-05 2024-09-13T10:11:49.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.043840+0000", 2024-09-13T10:11:49.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1214999999999997e-05 2024-09-13T10:11:49.655 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.655 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.655 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.655 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3179 4.19 4.8fd9b199 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:49.655 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.076610+0000", 2024-09-13T10:11:49.655 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.4397252270000003, 2024-09-13T10:11:49.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.051258910999999997, 2024-09-13T10:11:49.655 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.655 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.656 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:11:49.656 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.656 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.656 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.656 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3179 2024-09-13T10:11:49.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.656 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076610+0000", 2024-09-13T10:11:49.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076610+0000", 2024-09-13T10:11:49.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076614+0000", 2024-09-13T10:11:49.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4570000000000001e-06 2024-09-13T10:11:49.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076628+0000", 2024-09-13T10:11:49.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4409e-05 2024-09-13T10:11:49.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076658+0000", 2024-09-13T10:11:49.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0049e-05 2024-09-13T10:11:49.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076681+0000", 2024-09-13T10:11:49.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2246e-05 2024-09-13T10:11:49.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077042+0000", 2024-09-13T10:11:49.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036140400000000001 2024-09-13T10:11:49.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:49.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077069+0000", 2024-09-13T10:11:49.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6846e-05 2024-09-13T10:11:49.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.127848+0000", 2024-09-13T10:11:49.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.050779040999999997 2024-09-13T10:11:49.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.127869+0000", 2024-09-13T10:11:49.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1458999999999999e-05 2024-09-13T10:11:49.660 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.660 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.660 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.660 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3180 4.19 4.8fd9b199 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:49.660 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.076729+0000", 2024-09-13T10:11:49.661 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.4396068079999997, 2024-09-13T10:11:49.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.052035665000000002, 2024-09-13T10:11:49.661 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.661 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.661 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:11:49.661 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.661 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.661 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.661 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3180 2024-09-13T10:11:49.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.661 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076729+0000", 2024-09-13T10:11:49.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076729+0000", 2024-09-13T10:11:49.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076731+0000", 2024-09-13T10:11:49.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5969999999999999e-06 2024-09-13T10:11:49.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076739+0000", 2024-09-13T10:11:49.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7700000000000007e-06 2024-09-13T10:11:49.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076753+0000", 2024-09-13T10:11:49.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3896e-05 2024-09-13T10:11:49.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076764+0000", 2024-09-13T10:11:49.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0509999999999999e-05 2024-09-13T10:11:49.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077140+0000", 2024-09-13T10:11:49.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037611300000000002 2024-09-13T10:11:49.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:49.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077159+0000", 2024-09-13T10:11:49.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9453000000000002e-05 2024-09-13T10:11:49.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.128726+0000", 2024-09-13T10:11:49.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.051567108 2024-09-13T10:11:49.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.128765+0000", 2024-09-13T10:11:49.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8217999999999999e-05 2024-09-13T10:11:49.665 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.665 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.665 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.666 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3181 4.19 4.8fd9b199 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:49.666 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.076780+0000", 2024-09-13T10:11:49.666 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.4395555699999996, 2024-09-13T10:11:49.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.052105797000000002, 2024-09-13T10:11:49.666 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.666 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.666 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:11:49.666 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.666 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.666 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.666 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3181 2024-09-13T10:11:49.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.667 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076780+0000", 2024-09-13T10:11:49.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076780+0000", 2024-09-13T10:11:49.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076781+0000", 2024-09-13T10:11:49.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1699999999999997e-07 2024-09-13T10:11:49.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076785+0000", 2024-09-13T10:11:49.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1899999999999997e-06 2024-09-13T10:11:49.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076797+0000", 2024-09-13T10:11:49.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1579999999999999e-05 2024-09-13T10:11:49.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076805+0000", 2024-09-13T10:11:49.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8890000000000002e-06 2024-09-13T10:11:49.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077227+0000", 2024-09-13T10:11:49.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042249300000000002 2024-09-13T10:11:49.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:49.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077247+0000", 2024-09-13T10:11:49.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9644999999999999e-05 2024-09-13T10:11:49.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.128862+0000", 2024-09-13T10:11:49.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.051615227 2024-09-13T10:11:49.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.128886+0000", 2024-09-13T10:11:49.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3955999999999999e-05 2024-09-13T10:11:49.673 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.673 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.673 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.674 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3179 4.19 4:998d9bf1:::smithi19530714-33:head [read 1373852~878319,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:49.674 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.100709+0000", 2024-09-13T10:11:49.674 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.4156265980000002, 2024-09-13T10:11:49.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033184096000000003, 2024-09-13T10:11:49.674 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.674 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.674 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:49.674 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.674 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.674 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.675 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3179 2024-09-13T10:11:49.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.675 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100709+0000", 2024-09-13T10:11:49.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100709+0000", 2024-09-13T10:11:49.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100720+0000", 2024-09-13T10:11:49.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0871e-05 2024-09-13T10:11:49.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100736+0000", 2024-09-13T10:11:49.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6504999999999999e-05 2024-09-13T10:11:49.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100827+0000", 2024-09-13T10:11:49.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0713999999999999e-05 2024-09-13T10:11:49.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100852+0000", 2024-09-13T10:11:49.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5168e-05 2024-09-13T10:11:49.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101036+0000", 2024-09-13T10:11:49.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018362900000000001 2024-09-13T10:11:49.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:49.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101065+0000", 2024-09-13T10:11:49.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9352e-05 2024-09-13T10:11:49.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129506+0000", 2024-09-13T10:11:49.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028440811 2024-09-13T10:11:49.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129801+0000", 2024-09-13T10:11:49.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000295257 2024-09-13T10:11:49.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.133893+0000", 2024-09-13T10:11:49.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040917890000000002 2024-09-13T10:11:49.679 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.679 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.679 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.679 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3180 4.19 4:998d9bf1:::smithi19530714-33:head [read 0~2252171,checksum] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:49.679 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.100905+0000", 2024-09-13T10:11:49.680 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.4154308420000001, 2024-09-13T10:11:49.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.038171497999999998, 2024-09-13T10:11:49.680 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.680 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.680 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:49.680 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.680 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.680 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.680 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3180 2024-09-13T10:11:49.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.680 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100905+0000", 2024-09-13T10:11:49.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100905+0000", 2024-09-13T10:11:49.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100906+0000", 2024-09-13T10:11:49.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.494e-06 2024-09-13T10:11:49.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100915+0000", 2024-09-13T10:11:49.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7630000000000006e-06 2024-09-13T10:11:49.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100934+0000", 2024-09-13T10:11:49.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8722000000000001e-05 2024-09-13T10:11:49.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100952+0000", 2024-09-13T10:11:49.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7819e-05 2024-09-13T10:11:49.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101140+0000", 2024-09-13T10:11:49.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018818900000000001 2024-09-13T10:11:49.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:49.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101160+0000", 2024-09-13T10:11:49.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0429999999999999e-05 2024-09-13T10:11:49.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.133977+0000", 2024-09-13T10:11:49.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032816970000000001 2024-09-13T10:11:49.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.134128+0000", 2024-09-13T10:11:49.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015088100000000001 2024-09-13T10:11:49.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.139076+0000", 2024-09-13T10:11:49.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049482299999999996 2024-09-13T10:11:49.685 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.685 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.685 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.685 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3181 4.19 4:998d9bf1:::smithi19530714-33:head [read 1151249~1100922,checksum] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:49.685 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.100999+0000", 2024-09-13T10:11:49.685 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.4153371029999997, 2024-09-13T10:11:49.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040765649000000001, 2024-09-13T10:11:49.685 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.686 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.686 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:49.686 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.686 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.686 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.686 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3181 2024-09-13T10:11:49.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.686 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100999+0000", 2024-09-13T10:11:49.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100999+0000", 2024-09-13T10:11:49.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101000+0000", 2024-09-13T10:11:49.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3090000000000001e-06 2024-09-13T10:11:49.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101007+0000", 2024-09-13T10:11:49.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0430000000000002e-06 2024-09-13T10:11:49.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101018+0000", 2024-09-13T10:11:49.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1239e-05 2024-09-13T10:11:49.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101030+0000", 2024-09-13T10:11:49.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1986e-05 2024-09-13T10:11:49.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101229+0000", 2024-09-13T10:11:49.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000198681 2024-09-13T10:11:49.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:49.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101248+0000", 2024-09-13T10:11:49.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8766999999999998e-05 2024-09-13T10:11:49.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.139403+0000", 2024-09-13T10:11:49.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.038155715999999999 2024-09-13T10:11:49.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.139715+0000", 2024-09-13T10:11:49.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000311787 2024-09-13T10:11:49.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.141764+0000", 2024-09-13T10:11:49.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020491210000000001 2024-09-13T10:11:49.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.690 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.691 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.691 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3286 4.19 4:9bec18ef:::smithi19530714-47:head [sparse-read 0~1464604,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:49.691 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.129001+0000", 2024-09-13T10:11:49.691 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.3873346470000003, 2024-09-13T10:11:49.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018508512000000001, 2024-09-13T10:11:49.691 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.691 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.691 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:49.692 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.692 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.692 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.692 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3286 2024-09-13T10:11:49.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.692 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129001+0000", 2024-09-13T10:11:49.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129001+0000", 2024-09-13T10:11:49.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129003+0000", 2024-09-13T10:11:49.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3199999999999998e-06 2024-09-13T10:11:49.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129036+0000", 2024-09-13T10:11:49.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3015000000000002e-05 2024-09-13T10:11:49.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129057+0000", 2024-09-13T10:11:49.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1078999999999999e-05 2024-09-13T10:11:49.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129071+0000", 2024-09-13T10:11:49.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3743999999999999e-05 2024-09-13T10:11:49.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.141927+0000", 2024-09-13T10:11:49.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012855729 2024-09-13T10:11:49.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.142359+0000", 2024-09-13T10:11:49.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043179600000000002 2024-09-13T10:11:49.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.147510+0000", 2024-09-13T10:11:49.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005150829 2024-09-13T10:11:49.695 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.696 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.696 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.696 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3287 4.19 4:9bec18ef:::smithi19530714-47:head [sparse-read 2111652~258168] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:49.696 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.129097+0000", 2024-09-13T10:11:49.696 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.3872384100000001, 2024-09-13T10:11:49.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018987459000000002, 2024-09-13T10:11:49.696 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.696 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.696 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:49.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.697 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.697 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.697 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3287 2024-09-13T10:11:49.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.697 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129097+0000", 2024-09-13T10:11:49.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129097+0000", 2024-09-13T10:11:49.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129098+0000", 2024-09-13T10:11:49.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6299999999999993e-07 2024-09-13T10:11:49.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129103+0000", 2024-09-13T10:11:49.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.763e-06 2024-09-13T10:11:49.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129113+0000", 2024-09-13T10:11:49.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8230000000000006e-06 2024-09-13T10:11:49.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129121+0000", 2024-09-13T10:11:49.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1459999999999996e-06 2024-09-13T10:11:49.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.147587+0000", 2024-09-13T10:11:49.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018466104000000001 2024-09-13T10:11:49.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.147732+0000", 2024-09-13T10:11:49.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014455499999999999 2024-09-13T10:11:49.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.148085+0000", 2024-09-13T10:11:49.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035310499999999999 2024-09-13T10:11:49.702 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.703 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.703 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.703 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3288 4.19 4:9bec18ef:::smithi19530714-47:head [sparse-read 2190457~179363] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:49.703 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.129133+0000", 2024-09-13T10:11:49.703 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.387202448, 2024-09-13T10:11:49.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019922197999999999, 2024-09-13T10:11:49.703 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.704 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.704 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:49.704 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.704 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.704 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.704 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3288 2024-09-13T10:11:49.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.704 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129133+0000", 2024-09-13T10:11:49.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129133+0000", 2024-09-13T10:11:49.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129134+0000", 2024-09-13T10:11:49.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3e-07 2024-09-13T10:11:49.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129137+0000", 2024-09-13T10:11:49.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1930000000000002e-06 2024-09-13T10:11:49.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129146+0000", 2024-09-13T10:11:49.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7790000000000001e-06 2024-09-13T10:11:49.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129154+0000", 2024-09-13T10:11:49.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6539999999999999e-06 2024-09-13T10:11:49.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.148377+0000", 2024-09-13T10:11:49.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019222965000000002 2024-09-13T10:11:49.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.148497+0000", 2024-09-13T10:11:49.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012054500000000001 2024-09-13T10:11:49.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.149055+0000", 2024-09-13T10:11:49.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00055833200000000003 2024-09-13T10:11:49.708 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.708 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.708 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.708 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3516 4.17 4:e81d9ad0:::smithi19530714-4:head [write 460591~723410] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-13T10:11:49.708 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:48.935216+0000", 2024-09-13T10:11:49.708 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.581119732, 2024-09-13T10:11:49.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010700148, 2024-09-13T10:11:49.708 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:49.708 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:49.709 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:49.709 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:49.709 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:49.709 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:49.709 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3516 2024-09-13T10:11:49.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.709 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:49.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:49.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:48.935216+0000", 2024-09-13T10:11:49.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:49.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:48.935216+0000", 2024-09-13T10:11:49.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:49.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:49.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:48.935218+0000", 2024-09-13T10:11:49.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0940000000000002e-06 2024-09-13T10:11:49.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:49.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:48.936198+0000", 2024-09-13T10:11:49.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0009796309999999999 2024-09-13T10:11:49.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:49.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:48.936210+0000", 2024-09-13T10:11:49.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2636999999999999e-05 2024-09-13T10:11:49.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:49.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:48.936216+0000", 2024-09-13T10:11:49.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.93e-06 2024-09-13T10:11:49.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:49.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:48.936352+0000", 2024-09-13T10:11:49.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000135749 2024-09-13T10:11:49.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:49.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:48.936613+0000", 2024-09-13T10:11:49.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000261389 2024-09-13T10:11:49.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0,1", 2024-09-13T10:11:49.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:48.937067+0000", 2024-09-13T10:11:49.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00045311 2024-09-13T10:11:49.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:49.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:48.941345+0000", 2024-09-13T10:11:49.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004278174 2024-09-13T10:11:49.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:48.945472+0000", 2024-09-13T10:11:49.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041271700000000003 2024-09-13T10:11:49.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:49.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:48.945575+0000", 2024-09-13T10:11:49.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000103064 2024-09-13T10:11:49.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:49.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:48.945778+0000", 2024-09-13T10:11:49.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000202974 2024-09-13T10:11:49.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:49.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:49.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:49.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:48.945916+0000", 2024-09-13T10:11:49.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138226 2024-09-13T10:11:49.715 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.715 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.715 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.715 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:49.715 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:49.715 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:49.716 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-13T10:11:49.718 INFO:tasks.rados.rados.0.smithi195.stdout:1524: finishing write tid 1 to smithi19530714-5 2024-09-13T10:11:49.718 INFO:tasks.rados.rados.0.smithi195.stdout:1523: expect (ObjNum 287 snap 0 seq_num 287) 2024-09-13T10:11:49.718 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 968304 2024-09-13T10:11:49.718 INFO:tasks.rados.rados.0.smithi195.stdout:1524: finishing write tid 2 to smithi19530714-5 2024-09-13T10:11:49.718 INFO:tasks.rados.rados.0.smithi195.stdout:1524: finishing write tid 3 to smithi19530714-5 2024-09-13T10:11:49.718 INFO:tasks.rados.rados.0.smithi195.stdout:1524: finishing write tid 4 to smithi19530714-5 2024-09-13T10:11:49.718 INFO:tasks.rados.rados.0.smithi195.stdout:1524: finishing write tid 5 to smithi19530714-5 2024-09-13T10:11:49.718 INFO:tasks.rados.rados.0.smithi195.stdout:1524: oid 5 updating version 0 to 21 2024-09-13T10:11:49.718 INFO:tasks.rados.rados.0.smithi195.stdout:1524: oid 5 version 21 is already newer than 20 2024-09-13T10:11:49.718 INFO:tasks.rados.rados.0.smithi195.stdout:1524: oid 5 version 21 is already newer than 19 2024-09-13T10:11:49.718 INFO:tasks.rados.rados.0.smithi195.stdout:1524: oid 5 updating version 21 to 22 2024-09-13T10:11:49.719 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 22 (ObjNum 310 snap 0 seq_num 310) dirty exists 2024-09-13T10:11:49.719 INFO:tasks.rados.rados.0.smithi195.stdout:1524: left oid 5 (ObjNum 310 snap 0 seq_num 310) 2024-09-13T10:11:49.719 INFO:tasks.rados.rados.0.smithi195.stdout:1525: expect (ObjNum 297 snap 0 seq_num 297) 2024-09-13T10:11:49.719 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:49.720 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:49.711+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:49.728 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:11:49.781 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:49.818 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:49.811+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:49.825 INFO:tasks.rados.rados.0.smithi195.stdout:1526: finishing write tid 1 to smithi19530714-45 2024-09-13T10:11:49.825 INFO:tasks.rados.rados.0.smithi195.stdout:1526: finishing write tid 2 to smithi19530714-45 2024-09-13T10:11:49.825 INFO:tasks.rados.rados.0.smithi195.stdout:1526: finishing write tid 3 to smithi19530714-45 2024-09-13T10:11:49.825 INFO:tasks.rados.rados.0.smithi195.stdout:1526: finishing write tid 4 to smithi19530714-45 2024-09-13T10:11:49.825 INFO:tasks.rados.rados.0.smithi195.stdout:1526: finishing write tid 5 to smithi19530714-45 2024-09-13T10:11:49.826 INFO:tasks.rados.rados.0.smithi195.stdout:1526: oid 45 updating version 0 to 143 2024-09-13T10:11:49.826 INFO:tasks.rados.rados.0.smithi195.stdout:1526: oid 45 updating version 143 to 145 2024-09-13T10:11:49.826 INFO:tasks.rados.rados.0.smithi195.stdout:1526: oid 45 version 145 is already newer than 142 2024-09-13T10:11:49.826 INFO:tasks.rados.rados.0.smithi195.stdout:1526: oid 45 version 145 is already newer than 144 2024-09-13T10:11:49.826 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 145 (ObjNum 311 snap 0 seq_num 311) dirty exists 2024-09-13T10:11:49.826 INFO:tasks.rados.rados.0.smithi195.stdout:1526: left oid 45 (ObjNum 311 snap 0 seq_num 311) 2024-09-13T10:11:49.826 INFO:tasks.rados.rados.0.smithi195.stdout:1527: finishing write tid 1 to smithi19530714-3 2024-09-13T10:11:49.826 INFO:tasks.rados.rados.0.smithi195.stdout:1527: finishing write tid 2 to smithi19530714-3 2024-09-13T10:11:49.826 INFO:tasks.rados.rados.0.smithi195.stdout:1527: finishing write tid 3 to smithi19530714-3 2024-09-13T10:11:49.826 INFO:tasks.rados.rados.0.smithi195.stdout:1527: finishing write tid 4 to smithi19530714-3 2024-09-13T10:11:49.826 INFO:tasks.rados.rados.0.smithi195.stdout:1527: finishing write tid 5 to smithi19530714-3 2024-09-13T10:11:49.826 INFO:tasks.rados.rados.0.smithi195.stdout:1527: oid 3 updating version 0 to 70 2024-09-13T10:11:49.827 INFO:tasks.rados.rados.0.smithi195.stdout:1527: oid 3 version 70 is already newer than 67 2024-09-13T10:11:49.827 INFO:tasks.rados.rados.0.smithi195.stdout:1527: oid 3 version 70 is already newer than 68 2024-09-13T10:11:49.827 INFO:tasks.rados.rados.0.smithi195.stdout:1527: oid 3 version 70 is already newer than 69 2024-09-13T10:11:49.827 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 70 (ObjNum 312 snap 0 seq_num 312) dirty exists 2024-09-13T10:11:49.827 INFO:tasks.rados.rados.0.smithi195.stdout:1527: left oid 3 (ObjNum 312 snap 0 seq_num 312) 2024-09-13T10:11:49.827 INFO:tasks.rados.rados.0.smithi195.stdout:1528: expect (ObjNum 160 snap 0 seq_num 160) 2024-09-13T10:11:49.847 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:49.848 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:49.848 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:11:49.848 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:49.862 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-13T10:11:49.905 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:49.906 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:49.919 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:49.913+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:49.924 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:49.924 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:49.945 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:49.966 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:49.966 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:49.983 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:49.983 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:50.002 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:50.011 INFO:tasks.rados.rados.0.smithi195.stdout:1529: finishing write tid 1 to smithi19530714-4 2024-09-13T10:11:50.011 INFO:tasks.rados.rados.0.smithi195.stdout:1529: finishing write tid 2 to smithi19530714-4 2024-09-13T10:11:50.011 INFO:tasks.rados.rados.0.smithi195.stdout:1529: finishing write tid 3 to smithi19530714-4 2024-09-13T10:11:50.011 INFO:tasks.rados.rados.0.smithi195.stdout:1529: finishing write tid 4 to smithi19530714-4 2024-09-13T10:11:50.011 INFO:tasks.rados.rados.0.smithi195.stdout:1529: finishing write tid 5 to smithi19530714-4 2024-09-13T10:11:50.012 INFO:tasks.rados.rados.0.smithi195.stdout:1529: oid 4 updating version 0 to 112 2024-09-13T10:11:50.012 INFO:tasks.rados.rados.0.smithi195.stdout:1529: oid 4 updating version 112 to 113 2024-09-13T10:11:50.012 INFO:tasks.rados.rados.0.smithi195.stdout:1529: oid 4 version 113 is already newer than 110 2024-09-13T10:11:50.012 INFO:tasks.rados.rados.0.smithi195.stdout:1529: oid 4 version 113 is already newer than 111 2024-09-13T10:11:50.012 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 113 (ObjNum 313 snap 0 seq_num 313) dirty exists 2024-09-13T10:11:50.012 INFO:tasks.rados.rados.0.smithi195.stdout:1529: left oid 4 (ObjNum 313 snap 0 seq_num 313) 2024-09-13T10:11:50.012 INFO:tasks.rados.rados.0.smithi195.stdout:1530: finishing write tid 1 to smithi19530714-46 2024-09-13T10:11:50.012 INFO:tasks.rados.rados.0.smithi195.stdout:1530: finishing write tid 2 to smithi19530714-46 2024-09-13T10:11:50.012 INFO:tasks.rados.rados.0.smithi195.stdout:1530: finishing write tid 3 to smithi19530714-46 2024-09-13T10:11:50.012 INFO:tasks.rados.rados.0.smithi195.stdout:1530: finishing write tid 4 to smithi19530714-46 2024-09-13T10:11:50.013 INFO:tasks.rados.rados.0.smithi195.stdout:1530: finishing write tid 5 to smithi19530714-46 2024-09-13T10:11:50.013 INFO:tasks.rados.rados.0.smithi195.stdout:1530: oid 46 updating version 0 to 55 2024-09-13T10:11:50.013 INFO:tasks.rados.rados.0.smithi195.stdout:1530: oid 46 updating version 55 to 58 2024-09-13T10:11:50.013 INFO:tasks.rados.rados.0.smithi195.stdout:1530: oid 46 version 58 is already newer than 57 2024-09-13T10:11:50.013 INFO:tasks.rados.rados.0.smithi195.stdout:1530: oid 46 version 58 is already newer than 56 2024-09-13T10:11:50.013 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 58 (ObjNum 314 snap 0 seq_num 314) dirty exists 2024-09-13T10:11:50.013 INFO:tasks.rados.rados.0.smithi195.stdout:1530: left oid 46 (ObjNum 314 snap 0 seq_num 314) 2024-09-13T10:11:50.013 INFO:tasks.rados.rados.0.smithi195.stdout:1531: expect (ObjNum 308 snap 0 seq_num 308) 2024-09-13T10:11:50.013 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:50.021 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:50.018+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:50.025 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:50.025 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:50.025 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:11:50.025 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:11:50.025 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:50.025 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:50.026 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:50.039 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-13T10:11:50.041 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:11:50.042 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:11:50.044 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:50.045 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:50.060 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-09-13T10:11:48.950617+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":217554231,"num_objects":179,"num_object_clones":0,"num_object_copies":537,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":3934,"num_read_kb":2581189,"num_write":2547,"num_write_kb":564617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":23489322,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":23,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":117,"acting":117,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":964940,"kb_used_data":479968,"kb_used_omap":24,"kb_used_meta":477863,"kb_avail":1508984500,"statfs":{"total":1546188226560,"available":1545200128000,"internally_reserved":0,"allocated":491487232,"data_stored":483952952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":489331791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-3814306,"num_objects":-14,"num_object_clones":0,"num_object_copies":-42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-15,"num_objects_unfound":0,"num_objects_dirty":-14,"num_whiteouts":0,"num_read":353,"num_read_kb":269210,"num_write":96,"num_write_kb":22637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":5911894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.871582"},"pg_stats":[{"pgid":"4.19","version":"84'55","reported_seq":270,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.552777+0000","last_change":"2024-09-13T10:11:45.089925+0000","last_active":"2024-09-13T10:11:47.552777+0000","last_peered":"2024-09-13T10:11:47.552777+0000","last_clean":"2024-09-13T10:11:47.552777+0000","last_became_active":"2024-09-13T10:11:45.087574+0000","last_became_peered":"2024-09-13T10:11:45.087574+0000","last_unstale":"2024-09-13T10:11:47.552777+0000","last_undegraded":"2024-09-13T10:11:47.552777+0000","last_fullsized":"2024-09-13T10:11:47.552777+0000","mapping_epoch":82,"log_start":"74'52","ondisk_log_start":"74'52","created":41,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'28","last_scrub_stamp":"2024-09-13T10:11:00.388132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.388132+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":52,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:22.382648+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4621991,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":93352,"num_write":31,"num_write_kb":7941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2376704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"79'18","reported_seq":192,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.107365+0000","last_change":"2024-09-13T10:11:43.078139+0000","last_active":"2024-09-13T10:11:47.107365+0000","last_peered":"2024-09-13T10:11:47.107365+0000","last_clean":"2024-09-13T10:11:47.107365+0000","last_became_active":"2024-09-13T10:11:43.076166+0000","last_became_peered":"2024-09-13T10:11:43.076166+0000","last_unstale":"2024-09-13T10:11:47.107365+0000","last_undegraded":"2024-09-13T10:11:47.107365+0000","last_fullsized":"2024-09-13T10:11:47.107365+0000","mapping_epoch":80,"log_start":"68'15","ondisk_log_start":"68'15","created":41,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'5","last_scrub_stamp":"2024-09-13T10:11:01.783496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.783496+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":15,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:03.632372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3882417,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":52,"num_read_kb":38140,"num_write":14,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2436277,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,13],"acting":[10,8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.17","version":"84'109","reported_seq":378,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.708432+0000","last_change":"2024-09-13T10:11:34.723291+0000","last_active":"2024-09-13T10:11:47.708432+0000","last_peered":"2024-09-13T10:11:47.708432+0000","last_clean":"2024-09-13T10:11:47.708432+0000","last_became_active":"2024-09-13T10:11:34.720812+0000","last_became_peered":"2024-09-13T10:11:34.720812+0000","last_unstale":"2024-09-13T10:11:47.708432+0000","last_undegraded":"2024-09-13T10:11:47.708432+0000","last_fullsized":"2024-09-13T10:11:47.708432+0000","mapping_epoch":73,"log_start":"80'106","ondisk_log_start":"80'106","created":41,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'34","last_scrub_stamp":"2024-09-13T10:11:00.442413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.442413+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":106,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:13.441116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8067459,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":150,"num_read_kb":120929,"num_write":99,"num_write_kb":28972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3911454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"69'66","reported_seq":278,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-13T10:11:43.098346+0000","last_change":"2024-09-13T10:11:25.825915+0000","last_active":"2024-09-13T10:11:43.098346+0000","last_peered":"2024-09-13T10:11:43.098346+0000","last_clean":"2024-09-13T10:11:43.098346+0000","last_became_active":"2024-09-13T10:11:25.825236+0000","last_became_peered":"2024-09-13T10:11:25.825236+0000","last_unstale":"2024-09-13T10:11:43.098346+0000","last_undegraded":"2024-09-13T10:11:43.098346+0000","last_fullsized":"2024-09-13T10:11:43.098346+0000","mapping_epoch":66,"log_start":"69'63","ondisk_log_start":"69'63","created":41,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":5,"last_scrub":"46'44","last_scrub_stamp":"2024-09-13T10:11:00.456457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.456457+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":60,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:04.006000+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6672379,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":70,"num_read_kb":80777,"num_write":24,"num_write_kb":6198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3338234,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,13,5],"acting":[14,13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.15","version":"86'41","reported_seq":250,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T10:11:48.261430+0000","last_change":"2024-09-13T10:11:23.706313+0000","last_active":"2024-09-13T10:11:48.261430+0000","last_peered":"2024-09-13T10:11:48.261430+0000","last_clean":"2024-09-13T10:11:48.261430+0000","last_became_active":"2024-09-13T10:11:23.705370+0000","last_became_peered":"2024-09-13T10:11:23.705370+0000","last_unstale":"2024-09-13T10:11:48.261430+0000","last_undegraded":"2024-09-13T10:11:48.261430+0000","last_fullsized":"2024-09-13T10:11:48.261430+0000","mapping_epoch":64,"log_start":"74'38","ondisk_log_start":"74'38","created":41,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'36","last_scrub_stamp":"2024-09-13T10:11:05.611551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.611551+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":36,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:14.570465+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3193363,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":37704,"num_write":5,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1702851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,13],"acting":[6,7,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"39'40","reported_seq":162,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.717744+0000","last_change":"2024-09-13T10:11:47.716853+0000","last_active":"2024-09-13T10:11:47.717744+0000","last_peered":"2024-09-13T10:11:47.717744+0000","last_clean":"2024-09-13T10:11:47.717744+0000","last_became_active":"2024-09-13T10:11:47.715516+0000","last_became_peered":"2024-09-13T10:11:47.715516+0000","last_unstale":"2024-09-13T10:11:47.717744+0000","last_undegraded":"2024-09-13T10:11:47.717744+0000","last_fullsized":"2024-09-13T10:11:47.717744+0000","mapping_epoch":85,"log_start":"39'37","ondisk_log_start":"39'37","created":41,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'40","last_scrub_stamp":"2024-09-13T10:11:04.595221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.595221+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":37,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7831963,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":112,"num_write":16,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.13","version":"84'36","reported_seq":244,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.552727+0000","last_change":"2024-09-13T10:11:15.761185+0000","last_active":"2024-09-13T10:11:47.552727+0000","last_peered":"2024-09-13T10:11:47.552727+0000","last_clean":"2024-09-13T10:11:47.552727+0000","last_became_active":"2024-09-13T10:11:15.757364+0000","last_became_peered":"2024-09-13T10:11:15.757364+0000","last_unstale":"2024-09-13T10:11:47.552727+0000","last_undegraded":"2024-09-13T10:11:47.552727+0000","last_fullsized":"2024-09-13T10:11:47.552727+0000","mapping_epoch":58,"log_start":"80'33","ondisk_log_start":"80'33","created":41,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'17","last_scrub_stamp":"2024-09-13T10:11:03.220846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:03.220846+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":31,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:07.079181+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":45283,"num_write":22,"num_write_kb":5429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1493592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"81'42","reported_seq":288,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-13T10:11:45.826373+0000","last_change":"2024-09-13T10:11:43.083488+0000","last_active":"2024-09-13T10:11:45.826373+0000","last_peered":"2024-09-13T10:11:45.826373+0000","last_clean":"2024-09-13T10:11:45.826373+0000","last_became_active":"2024-09-13T10:11:43.081950+0000","last_became_peered":"2024-09-13T10:11:43.081950+0000","last_unstale":"2024-09-13T10:11:45.826373+0000","last_undegraded":"2024-09-13T10:11:45.826373+0000","last_fullsized":"2024-09-13T10:11:45.826373+0000","mapping_epoch":80,"log_start":"76'39","ondisk_log_start":"76'39","created":41,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'17","last_scrub_stamp":"2024-09-13T10:11:02.582829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:02.582829+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":39,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:25.166178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5428727,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":122521,"num_write":26,"num_write_kb":7451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3535190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,3,9],"acting":[13,3,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.11","version":"81'18","reported_seq":209,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-13T10:11:45.826022+0000","last_change":"2024-09-13T10:11:40.065958+0000","last_active":"2024-09-13T10:11:45.826022+0000","last_peered":"2024-09-13T10:11:45.826022+0000","last_clean":"2024-09-13T10:11:45.826022+0000","last_became_active":"2024-09-13T10:11:40.064608+0000","last_became_peered":"2024-09-13T10:11:40.064608+0000","last_unstale":"2024-09-13T10:11:45.826022+0000","last_undegraded":"2024-09-13T10:11:45.826022+0000","last_fullsized":"2024-09-13T10:11:45.826022+0000","mapping_epoch":77,"log_start":"66'15","ondisk_log_start":"66'15","created":41,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":5,"last_scrub":"40'13","last_scrub_stamp":"2024-09-13T10:11:04.242723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.242723+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":12,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:11.098336+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2991679,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":54,"num_read_kb":41076,"num_write":6,"num_write_kb":2015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1805029,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,12],"acting":[13,4,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.10","version":"84'47","reported_seq":282,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-13T10:11:46.719298+0000","last_change":"2024-09-13T10:11:40.068589+0000","last_active":"2024-09-13T10:11:46.719298+0000","last_peered":"2024-09-13T10:11:46.719298+0000","last_clean":"2024-09-13T10:11:46.719298+0000","last_became_active":"2024-09-13T10:11:40.063380+0000","last_became_peered":"2024-09-13T10:11:40.063380+0000","last_unstale":"2024-09-13T10:11:46.719298+0000","last_undegraded":"2024-09-13T10:11:46.719298+0000","last_fullsized":"2024-09-13T10:11:46.719298+0000","mapping_epoch":77,"log_start":"80'44","ondisk_log_start":"80'44","created":41,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'16","last_scrub_stamp":"2024-09-13T10:11:00.706874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.706874+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":42,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:17.975637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5714446,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":98679,"num_write":37,"num_write_kb":10257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2889991,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,14,3],"acting":[9,14,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.552988+0000","last_change":"2024-09-13T10:11:12.744453+0000","last_active":"2024-09-13T10:11:47.552988+0000","last_peered":"2024-09-13T10:11:47.552988+0000","last_clean":"2024-09-13T10:11:47.552988+0000","last_became_active":"2024-09-13T10:11:12.744090+0000","last_became_peered":"2024-09-13T10:11:12.744090+0000","last_unstale":"2024-09-13T10:11:47.552988+0000","last_undegraded":"2024-09-13T10:11:47.552988+0000","last_fullsized":"2024-09-13T10:11:47.552988+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:57.001325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,12],"acting":[3,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"39'12","reported_seq":176,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.970535+0000","last_change":"2024-09-13T10:11:01.537080+0000","last_active":"2024-09-13T10:11:47.970535+0000","last_peered":"2024-09-13T10:11:47.970535+0000","last_clean":"2024-09-13T10:11:47.970535+0000","last_became_active":"2024-09-13T10:10:54.318519+0000","last_became_peered":"2024-09-13T10:10:54.318519+0000","last_unstale":"2024-09-13T10:11:47.970535+0000","last_undegraded":"2024-09-13T10:11:47.970535+0000","last_fullsized":"2024-09-13T10:11:47.970535+0000","mapping_epoch":31,"log_start":"38'9","ondisk_log_start":"38'9","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'12","last_scrub_stamp":"2024-09-13T10:11:01.536963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.536963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":9,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:06.231077+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,14],"acting":[6,8,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":135,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.553040+0000","last_change":"2024-09-13T10:10:32.932079+0000","last_active":"2024-09-13T10:11:47.553040+0000","last_peered":"2024-09-13T10:11:47.553040+0000","last_clean":"2024-09-13T10:11:47.553040+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:11:47.553040+0000","last_undegraded":"2024-09-13T10:11:47.553040+0000","last_fullsized":"2024-09-13T10:11:47.553040+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:57.533673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"79'83","reported_seq":292,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-13T10:11:43.123399+0000","last_change":"2024-09-13T10:11:01.219554+0000","last_active":"2024-09-13T10:11:43.123399+0000","last_peered":"2024-09-13T10:11:43.123399+0000","last_clean":"2024-09-13T10:11:43.123399+0000","last_became_active":"2024-09-13T10:10:54.314545+0000","last_became_peered":"2024-09-13T10:10:54.314545+0000","last_unstale":"2024-09-13T10:11:43.123399+0000","last_undegraded":"2024-09-13T10:11:43.123399+0000","last_fullsized":"2024-09-13T10:11:43.123399+0000","mapping_epoch":31,"log_start":"75'80","ondisk_log_start":"75'80","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'27","last_scrub_stamp":"2024-09-13T10:11:01.219486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.219486+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":79,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:09.956363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2487469,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":98,"num_read_kb":90427,"num_write":71,"num_write_kb":20315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,8],"acting":[2,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":135,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.455019+0000","last_change":"2024-09-13T10:10:32.177488+0000","last_active":"2024-09-13T10:11:47.455019+0000","last_peered":"2024-09-13T10:11:47.455019+0000","last_clean":"2024-09-13T10:11:47.455019+0000","last_became_active":"2024-09-13T10:10:30.128658+0000","last_became_peered":"2024-09-13T10:10:30.128658+0000","last_unstale":"2024-09-13T10:11:47.455019+0000","last_undegraded":"2024-09-13T10:11:47.455019+0000","last_fullsized":"2024-09-13T10:11:47.455019+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:50.278429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.0","version":"73'26","reported_seq":221,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.708167+0000","last_change":"2024-09-13T10:11:05.681812+0000","last_active":"2024-09-13T10:11:47.708167+0000","last_peered":"2024-09-13T10:11:47.708167+0000","last_clean":"2024-09-13T10:11:47.708167+0000","last_became_active":"2024-09-13T10:10:54.320014+0000","last_became_peered":"2024-09-13T10:10:54.320014+0000","last_unstale":"2024-09-13T10:11:47.708167+0000","last_undegraded":"2024-09-13T10:11:47.708167+0000","last_fullsized":"2024-09-13T10:11:47.708167+0000","mapping_epoch":31,"log_start":"60'23","ondisk_log_start":"60'23","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'14","last_scrub_stamp":"2024-09-13T10:11:05.681653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.681653+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":22,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:21.666096+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":47,"num_read_kb":43456,"num_write":11,"num_write_kb":3373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":136,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-13T10:11:43.098216+0000","last_change":"2024-09-13T10:11:41.627608+0000","last_active":"2024-09-13T10:11:43.098216+0000","last_peered":"2024-09-13T10:11:43.098216+0000","last_clean":"2024-09-13T10:11:43.098216+0000","last_became_active":"2024-09-13T10:10:30.126588+0000","last_became_peered":"2024-09-13T10:10:30.126588+0000","last_unstale":"2024-09-13T10:11:43.098216+0000","last_undegraded":"2024-09-13T10:11:43.098216+0000","last_fullsized":"2024-09-13T10:11:43.098216+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:11:41.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:41.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:06.468025+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00089379399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.1","version":"81'26","reported_seq":202,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-13T10:11:43.136736+0000","last_change":"2024-09-13T10:11:02.190704+0000","last_active":"2024-09-13T10:11:43.136736+0000","last_peered":"2024-09-13T10:11:43.136736+0000","last_clean":"2024-09-13T10:11:43.136736+0000","last_became_active":"2024-09-13T10:10:54.317600+0000","last_became_peered":"2024-09-13T10:10:54.317600+0000","last_unstale":"2024-09-13T10:11:43.136736+0000","last_undegraded":"2024-09-13T10:11:43.136736+0000","last_fullsized":"2024-09-13T10:11:43.136736+0000","mapping_epoch":31,"log_start":"65'23","ondisk_log_start":"65'23","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'18","last_scrub_stamp":"2024-09-13T10:11:02.190506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:02.190506+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":23,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:17.042562+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2974872,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":54,"num_read_kb":58406,"num_write":9,"num_write_kb":2797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,12],"acting":[2,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"81'228","reported_seq":360,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-13T10:11:45.826052+0000","last_change":"2024-09-13T10:10:43.231291+0000","last_active":"2024-09-13T10:11:45.826052+0000","last_peered":"2024-09-13T10:11:45.826052+0000","last_clean":"2024-09-13T10:11:45.826052+0000","last_became_active":"2024-09-13T10:10:43.230866+0000","last_became_peered":"2024-09-13T10:10:43.230866+0000","last_unstale":"2024-09-13T10:11:45.826052+0000","last_undegraded":"2024-09-13T10:11:45.826052+0000","last_fullsized":"2024-09-13T10:11:45.826052+0000","mapping_epoch":30,"log_start":"81'225","ondisk_log_start":"81'225","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":240,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:54.564637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21429023,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":215,"num_read_kb":3185,"num_write":299,"num_write_kb":58085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,4,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":218,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.553039+0000","last_change":"2024-09-13T10:11:12.699917+0000","last_active":"2024-09-13T10:11:47.553039+0000","last_peered":"2024-09-13T10:11:47.553039+0000","last_clean":"2024-09-13T10:11:47.553039+0000","last_became_active":"2024-09-13T10:11:12.699043+0000","last_became_peered":"2024-09-13T10:11:12.699043+0000","last_unstale":"2024-09-13T10:11:47.553039+0000","last_undegraded":"2024-09-13T10:11:47.553039+0000","last_fullsized":"2024-09-13T10:11:47.553039+0000","mapping_epoch":55,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:48.879395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,7],"acting":[3,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":47,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-13T10:11:45.826074+0000","last_change":"2024-09-13T10:11:38.808286+0000","last_active":"2024-09-13T10:11:45.826074+0000","last_peered":"2024-09-13T10:11:45.826074+0000","last_clean":"2024-09-13T10:11:45.826074+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:11:45.826074+0000","last_undegraded":"2024-09-13T10:11:45.826074+0000","last_fullsized":"2024-09-13T10:11:45.826074+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T10:11:38.808119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:38.808119+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:43.158000+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.6","version":"75'73","reported_seq":285,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-13T10:11:45.826387+0000","last_change":"2024-09-13T10:11:39.048972+0000","last_active":"2024-09-13T10:11:45.826387+0000","last_peered":"2024-09-13T10:11:45.826387+0000","last_clean":"2024-09-13T10:11:45.826387+0000","last_became_active":"2024-09-13T10:11:39.047906+0000","last_became_peered":"2024-09-13T10:11:39.047906+0000","last_unstale":"2024-09-13T10:11:45.826387+0000","last_undegraded":"2024-09-13T10:11:45.826387+0000","last_fullsized":"2024-09-13T10:11:45.826387+0000","mapping_epoch":76,"log_start":"74'70","ondisk_log_start":"74'70","created":31,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'44","last_scrub_stamp":"2024-09-13T10:11:05.396838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.396838+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":70,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:19.555361+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6302658,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":63258,"num_write":33,"num_write_kb":9547,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,1],"acting":[13,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"85'283","reported_seq":419,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.557627+0000","last_change":"2024-09-13T10:10:43.220235+0000","last_active":"2024-09-13T10:11:47.557627+0000","last_peered":"2024-09-13T10:11:47.557627+0000","last_clean":"2024-09-13T10:11:47.557627+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:11:47.557627+0000","last_undegraded":"2024-09-13T10:11:47.557627+0000","last_fullsized":"2024-09-13T10:11:47.557627+0000","mapping_epoch":30,"log_start":"82'280","ondisk_log_start":"82'280","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":292,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:55.166722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28796781,"num_objects":44,"num_object_clones":0,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":259,"num_read_kb":7952,"num_write":377,"num_write_kb":79976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":139,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-13T10:11:43.123279+0000","last_change":"2024-09-13T10:11:30.291380+0000","last_active":"2024-09-13T10:11:43.123279+0000","last_peered":"2024-09-13T10:11:43.123279+0000","last_clean":"2024-09-13T10:11:43.123279+0000","last_became_active":"2024-09-13T10:10:30.127232+0000","last_became_peered":"2024-09-13T10:10:30.127232+0000","last_unstale":"2024-09-13T10:11:43.123279+0000","last_undegraded":"2024-09-13T10:11:43.123279+0000","last_fullsized":"2024-09-13T10:11:43.123279+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-13T10:11:30.291157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:30.291157+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:31.417661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0034622780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"84'78","reported_seq":299,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.713577+0000","last_change":"2024-09-13T10:11:05.436449+0000","last_active":"2024-09-13T10:11:47.713577+0000","last_peered":"2024-09-13T10:11:47.713577+0000","last_clean":"2024-09-13T10:11:47.713577+0000","last_became_active":"2024-09-13T10:10:54.314465+0000","last_became_peered":"2024-09-13T10:10:54.314465+0000","last_unstale":"2024-09-13T10:11:47.713577+0000","last_undegraded":"2024-09-13T10:11:47.713577+0000","last_fullsized":"2024-09-13T10:11:47.713577+0000","mapping_epoch":31,"log_start":"84'75","ondisk_log_start":"84'75","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'40","last_scrub_stamp":"2024-09-13T10:11:05.436272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.436272+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":72,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:08.571155+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5563183,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":73,"num_read_kb":64902,"num_write":45,"num_write_kb":13246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,10],"acting":[1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"85'261","reported_seq":389,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.569660+0000","last_change":"2024-09-13T10:10:43.220955+0000","last_active":"2024-09-13T10:11:47.569660+0000","last_peered":"2024-09-13T10:11:47.569660+0000","last_clean":"2024-09-13T10:11:47.569660+0000","last_became_active":"2024-09-13T10:10:43.220074+0000","last_became_peered":"2024-09-13T10:10:43.220074+0000","last_unstale":"2024-09-13T10:11:47.569660+0000","last_undegraded":"2024-09-13T10:11:47.569660+0000","last_fullsized":"2024-09-13T10:11:47.569660+0000","mapping_epoch":30,"log_start":"85'258","ondisk_log_start":"85'258","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":270,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:54.562129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21063132,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":234,"num_read_kb":4392,"num_write":338,"num_write_kb":68578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.454952+0000","last_change":"2024-09-13T10:11:39.046876+0000","last_active":"2024-09-13T10:11:47.454952+0000","last_peered":"2024-09-13T10:11:47.454952+0000","last_clean":"2024-09-13T10:11:47.454952+0000","last_became_active":"2024-09-13T10:11:39.046090+0000","last_became_peered":"2024-09-13T10:11:39.046090+0000","last_unstale":"2024-09-13T10:11:47.454952+0000","last_undegraded":"2024-09-13T10:11:47.454952+0000","last_fullsized":"2024-09-13T10:11:47.454952+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:49.333931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,7,9],"acting":[15,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.4","version":"84'141","reported_seq":507,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T10:11:48.266232+0000","last_change":"2024-09-13T10:11:03.633738+0000","last_active":"2024-09-13T10:11:48.266232+0000","last_peered":"2024-09-13T10:11:48.266232+0000","last_clean":"2024-09-13T10:11:48.266232+0000","last_became_active":"2024-09-13T10:10:54.316518+0000","last_became_peered":"2024-09-13T10:10:54.316518+0000","last_unstale":"2024-09-13T10:11:48.266232+0000","last_undegraded":"2024-09-13T10:11:48.266232+0000","last_fullsized":"2024-09-13T10:11:48.266232+0000","mapping_epoch":31,"log_start":"82'138","ondisk_log_start":"82'138","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'66","last_scrub_stamp":"2024-09-13T10:11:03.633499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:03.633499+0000","objects_scrubbed":5,"log_size":3,"log_dups_size":138,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:14.513606+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17597028,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":368,"num_read_kb":352152,"num_write":85,"num_write_kb":24037,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10],"acting":[6,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"81'317","reported_seq":442,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-13T10:11:43.142136+0000","last_change":"2024-09-13T10:10:43.230767+0000","last_active":"2024-09-13T10:11:43.142136+0000","last_peered":"2024-09-13T10:11:43.142136+0000","last_clean":"2024-09-13T10:11:43.142136+0000","last_became_active":"2024-09-13T10:10:43.229961+0000","last_became_peered":"2024-09-13T10:10:43.229961+0000","last_unstale":"2024-09-13T10:11:43.142136+0000","last_undegraded":"2024-09-13T10:11:43.142136+0000","last_fullsized":"2024-09-13T10:11:43.142136+0000","mapping_epoch":30,"log_start":"80'313","ondisk_log_start":"80'313","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":322,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:53.856674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12596224,"num_objects":25,"num_object_clones":0,"num_object_copies":75,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":292,"num_read_kb":3436,"num_write":393,"num_write_kb":74791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":138,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.970504+0000","last_change":"2024-09-13T10:10:32.149042+0000","last_active":"2024-09-13T10:11:47.970504+0000","last_peered":"2024-09-13T10:11:47.970504+0000","last_clean":"2024-09-13T10:11:47.970504+0000","last_became_active":"2024-09-13T10:10:30.272007+0000","last_became_peered":"2024-09-13T10:10:30.272007+0000","last_unstale":"2024-09-13T10:11:47.970504+0000","last_undegraded":"2024-09-13T10:11:47.970504+0000","last_fullsized":"2024-09-13T10:11:47.970504+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:49.552306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000987604,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"82'130","reported_seq":454,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T10:11:48.163513+0000","last_change":"2024-09-13T10:11:00.541038+0000","last_active":"2024-09-13T10:11:48.163513+0000","last_peered":"2024-09-13T10:11:48.163513+0000","last_clean":"2024-09-13T10:11:48.163513+0000","last_became_active":"2024-09-13T10:10:54.315196+0000","last_became_peered":"2024-09-13T10:10:54.315196+0000","last_unstale":"2024-09-13T10:11:48.163513+0000","last_undegraded":"2024-09-13T10:11:48.163513+0000","last_fullsized":"2024-09-13T10:11:48.163513+0000","mapping_epoch":31,"log_start":"79'127","ondisk_log_start":"79'127","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'48","last_scrub_stamp":"2024-09-13T10:11:00.540905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.540905+0000","objects_scrubbed":5,"log_size":3,"log_dups_size":126,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:04.875627+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15059138,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":261,"num_read_kb":212364,"num_write":102,"num_write_kb":28228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,0],"acting":[6,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"39'5","reported_seq":157,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.552724+0000","last_change":"2024-09-13T10:11:00.786261+0000","last_active":"2024-09-13T10:11:47.552724+0000","last_peered":"2024-09-13T10:11:47.552724+0000","last_clean":"2024-09-13T10:11:47.552724+0000","last_became_active":"2024-09-13T10:10:54.314935+0000","last_became_peered":"2024-09-13T10:10:54.314935+0000","last_unstale":"2024-09-13T10:11:47.552724+0000","last_undegraded":"2024-09-13T10:11:47.552724+0000","last_fullsized":"2024-09-13T10:11:47.552724+0000","mapping_epoch":31,"log_start":"38'2","ondisk_log_start":"38'2","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'5","last_scrub_stamp":"2024-09-13T10:11:00.786110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.786110+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:03.174485+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,0],"acting":[3,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"79'42","reported_seq":234,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.709986+0000","last_change":"2024-09-13T10:11:04.446852+0000","last_active":"2024-09-13T10:11:47.709986+0000","last_peered":"2024-09-13T10:11:47.709986+0000","last_clean":"2024-09-13T10:11:47.709986+0000","last_became_active":"2024-09-13T10:10:54.316057+0000","last_became_peered":"2024-09-13T10:10:54.316057+0000","last_unstale":"2024-09-13T10:11:47.709986+0000","last_undegraded":"2024-09-13T10:11:47.709986+0000","last_fullsized":"2024-09-13T10:11:47.709986+0000","mapping_epoch":31,"log_start":"74'39","ondisk_log_start":"74'39","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'29","last_scrub_stamp":"2024-09-13T10:11:04.446717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.446717+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":36,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:17.702973+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2806802,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":57,"num_read_kb":40254,"num_write":13,"num_write_kb":3657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,9],"acting":[11,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.a","version":"69'22","reported_seq":188,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.970475+0000","last_change":"2024-09-13T10:11:12.708135+0000","last_active":"2024-09-13T10:11:47.970475+0000","last_peered":"2024-09-13T10:11:47.970475+0000","last_clean":"2024-09-13T10:11:47.970475+0000","last_became_active":"2024-09-13T10:11:12.707306+0000","last_became_peered":"2024-09-13T10:11:12.707306+0000","last_unstale":"2024-09-13T10:11:47.970475+0000","last_undegraded":"2024-09-13T10:11:47.970475+0000","last_fullsized":"2024-09-13T10:11:47.970475+0000","mapping_epoch":55,"log_start":"54'19","ondisk_log_start":"54'19","created":31,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":19,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:06.698931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":54207,"num_write":21,"num_write_kb":7244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,13],"acting":[6,8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"79'66","reported_seq":298,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-13T10:11:43.098210+0000","last_change":"2024-09-13T10:10:44.227804+0000","last_active":"2024-09-13T10:11:43.098210+0000","last_peered":"2024-09-13T10:11:43.098210+0000","last_clean":"2024-09-13T10:11:43.098210+0000","last_became_active":"2024-09-13T10:10:44.226956+0000","last_became_peered":"2024-09-13T10:10:44.226956+0000","last_unstale":"2024-09-13T10:11:43.098210+0000","last_undegraded":"2024-09-13T10:11:43.098210+0000","last_fullsized":"2024-09-13T10:11:43.098210+0000","mapping_epoch":31,"log_start":"67'63","ondisk_log_start":"67'63","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":63,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:05.539518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9764867,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":260,"num_read_kb":231479,"num_write":60,"num_write_kb":16720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,4],"acting":[14,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.c","version":"86'102","reported_seq":405,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-13T10:11:48.271838+0000","last_change":"2024-09-13T10:10:59.373229+0000","last_active":"2024-09-13T10:11:48.271838+0000","last_peered":"2024-09-13T10:11:48.271838+0000","last_clean":"2024-09-13T10:11:48.271838+0000","last_became_active":"2024-09-13T10:10:59.366341+0000","last_became_peered":"2024-09-13T10:10:59.366341+0000","last_unstale":"2024-09-13T10:11:48.271838+0000","last_undegraded":"2024-09-13T10:11:48.271838+0000","last_fullsized":"2024-09-13T10:11:48.271838+0000","mapping_epoch":44,"log_start":"84'99","ondisk_log_start":"84'99","created":31,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":99,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:04.079822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11042298,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":389,"num_read_kb":351418,"num_write":103,"num_write_kb":27897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,12,3],"acting":[6,12,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"85'54","reported_seq":241,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-13T10:11:47.484363+0000","last_change":"2024-09-13T10:10:59.365358+0000","last_active":"2024-09-13T10:11:47.484363+0000","last_peered":"2024-09-13T10:11:47.484363+0000","last_clean":"2024-09-13T10:11:47.484363+0000","last_became_active":"2024-09-13T10:10:59.364443+0000","last_became_peered":"2024-09-13T10:10:59.364443+0000","last_unstale":"2024-09-13T10:11:47.484363+0000","last_undegraded":"2024-09-13T10:11:47.484363+0000","last_fullsized":"2024-09-13T10:11:47.484363+0000","mapping_epoch":44,"log_start":"84'51","ondisk_log_start":"84'51","created":31,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":51,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:02.930127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6084200,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":103117,"num_write":52,"num_write_kb":14282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,12,7],"acting":[15,12,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.e","version":"74'21","reported_seq":134,"reported_epoch":81,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:11:43.123350+0000","last_change":"2024-09-13T10:11:20.876764+0000","last_active":"2024-09-13T10:11:43.123350+0000","last_peered":"2024-09-13T10:11:43.123350+0000","last_clean":"2024-09-13T10:11:03.155929+0000","last_became_active":"2024-09-13T10:11:20.874865+0000","last_became_peered":"2024-09-13T10:11:20.874865+0000","last_unstale":"2024-09-13T10:11:43.123350+0000","last_undegraded":"2024-09-13T10:11:43.123350+0000","last_fullsized":"2024-09-13T10:11:43.123350+0000","mapping_epoch":61,"log_start":"67'18","ondisk_log_start":"67'18","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":18,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":23639,"num_write":20,"num_write_kb":5797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,15],"acting":[2,8,4],"avail_no_missing":["2","3","4","8","15"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"80'83","reported_seq":294,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-13T10:11:45.826399+0000","last_change":"2024-09-13T10:10:44.224347+0000","last_active":"2024-09-13T10:11:45.826399+0000","last_peered":"2024-09-13T10:11:45.826399+0000","last_clean":"2024-09-13T10:11:45.826399+0000","last_became_active":"2024-09-13T10:10:44.223700+0000","last_became_peered":"2024-09-13T10:10:44.223700+0000","last_unstale":"2024-09-13T10:11:45.826399+0000","last_undegraded":"2024-09-13T10:11:45.826399+0000","last_fullsized":"2024-09-13T10:11:45.826399+0000","mapping_epoch":31,"log_start":"76'80","ondisk_log_start":"76'80","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":80,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:11:50.757686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4991745,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":194499,"num_write":88,"num_write_kb":24842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,7],"acting":[13,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":26,"stat_sum":{"num_bytes":133078684,"num_objects":46,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":2848,"num_read_kb":2562151,"num_write":993,"num_write_kb":280513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":23489322,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":23,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":243793920,"data_stored":243122399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":78,"acting":78,"num_store_stats":16},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":83885160,"num_objects":129,"num_object_clones":0,"num_object_copies":387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":0,"num_read":1000,"num_read_kb":18965,"num_write":1407,"num_write_kb":281430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":235552768,"data_stored":234813055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":12}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247826,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78684,"kb_used_data":47500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94293156,"statfs":{"total":96636764160,"available":96556191744,"internally_reserved":0,"allocated":48640000,"data_stored":48132975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247828,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41448,"kb_used_data":14424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330392,"statfs":{"total":96636764160,"available":96594321408,"internally_reserved":0,"allocated":14770176,"data_stored":14363063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247828,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57680,"kb_used_data":26496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94314160,"statfs":{"total":96636764160,"available":96577699840,"internally_reserved":0,"allocated":27131904,"data_stored":26680469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313236,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36488,"kb_used_data":9464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335352,"statfs":{"total":96636764160,"available":96599400448,"internally_reserved":0,"allocated":9691136,"data_stored":9297987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,5,6,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313235,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70264,"kb_used_data":43240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94301576,"statfs":{"total":96636764160,"available":96564813824,"internally_reserved":0,"allocated":44277760,"data_stored":43834799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7,8,9,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247826,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54776,"kb_used_data":23584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94317064,"statfs":{"total":96636764160,"available":96580673536,"internally_reserved":0,"allocated":24150016,"data_stored":23643067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280531,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59076,"kb_used_data":24700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312764,"statfs":{"total":96636764160,"available":96576270336,"internally_reserved":0,"allocated":25292800,"data_stored":24801092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280531,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94576,"kb_used_data":63392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94277264,"statfs":{"total":96636764160,"available":96539918336,"internally_reserved":0,"allocated":64913408,"data_stored":64349424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280532,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89516,"kb_used_data":58332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94282324,"statfs":{"total":96636764160,"available":96545099776,"internally_reserved":0,"allocated":59731968,"data_stored":59180227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280530,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54544,"kb_used_data":27520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94317296,"statfs":{"total":96636764160,"available":96580911104,"internally_reserved":0,"allocated":28180480,"data_stored":27755414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280531,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":71168,"kb_used_data":39984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94300672,"statfs":{"total":96636764160,"available":96563888128,"internally_reserved":0,"allocated":40943616,"data_stored":40471155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280531,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74412,"kb_used_data":42312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94297428,"statfs":{"total":96636764160,"available":96560566272,"internally_reserved":0,"allocated":43327488,"data_stored":42760473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280531,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35912,"kb_used_data":4728,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335928,"statfs":{"total":96636764160,"available":96599990272,"internally_reserved":0,"allocated":4841472,"data_stored":4445622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,5,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280531,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29940,"kb_used_data":2916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341900,"statfs":{"total":96636764160,"available":96606105600,"internally_reserved":0,"allocated":2985984,"data_stored":2595132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247826,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47336,"kb_used_data":16152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324504,"statfs":{"total":96636764160,"available":96588292096,"internally_reserved":0,"allocated":16539648,"data_stored":16131621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247826,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69120,"kb_used_data":35224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302720,"statfs":{"total":96636764160,"available":96565985280,"internally_reserved":0,"allocated":36069376,"data_stored":35510432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8855552,"data_stored":8810496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13107200,"data_stored":13056387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":45559808,"data_stored":45416013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":51249152,"data_stored":51108624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28876800,"data_stored":28796781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":19181568,"data_stored":19114097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":13107200,"data_stored":13056387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":21483520,"data_stored":21429023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":34131968,"data_stored":34025247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14376960,"data_stored":14289660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13291520,"data_stored":13233189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":15618048,"data_stored":15588691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":17793024,"data_stored":17766950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7516160,"data_stored":7495510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":27213824,"data_stored":27179321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":34279424,"data_stored":34111857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12251136,"data_stored":12234898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":6438912,"data_stored":6433093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":9048064,"data_stored":9038077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":29442048,"data_stored":29313516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":3874816,"data_stored":3869529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":13852672,"data_stored":13820152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":21864448,"data_stored":21854267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":13590528,"data_stored":13564817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":3342336,"data_stored":3328872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:11:50.065 INFO:tasks.thrashosds.thrasher:Setting 4.12 to [1, 4, 13] 2024-09-13T10:11:50.065 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '4.12', '1', '4', '13'] 2024-09-13T10:11:50.065 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 4.12 1 4 13 2024-09-13T10:11:50.066 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:50.081 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:50.082 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:50.094 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:50.102 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:50.102 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:50.121 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:50.115+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:50.122 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:50.123 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:50.139 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:50.140 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:50.160 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:50.160 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:50.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:50.155+0000 7f906dc9a640 0 -- 172.21.15.73:0/942467526 >> v2:172.21.15.181:6804/1173379175 conn(0x7f9068222230 msgr2=0x7f9068224620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:11:50.161 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:50.161 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:50.175 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-13T10:11:50.194 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1673783 2024-09-13T10:11:50.222 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:50.215+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:50.253 INFO:tasks.rados.rados.0.smithi195.stdout:1532: finishing write tid 1 to smithi19530714-20 2024-09-13T10:11:50.253 INFO:tasks.rados.rados.0.smithi195.stdout:1532: finishing write tid 2 to smithi19530714-20 2024-09-13T10:11:50.253 INFO:tasks.rados.rados.0.smithi195.stdout:1532: finishing write tid 3 to smithi19530714-20 2024-09-13T10:11:50.254 INFO:tasks.rados.rados.0.smithi195.stdout:1532: finishing write tid 4 to smithi19530714-20 2024-09-13T10:11:50.254 INFO:tasks.rados.rados.0.smithi195.stdout:1532: finishing write tid 5 to smithi19530714-20 2024-09-13T10:11:50.254 INFO:tasks.rados.rados.0.smithi195.stdout:1532: oid 20 updating version 0 to 91 2024-09-13T10:11:50.254 INFO:tasks.rados.rados.0.smithi195.stdout:1532: oid 20 version 91 is already newer than 89 2024-09-13T10:11:50.254 INFO:tasks.rados.rados.0.smithi195.stdout:1532: oid 20 version 91 is already newer than 90 2024-09-13T10:11:50.254 INFO:tasks.rados.rados.0.smithi195.stdout:1532: oid 20 updating version 91 to 92 2024-09-13T10:11:50.254 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 92 (ObjNum 315 snap 0 seq_num 315) dirty exists 2024-09-13T10:11:50.254 INFO:tasks.rados.rados.0.smithi195.stdout:1532: left oid 20 (ObjNum 315 snap 0 seq_num 315) 2024-09-13T10:11:50.254 INFO:tasks.rados.rados.0.smithi195.stdout:1516: done (15 left) 2024-09-13T10:11:50.254 INFO:tasks.rados.rados.0.smithi195.stdout:1518: done (14 left) 2024-09-13T10:11:50.254 INFO:tasks.rados.rados.0.smithi195.stdout:1519: done (13 left) 2024-09-13T10:11:50.254 INFO:tasks.rados.rados.0.smithi195.stdout:1520: done (12 left) 2024-09-13T10:11:50.254 INFO:tasks.rados.rados.0.smithi195.stdout:1521: done (11 left) 2024-09-13T10:11:50.254 INFO:tasks.rados.rados.0.smithi195.stdout:1522: done (10 left) 2024-09-13T10:11:50.255 INFO:tasks.rados.rados.0.smithi195.stdout:1523: done (9 left) 2024-09-13T10:11:50.255 INFO:tasks.rados.rados.0.smithi195.stdout:1524: done (8 left) 2024-09-13T10:11:50.255 INFO:tasks.rados.rados.0.smithi195.stdout:1525: done (7 left) 2024-09-13T10:11:50.255 INFO:tasks.rados.rados.0.smithi195.stdout:1526: done (6 left) 2024-09-13T10:11:50.255 INFO:tasks.rados.rados.0.smithi195.stdout:1527: done (5 left) 2024-09-13T10:11:50.255 INFO:tasks.rados.rados.0.smithi195.stdout:1528: done (4 left) 2024-09-13T10:11:50.255 INFO:tasks.rados.rados.0.smithi195.stdout:1529: done (3 left) 2024-09-13T10:11:50.255 INFO:tasks.rados.rados.0.smithi195.stdout:1530: done (2 left) 2024-09-13T10:11:50.255 INFO:tasks.rados.rados.0.smithi195.stdout:1531: done (1 left) 2024-09-13T10:11:50.255 INFO:tasks.rados.rados.0.smithi195.stdout:1532: done (0 left) 2024-09-13T10:11:50.255 INFO:tasks.rados.rados.0.smithi195.stdout:1533: read oid 27 snap -1 2024-09-13T10:11:50.255 INFO:tasks.rados.rados.0.smithi195.stdout:1533: expect (ObjNum 221 snap 0 seq_num 221) 2024-09-13T10:11:50.255 INFO:tasks.rados.rados.0.smithi195.stdout:1534: write (excl) oid 39 current snap is 0 2024-09-13T10:11:50.255 INFO:tasks.rados.rados.0.smithi195.stdout:1534: seq_num 316 ranges {511012=549157,1775251=550760,3095195=215817} 2024-09-13T10:11:50.270 INFO:tasks.rados.rados.0.smithi195.stdout:1534: writing smithi19530714-39 from 511012 to 1060169 tid 1 2024-09-13T10:11:50.291 INFO:tasks.rados.rados.0.smithi195.stdout:1534: writing smithi19530714-39 from 1775251 to 2326011 tid 2 2024-09-13T10:11:50.298 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:50.298 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:50.306 INFO:tasks.rados.rados.0.smithi195.stdout:1534: writing smithi19530714-39 from 3095195 to 3311012 tid 3 2024-09-13T10:11:50.306 INFO:tasks.rados.rados.0.smithi195.stdout:1535: write oid 8 current snap is 0 2024-09-13T10:11:50.306 INFO:tasks.rados.rados.0.smithi195.stdout:1535: seq_num 317 ranges {527819=624406,1833733=410771,2722618=601581} 2024-09-13T10:11:50.313 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-13T10:11:50.320 INFO:tasks.rados.rados.0.smithi195.stdout:1535: writing smithi19530714-8 from 527819 to 1152225 tid 1 2024-09-13T10:11:50.322 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:50.317+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:50.330 INFO:tasks.rados.rados.0.smithi195.stdout:1535: writing smithi19530714-8 from 1833733 to 2244504 tid 2 2024-09-13T10:11:50.339 INFO:tasks.rados.rados.0.smithi195.stdout:1535: writing smithi19530714-8 from 2722618 to 3324199 tid 3 2024-09-13T10:11:50.340 INFO:tasks.rados.rados.0.smithi195.stdout:1536: set_chunk oid 37 target oid 2024-09-13T10:11:50.340 INFO:tasks.rados.rados.0.smithi195.stdout:1536: set_chunk oid 37 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:50.340 INFO:tasks.rados.rados.0.smithi195.stdout:1537: write (excl) oid 16 current snap is 0 2024-09-13T10:11:50.340 INFO:tasks.rados.rados.0.smithi195.stdout:1537: seq_num 318 ranges {544626=699655,1892215=554429,3150041=194585} 2024-09-13T10:11:50.349 INFO:tasks.rados.rados.0.smithi195.stdout:1537: writing smithi19530714-16 from 544626 to 1244281 tid 1 2024-09-13T10:11:50.357 INFO:tasks.rados.rados.0.smithi195.stdout:1537: writing smithi19530714-16 from 1892215 to 2446644 tid 2 2024-09-13T10:11:50.363 INFO:tasks.rados.rados.0.smithi195.stdout:1537: writing smithi19530714-16 from 3150041 to 3344626 tid 3 2024-09-13T10:11:50.363 INFO:tasks.rados.rados.0.smithi195.stdout:1538: delete oid 30 current snap is 0 2024-09-13T10:11:50.363 INFO:tasks.rados.rados.0.smithi195.stdout:1533: expect (ObjNum 221 snap 0 seq_num 221) 2024-09-13T10:11:50.385 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:50.385 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:50.415 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:50.416 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:50.423 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:50.417+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:50.477 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:50.477 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:50.491 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-13T10:11:50.524 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:50.518+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:50.577 INFO:tasks.rados.rados.0.smithi195.stdout:1534: finishing write tid 1 to smithi19530714-39 2024-09-13T10:11:50.578 INFO:tasks.rados.rados.0.smithi195.stdout:1534: finishing write tid 2 to smithi19530714-39 2024-09-13T10:11:50.578 INFO:tasks.rados.rados.0.smithi195.stdout:1534: finishing write tid 3 to smithi19530714-39 2024-09-13T10:11:50.578 INFO:tasks.rados.rados.0.smithi195.stdout:1534: finishing write tid 4 to smithi19530714-39 2024-09-13T10:11:50.578 INFO:tasks.rados.rados.0.smithi195.stdout:1534: finishing write tid 5 to smithi19530714-39 2024-09-13T10:11:50.578 INFO:tasks.rados.rados.0.smithi195.stdout:1534: oid 39 updating version 0 to 39 2024-09-13T10:11:50.578 INFO:tasks.rados.rados.0.smithi195.stdout:1534: oid 39 version 39 is already newer than 37 2024-09-13T10:11:50.578 INFO:tasks.rados.rados.0.smithi195.stdout:1534: oid 39 version 39 is already newer than 38 2024-09-13T10:11:50.578 INFO:tasks.rados.rados.0.smithi195.stdout:1534: oid 39 updating version 39 to 40 2024-09-13T10:11:50.578 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 40 (ObjNum 316 snap 0 seq_num 316) dirty exists 2024-09-13T10:11:50.578 INFO:tasks.rados.rados.0.smithi195.stdout:1534: left oid 39 (ObjNum 316 snap 0 seq_num 316) 2024-09-13T10:11:50.579 INFO:tasks.rados.rados.0.smithi195.stdout:1535: finishing write tid 1 to smithi19530714-8 2024-09-13T10:11:50.579 INFO:tasks.rados.rados.0.smithi195.stdout:1535: finishing write tid 2 to smithi19530714-8 2024-09-13T10:11:50.579 INFO:tasks.rados.rados.0.smithi195.stdout:1536: finishing set_chunk to oid 37 2024-09-13T10:11:50.579 INFO:tasks.rados.rados.0.smithi195.stdout:1536: got expected ENOENT (src dne) 2024-09-13T10:11:50.579 INFO:tasks.rados.rados.0.smithi195.stdout:1535: finishing write tid 3 to smithi19530714-8 2024-09-13T10:11:50.579 INFO:tasks.rados.rados.0.smithi195.stdout:1535: finishing write tid 4 to smithi19530714-8 2024-09-13T10:11:50.579 INFO:tasks.rados.rados.0.smithi195.stdout:1535: finishing write tid 5 to smithi19530714-8 2024-09-13T10:11:50.579 INFO:tasks.rados.rados.0.smithi195.stdout:1535: oid 8 updating version 0 to 24 2024-09-13T10:11:50.579 INFO:tasks.rados.rados.0.smithi195.stdout:1535: oid 8 version 24 is already newer than 22 2024-09-13T10:11:50.579 INFO:tasks.rados.rados.0.smithi195.stdout:1535: oid 8 updating version 24 to 25 2024-09-13T10:11:50.579 INFO:tasks.rados.rados.0.smithi195.stdout:1535: oid 8 version 25 is already newer than 23 2024-09-13T10:11:50.579 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 25 (ObjNum 317 snap 0 seq_num 317) dirty exists 2024-09-13T10:11:50.580 INFO:tasks.rados.rados.0.smithi195.stdout:1535: left oid 8 (ObjNum 317 snap 0 seq_num 317) 2024-09-13T10:11:50.580 INFO:tasks.rados.rados.0.smithi195.stdout:1537: finishing write tid 1 to smithi19530714-16 2024-09-13T10:11:50.580 INFO:tasks.rados.rados.0.smithi195.stdout:1537: finishing write tid 2 to smithi19530714-16 2024-09-13T10:11:50.580 INFO:tasks.rados.rados.0.smithi195.stdout:1537: finishing write tid 3 to smithi19530714-16 2024-09-13T10:11:50.580 INFO:tasks.rados.rados.0.smithi195.stdout:1537: finishing write tid 4 to smithi19530714-16 2024-09-13T10:11:50.580 INFO:tasks.rados.rados.0.smithi195.stdout:1537: finishing write tid 5 to smithi19530714-16 2024-09-13T10:11:50.580 INFO:tasks.rados.rados.0.smithi195.stdout:1537: oid 16 updating version 0 to 50 2024-09-13T10:11:50.580 INFO:tasks.rados.rados.0.smithi195.stdout:1537: oid 16 updating version 50 to 51 2024-09-13T10:11:50.580 INFO:tasks.rados.rados.0.smithi195.stdout:1537: oid 16 version 51 is already newer than 48 2024-09-13T10:11:50.580 INFO:tasks.rados.rados.0.smithi195.stdout:1537: oid 16 version 51 is already newer than 49 2024-09-13T10:11:50.580 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 51 (ObjNum 318 snap 0 seq_num 318) dirty exists 2024-09-13T10:11:50.580 INFO:tasks.rados.rados.0.smithi195.stdout:1537: left oid 16 (ObjNum 318 snap 0 seq_num 318) 2024-09-13T10:11:50.580 INFO:tasks.rados.rados.0.smithi195.stdout:1533: done (5 left) 2024-09-13T10:11:50.581 INFO:tasks.rados.rados.0.smithi195.stdout:1534: done (4 left) 2024-09-13T10:11:50.581 INFO:tasks.rados.rados.0.smithi195.stdout:1535: done (3 left) 2024-09-13T10:11:50.581 INFO:tasks.rados.rados.0.smithi195.stdout:1536: done (2 left) 2024-09-13T10:11:50.581 INFO:tasks.rados.rados.0.smithi195.stdout:1537: done (1 left) 2024-09-13T10:11:50.581 INFO:tasks.rados.rados.0.smithi195.stdout:1538: done (0 left) 2024-09-13T10:11:50.581 INFO:tasks.rados.rados.0.smithi195.stdout:1539: tier_flush oid 20 2024-09-13T10:11:50.581 INFO:tasks.rados.rados.0.smithi195.stdout:1539: tier_flush oid 20 snap -1 2024-09-13T10:11:50.581 INFO:tasks.rados.rados.0.smithi195.stdout:1540: read oid 26 snap -1 2024-09-13T10:11:50.581 INFO:tasks.rados.rados.0.smithi195.stdout:1540: expect (ObjNum 292 snap 0 seq_num 292) 2024-09-13T10:11:50.581 INFO:tasks.rados.rados.0.smithi195.stdout:1541: read oid 1 snap -1 2024-09-13T10:11:50.581 INFO:tasks.rados.rados.0.smithi195.stdout:1541: expect deleted 2024-09-13T10:11:50.581 INFO:tasks.rados.rados.0.smithi195.stdout:1542: write (excl) oid 32 current snap is 0 2024-09-13T10:11:50.582 INFO:tasks.rados.rados.0.smithi195.stdout:1542: seq_num 320 ranges {578240=566506,1609179=558098,2804887=551750} 2024-09-13T10:11:50.585 INFO:tasks.rados.rados.0.smithi195.stdout:1542: writing smithi19530714-32 from 578240 to 1144746 tid 1 2024-09-13T10:11:50.593 INFO:tasks.rados.rados.0.smithi195.stdout:1542: writing smithi19530714-32 from 1609179 to 2167277 tid 2 2024-09-13T10:11:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:1542: writing smithi19530714-32 from 2804887 to 3356637 tid 3 2024-09-13T10:11:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:1543: read oid 47 snap -1 2024-09-13T10:11:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:1543: expect (ObjNum 260 snap 0 seq_num 260) 2024-09-13T10:11:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:1544: read oid 50 snap -1 2024-09-13T10:11:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:1544: expect (ObjNum 303 snap 0 seq_num 303) 2024-09-13T10:11:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:1545: read oid 44 snap -1 2024-09-13T10:11:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:1545: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:11:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:1546: read oid 10 snap -1 2024-09-13T10:11:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:1546: expect (ObjNum 296 snap 0 seq_num 296) 2024-09-13T10:11:50.603 INFO:tasks.rados.rados.0.smithi195.stdout:1547: read oid 12 snap -1 2024-09-13T10:11:50.604 INFO:tasks.rados.rados.0.smithi195.stdout:1547: expect (ObjNum 287 snap 0 seq_num 287) 2024-09-13T10:11:50.604 INFO:tasks.rados.rados.0.smithi195.stdout:1548: write (excl) oid 17 current snap is 0 2024-09-13T10:11:50.604 INFO:tasks.rados.rados.0.smithi195.stdout:1548: seq_num 321 ranges {595047=641755,1667661=418109,2548663=763022} 2024-09-13T10:11:50.613 INFO:tasks.rados.rados.0.smithi195.stdout:1548: writing smithi19530714-17 from 595047 to 1236802 tid 1 2024-09-13T10:11:50.620 INFO:tasks.rados.rados.0.smithi195.stdout:1548: writing smithi19530714-17 from 1667661 to 2085770 tid 2 2024-09-13T10:11:50.624 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:50.621+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:50.627 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:50.627 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:50.631 INFO:tasks.rados.rados.0.smithi195.stdout:1548: writing smithi19530714-17 from 2548663 to 3311685 tid 3 2024-09-13T10:11:50.632 INFO:tasks.rados.rados.0.smithi195.stdout:1549: read oid 45 snap -1 2024-09-13T10:11:50.632 INFO:tasks.rados.rados.0.smithi195.stdout:1549: expect (ObjNum 311 snap 0 seq_num 311) 2024-09-13T10:11:50.632 INFO:tasks.rados.rados.0.smithi195.stdout:1550: read oid 28 snap -1 2024-09-13T10:11:50.632 INFO:tasks.rados.rados.0.smithi195.stdout:1550: expect (ObjNum 180 snap 0 seq_num 180) 2024-09-13T10:11:50.632 INFO:tasks.rados.rados.0.smithi195.stdout:1551: read oid 2 snap -1 2024-09-13T10:11:50.632 INFO:tasks.rados.rados.0.smithi195.stdout:1551: expect (ObjNum 280 snap 0 seq_num 280) 2024-09-13T10:11:50.632 INFO:tasks.rados.rados.0.smithi195.stdout:1552: read oid 5 snap -1 2024-09-13T10:11:50.632 INFO:tasks.rados.rados.0.smithi195.stdout:1552: expect (ObjNum 310 snap 0 seq_num 310) 2024-09-13T10:11:50.632 INFO:tasks.rados.rados.0.smithi195.stdout:1553: read oid 35 snap -1 2024-09-13T10:11:50.632 INFO:tasks.rados.rados.0.smithi195.stdout:1553: expect (ObjNum 300 snap 0 seq_num 300) 2024-09-13T10:11:50.632 INFO:tasks.rados.rados.0.smithi195.stdout:1554: set_chunk oid 40 target oid 2024-09-13T10:11:50.632 INFO:tasks.rados.rados.0.smithi195.stdout:1554: set_chunk oid 40 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:50.633 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:50.633 INFO:tasks.rados.rados.0.smithi195.stdout:1540: expect (ObjNum 292 snap 0 seq_num 292) 2024-09-13T10:11:50.633 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:50.648 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T10:11:50.726 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:50.722+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:50.726 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T10:11:50.749 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:50.772 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:50.772 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:50.787 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T10:11:50.806 INFO:tasks.rados.rados.0.smithi195.stdout:1542: finishing write tid 1 to smithi19530714-32 2024-09-13T10:11:50.807 INFO:tasks.rados.rados.0.smithi195.stdout:1542: finishing write tid 2 to smithi19530714-32 2024-09-13T10:11:50.807 INFO:tasks.rados.rados.0.smithi195.stdout:1543: expect (ObjNum 260 snap 0 seq_num 260) 2024-09-13T10:11:50.826 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:50.821+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:50.860 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1494122 2024-09-13T10:11:50.921 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2328648 2024-09-13T10:11:50.922 INFO:tasks.rados.rados.0.smithi195.stdout:1542: finishing write tid 3 to smithi19530714-32 2024-09-13T10:11:50.922 INFO:tasks.rados.rados.0.smithi195.stdout:1542: finishing write tid 4 to smithi19530714-32 2024-09-13T10:11:50.922 INFO:tasks.rados.rados.0.smithi195.stdout:1542: finishing write tid 5 to smithi19530714-32 2024-09-13T10:11:50.922 INFO:tasks.rados.rados.0.smithi195.stdout:1542: oid 32 updating version 0 to 80 2024-09-13T10:11:50.922 INFO:tasks.rados.rados.0.smithi195.stdout:1542: oid 32 updating version 80 to 81 2024-09-13T10:11:50.922 INFO:tasks.rados.rados.0.smithi195.stdout:1542: oid 32 updating version 81 to 82 2024-09-13T10:11:50.923 INFO:tasks.rados.rados.0.smithi195.stdout:1542: oid 32 version 82 is already newer than 79 2024-09-13T10:11:50.923 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 82 (ObjNum 320 snap 0 seq_num 320) dirty exists 2024-09-13T10:11:50.923 INFO:tasks.rados.rados.0.smithi195.stdout:1542: left oid 32 (ObjNum 320 snap 0 seq_num 320) 2024-09-13T10:11:50.923 INFO:tasks.rados.rados.0.smithi195.stdout:1547: expect (ObjNum 287 snap 0 seq_num 287) 2024-09-13T10:11:50.924 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1386963 2024-09-13T10:11:50.927 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:50.921+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:50.953 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:50.954 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:50.968 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T10:11:50.970 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 272909 2024-09-13T10:11:50.996 INFO:tasks.rados.rados.0.smithi195.stdout:1544: expect (ObjNum 303 snap 0 seq_num 303) 2024-09-13T10:11:51.030 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:51.023+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:51.090 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:51.090 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:51.104 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T10:11:51.113 INFO:teuthology.orchestra.run.smithi073.stderr:set 4.12 pg_upmap mapping to [1,4,13] 2024-09-13T10:11:51.114 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-13T10:11:51.130 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:51.125+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:51.137 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 990277 2024-09-13T10:11:51.183 INFO:tasks.rados.rados.0.smithi195.stdout:1546: expect (ObjNum 296 snap 0 seq_num 296) 2024-09-13T10:11:51.184 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 454591 2024-09-13T10:11:51.226 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:51.226 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:51.231 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:51.227+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:51.240 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T10:11:51.265 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 738767 2024-09-13T10:11:51.332 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:51.326+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:51.333 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:51.361 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:51.361 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:51.375 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T10:11:51.412 INFO:tasks.rados.rados.0.smithi195.stdout:1554: finishing set_chunk to oid 40 2024-09-13T10:11:51.412 INFO:tasks.rados.rados.0.smithi195.stdout:1554: got expected ENOENT (src dne) 2024-09-13T10:11:51.412 INFO:tasks.rados.rados.0.smithi195.stdout:1548: finishing write tid 1 to smithi19530714-17 2024-09-13T10:11:51.412 INFO:tasks.rados.rados.0.smithi195.stdout:1545: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:11:51.416 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 925220 2024-09-13T10:11:51.433 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:51.427+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:51.441 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:51.476 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:51.510 INFO:tasks.rados.rados.0.smithi195.stdout:1548: finishing write tid 2 to smithi19530714-17 2024-09-13T10:11:51.510 INFO:tasks.rados.rados.0.smithi195.stdout:1539: got (0) Success 2024-09-13T10:11:51.511 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 0 (ObjNum 315 snap 0 seq_num 315) dirty exists 2024-09-13T10:11:51.511 INFO:tasks.rados.rados.0.smithi195.stdout:1548: finishing write tid 3 to smithi19530714-17 2024-09-13T10:11:51.511 INFO:tasks.rados.rados.0.smithi195.stdout:1553: expect (ObjNum 300 snap 0 seq_num 300) 2024-09-13T10:11:51.534 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:51.527+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:51.539 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:51.539 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:51.555 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-13T10:11:51.621 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2894482 2024-09-13T10:11:51.628 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:51.635 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:51.631+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:51.677 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:51.677 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:51.691 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-13T10:11:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:1548: finishing write tid 4 to smithi19530714-17 2024-09-13T10:11:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:1548: finishing write tid 5 to smithi19530714-17 2024-09-13T10:11:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:1548: oid 17 updating version 0 to 85 2024-09-13T10:11:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:1548: oid 17 version 85 is already newer than 83 2024-09-13T10:11:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:1548: oid 17 version 85 is already newer than 84 2024-09-13T10:11:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:1548: oid 17 updating version 85 to 86 2024-09-13T10:11:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 86 (ObjNum 321 snap 0 seq_num 321) dirty exists 2024-09-13T10:11:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:1548: left oid 17 (ObjNum 321 snap 0 seq_num 321) 2024-09-13T10:11:51.695 INFO:tasks.rados.rados.0.smithi195.stdout:1552: expect (ObjNum 310 snap 0 seq_num 310) 2024-09-13T10:11:51.736 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:51.735+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:51.787 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2557405 2024-09-13T10:11:51.808 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:51.808 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:51.814 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 241787 2024-09-13T10:11:51.823 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-13T10:11:51.837 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:51.831+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:51.878 INFO:tasks.rados.rados.0.smithi195.stdout:1549: expect (ObjNum 311 snap 0 seq_num 311) 2024-09-13T10:11:51.938 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:51.933+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:51.942 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:51.942 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:51.956 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-13T10:11:51.965 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1287252 2024-09-13T10:11:52.002 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:52.038 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:52.035+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:52.070 INFO:tasks.rados.rados.0.smithi195.stdout:1551: expect (ObjNum 280 snap 0 seq_num 280) 2024-09-13T10:11:52.074 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:52.074 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:52.083 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1112039 2024-09-13T10:11:52.088 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-13T10:11:52.139 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:52.133+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:52.198 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1544465 2024-09-13T10:11:52.214 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:52.214 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:52.222 INFO:tasks.rados.rados.0.smithi195.stdout:1550: expect (ObjNum 180 snap 0 seq_num 180) 2024-09-13T10:11:52.222 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:52.230 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-13T10:11:52.277 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:52.274+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:52.341 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:52.335+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:52.354 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:52.355 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:52.369 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T10:11:52.378 INFO:tasks.rados.rados.0.smithi195.stdout:1539: done (15 left) 2024-09-13T10:11:52.378 INFO:tasks.rados.rados.0.smithi195.stdout:1540: done (14 left) 2024-09-13T10:11:52.379 INFO:tasks.rados.rados.0.smithi195.stdout:1541: done (13 left) 2024-09-13T10:11:52.379 INFO:tasks.rados.rados.0.smithi195.stdout:1542: done (12 left) 2024-09-13T10:11:52.379 INFO:tasks.rados.rados.0.smithi195.stdout:1543: done (11 left) 2024-09-13T10:11:52.379 INFO:tasks.rados.rados.0.smithi195.stdout:1544: done (10 left) 2024-09-13T10:11:52.379 INFO:tasks.rados.rados.0.smithi195.stdout:1545: done (9 left) 2024-09-13T10:11:52.379 INFO:tasks.rados.rados.0.smithi195.stdout:1546: done (8 left) 2024-09-13T10:11:52.379 INFO:tasks.rados.rados.0.smithi195.stdout:1547: done (7 left) 2024-09-13T10:11:52.379 INFO:tasks.rados.rados.0.smithi195.stdout:1548: done (6 left) 2024-09-13T10:11:52.379 INFO:tasks.rados.rados.0.smithi195.stdout:1549: done (5 left) 2024-09-13T10:11:52.379 INFO:tasks.rados.rados.0.smithi195.stdout:1550: done (4 left) 2024-09-13T10:11:52.379 INFO:tasks.rados.rados.0.smithi195.stdout:1551: done (3 left) 2024-09-13T10:11:52.380 INFO:tasks.rados.rados.0.smithi195.stdout:1552: done (2 left) 2024-09-13T10:11:52.380 INFO:tasks.rados.rados.0.smithi195.stdout:1553: done (1 left) 2024-09-13T10:11:52.380 INFO:tasks.rados.rados.0.smithi195.stdout:1554: done (0 left) 2024-09-13T10:11:52.380 INFO:tasks.rados.rados.0.smithi195.stdout:1555: write (excl) oid 5 current snap is 0 2024-09-13T10:11:52.380 INFO:tasks.rados.rados.0.smithi195.stdout:1555: seq_num 322 ranges {611854=717004,2126143=561767,3259733=152121} 2024-09-13T10:11:52.399 INFO:tasks.rados.rados.0.smithi195.stdout:1555: writing smithi19530714-5 from 611854 to 1328858 tid 1 2024-09-13T10:11:52.422 INFO:tasks.rados.rados.0.smithi195.stdout:1555: writing smithi19530714-5 from 2126143 to 2687910 tid 2 2024-09-13T10:11:52.432 INFO:tasks.rados.rados.0.smithi195.stdout:1555: writing smithi19530714-5 from 3259733 to 3411854 tid 3 2024-09-13T10:11:52.432 INFO:tasks.rados.rados.0.smithi195.stdout:1556: read oid 43 snap -1 2024-09-13T10:11:52.432 INFO:tasks.rados.rados.0.smithi195.stdout:1556: expect (ObjNum 263 snap 0 seq_num 263) 2024-09-13T10:11:52.432 INFO:tasks.rados.rados.0.smithi195.stdout:1557: write oid 17 current snap is 0 2024-09-13T10:11:52.433 INFO:tasks.rados.rados.0.smithi195.stdout:1555: finishing write tid 1 to smithi19530714-5 2024-09-13T10:11:52.433 INFO:tasks.rados.rados.0.smithi195.stdout:1555: finishing write tid 2 to smithi19530714-5 2024-09-13T10:11:52.433 INFO:tasks.rados.rados.0.smithi195.stdout:1557: seq_num 323 ranges {628661=792253,2184625=421778,3403509=25152} 2024-09-13T10:11:52.439 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:11:52.441 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:52.435+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:52.449 INFO:tasks.rados.rados.0.smithi195.stdout:1557: writing smithi19530714-17 from 628661 to 1420914 tid 1 2024-09-13T10:11:52.463 INFO:tasks.rados.rados.0.smithi195.stdout:1557: writing smithi19530714-17 from 2184625 to 2606403 tid 2 2024-09-13T10:11:52.473 INFO:tasks.rados.rados.0.smithi195.stdout:1557: writing smithi19530714-17 from 3403509 to 3428661 tid 3 2024-09-13T10:11:52.473 INFO:tasks.rados.rados.0.smithi195.stdout:1558: read oid 40 snap -1 2024-09-13T10:11:52.473 INFO:tasks.rados.rados.0.smithi195.stdout:1558: expect deleted 2024-09-13T10:11:52.473 INFO:tasks.rados.rados.0.smithi195.stdout:1559: write (excl) oid 25 current snap is 0 2024-09-13T10:11:52.473 INFO:tasks.rados.rados.0.smithi195.stdout:1559: seq_num 324 ranges {645468=467502,1726754=565436,2798226=596838} 2024-09-13T10:11:52.482 INFO:tasks.rados.rados.0.smithi195.stdout:1559: writing smithi19530714-25 from 645468 to 1112970 tid 1 2024-09-13T10:11:52.491 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:52.491 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:52.492 INFO:tasks.rados.rados.0.smithi195.stdout:1559: writing smithi19530714-25 from 1726754 to 2292190 tid 2 2024-09-13T10:11:52.500 INFO:tasks.rados.rados.0.smithi195.stdout:1559: writing smithi19530714-25 from 2798226 to 3395064 tid 3 2024-09-13T10:11:52.501 INFO:tasks.rados.rados.0.smithi195.stdout:1560: read oid 41 snap -1 2024-09-13T10:11:52.501 INFO:tasks.rados.rados.0.smithi195.stdout:1560: expect (ObjNum 305 snap 0 seq_num 305) 2024-09-13T10:11:52.501 INFO:tasks.rados.rados.0.smithi195.stdout:1561: write oid 16 current snap is 0 2024-09-13T10:11:52.501 INFO:tasks.rados.rados.0.smithi195.stdout:1555: finishing write tid 3 to smithi19530714-5 2024-09-13T10:11:52.501 INFO:tasks.rados.rados.0.smithi195.stdout:1555: finishing write tid 4 to smithi19530714-5 2024-09-13T10:11:52.501 INFO:tasks.rados.rados.0.smithi195.stdout:1555: finishing write tid 5 to smithi19530714-5 2024-09-13T10:11:52.501 INFO:tasks.rados.rados.0.smithi195.stdout:1555: oid 5 updating version 0 to 25 2024-09-13T10:11:52.501 INFO:tasks.rados.rados.0.smithi195.stdout:1555: oid 5 version 25 is already newer than 24 2024-09-13T10:11:52.501 INFO:tasks.rados.rados.0.smithi195.stdout:1555: oid 5 updating version 25 to 26 2024-09-13T10:11:52.501 INFO:tasks.rados.rados.0.smithi195.stdout:1555: oid 5 version 26 is already newer than 23 2024-09-13T10:11:52.502 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 26 (ObjNum 322 snap 0 seq_num 322) dirty exists 2024-09-13T10:11:52.502 INFO:tasks.rados.rados.0.smithi195.stdout:1555: left oid 5 (ObjNum 322 snap 0 seq_num 322) 2024-09-13T10:11:52.502 INFO:tasks.rados.rados.0.smithi195.stdout:1556: expect (ObjNum 263 snap 0 seq_num 263) 2024-09-13T10:11:52.506 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T10:11:52.542 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:52.535+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:52.585 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2050567 2024-09-13T10:11:52.598 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:52.598 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:52.606 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:52.619 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:52.619 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:52.640 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:52.640 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:52.643 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:52.639+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:52.652 INFO:tasks.rados.rados.0.smithi195.stdout:1557: finishing write tid 1 to smithi19530714-17 2024-09-13T10:11:52.652 INFO:tasks.rados.rados.0.smithi195.stdout:1557: finishing write tid 2 to smithi19530714-17 2024-09-13T10:11:52.652 INFO:tasks.rados.rados.0.smithi195.stdout:1557: finishing write tid 3 to smithi19530714-17 2024-09-13T10:11:52.653 INFO:tasks.rados.rados.0.smithi195.stdout:1557: finishing write tid 4 to smithi19530714-17 2024-09-13T10:11:52.653 INFO:tasks.rados.rados.0.smithi195.stdout:1557: finishing write tid 5 to smithi19530714-17 2024-09-13T10:11:52.653 INFO:tasks.rados.rados.0.smithi195.stdout:1557: oid 17 updating version 0 to 90 2024-09-13T10:11:52.653 INFO:tasks.rados.rados.0.smithi195.stdout:1557: oid 17 version 90 is already newer than 89 2024-09-13T10:11:52.653 INFO:tasks.rados.rados.0.smithi195.stdout:1557: oid 17 version 90 is already newer than 87 2024-09-13T10:11:52.653 INFO:tasks.rados.rados.0.smithi195.stdout:1557: oid 17 version 90 is already newer than 88 2024-09-13T10:11:52.653 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 90 (ObjNum 323 snap 0 seq_num 323) dirty exists 2024-09-13T10:11:52.653 INFO:tasks.rados.rados.0.smithi195.stdout:1557: left oid 17 (ObjNum 323 snap 0 seq_num 323) 2024-09-13T10:11:52.653 INFO:tasks.rados.rados.0.smithi195.stdout:1559: finishing write tid 1 to smithi19530714-25 2024-09-13T10:11:52.653 INFO:tasks.rados.rados.0.smithi195.stdout:1559: finishing write tid 2 to smithi19530714-25 2024-09-13T10:11:52.653 INFO:tasks.rados.rados.0.smithi195.stdout:1559: finishing write tid 3 to smithi19530714-25 2024-09-13T10:11:52.654 INFO:tasks.rados.rados.0.smithi195.stdout:1559: finishing write tid 4 to smithi19530714-25 2024-09-13T10:11:52.654 INFO:tasks.rados.rados.0.smithi195.stdout:1559: finishing write tid 5 to smithi19530714-25 2024-09-13T10:11:52.654 INFO:tasks.rados.rados.0.smithi195.stdout:1559: oid 25 updating version 0 to 149 2024-09-13T10:11:52.654 INFO:tasks.rados.rados.0.smithi195.stdout:1559: oid 25 version 149 is already newer than 148 2024-09-13T10:11:52.654 INFO:tasks.rados.rados.0.smithi195.stdout:1559: oid 25 version 149 is already newer than 146 2024-09-13T10:11:52.654 INFO:tasks.rados.rados.0.smithi195.stdout:1559: oid 25 version 149 is already newer than 147 2024-09-13T10:11:52.654 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 149 (ObjNum 324 snap 0 seq_num 324) dirty exists 2024-09-13T10:11:52.654 INFO:tasks.rados.rados.0.smithi195.stdout:1559: left oid 25 (ObjNum 324 snap 0 seq_num 324) 2024-09-13T10:11:52.654 INFO:tasks.rados.rados.0.smithi195.stdout:1560: expect (ObjNum 305 snap 0 seq_num 305) 2024-09-13T10:11:52.658 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:52.658 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:52.666 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:52.666 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:52.666 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:11:52.666 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:11:52.667 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:52.674 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:52.680 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T10:11:52.736 INFO:tasks.rados.rados.0.smithi195.stdout:1561: seq_num 325 ranges {662275=542751,1785236=425447,2942002=520273} 2024-09-13T10:11:52.743 INFO:tasks.rados.rados.0.smithi195.stdout:1561: writing smithi19530714-16 from 662275 to 1205026 tid 1 2024-09-13T10:11:52.744 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:52.738+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:52.749 INFO:tasks.rados.rados.0.smithi195.stdout:1561: writing smithi19530714-16 from 1785236 to 2210683 tid 2 2024-09-13T10:11:52.757 INFO:tasks.rados.rados.0.smithi195.stdout:1561: writing smithi19530714-16 from 2942002 to 3462275 tid 3 2024-09-13T10:11:52.757 INFO:tasks.rados.rados.0.smithi195.stdout:1555: done (6 left) 2024-09-13T10:11:52.757 INFO:tasks.rados.rados.0.smithi195.stdout:1556: done (5 left) 2024-09-13T10:11:52.757 INFO:tasks.rados.rados.0.smithi195.stdout:1557: done (4 left) 2024-09-13T10:11:52.757 INFO:tasks.rados.rados.0.smithi195.stdout:1558: done (3 left) 2024-09-13T10:11:52.757 INFO:tasks.rados.rados.0.smithi195.stdout:1559: done (2 left) 2024-09-13T10:11:52.757 INFO:tasks.rados.rados.0.smithi195.stdout:1560: done (1 left) 2024-09-13T10:11:52.757 INFO:tasks.rados.rados.0.smithi195.stdout:1562: delete oid 22 current snap is 0 2024-09-13T10:11:52.757 INFO:tasks.rados.rados.0.smithi195.stdout:1561: finishing write tid 1 to smithi19530714-16 2024-09-13T10:11:52.758 INFO:tasks.rados.rados.0.smithi195.stdout:1561: finishing write tid 2 to smithi19530714-16 2024-09-13T10:11:52.762 INFO:tasks.rados.rados.0.smithi195.stdout:1561: finishing write tid 3 to smithi19530714-16 2024-09-13T10:11:52.764 INFO:tasks.rados.rados.0.smithi195.stdout:1561: finishing write tid 4 to smithi19530714-16 2024-09-13T10:11:52.764 INFO:tasks.rados.rados.0.smithi195.stdout:1561: finishing write tid 5 to smithi19530714-16 2024-09-13T10:11:52.764 INFO:tasks.rados.rados.0.smithi195.stdout:1561: oid 16 updating version 0 to 55 2024-09-13T10:11:52.764 INFO:tasks.rados.rados.0.smithi195.stdout:1561: oid 16 version 55 is already newer than 53 2024-09-13T10:11:52.765 INFO:tasks.rados.rados.0.smithi195.stdout:1561: oid 16 version 55 is already newer than 52 2024-09-13T10:11:52.765 INFO:tasks.rados.rados.0.smithi195.stdout:1561: oid 16 version 55 is already newer than 54 2024-09-13T10:11:52.765 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 55 (ObjNum 325 snap 0 seq_num 325) dirty exists 2024-09-13T10:11:52.765 INFO:tasks.rados.rados.0.smithi195.stdout:1561: left oid 16 (ObjNum 325 snap 0 seq_num 325) 2024-09-13T10:11:52.766 INFO:tasks.rados.rados.0.smithi195.stdout:1561: done (1 left) 2024-09-13T10:11:52.766 INFO:tasks.rados.rados.0.smithi195.stdout:1562: done (0 left) 2024-09-13T10:11:52.766 INFO:tasks.rados.rados.0.smithi195.stdout:1563: read oid 43 snap -1 2024-09-13T10:11:52.767 INFO:tasks.rados.rados.0.smithi195.stdout:1563: expect (ObjNum 263 snap 0 seq_num 263) 2024-09-13T10:11:52.767 INFO:tasks.rados.rados.0.smithi195.stdout:1564: write (excl) oid 46 current snap is 0 2024-09-13T10:11:52.767 INFO:tasks.rados.rados.0.smithi195.stdout:1564: seq_num 327 ranges {695889=409602,1618553=712763,2996848=499041} 2024-09-13T10:11:52.773 INFO:tasks.rados.rados.0.smithi195.stdout:1564: writing smithi19530714-46 from 695889 to 1105491 tid 1 2024-09-13T10:11:52.780 INFO:tasks.rados.rados.0.smithi195.stdout:1564: writing smithi19530714-46 from 1618553 to 2331316 tid 2 2024-09-13T10:11:52.787 INFO:tasks.rados.rados.0.smithi195.stdout:1564: writing smithi19530714-46 from 2996848 to 3495889 tid 3 2024-09-13T10:11:52.787 INFO:tasks.rados.rados.0.smithi195.stdout:1565: tier_promote oid 26 2024-09-13T10:11:52.787 INFO:tasks.rados.rados.0.smithi195.stdout:1563: expect (ObjNum 263 snap 0 seq_num 263) 2024-09-13T10:11:52.813 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1984743 2024-09-13T10:11:52.845 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:52.839+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:52.848 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 546846 2024-09-13T10:11:52.852 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:52.852 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:11:52.852 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:11:52.852 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:11:52.852 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.853 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:207 3.0 3:332cd44b:::smithi19530714-8 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:52.853 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:11:52.853 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.857131187, 2024-09-13T10:11:52.853 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074034500000000003, 2024-09-13T10:11:52.853 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.853 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.853 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.853 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:52.853 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:52.853 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:52.853 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 207 2024-09-13T10:11:52.853 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.854 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:11:52.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.854 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:11:52.854 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.854 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.854 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.854 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991271+0000", 2024-09-13T10:11:52.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9289999999999998e-06 2024-09-13T10:11:52.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991284+0000", 2024-09-13T10:11:52.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2597e-05 2024-09-13T10:11:52.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.855 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.855 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991313+0000", 2024-09-13T10:11:52.855 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8982000000000001e-05 2024-09-13T10:11:52.855 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.855 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991334+0000", 2024-09-13T10:11:52.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1046000000000001e-05 2024-09-13T10:11:52.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991416+0000", 2024-09-13T10:11:52.856 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.153e-05 2024-09-13T10:11:52.856 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.856 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.856 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.856 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991504+0000", 2024-09-13T10:11:52.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8220999999999994e-05 2024-09-13T10:11:52.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:52.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017232+0000", 2024-09-13T10:11:52.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025728328000000002 2024-09-13T10:11:52.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.857 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.857 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:52.857 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.031588+0000", 2024-09-13T10:11:52.857 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014355557 2024-09-13T10:11:52.857 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034211+0000", 2024-09-13T10:11:52.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026232790000000001 2024-09-13T10:11:52.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065168+0000", 2024-09-13T10:11:52.858 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030956833999999999 2024-09-13T10:11:52.858 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.858 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.858 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.858 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065260+0000", 2024-09-13T10:11:52.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2044000000000002e-05 2024-09-13T10:11:52.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.859 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.859 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065303+0000", 2024-09-13T10:11:52.859 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3152999999999998e-05 2024-09-13T10:11:52.859 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.859 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.859 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.859 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.859 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.859 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:211 3.0 3:0596651d:::smithi19530714-12 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:52.859 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:11:52.859 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.856407808999997, 2024-09-13T10:11:52.860 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.042923122000000001, 2024-09-13T10:11:52.860 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.860 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.860 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.860 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:52.860 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:52.860 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:52.860 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 211 2024-09-13T10:11:52.860 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.860 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.860 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.860 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:11:52.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.861 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.861 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:11:52.861 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.861 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.861 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991995+0000", 2024-09-13T10:11:52.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7640000000000001e-06 2024-09-13T10:11:52.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992007+0000", 2024-09-13T10:11:52.862 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2058999999999999e-05 2024-09-13T10:11:52.862 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.862 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.862 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.862 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992040+0000", 2024-09-13T10:11:52.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3466000000000001e-05 2024-09-13T10:11:52.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992059+0000", 2024-09-13T10:11:52.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8485999999999999e-05 2024-09-13T10:11:52.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.863 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.863 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992136+0000", 2024-09-13T10:11:52.863 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6994000000000002e-05 2024-09-13T10:11:52.863 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.863 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992207+0000", 2024-09-13T10:11:52.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1357000000000001e-05 2024-09-13T10:11:52.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:52.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.006377+0000", 2024-09-13T10:11:52.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014170455 2024-09-13T10:11:52.864 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.864 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.864 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:52.864 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.022844+0000", 2024-09-13T10:11:52.864 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016466275999999998 2024-09-13T10:11:52.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023031+0000", 2024-09-13T10:11:52.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018686000000000001 2024-09-13T10:11:52.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.865 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034788+0000", 2024-09-13T10:11:52.865 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011756967 2024-09-13T10:11:52.865 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.865 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.865 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034857+0000", 2024-09-13T10:11:52.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9169000000000001e-05 2024-09-13T10:11:52.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.866 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.866 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034915+0000", 2024-09-13T10:11:52.866 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8269000000000001e-05 2024-09-13T10:11:52.866 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.866 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.866 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.866 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.866 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.866 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:212 3.0 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:52.866 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:11:52.867 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.856301475999999, 2024-09-13T10:11:52.867 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.073011809999999996, 2024-09-13T10:11:52.867 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.867 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.867 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.867 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:52.867 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:52.867 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:52.867 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 212 2024-09-13T10:11:52.867 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.867 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.867 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.867 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.867 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:11:52.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:11:52.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.868 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.868 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.868 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992099+0000", 2024-09-13T10:11:52.868 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4899999999999995e-07 2024-09-13T10:11:52.868 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992103+0000", 2024-09-13T10:11:52.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.33e-06 2024-09-13T10:11:52.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992110+0000", 2024-09-13T10:11:52.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8399999999999997e-06 2024-09-13T10:11:52.869 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.869 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.869 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.869 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992116+0000", 2024-09-13T10:11:52.869 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6790000000000002e-06 2024-09-13T10:11:52.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992292+0000", 2024-09-13T10:11:52.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017659199999999999 2024-09-13T10:11:52.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992385+0000", 2024-09-13T10:11:52.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2608000000000005e-05 2024-09-13T10:11:52.870 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.870 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.870 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:52.870 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011428+0000", 2024-09-13T10:11:52.870 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.019042631000000001 2024-09-13T10:11:52.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:52.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.022852+0000", 2024-09-13T10:11:52.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011424528999999999 2024-09-13T10:11:52.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.871 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.871 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.031660+0000", 2024-09-13T10:11:52.871 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088079899999999999 2024-09-13T10:11:52.871 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.871 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065032+0000", 2024-09-13T10:11:52.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033372247000000001 2024-09-13T10:11:52.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065073+0000", 2024-09-13T10:11:52.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0598e-05 2024-09-13T10:11:52.872 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.872 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.872 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.872 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065110+0000", 2024-09-13T10:11:52.872 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7116999999999997e-05 2024-09-13T10:11:52.873 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.873 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.873 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.873 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.873 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:202 3.2 e38/30 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:52.873 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:11:52.873 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.846810869999999, 2024-09-13T10:11:52.873 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017280298999999999, 2024-09-13T10:11:52.873 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.873 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.873 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.873 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.873 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:11:52.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:11:52.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.874 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.874 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.874 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.874 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001594+0000", 2024-09-13T10:11:52.874 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5550000000000002e-06 2024-09-13T10:11:52.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009486+0000", 2024-09-13T10:11:52.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078916799999999999 2024-09-13T10:11:52.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009519+0000", 2024-09-13T10:11:52.875 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2898999999999998e-05 2024-09-13T10:11:52.875 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.875 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.875 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.875 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009540+0000", 2024-09-13T10:11:52.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1559000000000001e-05 2024-09-13T10:11:52.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009655+0000", 2024-09-13T10:11:52.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114271 2024-09-13T10:11:52.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.876 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.876 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009683+0000", 2024-09-13T10:11:52.876 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8379000000000001e-05 2024-09-13T10:11:52.876 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.876 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.018766+0000", 2024-09-13T10:11:52.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0090829589999999998 2024-09-13T10:11:52.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.877 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.877 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.018869+0000", 2024-09-13T10:11:52.877 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102997 2024-09-13T10:11:52.877 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.877 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.877 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.877 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.877 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.878 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:216 3.0 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:52.878 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:11:52.878 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.843798333999999, 2024-09-13T10:11:52.878 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.063936142000000001, 2024-09-13T10:11:52.878 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.878 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.878 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.878 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:52.878 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:52.878 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:52.878 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 216 2024-09-13T10:11:52.878 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.878 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.878 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:11:52.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:11:52.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.879 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.879 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.879 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004603+0000", 2024-09-13T10:11:52.879 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5540000000000001e-06 2024-09-13T10:11:52.879 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004615+0000", 2024-09-13T10:11:52.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2597e-05 2024-09-13T10:11:52.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.880 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004637+0000", 2024-09-13T10:11:52.880 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1713e-05 2024-09-13T10:11:52.880 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.880 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.880 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004651+0000", 2024-09-13T10:11:52.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3878e-05 2024-09-13T10:11:52.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004985+0000", 2024-09-13T10:11:52.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033347299999999998 2024-09-13T10:11:52.881 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.881 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.881 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.881 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.005706+0000", 2024-09-13T10:11:52.881 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00072122199999999999 2024-09-13T10:11:52.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:52.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023801+0000", 2024-09-13T10:11:52.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018095466000000001 2024-09-13T10:11:52.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:52.882 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.032153+0000", 2024-09-13T10:11:52.882 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083516010000000002 2024-09-13T10:11:52.882 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.882 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.882 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.035250+0000", 2024-09-13T10:11:52.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030974409999999998 2024-09-13T10:11:52.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068439+0000", 2024-09-13T10:11:52.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033189042000000002 2024-09-13T10:11:52.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.883 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.883 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068499+0000", 2024-09-13T10:11:52.883 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9969000000000001e-05 2024-09-13T10:11:52.883 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.883 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.884 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.884 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068537+0000", 2024-09-13T10:11:52.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8186000000000003e-05 2024-09-13T10:11:52.884 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.884 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.884 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.884 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.884 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.884 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:221 3.0 3:18ae852b:::smithi19530714-22 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:52.884 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:11:52.884 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.809873826999997, 2024-09-13T10:11:52.884 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039809298, 2024-09-13T10:11:52.885 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.885 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.885 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.885 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:52.885 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:52.885 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:52.885 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 221 2024-09-13T10:11:52.885 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.885 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.885 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.885 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.885 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:11:52.885 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:11:52.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.886 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.886 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038528+0000", 2024-09-13T10:11:52.886 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8819999999999999e-06 2024-09-13T10:11:52.886 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.886 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038534+0000", 2024-09-13T10:11:52.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0730000000000001e-06 2024-09-13T10:11:52.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038546+0000", 2024-09-13T10:11:52.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.219e-05 2024-09-13T10:11:52.887 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.887 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.887 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.887 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038566+0000", 2024-09-13T10:11:52.887 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9867000000000001e-05 2024-09-13T10:11:52.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038608+0000", 2024-09-13T10:11:52.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1977999999999997e-05 2024-09-13T10:11:52.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.888 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.888 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.888 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038707+0000", 2024-09-13T10:11:52.888 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8708000000000004e-05 2024-09-13T10:11:52.888 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:52.889 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.057374+0000", 2024-09-13T10:11:52.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018667652999999999 2024-09-13T10:11:52.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:52.889 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061983+0000", 2024-09-13T10:11:52.889 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0046087089999999999 2024-09-13T10:11:52.889 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.889 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.889 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.063027+0000", 2024-09-13T10:11:52.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001044239 2024-09-13T10:11:52.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078233+0000", 2024-09-13T10:11:52.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015205639999999999 2024-09-13T10:11:52.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.890 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078298+0000", 2024-09-13T10:11:52.890 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5180999999999998e-05 2024-09-13T10:11:52.890 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.890 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.890 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.891 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078335+0000", 2024-09-13T10:11:52.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7178000000000002e-05 2024-09-13T10:11:52.891 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.891 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.891 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.891 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.891 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.891 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:225 3.0 3:26960251:::smithi19530714-26 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:52.891 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:11:52.891 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.789947552999998, 2024-09-13T10:11:52.891 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030704875999999999, 2024-09-13T10:11:52.891 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.891 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.892 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.892 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:52.892 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:52.892 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:52.892 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 225 2024-09-13T10:11:52.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.892 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.892 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.892 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:11:52.892 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.892 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.892 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:11:52.893 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058455+0000", 2024-09-13T10:11:52.893 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3220000000000001e-06 2024-09-13T10:11:52.893 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.893 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.893 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.893 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058467+0000", 2024-09-13T10:11:52.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1987999999999999e-05 2024-09-13T10:11:52.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.894 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058502+0000", 2024-09-13T10:11:52.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4873000000000003e-05 2024-09-13T10:11:52.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.894 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.894 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058520+0000", 2024-09-13T10:11:52.894 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7977000000000001e-05 2024-09-13T10:11:52.894 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.894 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060066+0000", 2024-09-13T10:11:52.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001545261 2024-09-13T10:11:52.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060680+0000", 2024-09-13T10:11:52.895 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00061431799999999998 2024-09-13T10:11:52.895 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.895 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.895 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:52.895 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070650+0000", 2024-09-13T10:11:52.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0099703099999999996 2024-09-13T10:11:52.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.896 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:52.896 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078192+0000", 2024-09-13T10:11:52.896 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0075419709999999997 2024-09-13T10:11:52.896 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.896 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084486+0000", 2024-09-13T10:11:52.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0062943779999999998 2024-09-13T10:11:52.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089029+0000", 2024-09-13T10:11:52.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045422379999999997 2024-09-13T10:11:52.897 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.897 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.897 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.897 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089092+0000", 2024-09-13T10:11:52.897 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3435999999999994e-05 2024-09-13T10:11:52.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.898 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.898 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089157+0000", 2024-09-13T10:11:52.898 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4803999999999998e-05 2024-09-13T10:11:52.898 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.898 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.898 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.898 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.898 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.898 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:228 3.0 3:0168a995:::smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:52.899 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:11:52.899 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.788146595000001, 2024-09-13T10:11:52.899 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030139446, 2024-09-13T10:11:52.899 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.899 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.899 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.899 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:52.899 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:52.899 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:52.899 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 228 2024-09-13T10:11:52.899 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.899 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.899 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:11:52.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:11:52.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.900 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.900 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.900 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060255+0000", 2024-09-13T10:11:52.900 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.787e-06 2024-09-13T10:11:52.900 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060262+0000", 2024-09-13T10:11:52.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8630000000000002e-06 2024-09-13T10:11:52.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060285+0000", 2024-09-13T10:11:52.901 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2935e-05 2024-09-13T10:11:52.901 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.901 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.901 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.901 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060294+0000", 2024-09-13T10:11:52.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3940000000000002e-06 2024-09-13T10:11:52.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061041+0000", 2024-09-13T10:11:52.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00074706700000000002 2024-09-13T10:11:52.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.902 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.902 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.902 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061148+0000", 2024-09-13T10:11:52.902 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107364 2024-09-13T10:11:52.902 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:52.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074199+0000", 2024-09-13T10:11:52.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013050447999999999 2024-09-13T10:11:52.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.903 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:52.903 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084506+0000", 2024-09-13T10:11:52.903 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010306789 2024-09-13T10:11:52.903 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.903 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089222+0000", 2024-09-13T10:11:52.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047163480000000004 2024-09-13T10:11:52.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.904 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090310+0000", 2024-09-13T10:11:52.904 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010880919999999999 2024-09-13T10:11:52.904 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.904 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.904 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090356+0000", 2024-09-13T10:11:52.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5522000000000002e-05 2024-09-13T10:11:52.905 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.905 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.905 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.905 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090393+0000", 2024-09-13T10:11:52.905 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6837e-05 2024-09-13T10:11:52.905 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.905 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.905 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.906 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.906 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.906 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:229 3.0 3:1e6cc58c:::smithi19530714-30 low_tier:head [copy-from ver 18] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:52.906 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:11:52.906 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.786840220000002, 2024-09-13T10:11:52.906 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033410479, 2024-09-13T10:11:52.906 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.906 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.906 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.906 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:52.906 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:52.907 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:52.907 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 229 2024-09-13T10:11:52.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.907 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.907 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.907 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:11:52.907 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.907 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.907 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:11:52.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.908 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061562+0000", 2024-09-13T10:11:52.908 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9730000000000002e-06 2024-09-13T10:11:52.908 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.908 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.908 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.909 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061577+0000", 2024-09-13T10:11:52.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5044e-05 2024-09-13T10:11:52.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.909 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061602+0000", 2024-09-13T10:11:52.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4494999999999998e-05 2024-09-13T10:11:52.909 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.909 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.909 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.909 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061615+0000", 2024-09-13T10:11:52.909 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3207000000000001e-05 2024-09-13T10:11:52.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061701+0000", 2024-09-13T10:11:52.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5694999999999999e-05 2024-09-13T10:11:52.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.910 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.910 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.910 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061871+0000", 2024-09-13T10:11:52.910 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017004100000000001 2024-09-13T10:11:52.910 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:52.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.079511+0000", 2024-09-13T10:11:52.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017639708 2024-09-13T10:11:52.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089295+0000", 2024-09-13T10:11:52.911 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0097846359999999993 2024-09-13T10:11:52.911 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.911 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.911 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:52.911 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089316+0000", 2024-09-13T10:11:52.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0288999999999998e-05 2024-09-13T10:11:52.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094875+0000", 2024-09-13T10:11:52.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0055598389999999996 2024-09-13T10:11:52.912 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.912 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.912 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.912 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094931+0000", 2024-09-13T10:11:52.912 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5556999999999997e-05 2024-09-13T10:11:52.913 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.913 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.913 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.913 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094970+0000", 2024-09-13T10:11:52.913 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8995000000000002e-05 2024-09-13T10:11:52.913 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.913 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.913 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.913 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.913 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.913 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:233 3.0 3:24ead787:::smithi19530714-34 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:52.914 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:11:52.914 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.782663716000002, 2024-09-13T10:11:52.914 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047200234000000001, 2024-09-13T10:11:52.914 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.914 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.914 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.914 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:52.914 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:52.914 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:52.914 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 233 2024-09-13T10:11:52.914 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.915 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:11:52.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.915 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:11:52.915 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.915 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.915 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.915 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065737+0000", 2024-09-13T10:11:52.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.283e-06 2024-09-13T10:11:52.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.916 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065745+0000", 2024-09-13T10:11:52.916 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3130000000000001e-06 2024-09-13T10:11:52.916 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.916 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.916 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065757+0000", 2024-09-13T10:11:52.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2627e-05 2024-09-13T10:11:52.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065765+0000", 2024-09-13T10:11:52.917 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4669999999999998e-06 2024-09-13T10:11:52.917 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.917 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.917 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.917 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065816+0000", 2024-09-13T10:11:52.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1545000000000002e-05 2024-09-13T10:11:52.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065940+0000", 2024-09-13T10:11:52.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012366699999999999 2024-09-13T10:11:52.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.918 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.918 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:52.918 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.086064+0000", 2024-09-13T10:11:52.918 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.020124487 2024-09-13T10:11:52.918 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.919 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:52.919 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090747+0000", 2024-09-13T10:11:52.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004682795 2024-09-13T10:11:52.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.919 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.919 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.091235+0000", 2024-09-13T10:11:52.919 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048779900000000002 2024-09-13T10:11:52.919 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.919 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.919 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.920 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112809+0000", 2024-09-13T10:11:52.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.021573967999999999 2024-09-13T10:11:52.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.920 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.920 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112878+0000", 2024-09-13T10:11:52.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9413999999999996e-05 2024-09-13T10:11:52.920 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.920 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.920 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.920 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112936+0000", 2024-09-13T10:11:52.920 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7868999999999998e-05 2024-09-13T10:11:52.921 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.921 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.921 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.921 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.921 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.921 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:227 3.2 e38/30 3:6ac7800c:::smithi19530714-28 low_tier:head v 38'8, mlcod=38'6)", 2024-09-13T10:11:52.921 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:11:52.921 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.774300652999997, 2024-09-13T10:11:52.921 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014433425999999999, 2024-09-13T10:11:52.921 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.921 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.921 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.922 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:11:52.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:11:52.922 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.922 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.922 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.922 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.922 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074101+0000", 2024-09-13T10:11:52.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.511e-06 2024-09-13T10:11:52.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083754+0000", 2024-09-13T10:11:52.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096533799999999996 2024-09-13T10:11:52.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083772+0000", 2024-09-13T10:11:52.923 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8241e-05 2024-09-13T10:11:52.923 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.923 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.923 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.923 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083782+0000", 2024-09-13T10:11:52.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.004e-05 2024-09-13T10:11:52.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083875+0000", 2024-09-13T10:11:52.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.268e-05 2024-09-13T10:11:52.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.924 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.924 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083918+0000", 2024-09-13T10:11:52.924 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-13T10:11:52.924 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.924 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.088456+0000", 2024-09-13T10:11:52.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045385540000000002 2024-09-13T10:11:52.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.925 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.925 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.088532+0000", 2024-09-13T10:11:52.925 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6069000000000006e-05 2024-09-13T10:11:52.925 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.925 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.925 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.925 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.925 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.926 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:239 3.0 3:078df4f4:::smithi19530714-40 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:52.926 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:11:52.926 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.759097140999998, 2024-09-13T10:11:52.926 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025192791999999999, 2024-09-13T10:11:52.926 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.926 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.926 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.926 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:52.926 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:52.926 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:52.926 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 239 2024-09-13T10:11:52.926 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.926 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.926 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:11:52.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:11:52.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.927 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.927 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.927 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.927 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089305+0000", 2024-09-13T10:11:52.927 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1710000000000001e-06 2024-09-13T10:11:52.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089312+0000", 2024-09-13T10:11:52.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6860000000000006e-06 2024-09-13T10:11:52.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089323+0000", 2024-09-13T10:11:52.928 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0467e-05 2024-09-13T10:11:52.928 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.928 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.928 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.928 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089330+0000", 2024-09-13T10:11:52.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3939999999999998e-06 2024-09-13T10:11:52.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089379+0000", 2024-09-13T10:11:52.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8990999999999999e-05 2024-09-13T10:11:52.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.929 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089543+0000", 2024-09-13T10:11:52.929 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016351700000000001 2024-09-13T10:11:52.929 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.929 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.929 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:52.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.098886+0000", 2024-09-13T10:11:52.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0093434980000000004 2024-09-13T10:11:52.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.105855+0000", 2024-09-13T10:11:52.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.006968279 2024-09-13T10:11:52.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.930 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:52.930 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.105868+0000", 2024-09-13T10:11:52.930 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3560999999999999e-05 2024-09-13T10:11:52.930 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.930 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114382+0000", 2024-09-13T10:11:52.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0085134730000000006 2024-09-13T10:11:52.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114458+0000", 2024-09-13T10:11:52.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6143999999999995e-05 2024-09-13T10:11:52.931 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.931 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.931 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.931 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114495+0000", 2024-09-13T10:11:52.931 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7611000000000003e-05 2024-09-13T10:11:52.931 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.931 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.932 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.932 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.932 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.932 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:241 3.0 3:388ca573:::smithi19530714-42 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:52.932 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:11:52.932 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.753223419000001, 2024-09-13T10:11:52.932 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057595649999999998, 2024-09-13T10:11:52.932 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.932 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.932 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.932 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:52.933 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:52.933 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:52.933 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 241 2024-09-13T10:11:52.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.933 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.933 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.933 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:11:52.933 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.933 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.933 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:11:52.934 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.934 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.934 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.934 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.934 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095178+0000", 2024-09-13T10:11:52.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.345e-06 2024-09-13T10:11:52.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095184+0000", 2024-09-13T10:11:52.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4239999999999998e-06 2024-09-13T10:11:52.935 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.935 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.935 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.935 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095198+0000", 2024-09-13T10:11:52.935 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4037e-05 2024-09-13T10:11:52.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095206+0000", 2024-09-13T10:11:52.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1629999999999996e-06 2024-09-13T10:11:52.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.936 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095274+0000", 2024-09-13T10:11:52.936 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7948999999999996e-05 2024-09-13T10:11:52.936 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.936 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.936 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095485+0000", 2024-09-13T10:11:52.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021050100000000001 2024-09-13T10:11:52.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:52.937 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.134651+0000", 2024-09-13T10:11:52.937 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039166200999999998 2024-09-13T10:11:52.937 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.937 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.937 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:52.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.140536+0000", 2024-09-13T10:11:52.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058849719999999996 2024-09-13T10:11:52.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.140980+0000", 2024-09-13T10:11:52.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044394200000000001 2024-09-13T10:11:52.938 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.938 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.938 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.938 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152700+0000", 2024-09-13T10:11:52.938 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011720431999999999 2024-09-13T10:11:52.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152739+0000", 2024-09-13T10:11:52.939 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8448999999999997e-05 2024-09-13T10:11:52.939 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.939 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.939 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.939 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152772+0000", 2024-09-13T10:11:52.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3235000000000003e-05 2024-09-13T10:11:52.940 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.940 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.940 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.940 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.940 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.940 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:247 3.0 3:0b30fd85:::smithi19530714-48 low_tier:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:52.940 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:11:52.940 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.735323768000001, 2024-09-13T10:11:52.940 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032448926000000003, 2024-09-13T10:11:52.940 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.940 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.940 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.941 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:52.941 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:52.941 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:52.941 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 247 2024-09-13T10:11:52.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.941 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.941 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.941 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:11:52.941 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.941 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.941 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:11:52.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113078+0000", 2024-09-13T10:11:52.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8670000000000001e-06 2024-09-13T10:11:52.942 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.942 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.942 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.942 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113086+0000", 2024-09-13T10:11:52.942 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2260000000000005e-06 2024-09-13T10:11:52.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.943 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113104+0000", 2024-09-13T10:11:52.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7887e-05 2024-09-13T10:11:52.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.943 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113115+0000", 2024-09-13T10:11:52.943 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0763e-05 2024-09-13T10:11:52.943 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.943 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.943 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.943 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113163+0000", 2024-09-13T10:11:52.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8541999999999997e-05 2024-09-13T10:11:52.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113322+0000", 2024-09-13T10:11:52.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000159121 2024-09-13T10:11:52.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.944 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:52.944 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.125165+0000", 2024-09-13T10:11:52.944 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011842320999999999 2024-09-13T10:11:52.944 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.944 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.131189+0000", 2024-09-13T10:11:52.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.006024728 2024-09-13T10:11:52.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:52.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.131203+0000", 2024-09-13T10:11:52.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3551e-05 2024-09-13T10:11:52.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.945 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.945 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.945 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145435+0000", 2024-09-13T10:11:52.945 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014232273 2024-09-13T10:11:52.945 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145481+0000", 2024-09-13T10:11:52.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5918000000000003e-05 2024-09-13T10:11:52.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.946 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.946 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.946 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145525+0000", 2024-09-13T10:11:52.946 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3729000000000001e-05 2024-09-13T10:11:52.946 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.946 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.946 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.946 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.947 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:244 3.2 e38/30 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'14, mlcod=38'10)", 2024-09-13T10:11:52.947 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:11:52.947 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.721857964999998, 2024-09-13T10:11:52.947 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014967827, 2024-09-13T10:11:52.947 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.947 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.947 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.947 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.947 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.947 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.947 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:11:52.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.948 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:11:52.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.948 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.948 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126544+0000", 2024-09-13T10:11:52.948 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.03e-06 2024-09-13T10:11:52.948 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.948 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133374+0000", 2024-09-13T10:11:52.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068299119999999996 2024-09-13T10:11:52.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133398+0000", 2024-09-13T10:11:52.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4209000000000001e-05 2024-09-13T10:11:52.949 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.949 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.949 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.949 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133404+0000", 2024-09-13T10:11:52.949 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3570000000000003e-06 2024-09-13T10:11:52.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133457+0000", 2024-09-13T10:11:52.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2320000000000001e-05 2024-09-13T10:11:52.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.950 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.950 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133477+0000", 2024-09-13T10:11:52.950 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0721999999999999e-05 2024-09-13T10:11:52.950 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.950 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.141475+0000", 2024-09-13T10:11:52.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0079973649999999993 2024-09-13T10:11:52.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.951 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.951 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.141510+0000", 2024-09-13T10:11:52.951 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4912e-05 2024-09-13T10:11:52.951 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.951 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.951 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.951 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.951 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.952 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:254 4.8 4:1de9da87:::smithi19530714-5:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:52.952 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.214446+0000", 2024-09-13T10:11:52.952 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 61.633953626999997, 2024-09-13T10:11:52.952 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014408274, 2024-09-13T10:11:52.952 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.952 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.952 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.952 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:52.952 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:52.952 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:52.952 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 254 2024-09-13T10:11:52.952 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.953 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214446+0000", 2024-09-13T10:11:52.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.953 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214446+0000", 2024-09-13T10:11:52.953 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.953 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.953 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.953 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214449+0000", 2024-09-13T10:11:52.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7860000000000001e-06 2024-09-13T10:11:52.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214464+0000", 2024-09-13T10:11:52.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5290000000000001e-05 2024-09-13T10:11:52.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.954 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.954 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.954 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214499+0000", 2024-09-13T10:11:52.954 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5119e-05 2024-09-13T10:11:52.954 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214517+0000", 2024-09-13T10:11:52.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7563000000000001e-05 2024-09-13T10:11:52.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214680+0000", 2024-09-13T10:11:52.955 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016348799999999999 2024-09-13T10:11:52.955 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.955 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.955 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.955 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214937+0000", 2024-09-13T10:11:52.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025682599999999998 2024-09-13T10:11:52.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0,11", 2024-09-13T10:11:52.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225660+0000", 2024-09-13T10:11:52.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010722768000000001 2024-09-13T10:11:52.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.956 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:52.956 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.227197+0000", 2024-09-13T10:11:52.956 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015373749999999999 2024-09-13T10:11:52.956 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.956 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228428+0000", 2024-09-13T10:11:52.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001230863 2024-09-13T10:11:52.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228646+0000", 2024-09-13T10:11:52.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021769700000000001 2024-09-13T10:11:52.957 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.957 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.957 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.957 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228755+0000", 2024-09-13T10:11:52.957 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109551 2024-09-13T10:11:52.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.958 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.958 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228854+0000", 2024-09-13T10:11:52.958 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8948000000000002e-05 2024-09-13T10:11:52.958 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.958 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.958 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.958 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.958 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.958 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1836 3.0 3:0cf17fa4:::e22ed036f41f6a390924cf1e5e56fa98e0edea9e6c33080c396679f1537fbc7a:head [write 0~2655744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-13T10:11:52.958 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:10.675934+0000", 2024-09-13T10:11:52.958 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.172465330999998, 2024-09-13T10:11:52.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015481818, 2024-09-13T10:11:52.959 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.959 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.959 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.959 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:52.959 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:52.959 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:52.959 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1836 2024-09-13T10:11:52.959 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.959 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.959 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.959 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.959 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.675934+0000", 2024-09-13T10:11:52.959 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.675934+0000", 2024-09-13T10:11:52.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.960 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.675939+0000", 2024-09-13T10:11:52.960 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2830000000000003e-06 2024-09-13T10:11:52.960 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.960 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.960 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677769+0000", 2024-09-13T10:11:52.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0018299830000000001 2024-09-13T10:11:52.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677796+0000", 2024-09-13T10:11:52.961 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7858999999999999e-05 2024-09-13T10:11:52.961 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.961 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.961 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.961 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677809+0000", 2024-09-13T10:11:52.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2291e-05 2024-09-13T10:11:52.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677895+0000", 2024-09-13T10:11:52.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6581000000000003e-05 2024-09-13T10:11:52.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.962 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.962 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.678079+0000", 2024-09-13T10:11:52.962 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000183802 2024-09-13T10:11:52.962 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.962 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:52.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.678268+0000", 2024-09-13T10:11:52.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000188383 2024-09-13T10:11:52.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:52.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.683389+0000", 2024-09-13T10:11:52.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0051215840000000002 2024-09-13T10:11:52.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.963 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.963 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.684549+0000", 2024-09-13T10:11:52.963 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0011600169999999999 2024-09-13T10:11:52.963 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.963 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.691333+0000", 2024-09-13T10:11:52.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0067840510000000001 2024-09-13T10:11:52.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.691377+0000", 2024-09-13T10:11:52.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4313000000000001e-05 2024-09-13T10:11:52.964 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.964 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.964 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.964 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.691416+0000", 2024-09-13T10:11:52.964 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8671000000000003e-05 2024-09-13T10:11:52.964 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.965 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.965 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.965 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.965 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.965 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(osd.6.23:97 3.0 3:129f02f0:::86c832e6e94279c78c969e8f58073998e1521a4d8ea062f95fd8fe010a0a109b:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:52.965 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:11.785962+0000", 2024-09-13T10:11:52.965 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.062438121, 2024-09-13T10:11:52.965 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013473951, 2024-09-13T10:11:52.965 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.965 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.965 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.966 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.966 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.785962+0000", 2024-09-13T10:11:52.966 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.966 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.966 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.966 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.785962+0000", 2024-09-13T10:11:52.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.785965+0000", 2024-09-13T10:11:52.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2169999999999999e-06 2024-09-13T10:11:52.967 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.967 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.967 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.967 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.787438+0000", 2024-09-13T10:11:52.967 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001473689 2024-09-13T10:11:52.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.787476+0000", 2024-09-13T10:11:52.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7866e-05 2024-09-13T10:11:52.968 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.968 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.968 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.968 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.787502+0000", 2024-09-13T10:11:52.968 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5695999999999999e-05 2024-09-13T10:11:52.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.787610+0000", 2024-09-13T10:11:52.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010799899999999999 2024-09-13T10:11:52.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.969 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.969 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.969 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.787729+0000", 2024-09-13T10:11:52.969 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000119094 2024-09-13T10:11:52.969 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:52.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.787929+0000", 2024-09-13T10:11:52.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019958700000000001 2024-09-13T10:11:52.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.970 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.970 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:52.970 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.794652+0000", 2024-09-13T10:11:52.970 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0067231000000000001 2024-09-13T10:11:52.970 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.797887+0000", 2024-09-13T10:11:52.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032356770000000002 2024-09-13T10:11:52.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.799299+0000", 2024-09-13T10:11:52.971 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0014111239999999999 2024-09-13T10:11:52.971 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.971 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.971 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.971 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.799383+0000", 2024-09-13T10:11:52.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4439999999999998e-05 2024-09-13T10:11:52.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.972 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.972 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.799435+0000", 2024-09-13T10:11:52.972 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2462e-05 2024-09-13T10:11:52.972 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.972 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.972 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.972 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.972 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.972 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(osd.6.23:144 3.0 3:037b20b5:::49b862fdd3fdabd6bd09ef286a067d3d49fec0ab014d5ef65dd18c71075e148b:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:11:52.972 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:17.489205+0000", 2024-09-13T10:11:52.972 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 35.359195135, 2024-09-13T10:11:52.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013662446, 2024-09-13T10:11:52.973 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.973 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.973 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.973 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.973 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.973 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489205+0000", 2024-09-13T10:11:52.973 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.973 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.973 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489205+0000", 2024-09-13T10:11:52.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489205+0000", 2024-09-13T10:11:52.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9500000000000003e-07 2024-09-13T10:11:52.974 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.974 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.974 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.974 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489258+0000", 2024-09-13T10:11:52.974 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2784000000000001e-05 2024-09-13T10:11:52.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489264+0000", 2024-09-13T10:11:52.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6969999999999996e-06 2024-09-13T10:11:52.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.975 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.975 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.975 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.489270+0000", 2024-09-13T10:11:52.975 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2789999999999998e-06 2024-09-13T10:11:52.975 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.499601+0000", 2024-09-13T10:11:52.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010331252000000001 2024-09-13T10:11:52.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.976 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.976 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.499863+0000", 2024-09-13T10:11:52.976 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026172599999999999 2024-09-13T10:11:52.976 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.976 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:11:52.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.500329+0000", 2024-09-13T10:11:52.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00046600299999999999 2024-09-13T10:11:52.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.977 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:52.977 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.502566+0000", 2024-09-13T10:11:52.977 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002237447 2024-09-13T10:11:52.977 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.977 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.502638+0000", 2024-09-13T10:11:52.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2166e-05 2024-09-13T10:11:52.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:52.978 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.502704+0000", 2024-09-13T10:11:52.978 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5209000000000005e-05 2024-09-13T10:11:52.978 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.978 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.978 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.502782+0000", 2024-09-13T10:11:52.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8801999999999995e-05 2024-09-13T10:11:52.979 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.979 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.979 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.979 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:17.502867+0000", 2024-09-13T10:11:52.979 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4586000000000006e-05 2024-09-13T10:11:52.979 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.979 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.979 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.980 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.980 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.980 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:3445 3.2 e86/30 3:7ba83ace:::4c246a1df6a0d5acaf0a3a85249ff707b21f90a5f3db763dc6d544e5b7cbec73:head v 86'330, mlcod=85'328)", 2024-09-13T10:11:52.980 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:48.129415+0000", 2024-09-13T10:11:52.980 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.7189847470000004, 2024-09-13T10:11:52.980 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014173302, 2024-09-13T10:11:52.980 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:52.980 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:52.980 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:52.981 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:52.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:52.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.129415+0000", 2024-09-13T10:11:52.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.981 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.981 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:52.981 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.129415+0000", 2024-09-13T10:11:52.981 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:52.981 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:52.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.129417+0000", 2024-09-13T10:11:52.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0269999999999998e-06 2024-09-13T10:11:52.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:52.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.137191+0000", 2024-09-13T10:11:52.982 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0077741859999999998 2024-09-13T10:11:52.982 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.982 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.982 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:52.982 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.137220+0000", 2024-09-13T10:11:52.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9062999999999999e-05 2024-09-13T10:11:52.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:52.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.137236+0000", 2024-09-13T10:11:52.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6069000000000001e-05 2024-09-13T10:11:52.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.983 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:52.983 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.137359+0000", 2024-09-13T10:11:52.983 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012320399999999999 2024-09-13T10:11:52.983 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.983 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:52.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.137403+0000", 2024-09-13T10:11:52.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3538999999999998e-05 2024-09-13T10:11:52.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:52.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.143508+0000", 2024-09-13T10:11:52.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061054129999999996 2024-09-13T10:11:52.984 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:52.984 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:52.984 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:52.984 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.143588+0000", 2024-09-13T10:11:52.984 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9801000000000004e-05 2024-09-13T10:11:52.985 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.985 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.985 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.985 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:52.985 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:52.985 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:52.986 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:52.987 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:52.987 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:52.988 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:52.988 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T10:11:52.989 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:52.990 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:52.990 INFO:tasks.rados.rados.0.smithi195.stdout:1564: finishing write tid 1 to smithi19530714-46 2024-09-13T10:11:52.990 INFO:tasks.rados.rados.0.smithi195.stdout:1564: finishing write tid 2 to smithi19530714-46 2024-09-13T10:11:52.990 INFO:tasks.rados.rados.0.smithi195.stdout:1564: finishing write tid 3 to smithi19530714-46 2024-09-13T10:11:52.990 INFO:tasks.rados.rados.0.smithi195.stdout:1564: finishing write tid 4 to smithi19530714-46 2024-09-13T10:11:52.990 INFO:tasks.rados.rados.0.smithi195.stdout:1564: finishing write tid 5 to smithi19530714-46 2024-09-13T10:11:52.990 INFO:tasks.rados.rados.0.smithi195.stdout:1564: oid 46 updating version 0 to 60 2024-09-13T10:11:52.990 INFO:tasks.rados.rados.0.smithi195.stdout:1564: oid 46 updating version 60 to 62 2024-09-13T10:11:52.991 INFO:tasks.rados.rados.0.smithi195.stdout:1564: oid 46 version 62 is already newer than 61 2024-09-13T10:11:52.991 INFO:tasks.rados.rados.0.smithi195.stdout:1564: oid 46 version 62 is already newer than 59 2024-09-13T10:11:52.991 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 62 (ObjNum 327 snap 0 seq_num 327) dirty exists 2024-09-13T10:11:52.991 INFO:tasks.rados.rados.0.smithi195.stdout:1564: left oid 46 (ObjNum 327 snap 0 seq_num 327) 2024-09-13T10:11:52.991 INFO:tasks.rados.rados.0.smithi195.stdout:1563: done (2 left) 2024-09-13T10:11:52.991 INFO:tasks.rados.rados.0.smithi195.stdout:1564: done (1 left) 2024-09-13T10:11:52.991 INFO:tasks.rados.rados.0.smithi195.stdout:1566: set_chunk oid 45 target oid 2024-09-13T10:11:52.991 INFO:tasks.rados.rados.0.smithi195.stdout:1566: set_chunk oid 45 offset: 882688 length: 1654272 target oid aa1ebf9e30ef78119f14a842b71edb9b2f1f0f51eeae4f9d2782065636bce84a offset: 0 snap -1 2024-09-13T10:11:52.991 INFO:tasks.rados.rados.0.smithi195.stdout:1567: write oid 3 current snap is 0 2024-09-13T10:11:52.991 INFO:tasks.rados.rados.0.smithi195.stdout:1567: seq_num 328 ranges {712696=484851,1960682=572774,3307918=204778} 2024-09-13T10:11:52.992 INFO:tasks.rados.rados.0.smithi195.stdout:1567: writing smithi19530714-3 from 712696 to 1197547 tid 1 2024-09-13T10:11:52.992 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:52.981+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:52.992 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:52.992 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:52.995 INFO:tasks.rados.rados.0.smithi195.stdout:1567: writing smithi19530714-3 from 1960682 to 2533456 tid 2 2024-09-13T10:11:53.007 INFO:tasks.rados.rados.0.smithi195.stdout:1567: writing smithi19530714-3 from 3307918 to 3512696 tid 3 2024-09-13T10:11:53.007 INFO:tasks.rados.rados.0.smithi195.stdout:1568: write (excl) oid 46 current snap is 0 2024-09-13T10:11:53.007 INFO:tasks.rados.rados.0.smithi195.stdout:1568: seq_num 329 ranges {729503=560100,2019164=716432,3335341=194162} 2024-09-13T10:11:53.009 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:53.009 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:53.020 INFO:tasks.rados.rados.0.smithi195.stdout:1568: writing smithi19530714-46 from 729503 to 1289603 tid 1 2024-09-13T10:11:53.033 INFO:tasks.rados.rados.0.smithi195.stdout:1568: writing smithi19530714-46 from 2019164 to 2735596 tid 2 2024-09-13T10:11:53.038 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:53.038 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:53.039 INFO:tasks.rados.rados.0.smithi195.stdout:1568: writing smithi19530714-46 from 3335341 to 3529503 tid 3 2024-09-13T10:11:53.039 INFO:tasks.rados.rados.0.smithi195.stdout:1569: read oid 44 snap -1 2024-09-13T10:11:53.040 INFO:tasks.rados.rados.0.smithi195.stdout:1569: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:11:53.040 INFO:tasks.rados.rados.0.smithi195.stdout:1570: write oid 34 current snap is 0 2024-09-13T10:11:53.040 INFO:tasks.rados.rados.0.smithi195.stdout:1566: finishing set_chunk to oid 45 2024-09-13T10:11:53.040 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 145 (ObjNum 311 snap 0 seq_num 311) dirty exists 2024-09-13T10:11:53.040 INFO:tasks.rados.rados.0.smithi195.stdout:1567: finishing write tid 1 to smithi19530714-3 2024-09-13T10:11:53.040 INFO:tasks.rados.rados.0.smithi195.stdout:1567: finishing write tid 2 to smithi19530714-3 2024-09-13T10:11:53.040 INFO:tasks.rados.rados.0.smithi195.stdout:1567: finishing write tid 3 to smithi19530714-3 2024-09-13T10:11:53.040 INFO:tasks.rados.rados.0.smithi195.stdout:1567: finishing write tid 4 to smithi19530714-3 2024-09-13T10:11:53.040 INFO:tasks.rados.rados.0.smithi195.stdout:1567: finishing write tid 5 to smithi19530714-3 2024-09-13T10:11:53.040 INFO:tasks.rados.rados.0.smithi195.stdout:1567: oid 3 updating version 0 to 73 2024-09-13T10:11:53.040 INFO:tasks.rados.rados.0.smithi195.stdout:1567: oid 3 updating version 73 to 74 2024-09-13T10:11:53.040 INFO:tasks.rados.rados.0.smithi195.stdout:1567: oid 3 version 74 is already newer than 71 2024-09-13T10:11:53.041 INFO:tasks.rados.rados.0.smithi195.stdout:1567: oid 3 version 74 is already newer than 72 2024-09-13T10:11:53.041 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 74 (ObjNum 328 snap 0 seq_num 328) dirty exists 2024-09-13T10:11:53.041 INFO:tasks.rados.rados.0.smithi195.stdout:1567: left oid 3 (ObjNum 328 snap 0 seq_num 328) 2024-09-13T10:11:53.041 INFO:tasks.rados.rados.0.smithi195.stdout:1568: finishing write tid 1 to smithi19530714-46 2024-09-13T10:11:53.041 INFO:tasks.rados.rados.0.smithi195.stdout:1568: finishing write tid 2 to smithi19530714-46 2024-09-13T10:11:53.041 INFO:tasks.rados.rados.0.smithi195.stdout:1570: seq_num 330 ranges {746310=635349,2077646=576443,3362764=183546} 2024-09-13T10:11:53.048 INFO:tasks.rados.rados.0.smithi195.stdout:1570: writing smithi19530714-34 from 746310 to 1381659 tid 1 2024-09-13T10:11:53.058 INFO:tasks.rados.rados.0.smithi195.stdout:1570: writing smithi19530714-34 from 2077646 to 2654089 tid 2 2024-09-13T10:11:53.065 INFO:tasks.rados.rados.0.smithi195.stdout:1570: writing smithi19530714-34 from 3362764 to 3546310 tid 3 2024-09-13T10:11:53.065 INFO:tasks.rados.rados.0.smithi195.stdout:1566: done (5 left) 2024-09-13T10:11:53.065 INFO:tasks.rados.rados.0.smithi195.stdout:1567: done (4 left) 2024-09-13T10:11:53.065 INFO:tasks.rados.rados.0.smithi195.stdout:1571: read oid 11 snap -1 2024-09-13T10:11:53.065 INFO:tasks.rados.rados.0.smithi195.stdout:1571: expect deleted 2024-09-13T10:11:53.065 INFO:tasks.rados.rados.0.smithi195.stdout:1572: read oid 12 snap -1 2024-09-13T10:11:53.065 INFO:tasks.rados.rados.0.smithi195.stdout:1572: expect (ObjNum 287 snap 0 seq_num 287) 2024-09-13T10:11:53.065 INFO:tasks.rados.rados.0.smithi195.stdout:1573: read oid 19 snap -1 2024-09-13T10:11:53.065 INFO:tasks.rados.rados.0.smithi195.stdout:1573: expect (ObjNum 160 snap 0 seq_num 160) 2024-09-13T10:11:53.065 INFO:tasks.rados.rados.0.smithi195.stdout:1574: delete oid 49 current snap is 0 2024-09-13T10:11:53.066 INFO:tasks.rados.rados.0.smithi195.stdout:1568: finishing write tid 3 to smithi19530714-46 2024-09-13T10:11:53.066 INFO:tasks.rados.rados.0.smithi195.stdout:1568: finishing write tid 4 to smithi19530714-46 2024-09-13T10:11:53.066 INFO:tasks.rados.rados.0.smithi195.stdout:1568: finishing write tid 5 to smithi19530714-46 2024-09-13T10:11:53.066 INFO:tasks.rados.rados.0.smithi195.stdout:1568: oid 46 updating version 0 to 64 2024-09-13T10:11:53.066 INFO:tasks.rados.rados.0.smithi195.stdout:1568: oid 46 version 64 is already newer than 63 2024-09-13T10:11:53.066 INFO:tasks.rados.rados.0.smithi195.stdout:1568: oid 46 updating version 64 to 65 2024-09-13T10:11:53.066 INFO:tasks.rados.rados.0.smithi195.stdout:1568: oid 46 updating version 65 to 66 2024-09-13T10:11:53.066 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 66 (ObjNum 329 snap 0 seq_num 329) dirty exists 2024-09-13T10:11:53.066 INFO:tasks.rados.rados.0.smithi195.stdout:1568: left oid 46 (ObjNum 329 snap 0 seq_num 329) 2024-09-13T10:11:53.066 INFO:tasks.rados.rados.0.smithi195.stdout:1569: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:11:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:53.071 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:53.088 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:53.083+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:53.099 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:53.099 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:53.122 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:53.122 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:53.122 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:11:53.122 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:53.124 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:53.124 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:53.134 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T10:11:53.140 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T10:11:53.151 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:53.151 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:53.185 INFO:tasks.rados.rados.0.smithi195.stdout:1570: finishing write tid 1 to smithi19530714-34 2024-09-13T10:11:53.185 INFO:tasks.rados.rados.0.smithi195.stdout:1573: expect (ObjNum 160 snap 0 seq_num 160) 2024-09-13T10:11:53.189 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:53.183+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:53.190 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:53.190 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:53.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:53.187+0000 7fe9020d7640 0 -- 172.21.15.73:0/2153658646 >> v2:172.21.15.195:6805/848686366 conn(0x7fe8fc22bf70 msgr2=0x7fe8fc22e360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:11:53.290 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:53.285+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:53.296 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2303512 2024-09-13T10:11:53.299 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:53.299 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:53.299 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:11:53.299 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:11:53.299 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:53.313 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T10:11:53.373 INFO:tasks.rados.rados.0.smithi195.stdout:1570: finishing write tid 2 to smithi19530714-34 2024-09-13T10:11:53.373 INFO:tasks.rados.rados.0.smithi195.stdout:1570: finishing write tid 3 to smithi19530714-34 2024-09-13T10:11:53.373 INFO:tasks.rados.rados.0.smithi195.stdout:1570: finishing write tid 4 to smithi19530714-34 2024-09-13T10:11:53.373 INFO:tasks.rados.rados.0.smithi195.stdout:1570: finishing write tid 5 to smithi19530714-34 2024-09-13T10:11:53.373 INFO:tasks.rados.rados.0.smithi195.stdout:1570: oid 34 updating version 0 to 72 2024-09-13T10:11:53.373 INFO:tasks.rados.rados.0.smithi195.stdout:1570: oid 34 updating version 72 to 73 2024-09-13T10:11:53.373 INFO:tasks.rados.rados.0.smithi195.stdout:1570: oid 34 version 73 is already newer than 71 2024-09-13T10:11:53.373 INFO:tasks.rados.rados.0.smithi195.stdout:1570: oid 34 updating version 73 to 74 2024-09-13T10:11:53.373 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 34 v 74 (ObjNum 330 snap 0 seq_num 330) dirty exists 2024-09-13T10:11:53.373 INFO:tasks.rados.rados.0.smithi195.stdout:1570: left oid 34 (ObjNum 330 snap 0 seq_num 330) 2024-09-13T10:11:53.374 INFO:tasks.rados.rados.0.smithi195.stdout:1568: done (7 left) 2024-09-13T10:11:53.374 INFO:tasks.rados.rados.0.smithi195.stdout:1569: done (6 left) 2024-09-13T10:11:53.374 INFO:tasks.rados.rados.0.smithi195.stdout:1570: done (5 left) 2024-09-13T10:11:53.374 INFO:tasks.rados.rados.0.smithi195.stdout:1571: done (4 left) 2024-09-13T10:11:53.374 INFO:tasks.rados.rados.0.smithi195.stdout:1573: done (3 left) 2024-09-13T10:11:53.374 INFO:tasks.rados.rados.0.smithi195.stdout:1574: done (2 left) 2024-09-13T10:11:53.374 INFO:tasks.rados.rados.0.smithi195.stdout:1575: read oid 44 snap -1 2024-09-13T10:11:53.374 INFO:tasks.rados.rados.0.smithi195.stdout:1575: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:11:53.374 INFO:tasks.rados.rados.0.smithi195.stdout:1576: write oid 43 current snap is 0 2024-09-13T10:11:53.374 INFO:tasks.rados.rados.0.smithi195.stdout:1572: expect (ObjNum 287 snap 0 seq_num 287) 2024-09-13T10:11:53.375 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 462785 2024-09-13T10:11:53.391 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:53.387+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:53.405 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 787334 2024-09-13T10:11:53.417 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:53.417 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:53.427 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:53.457 INFO:tasks.rados.rados.0.smithi195.stdout:1575: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:11:53.483 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:53.483 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:11:53.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:11:53.483 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:11:53.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.484 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4281.0:45 2.0 e25/24 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 25'27, mlcod=25'19)", 2024-09-13T10:11:53.484 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:30.286596+0000", 2024-09-13T10:11:53.484 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 83.192510708, 2024-09-13T10:11:53.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0093831250000000008, 2024-09-13T10:11:53.484 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.484 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.484 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.484 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.485 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286596+0000", 2024-09-13T10:11:53.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.485 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.485 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286596+0000", 2024-09-13T10:11:53.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.485 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.485 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286598+0000", 2024-09-13T10:11:53.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8500000000000001e-06 2024-09-13T10:11:53.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286611+0000", 2024-09-13T10:11:53.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3456e-05 2024-09-13T10:11:53.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286635+0000", 2024-09-13T10:11:53.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4165e-05 2024-09-13T10:11:53.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.286655+0000", 2024-09-13T10:11:53.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9383000000000001e-05 2024-09-13T10:11:53.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289627+0000", 2024-09-13T10:11:53.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029728900000000002 2024-09-13T10:11:53.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.289685+0000", 2024-09-13T10:11:53.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7040000000000003e-05 2024-09-13T10:11:53.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.295871+0000", 2024-09-13T10:11:53.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061868629999999999 2024-09-13T10:11:53.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:30.295979+0000", 2024-09-13T10:11:53.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107478 2024-09-13T10:11:53.488 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.488 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.488 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.489 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:211 3.0 e38/30 3:0596651d:::smithi19530714-12 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:53.489 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:11:53.489 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.470787543999997, 2024-09-13T10:11:53.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013316957000000001, 2024-09-13T10:11:53.489 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.489 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.489 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.489 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:11:53.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:11:53.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008321+0000", 2024-09-13T10:11:53.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-13T10:11:53.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011069+0000", 2024-09-13T10:11:53.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002747893 2024-09-13T10:11:53.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011113+0000", 2024-09-13T10:11:53.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3921000000000002e-05 2024-09-13T10:11:53.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.491 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011140+0000", 2024-09-13T10:11:53.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7662999999999999e-05 2024-09-13T10:11:53.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011313+0000", 2024-09-13T10:11:53.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017232200000000001 2024-09-13T10:11:53.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011366+0000", 2024-09-13T10:11:53.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3888000000000003e-05 2024-09-13T10:11:53.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021577+0000", 2024-09-13T10:11:53.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010210472999999999 2024-09-13T10:11:53.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021636+0000", 2024-09-13T10:11:53.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8953999999999998e-05 2024-09-13T10:11:53.493 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.493 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.493 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.494 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:212 3.0 e38/30 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:53.494 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:11:53.494 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.467470437999999, 2024-09-13T10:11:53.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014217904, 2024-09-13T10:11:53.494 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.494 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.494 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.494 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:11:53.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:11:53.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011638+0000", 2024-09-13T10:11:53.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.672e-06 2024-09-13T10:11:53.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013289+0000", 2024-09-13T10:11:53.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001651511 2024-09-13T10:11:53.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013310+0000", 2024-09-13T10:11:53.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0917999999999999e-05 2024-09-13T10:11:53.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013322+0000", 2024-09-13T10:11:53.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2067e-05 2024-09-13T10:11:53.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.016882+0000", 2024-09-13T10:11:53.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035599130000000001 2024-09-13T10:11:53.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.016934+0000", 2024-09-13T10:11:53.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1715999999999997e-05 2024-09-13T10:11:53.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025787+0000", 2024-09-13T10:11:53.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088531809999999999 2024-09-13T10:11:53.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.498 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.498 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025854+0000", 2024-09-13T10:11:53.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6926000000000003e-05 2024-09-13T10:11:53.499 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.499 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.499 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.499 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:207 3.0 e38/30 3:332cd44b:::smithi19530714-8 low_tier:head v 38'3, mlcod=0'0)", 2024-09-13T10:11:53.499 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:11:53.499 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.461676996999998, 2024-09-13T10:11:53.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016660892, 2024-09-13T10:11:53.499 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.499 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.500 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.500 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:11:53.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:11:53.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017432+0000", 2024-09-13T10:11:53.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1490000000000001e-06 2024-09-13T10:11:53.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021310+0000", 2024-09-13T10:11:53.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003877985 2024-09-13T10:11:53.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021351+0000", 2024-09-13T10:11:53.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1105e-05 2024-09-13T10:11:53.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021376+0000", 2024-09-13T10:11:53.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5185999999999999e-05 2024-09-13T10:11:53.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021727+0000", 2024-09-13T10:11:53.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035131300000000002 2024-09-13T10:11:53.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021792+0000", 2024-09-13T10:11:53.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4368000000000004e-05 2024-09-13T10:11:53.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034027+0000", 2024-09-13T10:11:53.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012235582 2024-09-13T10:11:53.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034090+0000", 2024-09-13T10:11:53.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3204e-05 2024-09-13T10:11:53.503 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.504 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.504 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.504 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:216 3.0 e38/30 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:11:53.504 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:11:53.504 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.455111426999999, 2024-09-13T10:11:53.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011044523000000001, 2024-09-13T10:11:53.504 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.504 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.504 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.504 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:11:53.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:11:53.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023997+0000", 2024-09-13T10:11:53.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.533e-06 2024-09-13T10:11:53.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024872+0000", 2024-09-13T10:11:53.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087517699999999999 2024-09-13T10:11:53.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024894+0000", 2024-09-13T10:11:53.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2571000000000001e-05 2024-09-13T10:11:53.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024909+0000", 2024-09-13T10:11:53.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5158e-05 2024-09-13T10:11:53.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025979+0000", 2024-09-13T10:11:53.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010692329999999999 2024-09-13T10:11:53.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.026027+0000", 2024-09-13T10:11:53.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8158000000000002e-05 2024-09-13T10:11:53.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034992+0000", 2024-09-13T10:11:53.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0089650960000000005 2024-09-13T10:11:53.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.035040+0000", 2024-09-13T10:11:53.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7596999999999998e-05 2024-09-13T10:11:53.508 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.508 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.508 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.509 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:225 3.0 e38/30 3:26960251:::smithi19530714-26 low_tier:head v 38'6, mlcod=38'3)", 2024-09-13T10:11:53.509 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:11:53.509 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.408214350000002, 2024-09-13T10:11:53.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013205889, 2024-09-13T10:11:53.509 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.509 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.509 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.509 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.509 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:11:53.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:11:53.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070894+0000", 2024-09-13T10:11:53.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5e-06 2024-09-13T10:11:53.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072562+0000", 2024-09-13T10:11:53.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016683469999999999 2024-09-13T10:11:53.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072632+0000", 2024-09-13T10:11:53.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9665000000000005e-05 2024-09-13T10:11:53.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072661+0000", 2024-09-13T10:11:53.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9714999999999999e-05 2024-09-13T10:11:53.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072800+0000", 2024-09-13T10:11:53.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013886000000000001 2024-09-13T10:11:53.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072828+0000", 2024-09-13T10:11:53.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7886e-05 2024-09-13T10:11:53.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084041+0000", 2024-09-13T10:11:53.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011213064999999999 2024-09-13T10:11:53.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084098+0000", 2024-09-13T10:11:53.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6851000000000002e-05 2024-09-13T10:11:53.513 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.513 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.513 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.513 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:228 3.0 e38/30 3:0168a995:::smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'7, mlcod=38'3)", 2024-09-13T10:11:53.513 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:11:53.513 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.404430769999998, 2024-09-13T10:11:53.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094647819999999997, 2024-09-13T10:11:53.514 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.514 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.514 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.514 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:11:53.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:11:53.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074677+0000", 2024-09-13T10:11:53.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7099999999999999e-06 2024-09-13T10:11:53.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076599+0000", 2024-09-13T10:11:53.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019212459999999999 2024-09-13T10:11:53.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076616+0000", 2024-09-13T10:11:53.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7008999999999999e-05 2024-09-13T10:11:53.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076641+0000", 2024-09-13T10:11:53.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5380000000000001e-05 2024-09-13T10:11:53.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076772+0000", 2024-09-13T10:11:53.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013101599999999999 2024-09-13T10:11:53.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076867+0000", 2024-09-13T10:11:53.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5262999999999999e-05 2024-09-13T10:11:53.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084106+0000", 2024-09-13T10:11:53.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0072390859999999996 2024-09-13T10:11:53.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084140+0000", 2024-09-13T10:11:53.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4072000000000002e-05 2024-09-13T10:11:53.518 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.518 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.518 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.518 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:252 4.6 4:6ddb2b23:::smithi19530714-3:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:53.518 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:11:53.518 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.357042872000001, 2024-09-13T10:11:53.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.03110711, 2024-09-13T10:11:53.518 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.519 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.519 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.519 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:53.519 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:53.519 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:53.519 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 252 2024-09-13T10:11:53.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.519 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:11:53.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:11:53.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122066+0000", 2024-09-13T10:11:53.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4619999999999999e-06 2024-09-13T10:11:53.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122097+0000", 2024-09-13T10:11:53.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0488999999999999e-05 2024-09-13T10:11:53.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122120+0000", 2024-09-13T10:11:53.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3649999999999999e-05 2024-09-13T10:11:53.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122130+0000", 2024-09-13T10:11:53.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7950000000000002e-06 2024-09-13T10:11:53.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122212+0000", 2024-09-13T10:11:53.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2090999999999997e-05 2024-09-13T10:11:53.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122317+0000", 2024-09-13T10:11:53.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104954 2024-09-13T10:11:53.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:53.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.151952+0000", 2024-09-13T10:11:53.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.029634908000000001 2024-09-13T10:11:53.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:53.523 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152580+0000", 2024-09-13T10:11:53.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062796499999999997 2024-09-13T10:11:53.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.523 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152891+0000", 2024-09-13T10:11:53.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000310747 2024-09-13T10:11:53.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.523 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153077+0000", 2024-09-13T10:11:53.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018588200000000001 2024-09-13T10:11:53.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153130+0000", 2024-09-13T10:11:53.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3016000000000002e-05 2024-09-13T10:11:53.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153171+0000", 2024-09-13T10:11:53.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1151e-05 2024-09-13T10:11:53.524 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.524 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.525 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.525 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:255 4.6 4:6150ba38:::smithi19530714-6:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:53.525 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:11:53.525 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.264628668, 2024-09-13T10:11:53.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011708387000000001, 2024-09-13T10:11:53.525 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.525 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.525 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.525 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:53.525 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:53.525 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:53.526 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 255 2024-09-13T10:11:53.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.526 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:11:53.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:11:53.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214482+0000", 2024-09-13T10:11:53.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8139999999999998e-06 2024-09-13T10:11:53.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214498+0000", 2024-09-13T10:11:53.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6636000000000001e-05 2024-09-13T10:11:53.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214537+0000", 2024-09-13T10:11:53.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8668999999999999e-05 2024-09-13T10:11:53.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.528 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214562+0000", 2024-09-13T10:11:53.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5242999999999999e-05 2024-09-13T10:11:53.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.528 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214770+0000", 2024-09-13T10:11:53.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020771299999999999 2024-09-13T10:11:53.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214996+0000", 2024-09-13T10:11:53.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022615999999999999 2024-09-13T10:11:53.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:53.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.223455+0000", 2024-09-13T10:11:53.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0084591960000000004 2024-09-13T10:11:53.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:53.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225042+0000", 2024-09-13T10:11:53.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001586896 2024-09-13T10:11:53.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225719+0000", 2024-09-13T10:11:53.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00067701700000000003 2024-09-13T10:11:53.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225845+0000", 2024-09-13T10:11:53.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012626299999999999 2024-09-13T10:11:53.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226070+0000", 2024-09-13T10:11:53.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022411600000000001 2024-09-13T10:11:53.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226186+0000", 2024-09-13T10:11:53.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011666400000000001 2024-09-13T10:11:53.531 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.531 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.531 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.532 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:268 4.6 4:6fb28ed4:::smithi19530714-19:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:53.532 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:11:53.532 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.263710107999998, 2024-09-13T10:11:53.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041608707000000002, 2024-09-13T10:11:53.532 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.532 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.532 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.532 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:53.532 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:53.532 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:53.532 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 268 2024-09-13T10:11:53.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.533 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:11:53.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:11:53.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215399+0000", 2024-09-13T10:11:53.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8669999999999998e-06 2024-09-13T10:11:53.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215416+0000", 2024-09-13T10:11:53.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6792000000000001e-05 2024-09-13T10:11:53.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215480+0000", 2024-09-13T10:11:53.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3805000000000003e-05 2024-09-13T10:11:53.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215499+0000", 2024-09-13T10:11:53.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9349999999999999e-05 2024-09-13T10:11:53.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215600+0000", 2024-09-13T10:11:53.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010119 2024-09-13T10:11:53.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215715+0000", 2024-09-13T10:11:53.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011491 2024-09-13T10:11:53.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:53.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.253967+0000", 2024-09-13T10:11:53.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038251470000000003 2024-09-13T10:11:53.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:53.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.255748+0000", 2024-09-13T10:11:53.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017815089999999999 2024-09-13T10:11:53.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256188+0000", 2024-09-13T10:11:53.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00043959799999999997 2024-09-13T10:11:53.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256840+0000", 2024-09-13T10:11:53.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000652591 2024-09-13T10:11:53.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256932+0000", 2024-09-13T10:11:53.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1691e-05 2024-09-13T10:11:53.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.257005+0000", 2024-09-13T10:11:53.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2934000000000003e-05 2024-09-13T10:11:53.538 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.539 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.539 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.539 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:305 4.6 4:6150ba38:::smithi19530714-6:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:53.539 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:11:53.539 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.210479415000002, 2024-09-13T10:11:53.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010323297, 2024-09-13T10:11:53.539 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.539 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.539 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.539 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:53.539 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:53.540 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:53.540 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 305 2024-09-13T10:11:53.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.540 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.540 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.540 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:11:53.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.540 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.540 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:11:53.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268630+0000", 2024-09-13T10:11:53.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.548e-06 2024-09-13T10:11:53.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268644+0000", 2024-09-13T10:11:53.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.394e-05 2024-09-13T10:11:53.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268672+0000", 2024-09-13T10:11:53.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8242000000000002e-05 2024-09-13T10:11:53.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268689+0000", 2024-09-13T10:11:53.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7062e-05 2024-09-13T10:11:53.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.542 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.542 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268789+0000", 2024-09-13T10:11:53.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010026600000000001 2024-09-13T10:11:53.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268905+0000", 2024-09-13T10:11:53.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115908 2024-09-13T10:11:53.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:53.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.276776+0000", 2024-09-13T10:11:53.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078712000000000001 2024-09-13T10:11:53.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:53.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278412+0000", 2024-09-13T10:11:53.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016358670000000001 2024-09-13T10:11:53.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278664+0000", 2024-09-13T10:11:53.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025162600000000002 2024-09-13T10:11:53.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278785+0000", 2024-09-13T10:11:53.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012162100000000001 2024-09-13T10:11:53.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278895+0000", 2024-09-13T10:11:53.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109774 2024-09-13T10:11:53.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278950+0000", 2024-09-13T10:11:53.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5242999999999999e-05 2024-09-13T10:11:53.545 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.545 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.545 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.546 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1023 4.16 4:6fb28ed4:::smithi19530714-19:head [write 1798066~477679] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:53.546 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.078200+0000", 2024-09-13T10:11:53.546 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 57.400906055, 2024-09-13T10:11:53.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0095557869999999996, 2024-09-13T10:11:53.546 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.546 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.546 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.546 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:53.546 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:53.546 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:53.547 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1023 2024-09-13T10:11:53.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.547 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078200+0000", 2024-09-13T10:11:53.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078200+0000", 2024-09-13T10:11:53.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078202+0000", 2024-09-13T10:11:53.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2529999999999999e-06 2024-09-13T10:11:53.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078505+0000", 2024-09-13T10:11:53.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000303228 2024-09-13T10:11:53.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078519+0000", 2024-09-13T10:11:53.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3918e-05 2024-09-13T10:11:53.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.549 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.549 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078526+0000", 2024-09-13T10:11:53.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.311e-06 2024-09-13T10:11:53.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.549 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.549 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078577+0000", 2024-09-13T10:11:53.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1125999999999998e-05 2024-09-13T10:11:53.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078682+0000", 2024-09-13T10:11:53.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104671 2024-09-13T10:11:53.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:53.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078864+0000", 2024-09-13T10:11:53.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018166999999999999 2024-09-13T10:11:53.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:53.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.080583+0000", 2024-09-13T10:11:53.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001719138 2024-09-13T10:11:53.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.085104+0000", 2024-09-13T10:11:53.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045216950000000001 2024-09-13T10:11:53.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.087686+0000", 2024-09-13T10:11:53.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025817499999999998 2024-09-13T10:11:53.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.087721+0000", 2024-09-13T10:11:53.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5009999999999999e-05 2024-09-13T10:11:53.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.087756+0000", 2024-09-13T10:11:53.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5017e-05 2024-09-13T10:11:53.552 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.552 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.552 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.553 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1040 4.6 4:61ec6a5a:::smithi19530714-35:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:53.553 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.095747+0000", 2024-09-13T10:11:53.553 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 57.383359077999998, 2024-09-13T10:11:53.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036815318999999999, 2024-09-13T10:11:53.553 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.553 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.553 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:53.553 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:53.553 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:53.553 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:53.553 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1040 2024-09-13T10:11:53.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.553 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095747+0000", 2024-09-13T10:11:53.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095747+0000", 2024-09-13T10:11:53.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095749+0000", 2024-09-13T10:11:53.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9039999999999999e-06 2024-09-13T10:11:53.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095757+0000", 2024-09-13T10:11:53.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6660000000000008e-06 2024-09-13T10:11:53.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095784+0000", 2024-09-13T10:11:53.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6860999999999999e-05 2024-09-13T10:11:53.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095794+0000", 2024-09-13T10:11:53.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7149999999999993e-06 2024-09-13T10:11:53.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095860+0000", 2024-09-13T10:11:53.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6699999999999995e-05 2024-09-13T10:11:53.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.556 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.556 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095984+0000", 2024-09-13T10:11:53.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123676 2024-09-13T10:11:53.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.557 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.557 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.132563+0000", 2024-09-13T10:11:53.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036578797000000003 2024-09-13T10:11:53.557 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.557 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.557 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.557 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1045 4.6 4:6150ba38:::smithi19530714-6:head [write 527753~708201,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:53.557 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.313424+0000", 2024-09-13T10:11:53.557 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 57.165682144000002, 2024-09-13T10:11:53.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011098433, 2024-09-13T10:11:53.557 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.558 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.558 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.558 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:53.558 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:53.558 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:53.558 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1045 2024-09-13T10:11:53.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.558 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313424+0000", 2024-09-13T10:11:53.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313424+0000", 2024-09-13T10:11:53.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313427+0000", 2024-09-13T10:11:53.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0230000000000001e-06 2024-09-13T10:11:53.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313756+0000", 2024-09-13T10:11:53.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032905500000000003 2024-09-13T10:11:53.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313789+0000", 2024-09-13T10:11:53.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2359e-05 2024-09-13T10:11:53.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313807+0000", 2024-09-13T10:11:53.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8130000000000001e-05 2024-09-13T10:11:53.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313953+0000", 2024-09-13T10:11:53.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014562200000000001 2024-09-13T10:11:53.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.314185+0000", 2024-09-13T10:11:53.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023246399999999999 2024-09-13T10:11:53.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:53.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.314442+0000", 2024-09-13T10:11:53.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000257023 2024-09-13T10:11:53.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:53.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.317258+0000", 2024-09-13T10:11:53.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0028160149999999998 2024-09-13T10:11:53.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.319440+0000", 2024-09-13T10:11:53.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021817149999999999 2024-09-13T10:11:53.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.324214+0000", 2024-09-13T10:11:53.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047739189999999997 2024-09-13T10:11:53.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.324440+0000", 2024-09-13T10:11:53.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000225944 2024-09-13T10:11:53.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.324523+0000", 2024-09-13T10:11:53.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3164000000000005e-05 2024-09-13T10:11:53.564 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.564 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.564 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.564 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2603 4.6 4:6150ba38:::smithi19530714-6:head [setxattr _header (55) in=7b,truncate 3848803] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T10:11:53.564 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:26.522327+0000", 2024-09-13T10:11:53.565 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 26.956779337, 2024-09-13T10:11:53.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0099854200000000001, 2024-09-13T10:11:53.565 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.565 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.565 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.565 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:53.565 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:53.565 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:53.565 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2603 2024-09-13T10:11:53.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.566 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522327+0000", 2024-09-13T10:11:53.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522327+0000", 2024-09-13T10:11:53.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522329+0000", 2024-09-13T10:11:53.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.813e-06 2024-09-13T10:11:53.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522338+0000", 2024-09-13T10:11:53.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1269999999999992e-06 2024-09-13T10:11:53.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522359+0000", 2024-09-13T10:11:53.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1242000000000001e-05 2024-09-13T10:11:53.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522374+0000", 2024-09-13T10:11:53.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5085999999999999e-05 2024-09-13T10:11:53.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.523648+0000", 2024-09-13T10:11:53.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0012740049999999999 2024-09-13T10:11:53.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.568 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.568 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.523917+0000", 2024-09-13T10:11:53.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026876100000000001 2024-09-13T10:11:53.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:53.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.524319+0000", 2024-09-13T10:11:53.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040155200000000001 2024-09-13T10:11:53.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:53.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.525257+0000", 2024-09-13T10:11:53.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00093831600000000002 2024-09-13T10:11:53.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532117+0000", 2024-09-13T10:11:53.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068596530000000003 2024-09-13T10:11:53.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532185+0000", 2024-09-13T10:11:53.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8785e-05 2024-09-13T10:11:53.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532243+0000", 2024-09-13T10:11:53.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7627999999999998e-05 2024-09-13T10:11:53.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532313+0000", 2024-09-13T10:11:53.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9451999999999998e-05 2024-09-13T10:11:53.571 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.571 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.571 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.571 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2604 4.6 4:6150ba38:::smithi19530714-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-09-13T10:11:53.571 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:26.522408+0000", 2024-09-13T10:11:53.571 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 26.95669887, 2024-09-13T10:11:53.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010226071999999999, 2024-09-13T10:11:53.571 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.572 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.572 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:53.572 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:53.572 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:53.572 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:53.572 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2604 2024-09-13T10:11:53.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.572 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522408+0000", 2024-09-13T10:11:53.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522408+0000", 2024-09-13T10:11:53.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522409+0000", 2024-09-13T10:11:53.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3999999999999999e-06 2024-09-13T10:11:53.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522415+0000", 2024-09-13T10:11:53.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7790000000000001e-06 2024-09-13T10:11:53.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522433+0000", 2024-09-13T10:11:53.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7972999999999999e-05 2024-09-13T10:11:53.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522449+0000", 2024-09-13T10:11:53.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5899000000000001e-05 2024-09-13T10:11:53.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.524993+0000", 2024-09-13T10:11:53.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002544499 2024-09-13T10:11:53.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:53.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.525196+0000", 2024-09-13T10:11:53.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202721 2024-09-13T10:11:53.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532374+0000", 2024-09-13T10:11:53.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0071786259999999996 2024-09-13T10:11:53.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532498+0000", 2024-09-13T10:11:53.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012336899999999999 2024-09-13T10:11:53.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532634+0000", 2024-09-13T10:11:53.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013580600000000001 2024-09-13T10:11:53.576 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.576 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.576 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.577 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2991 4.6 4:61ec6a5a:::smithi19530714-35:head [write 453855~696987,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T10:11:53.577 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:37.300931+0000", 2024-09-13T10:11:53.577 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.178175849999999, 2024-09-13T10:11:53.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012633428, 2024-09-13T10:11:53.577 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.577 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.577 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.577 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:53.577 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:53.577 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:53.577 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2991 2024-09-13T10:11:53.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.578 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.300931+0000", 2024-09-13T10:11:53.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.300931+0000", 2024-09-13T10:11:53.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.300934+0000", 2024-09-13T10:11:53.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.856e-06 2024-09-13T10:11:53.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.301789+0000", 2024-09-13T10:11:53.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00085478099999999999 2024-09-13T10:11:53.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.301821+0000", 2024-09-13T10:11:53.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1301999999999999e-05 2024-09-13T10:11:53.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.301838+0000", 2024-09-13T10:11:53.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7229999999999999e-05 2024-09-13T10:11:53.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.302009+0000", 2024-09-13T10:11:53.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017093699999999999 2024-09-13T10:11:53.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op must wait for map", 2024-09-13T10:11:53.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.302027+0000", 2024-09-13T10:11:53.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8487000000000001e-05 2024-09-13T10:11:53.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.304197+0000", 2024-09-13T10:11:53.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002169706 2024-09-13T10:11:53.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.581 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.581 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.304321+0000", 2024-09-13T10:11:53.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012425899999999999 2024-09-13T10:11:53.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:53.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.304430+0000", 2024-09-13T10:11:53.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109323 2024-09-13T10:11:53.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:53.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.307797+0000", 2024-09-13T10:11:53.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033668000000000001 2024-09-13T10:11:53.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313063+0000", 2024-09-13T10:11:53.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.005265947 2024-09-13T10:11:53.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313113+0000", 2024-09-13T10:11:53.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9592000000000002e-05 2024-09-13T10:11:53.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313513+0000", 2024-09-13T10:11:53.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040035 2024-09-13T10:11:53.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313564+0000", 2024-09-13T10:11:53.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0858e-05 2024-09-13T10:11:53.584 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.584 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.584 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.584 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2992 4.6 4:61ec6a5a:::smithi19530714-35:head [write 1890787~548083] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T10:11:53.584 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:37.308827+0000", 2024-09-13T10:11:53.584 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.170279403999999, 2024-09-13T10:11:53.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011134115, 2024-09-13T10:11:53.584 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.584 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.585 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.585 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:53.585 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:53.585 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:53.585 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2992 2024-09-13T10:11:53.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.585 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.308827+0000", 2024-09-13T10:11:53.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.308827+0000", 2024-09-13T10:11:53.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.308830+0000", 2024-09-13T10:11:53.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4000000000000001e-06 2024-09-13T10:11:53.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309309+0000", 2024-09-13T10:11:53.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000478901 2024-09-13T10:11:53.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309348+0000", 2024-09-13T10:11:53.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8182000000000001e-05 2024-09-13T10:11:53.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309376+0000", 2024-09-13T10:11:53.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8082e-05 2024-09-13T10:11:53.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309433+0000", 2024-09-13T10:11:53.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6981000000000002e-05 2024-09-13T10:11:53.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309508+0000", 2024-09-13T10:11:53.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5378999999999998e-05 2024-09-13T10:11:53.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.588 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:53.588 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309603+0000", 2024-09-13T10:11:53.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4871999999999996e-05 2024-09-13T10:11:53.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:53.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313952+0000", 2024-09-13T10:11:53.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0043492139999999997 2024-09-13T10:11:53.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.317432+0000", 2024-09-13T10:11:53.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034796969999999999 2024-09-13T10:11:53.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.318860+0000", 2024-09-13T10:11:53.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001428627 2024-09-13T10:11:53.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.319830+0000", 2024-09-13T10:11:53.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00096928100000000001 2024-09-13T10:11:53.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.319961+0000", 2024-09-13T10:11:53.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013149900000000001 2024-09-13T10:11:53.590 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.590 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.591 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.591 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2993 4.6 4:61ec6a5a:::smithi19530714-35:head [setxattr _header (56) in=7b,truncate 2453855] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T10:11:53.591 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:37.309438+0000", 2024-09-13T10:11:53.591 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.169668563999998, 2024-09-13T10:11:53.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010978862000000001, 2024-09-13T10:11:53.591 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.591 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.591 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:53.591 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:53.592 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:53.592 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:53.592 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2993 2024-09-13T10:11:53.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.592 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309438+0000", 2024-09-13T10:11:53.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309438+0000", 2024-09-13T10:11:53.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309439+0000", 2024-09-13T10:11:53.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3370000000000001e-06 2024-09-13T10:11:53.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309449+0000", 2024-09-13T10:11:53.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0159e-05 2024-09-13T10:11:53.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309463+0000", 2024-09-13T10:11:53.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3305999999999999e-05 2024-09-13T10:11:53.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309473+0000", 2024-09-13T10:11:53.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8169999999999994e-06 2024-09-13T10:11:53.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.310756+0000", 2024-09-13T10:11:53.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0012835100000000001 2024-09-13T10:11:53.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.311065+0000", 2024-09-13T10:11:53.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030920900000000001 2024-09-13T10:11:53.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:11:53.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.311478+0000", 2024-09-13T10:11:53.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000413106 2024-09-13T10:11:53.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:53.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313969+0000", 2024-09-13T10:11:53.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002490216 2024-09-13T10:11:53.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.318098+0000", 2024-09-13T10:11:53.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0041298020000000001 2024-09-13T10:11:53.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:53.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320032+0000", 2024-09-13T10:11:53.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001933349 2024-09-13T10:11:53.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:53.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320356+0000", 2024-09-13T10:11:53.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032416699999999999 2024-09-13T10:11:53.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320417+0000", 2024-09-13T10:11:53.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0884000000000002e-05 2024-09-13T10:11:53.598 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.598 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.598 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.599 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2994 4.6 4:61ec6a5a:::smithi19530714-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e75)", 2024-09-13T10:11:53.599 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:37.309490+0000", 2024-09-13T10:11:53.599 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.169616246, 2024-09-13T10:11:53.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011140761000000001, 2024-09-13T10:11:53.599 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:53.599 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:53.599 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:53.599 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:53.599 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:53.599 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:53.599 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2994 2024-09-13T10:11:53.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.599 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:53.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:53.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309490+0000", 2024-09-13T10:11:53.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:53.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309490+0000", 2024-09-13T10:11:53.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:53.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:53.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309491+0000", 2024-09-13T10:11:53.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2699999999999998e-07 2024-09-13T10:11:53.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:53.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309495+0000", 2024-09-13T10:11:53.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7459999999999998e-06 2024-09-13T10:11:53.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:53.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309505+0000", 2024-09-13T10:11:53.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7969999999999995e-06 2024-09-13T10:11:53.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:53.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309516+0000", 2024-09-13T10:11:53.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1222e-05 2024-09-13T10:11:53.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313661+0000", 2024-09-13T10:11:53.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0041448309999999999 2024-09-13T10:11:53.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:53.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313920+0000", 2024-09-13T10:11:53.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025941099999999997 2024-09-13T10:11:53.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:53.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320456+0000", 2024-09-13T10:11:53.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0065354030000000004 2024-09-13T10:11:53.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:53.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320539+0000", 2024-09-13T10:11:53.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3935000000000002e-05 2024-09-13T10:11:53.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:53.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:53.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:53.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320631+0000", 2024-09-13T10:11:53.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1489000000000004e-05 2024-09-13T10:11:53.604 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.604 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.604 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.604 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:53.604 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:53.604 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:53.606 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-13T10:11:53.608 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1076712 2024-09-13T10:11:53.608 INFO:tasks.rados.rados.0.smithi195.stdout:1576: seq_num 332 ranges {779924=785847,2078257=580112,3301257=278667} 2024-09-13T10:11:53.608 INFO:tasks.rados.rados.0.smithi195.stdout:1576: writing smithi19530714-43 from 779924 to 1565771 tid 1 2024-09-13T10:11:53.608 INFO:tasks.rados.rados.0.smithi195.stdout:1576: writing smithi19530714-43 from 2078257 to 2658369 tid 2 2024-09-13T10:11:53.608 INFO:tasks.rados.rados.0.smithi195.stdout:1576: writing smithi19530714-43 from 3301257 to 3579924 tid 3 2024-09-13T10:11:53.608 INFO:tasks.rados.rados.0.smithi195.stdout:1572: done (3 left) 2024-09-13T10:11:53.608 INFO:tasks.rados.rados.0.smithi195.stdout:1575: done (2 left) 2024-09-13T10:11:53.608 INFO:tasks.rados.rados.0.smithi195.stdout:1577: read oid 14 snap -1 2024-09-13T10:11:53.608 INFO:tasks.rados.rados.0.smithi195.stdout:1577: expect (ObjNum 243 snap 0 seq_num 243) 2024-09-13T10:11:53.609 INFO:tasks.rados.rados.0.smithi195.stdout:1578: write oid 13 current snap is 0 2024-09-13T10:11:53.609 INFO:tasks.rados.rados.0.smithi195.stdout:1576: finishing write tid 1 to smithi19530714-43 2024-09-13T10:11:53.609 INFO:tasks.rados.rados.0.smithi195.stdout:1578: seq_num 333 ranges {796731=461096,1736739=723770,2928680=668051} 2024-09-13T10:11:53.609 INFO:tasks.rados.rados.0.smithi195.stdout:1578: writing smithi19530714-13 from 796731 to 1257827 tid 1 2024-09-13T10:11:53.609 INFO:tasks.rados.rados.0.smithi195.stdout:1578: writing smithi19530714-13 from 1736739 to 2460509 tid 2 2024-09-13T10:11:53.609 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:53.599+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:53.614 INFO:tasks.rados.rados.0.smithi195.stdout:1578: writing smithi19530714-13 from 2928680 to 3596731 tid 3 2024-09-13T10:11:53.614 INFO:tasks.rados.rados.0.smithi195.stdout:1579: read oid 38 snap -1 2024-09-13T10:11:53.614 INFO:tasks.rados.rados.0.smithi195.stdout:1579: expect (ObjNum 298 snap 0 seq_num 298) 2024-09-13T10:11:53.614 INFO:tasks.rados.rados.0.smithi195.stdout:1580: write (excl) oid 12 current snap is 0 2024-09-13T10:11:53.614 INFO:tasks.rados.rados.0.smithi195.stdout:1580: seq_num 334 ranges {413538=536345,1395221=583781,2556103=658617} 2024-09-13T10:11:53.619 INFO:tasks.rados.rados.0.smithi195.stdout:1580: writing smithi19530714-12 from 413538 to 949883 tid 1 2024-09-13T10:11:53.625 INFO:tasks.rados.rados.0.smithi195.stdout:1580: writing smithi19530714-12 from 1395221 to 1979002 tid 2 2024-09-13T10:11:53.633 INFO:tasks.rados.rados.0.smithi195.stdout:1580: writing smithi19530714-12 from 2556103 to 3214720 tid 3 2024-09-13T10:11:53.633 INFO:tasks.rados.rados.0.smithi195.stdout:1581: write oid 42 current snap is 0 2024-09-13T10:11:53.633 INFO:tasks.rados.rados.0.smithi195.stdout:1576: finishing write tid 2 to smithi19530714-43 2024-09-13T10:11:53.633 INFO:tasks.rados.rados.0.smithi195.stdout:1581: seq_num 335 ranges {430345=727947,1570056=727439,2699879=469889} 2024-09-13T10:11:53.639 INFO:tasks.rados.rados.0.smithi195.stdout:1581: writing smithi19530714-42 from 430345 to 1158292 tid 1 2024-09-13T10:11:53.646 INFO:tasks.rados.rados.0.smithi195.stdout:1581: writing smithi19530714-42 from 1570056 to 2297495 tid 2 2024-09-13T10:11:53.650 INFO:tasks.rados.rados.0.smithi195.stdout:1581: writing smithi19530714-42 from 2699879 to 3169768 tid 3 2024-09-13T10:11:53.651 INFO:tasks.rados.rados.0.smithi195.stdout:1582: write (excl) oid 24 current snap is 0 2024-09-13T10:11:53.651 INFO:tasks.rados.rados.0.smithi195.stdout:1582: seq_num 336 ranges {447152=403196,1512185=587450,2727302=681161} 2024-09-13T10:11:53.655 INFO:tasks.rados.rados.0.smithi195.stdout:1582: writing smithi19530714-24 from 447152 to 850348 tid 1 2024-09-13T10:11:53.662 INFO:tasks.rados.rados.0.smithi195.stdout:1582: writing smithi19530714-24 from 1512185 to 2099635 tid 2 2024-09-13T10:11:53.670 INFO:tasks.rados.rados.0.smithi195.stdout:1582: writing smithi19530714-24 from 2727302 to 3408463 tid 3 2024-09-13T10:11:53.670 INFO:tasks.rados.rados.0.smithi195.stdout:1583: set_chunk oid 22 target oid 2024-09-13T10:11:53.670 INFO:tasks.rados.rados.0.smithi195.stdout:1583: set_chunk oid 22 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:53.670 INFO:tasks.rados.rados.0.smithi195.stdout:1584: read oid 23 snap -1 2024-09-13T10:11:53.670 INFO:tasks.rados.rados.0.smithi195.stdout:1584: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:11:53.670 INFO:tasks.rados.rados.0.smithi195.stdout:1585: tier_promote oid 36 2024-09-13T10:11:53.670 INFO:tasks.rados.rados.0.smithi195.stdout:1586: set_chunk oid 28 target oid 2024-09-13T10:11:53.700 INFO:tasks.rados.rados.0.smithi195.stdout:1586: set_chunk oid 28 offset: 93184 length: 2083840 target oid 649b3a8d88e73db7ee38b8af18bbb809b2a710632140455a6623d76ac2823066 offset: 0 snap -1 2024-09-13T10:11:53.700 INFO:tasks.rados.rados.0.smithi195.stdout:1587: read oid 50 snap -1 2024-09-13T10:11:53.701 INFO:tasks.rados.rados.0.smithi195.stdout:1587: expect (ObjNum 303 snap 0 seq_num 303) 2024-09-13T10:11:53.701 INFO:tasks.rados.rados.0.smithi195.stdout:1588: write oid 6 current snap is 0 2024-09-13T10:11:53.701 INFO:tasks.rados.rados.0.smithi195.stdout:1588: seq_num 337 ranges {463959=478445,1570667=731108,3038372=492433} 2024-09-13T10:11:53.707 INFO:tasks.rados.rados.0.smithi195.stdout:1588: writing smithi19530714-6 from 463959 to 942404 tid 1 2024-09-13T10:11:53.707 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:53.703+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:53.714 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-13T10:11:53.715 INFO:tasks.rados.rados.0.smithi195.stdout:1588: writing smithi19530714-6 from 1570667 to 2301775 tid 2 2024-09-13T10:11:53.724 INFO:tasks.rados.rados.0.smithi195.stdout:1588: writing smithi19530714-6 from 3038372 to 3530805 tid 3 2024-09-13T10:11:53.724 INFO:tasks.rados.rados.0.smithi195.stdout:1589: read oid 15 snap -1 2024-09-13T10:11:53.724 INFO:tasks.rados.rados.0.smithi195.stdout:1589: expect deleted 2024-09-13T10:11:53.724 INFO:tasks.rados.rados.0.smithi195.stdout:1590: write oid 18 current snap is 0 2024-09-13T10:11:53.724 INFO:tasks.rados.rados.0.smithi195.stdout:1576: finishing write tid 3 to smithi19530714-43 2024-09-13T10:11:53.724 INFO:tasks.rados.rados.0.smithi195.stdout:1576: finishing write tid 4 to smithi19530714-43 2024-09-13T10:11:53.724 INFO:tasks.rados.rados.0.smithi195.stdout:1576: finishing write tid 5 to smithi19530714-43 2024-09-13T10:11:53.724 INFO:tasks.rados.rados.0.smithi195.stdout:1576: oid 43 updating version 0 to 87 2024-09-13T10:11:53.725 INFO:tasks.rados.rados.0.smithi195.stdout:1576: oid 43 version 87 is already newer than 86 2024-09-13T10:11:53.725 INFO:tasks.rados.rados.0.smithi195.stdout:1576: oid 43 version 87 is already newer than 84 2024-09-13T10:11:53.725 INFO:tasks.rados.rados.0.smithi195.stdout:1576: oid 43 version 87 is already newer than 85 2024-09-13T10:11:53.725 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 87 (ObjNum 332 snap 0 seq_num 332) dirty exists 2024-09-13T10:11:53.725 INFO:tasks.rados.rados.0.smithi195.stdout:1576: left oid 43 (ObjNum 332 snap 0 seq_num 332) 2024-09-13T10:11:53.725 INFO:tasks.rados.rados.0.smithi195.stdout:1577: expect (ObjNum 243 snap 0 seq_num 243) 2024-09-13T10:11:53.749 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:53.749 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:53.749 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:11:53.749 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:53.762 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-13T10:11:53.796 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:53.832 INFO:tasks.rados.rados.0.smithi195.stdout:1578: finishing write tid 1 to smithi19530714-13 2024-09-13T10:11:53.833 INFO:tasks.rados.rados.0.smithi195.stdout:1590: seq_num 338 ranges {480766=553694,1629149=591119,2782148=420058} 2024-09-13T10:11:53.833 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:53.829+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:53.839 INFO:tasks.rados.rados.0.smithi195.stdout:1590: writing smithi19530714-18 from 480766 to 1034460 tid 1 2024-09-13T10:11:53.847 INFO:tasks.rados.rados.0.smithi195.stdout:1590: writing smithi19530714-18 from 1629149 to 2220268 tid 2 2024-09-13T10:11:53.853 INFO:tasks.rados.rados.0.smithi195.stdout:1590: writing smithi19530714-18 from 2782148 to 3202206 tid 3 2024-09-13T10:11:53.853 INFO:tasks.rados.rados.0.smithi195.stdout:1576: done (15 left) 2024-09-13T10:11:53.853 INFO:tasks.rados.rados.0.smithi195.stdout:1577: done (14 left) 2024-09-13T10:11:53.854 INFO:tasks.rados.rados.0.smithi195.stdout:1591: set_chunk oid 16 target oid 2024-09-13T10:11:53.880 INFO:tasks.rados.rados.0.smithi195.stdout:1591: set_chunk oid 16 offset: 344576 length: 1782272 target oid 989de9f18158d38e3c2846cd6c5625b2e4406737a04dca59570c2ce866cf967c offset: 0 snap -1 2024-09-13T10:11:53.880 INFO:tasks.rados.rados.0.smithi195.stdout:1592: read oid 31 snap -1 2024-09-13T10:11:53.880 INFO:tasks.rados.rados.0.smithi195.stdout:1592: expect (ObjNum 272 snap 0 seq_num 272) 2024-09-13T10:11:53.880 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:11:53.881 INFO:tasks.rados.rados.0.smithi195.stdout:1578: finishing write tid 2 to smithi19530714-13 2024-09-13T10:11:53.881 INFO:tasks.rados.rados.0.smithi195.stdout:1578: finishing write tid 3 to smithi19530714-13 2024-09-13T10:11:53.881 INFO:tasks.rados.rados.0.smithi195.stdout:1578: finishing write tid 4 to smithi19530714-13 2024-09-13T10:11:53.881 INFO:tasks.rados.rados.0.smithi195.stdout:1578: finishing write tid 5 to smithi19530714-13 2024-09-13T10:11:53.881 INFO:tasks.rados.rados.0.smithi195.stdout:1578: oid 13 updating version 0 to 75 2024-09-13T10:11:53.881 INFO:tasks.rados.rados.0.smithi195.stdout:1578: oid 13 updating version 75 to 78 2024-09-13T10:11:53.881 INFO:tasks.rados.rados.0.smithi195.stdout:1578: oid 13 version 78 is already newer than 76 2024-09-13T10:11:53.881 INFO:tasks.rados.rados.0.smithi195.stdout:1578: oid 13 version 78 is already newer than 77 2024-09-13T10:11:53.881 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 78 (ObjNum 333 snap 0 seq_num 333) dirty exists 2024-09-13T10:11:53.881 INFO:tasks.rados.rados.0.smithi195.stdout:1578: left oid 13 (ObjNum 333 snap 0 seq_num 333) 2024-09-13T10:11:53.881 INFO:tasks.rados.rados.0.smithi195.stdout:1579: expect (ObjNum 298 snap 0 seq_num 298) 2024-09-13T10:11:53.881 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 81571 2024-09-13T10:11:53.909 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:53.905+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:53.925 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:53.926 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:53.926 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:11:53.926 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:11:53.926 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:53.940 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-13T10:11:54.010 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:54.007+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:54.110 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:54.110 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:11:54.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:11:54.111 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:11:54.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.111 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:89 4.9 4:96fcf081:::smithi19530714-23:head [write 769754~688184,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:54.111 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.809155+0000", 2024-09-13T10:11:54.111 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.297050030999998, 2024-09-13T10:11:54.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011562566, 2024-09-13T10:11:54.111 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.111 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.111 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.111 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.111 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.111 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.111 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 89 2024-09-13T10:11:54.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.112 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.809155+0000", 2024-09-13T10:11:54.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.809155+0000", 2024-09-13T10:11:54.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.809158+0000", 2024-09-13T10:11:54.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5770000000000001e-06 2024-09-13T10:11:54.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810210+0000", 2024-09-13T10:11:54.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010519889999999999 2024-09-13T10:11:54.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810238+0000", 2024-09-13T10:11:54.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8449000000000002e-05 2024-09-13T10:11:54.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810265+0000", 2024-09-13T10:11:54.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6832e-05 2024-09-13T10:11:54.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810374+0000", 2024-09-13T10:11:54.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109274 2024-09-13T10:11:54.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810519+0000", 2024-09-13T10:11:54.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014449299999999999 2024-09-13T10:11:54.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:54.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810684+0000", 2024-09-13T10:11:54.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000165744 2024-09-13T10:11:54.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816442+0000", 2024-09-13T10:11:54.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057578990000000004 2024-09-13T10:11:54.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816685+0000", 2024-09-13T10:11:54.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024254999999999999 2024-09-13T10:11:54.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820614+0000", 2024-09-13T10:11:54.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0039290970000000003 2024-09-13T10:11:54.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820668+0000", 2024-09-13T10:11:54.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3567999999999999e-05 2024-09-13T10:11:54.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820718+0000", 2024-09-13T10:11:54.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0093999999999998e-05 2024-09-13T10:11:54.117 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.117 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.117 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.117 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:90 4.9 4:96fcf081:::smithi19530714-23:head [write 2021192~348562] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:54.117 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810322+0000", 2024-09-13T10:11:54.118 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.295882796000001, 2024-09-13T10:11:54.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01166355, 2024-09-13T10:11:54.118 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.118 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.118 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.118 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.118 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.118 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.118 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 90 2024-09-13T10:11:54.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.118 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810322+0000", 2024-09-13T10:11:54.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810322+0000", 2024-09-13T10:11:54.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810324+0000", 2024-09-13T10:11:54.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.097e-06 2024-09-13T10:11:54.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810494+0000", 2024-09-13T10:11:54.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016961499999999999 2024-09-13T10:11:54.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810515+0000", 2024-09-13T10:11:54.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0939999999999999e-05 2024-09-13T10:11:54.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810533+0000", 2024-09-13T10:11:54.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7909000000000001e-05 2024-09-13T10:11:54.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.814429+0000", 2024-09-13T10:11:54.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0038957409999999999 2024-09-13T10:11:54.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.814603+0000", 2024-09-13T10:11:54.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174294 2024-09-13T10:11:54.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:54.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.814804+0000", 2024-09-13T10:11:54.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020149799999999999 2024-09-13T10:11:54.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820569+0000", 2024-09-13T10:11:54.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057648389999999999 2024-09-13T10:11:54.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820761+0000", 2024-09-13T10:11:54.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019123 2024-09-13T10:11:54.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821893+0000", 2024-09-13T10:11:54.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001132051 2024-09-13T10:11:54.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821949+0000", 2024-09-13T10:11:54.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6832000000000001e-05 2024-09-13T10:11:54.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821986+0000", 2024-09-13T10:11:54.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6504000000000002e-05 2024-09-13T10:11:54.124 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.124 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.124 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.125 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:91 4.9 4:96fcf081:::smithi19530714-23:head [setxattr _header (56) in=7b,truncate 2369754] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:54.125 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810564+0000", 2024-09-13T10:11:54.125 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.295641586000002, 2024-09-13T10:11:54.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01178625, 2024-09-13T10:11:54.125 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.125 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.125 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.125 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.125 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.125 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.125 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 91 2024-09-13T10:11:54.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.126 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810564+0000", 2024-09-13T10:11:54.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810564+0000", 2024-09-13T10:11:54.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810565+0000", 2024-09-13T10:11:54.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3960000000000001e-06 2024-09-13T10:11:54.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810574+0000", 2024-09-13T10:11:54.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9339999999999995e-06 2024-09-13T10:11:54.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810590+0000", 2024-09-13T10:11:54.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5858999999999998e-05 2024-09-13T10:11:54.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810604+0000", 2024-09-13T10:11:54.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4533e-05 2024-09-13T10:11:54.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816017+0000", 2024-09-13T10:11:54.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0054122390000000001 2024-09-13T10:11:54.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816092+0000", 2024-09-13T10:11:54.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5537999999999994e-05 2024-09-13T10:11:54.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:54.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816206+0000", 2024-09-13T10:11:54.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114298 2024-09-13T10:11:54.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820805+0000", 2024-09-13T10:11:54.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004598347 2024-09-13T10:11:54.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821735+0000", 2024-09-13T10:11:54.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000930348 2024-09-13T10:11:54.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822118+0000", 2024-09-13T10:11:54.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038271099999999999 2024-09-13T10:11:54.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822208+0000", 2024-09-13T10:11:54.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0271000000000004e-05 2024-09-13T10:11:54.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.131 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822350+0000", 2024-09-13T10:11:54.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014177600000000001 2024-09-13T10:11:54.131 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.131 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.131 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.132 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:92 4.9 4:96fcf081:::smithi19530714-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:54.132 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810624+0000", 2024-09-13T10:11:54.132 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.295581161000001, 2024-09-13T10:11:54.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012449517, 2024-09-13T10:11:54.132 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.132 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.132 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:54.132 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.132 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.132 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.132 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 92 2024-09-13T10:11:54.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.133 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810624+0000", 2024-09-13T10:11:54.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810624+0000", 2024-09-13T10:11:54.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810625+0000", 2024-09-13T10:11:54.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3459999999999999e-06 2024-09-13T10:11:54.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810638+0000", 2024-09-13T10:11:54.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2853e-05 2024-09-13T10:11:54.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810655+0000", 2024-09-13T10:11:54.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6370000000000001e-05 2024-09-13T10:11:54.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810668+0000", 2024-09-13T10:11:54.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3906e-05 2024-09-13T10:11:54.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816513+0000", 2024-09-13T10:11:54.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058443369999999998 2024-09-13T10:11:54.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:54.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816657+0000", 2024-09-13T10:11:54.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014419199999999999 2024-09-13T10:11:54.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822492+0000", 2024-09-13T10:11:54.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.005835107 2024-09-13T10:11:54.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822740+0000", 2024-09-13T10:11:54.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024784600000000003 2024-09-13T10:11:54.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823074+0000", 2024-09-13T10:11:54.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033356 2024-09-13T10:11:54.137 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.137 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.137 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.137 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:129 4.9 4:998d9bf1:::smithi19530714-33:head [write 537824~473380,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:54.137 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810689+0000", 2024-09-13T10:11:54.138 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.295516386000003, 2024-09-13T10:11:54.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01301894, 2024-09-13T10:11:54.138 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.138 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.138 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.138 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.138 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.138 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.138 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 129 2024-09-13T10:11:54.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.138 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.138 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810689+0000", 2024-09-13T10:11:54.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810689+0000", 2024-09-13T10:11:54.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810690+0000", 2024-09-13T10:11:54.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2330000000000001e-06 2024-09-13T10:11:54.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811247+0000", 2024-09-13T10:11:54.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055654999999999995 2024-09-13T10:11:54.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811264+0000", 2024-09-13T10:11:54.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7813e-05 2024-09-13T10:11:54.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811280+0000", 2024-09-13T10:11:54.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5676e-05 2024-09-13T10:11:54.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816755+0000", 2024-09-13T10:11:54.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0054754139999999996 2024-09-13T10:11:54.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816940+0000", 2024-09-13T10:11:54.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184091 2024-09-13T10:11:54.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:54.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.817141+0000", 2024-09-13T10:11:54.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020179099999999999 2024-09-13T10:11:54.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821746+0000", 2024-09-13T10:11:54.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004604543 2024-09-13T10:11:54.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823211+0000", 2024-09-13T10:11:54.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001464824 2024-09-13T10:11:54.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823536+0000", 2024-09-13T10:11:54.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032508599999999998 2024-09-13T10:11:54.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823620+0000", 2024-09-13T10:11:54.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4524999999999995e-05 2024-09-13T10:11:54.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823708+0000", 2024-09-13T10:11:54.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7393999999999997e-05 2024-09-13T10:11:54.144 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.144 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.144 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.144 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:130 4.9 4:998d9bf1:::smithi19530714-33:head [write 1806012~625998] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:54.144 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.811308+0000", 2024-09-13T10:11:54.144 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.294897421999998, 2024-09-13T10:11:54.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012775038000000001, 2024-09-13T10:11:54.144 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.144 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.144 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.145 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.145 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.145 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.145 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 130 2024-09-13T10:11:54.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.145 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811308+0000", 2024-09-13T10:11:54.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811308+0000", 2024-09-13T10:11:54.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811310+0000", 2024-09-13T10:11:54.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9400000000000001e-06 2024-09-13T10:11:54.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811989+0000", 2024-09-13T10:11:54.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00067975999999999996 2024-09-13T10:11:54.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812006+0000", 2024-09-13T10:11:54.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6175999999999999e-05 2024-09-13T10:11:54.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812024+0000", 2024-09-13T10:11:54.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8813000000000001e-05 2024-09-13T10:11:54.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.818639+0000", 2024-09-13T10:11:54.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066148639999999998 2024-09-13T10:11:54.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.818763+0000", 2024-09-13T10:11:54.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012372300000000001 2024-09-13T10:11:54.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:54.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.818947+0000", 2024-09-13T10:11:54.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184394 2024-09-13T10:11:54.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822370+0000", 2024-09-13T10:11:54.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034224989999999999 2024-09-13T10:11:54.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823413+0000", 2024-09-13T10:11:54.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001042739 2024-09-13T10:11:54.149 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.149 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.149 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.149 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823829+0000", 2024-09-13T10:11:54.149 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000416477 2024-09-13T10:11:54.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823982+0000", 2024-09-13T10:11:54.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015238599999999999 2024-09-13T10:11:54.150 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.150 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.150 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.150 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824083+0000", 2024-09-13T10:11:54.150 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000101267 2024-09-13T10:11:54.150 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.150 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.150 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.151 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.151 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:131 4.9 4:998d9bf1:::smithi19530714-33:head [setxattr _header (56) in=7b,truncate 2537824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:54.151 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.812049+0000", 2024-09-13T10:11:54.151 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.294155664000002, 2024-09-13T10:11:54.151 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012575461, 2024-09-13T10:11:54.151 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.151 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.151 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.151 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.151 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 131 2024-09-13T10:11:54.151 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.152 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812049+0000", 2024-09-13T10:11:54.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.152 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.152 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812049+0000", 2024-09-13T10:11:54.152 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.152 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.152 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812051+0000", 2024-09-13T10:11:54.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-13T10:11:54.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812060+0000", 2024-09-13T10:11:54.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9930000000000001e-06 2024-09-13T10:11:54.153 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.153 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.153 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.153 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812078+0000", 2024-09-13T10:11:54.153 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8266000000000001e-05 2024-09-13T10:11:54.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812093+0000", 2024-09-13T10:11:54.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5153e-05 2024-09-13T10:11:54.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820907+0000", 2024-09-13T10:11:54.154 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088138650000000006 2024-09-13T10:11:54.154 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.154 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.154 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.154 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821046+0000", 2024-09-13T10:11:54.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013850900000000001 2024-09-13T10:11:54.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:54.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821338+0000", 2024-09-13T10:11:54.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029241800000000002 2024-09-13T10:11:54.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.155 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.155 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823249+0000", 2024-09-13T10:11:54.155 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019105610000000001 2024-09-13T10:11:54.155 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.155 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824157+0000", 2024-09-13T10:11:54.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090782899999999997 2024-09-13T10:11:54.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824422+0000", 2024-09-13T10:11:54.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026587900000000001 2024-09-13T10:11:54.156 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.156 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.156 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.156 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824497+0000", 2024-09-13T10:11:54.156 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4443999999999994e-05 2024-09-13T10:11:54.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.157 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.157 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824625+0000", 2024-09-13T10:11:54.157 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012811699999999999 2024-09-13T10:11:54.157 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.157 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.157 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.157 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.157 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.157 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:132 4.9 4:998d9bf1:::smithi19530714-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:54.157 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.812113+0000", 2024-09-13T10:11:54.157 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.294091651999999, 2024-09-13T10:11:54.158 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01281326, 2024-09-13T10:11:54.158 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.158 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.158 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:54.158 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.158 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.158 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.158 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 132 2024-09-13T10:11:54.158 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.158 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.158 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.158 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.158 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812113+0000", 2024-09-13T10:11:54.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812113+0000", 2024-09-13T10:11:54.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.159 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812115+0000", 2024-09-13T10:11:54.159 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.826e-06 2024-09-13T10:11:54.159 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.159 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.159 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812121+0000", 2024-09-13T10:11:54.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9540000000000001e-06 2024-09-13T10:11:54.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812158+0000", 2024-09-13T10:11:54.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6619000000000003e-05 2024-09-13T10:11:54.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.160 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.160 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.160 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812175+0000", 2024-09-13T10:11:54.160 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6662000000000001e-05 2024-09-13T10:11:54.160 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821791+0000", 2024-09-13T10:11:54.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.009616187 2024-09-13T10:11:54.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:54.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821855+0000", 2024-09-13T10:11:54.161 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4183999999999994e-05 2024-09-13T10:11:54.161 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.161 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.161 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.161 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824680+0000", 2024-09-13T10:11:54.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0028252749999999999 2024-09-13T10:11:54.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824784+0000", 2024-09-13T10:11:54.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010365 2024-09-13T10:11:54.162 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.162 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.162 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.162 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824927+0000", 2024-09-13T10:11:54.162 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000142903 2024-09-13T10:11:54.162 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.162 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.162 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.163 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.163 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.163 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:272 4.9 4:96fcf081:::smithi19530714-23:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:54.163 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.226803+0000", 2024-09-13T10:11:54.163 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.879401938999997, 2024-09-13T10:11:54.163 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0087344910000000005, 2024-09-13T10:11:54.163 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.163 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.163 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.163 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.163 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.163 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.163 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 272 2024-09-13T10:11:54.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.164 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226803+0000", 2024-09-13T10:11:54.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.164 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.164 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226803+0000", 2024-09-13T10:11:54.164 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.164 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.164 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226807+0000", 2024-09-13T10:11:54.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8829999999999999e-06 2024-09-13T10:11:54.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226823+0000", 2024-09-13T10:11:54.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5979999999999999e-05 2024-09-13T10:11:54.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.165 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.165 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.165 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226859+0000", 2024-09-13T10:11:54.165 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6331999999999997e-05 2024-09-13T10:11:54.165 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226882+0000", 2024-09-13T10:11:54.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2452000000000001e-05 2024-09-13T10:11:54.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.227037+0000", 2024-09-13T10:11:54.166 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000155563 2024-09-13T10:11:54.166 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.166 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.166 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.166 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.227232+0000", 2024-09-13T10:11:54.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019476500000000001 2024-09-13T10:11:54.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:54.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.233676+0000", 2024-09-13T10:11:54.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0064435719999999998 2024-09-13T10:11:54.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.167 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235081+0000", 2024-09-13T10:11:54.167 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001404953 2024-09-13T10:11:54.167 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.167 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.167 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235114+0000", 2024-09-13T10:11:54.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2996000000000001e-05 2024-09-13T10:11:54.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235286+0000", 2024-09-13T10:11:54.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171965 2024-09-13T10:11:54.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.168 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.168 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.168 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235431+0000", 2024-09-13T10:11:54.168 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000145205 2024-09-13T10:11:54.168 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.169 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.169 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235538+0000", 2024-09-13T10:11:54.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010682499999999999 2024-09-13T10:11:54.169 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.169 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.169 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.169 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.169 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.169 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:282 4.9 4:998d9bf1:::smithi19530714-33:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:54.169 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.240459+0000", 2024-09-13T10:11:54.169 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.865746283999997, 2024-09-13T10:11:54.169 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.022877255999999999, 2024-09-13T10:11:54.170 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.170 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.170 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.170 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.170 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.170 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.170 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 282 2024-09-13T10:11:54.170 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.170 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.170 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.170 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.170 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240459+0000", 2024-09-13T10:11:54.170 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240459+0000", 2024-09-13T10:11:54.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.171 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.171 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240462+0000", 2024-09-13T10:11:54.171 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.971e-06 2024-09-13T10:11:54.171 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.171 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240479+0000", 2024-09-13T10:11:54.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6937000000000001e-05 2024-09-13T10:11:54.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240512+0000", 2024-09-13T10:11:54.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3012999999999997e-05 2024-09-13T10:11:54.172 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.172 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.172 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.172 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240533+0000", 2024-09-13T10:11:54.172 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1138999999999999e-05 2024-09-13T10:11:54.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240687+0000", 2024-09-13T10:11:54.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000154275 2024-09-13T10:11:54.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.173 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240829+0000", 2024-09-13T10:11:54.173 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000141619 2024-09-13T10:11:54.173 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.173 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.173 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:54.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.260691+0000", 2024-09-13T10:11:54.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.019861706 2024-09-13T10:11:54.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.262189+0000", 2024-09-13T10:11:54.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0014983069999999999 2024-09-13T10:11:54.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.174 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.174 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.262975+0000", 2024-09-13T10:11:54.174 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00078655900000000004 2024-09-13T10:11:54.174 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.174 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.263100+0000", 2024-09-13T10:11:54.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012431899999999999 2024-09-13T10:11:54.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.263232+0000", 2024-09-13T10:11:54.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013227399999999999 2024-09-13T10:11:54.175 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.175 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.175 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.175 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.263336+0000", 2024-09-13T10:11:54.175 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104137 2024-09-13T10:11:54.175 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.176 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.176 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.176 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.176 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.176 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:296 4.9 4:9bec18ef:::smithi19530714-47:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:54.176 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.261575+0000", 2024-09-13T10:11:54.176 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.844630342000002, 2024-09-13T10:11:54.176 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0097761170000000008, 2024-09-13T10:11:54.176 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.176 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.176 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.176 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.176 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.177 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.177 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 296 2024-09-13T10:11:54.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.177 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261575+0000", 2024-09-13T10:11:54.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.177 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.177 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.177 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.177 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261575+0000", 2024-09-13T10:11:54.177 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261579+0000", 2024-09-13T10:11:54.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7740000000000002e-06 2024-09-13T10:11:54.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.178 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261594+0000", 2024-09-13T10:11:54.178 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4973999999999999e-05 2024-09-13T10:11:54.178 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.178 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.178 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261625+0000", 2024-09-13T10:11:54.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1519000000000001e-05 2024-09-13T10:11:54.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261645+0000", 2024-09-13T10:11:54.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9568e-05 2024-09-13T10:11:54.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.179 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.179 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261758+0000", 2024-09-13T10:11:54.179 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011301700000000001 2024-09-13T10:11:54.179 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.179 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261902+0000", 2024-09-13T10:11:54.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014435300000000001 2024-09-13T10:11:54.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:54.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.269548+0000", 2024-09-13T10:11:54.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0076456850000000002 2024-09-13T10:11:54.180 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.180 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.180 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.180 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.270867+0000", 2024-09-13T10:11:54.180 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013193930000000001 2024-09-13T10:11:54.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.270884+0000", 2024-09-13T10:11:54.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7116999999999999e-05 2024-09-13T10:11:54.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.181 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.271218+0000", 2024-09-13T10:11:54.181 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033386800000000002 2024-09-13T10:11:54.181 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.181 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.181 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.271305+0000", 2024-09-13T10:11:54.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6738999999999997e-05 2024-09-13T10:11:54.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.182 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.182 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.271351+0000", 2024-09-13T10:11:54.182 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6109999999999997e-05 2024-09-13T10:11:54.182 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.182 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.182 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.182 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.182 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.182 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:346 4.9 4:9bec18ef:::smithi19530714-47:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:54.183 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.285952+0000", 2024-09-13T10:11:54.183 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.820253493000003, 2024-09-13T10:11:54.183 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0081934079999999992, 2024-09-13T10:11:54.183 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.183 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.183 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.183 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.183 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.183 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.183 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 346 2024-09-13T10:11:54.183 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.183 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.183 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.183 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285952+0000", 2024-09-13T10:11:54.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285952+0000", 2024-09-13T10:11:54.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.184 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.184 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.184 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285954+0000", 2024-09-13T10:11:54.184 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3530000000000002e-06 2024-09-13T10:11:54.184 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285965+0000", 2024-09-13T10:11:54.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0718e-05 2024-09-13T10:11:54.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.185 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285988+0000", 2024-09-13T10:11:54.185 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3095000000000002e-05 2024-09-13T10:11:54.185 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.185 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.185 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286005+0000", 2024-09-13T10:11:54.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7147999999999999e-05 2024-09-13T10:11:54.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286119+0000", 2024-09-13T10:11:54.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000113999 2024-09-13T10:11:54.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.186 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.186 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.186 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286281+0000", 2024-09-13T10:11:54.186 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000161948 2024-09-13T10:11:54.186 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:54.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.291713+0000", 2024-09-13T10:11:54.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0054324810000000003 2024-09-13T10:11:54.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293743+0000", 2024-09-13T10:11:54.187 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020298439999999998 2024-09-13T10:11:54.187 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.187 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.187 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.187 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293904+0000", 2024-09-13T10:11:54.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000160853 2024-09-13T10:11:54.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293952+0000", 2024-09-13T10:11:54.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7991999999999997e-05 2024-09-13T10:11:54.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.188 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.188 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294052+0000", 2024-09-13T10:11:54.188 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9893000000000001e-05 2024-09-13T10:11:54.188 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.188 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.189 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.189 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294145+0000", 2024-09-13T10:11:54.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3084000000000005e-05 2024-09-13T10:11:54.189 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.189 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.189 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.189 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.189 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.189 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:801 4.9 4:96fcf081:::smithi19530714-23:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:11:54.189 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.798334+0000", 2024-09-13T10:11:54.189 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 62.307871134999999, 2024-09-13T10:11:54.189 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056549887, 2024-09-13T10:11:54.189 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.190 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.190 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:54.190 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.190 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.190 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.190 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 801 2024-09-13T10:11:54.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.190 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.190 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.190 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798334+0000", 2024-09-13T10:11:54.190 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.190 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.190 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798334+0000", 2024-09-13T10:11:54.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798338+0000", 2024-09-13T10:11:54.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3239999999999996e-06 2024-09-13T10:11:54.191 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.191 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.191 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.191 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798355+0000", 2024-09-13T10:11:54.191 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7149000000000001e-05 2024-09-13T10:11:54.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798403+0000", 2024-09-13T10:11:54.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7132000000000002e-05 2024-09-13T10:11:54.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.192 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798431+0000", 2024-09-13T10:11:54.192 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8365999999999999e-05 2024-09-13T10:11:54.192 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.192 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.192 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798534+0000", 2024-09-13T10:11:54.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102545 2024-09-13T10:11:54.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798715+0000", 2024-09-13T10:11:54.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018161800000000001 2024-09-13T10:11:54.193 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.193 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.193 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.193 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.854884+0000", 2024-09-13T10:11:54.193 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056168753000000002 2024-09-13T10:11:54.193 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.193 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.194 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.194 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.194 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.194 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1065 4.19 4:998d9bf1:::smithi19530714-33:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:54.194 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.618527+0000", 2024-09-13T10:11:54.194 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 57.487678418999998, 2024-09-13T10:11:54.194 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010820649999999999, 2024-09-13T10:11:54.194 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.194 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.194 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:54.194 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.194 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.194 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.195 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1065 2024-09-13T10:11:54.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.195 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618527+0000", 2024-09-13T10:11:54.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.195 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.195 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618527+0000", 2024-09-13T10:11:54.195 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.195 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.195 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618531+0000", 2024-09-13T10:11:54.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9899999999999999e-06 2024-09-13T10:11:54.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618546+0000", 2024-09-13T10:11:54.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5733e-05 2024-09-13T10:11:54.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.196 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.196 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.196 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618603+0000", 2024-09-13T10:11:54.196 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6075000000000001e-05 2024-09-13T10:11:54.196 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618635+0000", 2024-09-13T10:11:54.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2053e-05 2024-09-13T10:11:54.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618731+0000", 2024-09-13T10:11:54.197 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6514000000000005e-05 2024-09-13T10:11:54.197 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.197 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.197 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.197 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618887+0000", 2024-09-13T10:11:54.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000155415 2024-09-13T10:11:54.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.198 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.198 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.629347+0000", 2024-09-13T10:11:54.198 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010460870000000001 2024-09-13T10:11:54.198 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.198 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.198 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.198 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.198 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.198 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(osd.2.0:47 4.3 e46/41 4:c5b5ff75:::smithi19530714-20:head v 46'28, mlcod=43'26)", 2024-09-13T10:11:54.198 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:01.500294+0000", 2024-09-13T10:11:54.198 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 52.605911257000002, 2024-09-13T10:11:54.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010309196, 2024-09-13T10:11:54.199 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.199 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.199 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.199 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.500294+0000", 2024-09-13T10:11:54.199 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.199 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.199 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.199 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.199 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.500294+0000", 2024-09-13T10:11:54.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.500296+0000", 2024-09-13T10:11:54.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2800000000000002e-06 2024-09-13T10:11:54.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.200 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.200 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505222+0000", 2024-09-13T10:11:54.200 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0049261269999999998 2024-09-13T10:11:54.200 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.200 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505252+0000", 2024-09-13T10:11:54.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9272999999999999e-05 2024-09-13T10:11:54.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505266+0000", 2024-09-13T10:11:54.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4459e-05 2024-09-13T10:11:54.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.201 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.201 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.201 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505445+0000", 2024-09-13T10:11:54.201 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017880899999999999 2024-09-13T10:11:54.201 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.202 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.202 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505501+0000", 2024-09-13T10:11:54.202 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5926e-05 2024-09-13T10:11:54.202 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.202 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.510523+0000", 2024-09-13T10:11:54.204 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0050222440000000004 2024-09-13T10:11:54.204 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.204 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.204 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.204 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.510603+0000", 2024-09-13T10:11:54.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0078000000000001e-05 2024-09-13T10:11:54.205 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.205 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.205 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.205 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.205 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.205 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1619 4.19 4:998d9bf1:::smithi19530714-33:head [write 1750386~495032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:54.205 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.073959+0000", 2024-09-13T10:11:54.205 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 48.03224625, 2024-09-13T10:11:54.205 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0081975420000000004, 2024-09-13T10:11:54.205 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.205 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.205 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.206 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.206 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.206 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.206 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1619 2024-09-13T10:11:54.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.206 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.206 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.073959+0000", 2024-09-13T10:11:54.206 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.206 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.206 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.206 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.073959+0000", 2024-09-13T10:11:54.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.073961+0000", 2024-09-13T10:11:54.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3829999999999999e-06 2024-09-13T10:11:54.207 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.207 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.207 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.207 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074180+0000", 2024-09-13T10:11:54.207 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000218352 2024-09-13T10:11:54.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074208+0000", 2024-09-13T10:11:54.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7880000000000001e-05 2024-09-13T10:11:54.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.208 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.208 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074222+0000", 2024-09-13T10:11:54.208 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4250000000000001e-05 2024-09-13T10:11:54.208 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.208 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074360+0000", 2024-09-13T10:11:54.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000138023 2024-09-13T10:11:54.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074587+0000", 2024-09-13T10:11:54.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000226789 2024-09-13T10:11:54.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.209 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.209 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:54.209 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074984+0000", 2024-09-13T10:11:54.209 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000397031 2024-09-13T10:11:54.209 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.078275+0000", 2024-09-13T10:11:54.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032917250000000001 2024-09-13T10:11:54.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.080241+0000", 2024-09-13T10:11:54.210 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019660910000000001 2024-09-13T10:11:54.210 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.210 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.210 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.210 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.081869+0000", 2024-09-13T10:11:54.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001627325 2024-09-13T10:11:54.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082041+0000", 2024-09-13T10:11:54.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017206599999999999 2024-09-13T10:11:54.211 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.211 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.211 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.211 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082156+0000", 2024-09-13T10:11:54.211 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115627 2024-09-13T10:11:54.211 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.211 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.212 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.212 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.212 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.212 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1620 4.19 4:998d9bf1:::smithi19530714-33:head [setxattr _header (56) in=7b,truncate 2302559] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:54.212 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.074286+0000", 2024-09-13T10:11:54.212 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 48.031918884, 2024-09-13T10:11:54.212 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0086728220000000002, 2024-09-13T10:11:54.212 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.212 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.212 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.212 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.212 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.213 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.213 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1620 2024-09-13T10:11:54.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.213 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074286+0000", 2024-09-13T10:11:54.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.213 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.213 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.213 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.213 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074286+0000", 2024-09-13T10:11:54.213 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074288+0000", 2024-09-13T10:11:54.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.677e-06 2024-09-13T10:11:54.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074298+0000", 2024-09-13T10:11:54.214 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0020000000000001e-05 2024-09-13T10:11:54.214 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.214 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.214 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.214 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074316+0000", 2024-09-13T10:11:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7546000000000001e-05 2024-09-13T10:11:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074330+0000", 2024-09-13T10:11:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4399999999999999e-05 2024-09-13T10:11:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.076843+0000", 2024-09-13T10:11:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025133289999999999 2024-09-13T10:11:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.215 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.076976+0000", 2024-09-13T10:11:54.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013269000000000001 2024-09-13T10:11:54.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:54.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.077195+0000", 2024-09-13T10:11:54.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000219278 2024-09-13T10:11:54.216 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.216 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.216 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.216 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.078998+0000", 2024-09-13T10:11:54.216 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0018027379999999999 2024-09-13T10:11:54.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082240+0000", 2024-09-13T10:11:54.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032419929999999999 2024-09-13T10:11:54.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.217 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082653+0000", 2024-09-13T10:11:54.217 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041340499999999999 2024-09-13T10:11:54.217 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.217 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.217 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082806+0000", 2024-09-13T10:11:54.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015261300000000001 2024-09-13T10:11:54.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.218 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.218 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082959+0000", 2024-09-13T10:11:54.218 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015313300000000001 2024-09-13T10:11:54.218 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.218 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.218 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.218 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.218 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.219 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1621 4.19 4:998d9bf1:::smithi19530714-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:11:54.219 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.074359+0000", 2024-09-13T10:11:54.219 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 48.031846105, 2024-09-13T10:11:54.219 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0092381359999999992, 2024-09-13T10:11:54.219 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.219 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.219 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:54.219 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.219 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.219 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.219 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1621 2024-09-13T10:11:54.219 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.220 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074359+0000", 2024-09-13T10:11:54.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.220 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.220 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074359+0000", 2024-09-13T10:11:54.220 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.220 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.220 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074361+0000", 2024-09-13T10:11:54.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4699999999999999e-06 2024-09-13T10:11:54.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074387+0000", 2024-09-13T10:11:54.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6472e-05 2024-09-13T10:11:54.221 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.221 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.221 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.221 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074402+0000", 2024-09-13T10:11:54.221 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5437000000000001e-05 2024-09-13T10:11:54.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074411+0000", 2024-09-13T10:11:54.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8699999999999998e-06 2024-09-13T10:11:54.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.222 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.078356+0000", 2024-09-13T10:11:54.222 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003944894 2024-09-13T10:11:54.222 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.222 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.222 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:54.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.078565+0000", 2024-09-13T10:11:54.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020851800000000001 2024-09-13T10:11:54.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.223 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.083093+0000", 2024-09-13T10:11:54.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045284660000000001 2024-09-13T10:11:54.223 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.223 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.223 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.223 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.083319+0000", 2024-09-13T10:11:54.223 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022566200000000001 2024-09-13T10:11:54.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.224 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.224 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.083597+0000", 2024-09-13T10:11:54.224 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027834700000000002 2024-09-13T10:11:54.224 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.224 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.224 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.224 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.224 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.224 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1989 4.19 4:998d9bf1:::smithi19530714-33:head [write 574839~795192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T10:11:54.224 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:13.093772+0000", 2024-09-13T10:11:54.224 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.012433506999997, 2024-09-13T10:11:54.225 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.008334045, 2024-09-13T10:11:54.225 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.225 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.225 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.225 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.225 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.225 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.225 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1989 2024-09-13T10:11:54.225 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.225 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.225 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.225 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.093772+0000", 2024-09-13T10:11:54.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.093772+0000", 2024-09-13T10:11:54.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.226 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.226 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.226 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.093774+0000", 2024-09-13T10:11:54.226 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1519999999999999e-06 2024-09-13T10:11:54.226 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094142+0000", 2024-09-13T10:11:54.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036777400000000002 2024-09-13T10:11:54.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094158+0000", 2024-09-13T10:11:54.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6574000000000001e-05 2024-09-13T10:11:54.227 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.227 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.227 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.227 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094167+0000", 2024-09-13T10:11:54.227 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9299999999999992e-06 2024-09-13T10:11:54.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.228 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.228 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094284+0000", 2024-09-13T10:11:54.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116895 2024-09-13T10:11:54.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.228 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.228 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094428+0000", 2024-09-13T10:11:54.228 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014431099999999999 2024-09-13T10:11:54.228 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.228 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.228 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:54.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094658+0000", 2024-09-13T10:11:54.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000230106 2024-09-13T10:11:54.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.098293+0000", 2024-09-13T10:11:54.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003634918 2024-09-13T10:11:54.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.229 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.229 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.229 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.100887+0000", 2024-09-13T10:11:54.229 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025936309999999999 2024-09-13T10:11:54.229 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.101823+0000", 2024-09-13T10:11:54.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00093637600000000005 2024-09-13T10:11:54.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.101980+0000", 2024-09-13T10:11:54.230 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015709400000000001 2024-09-13T10:11:54.230 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.230 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.230 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.230 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.102106+0000", 2024-09-13T10:11:54.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125284 2024-09-13T10:11:54.231 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.231 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.231 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.231 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.231 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.231 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2416 4.19 4:998d9bf1:::smithi19530714-33:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e63)", 2024-09-13T10:11:54.231 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:22.232954+0000", 2024-09-13T10:11:54.231 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 31.873251369999998, 2024-09-13T10:11:54.231 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010811217999999999, 2024-09-13T10:11:54.231 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.231 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.231 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.232 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.232 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.232 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.232 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2416 2024-09-13T10:11:54.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.232 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.232 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.232 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.232954+0000", 2024-09-13T10:11:54.232 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.232 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.232 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.232 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.232954+0000", 2024-09-13T10:11:54.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.232957+0000", 2024-09-13T10:11:54.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0240000000000002e-06 2024-09-13T10:11:54.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.233 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.233 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.233 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.232969+0000", 2024-09-13T10:11:54.233 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2602000000000001e-05 2024-09-13T10:11:54.233 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.233001+0000", 2024-09-13T10:11:54.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1486000000000003e-05 2024-09-13T10:11:54.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.233017+0000", 2024-09-13T10:11:54.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6565000000000001e-05 2024-09-13T10:11:54.234 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.234 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.234 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.234 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.233160+0000", 2024-09-13T10:11:54.234 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014236000000000001 2024-09-13T10:11:54.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op must wait for map", 2024-09-13T10:11:54.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.233170+0000", 2024-09-13T10:11:54.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9599999999999995e-06 2024-09-13T10:11:54.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.235 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.236247+0000", 2024-09-13T10:11:54.235 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030776990000000001 2024-09-13T10:11:54.235 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.235 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.235 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.236 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.236509+0000", 2024-09-13T10:11:54.236 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026104399999999999 2024-09-13T10:11:54.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.236 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.236 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:11:54.236 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.236883+0000", 2024-09-13T10:11:54.236 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00037440500000000002 2024-09-13T10:11:54.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.236 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.236 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.236 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.240643+0000", 2024-09-13T10:11:54.236 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003760354 2024-09-13T10:11:54.236 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.241525+0000", 2024-09-13T10:11:54.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00088204699999999995 2024-09-13T10:11:54.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.237 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.243591+0000", 2024-09-13T10:11:54.237 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020659039999999999 2024-09-13T10:11:54.237 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.237 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.237 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.243723+0000", 2024-09-13T10:11:54.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013196 2024-09-13T10:11:54.238 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.238 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.238 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.238 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.243765+0000", 2024-09-13T10:11:54.238 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1807999999999997e-05 2024-09-13T10:11:54.238 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.238 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.238 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.238 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.238 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.238 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:54.240 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-13T10:11:54.241 INFO:tasks.rados.rados.0.smithi195.stdout:1580: finishing write tid 1 to smithi19530714-12 2024-09-13T10:11:54.241 INFO:tasks.rados.rados.0.smithi195.stdout:1580: finishing write tid 2 to smithi19530714-12 2024-09-13T10:11:54.242 INFO:tasks.rados.rados.0.smithi195.stdout:1580: finishing write tid 3 to smithi19530714-12 2024-09-13T10:11:54.242 INFO:tasks.rados.rados.0.smithi195.stdout:1580: finishing write tid 4 to smithi19530714-12 2024-09-13T10:11:54.242 INFO:tasks.rados.rados.0.smithi195.stdout:1580: finishing write tid 5 to smithi19530714-12 2024-09-13T10:11:54.242 INFO:tasks.rados.rados.0.smithi195.stdout:1580: oid 12 updating version 0 to 105 2024-09-13T10:11:54.242 INFO:tasks.rados.rados.0.smithi195.stdout:1580: oid 12 version 105 is already newer than 104 2024-09-13T10:11:54.242 INFO:tasks.rados.rados.0.smithi195.stdout:1580: oid 12 version 105 is already newer than 103 2024-09-13T10:11:54.242 INFO:tasks.rados.rados.0.smithi195.stdout:1580: oid 12 updating version 105 to 106 2024-09-13T10:11:54.242 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 106 (ObjNum 334 snap 0 seq_num 334) dirty exists 2024-09-13T10:11:54.242 INFO:tasks.rados.rados.0.smithi195.stdout:1580: left oid 12 (ObjNum 334 snap 0 seq_num 334) 2024-09-13T10:11:54.242 INFO:tasks.rados.rados.0.smithi195.stdout:1581: finishing write tid 1 to smithi19530714-42 2024-09-13T10:11:54.242 INFO:tasks.rados.rados.0.smithi195.stdout:1581: finishing write tid 2 to smithi19530714-42 2024-09-13T10:11:54.242 INFO:tasks.rados.rados.0.smithi195.stdout:1578: done (15 left) 2024-09-13T10:11:54.242 INFO:tasks.rados.rados.0.smithi195.stdout:1579: done (14 left) 2024-09-13T10:11:54.243 INFO:tasks.rados.rados.0.smithi195.stdout:1580: done (13 left) 2024-09-13T10:11:54.243 INFO:tasks.rados.rados.0.smithi195.stdout:1593: write (excl) oid 35 current snap is 0 2024-09-13T10:11:54.243 INFO:tasks.rados.rados.0.smithi195.stdout:1581: finishing write tid 3 to smithi19530714-42 2024-09-13T10:11:54.243 INFO:tasks.rados.rados.0.smithi195.stdout:1581: finishing write tid 4 to smithi19530714-42 2024-09-13T10:11:54.243 INFO:tasks.rados.rados.0.smithi195.stdout:1581: finishing write tid 5 to smithi19530714-42 2024-09-13T10:11:54.243 INFO:tasks.rados.rados.0.smithi195.stdout:1581: oid 42 updating version 0 to 22 2024-09-13T10:11:54.243 INFO:tasks.rados.rados.0.smithi195.stdout:1581: oid 42 version 22 is already newer than 20 2024-09-13T10:11:54.243 INFO:tasks.rados.rados.0.smithi195.stdout:1581: oid 42 version 22 is already newer than 19 2024-09-13T10:11:54.243 INFO:tasks.rados.rados.0.smithi195.stdout:1581: oid 42 version 22 is already newer than 21 2024-09-13T10:11:54.243 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 22 (ObjNum 335 snap 0 seq_num 335) dirty exists 2024-09-13T10:11:54.243 INFO:tasks.rados.rados.0.smithi195.stdout:1581: left oid 42 (ObjNum 335 snap 0 seq_num 335) 2024-09-13T10:11:54.243 INFO:tasks.rados.rados.0.smithi195.stdout:1582: finishing write tid 1 to smithi19530714-24 2024-09-13T10:11:54.243 INFO:tasks.rados.rados.0.smithi195.stdout:1583: finishing set_chunk to oid 22 2024-09-13T10:11:54.244 INFO:tasks.rados.rados.0.smithi195.stdout:1583: got expected ENOENT (src dne) 2024-09-13T10:11:54.244 INFO:tasks.rados.rados.0.smithi195.stdout:1584: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:11:54.244 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:54.244 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2499790 2024-09-13T10:11:54.244 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:54.239+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:54.251 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1802404 2024-09-13T10:11:54.269 INFO:tasks.rados.rados.0.smithi195.stdout:1582: finishing write tid 2 to smithi19530714-24 2024-09-13T10:11:54.269 INFO:tasks.rados.rados.0.smithi195.stdout:1582: finishing write tid 3 to smithi19530714-24 2024-09-13T10:11:54.269 INFO:tasks.rados.rados.0.smithi195.stdout:1582: finishing write tid 4 to smithi19530714-24 2024-09-13T10:11:54.269 INFO:tasks.rados.rados.0.smithi195.stdout:1582: finishing write tid 5 to smithi19530714-24 2024-09-13T10:11:54.269 INFO:tasks.rados.rados.0.smithi195.stdout:1582: oid 24 updating version 0 to 154 2024-09-13T10:11:54.269 INFO:tasks.rados.rados.0.smithi195.stdout:1582: oid 24 version 154 is already newer than 152 2024-09-13T10:11:54.269 INFO:tasks.rados.rados.0.smithi195.stdout:1582: oid 24 updating version 154 to 155 2024-09-13T10:11:54.269 INFO:tasks.rados.rados.0.smithi195.stdout:1582: oid 24 version 155 is already newer than 153 2024-09-13T10:11:54.269 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 155 (ObjNum 336 snap 0 seq_num 336) dirty exists 2024-09-13T10:11:54.270 INFO:tasks.rados.rados.0.smithi195.stdout:1582: left oid 24 (ObjNum 336 snap 0 seq_num 336) 2024-09-13T10:11:54.270 INFO:tasks.rados.rados.0.smithi195.stdout:1586: finishing set_chunk to oid 28 2024-09-13T10:11:54.270 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 28 set_chunk returned error code -95 offset: 93184 length: 2083840 2024-09-13T10:11:54.270 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 0 (ObjNum 180 snap 0 seq_num 180) dirty exists 2024-09-13T10:11:54.270 INFO:tasks.rados.rados.0.smithi195.stdout:1587: expect (ObjNum 303 snap 0 seq_num 303) 2024-09-13T10:11:54.341 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:54.338+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:54.370 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:54.370 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:54.370 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:11:54.370 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:54.383 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-13T10:11:54.384 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2996055 2024-09-13T10:11:54.442 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:54.438+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:54.453 INFO:tasks.rados.rados.0.smithi195.stdout:1588: finishing write tid 1 to smithi19530714-6 2024-09-13T10:11:54.453 INFO:tasks.rados.rados.0.smithi195.stdout:1588: finishing write tid 2 to smithi19530714-6 2024-09-13T10:11:54.453 INFO:tasks.rados.rados.0.smithi195.stdout:1588: finishing write tid 3 to smithi19530714-6 2024-09-13T10:11:54.453 INFO:tasks.rados.rados.0.smithi195.stdout:1588: finishing write tid 4 to smithi19530714-6 2024-09-13T10:11:54.453 INFO:tasks.rados.rados.0.smithi195.stdout:1588: finishing write tid 5 to smithi19530714-6 2024-09-13T10:11:54.453 INFO:tasks.rados.rados.0.smithi195.stdout:1588: oid 6 updating version 0 to 77 2024-09-13T10:11:54.453 INFO:tasks.rados.rados.0.smithi195.stdout:1588: oid 6 updating version 77 to 79 2024-09-13T10:11:54.454 INFO:tasks.rados.rados.0.smithi195.stdout:1588: oid 6 updating version 79 to 80 2024-09-13T10:11:54.454 INFO:tasks.rados.rados.0.smithi195.stdout:1588: oid 6 version 80 is already newer than 78 2024-09-13T10:11:54.454 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 80 (ObjNum 337 snap 0 seq_num 337) dirty exists 2024-09-13T10:11:54.454 INFO:tasks.rados.rados.0.smithi195.stdout:1588: left oid 6 (ObjNum 337 snap 0 seq_num 337) 2024-09-13T10:11:54.454 INFO:tasks.rados.rados.0.smithi195.stdout:1565: got (0) Success 2024-09-13T10:11:54.454 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 42 (ObjNum 292 snap 0 seq_num 292) dirty exists 2024-09-13T10:11:54.454 INFO:tasks.rados.rados.0.smithi195.stdout:1585: got (0) Success 2024-09-13T10:11:54.454 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 36 (ObjNum 269 snap 0 seq_num 269) dirty exists 2024-09-13T10:11:54.454 INFO:tasks.rados.rados.0.smithi195.stdout:1590: finishing write tid 1 to smithi19530714-18 2024-09-13T10:11:54.454 INFO:tasks.rados.rados.0.smithi195.stdout:1590: finishing write tid 2 to smithi19530714-18 2024-09-13T10:11:54.454 INFO:tasks.rados.rados.0.smithi195.stdout:1590: finishing write tid 3 to smithi19530714-18 2024-09-13T10:11:54.454 INFO:tasks.rados.rados.0.smithi195.stdout:1590: finishing write tid 4 to smithi19530714-18 2024-09-13T10:11:54.454 INFO:tasks.rados.rados.0.smithi195.stdout:1590: finishing write tid 5 to smithi19530714-18 2024-09-13T10:11:54.455 INFO:tasks.rados.rados.0.smithi195.stdout:1590: oid 18 updating version 0 to 116 2024-09-13T10:11:54.455 INFO:tasks.rados.rados.0.smithi195.stdout:1590: oid 18 version 116 is already newer than 114 2024-09-13T10:11:54.455 INFO:tasks.rados.rados.0.smithi195.stdout:1590: oid 18 updating version 116 to 117 2024-09-13T10:11:54.455 INFO:tasks.rados.rados.0.smithi195.stdout:1590: oid 18 version 117 is already newer than 115 2024-09-13T10:11:54.455 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 117 (ObjNum 338 snap 0 seq_num 338) dirty exists 2024-09-13T10:11:54.455 INFO:tasks.rados.rados.0.smithi195.stdout:1590: left oid 18 (ObjNum 338 snap 0 seq_num 338) 2024-09-13T10:11:54.455 INFO:tasks.rados.rados.0.smithi195.stdout:1592: expect (ObjNum 272 snap 0 seq_num 272) 2024-09-13T10:11:54.510 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:54.510 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:11:54.510 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:11:54.510 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:11:54.510 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:54.523 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-13T10:11:54.543 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:54.539+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:54.544 INFO:tasks.rados.rados.0.smithi195.stdout:1591: finishing set_chunk to oid 16 2024-09-13T10:11:54.544 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 55 (ObjNum 325 snap 0 seq_num 325) dirty exists 2024-09-13T10:11:54.544 INFO:tasks.rados.rados.0.smithi195.stdout:1593: seq_num 339 ranges {497573=628943,1687631=734777,3093218=604355} 2024-09-13T10:11:54.552 INFO:tasks.rados.rados.0.smithi195.stdout:1593: writing smithi19530714-35 from 497573 to 1126516 tid 1 2024-09-13T10:11:54.562 INFO:tasks.rados.rados.0.smithi195.stdout:1593: writing smithi19530714-35 from 1687631 to 2422408 tid 2 2024-09-13T10:11:54.573 INFO:tasks.rados.rados.0.smithi195.stdout:1593: writing smithi19530714-35 from 3093218 to 3697573 tid 3 2024-09-13T10:11:54.573 INFO:tasks.rados.rados.0.smithi195.stdout:1565: done (13 left) 2024-09-13T10:11:54.573 INFO:tasks.rados.rados.0.smithi195.stdout:1581: done (12 left) 2024-09-13T10:11:54.573 INFO:tasks.rados.rados.0.smithi195.stdout:1582: done (11 left) 2024-09-13T10:11:54.573 INFO:tasks.rados.rados.0.smithi195.stdout:1583: done (10 left) 2024-09-13T10:11:54.573 INFO:tasks.rados.rados.0.smithi195.stdout:1584: done (9 left) 2024-09-13T10:11:54.573 INFO:tasks.rados.rados.0.smithi195.stdout:1585: done (8 left) 2024-09-13T10:11:54.573 INFO:tasks.rados.rados.0.smithi195.stdout:1586: done (7 left) 2024-09-13T10:11:54.573 INFO:tasks.rados.rados.0.smithi195.stdout:1587: done (6 left) 2024-09-13T10:11:54.574 INFO:tasks.rados.rados.0.smithi195.stdout:1588: done (5 left) 2024-09-13T10:11:54.574 INFO:tasks.rados.rados.0.smithi195.stdout:1589: done (4 left) 2024-09-13T10:11:54.574 INFO:tasks.rados.rados.0.smithi195.stdout:1590: done (3 left) 2024-09-13T10:11:54.574 INFO:tasks.rados.rados.0.smithi195.stdout:1591: done (2 left) 2024-09-13T10:11:54.574 INFO:tasks.rados.rados.0.smithi195.stdout:1592: done (1 left) 2024-09-13T10:11:54.574 INFO:tasks.rados.rados.0.smithi195.stdout:1594: read oid 24 snap -1 2024-09-13T10:11:54.574 INFO:tasks.rados.rados.0.smithi195.stdout:1594: expect (ObjNum 336 snap 0 seq_num 336) 2024-09-13T10:11:54.574 INFO:tasks.rados.rados.0.smithi195.stdout:1595: set_chunk oid 21 target oid 2024-09-13T10:11:54.630 INFO:tasks.rados.rados.0.smithi195.stdout:1595: set_chunk oid 21 offset: 108032 length: 3512832 target oid f6d96d7a8dac6a7d56d3d9df9ef9c133dce9c515f90d85faf1e037f770fc9485 offset: 0 snap -1 2024-09-13T10:11:54.630 INFO:tasks.rados.rados.0.smithi195.stdout:1596: read oid 43 snap -1 2024-09-13T10:11:54.630 INFO:tasks.rados.rados.0.smithi195.stdout:1596: expect (ObjNum 332 snap 0 seq_num 332) 2024-09-13T10:11:54.631 INFO:tasks.rados.rados.0.smithi195.stdout:1597: set_chunk oid 30 target oid 2024-09-13T10:11:54.631 INFO:tasks.rados.rados.0.smithi195.stdout:1597: set_chunk oid 30 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:54.631 INFO:tasks.rados.rados.0.smithi195.stdout:1598: set_chunk oid 31 target oid 2024-09-13T10:11:54.644 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:54.641+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:54.649 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:54.650 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:11:54.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:11:54.650 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:11:54.650 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.650 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:161 4.d e38/31 4:b19b932a:::smithi19530714-41:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:54.650 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:11:54.650 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 63.792024015999999, 2024-09-13T10:11:54.650 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056705289999999997, 2024-09-13T10:11:54.650 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.650 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.650 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.651 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:11:54.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.651 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.651 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:11:54.651 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.651 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.651 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:11:54.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0999999999999998e-07 2024-09-13T10:11:54.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.652 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.652 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856051+0000", 2024-09-13T10:11:54.652 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020382130000000001 2024-09-13T10:11:54.652 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.652 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856079+0000", 2024-09-13T10:11:54.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8345999999999999e-05 2024-09-13T10:11:54.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.653 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856093+0000", 2024-09-13T10:11:54.653 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4353e-05 2024-09-13T10:11:54.653 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.653 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.653 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856232+0000", 2024-09-13T10:11:54.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013922199999999999 2024-09-13T10:11:54.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856267+0000", 2024-09-13T10:11:54.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4823000000000002e-05 2024-09-13T10:11:54.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.654 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.654 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.654 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.859505+0000", 2024-09-13T10:11:54.654 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032373269999999999 2024-09-13T10:11:54.654 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.655 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.655 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.859682+0000", 2024-09-13T10:11:54.655 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017753499999999999 2024-09-13T10:11:54.655 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.655 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.655 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.655 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.655 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.655 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1665 4.d 4:b19b932a:::smithi19530714-41:head [write 769787~704463,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-13T10:11:54.655 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.752922+0000", 2024-09-13T10:11:54.655 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.893113597000003, 2024-09-13T10:11:54.656 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014152090000000001, 2024-09-13T10:11:54.656 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.656 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.656 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.656 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.656 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.656 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.656 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1665 2024-09-13T10:11:54.656 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.656 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.656 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.656 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.752922+0000", 2024-09-13T10:11:54.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.752922+0000", 2024-09-13T10:11:54.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.657 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.657 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.657 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.752927+0000", 2024-09-13T10:11:54.657 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7430000000000002e-06 2024-09-13T10:11:54.657 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.753904+0000", 2024-09-13T10:11:54.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00097707799999999993 2024-09-13T10:11:54.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.753959+0000", 2024-09-13T10:11:54.658 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5099000000000002e-05 2024-09-13T10:11:54.658 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.658 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.658 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.658 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.753987+0000", 2024-09-13T10:11:54.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7736999999999999e-05 2024-09-13T10:11:54.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.754289+0000", 2024-09-13T10:11:54.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000301926 2024-09-13T10:11:54.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.659 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.659 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.755253+0000", 2024-09-13T10:11:54.659 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00096452500000000004 2024-09-13T10:11:54.659 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.659 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:54.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.756279+0000", 2024-09-13T10:11:54.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010256919999999999 2024-09-13T10:11:54.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.765950+0000", 2024-09-13T10:11:54.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096707410000000001 2024-09-13T10:11:54.660 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.660 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.660 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.660 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766011+0000", 2024-09-13T10:11:54.660 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1826000000000001e-05 2024-09-13T10:11:54.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766431+0000", 2024-09-13T10:11:54.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041941400000000001 2024-09-13T10:11:54.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.661 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.767009+0000", 2024-09-13T10:11:54.661 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00057826500000000001 2024-09-13T10:11:54.661 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.661 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.661 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.662 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.767074+0000", 2024-09-13T10:11:54.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5043999999999996e-05 2024-09-13T10:11:54.662 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.662 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.662 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.662 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.662 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.662 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1666 4.d 4:b19b932a:::smithi19530714-41:head [write 2267961~101826] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-13T10:11:54.662 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.762495+0000", 2024-09-13T10:11:54.662 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.883541065999999, 2024-09-13T10:11:54.662 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061889800000000002, 2024-09-13T10:11:54.662 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.662 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.663 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.663 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.663 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.663 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.663 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1666 2024-09-13T10:11:54.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.663 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.663 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762495+0000", 2024-09-13T10:11:54.663 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.663 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.663 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.663 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762495+0000", 2024-09-13T10:11:54.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762542+0000", 2024-09-13T10:11:54.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7382999999999997e-05 2024-09-13T10:11:54.664 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.664 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.664 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.664 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762850+0000", 2024-09-13T10:11:54.664 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030803999999999998 2024-09-13T10:11:54.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762888+0000", 2024-09-13T10:11:54.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7645999999999998e-05 2024-09-13T10:11:54.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.665 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762919+0000", 2024-09-13T10:11:54.665 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0879e-05 2024-09-13T10:11:54.665 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.665 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.665 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763057+0000", 2024-09-13T10:11:54.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000138672 2024-09-13T10:11:54.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763221+0000", 2024-09-13T10:11:54.666 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016337200000000001 2024-09-13T10:11:54.666 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.666 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.666 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:54.666 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763457+0000", 2024-09-13T10:11:54.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023669399999999999 2024-09-13T10:11:54.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766139+0000", 2024-09-13T10:11:54.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026813819999999999 2024-09-13T10:11:54.667 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.667 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.667 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.667 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.767138+0000", 2024-09-13T10:11:54.667 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0009992200000000001 2024-09-13T10:11:54.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768189+0000", 2024-09-13T10:11:54.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001051614 2024-09-13T10:11:54.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.668 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.668 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.668 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768616+0000", 2024-09-13T10:11:54.668 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042670000000000002 2024-09-13T10:11:54.668 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.669 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.669 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768684+0000", 2024-09-13T10:11:54.669 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7378000000000004e-05 2024-09-13T10:11:54.669 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.669 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.669 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.669 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.669 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.669 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1667 4.d 4:b19b932a:::smithi19530714-41:head [setxattr _header (56) in=7b,truncate 2369787] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-13T10:11:54.670 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.762978+0000", 2024-09-13T10:11:54.670 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.883057503000003, 2024-09-13T10:11:54.670 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0077634310000000003, 2024-09-13T10:11:54.670 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.670 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.670 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.670 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.670 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.670 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.670 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1667 2024-09-13T10:11:54.670 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.670 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.670 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762978+0000", 2024-09-13T10:11:54.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.671 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.671 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.671 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762978+0000", 2024-09-13T10:11:54.671 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.671 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762980+0000", 2024-09-13T10:11:54.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.654e-06 2024-09-13T10:11:54.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.672 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762990+0000", 2024-09-13T10:11:54.672 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9629999999999993e-06 2024-09-13T10:11:54.672 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.672 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.672 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763008+0000", 2024-09-13T10:11:54.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8122e-05 2024-09-13T10:11:54.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.673 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.673 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763026+0000", 2024-09-13T10:11:54.673 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7747000000000001e-05 2024-09-13T10:11:54.673 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.673 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.764724+0000", 2024-09-13T10:11:54.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001698726 2024-09-13T10:11:54.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.674 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.764905+0000", 2024-09-13T10:11:54.674 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018053200000000001 2024-09-13T10:11:54.674 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.674 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.674 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:54.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.765167+0000", 2024-09-13T10:11:54.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026248100000000001 2024-09-13T10:11:54.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.675 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.767098+0000", 2024-09-13T10:11:54.675 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019303949999999999 2024-09-13T10:11:54.675 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.675 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.675 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768108+0000", 2024-09-13T10:11:54.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010101820000000001 2024-09-13T10:11:54.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.676 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.676 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768766+0000", 2024-09-13T10:11:54.676 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00065837299999999997 2024-09-13T10:11:54.676 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.676 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770530+0000", 2024-09-13T10:11:54.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00176384 2024-09-13T10:11:54.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.677 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.677 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.677 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770742+0000", 2024-09-13T10:11:54.677 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000211416 2024-09-13T10:11:54.677 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.677 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.677 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.677 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.678 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.678 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1668 4.d 4:b19b932a:::smithi19530714-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-09-13T10:11:54.678 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.763054+0000", 2024-09-13T10:11:54.678 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 47.882981405000002, 2024-09-13T10:11:54.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0080519860000000006, 2024-09-13T10:11:54.678 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.678 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.678 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:54.678 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.678 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.678 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.678 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1668 2024-09-13T10:11:54.678 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.679 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763054+0000", 2024-09-13T10:11:54.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763054+0000", 2024-09-13T10:11:54.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.679 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.679 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.679 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763056+0000", 2024-09-13T10:11:54.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.404e-06 2024-09-13T10:11:54.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763062+0000", 2024-09-13T10:11:54.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1260000000000003e-06 2024-09-13T10:11:54.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763080+0000", 2024-09-13T10:11:54.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7864e-05 2024-09-13T10:11:54.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763093+0000", 2024-09-13T10:11:54.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3702e-05 2024-09-13T10:11:54.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766050+0000", 2024-09-13T10:11:54.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029562400000000002 2024-09-13T10:11:54.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:54.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766127+0000", 2024-09-13T10:11:54.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7469e-05 2024-09-13T10:11:54.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770802+0000", 2024-09-13T10:11:54.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0046747309999999997 2024-09-13T10:11:54.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770954+0000", 2024-09-13T10:11:54.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015184000000000001 2024-09-13T10:11:54.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.771106+0000", 2024-09-13T10:11:54.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015260999999999999 2024-09-13T10:11:54.683 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.683 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.683 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.683 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1920 4.d 4:b19b932a:::smithi19530714-41:head [read 0~2369787,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:54.683 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:11.758355+0000", 2024-09-13T10:11:54.683 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.887680193999998, 2024-09-13T10:11:54.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0052833139999999999, 2024-09-13T10:11:54.684 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.684 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.684 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:54.684 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.684 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.684 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.684 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1920 2024-09-13T10:11:54.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.684 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758355+0000", 2024-09-13T10:11:54.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758355+0000", 2024-09-13T10:11:54.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758358+0000", 2024-09-13T10:11:54.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7130000000000001e-06 2024-09-13T10:11:54.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758369+0000", 2024-09-13T10:11:54.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0974e-05 2024-09-13T10:11:54.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758405+0000", 2024-09-13T10:11:54.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6272000000000001e-05 2024-09-13T10:11:54.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758422+0000", 2024-09-13T10:11:54.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6322999999999999e-05 2024-09-13T10:11:54.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758592+0000", 2024-09-13T10:11:54.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000170147 2024-09-13T10:11:54.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.687 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.687 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758876+0000", 2024-09-13T10:11:54.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028450099999999999 2024-09-13T10:11:54.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.763639+0000", 2024-09-13T10:11:54.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047623839999999997 2024-09-13T10:11:54.688 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.688 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.688 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.688 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1921 4.d 4:b19b932a:::smithi19530714-41:head [read 1405629~964158,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:54.688 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:11.758460+0000", 2024-09-13T10:11:54.688 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.887576049000003, 2024-09-13T10:11:54.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068984509999999999, 2024-09-13T10:11:54.689 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.689 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.689 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:54.689 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.689 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.689 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.689 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1921 2024-09-13T10:11:54.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.689 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758460+0000", 2024-09-13T10:11:54.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758460+0000", 2024-09-13T10:11:54.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758461+0000", 2024-09-13T10:11:54.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-13T10:11:54.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758469+0000", 2024-09-13T10:11:54.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8029999999999997e-06 2024-09-13T10:11:54.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758490+0000", 2024-09-13T10:11:54.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1365999999999998e-05 2024-09-13T10:11:54.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758503+0000", 2024-09-13T10:11:54.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3200000000000001e-05 2024-09-13T10:11:54.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.763779+0000", 2024-09-13T10:11:54.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0052761079999999998 2024-09-13T10:11:54.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.764101+0000", 2024-09-13T10:11:54.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032119700000000001 2024-09-13T10:11:54.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.765358+0000", 2024-09-13T10:11:54.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001257467 2024-09-13T10:11:54.693 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.693 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.693 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.693 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1922 4.d 4:b19b932a:::smithi19530714-41:head [sparse-read 1834913~534874] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:54.693 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:11.758578+0000", 2024-09-13T10:11:54.693 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 42.887457425000001, 2024-09-13T10:11:54.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007449954, 2024-09-13T10:11:54.693 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.693 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.694 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:54.694 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.694 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.694 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.694 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1922 2024-09-13T10:11:54.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.694 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758578+0000", 2024-09-13T10:11:54.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758578+0000", 2024-09-13T10:11:54.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758580+0000", 2024-09-13T10:11:54.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.79e-06 2024-09-13T10:11:54.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758587+0000", 2024-09-13T10:11:54.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5599999999999999e-06 2024-09-13T10:11:54.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758620+0000", 2024-09-13T10:11:54.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3649000000000003e-05 2024-09-13T10:11:54.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758628+0000", 2024-09-13T10:11:54.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2400000000000007e-06 2024-09-13T10:11:54.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.765429+0000", 2024-09-13T10:11:54.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068003600000000001 2024-09-13T10:11:54.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.765651+0000", 2024-09-13T10:11:54.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022221900000000001 2024-09-13T10:11:54.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.766028+0000", 2024-09-13T10:11:54.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00037713599999999998 2024-09-13T10:11:54.698 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.698 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.698 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.698 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2712 4.d 4:b19b932a:::smithi19530714-41:head [write 1860221~469412] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:54.698 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:29.375231+0000", 2024-09-13T10:11:54.699 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.270804140999999, 2024-09-13T10:11:54.699 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.005573349, 2024-09-13T10:11:54.699 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.699 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.699 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.699 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.699 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.699 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.699 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2712 2024-09-13T10:11:54.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.699 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375231+0000", 2024-09-13T10:11:54.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375231+0000", 2024-09-13T10:11:54.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.700 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.700 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.700 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375233+0000", 2024-09-13T10:11:54.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.517e-06 2024-09-13T10:11:54.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375610+0000", 2024-09-13T10:11:54.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00037666799999999998 2024-09-13T10:11:54.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.701 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.701 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375628+0000", 2024-09-13T10:11:54.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7918e-05 2024-09-13T10:11:54.701 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375639+0000", 2024-09-13T10:11:54.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1345000000000001e-05 2024-09-13T10:11:54.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375690+0000", 2024-09-13T10:11:54.702 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0603000000000003e-05 2024-09-13T10:11:54.702 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.702 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.702 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.702 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375755+0000", 2024-09-13T10:11:54.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5331000000000002e-05 2024-09-13T10:11:54.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:54.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375864+0000", 2024-09-13T10:11:54.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000108695 2024-09-13T10:11:54.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.703 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.703 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.703 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.377367+0000", 2024-09-13T10:11:54.703 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015037920000000001 2024-09-13T10:11:54.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.378220+0000", 2024-09-13T10:11:54.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00085309600000000002 2024-09-13T10:11:54.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.380433+0000", 2024-09-13T10:11:54.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0022124760000000001 2024-09-13T10:11:54.704 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.380726+0000", 2024-09-13T10:11:54.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029259799999999998 2024-09-13T10:11:54.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.380805+0000", 2024-09-13T10:11:54.705 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9309999999999998e-05 2024-09-13T10:11:54.705 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.705 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.705 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.706 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2714 4.d 4:b19b932a:::smithi19530714-41:head [setxattr _header (56) in=7b,truncate 3966452] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:54.706 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:29.386719+0000", 2024-09-13T10:11:54.706 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.259316643999998, 2024-09-13T10:11:54.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0048978509999999999, 2024-09-13T10:11:54.706 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.706 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.706 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.706 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.707 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.707 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.707 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2714 2024-09-13T10:11:54.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.707 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386719+0000", 2024-09-13T10:11:54.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386719+0000", 2024-09-13T10:11:54.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386720+0000", 2024-09-13T10:11:54.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4300000000000003e-07 2024-09-13T10:11:54.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386723+0000", 2024-09-13T10:11:54.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8899999999999999e-06 2024-09-13T10:11:54.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386735+0000", 2024-09-13T10:11:54.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2052000000000001e-05 2024-09-13T10:11:54.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386740+0000", 2024-09-13T10:11:54.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9629999999999997e-06 2024-09-13T10:11:54.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.388114+0000", 2024-09-13T10:11:54.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001374073 2024-09-13T10:11:54.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.388324+0000", 2024-09-13T10:11:54.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020999500000000001 2024-09-13T10:11:54.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:54.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.388765+0000", 2024-09-13T10:11:54.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044132600000000002 2024-09-13T10:11:54.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.390618+0000", 2024-09-13T10:11:54.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001852993 2024-09-13T10:11:54.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.712 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.390672+0000", 2024-09-13T10:11:54.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3949999999999997e-05 2024-09-13T10:11:54.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.391515+0000", 2024-09-13T10:11:54.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00084309200000000004 2024-09-13T10:11:54.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.391563+0000", 2024-09-13T10:11:54.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8133000000000001e-05 2024-09-13T10:11:54.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.391617+0000", 2024-09-13T10:11:54.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3740999999999999e-05 2024-09-13T10:11:54.714 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.714 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.714 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.714 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2715 4.d 4:b19b932a:::smithi19530714-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:54.714 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:29.386766+0000", 2024-09-13T10:11:54.715 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 25.259270029, 2024-09-13T10:11:54.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0051525249999999998, 2024-09-13T10:11:54.715 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.715 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.715 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:54.715 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.715 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.715 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.715 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2715 2024-09-13T10:11:54.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.715 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.715 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386766+0000", 2024-09-13T10:11:54.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386766+0000", 2024-09-13T10:11:54.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386766+0000", 2024-09-13T10:11:54.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5200000000000002e-07 2024-09-13T10:11:54.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386768+0000", 2024-09-13T10:11:54.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2500000000000001e-06 2024-09-13T10:11:54.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386774+0000", 2024-09-13T10:11:54.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2050000000000001e-06 2024-09-13T10:11:54.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.386778+0000", 2024-09-13T10:11:54.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.318e-06 2024-09-13T10:11:54.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.389475+0000", 2024-09-13T10:11:54.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026974130000000001 2024-09-13T10:11:54.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:54.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.389685+0000", 2024-09-13T10:11:54.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020930800000000001 2024-09-13T10:11:54.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.391672+0000", 2024-09-13T10:11:54.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001987716 2024-09-13T10:11:54.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.391777+0000", 2024-09-13T10:11:54.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010505699999999999 2024-09-13T10:11:54.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.391918+0000", 2024-09-13T10:11:54.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014080599999999999 2024-09-13T10:11:54.720 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.720 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.720 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.720 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3087 4.d 4:b19b932a:::smithi19530714-41:head [read 0~3966452,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-13T10:11:54.720 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:39.459942+0000", 2024-09-13T10:11:54.721 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.186093175, 2024-09-13T10:11:54.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047727209999999997, 2024-09-13T10:11:54.721 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.721 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.721 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:54.721 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.721 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.721 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.721 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3087 2024-09-13T10:11:54.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.721 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.721 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.459942+0000", 2024-09-13T10:11:54.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.459942+0000", 2024-09-13T10:11:54.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.459944+0000", 2024-09-13T10:11:54.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5099999999999999e-06 2024-09-13T10:11:54.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.459955+0000", 2024-09-13T10:11:54.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0796e-05 2024-09-13T10:11:54.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.459972+0000", 2024-09-13T10:11:54.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6762999999999999e-05 2024-09-13T10:11:54.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.459989+0000", 2024-09-13T10:11:54.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7719000000000001e-05 2024-09-13T10:11:54.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.461420+0000", 2024-09-13T10:11:54.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001430792 2024-09-13T10:11:54.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.461542+0000", 2024-09-13T10:11:54.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000122092 2024-09-13T10:11:54.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.464715+0000", 2024-09-13T10:11:54.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0031730489999999998 2024-09-13T10:11:54.725 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.726 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.726 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.726 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3088 4.d 4:b19b932a:::smithi19530714-41:head [sparse-read 0~2316164] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-09-13T10:11:54.726 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:39.460018+0000", 2024-09-13T10:11:54.726 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.186017745000001, 2024-09-13T10:11:54.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0050258610000000004, 2024-09-13T10:11:54.726 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.726 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.726 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:54.726 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.727 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.727 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.727 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3088 2024-09-13T10:11:54.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.727 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.460018+0000", 2024-09-13T10:11:54.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.727 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.460018+0000", 2024-09-13T10:11:54.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.460019+0000", 2024-09-13T10:11:54.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0699999999999999e-06 2024-09-13T10:11:54.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.460024+0000", 2024-09-13T10:11:54.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.967e-06 2024-09-13T10:11:54.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.460037+0000", 2024-09-13T10:11:54.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.328e-05 2024-09-13T10:11:54.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.729 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.460049+0000", 2024-09-13T10:11:54.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1732e-05 2024-09-13T10:11:54.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.729 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.464751+0000", 2024-09-13T10:11:54.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047016519999999997 2024-09-13T10:11:54.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.464817+0000", 2024-09-13T10:11:54.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6011000000000002e-05 2024-09-13T10:11:54.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:39.465044+0000", 2024-09-13T10:11:54.730 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000227149 2024-09-13T10:11:54.730 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.731 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.731 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.731 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3259 4.d 4:ba7d2bc5:::smithi19530714-46:head [write 558065~619869,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-13T10:11:54.731 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:43.057116+0000", 2024-09-13T10:11:54.731 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.588919327999999, 2024-09-13T10:11:54.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0075474399999999999, 2024-09-13T10:11:54.731 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.731 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.731 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.731 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.731 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.732 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.732 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3259 2024-09-13T10:11:54.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.732 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.057116+0000", 2024-09-13T10:11:54.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.057116+0000", 2024-09-13T10:11:54.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.057119+0000", 2024-09-13T10:11:54.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0630000000000002e-06 2024-09-13T10:11:54.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.057773+0000", 2024-09-13T10:11:54.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00065407799999999999 2024-09-13T10:11:54.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.057806+0000", 2024-09-13T10:11:54.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2781999999999999e-05 2024-09-13T10:11:54.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.057823+0000", 2024-09-13T10:11:54.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7039999999999999e-05 2024-09-13T10:11:54.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.058151+0000", 2024-09-13T10:11:54.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032804399999999999 2024-09-13T10:11:54.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.058757+0000", 2024-09-13T10:11:54.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00060580900000000001 2024-09-13T10:11:54.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:54.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.059346+0000", 2024-09-13T10:11:54.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00058853599999999996 2024-09-13T10:11:54.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.062924+0000", 2024-09-13T10:11:54.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035781150000000002 2024-09-13T10:11:54.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.063034+0000", 2024-09-13T10:11:54.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109943 2024-09-13T10:11:54.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.064323+0000", 2024-09-13T10:11:54.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0012888369999999999 2024-09-13T10:11:54.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.064505+0000", 2024-09-13T10:11:54.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000182766 2024-09-13T10:11:54.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.064664+0000", 2024-09-13T10:11:54.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015842699999999999 2024-09-13T10:11:54.737 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.737 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.737 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.738 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3260 4.d 4:ba7d2bc5:::smithi19530714-46:head [write 1579835~486765] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-13T10:11:54.738 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:43.070681+0000", 2024-09-13T10:11:54.738 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.575354517999999, 2024-09-13T10:11:54.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056142930000000002, 2024-09-13T10:11:54.738 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.738 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.738 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.738 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.738 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.738 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.739 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3260 2024-09-13T10:11:54.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.739 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.070681+0000", 2024-09-13T10:11:54.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.070681+0000", 2024-09-13T10:11:54.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.070684+0000", 2024-09-13T10:11:54.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6869999999999999e-06 2024-09-13T10:11:54.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.070967+0000", 2024-09-13T10:11:54.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000282766 2024-09-13T10:11:54.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.070986+0000", 2024-09-13T10:11:54.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9521000000000002e-05 2024-09-13T10:11:54.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.070997+0000", 2024-09-13T10:11:54.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.06e-05 2024-09-13T10:11:54.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.071274+0000", 2024-09-13T10:11:54.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027780099999999999 2024-09-13T10:11:54.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.071763+0000", 2024-09-13T10:11:54.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000488665 2024-09-13T10:11:54.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:54.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.072179+0000", 2024-09-13T10:11:54.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041585899999999998 2024-09-13T10:11:54.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.075091+0000", 2024-09-13T10:11:54.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029115349999999998 2024-09-13T10:11:54.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.075125+0000", 2024-09-13T10:11:54.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4442e-05 2024-09-13T10:11:54.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.076114+0000", 2024-09-13T10:11:54.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098851899999999999 2024-09-13T10:11:54.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.076209+0000", 2024-09-13T10:11:54.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5087000000000006e-05 2024-09-13T10:11:54.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.076295+0000", 2024-09-13T10:11:54.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6811000000000006e-05 2024-09-13T10:11:54.744 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.744 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.744 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.744 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3261 4.d 4:ba7d2bc5:::smithi19530714-46:head [write 2468371~489694] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:54.744 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:43.082814+0000", 2024-09-13T10:11:54.745 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.563221262000001, 2024-09-13T10:11:54.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0079547009999999998, 2024-09-13T10:11:54.745 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.745 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.745 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.745 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.745 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.745 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.745 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3261 2024-09-13T10:11:54.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.745 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.082814+0000", 2024-09-13T10:11:54.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.082814+0000", 2024-09-13T10:11:54.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.746 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.082817+0000", 2024-09-13T10:11:54.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9670000000000001e-06 2024-09-13T10:11:54.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083161+0000", 2024-09-13T10:11:54.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034351199999999997 2024-09-13T10:11:54.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083186+0000", 2024-09-13T10:11:54.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5194e-05 2024-09-13T10:11:54.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083199+0000", 2024-09-13T10:11:54.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2614e-05 2024-09-13T10:11:54.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083318+0000", 2024-09-13T10:11:54.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011906299999999999 2024-09-13T10:11:54.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op must wait for map", 2024-09-13T10:11:54.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083330+0000", 2024-09-13T10:11:54.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2558000000000001e-05 2024-09-13T10:11:54.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.085615+0000", 2024-09-13T10:11:54.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0022843540000000002 2024-09-13T10:11:54.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.085700+0000", 2024-09-13T10:11:54.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5177000000000001e-05 2024-09-13T10:11:54.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:54.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.085838+0000", 2024-09-13T10:11:54.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013857499999999999 2024-09-13T10:11:54.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.089886+0000", 2024-09-13T10:11:54.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0040475290000000002 2024-09-13T10:11:54.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.090273+0000", 2024-09-13T10:11:54.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387439 2024-09-13T10:11:54.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.090564+0000", 2024-09-13T10:11:54.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029072100000000002 2024-09-13T10:11:54.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.090655+0000", 2024-09-13T10:11:54.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0823999999999996e-05 2024-09-13T10:11:54.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.090769+0000", 2024-09-13T10:11:54.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011417399999999999 2024-09-13T10:11:54.752 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.752 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.752 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.752 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3262 4.d 4:ba7d2bc5:::smithi19530714-46:head [setxattr _header (56) in=7b,truncate 2958065] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:54.752 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:43.083231+0000", 2024-09-13T10:11:54.752 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.562804313999999, 2024-09-13T10:11:54.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0079301509999999999, 2024-09-13T10:11:54.753 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.753 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.753 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.753 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.753 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.753 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.753 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3262 2024-09-13T10:11:54.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.753 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.753 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083231+0000", 2024-09-13T10:11:54.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083231+0000", 2024-09-13T10:11:54.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083233+0000", 2024-09-13T10:11:54.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.595e-06 2024-09-13T10:11:54.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083240+0000", 2024-09-13T10:11:54.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7340000000000002e-06 2024-09-13T10:11:54.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083248+0000", 2024-09-13T10:11:54.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8970000000000008e-06 2024-09-13T10:11:54.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083254+0000", 2024-09-13T10:11:54.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3300000000000004e-06 2024-09-13T10:11:54.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083526+0000", 2024-09-13T10:11:54.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027166799999999997 2024-09-13T10:11:54.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting_for_map not empty", 2024-09-13T10:11:54.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083546+0000", 2024-09-13T10:11:54.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0092999999999999e-05 2024-09-13T10:11:54.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.087668+0000", 2024-09-13T10:11:54.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0041219719999999998 2024-09-13T10:11:54.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.087976+0000", 2024-09-13T10:11:54.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030828399999999998 2024-09-13T10:11:54.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:54.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.088483+0000", 2024-09-13T10:11:54.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00050680600000000005 2024-09-13T10:11:54.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.090406+0000", 2024-09-13T10:11:54.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001922864 2024-09-13T10:11:54.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.090523+0000", 2024-09-13T10:11:54.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000117627 2024-09-13T10:11:54.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.090794+0000", 2024-09-13T10:11:54.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027118500000000001 2024-09-13T10:11:54.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.090881+0000", 2024-09-13T10:11:54.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6930999999999998e-05 2024-09-13T10:11:54.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.091161+0000", 2024-09-13T10:11:54.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028016500000000003 2024-09-13T10:11:54.760 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.760 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.760 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.760 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3263 4.d 4:ba7d2bc5:::smithi19530714-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:54.760 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:43.083269+0000", 2024-09-13T10:11:54.761 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 11.562766591000001, 2024-09-13T10:11:54.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0081338560000000001, 2024-09-13T10:11:54.761 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.761 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.761 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:11:54.761 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.761 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.761 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.761 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3263 2024-09-13T10:11:54.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.761 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083269+0000", 2024-09-13T10:11:54.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.762 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083269+0000", 2024-09-13T10:11:54.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.762 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.762 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083270+0000", 2024-09-13T10:11:54.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9800000000000003e-07 2024-09-13T10:11:54.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083273+0000", 2024-09-13T10:11:54.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.416e-06 2024-09-13T10:11:54.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083283+0000", 2024-09-13T10:11:54.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9520000000000006e-06 2024-09-13T10:11:54.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083292+0000", 2024-09-13T10:11:54.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8680000000000005e-06 2024-09-13T10:11:54.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083595+0000", 2024-09-13T10:11:54.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030329100000000002 2024-09-13T10:11:54.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting_for_map not empty", 2024-09-13T10:11:54.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083599+0000", 2024-09-13T10:11:54.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9920000000000001e-06 2024-09-13T10:11:54.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.089551+0000", 2024-09-13T10:11:54.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059521210000000003 2024-09-13T10:11:54.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:54.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.089770+0000", 2024-09-13T10:11:54.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021871400000000001 2024-09-13T10:11:54.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.091250+0000", 2024-09-13T10:11:54.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001479389 2024-09-13T10:11:54.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.091314+0000", 2024-09-13T10:11:54.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4888999999999995e-05 2024-09-13T10:11:54.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.091403+0000", 2024-09-13T10:11:54.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8425999999999997e-05 2024-09-13T10:11:54.767 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.767 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.767 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.767 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3521 4.d 4:ba7d2bc5:::smithi19530714-46:head [write 477398~798659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-13T10:11:54.767 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:48.956271+0000", 2024-09-13T10:11:54.767 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.689764813, 2024-09-13T10:11:54.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0063851139999999999, 2024-09-13T10:11:54.768 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.768 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.768 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.768 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.768 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.768 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.768 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3521 2024-09-13T10:11:54.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.768 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.768 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.956271+0000", 2024-09-13T10:11:54.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.956271+0000", 2024-09-13T10:11:54.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.956275+0000", 2024-09-13T10:11:54.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9330000000000003e-06 2024-09-13T10:11:54.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.957235+0000", 2024-09-13T10:11:54.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00095993900000000004 2024-09-13T10:11:54.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.957270+0000", 2024-09-13T10:11:54.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5194999999999998e-05 2024-09-13T10:11:54.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.957287+0000", 2024-09-13T10:11:54.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7453e-05 2024-09-13T10:11:54.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.957359+0000", 2024-09-13T10:11:54.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1314000000000002e-05 2024-09-13T10:11:54.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.957427+0000", 2024-09-13T10:11:54.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8090999999999996e-05 2024-09-13T10:11:54.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:54.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.957534+0000", 2024-09-13T10:11:54.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107031 2024-09-13T10:11:54.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.959738+0000", 2024-09-13T10:11:54.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002204484 2024-09-13T10:11:54.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.961174+0000", 2024-09-13T10:11:54.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001435644 2024-09-13T10:11:54.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.962581+0000", 2024-09-13T10:11:54.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0014072640000000001 2024-09-13T10:11:54.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.962619+0000", 2024-09-13T10:11:54.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7904999999999997e-05 2024-09-13T10:11:54.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.962656+0000", 2024-09-13T10:11:54.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6860999999999998e-05 2024-09-13T10:11:54.774 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.774 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.774 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.774 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3522 4.d 4:ba7d2bc5:::smithi19530714-46:head [write 1774640~663444] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-13T10:11:54.774 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:48.963159+0000", 2024-09-13T10:11:54.774 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.682876244, 2024-09-13T10:11:54.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0055737549999999997, 2024-09-13T10:11:54.774 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:11:54.774 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:11:54.775 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:54.775 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:11:54.775 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:11:54.775 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:54.775 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3522 2024-09-13T10:11:54.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.775 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:11:54.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:11:54.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.963159+0000", 2024-09-13T10:11:54.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:11:54.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.963159+0000", 2024-09-13T10:11:54.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:11:54.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:11:54.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.963162+0000", 2024-09-13T10:11:54.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9869999999999999e-06 2024-09-13T10:11:54.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.776 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:11:54.776 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.963664+0000", 2024-09-13T10:11:54.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00050161699999999999 2024-09-13T10:11:54.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:11:54.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.963691+0000", 2024-09-13T10:11:54.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7158999999999999e-05 2024-09-13T10:11:54.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:11:54.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.963711+0000", 2024-09-13T10:11:54.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0196000000000001e-05 2024-09-13T10:11:54.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:11:54.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.963823+0000", 2024-09-13T10:11:54.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111841 2024-09-13T10:11:54.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:11:54.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.963991+0000", 2024-09-13T10:11:54.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168085 2024-09-13T10:11:54.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:11:54.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.964239+0000", 2024-09-13T10:11:54.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024774399999999999 2024-09-13T10:11:54.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:11:54.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.967726+0000", 2024-09-13T10:11:54.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003487244 2024-09-13T10:11:54.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.967978+0000", 2024-09-13T10:11:54.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025163600000000001 2024-09-13T10:11:54.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:54.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.968662+0000", 2024-09-13T10:11:54.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00068363600000000005 2024-09-13T10:11:54.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:11:54.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.968696+0000", 2024-09-13T10:11:54.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4764000000000001e-05 2024-09-13T10:11:54.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:11:54.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:11:54.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:11:54.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.968733+0000", 2024-09-13T10:11:54.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6845999999999999e-05 2024-09-13T10:11:54.780 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.780 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.781 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.781 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:11:54.781 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:11:54.781 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:54.782 INFO:tasks.rados.rados.0.smithi195.stdout:1598: set_chunk oid 31 offset: 911872 length: 1043968 target oid 3cc12f4d8b9050d83798b9f284e707dabafa16983d0365475c7502505eb6ce67 offset: 0 snap -1 2024-09-13T10:11:54.783 INFO:tasks.rados.rados.0.smithi195.stdout:1599: read oid 9 snap -1 2024-09-13T10:11:54.783 INFO:tasks.rados.rados.0.smithi195.stdout:1599: expect (ObjNum 259 snap 0 seq_num 259) 2024-09-13T10:11:54.783 INFO:tasks.rados.rados.0.smithi195.stdout:1600: read oid 41 snap -1 2024-09-13T10:11:54.783 INFO:tasks.rados.rados.0.smithi195.stdout:1600: expect (ObjNum 305 snap 0 seq_num 305) 2024-09-13T10:11:54.783 INFO:tasks.rados.rados.0.smithi195.stdout:1601: read oid 7 snap -1 2024-09-13T10:11:54.783 INFO:tasks.rados.rados.0.smithi195.stdout:1601: expect (ObjNum 276 snap 0 seq_num 276) 2024-09-13T10:11:54.783 INFO:tasks.rados.rados.0.smithi195.stdout:1602: delete oid 34 current snap is 0 2024-09-13T10:11:54.783 INFO:tasks.rados.rados.0.smithi195.stdout:1593: finishing write tid 1 to smithi19530714-35 2024-09-13T10:11:54.783 INFO:tasks.rados.rados.0.smithi195.stdout:1593: finishing write tid 2 to smithi19530714-35 2024-09-13T10:11:54.783 INFO:tasks.rados.rados.0.smithi195.stdout:1593: finishing write tid 3 to smithi19530714-35 2024-09-13T10:11:54.783 INFO:tasks.rados.rados.0.smithi195.stdout:1593: finishing write tid 4 to smithi19530714-35 2024-09-13T10:11:54.783 INFO:tasks.rados.rados.0.smithi195.stdout:1593: finishing write tid 5 to smithi19530714-35 2024-09-13T10:11:54.783 INFO:tasks.rados.rados.0.smithi195.stdout:1593: oid 35 updating version 0 to 81 2024-09-13T10:11:54.784 INFO:tasks.rados.rados.0.smithi195.stdout:1593: oid 35 updating version 81 to 83 2024-09-13T10:11:54.784 INFO:tasks.rados.rados.0.smithi195.stdout:1593: oid 35 version 83 is already newer than 82 2024-09-13T10:11:54.784 INFO:tasks.rados.rados.0.smithi195.stdout:1593: oid 35 updating version 83 to 84 2024-09-13T10:11:54.784 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 84 (ObjNum 339 snap 0 seq_num 339) dirty exists 2024-09-13T10:11:54.784 INFO:tasks.rados.rados.0.smithi195.stdout:1593: left oid 35 (ObjNum 339 snap 0 seq_num 339) 2024-09-13T10:11:54.784 INFO:tasks.rados.rados.0.smithi195.stdout:1594: expect (ObjNum 336 snap 0 seq_num 336) 2024-09-13T10:11:54.785 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 442043 2024-09-13T10:11:54.785 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T10:11:54.786 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:54.775+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:54.880 INFO:tasks.rados.rados.0.smithi195.stdout:1597: finishing set_chunk to oid 30 2024-09-13T10:11:54.880 INFO:tasks.rados.rados.0.smithi195.stdout:1597: got expected ENOENT (src dne) 2024-09-13T10:11:54.880 INFO:tasks.rados.rados.0.smithi195.stdout:1595: finishing set_chunk to oid 21 2024-09-13T10:11:54.880 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 114 (ObjNum 235 snap 0 seq_num 235) dirty exists 2024-09-13T10:11:54.880 INFO:tasks.rados.rados.0.smithi195.stdout:1596: expect (ObjNum 332 snap 0 seq_num 332) 2024-09-13T10:11:54.883 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:54.879+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:54.917 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:54.917 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:54.917 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:11:54.917 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:54.931 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T10:11:54.983 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:54.979+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:55.054 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:55.054 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:55.054 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:11:55.054 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:11:55.054 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:55.068 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T10:11:55.069 INFO:tasks.rados.rados.0.smithi195.stdout:1598: finishing set_chunk to oid 31 2024-09-13T10:11:55.069 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 31 set_chunk returned error code -95 offset: 911872 length: 1043968 2024-09-13T10:11:55.069 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 0 (ObjNum 272 snap 0 seq_num 272) dirty exists 2024-09-13T10:11:55.069 INFO:tasks.rados.rados.0.smithi195.stdout:1600: expect (ObjNum 305 snap 0 seq_num 305) 2024-09-13T10:11:55.078 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2852449 2024-09-13T10:11:55.084 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:55.079+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:55.120 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:55.154 INFO:tasks.rados.rados.0.smithi195.stdout:1599: expect (ObjNum 259 snap 0 seq_num 259) 2024-09-13T10:11:55.154 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:55.185 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:11:55.179+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:11:55.231 INFO:tasks.rados.rados.0.smithi195.stdout:1601: expect (ObjNum 276 snap 0 seq_num 276) 2024-09-13T10:11:55.231 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:55.236 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:55.236 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:11:55.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:11:55.236 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:11:55.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.236 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:202 3.2 e38/30 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'1, mlcod=0'0)", 2024-09-13T10:11:55.236 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:11:55.236 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.230393812000003, 2024-09-13T10:11:55.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016741669000000001, 2024-09-13T10:11:55.237 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.237 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.237 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.237 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:11:55.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:11:55.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002368+0000", 2024-09-13T10:11:55.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-13T10:11:55.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010047+0000", 2024-09-13T10:11:55.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007678748 2024-09-13T10:11:55.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010081+0000", 2024-09-13T10:11:55.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4011999999999999e-05 2024-09-13T10:11:55.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010098+0000", 2024-09-13T10:11:55.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7125999999999998e-05 2024-09-13T10:11:55.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010253+0000", 2024-09-13T10:11:55.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000154881 2024-09-13T10:11:55.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010294+0000", 2024-09-13T10:11:55.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1746000000000003e-05 2024-09-13T10:11:55.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.018922+0000", 2024-09-13T10:11:55.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0086279619999999994 2024-09-13T10:11:55.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.019109+0000", 2024-09-13T10:11:55.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000186057 2024-09-13T10:11:55.241 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.241 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.241 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.241 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:206 3.2 e38/30 3:4cbdbe9e:::smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:11:55.241 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:11:55.241 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.222009080999996, 2024-09-13T10:11:55.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011224615, 2024-09-13T10:11:55.242 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.242 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.242 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.242 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:11:55.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:11:55.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010754+0000", 2024-09-13T10:11:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9999999999999999e-06 2024-09-13T10:11:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016750+0000", 2024-09-13T10:11:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059966480000000003 2024-09-13T10:11:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016782+0000", 2024-09-13T10:11:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2218999999999998e-05 2024-09-13T10:11:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016799+0000", 2024-09-13T10:11:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6685999999999999e-05 2024-09-13T10:11:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016923+0000", 2024-09-13T10:11:55.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000124122 2024-09-13T10:11:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016962+0000", 2024-09-13T10:11:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8748999999999998e-05 2024-09-13T10:11:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021869+0000", 2024-09-13T10:11:55.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049073249999999997 2024-09-13T10:11:55.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021976+0000", 2024-09-13T10:11:55.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106866 2024-09-13T10:11:55.246 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.246 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.246 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.246 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:214 3.2 e38/30 3:68ba95ba:::smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'3, mlcod=0'0)", 2024-09-13T10:11:55.247 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:11:55.247 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.212306424999994, 2024-09-13T10:11:55.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011518149, 2024-09-13T10:11:55.247 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.247 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.247 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.247 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:11:55.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:11:55.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020456+0000", 2024-09-13T10:11:55.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0430000000000002e-06 2024-09-13T10:11:55.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026998+0000", 2024-09-13T10:11:55.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065420169999999998 2024-09-13T10:11:55.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027030+0000", 2024-09-13T10:11:55.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1976e-05 2024-09-13T10:11:55.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027048+0000", 2024-09-13T10:11:55.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7615e-05 2024-09-13T10:11:55.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027206+0000", 2024-09-13T10:11:55.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015788900000000001 2024-09-13T10:11:55.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027283+0000", 2024-09-13T10:11:55.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6942999999999998e-05 2024-09-13T10:11:55.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.031808+0000", 2024-09-13T10:11:55.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045256940000000002 2024-09-13T10:11:55.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.031972+0000", 2024-09-13T10:11:55.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016397199999999999 2024-09-13T10:11:55.251 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.251 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.251 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.251 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:210 3.2 e38/30 3:46ab5247:::smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:11:55.252 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:11:55.252 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.204440707000003, 2024-09-13T10:11:55.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015681245, 2024-09-13T10:11:55.252 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.252 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.252 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.252 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:11:55.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:11:55.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028322+0000", 2024-09-13T10:11:55.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0200000000000001e-06 2024-09-13T10:11:55.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037903+0000", 2024-09-13T10:11:55.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095811639999999997 2024-09-13T10:11:55.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037945+0000", 2024-09-13T10:11:55.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1891000000000003e-05 2024-09-13T10:11:55.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037971+0000", 2024-09-13T10:11:55.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5829e-05 2024-09-13T10:11:55.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038125+0000", 2024-09-13T10:11:55.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000153852 2024-09-13T10:11:55.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038193+0000", 2024-09-13T10:11:55.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7995000000000002e-05 2024-09-13T10:11:55.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.043904+0000", 2024-09-13T10:11:55.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0057110340000000002 2024-09-13T10:11:55.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.044001+0000", 2024-09-13T10:11:55.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7460000000000005e-05 2024-09-13T10:11:55.257 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.257 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.257 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.257 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:218 3.2 e38/30 3:5915c927:::smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'5, mlcod=38'2)", 2024-09-13T10:11:55.257 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:11:55.257 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.191078121000004, 2024-09-13T10:11:55.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012988942, 2024-09-13T10:11:55.257 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.257 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.258 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.258 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:11:55.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:11:55.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041685+0000", 2024-09-13T10:11:55.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7769999999999998e-06 2024-09-13T10:11:55.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046575+0000", 2024-09-13T10:11:55.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048899520000000004 2024-09-13T10:11:55.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046633+0000", 2024-09-13T10:11:55.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7561e-05 2024-09-13T10:11:55.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046657+0000", 2024-09-13T10:11:55.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4223000000000002e-05 2024-09-13T10:11:55.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046819+0000", 2024-09-13T10:11:55.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016167699999999999 2024-09-13T10:11:55.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046882+0000", 2024-09-13T10:11:55.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3467999999999996e-05 2024-09-13T10:11:55.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054529+0000", 2024-09-13T10:11:55.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0076467289999999997 2024-09-13T10:11:55.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054671+0000", 2024-09-13T10:11:55.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014255499999999999 2024-09-13T10:11:55.262 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.262 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.262 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.262 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:217 3.2 e38/30 3:554c406d:::smithi19530714-18 low_tier:head v 38'6, mlcod=38'2)", 2024-09-13T10:11:55.262 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:11:55.262 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.186031330999995, 2024-09-13T10:11:55.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013473855999999999, 2024-09-13T10:11:55.263 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.263 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.263 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.263 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:11:55.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:11:55.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046731+0000", 2024-09-13T10:11:55.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.813e-06 2024-09-13T10:11:55.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054994+0000", 2024-09-13T10:11:55.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0082631279999999998 2024-09-13T10:11:55.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055024+0000", 2024-09-13T10:11:55.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9839e-05 2024-09-13T10:11:55.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055044+0000", 2024-09-13T10:11:55.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9507000000000001e-05 2024-09-13T10:11:55.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055186+0000", 2024-09-13T10:11:55.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014243099999999999 2024-09-13T10:11:55.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055240+0000", 2024-09-13T10:11:55.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4444999999999998e-05 2024-09-13T10:11:55.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060135+0000", 2024-09-13T10:11:55.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048942949999999999 2024-09-13T10:11:55.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060203+0000", 2024-09-13T10:11:55.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8398000000000005e-05 2024-09-13T10:11:55.267 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.267 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.268 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.268 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:220 3.2 e38/30 3:753d5785:::smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'7, mlcod=38'3)", 2024-09-13T10:11:55.268 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:11:55.268 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.177650063000002, 2024-09-13T10:11:55.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.009287455, 2024-09-13T10:11:55.268 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.268 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.268 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.268 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:11:55.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:11:55.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055112+0000", 2024-09-13T10:11:55.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6589999999999999e-06 2024-09-13T10:11:55.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057327+0000", 2024-09-13T10:11:55.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0022145229999999999 2024-09-13T10:11:55.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057354+0000", 2024-09-13T10:11:55.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7642000000000001e-05 2024-09-13T10:11:55.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057373+0000", 2024-09-13T10:11:55.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8768999999999999e-05 2024-09-13T10:11:55.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057511+0000", 2024-09-13T10:11:55.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013793900000000001 2024-09-13T10:11:55.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057550+0000", 2024-09-13T10:11:55.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9301999999999997e-05 2024-09-13T10:11:55.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.064324+0000", 2024-09-13T10:11:55.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0067737429999999996 2024-09-13T10:11:55.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.064398+0000", 2024-09-13T10:11:55.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3878e-05 2024-09-13T10:11:55.273 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.273 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.273 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.273 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:227 3.2 e38/30 3:6ac7800c:::smithi19530714-28 low_tier:head v 38'8, mlcod=38'6)", 2024-09-13T10:11:55.273 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:11:55.273 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.158059413000004, 2024-09-13T10:11:55.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018812816, 2024-09-13T10:11:55.273 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.273 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.273 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.273 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:11:55.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:11:55.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074708+0000", 2024-09-13T10:11:55.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.968e-06 2024-09-13T10:11:55.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085262+0000", 2024-09-13T10:11:55.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010553842000000001 2024-09-13T10:11:55.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085289+0000", 2024-09-13T10:11:55.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7288999999999999e-05 2024-09-13T10:11:55.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085305+0000", 2024-09-13T10:11:55.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5925000000000001e-05 2024-09-13T10:11:55.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085418+0000", 2024-09-13T10:11:55.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112584 2024-09-13T10:11:55.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085451+0000", 2024-09-13T10:11:55.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3605000000000001e-05 2024-09-13T10:11:55.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093417+0000", 2024-09-13T10:11:55.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0079654249999999999 2024-09-13T10:11:55.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093514+0000", 2024-09-13T10:11:55.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7177999999999997e-05 2024-09-13T10:11:55.277 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.277 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.278 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.278 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:230 3.2 e38/30 3:5d108080:::smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'9, mlcod=38'6)", 2024-09-13T10:11:55.278 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:11:55.278 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.143776383000002, 2024-09-13T10:11:55.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01953365, 2024-09-13T10:11:55.278 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.278 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.278 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.278 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:11:55.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:11:55.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088987+0000", 2024-09-13T10:11:55.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4600000000000002e-06 2024-09-13T10:11:55.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095842+0000", 2024-09-13T10:11:55.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0068548760000000002 2024-09-13T10:11:55.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095872+0000", 2024-09-13T10:11:55.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0476000000000001e-05 2024-09-13T10:11:55.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095890+0000", 2024-09-13T10:11:55.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7554000000000002e-05 2024-09-13T10:11:55.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.281 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.096006+0000", 2024-09-13T10:11:55.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011597199999999999 2024-09-13T10:11:55.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.281 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.096058+0000", 2024-09-13T10:11:55.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2274000000000001e-05 2024-09-13T10:11:55.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.282 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.108451+0000", 2024-09-13T10:11:55.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012392815 2024-09-13T10:11:55.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.282 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.108518+0000", 2024-09-13T10:11:55.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7223000000000003e-05 2024-09-13T10:11:55.282 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.282 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.282 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.283 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:232 3.2 e38/30 3:6538713c:::smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'10, mlcod=38'6)", 2024-09-13T10:11:55.283 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:11:55.283 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.136825501999994, 2024-09-13T10:11:55.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015307497, 2024-09-13T10:11:55.283 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.283 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.283 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.283 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:11:55.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:11:55.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095937+0000", 2024-09-13T10:11:55.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.367e-06 2024-09-13T10:11:55.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099841+0000", 2024-09-13T10:11:55.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0039043960000000001 2024-09-13T10:11:55.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099865+0000", 2024-09-13T10:11:55.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4538000000000001e-05 2024-09-13T10:11:55.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099881+0000", 2024-09-13T10:11:55.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5809000000000001e-05 2024-09-13T10:11:55.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099995+0000", 2024-09-13T10:11:55.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113843 2024-09-13T10:11:55.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.100043+0000", 2024-09-13T10:11:55.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8291999999999998e-05 2024-09-13T10:11:55.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111200+0000", 2024-09-13T10:11:55.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011156291 2024-09-13T10:11:55.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111243+0000", 2024-09-13T10:11:55.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2960999999999997e-05 2024-09-13T10:11:55.287 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.287 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.288 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.288 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:236 3.2 e38/30 3:73c2893b:::smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'11, mlcod=38'7)", 2024-09-13T10:11:55.288 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:11:55.288 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.127325384000002, 2024-09-13T10:11:55.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0098707670000000008, 2024-09-13T10:11:55.288 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.288 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.288 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.288 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:11:55.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:11:55.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105438+0000", 2024-09-13T10:11:55.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3010000000000001e-06 2024-09-13T10:11:55.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.290 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.290 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112262+0000", 2024-09-13T10:11:55.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.006824793 2024-09-13T10:11:55.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.290 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.290 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112279+0000", 2024-09-13T10:11:55.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6829000000000001e-05 2024-09-13T10:11:55.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112293+0000", 2024-09-13T10:11:55.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3801000000000001e-05 2024-09-13T10:11:55.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112381+0000", 2024-09-13T10:11:55.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7825999999999996e-05 2024-09-13T10:11:55.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112411+0000", 2024-09-13T10:11:55.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0434e-05 2024-09-13T10:11:55.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.115246+0000", 2024-09-13T10:11:55.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028352360000000001 2024-09-13T10:11:55.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.115306+0000", 2024-09-13T10:11:55.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9546999999999998e-05 2024-09-13T10:11:55.293 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.293 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.293 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.293 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:240 3.2 e38/30 3:7eee2f4a:::smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'12, mlcod=38'7)", 2024-09-13T10:11:55.293 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:11:55.293 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.120415597000004, 2024-09-13T10:11:55.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011743134000000001, 2024-09-13T10:11:55.293 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.293 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.293 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.294 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:11:55.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:11:55.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.295 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112346+0000", 2024-09-13T10:11:55.295 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.037e-06 2024-09-13T10:11:55.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.295 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.295 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.295 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120152+0000", 2024-09-13T10:11:55.295 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0078054700000000001 2024-09-13T10:11:55.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.295 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.295 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.295 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120173+0000", 2024-09-13T10:11:55.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1498999999999999e-05 2024-09-13T10:11:55.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120188+0000", 2024-09-13T10:11:55.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5159e-05 2024-09-13T10:11:55.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120313+0000", 2024-09-13T10:11:55.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012440199999999999 2024-09-13T10:11:55.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120358+0000", 2024-09-13T10:11:55.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-13T10:11:55.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.123945+0000", 2024-09-13T10:11:55.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035865490000000001 2024-09-13T10:11:55.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.124088+0000", 2024-09-13T10:11:55.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014318999999999999 2024-09-13T10:11:55.298 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.298 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.298 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.299 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:242 3.2 e38/30 3:501d22a1:::smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'13, mlcod=38'7)", 2024-09-13T10:11:55.299 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:11:55.299 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.112522588999994, 2024-09-13T10:11:55.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0088394330000000007, 2024-09-13T10:11:55.299 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.299 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.299 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.299 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.299 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.299 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:11:55.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.300 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:11:55.300 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.300 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120240+0000", 2024-09-13T10:11:55.300 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.731e-06 2024-09-13T10:11:55.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125395+0000", 2024-09-13T10:11:55.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0051547490000000001 2024-09-13T10:11:55.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125412+0000", 2024-09-13T10:11:55.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7833e-05 2024-09-13T10:11:55.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125426+0000", 2024-09-13T10:11:55.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3237999999999999e-05 2024-09-13T10:11:55.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.302 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.302 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.302 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125541+0000", 2024-09-13T10:11:55.302 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011566299999999999 2024-09-13T10:11:55.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.302 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.302 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.302 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125578+0000", 2024-09-13T10:11:55.302 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7197000000000003e-05 2024-09-13T10:11:55.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.302 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.303 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.303 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.128885+0000", 2024-09-13T10:11:55.303 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0033067119999999998 2024-09-13T10:11:55.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.303 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.303 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.303 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.129077+0000", 2024-09-13T10:11:55.303 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019231000000000001 2024-09-13T10:11:55.303 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.303 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.303 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.303 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.303 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:244 3.2 e38/30 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'14, mlcod=38'10)", 2024-09-13T10:11:55.304 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:11:55.304 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.106503220999997, 2024-09-13T10:11:55.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01536599, 2024-09-13T10:11:55.304 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.304 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.304 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.304 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.304 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:11:55.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:11:55.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126259+0000", 2024-09-13T10:11:55.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.528e-06 2024-09-13T10:11:55.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134336+0000", 2024-09-13T10:11:55.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080767410000000001 2024-09-13T10:11:55.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134356+0000", 2024-09-13T10:11:55.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9936999999999999e-05 2024-09-13T10:11:55.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134367+0000", 2024-09-13T10:11:55.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1352000000000001e-05 2024-09-13T10:11:55.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.307 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.307 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134532+0000", 2024-09-13T10:11:55.307 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001651 2024-09-13T10:11:55.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.307 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.307 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.307 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134578+0000", 2024-09-13T10:11:55.307 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5442000000000003e-05 2024-09-13T10:11:55.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.307 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.307 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.307 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.141521+0000", 2024-09-13T10:11:55.307 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069432720000000003 2024-09-13T10:11:55.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.308 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.308 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.308 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.141623+0000", 2024-09-13T10:11:55.308 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102618 2024-09-13T10:11:55.308 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.308 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.308 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.308 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.308 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:250 3.2 e38/30 3:78d40eb9:::smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'15, mlcod=38'10)", 2024-09-13T10:11:55.309 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:11:55.309 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.098306750000006, 2024-09-13T10:11:55.309 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095447210000000008, 2024-09-13T10:11:55.309 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.309 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.309 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.309 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.309 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.309 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.309 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:11:55.309 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.309 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.310 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.310 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:11:55.310 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.310 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.310 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.310 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134455+0000", 2024-09-13T10:11:55.310 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2699999999999999e-07 2024-09-13T10:11:55.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.310 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.310 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.310 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139352+0000", 2024-09-13T10:11:55.310 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048977810000000004 2024-09-13T10:11:55.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.311 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139374+0000", 2024-09-13T10:11:55.311 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1339e-05 2024-09-13T10:11:55.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.311 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139384+0000", 2024-09-13T10:11:55.311 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0183999999999999e-05 2024-09-13T10:11:55.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.312 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139477+0000", 2024-09-13T10:11:55.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2590999999999995e-05 2024-09-13T10:11:55.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.312 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.312 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139515+0000", 2024-09-13T10:11:55.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8624999999999997e-05 2024-09-13T10:11:55.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.312 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.312 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.143939+0000", 2024-09-13T10:11:55.313 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0044241189999999998 2024-09-13T10:11:55.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.313 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.313 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.313 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.143999+0000", 2024-09-13T10:11:55.313 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9354999999999997e-05 2024-09-13T10:11:55.313 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.313 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.313 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.313 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.313 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:245 3.2 e38/30 3:7837e849:::smithi19530714-46 low_tier:head v 38'16, mlcod=38'10)", 2024-09-13T10:11:55.313 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:11:55.314 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.093350396000005, 2024-09-13T10:11:55.314 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012687423999999999, 2024-09-13T10:11:55.314 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.314 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.314 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.314 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.314 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.314 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.314 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:11:55.314 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.314 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.314 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.315 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:11:55.315 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.315 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.315 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.315 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139411+0000", 2024-09-13T10:11:55.315 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6300000000000005e-07 2024-09-13T10:11:55.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.315 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.315 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.315 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144635+0000", 2024-09-13T10:11:55.315 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0052242169999999997 2024-09-13T10:11:55.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.316 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.316 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.316 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144651+0000", 2024-09-13T10:11:55.316 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6371e-05 2024-09-13T10:11:55.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.316 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.316 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.316 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144662+0000", 2024-09-13T10:11:55.316 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0003000000000001e-05 2024-09-13T10:11:55.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.316 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.316 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.317 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144741+0000", 2024-09-13T10:11:55.317 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9614e-05 2024-09-13T10:11:55.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.317 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.317 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.317 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144768+0000", 2024-09-13T10:11:55.317 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6826999999999999e-05 2024-09-13T10:11:55.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.317 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.317 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.317 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152015+0000", 2024-09-13T10:11:55.317 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0072470879999999996 2024-09-13T10:11:55.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.318 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.318 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.318 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152098+0000", 2024-09-13T10:11:55.318 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2640999999999997e-05 2024-09-13T10:11:55.318 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.318 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.318 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.318 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.318 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.318 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:2811 3.2 e71/30 3:5a2483ca:::890eeb33aa7613d6935d7cb9ed4860e522183188e5dc23a51d7045443a8d23dd:head v 71'264, mlcod=70'262)", 2024-09-13T10:11:55.318 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:32.583079+0000", 2024-09-13T10:11:55.318 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.649681199, 2024-09-13T10:11:55.318 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0094005459999999992, 2024-09-13T10:11:55.319 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.319 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.319 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.319 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.319 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.319 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.583079+0000", 2024-09-13T10:11:55.319 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.319 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.319 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.583079+0000", 2024-09-13T10:11:55.319 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.583082+0000", 2024-09-13T10:11:55.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3070000000000001e-06 2024-09-13T10:11:55.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.587892+0000", 2024-09-13T10:11:55.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048105889999999997 2024-09-13T10:11:55.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.321 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.587924+0000", 2024-09-13T10:11:55.321 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1217999999999998e-05 2024-09-13T10:11:55.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.321 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.321 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.587938+0000", 2024-09-13T10:11:55.321 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4341999999999999e-05 2024-09-13T10:11:55.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.321 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.321 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.588076+0000", 2024-09-13T10:11:55.321 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013836600000000001 2024-09-13T10:11:55.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.322 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.322 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.322 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.588116+0000", 2024-09-13T10:11:55.322 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9480999999999997e-05 2024-09-13T10:11:55.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.322 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.322 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.322 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.592358+0000", 2024-09-13T10:11:55.322 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042423740000000001 2024-09-13T10:11:55.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.322 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.323 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.323 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.592480+0000", 2024-09-13T10:11:55.323 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012186899999999999 2024-09-13T10:11:55.323 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.323 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.323 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.323 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.323 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:3250 3.2 e80/30 3:695c484f:::71f8cb809b47e1dc3dd3aea8719375a9e681992461b1c1cb1fc81e6c9519fc6b:head v 80'311, mlcod=80'309)", 2024-09-13T10:11:55.323 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:43.022696+0000", 2024-09-13T10:11:55.323 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.210064213000001, 2024-09-13T10:11:55.323 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007890144, 2024-09-13T10:11:55.324 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.324 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.324 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.324 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.324 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.324 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:43.022696+0000", 2024-09-13T10:11:55.324 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.324 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.324 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:43.022696+0000", 2024-09-13T10:11:55.324 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.325 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.325 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.325 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:43.022698+0000", 2024-09-13T10:11:55.325 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5600000000000001e-06 2024-09-13T10:11:55.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.325 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.326 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.326 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:43.027405+0000", 2024-09-13T10:11:55.326 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0047069829999999997 2024-09-13T10:11:55.326 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.326 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.326 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.326 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:43.027439+0000", 2024-09-13T10:11:55.326 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4505999999999998e-05 2024-09-13T10:11:55.326 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.326 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.326 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.326 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:43.027456+0000", 2024-09-13T10:11:55.327 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6685e-05 2024-09-13T10:11:55.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.327 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.327 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:43.027580+0000", 2024-09-13T10:11:55.327 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012377500000000001 2024-09-13T10:11:55.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.327 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.327 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:43.027620+0000", 2024-09-13T10:11:55.327 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0501999999999999e-05 2024-09-13T10:11:55.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.328 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.328 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:43.030500+0000", 2024-09-13T10:11:55.328 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028800470000000002 2024-09-13T10:11:55.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.328 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.328 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.328 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:43.030587+0000", 2024-09-13T10:11:55.328 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6086000000000002e-05 2024-09-13T10:11:55.328 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.328 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.328 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.328 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.329 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(osd.6.0:365 4.5 e82/41 4:a3098556:::smithi19530714-22:head v 82'129, mlcod=79'127)", 2024-09-13T10:11:55.329 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:44.595715+0000", 2024-09-13T10:11:55.329 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.637045556, 2024-09-13T10:11:55.329 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010604906000000001, 2024-09-13T10:11:55.329 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.329 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.329 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.329 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.329 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.329 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.329 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.595715+0000", 2024-09-13T10:11:55.330 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.330 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.330 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.330 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.595715+0000", 2024-09-13T10:11:55.330 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.330 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.330 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.330 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.595716+0000", 2024-09-13T10:11:55.330 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8400000000000004e-07 2024-09-13T10:11:55.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.330 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.331 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.331 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.601483+0000", 2024-09-13T10:11:55.331 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0057671789999999999 2024-09-13T10:11:55.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.331 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.331 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.331 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.601511+0000", 2024-09-13T10:11:55.331 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8218999999999999e-05 2024-09-13T10:11:55.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.331 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.331 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.331 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.601526+0000", 2024-09-13T10:11:55.331 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4956e-05 2024-09-13T10:11:55.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.332 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.332 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.332 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.601667+0000", 2024-09-13T10:11:55.332 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000140774 2024-09-13T10:11:55.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.332 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.332 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.332 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.601709+0000", 2024-09-13T10:11:55.332 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2688999999999997e-05 2024-09-13T10:11:55.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.333 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.333 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.333 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.606248+0000", 2024-09-13T10:11:55.333 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004538065 2024-09-13T10:11:55.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.333 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.333 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.333 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.606320+0000", 2024-09-13T10:11:55.333 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2440000000000004e-05 2024-09-13T10:11:55.333 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.333 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.333 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.334 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.334 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:3445 3.2 e86/30 3:7ba83ace:::4c246a1df6a0d5acaf0a3a85249ff707b21f90a5f3db763dc6d544e5b7cbec73:head v 86'330, mlcod=85'328)", 2024-09-13T10:11:55.334 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:48.129770+0000", 2024-09-13T10:11:55.334 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.1029908600000002, 2024-09-13T10:11:55.334 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010414537999999999, 2024-09-13T10:11:55.334 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:11:55.334 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:11:55.334 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:55.334 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:11:55.334 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.334 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:11:55.334 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.129770+0000", 2024-09-13T10:11:55.334 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.335 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.335 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:11:55.335 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.129770+0000", 2024-09-13T10:11:55.335 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:11:55.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.335 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.335 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:11:55.335 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.129771+0000", 2024-09-13T10:11:55.335 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.59e-06 2024-09-13T10:11:55.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.335 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.336 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:11:55.336 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.136002+0000", 2024-09-13T10:11:55.336 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0062306380000000001 2024-09-13T10:11:55.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.336 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.336 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:11:55.336 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.136024+0000", 2024-09-13T10:11:55.336 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2106000000000001e-05 2024-09-13T10:11:55.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.336 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.336 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:11:55.336 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.136035+0000", 2024-09-13T10:11:55.336 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1293e-05 2024-09-13T10:11:55.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.337 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.337 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:11:55.337 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.136115+0000", 2024-09-13T10:11:55.337 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9294000000000003e-05 2024-09-13T10:11:55.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.337 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.337 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:11:55.337 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.136136+0000", 2024-09-13T10:11:55.337 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1495999999999999e-05 2024-09-13T10:11:55.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.337 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.337 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:11:55.338 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.140076+0000", 2024-09-13T10:11:55.338 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0039398849999999997 2024-09-13T10:11:55.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:11:55.338 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:11:55.338 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:11:55.338 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.140184+0000", 2024-09-13T10:11:55.338 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108236 2024-09-13T10:11:55.338 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.338 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.338 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.338 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:11:55.338 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:11:55.338 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:55.341 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T10:11:55.341 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:55.341 INFO:tasks.rados.rados.0.smithi195.stdout:1593: done (9 left) 2024-09-13T10:11:55.342 INFO:tasks.rados.rados.0.smithi195.stdout:1594: done (8 left) 2024-09-13T10:11:55.342 INFO:tasks.rados.rados.0.smithi195.stdout:1595: done (7 left) 2024-09-13T10:11:55.342 INFO:tasks.rados.rados.0.smithi195.stdout:1596: done (6 left) 2024-09-13T10:11:55.342 INFO:tasks.rados.rados.0.smithi195.stdout:1597: done (5 left) 2024-09-13T10:11:55.342 INFO:tasks.rados.rados.0.smithi195.stdout:1598: done (4 left) 2024-09-13T10:11:55.342 INFO:tasks.rados.rados.0.smithi195.stdout:1599: done (3 left) 2024-09-13T10:11:55.342 INFO:tasks.rados.rados.0.smithi195.stdout:1600: done (2 left) 2024-09-13T10:11:55.342 INFO:tasks.rados.rados.0.smithi195.stdout:1601: done (1 left) 2024-09-13T10:11:55.342 INFO:tasks.rados.rados.0.smithi195.stdout:1602: done (0 left) 2024-09-13T10:11:55.342 INFO:tasks.rados.rados.0.smithi195.stdout:1603: read oid 34 snap -1 2024-09-13T10:11:55.342 INFO:tasks.rados.rados.0.smithi195.stdout:1603: expect deleted 2024-09-13T10:11:55.342 INFO:tasks.rados.rados.0.smithi195.stdout:1604: read oid 40 snap -1 2024-09-13T10:11:55.342 INFO:tasks.rados.rados.0.smithi195.stdout:1604: expect deleted 2024-09-13T10:11:55.343 INFO:tasks.rados.rados.0.smithi195.stdout:1605: read oid 5 snap -1 2024-09-13T10:11:55.343 INFO:tasks.rados.rados.0.smithi195.stdout:1605: expect (ObjNum 322 snap 0 seq_num 322) 2024-09-13T10:11:55.343 INFO:tasks.rados.rados.0.smithi195.stdout:1606: tier_promote oid 48 2024-09-13T10:11:55.343 INFO:tasks.rados.rados.0.smithi195.stdout:1607: set_chunk oid 45 target oid 2024-09-13T10:11:55.343 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:55.339+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:55.379 INFO:tasks.rados.rados.0.smithi195.stdout:1607: set_chunk oid 45 offset: 677376 length: 1920000 target oid ad561c8932fccc7c6f054ac53ec9d95308fb2f227ca4f656b65445b565943ca7 offset: 0 snap -1 2024-09-13T10:11:55.380 INFO:tasks.rados.rados.0.smithi195.stdout:1608: tier_promote oid 24 2024-09-13T10:11:55.380 INFO:tasks.rados.rados.0.smithi195.stdout:1609: read oid 20 snap -1 2024-09-13T10:11:55.380 INFO:tasks.rados.rados.0.smithi195.stdout:1609: expect (ObjNum 315 snap 0 seq_num 315) 2024-09-13T10:11:55.380 INFO:tasks.rados.rados.0.smithi195.stdout:1610: read oid 17 snap -1 2024-09-13T10:11:55.380 INFO:tasks.rados.rados.0.smithi195.stdout:1610: expect (ObjNum 323 snap 0 seq_num 323) 2024-09-13T10:11:55.380 INFO:tasks.rados.rados.0.smithi195.stdout:1611: write (excl) oid 22 current snap is 0 2024-09-13T10:11:55.380 INFO:tasks.rados.rados.0.smithi195.stdout:1606: got (0) Success 2024-09-13T10:11:55.380 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 88 (ObjNum 264 snap 0 seq_num 264) dirty exists 2024-09-13T10:11:55.380 INFO:tasks.rados.rados.0.smithi195.stdout:1605: expect (ObjNum 322 snap 0 seq_num 322) 2024-09-13T10:11:55.424 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2824087 2024-09-13T10:11:55.434 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:11:55.468 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:55.469 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:55.469 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:11:55.469 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:55.469 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:55.463+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:55.484 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T10:11:55.523 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 359834 2024-09-13T10:11:55.541 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:55.537+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:55.581 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:55.582 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:55.597 INFO:tasks.rados.rados.0.smithi195.stdout:1607: finishing set_chunk to oid 45 2024-09-13T10:11:55.598 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 45 set_chunk returned error code -95 offset: 677376 length: 1920000 2024-09-13T10:11:55.598 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 0 (ObjNum 311 snap 0 seq_num 311) dirty exists 2024-09-13T10:11:55.598 INFO:tasks.rados.rados.0.smithi195.stdout:1609: expect (ObjNum 315 snap 0 seq_num 315) 2024-09-13T10:11:55.603 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:55.603 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:55.605 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2230979 2024-09-13T10:11:55.624 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:55.624 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:55.643 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:55.638+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:55.646 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:55.646 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:55.658 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:11:55.658 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:11:55.658 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:11:55.658 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:11:55.658 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:11:55.667 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:55.667 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:55.673 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T10:11:55.677 INFO:tasks.rados.rados.0.smithi195.stdout:1608: got (0) Success 2024-09-13T10:11:55.677 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 155 (ObjNum 336 snap 0 seq_num 336) dirty exists 2024-09-13T10:11:55.678 INFO:tasks.rados.rados.0.smithi195.stdout:1610: expect (ObjNum 323 snap 0 seq_num 323) 2024-09-13T10:11:55.689 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:55.690 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:55.712 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:55.712 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:55.737 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:55.737 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:55.737 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T10:11:55.744 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:11:55.737+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:11:55.758 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:55.759 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:55.780 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:55.780 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:55.800 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:55.800 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:55.801 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:55.802 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:55.816 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-13T10:11:55.824 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:55.824 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:55.842 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:55.842 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:55.844 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:55.839+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:55.863 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:55.863 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:55.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:55.859+0000 7f979f7fe640 0 -- 172.21.15.73:0/574706821 >> v2:172.21.15.181:6804/1173379175 conn(0x7f97a0220d80 msgr2=0x7f97a0223170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:11:55.878 INFO:tasks.rados.rados.0.smithi195.stdout:1611: seq_num 341 ranges {531187=779441,2088242=738446,3431711=299476} 2024-09-13T10:11:55.896 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:55.897 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:55.899 INFO:tasks.rados.rados.0.smithi195.stdout:1611: writing smithi19530714-22 from 531187 to 1310628 tid 1 2024-09-13T10:11:55.918 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:55.919 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:55.924 INFO:tasks.rados.rados.0.smithi195.stdout:1611: writing smithi19530714-22 from 2088242 to 2826688 tid 2 2024-09-13T10:11:55.936 INFO:tasks.rados.rados.0.smithi195.stdout:1611: writing smithi19530714-22 from 3431711 to 3731187 tid 3 2024-09-13T10:11:55.936 INFO:tasks.rados.rados.0.smithi195.stdout:1603: done (8 left) 2024-09-13T10:11:55.936 INFO:tasks.rados.rados.0.smithi195.stdout:1604: done (7 left) 2024-09-13T10:11:55.936 INFO:tasks.rados.rados.0.smithi195.stdout:1605: done (6 left) 2024-09-13T10:11:55.936 INFO:tasks.rados.rados.0.smithi195.stdout:1606: done (5 left) 2024-09-13T10:11:55.936 INFO:tasks.rados.rados.0.smithi195.stdout:1607: done (4 left) 2024-09-13T10:11:55.937 INFO:tasks.rados.rados.0.smithi195.stdout:1608: done (3 left) 2024-09-13T10:11:55.937 INFO:tasks.rados.rados.0.smithi195.stdout:1609: done (2 left) 2024-09-13T10:11:55.937 INFO:tasks.rados.rados.0.smithi195.stdout:1610: done (1 left) 2024-09-13T10:11:55.937 INFO:tasks.rados.rados.0.smithi195.stdout:1612: read oid 19 snap -1 2024-09-13T10:11:55.937 INFO:tasks.rados.rados.0.smithi195.stdout:1612: expect (ObjNum 160 snap 0 seq_num 160) 2024-09-13T10:11:55.937 INFO:tasks.rados.rados.0.smithi195.stdout:1613: set_chunk oid 36 target oid 2024-09-13T10:11:55.945 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:55.945 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:55.945 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:55.939+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:55.960 INFO:tasks.rados.rados.0.smithi195.stdout:1613: set_chunk oid 36 offset: 858624 length: 1278464 target oid 211bbe939674e9586f830e545473ac9f9a3c3efe875f841551c173ccfe0d4057 offset: 0 snap -1 2024-09-13T10:11:55.960 INFO:tasks.rados.rados.0.smithi195.stdout:1614: read oid 8 snap -1 2024-09-13T10:11:55.960 INFO:tasks.rados.rados.0.smithi195.stdout:1614: expect (ObjNum 317 snap 0 seq_num 317) 2024-09-13T10:11:55.960 INFO:tasks.rados.rados.0.smithi195.stdout:1615: read oid 25 snap -1 2024-09-13T10:11:55.960 INFO:tasks.rados.rados.0.smithi195.stdout:1615: expect (ObjNum 324 snap 0 seq_num 324) 2024-09-13T10:11:55.960 INFO:tasks.rados.rados.0.smithi195.stdout:1616: write (excl) oid 13 current snap is 0 2024-09-13T10:11:55.960 INFO:tasks.rados.rados.0.smithi195.stdout:1611: finishing write tid 1 to smithi19530714-22 2024-09-13T10:11:55.960 INFO:tasks.rados.rados.0.smithi195.stdout:1616: seq_num 342 ranges {547994=454690,1746724=482104,2659134=465146,3605976=142018} 2024-09-13T10:11:55.961 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-13T10:11:55.972 INFO:tasks.rados.rados.0.smithi195.stdout:1616: writing smithi19530714-13 from 547994 to 1002684 tid 1 2024-09-13T10:11:55.988 INFO:tasks.rados.rados.0.smithi195.stdout:1616: writing smithi19530714-13 from 1746724 to 2228828 tid 2 2024-09-13T10:11:55.998 INFO:tasks.rados.rados.0.smithi195.stdout:1616: writing smithi19530714-13 from 2659134 to 3124280 tid 3 2024-09-13T10:11:56.005 INFO:tasks.rados.rados.0.smithi195.stdout:1616: writing smithi19530714-13 from 3605976 to 3747994 tid 4 2024-09-13T10:11:56.005 INFO:tasks.rados.rados.0.smithi195.stdout:1617: read oid 40 snap -1 2024-09-13T10:11:56.005 INFO:tasks.rados.rados.0.smithi195.stdout:1617: expect deleted 2024-09-13T10:11:56.005 INFO:tasks.rados.rados.0.smithi195.stdout:1618: tier_promote oid 9 2024-09-13T10:11:56.005 INFO:tasks.rados.rados.0.smithi195.stdout:1611: finishing write tid 2 to smithi19530714-22 2024-09-13T10:11:56.005 INFO:tasks.rados.rados.0.smithi195.stdout:1611: finishing write tid 3 to smithi19530714-22 2024-09-13T10:11:56.005 INFO:tasks.rados.rados.0.smithi195.stdout:1619: set_chunk oid 45 target oid 2024-09-13T10:11:56.015 INFO:tasks.rados.rados.0.smithi195.stdout:1619: set_chunk oid 45 offset: 2423296 length: 282624 target oid 355ce102c234bb70fbb51eeb1dd975a16cf70ae59d5440ed7140e460ec58ee3d offset: 0 snap -1 2024-09-13T10:11:56.015 INFO:tasks.rados.rados.0.smithi195.stdout:1620: write (excl) oid 42 current snap is 0 2024-09-13T10:11:56.016 INFO:tasks.rados.rados.0.smithi195.stdout:1620: seq_num 343 ranges {564801=646292,1921559=742115,3202910=561891} 2024-09-13T10:11:56.028 INFO:tasks.rados.rados.0.smithi195.stdout:1620: writing smithi19530714-42 from 564801 to 1211093 tid 1 2024-09-13T10:11:56.045 INFO:tasks.rados.rados.0.smithi195.stdout:1620: writing smithi19530714-42 from 1921559 to 2663674 tid 2 2024-09-13T10:11:56.046 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:11:56.043+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:11:56.054 INFO:tasks.rados.rados.0.smithi195.stdout:1620: writing smithi19530714-42 from 3202910 to 3764801 tid 3 2024-09-13T10:11:56.054 INFO:tasks.rados.rados.0.smithi195.stdout:1621: read oid 12 snap -1 2024-09-13T10:11:56.054 INFO:tasks.rados.rados.0.smithi195.stdout:1621: expect (ObjNum 334 snap 0 seq_num 334) 2024-09-13T10:11:56.054 INFO:tasks.rados.rados.0.smithi195.stdout:1622: write (excl) oid 43 current snap is 0 2024-09-13T10:11:56.054 INFO:tasks.rados.rados.0.smithi195.stdout:1611: finishing write tid 4 to smithi19530714-22 2024-09-13T10:11:56.054 INFO:tasks.rados.rados.0.smithi195.stdout:1611: finishing write tid 5 to smithi19530714-22 2024-09-13T10:11:56.054 INFO:tasks.rados.rados.0.smithi195.stdout:1611: oid 22 updating version 0 to 133 2024-09-13T10:11:56.054 INFO:tasks.rados.rados.0.smithi195.stdout:1611: oid 22 updating version 133 to 134 2024-09-13T10:11:56.054 INFO:tasks.rados.rados.0.smithi195.stdout:1611: oid 22 updating version 134 to 135 2024-09-13T10:11:56.055 INFO:tasks.rados.rados.0.smithi195.stdout:1611: oid 22 updating version 135 to 136 2024-09-13T10:11:56.055 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 22 v 136 (ObjNum 341 snap 0 seq_num 341) dirty exists 2024-09-13T10:11:56.055 INFO:tasks.rados.rados.0.smithi195.stdout:1611: left oid 22 (ObjNum 341 snap 0 seq_num 341) 2024-09-13T10:11:56.055 INFO:tasks.rados.rados.0.smithi195.stdout:1612: expect (ObjNum 160 snap 0 seq_num 160) 2024-09-13T10:11:56.118 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:56.118 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:56.132 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-13T10:11:56.138 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:11:56.139 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:11:56.139 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:11:56.139 INFO:tasks.thrashosds.thrasher:inject_pause on osd.15 2024-09-13T10:11:56.139 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T10:11:56.139 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T10:11:56.139 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok config set filestore_inject_stall 3 2024-09-13T10:11:56.147 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:11:56.142+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:11:56.162 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:56.248 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:56.246+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:56.260 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:11:56.260 INFO:teuthology.orchestra.run.smithi195.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T10:11:56.260 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:11:56.292 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:56.292 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:56.294 INFO:tasks.rados.rados.0.smithi195.stdout:1613: finishing set_chunk to oid 36 2024-09-13T10:11:56.294 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 36 (ObjNum 269 snap 0 seq_num 269) dirty exists 2024-09-13T10:11:56.294 INFO:tasks.rados.rados.0.smithi195.stdout:1615: expect (ObjNum 324 snap 0 seq_num 324) 2024-09-13T10:11:56.304 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-13T10:11:56.349 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:56.345+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:56.366 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 860389 2024-09-13T10:11:56.423 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:56.424 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:56.438 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-13T10:11:56.479 INFO:tasks.rados.rados.0.smithi195.stdout:1614: expect (ObjNum 317 snap 0 seq_num 317) 2024-09-13T10:11:56.482 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:56.479+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:56.522 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2188153 2024-09-13T10:11:56.542 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2512766 2024-09-13T10:11:56.550 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:56.547+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:56.553 INFO:tasks.rados.rados.0.smithi195.stdout:1616: finishing write tid 1 to smithi19530714-13 2024-09-13T10:11:56.553 INFO:tasks.rados.rados.0.smithi195.stdout:1616: finishing write tid 2 to smithi19530714-13 2024-09-13T10:11:56.553 INFO:tasks.rados.rados.0.smithi195.stdout:1616: finishing write tid 3 to smithi19530714-13 2024-09-13T10:11:56.554 INFO:tasks.rados.rados.0.smithi195.stdout:1618: got (0) Success 2024-09-13T10:11:56.554 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 123 (ObjNum 259 snap 0 seq_num 259) dirty exists 2024-09-13T10:11:56.554 INFO:tasks.rados.rados.0.smithi195.stdout:1616: finishing write tid 4 to smithi19530714-13 2024-09-13T10:11:56.554 INFO:tasks.rados.rados.0.smithi195.stdout:1616: finishing write tid 5 to smithi19530714-13 2024-09-13T10:11:56.554 INFO:tasks.rados.rados.0.smithi195.stdout:1616: finishing write tid 6 to smithi19530714-13 2024-09-13T10:11:56.554 INFO:tasks.rados.rados.0.smithi195.stdout:1616: oid 13 updating version 0 to 81 2024-09-13T10:11:56.554 INFO:tasks.rados.rados.0.smithi195.stdout:1616: oid 13 updating version 81 to 83 2024-09-13T10:11:56.554 INFO:tasks.rados.rados.0.smithi195.stdout:1616: oid 13 updating version 83 to 84 2024-09-13T10:11:56.554 INFO:tasks.rados.rados.0.smithi195.stdout:1616: oid 13 version 84 is already newer than 82 2024-09-13T10:11:56.554 INFO:tasks.rados.rados.0.smithi195.stdout:1616: oid 13 version 84 is already newer than 80 2024-09-13T10:11:56.554 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 84 (ObjNum 342 snap 0 seq_num 342) dirty exists 2024-09-13T10:11:56.554 INFO:tasks.rados.rados.0.smithi195.stdout:1616: left oid 13 (ObjNum 342 snap 0 seq_num 342) 2024-09-13T10:11:56.554 INFO:tasks.rados.rados.0.smithi195.stdout:1619: finishing set_chunk to oid 45 2024-09-13T10:11:56.555 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 45 set_chunk returned error code -95 offset: 2423296 length: 282624 2024-09-13T10:11:56.555 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 0 (ObjNum 311 snap 0 seq_num 311) dirty exists 2024-09-13T10:11:56.555 INFO:tasks.rados.rados.0.smithi195.stdout:1620: finishing write tid 1 to smithi19530714-42 2024-09-13T10:11:56.555 INFO:tasks.rados.rados.0.smithi195.stdout:1620: finishing write tid 2 to smithi19530714-42 2024-09-13T10:11:56.555 INFO:tasks.rados.rados.0.smithi195.stdout:1620: finishing write tid 3 to smithi19530714-42 2024-09-13T10:11:56.555 INFO:tasks.rados.rados.0.smithi195.stdout:1620: finishing write tid 4 to smithi19530714-42 2024-09-13T10:11:56.555 INFO:tasks.rados.rados.0.smithi195.stdout:1620: finishing write tid 5 to smithi19530714-42 2024-09-13T10:11:56.555 INFO:tasks.rados.rados.0.smithi195.stdout:1620: oid 42 updating version 0 to 24 2024-09-13T10:11:56.555 INFO:tasks.rados.rados.0.smithi195.stdout:1620: oid 42 updating version 24 to 26 2024-09-13T10:11:56.555 INFO:tasks.rados.rados.0.smithi195.stdout:1620: oid 42 version 26 is already newer than 23 2024-09-13T10:11:56.555 INFO:tasks.rados.rados.0.smithi195.stdout:1620: oid 42 version 26 is already newer than 25 2024-09-13T10:11:56.555 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 26 (ObjNum 343 snap 0 seq_num 343) dirty exists 2024-09-13T10:11:56.555 INFO:tasks.rados.rados.0.smithi195.stdout:1620: left oid 42 (ObjNum 343 snap 0 seq_num 343) 2024-09-13T10:11:56.555 INFO:tasks.rados.rados.0.smithi195.stdout:1621: expect (ObjNum 334 snap 0 seq_num 334) 2024-09-13T10:11:56.560 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:56.560 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:56.574 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-13T10:11:56.603 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:56.652 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:56.646+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:56.703 INFO:tasks.rados.rados.0.smithi195.stdout:1622: seq_num 344 ranges {581608=721541,1863688=485773,3113980=604043} 2024-09-13T10:11:56.723 INFO:tasks.rados.rados.0.smithi195.stdout:1622: writing smithi19530714-43 from 581608 to 1303149 tid 1 2024-09-13T10:11:56.726 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-13T10:11:56.741 INFO:tasks.rados.rados.0.smithi195.stdout:1622: writing smithi19530714-43 from 1863688 to 2349461 tid 2 2024-09-13T10:11:56.741 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:56.741 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:56.752 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:56.747+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:56.755 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T10:11:56.762 INFO:tasks.rados.rados.0.smithi195.stdout:1622: writing smithi19530714-43 from 3113980 to 3718023 tid 3 2024-09-13T10:11:56.762 INFO:tasks.rados.rados.0.smithi195.stdout:1611: done (11 left) 2024-09-13T10:11:56.762 INFO:tasks.rados.rados.0.smithi195.stdout:1612: done (10 left) 2024-09-13T10:11:56.762 INFO:tasks.rados.rados.0.smithi195.stdout:1613: done (9 left) 2024-09-13T10:11:56.762 INFO:tasks.rados.rados.0.smithi195.stdout:1614: done (8 left) 2024-09-13T10:11:56.762 INFO:tasks.rados.rados.0.smithi195.stdout:1615: done (7 left) 2024-09-13T10:11:56.763 INFO:tasks.rados.rados.0.smithi195.stdout:1616: done (6 left) 2024-09-13T10:11:56.763 INFO:tasks.rados.rados.0.smithi195.stdout:1617: done (5 left) 2024-09-13T10:11:56.763 INFO:tasks.rados.rados.0.smithi195.stdout:1618: done (4 left) 2024-09-13T10:11:56.763 INFO:tasks.rados.rados.0.smithi195.stdout:1619: done (3 left) 2024-09-13T10:11:56.763 INFO:tasks.rados.rados.0.smithi195.stdout:1620: done (2 left) 2024-09-13T10:11:56.763 INFO:tasks.rados.rados.0.smithi195.stdout:1621: done (1 left) 2024-09-13T10:11:56.763 INFO:tasks.rados.rados.0.smithi195.stdout:1623: read oid 45 snap -1 2024-09-13T10:11:56.763 INFO:tasks.rados.rados.0.smithi195.stdout:1623: expect (ObjNum 311 snap 0 seq_num 311) 2024-09-13T10:11:56.763 INFO:tasks.rados.rados.0.smithi195.stdout:1624: read oid 29 snap -1 2024-09-13T10:11:56.763 INFO:tasks.rados.rados.0.smithi195.stdout:1624: expect (ObjNum 220 snap 0 seq_num 220) 2024-09-13T10:11:56.763 INFO:tasks.rados.rados.0.smithi195.stdout:1625: set_chunk oid 11 target oid 2024-09-13T10:11:56.763 INFO:tasks.rados.rados.0.smithi195.stdout:1625: set_chunk oid 11 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:56.763 INFO:tasks.rados.rados.0.smithi195.stdout:1626: delete oid 33 current snap is 0 2024-09-13T10:11:56.763 INFO:tasks.rados.rados.0.smithi195.stdout:1622: finishing write tid 1 to smithi19530714-43 2024-09-13T10:11:56.764 INFO:tasks.rados.rados.0.smithi195.stdout:1622: finishing write tid 2 to smithi19530714-43 2024-09-13T10:11:56.764 INFO:tasks.rados.rados.0.smithi195.stdout:1625: finishing set_chunk to oid 11 2024-09-13T10:11:56.764 INFO:tasks.rados.rados.0.smithi195.stdout:1625: got expected ENOENT (src dne) 2024-09-13T10:11:56.770 INFO:tasks.rados.rados.0.smithi195.stdout:1625: done (4 left) 2024-09-13T10:11:56.770 INFO:tasks.rados.rados.0.smithi195.stdout:1626: done (3 left) 2024-09-13T10:11:56.770 INFO:tasks.rados.rados.0.smithi195.stdout:1627: read oid 26 snap -1 2024-09-13T10:11:56.770 INFO:tasks.rados.rados.0.smithi195.stdout:1627: expect (ObjNum 292 snap 0 seq_num 292) 2024-09-13T10:11:56.770 INFO:tasks.rados.rados.0.smithi195.stdout:1628: write (excl) oid 19 current snap is 0 2024-09-13T10:11:56.770 INFO:tasks.rados.rados.0.smithi195.stdout:1628: seq_num 346 ranges {615222=472039,1580652=489442,2768826=626587} 2024-09-13T10:11:56.784 INFO:tasks.rados.rados.0.smithi195.stdout:1628: writing smithi19530714-19 from 615222 to 1087261 tid 1 2024-09-13T10:11:56.797 INFO:tasks.rados.rados.0.smithi195.stdout:1628: writing smithi19530714-19 from 1580652 to 2070094 tid 2 2024-09-13T10:11:56.813 INFO:tasks.rados.rados.0.smithi195.stdout:1628: writing smithi19530714-19 from 2768826 to 3395413 tid 3 2024-09-13T10:11:56.813 INFO:tasks.rados.rados.0.smithi195.stdout:1629: write (excl) oid 17 current snap is 0 2024-09-13T10:11:56.814 INFO:tasks.rados.rados.0.smithi195.stdout:1622: finishing write tid 3 to smithi19530714-43 2024-09-13T10:11:56.814 INFO:tasks.rados.rados.0.smithi195.stdout:1622: finishing write tid 4 to smithi19530714-43 2024-09-13T10:11:56.814 INFO:tasks.rados.rados.0.smithi195.stdout:1622: finishing write tid 5 to smithi19530714-43 2024-09-13T10:11:56.814 INFO:tasks.rados.rados.0.smithi195.stdout:1622: oid 43 updating version 0 to 88 2024-09-13T10:11:56.814 INFO:tasks.rados.rados.0.smithi195.stdout:1622: oid 43 updating version 88 to 91 2024-09-13T10:11:56.814 INFO:tasks.rados.rados.0.smithi195.stdout:1622: oid 43 version 91 is already newer than 90 2024-09-13T10:11:56.814 INFO:tasks.rados.rados.0.smithi195.stdout:1622: oid 43 version 91 is already newer than 89 2024-09-13T10:11:56.814 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 91 (ObjNum 344 snap 0 seq_num 344) dirty exists 2024-09-13T10:11:56.814 INFO:tasks.rados.rados.0.smithi195.stdout:1622: left oid 43 (ObjNum 344 snap 0 seq_num 344) 2024-09-13T10:11:56.814 INFO:tasks.rados.rados.0.smithi195.stdout:1623: expect (ObjNum 311 snap 0 seq_num 311) 2024-09-13T10:11:56.853 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:56.847+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:56.887 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:56.891 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:56.891 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:56.905 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T10:11:56.954 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:56.947+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:56.989 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1988904 2024-09-13T10:11:57.015 INFO:tasks.rados.rados.0.smithi195.stdout:1627: expect (ObjNum 292 snap 0 seq_num 292) 2024-09-13T10:11:57.029 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:57.029 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:57.043 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T10:11:57.055 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:57.051+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:57.083 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 661409 2024-09-13T10:11:57.155 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:57.150+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:57.183 INFO:tasks.rados.rados.0.smithi195.stdout:1624: expect (ObjNum 220 snap 0 seq_num 220) 2024-09-13T10:11:57.203 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:57.207 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:57.207 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:57.223 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T10:11:57.262 INFO:tasks.rados.rados.0.smithi195.stdout:1628: finishing write tid 1 to smithi19530714-19 2024-09-13T10:11:57.262 INFO:tasks.rados.rados.0.smithi195.stdout:1628: finishing write tid 2 to smithi19530714-19 2024-09-13T10:11:57.262 INFO:tasks.rados.rados.0.smithi195.stdout:1629: seq_num 347 ranges {632029=547288,1639134=749453,2796249=437859,3653524=178505} 2024-09-13T10:11:57.269 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:11:57.263+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:11:57.281 INFO:tasks.rados.rados.0.smithi195.stdout:1629: writing smithi19530714-17 from 632029 to 1179317 tid 1 2024-09-13T10:11:57.301 INFO:tasks.rados.rados.0.smithi195.stdout:1629: writing smithi19530714-17 from 1639134 to 2388587 tid 2 2024-09-13T10:11:57.314 INFO:tasks.rados.rados.0.smithi195.stdout:1629: writing smithi19530714-17 from 2796249 to 3234108 tid 3 2024-09-13T10:11:57.322 INFO:tasks.rados.rados.0.smithi195.stdout:1629: writing smithi19530714-17 from 3653524 to 3832029 tid 4 2024-09-13T10:11:57.323 INFO:tasks.rados.rados.0.smithi195.stdout:1622: done (5 left) 2024-09-13T10:11:57.323 INFO:tasks.rados.rados.0.smithi195.stdout:1623: done (4 left) 2024-09-13T10:11:57.323 INFO:tasks.rados.rados.0.smithi195.stdout:1624: done (3 left) 2024-09-13T10:11:57.323 INFO:tasks.rados.rados.0.smithi195.stdout:1627: done (2 left) 2024-09-13T10:11:57.323 INFO:tasks.rados.rados.0.smithi195.stdout:1630: delete oid 24 current snap is 0 2024-09-13T10:11:57.323 INFO:tasks.rados.rados.0.smithi195.stdout:1628: finishing write tid 3 to smithi19530714-19 2024-09-13T10:11:57.323 INFO:tasks.rados.rados.0.smithi195.stdout:1628: finishing write tid 4 to smithi19530714-19 2024-09-13T10:11:57.323 INFO:tasks.rados.rados.0.smithi195.stdout:1628: finishing write tid 5 to smithi19530714-19 2024-09-13T10:11:57.323 INFO:tasks.rados.rados.0.smithi195.stdout:1628: oid 19 updating version 0 to 75 2024-09-13T10:11:57.323 INFO:tasks.rados.rados.0.smithi195.stdout:1628: oid 19 updating version 75 to 77 2024-09-13T10:11:57.323 INFO:tasks.rados.rados.0.smithi195.stdout:1628: oid 19 version 77 is already newer than 76 2024-09-13T10:11:57.323 INFO:tasks.rados.rados.0.smithi195.stdout:1628: oid 19 updating version 77 to 78 2024-09-13T10:11:57.323 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 78 (ObjNum 346 snap 0 seq_num 346) dirty exists 2024-09-13T10:11:57.324 INFO:tasks.rados.rados.0.smithi195.stdout:1628: left oid 19 (ObjNum 346 snap 0 seq_num 346) 2024-09-13T10:11:57.324 INFO:tasks.rados.rados.0.smithi195.stdout:1629: finishing write tid 1 to smithi19530714-17 2024-09-13T10:11:57.324 INFO:tasks.rados.rados.0.smithi195.stdout:1629: finishing write tid 2 to smithi19530714-17 2024-09-13T10:11:57.324 INFO:tasks.rados.rados.0.smithi195.stdout:1629: finishing write tid 3 to smithi19530714-17 2024-09-13T10:11:57.327 INFO:tasks.rados.rados.0.smithi195.stdout:1629: finishing write tid 4 to smithi19530714-17 2024-09-13T10:11:57.329 INFO:tasks.rados.rados.0.smithi195.stdout:1629: finishing write tid 5 to smithi19530714-17 2024-09-13T10:11:57.329 INFO:tasks.rados.rados.0.smithi195.stdout:1628: done (2 left) 2024-09-13T10:11:57.330 INFO:tasks.rados.rados.0.smithi195.stdout:1630: done (1 left) 2024-09-13T10:11:57.330 INFO:tasks.rados.rados.0.smithi195.stdout:1631: delete oid 31 current snap is 0 2024-09-13T10:11:57.330 INFO:tasks.rados.rados.0.smithi195.stdout:1629: finishing write tid 6 to smithi19530714-17 2024-09-13T10:11:57.330 INFO:tasks.rados.rados.0.smithi195.stdout:1629: oid 17 updating version 0 to 94 2024-09-13T10:11:57.331 INFO:tasks.rados.rados.0.smithi195.stdout:1629: oid 17 updating version 94 to 95 2024-09-13T10:11:57.331 INFO:tasks.rados.rados.0.smithi195.stdout:1629: oid 17 version 95 is already newer than 93 2024-09-13T10:11:57.331 INFO:tasks.rados.rados.0.smithi195.stdout:1629: oid 17 version 95 is already newer than 91 2024-09-13T10:11:57.331 INFO:tasks.rados.rados.0.smithi195.stdout:1629: oid 17 version 95 is already newer than 92 2024-09-13T10:11:57.331 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 95 (ObjNum 347 snap 0 seq_num 347) dirty exists 2024-09-13T10:11:57.331 INFO:tasks.rados.rados.0.smithi195.stdout:1629: left oid 17 (ObjNum 347 snap 0 seq_num 347) 2024-09-13T10:11:57.333 INFO:tasks.rados.rados.0.smithi195.stdout:1629: done (1 left) 2024-09-13T10:11:57.333 INFO:tasks.rados.rados.0.smithi195.stdout:1631: done (0 left) 2024-09-13T10:11:57.333 INFO:tasks.rados.rados.0.smithi195.stdout:1632: set_chunk oid 47 target oid 2024-09-13T10:11:57.341 INFO:tasks.rados.rados.0.smithi195.stdout:1632: set_chunk oid 47 offset: 1331712 length: 271872 target oid 608ccf8ca7a0c376093ff665bd05c8976e7002e125ade44b3e997e97696f7dda offset: 0 snap -1 2024-09-13T10:11:57.341 INFO:tasks.rados.rados.0.smithi195.stdout:1633: write (excl) oid 31 current snap is 0 2024-09-13T10:11:57.341 INFO:tasks.rados.rados.0.smithi195.stdout:1633: seq_num 350 ranges {682450=489388,1814580=496780,2878518=788028} 2024-09-13T10:11:57.346 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:57.346 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:57.352 INFO:tasks.rados.rados.0.smithi195.stdout:1633: writing smithi19530714-31 from 682450 to 1171838 tid 1 2024-09-13T10:11:57.357 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:57.353+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:57.362 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T10:11:57.363 INFO:tasks.rados.rados.0.smithi195.stdout:1633: writing smithi19530714-31 from 1814580 to 2311360 tid 2 2024-09-13T10:11:57.377 INFO:tasks.rados.rados.0.smithi195.stdout:1633: writing smithi19530714-31 from 2878518 to 3666546 tid 3 2024-09-13T10:11:57.377 INFO:tasks.rados.rados.0.smithi195.stdout:1634: set_chunk oid 17 target oid 2024-09-13T10:11:57.405 INFO:tasks.rados.rados.0.smithi195.stdout:1634: set_chunk oid 17 offset: 1043968 length: 2142720 target oid 52ced48b5b8c98b8f5805dad5ddb3d963508dff3f2820e9669dd447a70f434b5 offset: 0 snap -1 2024-09-13T10:11:57.405 INFO:tasks.rados.rados.0.smithi195.stdout:1635: set_chunk oid 15 target oid 2024-09-13T10:11:57.406 INFO:tasks.rados.rados.0.smithi195.stdout:1635: set_chunk oid 15 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:57.406 INFO:tasks.rados.rados.0.smithi195.stdout:1636: read oid 34 snap -1 2024-09-13T10:11:57.406 INFO:tasks.rados.rados.0.smithi195.stdout:1636: expect deleted 2024-09-13T10:11:57.406 INFO:tasks.rados.rados.0.smithi195.stdout:1637: write (excl) oid 8 current snap is 0 2024-09-13T10:11:57.406 INFO:tasks.rados.rados.0.smithi195.stdout:1637: seq_num 351 ranges {699257=564637,1873062=640438,3305941=593316} 2024-09-13T10:11:57.418 INFO:tasks.rados.rados.0.smithi195.stdout:1637: writing smithi19530714-8 from 699257 to 1263894 tid 1 2024-09-13T10:11:57.431 INFO:tasks.rados.rados.0.smithi195.stdout:1637: writing smithi19530714-8 from 1873062 to 2513500 tid 2 2024-09-13T10:11:57.444 INFO:tasks.rados.rados.0.smithi195.stdout:1637: writing smithi19530714-8 from 3305941 to 3899257 tid 3 2024-09-13T10:11:57.444 INFO:tasks.rados.rados.0.smithi195.stdout:1638: read oid 19 snap -1 2024-09-13T10:11:57.444 INFO:tasks.rados.rados.0.smithi195.stdout:1638: expect (ObjNum 346 snap 0 seq_num 346) 2024-09-13T10:11:57.444 INFO:tasks.rados.rados.0.smithi195.stdout:1639: write (excl) oid 4 current snap is 0 2024-09-13T10:11:57.444 INFO:tasks.rados.rados.0.smithi195.stdout:1632: finishing set_chunk to oid 47 2024-09-13T10:11:57.444 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 54 (ObjNum 260 snap 0 seq_num 260) dirty exists 2024-09-13T10:11:57.444 INFO:tasks.rados.rados.0.smithi195.stdout:1639: seq_num 352 ranges {716064=639886,1815191=500449,2817011=526925} 2024-09-13T10:11:57.455 INFO:tasks.rados.rados.0.smithi195.stdout:1639: writing smithi19530714-4 from 716064 to 1355950 tid 1 2024-09-13T10:11:57.458 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:57.454+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:57.462 INFO:tasks.rados.rados.0.smithi195.stdout:1639: writing smithi19530714-4 from 1815191 to 2315640 tid 2 2024-09-13T10:11:57.473 INFO:tasks.rados.rados.0.smithi195.stdout:1639: writing smithi19530714-4 from 2817011 to 3343936 tid 3 2024-09-13T10:11:57.473 INFO:tasks.rados.rados.0.smithi195.stdout:1632: done (7 left) 2024-09-13T10:11:57.474 INFO:tasks.rados.rados.0.smithi195.stdout:1640: set_chunk oid 1 target oid 2024-09-13T10:11:57.474 INFO:tasks.rados.rados.0.smithi195.stdout:1640: set_chunk oid 1 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:11:57.474 INFO:tasks.rados.rados.0.smithi195.stdout:1641: set_chunk oid 28 target oid 2024-09-13T10:11:57.483 INFO:tasks.rados.rados.0.smithi195.stdout:1641: set_chunk oid 28 offset: 1769984 length: 498176 target oid de3d022ab516452453fbce62932ec6534fc052415d7af55312bf63ead2c05ca5 offset: 0 snap -1 2024-09-13T10:11:57.483 INFO:tasks.rados.rados.0.smithi195.stdout:1642: read oid 7 snap -1 2024-09-13T10:11:57.483 INFO:tasks.rados.rados.0.smithi195.stdout:1642: expect (ObjNum 276 snap 0 seq_num 276) 2024-09-13T10:11:57.483 INFO:tasks.rados.rados.0.smithi195.stdout:1643: tier_evict oid 6 2024-09-13T10:11:57.483 INFO:tasks.rados.rados.0.smithi195.stdout:1643: tier_evict oid 6 snap -1 2024-09-13T10:11:57.483 INFO:tasks.rados.rados.0.smithi195.stdout:1633: finishing write tid 1 to smithi19530714-31 2024-09-13T10:11:57.483 INFO:tasks.rados.rados.0.smithi195.stdout:1633: finishing write tid 2 to smithi19530714-31 2024-09-13T10:11:57.483 INFO:tasks.rados.rados.0.smithi195.stdout:1633: finishing write tid 3 to smithi19530714-31 2024-09-13T10:11:57.483 INFO:tasks.rados.rados.0.smithi195.stdout:1644: read oid 45 snap -1 2024-09-13T10:11:57.483 INFO:tasks.rados.rados.0.smithi195.stdout:1644: expect (ObjNum 311 snap 0 seq_num 311) 2024-09-13T10:11:57.483 INFO:tasks.rados.rados.0.smithi195.stdout:1645: read oid 12 snap -1 2024-09-13T10:11:57.484 INFO:tasks.rados.rados.0.smithi195.stdout:1645: expect (ObjNum 334 snap 0 seq_num 334) 2024-09-13T10:11:57.484 INFO:tasks.rados.rados.0.smithi195.stdout:1646: read oid 2 snap -1 2024-09-13T10:11:57.484 INFO:tasks.rados.rados.0.smithi195.stdout:1646: expect (ObjNum 280 snap 0 seq_num 280) 2024-09-13T10:11:57.484 INFO:tasks.rados.rados.0.smithi195.stdout:1647: delete oid 30 current snap is 0 2024-09-13T10:11:57.484 INFO:tasks.rados.rados.0.smithi195.stdout:1633: finishing write tid 4 to smithi19530714-31 2024-09-13T10:11:57.484 INFO:tasks.rados.rados.0.smithi195.stdout:1633: finishing write tid 5 to smithi19530714-31 2024-09-13T10:11:57.484 INFO:tasks.rados.rados.0.smithi195.stdout:1633: oid 31 updating version 0 to 95 2024-09-13T10:11:57.484 INFO:tasks.rados.rados.0.smithi195.stdout:1633: oid 31 updating version 95 to 96 2024-09-13T10:11:57.484 INFO:tasks.rados.rados.0.smithi195.stdout:1633: oid 31 version 96 is already newer than 94 2024-09-13T10:11:57.484 INFO:tasks.rados.rados.0.smithi195.stdout:1633: oid 31 version 96 is already newer than 93 2024-09-13T10:11:57.484 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 96 (ObjNum 350 snap 0 seq_num 350) dirty exists 2024-09-13T10:11:57.484 INFO:tasks.rados.rados.0.smithi195.stdout:1633: left oid 31 (ObjNum 350 snap 0 seq_num 350) 2024-09-13T10:11:57.485 INFO:tasks.rados.rados.0.smithi195.stdout:1635: finishing set_chunk to oid 15 2024-09-13T10:11:57.485 INFO:tasks.rados.rados.0.smithi195.stdout:1635: got expected ENOENT (src dne) 2024-09-13T10:11:57.485 INFO:tasks.rados.rados.0.smithi195.stdout:1634: finishing set_chunk to oid 17 2024-09-13T10:11:57.485 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 95 (ObjNum 347 snap 0 seq_num 347) dirty exists 2024-09-13T10:11:57.485 INFO:tasks.rados.rados.0.smithi195.stdout:1637: finishing write tid 1 to smithi19530714-8 2024-09-13T10:11:57.485 INFO:tasks.rados.rados.0.smithi195.stdout:1637: finishing write tid 2 to smithi19530714-8 2024-09-13T10:11:57.485 INFO:tasks.rados.rados.0.smithi195.stdout:1637: finishing write tid 3 to smithi19530714-8 2024-09-13T10:11:57.485 INFO:tasks.rados.rados.0.smithi195.stdout:1637: finishing write tid 4 to smithi19530714-8 2024-09-13T10:11:57.485 INFO:tasks.rados.rados.0.smithi195.stdout:1637: finishing write tid 5 to smithi19530714-8 2024-09-13T10:11:57.485 INFO:tasks.rados.rados.0.smithi195.stdout:1637: oid 8 updating version 0 to 28 2024-09-13T10:11:57.485 INFO:tasks.rados.rados.0.smithi195.stdout:1637: oid 8 version 28 is already newer than 26 2024-09-13T10:11:57.485 INFO:tasks.rados.rados.0.smithi195.stdout:1637: oid 8 version 28 is already newer than 27 2024-09-13T10:11:57.485 INFO:tasks.rados.rados.0.smithi195.stdout:1637: oid 8 updating version 28 to 29 2024-09-13T10:11:57.485 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 29 (ObjNum 351 snap 0 seq_num 351) dirty exists 2024-09-13T10:11:57.486 INFO:tasks.rados.rados.0.smithi195.stdout:1637: left oid 8 (ObjNum 351 snap 0 seq_num 351) 2024-09-13T10:11:57.486 INFO:tasks.rados.rados.0.smithi195.stdout:1639: finishing write tid 1 to smithi19530714-4 2024-09-13T10:11:57.486 INFO:tasks.rados.rados.0.smithi195.stdout:1638: expect (ObjNum 346 snap 0 seq_num 346) 2024-09-13T10:11:57.533 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:57.533 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:57.547 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T10:11:57.591 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:57.587+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:57.614 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:57.660 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:57.655+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:57.669 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:57.669 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:57.683 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-13T10:11:57.719 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1239240 2024-09-13T10:11:57.760 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:57.755+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:57.772 INFO:tasks.rados.rados.0.smithi195.stdout:1639: finishing write tid 2 to smithi19530714-4 2024-09-13T10:11:57.772 INFO:tasks.rados.rados.0.smithi195.stdout:1640: finishing set_chunk to oid 1 2024-09-13T10:11:57.772 INFO:tasks.rados.rados.0.smithi195.stdout:1640: got expected ENOENT (src dne) 2024-09-13T10:11:57.772 INFO:tasks.rados.rados.0.smithi195.stdout:1639: finishing write tid 3 to smithi19530714-4 2024-09-13T10:11:57.772 INFO:tasks.rados.rados.0.smithi195.stdout:1639: finishing write tid 4 to smithi19530714-4 2024-09-13T10:11:57.772 INFO:tasks.rados.rados.0.smithi195.stdout:1639: finishing write tid 5 to smithi19530714-4 2024-09-13T10:11:57.772 INFO:tasks.rados.rados.0.smithi195.stdout:1639: oid 4 updating version 0 to 119 2024-09-13T10:11:57.773 INFO:tasks.rados.rados.0.smithi195.stdout:1639: oid 4 version 119 is already newer than 118 2024-09-13T10:11:57.773 INFO:tasks.rados.rados.0.smithi195.stdout:1639: oid 4 updating version 119 to 120 2024-09-13T10:11:57.773 INFO:tasks.rados.rados.0.smithi195.stdout:1639: oid 4 updating version 120 to 121 2024-09-13T10:11:57.773 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 121 (ObjNum 352 snap 0 seq_num 352) dirty exists 2024-09-13T10:11:57.773 INFO:tasks.rados.rados.0.smithi195.stdout:1639: left oid 4 (ObjNum 352 snap 0 seq_num 352) 2024-09-13T10:11:57.773 INFO:tasks.rados.rados.0.smithi195.stdout:1643: got (22) Invalid argument 2024-09-13T10:11:57.773 INFO:tasks.rados.rados.0.smithi195.stdout:1633: done (14 left) 2024-09-13T10:11:57.773 INFO:tasks.rados.rados.0.smithi195.stdout:1634: done (13 left) 2024-09-13T10:11:57.773 INFO:tasks.rados.rados.0.smithi195.stdout:1635: done (12 left) 2024-09-13T10:11:57.773 INFO:tasks.rados.rados.0.smithi195.stdout:1636: done (11 left) 2024-09-13T10:11:57.773 INFO:tasks.rados.rados.0.smithi195.stdout:1637: done (10 left) 2024-09-13T10:11:57.773 INFO:tasks.rados.rados.0.smithi195.stdout:1638: done (9 left) 2024-09-13T10:11:57.773 INFO:tasks.rados.rados.0.smithi195.stdout:1639: done (8 left) 2024-09-13T10:11:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:1640: done (7 left) 2024-09-13T10:11:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:1643: done (6 left) 2024-09-13T10:11:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:1647: done (5 left) 2024-09-13T10:11:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:1648: read oid 30 snap -1 2024-09-13T10:11:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:1648: expect deleted 2024-09-13T10:11:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:1649: read oid 44 snap -1 2024-09-13T10:11:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:1649: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:11:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:1650: read oid 47 snap -1 2024-09-13T10:11:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:1650: expect (ObjNum 260 snap 0 seq_num 260) 2024-09-13T10:11:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:1651: read oid 50 snap -1 2024-09-13T10:11:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:1651: expect (ObjNum 303 snap 0 seq_num 303) 2024-09-13T10:11:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:1652: read oid 20 snap -1 2024-09-13T10:11:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:1652: expect (ObjNum 315 snap 0 seq_num 315) 2024-09-13T10:11:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:1653: delete oid 31 current snap is 0 2024-09-13T10:11:57.774 INFO:tasks.rados.rados.0.smithi195.stdout:1641: finishing set_chunk to oid 28 2024-09-13T10:11:57.775 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 28 set_chunk returned error code -95 offset: 1769984 length: 498176 2024-09-13T10:11:57.775 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 0 (ObjNum 180 snap 0 seq_num 180) dirty exists 2024-09-13T10:11:57.775 INFO:tasks.rados.rados.0.smithi195.stdout:1642: expect (ObjNum 276 snap 0 seq_num 276) 2024-09-13T10:11:57.850 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:57.850 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:57.850 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1735500 2024-09-13T10:11:57.861 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:57.855+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:57.864 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-13T10:11:57.869 INFO:tasks.rados.rados.0.smithi195.stdout:1644: expect (ObjNum 311 snap 0 seq_num 311) 2024-09-13T10:11:57.939 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:11:57.961 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:11:57.957+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:11:58.031 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:58.031 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:58.045 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-13T10:11:58.062 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:58.055+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:58.066 INFO:tasks.rados.rados.0.smithi195.stdout:1646: expect (ObjNum 280 snap 0 seq_num 280) 2024-09-13T10:11:58.095 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:11:58.095 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:11:58.114 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:11:58.114 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:11:58.135 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:11:58.135 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:11:58.156 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:11:58.156 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:11:58.162 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:11:58.157+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:11:58.164 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:58.164 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:58.169 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:58.172 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:11:58.172 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:11:58.178 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-13T10:11:58.191 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:11:58.192 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:11:58.212 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:11:58.213 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:11:58.224 INFO:tasks.rados.rados.0.smithi195.stdout:1645: expect (ObjNum 334 snap 0 seq_num 334) 2024-09-13T10:11:58.232 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:11:58.233 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:11:58.242 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2280285 2024-09-13T10:11:58.252 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:11:58.253 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:11:58.263 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:11:58.262+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:11:58.266 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:58.272 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:11:58.272 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:11:58.293 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:11:58.293 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:11:58.301 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:11:58.301 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:11:58.314 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:11:58.315 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:11:58.316 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-13T10:11:58.320 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 688046 2024-09-13T10:11:58.336 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:11:58.336 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:11:58.357 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:11:58.357 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:11:58.363 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:58.358+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:58.364 INFO:tasks.rados.rados.0.smithi195.stdout:1651: expect (ObjNum 303 snap 0 seq_num 303) 2024-09-13T10:11:58.379 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:11:58.379 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:11:58.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:11:58.375+0000 7f1ec967c640 0 -- 172.21.15.73:0/1173210245 >> v2:172.21.15.195:6805/848686366 conn(0x7f1ec4224770 msgr2=0x7f1ec4226b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:11:58.403 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:11:58.403 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:11:58.413 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2913303 2024-09-13T10:11:58.441 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:58.441 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:11:58.441 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:11:58.442 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:11:58.442 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:58.456 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-13T10:11:58.499 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:58.495+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:58.541 INFO:tasks.rados.rados.0.smithi195.stdout:1650: expect (ObjNum 260 snap 0 seq_num 260) 2024-09-13T10:11:58.565 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:58.559+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:58.574 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:11:58.574 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:11:58.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:11:58.574 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:11:58.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.575 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:203 3.1 3:b033e801:::smithi19530714-4 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:58.575 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:11:58.575 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.580841192999998, 2024-09-13T10:11:58.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035547764000000003, 2024-09-13T10:11:58.575 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.575 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.575 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:58.575 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.575 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.575 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.575 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 203 2024-09-13T10:11:58.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.576 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:11:58.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:11:58.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991467+0000", 2024-09-13T10:11:58.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5030000000000002e-06 2024-09-13T10:11:58.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991483+0000", 2024-09-13T10:11:58.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5503e-05 2024-09-13T10:11:58.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991522+0000", 2024-09-13T10:11:58.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9399e-05 2024-09-13T10:11:58.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991544+0000", 2024-09-13T10:11:58.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2408999999999998e-05 2024-09-13T10:11:58.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991646+0000", 2024-09-13T10:11:58.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101798 2024-09-13T10:11:58.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991818+0000", 2024-09-13T10:11:58.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017204300000000001 2024-09-13T10:11:58.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:58.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.001618+0000", 2024-09-13T10:11:58.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097993490000000006 2024-09-13T10:11:58.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:58.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026785+0000", 2024-09-13T10:11:58.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025167446 2024-09-13T10:11:58.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026856+0000", 2024-09-13T10:11:58.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0575999999999996e-05 2024-09-13T10:11:58.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026915+0000", 2024-09-13T10:11:58.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9258000000000001e-05 2024-09-13T10:11:58.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:58.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026964+0000", 2024-09-13T10:11:58.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000003e-05 2024-09-13T10:11:58.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027011+0000", 2024-09-13T10:11:58.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7200000000000002e-05 2024-09-13T10:11:58.581 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.581 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.581 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.582 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:204 3.1 3:babcb876:::smithi19530714-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:58.582 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:11:58.582 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.580716676999998, 2024-09-13T10:11:58.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.22268644700000001, 2024-09-13T10:11:58.582 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.582 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.582 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:58.582 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.582 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.582 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.582 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 204 2024-09-13T10:11:58.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.583 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:11:58.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:11:58.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991589+0000", 2024-09-13T10:11:58.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3859999999999999e-06 2024-09-13T10:11:58.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991598+0000", 2024-09-13T10:11:58.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7669999999999992e-06 2024-09-13T10:11:58.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991616+0000", 2024-09-13T10:11:58.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7589e-05 2024-09-13T10:11:58.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991631+0000", 2024-09-13T10:11:58.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.47e-05 2024-09-13T10:11:58.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992050+0000", 2024-09-13T10:11:58.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041913399999999999 2024-09-13T10:11:58.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992229+0000", 2024-09-13T10:11:58.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017956499999999999 2024-09-13T10:11:58.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:58.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.203594+0000", 2024-09-13T10:11:58.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.211364303 2024-09-13T10:11:58.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:58.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.208526+0000", 2024-09-13T10:11:58.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049325020000000001 2024-09-13T10:11:58.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214019+0000", 2024-09-13T10:11:58.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054933270000000001 2024-09-13T10:11:58.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214108+0000", 2024-09-13T10:11:58.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8517000000000006e-05 2024-09-13T10:11:58.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:58.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214202+0000", 2024-09-13T10:11:58.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3763000000000003e-05 2024-09-13T10:11:58.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214275+0000", 2024-09-13T10:11:58.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2893999999999997e-05 2024-09-13T10:11:58.588 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.588 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.588 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.588 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:205 3.1 3:aec11a92:::smithi19530714-6 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:58.588 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:11:58.588 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.580641022999998, 2024-09-13T10:11:58.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056950259000000003, 2024-09-13T10:11:58.588 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.588 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.588 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:58.588 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.589 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.589 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.589 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 205 2024-09-13T10:11:58.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.589 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:11:58.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:11:58.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991665+0000", 2024-09-13T10:11:58.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1400000000000001e-06 2024-09-13T10:11:58.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991673+0000", 2024-09-13T10:11:58.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5730000000000008e-06 2024-09-13T10:11:58.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991689+0000", 2024-09-13T10:11:58.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5956000000000001e-05 2024-09-13T10:11:58.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991705+0000", 2024-09-13T10:11:58.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5539000000000001e-05 2024-09-13T10:11:58.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992393+0000", 2024-09-13T10:11:58.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068836600000000004 2024-09-13T10:11:58.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992509+0000", 2024-09-13T10:11:58.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115563 2024-09-13T10:11:58.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:58.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.011894+0000", 2024-09-13T10:11:58.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019385594999999999 2024-09-13T10:11:58.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:58.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027029+0000", 2024-09-13T10:11:58.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015134664000000001 2024-09-13T10:11:58.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.031277+0000", 2024-09-13T10:11:58.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004247804 2024-09-13T10:11:58.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048542+0000", 2024-09-13T10:11:58.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017265401999999999 2024-09-13T10:11:58.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:58.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048579+0000", 2024-09-13T10:11:58.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6238000000000001e-05 2024-09-13T10:11:58.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048614+0000", 2024-09-13T10:11:58.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5419000000000001e-05 2024-09-13T10:11:58.594 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.594 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.594 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.594 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:209 3.1 3:9798d58d:::smithi19530714-10 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:58.595 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:11:58.595 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.580512807999995, 2024-09-13T10:11:58.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.069824676000000002, 2024-09-13T10:11:58.595 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.595 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.595 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:58.595 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.595 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.595 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.595 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 209 2024-09-13T10:11:58.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.595 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:11:58.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:11:58.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991793+0000", 2024-09-13T10:11:58.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.057e-06 2024-09-13T10:11:58.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991803+0000", 2024-09-13T10:11:58.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7860000000000008e-06 2024-09-13T10:11:58.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991817+0000", 2024-09-13T10:11:58.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4066000000000001e-05 2024-09-13T10:11:58.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991828+0000", 2024-09-13T10:11:58.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1097000000000001e-05 2024-09-13T10:11:58.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992666+0000", 2024-09-13T10:11:58.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083755799999999997 2024-09-13T10:11:58.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992792+0000", 2024-09-13T10:11:58.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000126334 2024-09-13T10:11:58.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:58.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027675+0000", 2024-09-13T10:11:58.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034883299 2024-09-13T10:11:58.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:58.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.035648+0000", 2024-09-13T10:11:58.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007972972 2024-09-13T10:11:58.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.045764+0000", 2024-09-13T10:11:58.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010115832 2024-09-13T10:11:58.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061460+0000", 2024-09-13T10:11:58.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015695563999999999 2024-09-13T10:11:58.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:58.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061544+0000", 2024-09-13T10:11:58.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4866999999999999e-05 2024-09-13T10:11:58.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061617+0000", 2024-09-13T10:11:58.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2243999999999995e-05 2024-09-13T10:11:58.601 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.601 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.601 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.601 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:213 3.1 3:accf4e56:::smithi19530714-14 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:58.601 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:11:58.601 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.579738864999996, 2024-09-13T10:11:58.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.060593045999999998, 2024-09-13T10:11:58.601 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.601 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.601 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:58.601 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.602 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.602 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 213 2024-09-13T10:11:58.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.602 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:11:58.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:11:58.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992568+0000", 2024-09-13T10:11:58.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3640000000000002e-06 2024-09-13T10:11:58.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992580+0000", 2024-09-13T10:11:58.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1796e-05 2024-09-13T10:11:58.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992662+0000", 2024-09-13T10:11:58.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1814e-05 2024-09-13T10:11:58.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992683+0000", 2024-09-13T10:11:58.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0746000000000001e-05 2024-09-13T10:11:58.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.993024+0000", 2024-09-13T10:11:58.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034141299999999999 2024-09-13T10:11:58.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.993182+0000", 2024-09-13T10:11:58.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015817699999999999 2024-09-13T10:11:58.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:58.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.019852+0000", 2024-09-13T10:11:58.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026669470000000001 2024-09-13T10:11:58.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.032551+0000", 2024-09-13T10:11:58.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012699729999999999 2024-09-13T10:11:58.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:58.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.034351+0000", 2024-09-13T10:11:58.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017998269999999999 2024-09-13T10:11:58.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053106+0000", 2024-09-13T10:11:58.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018755236000000002 2024-09-13T10:11:58.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:58.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053131+0000", 2024-09-13T10:11:58.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4729e-05 2024-09-13T10:11:58.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053159+0000", 2024-09-13T10:11:58.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7744000000000001e-05 2024-09-13T10:11:58.607 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.607 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.608 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.608 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:219 3.1 3:87bfc7b1:::smithi19530714-20 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:58.608 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:11:58.608 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.539289792000005, 2024-09-13T10:11:58.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.037494069999999997, 2024-09-13T10:11:58.608 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.608 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.608 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:58.608 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.608 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.608 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.609 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 219 2024-09-13T10:11:58.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.609 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:11:58.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:11:58.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033019+0000", 2024-09-13T10:11:58.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.726e-06 2024-09-13T10:11:58.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033036+0000", 2024-09-13T10:11:58.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7329999999999998e-05 2024-09-13T10:11:58.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033060+0000", 2024-09-13T10:11:58.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4286000000000001e-05 2024-09-13T10:11:58.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033079+0000", 2024-09-13T10:11:58.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8872000000000002e-05 2024-09-13T10:11:58.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033155+0000", 2024-09-13T10:11:58.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6118999999999994e-05 2024-09-13T10:11:58.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033297+0000", 2024-09-13T10:11:58.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014154700000000001 2024-09-13T10:11:58.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:58.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.041666+0000", 2024-09-13T10:11:58.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083696180000000005 2024-09-13T10:11:58.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:58.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046621+0000", 2024-09-13T10:11:58.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049545759999999996 2024-09-13T10:11:58.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.052333+0000", 2024-09-13T10:11:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057120160000000003 2024-09-13T10:11:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070340+0000", 2024-09-13T10:11:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018006859 2024-09-13T10:11:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:58.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070447+0000", 2024-09-13T10:11:58.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106637 2024-09-13T10:11:58.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070509+0000", 2024-09-13T10:11:58.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2483999999999993e-05 2024-09-13T10:11:58.614 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.614 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.614 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.614 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:222 3.1 3:bdac4e61:::smithi19530714-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:58.614 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:11:58.614 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.525601817999998, 2024-09-13T10:11:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041434034000000002, 2024-09-13T10:11:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 222 2024-09-13T10:11:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:11:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:11:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046706+0000", 2024-09-13T10:11:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0089999999999999e-06 2024-09-13T10:11:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046719+0000", 2024-09-13T10:11:58.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2565000000000001e-05 2024-09-13T10:11:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046740+0000", 2024-09-13T10:11:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0979e-05 2024-09-13T10:11:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046752+0000", 2024-09-13T10:11:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2917e-05 2024-09-13T10:11:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046803+0000", 2024-09-13T10:11:58.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0417000000000002e-05 2024-09-13T10:11:58.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046908+0000", 2024-09-13T10:11:58.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105039 2024-09-13T10:11:58.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:58.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.066097+0000", 2024-09-13T10:11:58.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019189277000000001 2024-09-13T10:11:58.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:58.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070969+0000", 2024-09-13T10:11:58.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048721429999999998 2024-09-13T10:11:58.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.087852+0000", 2024-09-13T10:11:58.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016882489000000001 2024-09-13T10:11:58.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.087942+0000", 2024-09-13T10:11:58.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9738e-05 2024-09-13T10:11:58.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:58.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088076+0000", 2024-09-13T10:11:58.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013400700000000001 2024-09-13T10:11:58.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088137+0000", 2024-09-13T10:11:58.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1453999999999998e-05 2024-09-13T10:11:58.620 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.620 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.620 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.621 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:223 3.1 3:8759f4e0:::smithi19530714-24 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:58.621 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:11:58.621 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.512924948999995, 2024-09-13T10:11:58.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053691092000000003, 2024-09-13T10:11:58.626 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.626 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.626 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:58.626 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.626 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.626 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.626 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 223 2024-09-13T10:11:58.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.626 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:11:58.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:11:58.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059383+0000", 2024-09-13T10:11:58.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.847e-06 2024-09-13T10:11:58.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059399+0000", 2024-09-13T10:11:58.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6739e-05 2024-09-13T10:11:58.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059433+0000", 2024-09-13T10:11:58.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3946000000000003e-05 2024-09-13T10:11:58.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059456+0000", 2024-09-13T10:11:58.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2163000000000001e-05 2024-09-13T10:11:58.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059621+0000", 2024-09-13T10:11:58.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016546700000000001 2024-09-13T10:11:58.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.060003+0000", 2024-09-13T10:11:58.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000381982 2024-09-13T10:11:58.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:58.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.085339+0000", 2024-09-13T10:11:58.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025335610000000001 2024-09-13T10:11:58.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:58.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.089896+0000", 2024-09-13T10:11:58.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045575930000000004 2024-09-13T10:11:58.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.102515+0000", 2024-09-13T10:11:58.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012618664999999999 2024-09-13T10:11:58.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113009+0000", 2024-09-13T10:11:58.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010493784000000001 2024-09-13T10:11:58.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:58.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113044+0000", 2024-09-13T10:11:58.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5543999999999997e-05 2024-09-13T10:11:58.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113071+0000", 2024-09-13T10:11:58.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6752000000000001e-05 2024-09-13T10:11:58.631 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.631 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.631 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.632 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:231 3.1 3:9d247594:::smithi19530714-32 low_tier:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:58.632 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.065612+0000", 2024-09-13T10:11:58.632 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.506692290999993, 2024-09-13T10:11:58.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.031259105000000002, 2024-09-13T10:11:58.632 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.632 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.632 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:58.632 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.632 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.632 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.632 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 231 2024-09-13T10:11:58.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.632 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065612+0000", 2024-09-13T10:11:58.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065612+0000", 2024-09-13T10:11:58.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065614+0000", 2024-09-13T10:11:58.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7370000000000001e-06 2024-09-13T10:11:58.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065622+0000", 2024-09-13T10:11:58.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1110000000000008e-06 2024-09-13T10:11:58.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065640+0000", 2024-09-13T10:11:58.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7499999999999998e-05 2024-09-13T10:11:58.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.065655+0000", 2024-09-13T10:11:58.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5021999999999999e-05 2024-09-13T10:11:58.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069879+0000", 2024-09-13T10:11:58.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042246360000000004 2024-09-13T10:11:58.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070080+0000", 2024-09-13T10:11:58.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020099199999999999 2024-09-13T10:11:58.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:58.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.075971+0000", 2024-09-13T10:11:58.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058906030000000003 2024-09-13T10:11:58.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:58.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088156+0000", 2024-09-13T10:11:58.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012184868999999999 2024-09-13T10:11:58.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.095237+0000", 2024-09-13T10:11:58.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0070814279999999999 2024-09-13T10:11:58.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.096766+0000", 2024-09-13T10:11:58.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001529007 2024-09-13T10:11:58.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:58.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.096823+0000", 2024-09-13T10:11:58.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6789999999999997e-05 2024-09-13T10:11:58.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.096872+0000", 2024-09-13T10:11:58.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8409999999999999e-05 2024-09-13T10:11:58.638 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.638 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.638 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.638 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:234 3.1 3:bdce9caa:::smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:58.638 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:11:58.638 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.503205687999994, 2024-09-13T10:11:58.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043032263000000001, 2024-09-13T10:11:58.638 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.638 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.638 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:58.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.639 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 234 2024-09-13T10:11:58.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.639 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:11:58.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:11:58.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069102+0000", 2024-09-13T10:11:58.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.543e-06 2024-09-13T10:11:58.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069112+0000", 2024-09-13T10:11:58.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0091e-05 2024-09-13T10:11:58.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069133+0000", 2024-09-13T10:11:58.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1188000000000001e-05 2024-09-13T10:11:58.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069143+0000", 2024-09-13T10:11:58.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0456e-05 2024-09-13T10:11:58.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070588+0000", 2024-09-13T10:11:58.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001445046 2024-09-13T10:11:58.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070801+0000", 2024-09-13T10:11:58.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000212151 2024-09-13T10:11:58.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:58.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.082362+0000", 2024-09-13T10:11:58.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561781 2024-09-13T10:11:58.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:58.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088161+0000", 2024-09-13T10:11:58.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057986119999999999 2024-09-13T10:11:58.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.101972+0000", 2024-09-13T10:11:58.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013811498 2024-09-13T10:11:58.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112028+0000", 2024-09-13T10:11:58.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010055530999999999 2024-09-13T10:11:58.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:58.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112082+0000", 2024-09-13T10:11:58.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3974999999999997e-05 2024-09-13T10:11:58.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112131+0000", 2024-09-13T10:11:58.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9391000000000002e-05 2024-09-13T10:11:58.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.645 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.645 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:243 3.1 3:a4dd4282:::smithi19530714-44 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:58.645 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:11:58.645 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.474997982999994, 2024-09-13T10:11:58.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036759694000000002, 2024-09-13T10:11:58.645 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.645 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.645 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:58.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.646 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.646 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 243 2024-09-13T10:11:58.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.646 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:11:58.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:11:58.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097308+0000", 2024-09-13T10:11:58.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2099999999999995e-07 2024-09-13T10:11:58.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097314+0000", 2024-09-13T10:11:58.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9989999999999997e-06 2024-09-13T10:11:58.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097326+0000", 2024-09-13T10:11:58.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1787000000000001e-05 2024-09-13T10:11:58.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097332+0000", 2024-09-13T10:11:58.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6950000000000003e-06 2024-09-13T10:11:58.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097378+0000", 2024-09-13T10:11:58.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5744000000000001e-05 2024-09-13T10:11:58.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097489+0000", 2024-09-13T10:11:58.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111352 2024-09-13T10:11:58.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:58.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.118571+0000", 2024-09-13T10:11:58.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021081249999999999 2024-09-13T10:11:58.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:58.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.128494+0000", 2024-09-13T10:11:58.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099237270000000002 2024-09-13T10:11:58.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.128775+0000", 2024-09-13T10:11:58.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028027299999999999 2024-09-13T10:11:58.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.133963+0000", 2024-09-13T10:11:58.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005188191 2024-09-13T10:11:58.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:58.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134015+0000", 2024-09-13T10:11:58.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2438000000000002e-05 2024-09-13T10:11:58.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134066+0000", 2024-09-13T10:11:58.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1316999999999997e-05 2024-09-13T10:11:58.652 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.652 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.652 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.652 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:248 3.1 3:b0d90ce0:::smithi19530714-49 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 18] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:58.652 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.114744+0000", 2024-09-13T10:11:58.652 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.457561081999998, 2024-09-13T10:11:58.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028886754000000001, 2024-09-13T10:11:58.652 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.652 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.653 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:58.653 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.653 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.653 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.653 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 248 2024-09-13T10:11:58.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.653 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114744+0000", 2024-09-13T10:11:58.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114744+0000", 2024-09-13T10:11:58.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114745+0000", 2024-09-13T10:11:58.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.55e-06 2024-09-13T10:11:58.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114752+0000", 2024-09-13T10:11:58.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1670000000000002e-06 2024-09-13T10:11:58.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114768+0000", 2024-09-13T10:11:58.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5471000000000001e-05 2024-09-13T10:11:58.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114775+0000", 2024-09-13T10:11:58.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3799999999999996e-06 2024-09-13T10:11:58.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114826+0000", 2024-09-13T10:11:58.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0909999999999999e-05 2024-09-13T10:11:58.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.114981+0000", 2024-09-13T10:11:58.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015455699999999999 2024-09-13T10:11:58.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:11:58.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.125745+0000", 2024-09-13T10:11:58.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010764404 2024-09-13T10:11:58.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:58.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.129548+0000", 2024-09-13T10:11:58.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038027019999999998 2024-09-13T10:11:58.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.140602+0000", 2024-09-13T10:11:58.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011054082 2024-09-13T10:11:58.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.143523+0000", 2024-09-13T10:11:58.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002920962 2024-09-13T10:11:58.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:58.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.143586+0000", 2024-09-13T10:11:58.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2866000000000004e-05 2024-09-13T10:11:58.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.143630+0000", 2024-09-13T10:11:58.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4703000000000002e-05 2024-09-13T10:11:58.728 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.728 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.728 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.728 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:292 4.f 4:f6a0e8ce:::smithi19530714-43:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:58.728 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.257863+0000", 2024-09-13T10:11:58.728 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.314441861999995, 2024-09-13T10:11:58.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022747474, 2024-09-13T10:11:58.729 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.729 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.729 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:58.729 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.729 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.729 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.729 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 292 2024-09-13T10:11:58.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.729 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257863+0000", 2024-09-13T10:11:58.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257863+0000", 2024-09-13T10:11:58.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257865+0000", 2024-09-13T10:11:58.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.883e-06 2024-09-13T10:11:58.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257874+0000", 2024-09-13T10:11:58.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0699999999999996e-06 2024-09-13T10:11:58.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257894+0000", 2024-09-13T10:11:58.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0349000000000001e-05 2024-09-13T10:11:58.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257925+0000", 2024-09-13T10:11:58.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1092e-05 2024-09-13T10:11:58.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258040+0000", 2024-09-13T10:11:58.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114474 2024-09-13T10:11:58.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258126+0000", 2024-09-13T10:11:58.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6479999999999999e-05 2024-09-13T10:11:58.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 2,7", 2024-09-13T10:11:58.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.273691+0000", 2024-09-13T10:11:58.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015564626999999999 2024-09-13T10:11:58.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:58.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.275091+0000", 2024-09-13T10:11:58.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001400079 2024-09-13T10:11:58.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.276990+0000", 2024-09-13T10:11:58.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018994179999999999 2024-09-13T10:11:58.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.280333+0000", 2024-09-13T10:11:58.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033424320000000002 2024-09-13T10:11:58.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:58.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.280486+0000", 2024-09-13T10:11:58.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015310500000000001 2024-09-13T10:11:58.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.280610+0000", 2024-09-13T10:11:58.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012446500000000001 2024-09-13T10:11:58.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.735 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.735 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1272 4.f 4:f7ad6ef6:::smithi19530714-1:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e45)", 2024-09-13T10:11:58.735 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:11:58.735 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.696769807999999, 2024-09-13T10:11:58.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.050041173000000001, 2024-09-13T10:11:58.735 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.735 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.735 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:58.735 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.736 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.736 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.736 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1272 2024-09-13T10:11:58.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.736 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:11:58.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:11:58.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875537+0000", 2024-09-13T10:11:58.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2689999999999998e-06 2024-09-13T10:11:58.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875548+0000", 2024-09-13T10:11:58.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0291e-05 2024-09-13T10:11:58.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875567+0000", 2024-09-13T10:11:58.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9284999999999999e-05 2024-09-13T10:11:58.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875578+0000", 2024-09-13T10:11:58.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1256e-05 2024-09-13T10:11:58.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875763+0000", 2024-09-13T10:11:58.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000185396 2024-09-13T10:11:58.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875908+0000", 2024-09-13T10:11:58.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014412299999999999 2024-09-13T10:11:58.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.925576+0000", 2024-09-13T10:11:58.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049668552999999997 2024-09-13T10:11:58.739 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.739 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.739 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.740 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2719 4.f 4:f7ad6ef6:::smithi19530714-1:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:11:58.740 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:29.386743+0000", 2024-09-13T10:11:58.740 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.185561724999999, 2024-09-13T10:11:58.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047467252000000001, 2024-09-13T10:11:58.740 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.740 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.740 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:58.740 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.740 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.740 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.740 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2719 2024-09-13T10:11:58.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.741 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386743+0000", 2024-09-13T10:11:58.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386743+0000", 2024-09-13T10:11:58.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386746+0000", 2024-09-13T10:11:58.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2729999999999998e-06 2024-09-13T10:11:58.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386758+0000", 2024-09-13T10:11:58.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1703e-05 2024-09-13T10:11:58.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386789+0000", 2024-09-13T10:11:58.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0815000000000002e-05 2024-09-13T10:11:58.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386806+0000", 2024-09-13T10:11:58.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7595999999999999e-05 2024-09-13T10:11:58.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386996+0000", 2024-09-13T10:11:58.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000189521 2024-09-13T10:11:58.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.387151+0000", 2024-09-13T10:11:58.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015501600000000001 2024-09-13T10:11:58.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.434210+0000", 2024-09-13T10:11:58.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047059327999999997 2024-09-13T10:11:58.744 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.744 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.744 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.744 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3234 4.12 4:4ac406d9:::smithi19530714-26:head [write 507644~794122] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:58.744 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.075276+0000", 2024-09-13T10:11:58.744 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.497028665, 2024-09-13T10:11:58.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027827800999999999, 2024-09-13T10:11:58.745 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.745 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.745 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:58.745 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.745 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.745 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.745 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3234 2024-09-13T10:11:58.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.745 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.075276+0000", 2024-09-13T10:11:58.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.075276+0000", 2024-09-13T10:11:58.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.075280+0000", 2024-09-13T10:11:58.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5470000000000001e-06 2024-09-13T10:11:58.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076401+0000", 2024-09-13T10:11:58.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001121665 2024-09-13T10:11:58.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076434+0000", 2024-09-13T10:11:58.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2889000000000003e-05 2024-09-13T10:11:58.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076460+0000", 2024-09-13T10:11:58.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5415999999999999e-05 2024-09-13T10:11:58.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076602+0000", 2024-09-13T10:11:58.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000142201 2024-09-13T10:11:58.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:58.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076628+0000", 2024-09-13T10:11:58.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5829e-05 2024-09-13T10:11:58.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.084126+0000", 2024-09-13T10:11:58.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0074985110000000002 2024-09-13T10:11:58.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.084459+0000", 2024-09-13T10:11:58.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033248499999999999 2024-09-13T10:11:58.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3,9", 2024-09-13T10:11:58.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.084875+0000", 2024-09-13T10:11:58.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000415933 2024-09-13T10:11:58.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.095806+0000", 2024-09-13T10:11:58.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010931738 2024-09-13T10:11:58.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:58.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096485+0000", 2024-09-13T10:11:58.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000678427 2024-09-13T10:11:58.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.102831+0000", 2024-09-13T10:11:58.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063463490000000003 2024-09-13T10:11:58.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:58.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.102965+0000", 2024-09-13T10:11:58.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013418600000000001 2024-09-13T10:11:58.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.103104+0000", 2024-09-13T10:11:58.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013862499999999999 2024-09-13T10:11:58.752 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.752 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.752 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.752 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3235 4.12 4:4ac406d9:::smithi19530714-26:head [write 1920742~623085] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:58.752 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.076518+0000", 2024-09-13T10:11:58.752 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.495787274, 2024-09-13T10:11:58.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027319461999999999, 2024-09-13T10:11:58.752 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.752 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.752 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:58.753 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.753 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.753 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.753 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3235 2024-09-13T10:11:58.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.753 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076518+0000", 2024-09-13T10:11:58.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076518+0000", 2024-09-13T10:11:58.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076520+0000", 2024-09-13T10:11:58.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1069999999999999e-06 2024-09-13T10:11:58.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077261+0000", 2024-09-13T10:11:58.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074164200000000004 2024-09-13T10:11:58.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077293+0000", 2024-09-13T10:11:58.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1665000000000003e-05 2024-09-13T10:11:58.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077314+0000", 2024-09-13T10:11:58.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0729000000000001e-05 2024-09-13T10:11:58.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077453+0000", 2024-09-13T10:11:58.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000139009 2024-09-13T10:11:58.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:58.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077474+0000", 2024-09-13T10:11:58.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.101e-05 2024-09-13T10:11:58.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.089297+0000", 2024-09-13T10:11:58.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011823139999999999 2024-09-13T10:11:58.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.089526+0000", 2024-09-13T10:11:58.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022905899999999999 2024-09-13T10:11:58.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3,9", 2024-09-13T10:11:58.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.089959+0000", 2024-09-13T10:11:58.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043306200000000001 2024-09-13T10:11:58.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:58.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096499+0000", 2024-09-13T10:11:58.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065402569999999998 2024-09-13T10:11:58.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.098041+0000", 2024-09-13T10:11:58.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015419940000000001 2024-09-13T10:11:58.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.103618+0000", 2024-09-13T10:11:58.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055764489999999998 2024-09-13T10:11:58.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:58.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.103711+0000", 2024-09-13T10:11:58.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2923999999999993e-05 2024-09-13T10:11:58.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.103837+0000", 2024-09-13T10:11:58.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000126415 2024-09-13T10:11:58.759 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.759 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.759 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.759 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3236 4.12 4:4ac406d9:::smithi19530714-26:head [setxattr _header (56) in=7b,truncate 2907644] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:58.759 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.077362+0000", 2024-09-13T10:11:58.759 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.494942759000001, 2024-09-13T10:11:58.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027193532999999999, 2024-09-13T10:11:58.759 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.760 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.760 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:58.760 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.760 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.760 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.760 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3236 2024-09-13T10:11:58.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.760 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077362+0000", 2024-09-13T10:11:58.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077362+0000", 2024-09-13T10:11:58.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077364+0000", 2024-09-13T10:11:58.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.99e-06 2024-09-13T10:11:58.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077375+0000", 2024-09-13T10:11:58.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0563e-05 2024-09-13T10:11:58.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077396+0000", 2024-09-13T10:11:58.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1659e-05 2024-09-13T10:11:58.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077413+0000", 2024-09-13T10:11:58.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6399999999999999e-05 2024-09-13T10:11:58.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077554+0000", 2024-09-13T10:11:58.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014102799999999999 2024-09-13T10:11:58.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:58.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077570+0000", 2024-09-13T10:11:58.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6495999999999999e-05 2024-09-13T10:11:58.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.094403+0000", 2024-09-13T10:11:58.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016832973000000001 2024-09-13T10:11:58.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.094627+0000", 2024-09-13T10:11:58.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000224162 2024-09-13T10:11:58.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3,9", 2024-09-13T10:11:58.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.095016+0000", 2024-09-13T10:11:58.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000388234 2024-09-13T10:11:58.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:11:58.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096829+0000", 2024-09-13T10:11:58.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001813552 2024-09-13T10:11:58.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.098138+0000", 2024-09-13T10:11:58.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013090529999999999 2024-09-13T10:11:58.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:58.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.104375+0000", 2024-09-13T10:11:58.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0062366490000000004 2024-09-13T10:11:58.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:11:58.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.104464+0000", 2024-09-13T10:11:58.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8817e-05 2024-09-13T10:11:58.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.104556+0000", 2024-09-13T10:11:58.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1957000000000001e-05 2024-09-13T10:11:58.766 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.766 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.766 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.767 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3237 4.12 4:4ac406d9:::smithi19530714-26:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:58.767 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.077438+0000", 2024-09-13T10:11:58.767 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.494866937999999, 2024-09-13T10:11:58.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.027524142000000001, 2024-09-13T10:11:58.767 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.767 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.767 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:58.767 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.767 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.767 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.767 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3237 2024-09-13T10:11:58.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.767 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077438+0000", 2024-09-13T10:11:58.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077438+0000", 2024-09-13T10:11:58.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077439+0000", 2024-09-13T10:11:58.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5269999999999999e-06 2024-09-13T10:11:58.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077451+0000", 2024-09-13T10:11:58.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1846e-05 2024-09-13T10:11:58.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077470+0000", 2024-09-13T10:11:58.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9216e-05 2024-09-13T10:11:58.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077490+0000", 2024-09-13T10:11:58.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9256e-05 2024-09-13T10:11:58.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077643+0000", 2024-09-13T10:11:58.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015328799999999999 2024-09-13T10:11:58.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:11:58.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077660+0000", 2024-09-13T10:11:58.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7235999999999999e-05 2024-09-13T10:11:58.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.095924+0000", 2024-09-13T10:11:58.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018264256999999999 2024-09-13T10:11:58.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:58.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096130+0000", 2024-09-13T10:11:58.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020595300000000001 2024-09-13T10:11:58.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.104656+0000", 2024-09-13T10:11:58.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0085253359999999997 2024-09-13T10:11:58.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.104791+0000", 2024-09-13T10:11:58.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013517600000000001 2024-09-13T10:11:58.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.104962+0000", 2024-09-13T10:11:58.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000171051 2024-09-13T10:11:58.772 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.772 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.772 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.772 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3275 4.6 4:6150ba38:::smithi19530714-6:head [read 0~3848803,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:58.773 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.083653+0000", 2024-09-13T10:11:58.773 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.488651698, 2024-09-13T10:11:58.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.022619937999999999, 2024-09-13T10:11:58.773 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:11:58.773 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:11:58.773 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:11:58.773 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:11:58.773 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:11:58.773 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:58.773 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3275 2024-09-13T10:11:58.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.773 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:11:58.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:11:58.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083653+0000", 2024-09-13T10:11:58.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:11:58.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083653+0000", 2024-09-13T10:11:58.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:11:58.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:11:58.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083655+0000", 2024-09-13T10:11:58.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.533e-06 2024-09-13T10:11:58.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:11:58.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083664+0000", 2024-09-13T10:11:58.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1770000000000004e-06 2024-09-13T10:11:58.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:11:58.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083682+0000", 2024-09-13T10:11:58.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7722000000000001e-05 2024-09-13T10:11:58.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:11:58.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083703+0000", 2024-09-13T10:11:58.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1823000000000001e-05 2024-09-13T10:11:58.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:11:58.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096390+0000", 2024-09-13T10:11:58.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012686606 2024-09-13T10:11:58.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:11:58.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096610+0000", 2024-09-13T10:11:58.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021990900000000001 2024-09-13T10:11:58.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:11:58.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:11:58.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:11:58.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.106273+0000", 2024-09-13T10:11:58.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0096631679999999998 2024-09-13T10:11:58.777 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.777 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.777 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.777 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:11:58.777 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:11:58.777 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:11:58.777 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T10:11:58.779 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:58.779 INFO:tasks.rados.rados.0.smithi195.stdout:1652: expect (ObjNum 315 snap 0 seq_num 315) 2024-09-13T10:11:58.779 INFO:tasks.rados.rados.0.smithi195.stdout:1649: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:11:58.779 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:58.780 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T10:11:58.780 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:58.775+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:58.824 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:58.858 INFO:tasks.rados.rados.0.smithi195.stdout:1641: done (10 left) 2024-09-13T10:11:58.858 INFO:tasks.rados.rados.0.smithi195.stdout:1642: done (9 left) 2024-09-13T10:11:58.859 INFO:tasks.rados.rados.0.smithi195.stdout:1644: done (8 left) 2024-09-13T10:11:58.859 INFO:tasks.rados.rados.0.smithi195.stdout:1645: done (7 left) 2024-09-13T10:11:58.859 INFO:tasks.rados.rados.0.smithi195.stdout:1646: done (6 left) 2024-09-13T10:11:58.859 INFO:tasks.rados.rados.0.smithi195.stdout:1648: done (5 left) 2024-09-13T10:11:58.859 INFO:tasks.rados.rados.0.smithi195.stdout:1649: done (4 left) 2024-09-13T10:11:58.859 INFO:tasks.rados.rados.0.smithi195.stdout:1650: done (3 left) 2024-09-13T10:11:58.859 INFO:tasks.rados.rados.0.smithi195.stdout:1651: done (2 left) 2024-09-13T10:11:58.859 INFO:tasks.rados.rados.0.smithi195.stdout:1652: done (1 left) 2024-09-13T10:11:58.859 INFO:tasks.rados.rados.0.smithi195.stdout:1653: done (0 left) 2024-09-13T10:11:58.859 INFO:tasks.rados.rados.0.smithi195.stdout:1654: set_chunk oid 6 target oid 2024-09-13T10:11:58.875 INFO:tasks.rados.rados.0.smithi195.stdout:1654: set_chunk oid 6 offset: 1327104 length: 694272 target oid 51aed524c83f90bc0fd039e0e1f85f784897bc362bf06371b9184446a01dbfa3 offset: 0 snap -1 2024-09-13T10:11:58.875 INFO:tasks.rados.rados.0.smithi195.stdout:1655: set_chunk oid 4 target oid 2024-09-13T10:11:58.879 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:11:58.875+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:11:58.914 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:58.914 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:58.914 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:11:58.914 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:58.926 INFO:tasks.rados.rados.0.smithi195.stdout:1655: set_chunk oid 4 offset: 41984 length: 2900992 target oid a6332e51e016c24df8505825880ac4e703da29058ee795e9033d5e12bb724144 offset: 0 snap -1 2024-09-13T10:11:58.926 INFO:tasks.rados.rados.0.smithi195.stdout:1656: read oid 22 snap -1 2024-09-13T10:11:58.926 INFO:tasks.rados.rados.0.smithi195.stdout:1656: expect (ObjNum 341 snap 0 seq_num 341) 2024-09-13T10:11:58.926 INFO:tasks.rados.rados.0.smithi195.stdout:1657: set_chunk oid 9 target oid 2024-09-13T10:11:58.930 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T10:11:58.961 INFO:tasks.rados.rados.0.smithi195.stdout:1657: set_chunk oid 9 offset: 742912 length: 739840 target oid 738a31292ff419a743b884430bcb71525640f19f6cf5050bbbd9b5a0fd9045c9 offset: 0 snap -1 2024-09-13T10:11:58.961 INFO:tasks.rados.rados.0.smithi195.stdout:1658: read oid 35 snap -1 2024-09-13T10:11:58.961 INFO:tasks.rados.rados.0.smithi195.stdout:1658: expect (ObjNum 339 snap 0 seq_num 339) 2024-09-13T10:11:58.961 INFO:tasks.rados.rados.0.smithi195.stdout:1659: read oid 5 snap -1 2024-09-13T10:11:58.961 INFO:tasks.rados.rados.0.smithi195.stdout:1659: expect (ObjNum 322 snap 0 seq_num 322) 2024-09-13T10:11:58.962 INFO:tasks.rados.rados.0.smithi195.stdout:1660: tier_promote oid 28 2024-09-13T10:11:58.962 INFO:tasks.rados.rados.0.smithi195.stdout:1654: finishing set_chunk to oid 6 2024-09-13T10:11:58.962 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 80 (ObjNum 337 snap 0 seq_num 337) dirty exists 2024-09-13T10:11:58.962 INFO:tasks.rados.rados.0.smithi195.stdout:1655: finishing set_chunk to oid 4 2024-09-13T10:11:58.962 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 121 (ObjNum 352 snap 0 seq_num 352) dirty exists 2024-09-13T10:11:58.962 INFO:tasks.rados.rados.0.smithi195.stdout:1656: expect (ObjNum 341 snap 0 seq_num 341) 2024-09-13T10:11:58.980 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:11:58.975+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:11:59.005 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2657819 2024-09-13T10:11:59.029 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:59.058 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:59.058 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:59.058 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:11:59.058 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:11:59.058 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:59.072 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T10:11:59.079 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:59.081 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:59.077+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:59.116 INFO:tasks.rados.rados.0.smithi195.stdout:1657: finishing set_chunk to oid 9 2024-09-13T10:11:59.116 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 123 (ObjNum 259 snap 0 seq_num 259) dirty exists 2024-09-13T10:11:59.116 INFO:tasks.rados.rados.0.smithi195.stdout:1660: got (0) Success 2024-09-13T10:11:59.116 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 96 (ObjNum 180 snap 0 seq_num 180) dirty exists 2024-09-13T10:11:59.116 INFO:tasks.rados.rados.0.smithi195.stdout:1659: expect (ObjNum 322 snap 0 seq_num 322) 2024-09-13T10:11:59.123 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1018589 2024-09-13T10:11:59.182 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:59.175+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:59.188 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:59.199 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:59.199 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:11:59.199 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:11:59.199 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:11:59.199 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.199 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:259 4.1 4:85f1e656:::smithi19530714-10:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:59.200 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:11:59.200 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 67.980203001000007, 2024-09-13T10:11:59.200 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014693513, 2024-09-13T10:11:59.200 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.200 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.200 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.200 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.200 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.200 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.200 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 259 2024-09-13T10:11:59.200 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.200 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.201 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.201 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.201 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:11:59.201 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.201 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.201 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.201 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.201 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:11:59.201 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.201 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.201 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.201 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.201 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215171+0000", 2024-09-13T10:11:59.202 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.3329999999999998e-06 2024-09-13T10:11:59.202 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.202 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.202 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.202 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215190+0000", 2024-09-13T10:11:59.202 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8672999999999999e-05 2024-09-13T10:11:59.202 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.202 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.202 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.202 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215229+0000", 2024-09-13T10:11:59.202 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9076000000000003e-05 2024-09-13T10:11:59.203 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.203 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.203 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.203 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215287+0000", 2024-09-13T10:11:59.203 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8142e-05 2024-09-13T10:11:59.203 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.203 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.203 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.203 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215435+0000", 2024-09-13T10:11:59.203 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014800600000000001 2024-09-13T10:11:59.203 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.203 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.204 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.204 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215593+0000", 2024-09-13T10:11:59.204 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015864300000000001 2024-09-13T10:11:59.204 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.204 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.204 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:59.204 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226037+0000", 2024-09-13T10:11:59.204 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010443948 2024-09-13T10:11:59.204 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.204 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.204 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.204 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228312+0000", 2024-09-13T10:11:59.205 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022743009999999998 2024-09-13T10:11:59.205 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.205 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.205 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.205 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228708+0000", 2024-09-13T10:11:59.205 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039651999999999998 2024-09-13T10:11:59.205 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.205 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.205 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.205 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229625+0000", 2024-09-13T10:11:59.205 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00091692300000000004 2024-09-13T10:11:59.206 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.206 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.206 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.206 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229750+0000", 2024-09-13T10:11:59.206 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012532099999999999 2024-09-13T10:11:59.206 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.206 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.206 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.206 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229860+0000", 2024-09-13T10:11:59.206 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000109627 2024-09-13T10:11:59.206 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.206 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.206 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.207 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.207 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.207 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:260 4.3 4:c426503f:::smithi19530714-11:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:59.207 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:11:59.207 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 67.980035041999997, 2024-09-13T10:11:59.207 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041961871999999997, 2024-09-13T10:11:59.207 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.207 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.207 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.207 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.207 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.207 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.208 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 260 2024-09-13T10:11:59.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.208 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.208 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.208 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.208 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:11:59.208 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.208 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.208 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.208 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:11:59.208 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.209 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.209 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.209 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215336+0000", 2024-09-13T10:11:59.209 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9069999999999999e-06 2024-09-13T10:11:59.209 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.209 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.209 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.209 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215345+0000", 2024-09-13T10:11:59.209 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.6659999999999997e-06 2024-09-13T10:11:59.210 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.210 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.210 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.210 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215363+0000", 2024-09-13T10:11:59.210 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8153000000000001e-05 2024-09-13T10:11:59.210 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.210 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.210 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.210 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215388+0000", 2024-09-13T10:11:59.210 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4556e-05 2024-09-13T10:11:59.210 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.210 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.211 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.211 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215542+0000", 2024-09-13T10:11:59.211 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000154637 2024-09-13T10:11:59.211 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.211 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.211 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.211 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215825+0000", 2024-09-13T10:11:59.211 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028208600000000002 2024-09-13T10:11:59.211 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.211 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.211 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:59.211 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254572+0000", 2024-09-13T10:11:59.211 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.038747464000000002 2024-09-13T10:11:59.212 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.212 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.212 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.212 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.255694+0000", 2024-09-13T10:11:59.212 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011222929999999999 2024-09-13T10:11:59.212 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.212 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.212 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.212 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.256931+0000", 2024-09-13T10:11:59.212 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0012370980000000001 2024-09-13T10:11:59.212 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.212 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.212 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.213 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257080+0000", 2024-09-13T10:11:59.213 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014874800000000001 2024-09-13T10:11:59.213 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.213 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.213 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.213 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257211+0000", 2024-09-13T10:11:59.213 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000130674 2024-09-13T10:11:59.213 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.213 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.213 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.213 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257296+0000", 2024-09-13T10:11:59.213 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5589999999999999e-05 2024-09-13T10:11:59.214 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.214 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.214 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.214 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.214 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.214 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:269 4.3 4:c5b5ff75:::smithi19530714-20:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:59.214 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:11:59.214 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 67.979530804000007, 2024-09-13T10:11:59.214 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014988243, 2024-09-13T10:11:59.214 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.214 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.214 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.214 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.215 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.215 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.215 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 269 2024-09-13T10:11:59.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.215 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.215 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.215 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.215 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:11:59.215 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.215 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.215 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.215 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:11:59.216 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.216 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.216 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.216 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.216 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215841+0000", 2024-09-13T10:11:59.216 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0829999999999998e-06 2024-09-13T10:11:59.216 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.216 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.216 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.216 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215854+0000", 2024-09-13T10:11:59.216 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2976000000000001e-05 2024-09-13T10:11:59.216 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.216 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.216 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.217 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215875+0000", 2024-09-13T10:11:59.217 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1100000000000001e-05 2024-09-13T10:11:59.217 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.217 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.217 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.217 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215892+0000", 2024-09-13T10:11:59.217 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7388999999999999e-05 2024-09-13T10:11:59.217 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.217 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.217 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.218 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216353+0000", 2024-09-13T10:11:59.218 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00046100900000000002 2024-09-13T10:11:59.218 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.218 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.218 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.218 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216611+0000", 2024-09-13T10:11:59.218 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025716299999999998 2024-09-13T10:11:59.218 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.218 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.218 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:59.218 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.227177+0000", 2024-09-13T10:11:59.218 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010566676000000001 2024-09-13T10:11:59.219 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.219 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.219 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.219 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229574+0000", 2024-09-13T10:11:59.219 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023964469999999999 2024-09-13T10:11:59.219 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.219 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.219 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.219 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230619+0000", 2024-09-13T10:11:59.219 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0010455950000000001 2024-09-13T10:11:59.219 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.219 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.219 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.220 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230697+0000", 2024-09-13T10:11:59.220 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8021000000000004e-05 2024-09-13T10:11:59.220 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.220 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.220 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.220 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230767+0000", 2024-09-13T10:11:59.220 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9544999999999999e-05 2024-09-13T10:11:59.220 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.220 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.220 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.220 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230827+0000", 2024-09-13T10:11:59.220 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.0238999999999997e-05 2024-09-13T10:11:59.220 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.221 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.221 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.221 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.221 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.221 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:310 4.3 4:c426503f:::smithi19530714-11:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:59.221 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:11:59.221 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 67.922234731000003, 2024-09-13T10:11:59.221 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013782279999999999, 2024-09-13T10:11:59.221 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.221 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.221 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.221 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.221 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.222 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.222 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 310 2024-09-13T10:11:59.222 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.222 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.222 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.222 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.222 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:11:59.222 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.222 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.222 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.222 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.222 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:11:59.222 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.223 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.223 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.223 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.223 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273138+0000", 2024-09-13T10:11:59.223 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8459999999999999e-06 2024-09-13T10:11:59.223 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.223 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.223 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.223 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273150+0000", 2024-09-13T10:11:59.223 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2133999999999999e-05 2024-09-13T10:11:59.223 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.223 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.223 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.224 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273175+0000", 2024-09-13T10:11:59.224 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5321999999999999e-05 2024-09-13T10:11:59.224 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.224 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.224 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.224 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273192+0000", 2024-09-13T10:11:59.224 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.681e-05 2024-09-13T10:11:59.224 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.224 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.224 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.225 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273314+0000", 2024-09-13T10:11:59.225 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012228 2024-09-13T10:11:59.225 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.225 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.225 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.225 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273474+0000", 2024-09-13T10:11:59.225 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016019799999999999 2024-09-13T10:11:59.225 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.225 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.225 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:59.225 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.284136+0000", 2024-09-13T10:11:59.225 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010661761000000001 2024-09-13T10:11:59.226 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.226 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.226 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.226 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286085+0000", 2024-09-13T10:11:59.226 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019484929999999999 2024-09-13T10:11:59.226 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.226 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.226 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.226 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286111+0000", 2024-09-13T10:11:59.226 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5834e-05 2024-09-13T10:11:59.226 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.226 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.226 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.227 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286721+0000", 2024-09-13T10:11:59.227 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00061038599999999998 2024-09-13T10:11:59.227 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.227 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.227 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.227 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286838+0000", 2024-09-13T10:11:59.227 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117064 2024-09-13T10:11:59.227 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.227 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.227 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.227 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286917+0000", 2024-09-13T10:11:59.227 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9152000000000004e-05 2024-09-13T10:11:59.227 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.227 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.228 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.228 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.228 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.228 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1006 4.3 4:c5b5ff75:::smithi19530714-20:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:59.228 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:11:59.228 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 63.541321482999997, 2024-09-13T10:11:59.228 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.047764635, 2024-09-13T10:11:59.228 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.228 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.228 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.228 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.228 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.229 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.229 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1006 2024-09-13T10:11:59.229 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.229 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.229 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.229 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.229 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:11:59.229 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.229 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.229 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.229 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.229 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:11:59.229 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.230 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.230 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.230 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.230 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654049+0000", 2024-09-13T10:11:59.230 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5600000000000005e-07 2024-09-13T10:11:59.230 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.230 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.230 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.230 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654058+0000", 2024-09-13T10:11:59.230 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8200000000000003e-06 2024-09-13T10:11:59.230 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.230 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.230 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.231 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654067+0000", 2024-09-13T10:11:59.231 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.4599999999999992e-06 2024-09-13T10:11:59.231 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.231 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.231 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.231 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654075+0000", 2024-09-13T10:11:59.231 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.1070000000000005e-06 2024-09-13T10:11:59.231 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.231 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.231 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.231 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654188+0000", 2024-09-13T10:11:59.231 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000112734 2024-09-13T10:11:59.231 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.232 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.232 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.232 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654395+0000", 2024-09-13T10:11:59.232 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000207235 2024-09-13T10:11:59.232 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.232 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.232 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.232 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.701813+0000", 2024-09-13T10:11:59.232 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.047417523000000003 2024-09-13T10:11:59.232 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.232 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.232 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.232 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.233 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.233 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1013 4.3 4:c426503f:::smithi19530714-11:head [write 477332~766101] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:59.233 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:11:59.233 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 63.534000229, 2024-09-13T10:11:59.233 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027363838000000001, 2024-09-13T10:11:59.233 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.233 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.233 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.233 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.233 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.233 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.233 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1013 2024-09-13T10:11:59.234 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.234 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.234 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.234 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.234 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:11:59.234 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.234 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.234 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.234 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.234 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:11:59.234 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.234 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.235 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.235 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.235 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661374+0000", 2024-09-13T10:11:59.235 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.0930000000000002e-06 2024-09-13T10:11:59.235 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.235 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.235 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.235 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665828+0000", 2024-09-13T10:11:59.235 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044532119999999998 2024-09-13T10:11:59.235 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.235 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.235 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.236 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665864+0000", 2024-09-13T10:11:59.236 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5979000000000002e-05 2024-09-13T10:11:59.236 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.236 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.236 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.236 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665881+0000", 2024-09-13T10:11:59.236 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7836e-05 2024-09-13T10:11:59.236 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.236 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.236 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.236 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.678976+0000", 2024-09-13T10:11:59.236 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.01309448 2024-09-13T10:11:59.236 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.237 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.237 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.237 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.679188+0000", 2024-09-13T10:11:59.237 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000211945 2024-09-13T10:11:59.237 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.237 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.237 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:59.237 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.679468+0000", 2024-09-13T10:11:59.237 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028034000000000002 2024-09-13T10:11:59.237 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.237 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.237 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.237 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688233+0000", 2024-09-13T10:11:59.238 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0087645369999999993 2024-09-13T10:11:59.238 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.238 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.238 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.238 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688388+0000", 2024-09-13T10:11:59.238 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015488000000000001 2024-09-13T10:11:59.238 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.238 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.238 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.238 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688505+0000", 2024-09-13T10:11:59.238 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117114 2024-09-13T10:11:59.238 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.239 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.239 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.239 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688612+0000", 2024-09-13T10:11:59.239 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010741100000000001 2024-09-13T10:11:59.239 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.239 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.239 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.239 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688733+0000", 2024-09-13T10:11:59.239 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000121011 2024-09-13T10:11:59.239 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.239 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.239 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.239 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.239 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.240 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1014 4.3 4:c426503f:::smithi19530714-11:head [write 1681102~474010] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:59.240 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:11:59.240 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 63.528726843000001, 2024-09-13T10:11:59.240 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022694374, 2024-09-13T10:11:59.240 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.240 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.240 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.240 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.240 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.240 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.240 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1014 2024-09-13T10:11:59.240 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.241 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.241 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.241 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.241 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:11:59.241 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.241 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.241 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.241 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.241 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:11:59.241 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.241 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.241 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.242 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.242 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666645+0000", 2024-09-13T10:11:59.242 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2970000000000002e-06 2024-09-13T10:11:59.242 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.242 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.242 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.242 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667326+0000", 2024-09-13T10:11:59.242 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00068073199999999997 2024-09-13T10:11:59.242 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.242 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.242 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.242 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667346+0000", 2024-09-13T10:11:59.243 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0188999999999999e-05 2024-09-13T10:11:59.243 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.243 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.243 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.243 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667361+0000", 2024-09-13T10:11:59.243 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5299999999999999e-05 2024-09-13T10:11:59.243 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.243 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.243 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.243 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.683468+0000", 2024-09-13T10:11:59.243 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016106944000000002 2024-09-13T10:11:59.243 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.243 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.243 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.244 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.683714+0000", 2024-09-13T10:11:59.244 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024595699999999999 2024-09-13T10:11:59.244 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.244 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.244 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:59.244 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.684132+0000", 2024-09-13T10:11:59.244 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00041775800000000001 2024-09-13T10:11:59.244 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.244 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.244 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.244 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688771+0000", 2024-09-13T10:11:59.244 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0046394540000000003 2024-09-13T10:11:59.244 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.244 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.245 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.245 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688920+0000", 2024-09-13T10:11:59.245 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014886000000000001 2024-09-13T10:11:59.245 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.245 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.245 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.245 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689085+0000", 2024-09-13T10:11:59.245 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000164539 2024-09-13T10:11:59.245 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.245 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.245 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.245 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689230+0000", 2024-09-13T10:11:59.245 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000145394 2024-09-13T10:11:59.246 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.246 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.246 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.246 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689337+0000", 2024-09-13T10:11:59.246 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010695 2024-09-13T10:11:59.246 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.246 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.246 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.246 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.246 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.246 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1015 4.3 4:c426503f:::smithi19530714-11:head [write 2687912~589420] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:59.246 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:11:59.246 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 63.522410166, 2024-09-13T10:11:59.247 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.018489873, 2024-09-13T10:11:59.247 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.247 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.247 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.247 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.247 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.247 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.247 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1015 2024-09-13T10:11:59.247 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.247 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.247 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.247 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.247 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:11:59.248 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.248 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.248 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.248 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.248 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:11:59.248 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.248 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.248 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.248 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.248 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672962+0000", 2024-09-13T10:11:59.248 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.277e-06 2024-09-13T10:11:59.248 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.249 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.249 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.249 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673758+0000", 2024-09-13T10:11:59.249 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00079599100000000002 2024-09-13T10:11:59.249 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.249 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.249 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.249 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673787+0000", 2024-09-13T10:11:59.249 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9280000000000001e-05 2024-09-13T10:11:59.249 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.249 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.249 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.249 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673801+0000", 2024-09-13T10:11:59.250 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4493e-05 2024-09-13T10:11:59.250 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.250 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.250 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.250 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.685885+0000", 2024-09-13T10:11:59.250 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012083692 2024-09-13T10:11:59.250 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.250 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.250 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.250 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.686057+0000", 2024-09-13T10:11:59.250 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017133900000000001 2024-09-13T10:11:59.250 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.250 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.251 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:59.251 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.686396+0000", 2024-09-13T10:11:59.251 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000339809 2024-09-13T10:11:59.251 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.251 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.251 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.251 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691040+0000", 2024-09-13T10:11:59.251 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.004643215 2024-09-13T10:11:59.251 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.251 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.251 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.252 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691082+0000", 2024-09-13T10:11:59.252 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.2499000000000001e-05 2024-09-13T10:11:59.252 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.252 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.252 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.252 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691179+0000", 2024-09-13T10:11:59.252 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7044000000000001e-05 2024-09-13T10:11:59.252 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.252 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.252 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.252 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691324+0000", 2024-09-13T10:11:59.252 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014532299999999999 2024-09-13T10:11:59.253 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.253 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.253 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.253 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691449+0000", 2024-09-13T10:11:59.253 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012491100000000001 2024-09-13T10:11:59.253 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.253 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.253 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.253 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.253 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.253 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1016 4.3 4:c426503f:::smithi19530714-11:head [setxattr _header (56) in=7b,truncate 3277332] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:59.253 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:11:59.253 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 63.521541208999999, 2024-09-13T10:11:59.254 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.019820562999999999, 2024-09-13T10:11:59.254 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.254 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.254 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.254 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.254 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.254 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.254 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1016 2024-09-13T10:11:59.254 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.254 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.254 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.254 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.255 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:11:59.255 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.255 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.255 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.255 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.255 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:11:59.255 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.255 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.255 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.255 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.255 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673830+0000", 2024-09-13T10:11:59.255 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1170000000000001e-06 2024-09-13T10:11:59.255 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.256 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.256 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.256 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673835+0000", 2024-09-13T10:11:59.256 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5559999999999998e-06 2024-09-13T10:11:59.256 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.256 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.256 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.256 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673845+0000", 2024-09-13T10:11:59.256 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6330000000000008e-06 2024-09-13T10:11:59.257 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.257 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.257 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.257 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673856+0000", 2024-09-13T10:11:59.257 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0963e-05 2024-09-13T10:11:59.257 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.257 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.257 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.257 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689480+0000", 2024-09-13T10:11:59.257 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015624605 2024-09-13T10:11:59.257 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.257 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.257 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.258 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689739+0000", 2024-09-13T10:11:59.258 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025878700000000001 2024-09-13T10:11:59.258 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.258 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.258 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:59.258 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.690154+0000", 2024-09-13T10:11:59.258 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00041533800000000001 2024-09-13T10:11:59.258 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.258 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.258 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.258 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691858+0000", 2024-09-13T10:11:59.258 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001703811 2024-09-13T10:11:59.258 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.259 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.259 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.259 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691880+0000", 2024-09-13T10:11:59.259 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1997e-05 2024-09-13T10:11:59.259 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.259 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.259 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.259 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693384+0000", 2024-09-13T10:11:59.259 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0015040190000000001 2024-09-13T10:11:59.259 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.260 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.260 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.260 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693541+0000", 2024-09-13T10:11:59.260 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015723899999999999 2024-09-13T10:11:59.260 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.260 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.260 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.260 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693649+0000", 2024-09-13T10:11:59.260 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010749800000000001 2024-09-13T10:11:59.260 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.260 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.261 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.261 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.261 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.261 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1017 4.3 4:c426503f:::smithi19530714-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:11:59.261 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:11:59.261 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 63.521501196999999, 2024-09-13T10:11:59.261 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020166488999999999, 2024-09-13T10:11:59.261 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.261 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.261 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.261 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.261 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.261 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.262 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1017 2024-09-13T10:11:59.262 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.262 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.262 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.262 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.262 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:11:59.262 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.262 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.262 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.262 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.262 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:11:59.262 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.262 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.262 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.263 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.263 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673869+0000", 2024-09-13T10:11:59.263 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9699999999999995e-07 2024-09-13T10:11:59.263 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.263 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.263 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.263 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673873+0000", 2024-09-13T10:11:59.263 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.4970000000000001e-06 2024-09-13T10:11:59.263 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.263 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.263 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.264 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673885+0000", 2024-09-13T10:11:59.264 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2398999999999999e-05 2024-09-13T10:11:59.264 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.264 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.264 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.264 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673893+0000", 2024-09-13T10:11:59.264 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7530000000000001e-06 2024-09-13T10:11:59.264 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.264 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.264 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.264 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691638+0000", 2024-09-13T10:11:59.264 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017745007 2024-09-13T10:11:59.265 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.265 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.265 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for rw locks", 2024-09-13T10:11:59.265 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691789+0000", 2024-09-13T10:11:59.265 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015133400000000001 2024-09-13T10:11:59.265 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.265 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.265 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.265 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693731+0000", 2024-09-13T10:11:59.265 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019422459999999999 2024-09-13T10:11:59.265 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.265 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.265 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.266 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693870+0000", 2024-09-13T10:11:59.266 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013820299999999999 2024-09-13T10:11:59.266 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.266 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.266 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.266 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.694035+0000", 2024-09-13T10:11:59.266 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000165253 2024-09-13T10:11:59.266 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.266 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.266 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.266 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.266 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.267 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1405 4.3 4:c5b5ff75:::smithi19530714-20:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T10:11:59.267 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:01.492787+0000", 2024-09-13T10:11:59.267 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 57.702582552999999, 2024-09-13T10:11:59.267 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021225698000000001, 2024-09-13T10:11:59.267 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.267 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.267 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.267 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.267 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.267 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.267 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1405 2024-09-13T10:11:59.267 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.267 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.267 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.267 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.268 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492787+0000", 2024-09-13T10:11:59.268 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.268 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.268 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.268 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.268 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492787+0000", 2024-09-13T10:11:59.268 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.268 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.268 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.268 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.268 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492791+0000", 2024-09-13T10:11:59.268 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.89e-06 2024-09-13T10:11:59.268 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.269 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.269 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.269 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492805+0000", 2024-09-13T10:11:59.269 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4083000000000001e-05 2024-09-13T10:11:59.269 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.269 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.269 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.269 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492846+0000", 2024-09-13T10:11:59.269 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0849000000000001e-05 2024-09-13T10:11:59.269 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.269 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.269 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.270 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492867+0000", 2024-09-13T10:11:59.270 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1058999999999999e-05 2024-09-13T10:11:59.270 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.270 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.270 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.270 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492973+0000", 2024-09-13T10:11:59.270 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000106084 2024-09-13T10:11:59.270 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.270 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.270 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.270 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.493105+0000", 2024-09-13T10:11:59.270 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013215899999999999 2024-09-13T10:11:59.270 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.270 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.270 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.271 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.514013+0000", 2024-09-13T10:11:59.271 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020907574000000002 2024-09-13T10:11:59.271 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.271 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.271 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.271 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.271 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.271 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2816 4.3 4:c5b5ff75:::smithi19530714-20:head [assert-version v76,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:59.271 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.891848+0000", 2024-09-13T10:11:59.271 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.303521528000001, 2024-09-13T10:11:59.271 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020571445000000001, 2024-09-13T10:11:59.271 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.271 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.272 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.272 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.272 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.272 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.272 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2816 2024-09-13T10:11:59.272 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.272 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.272 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.272 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.272 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891848+0000", 2024-09-13T10:11:59.272 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.272 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.272 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.273 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.273 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891848+0000", 2024-09-13T10:11:59.273 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.273 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.273 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.273 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.273 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891852+0000", 2024-09-13T10:11:59.273 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1729999999999997e-06 2024-09-13T10:11:59.273 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.273 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.273 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.273 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891869+0000", 2024-09-13T10:11:59.273 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6639000000000001e-05 2024-09-13T10:11:59.274 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.274 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.274 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.274 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891910+0000", 2024-09-13T10:11:59.274 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1125999999999999e-05 2024-09-13T10:11:59.274 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.274 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.274 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.274 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891934+0000", 2024-09-13T10:11:59.274 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4039000000000001e-05 2024-09-13T10:11:59.274 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.274 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.274 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.275 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892099+0000", 2024-09-13T10:11:59.275 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016462000000000001 2024-09-13T10:11:59.275 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.275 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.275 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.275 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892368+0000", 2024-09-13T10:11:59.275 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026922699999999998 2024-09-13T10:11:59.275 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.275 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.275 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:59.275 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.907670+0000", 2024-09-13T10:11:59.275 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015302299 2024-09-13T10:11:59.275 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.276 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.276 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.276 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.910850+0000", 2024-09-13T10:11:59.276 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003179383 2024-09-13T10:11:59.276 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.276 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.276 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.276 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.910901+0000", 2024-09-13T10:11:59.276 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.1548000000000002e-05 2024-09-13T10:11:59.276 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.276 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.276 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.276 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.912089+0000", 2024-09-13T10:11:59.277 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011878170000000001 2024-09-13T10:11:59.277 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.277 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.277 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.277 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.912267+0000", 2024-09-13T10:11:59.277 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000178393 2024-09-13T10:11:59.277 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.277 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.277 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.277 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.912420+0000", 2024-09-13T10:11:59.277 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015218100000000001 2024-09-13T10:11:59.277 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.277 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.278 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.278 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.278 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.278 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2817 4.3 4:c426503f:::smithi19530714-11:head [read 0~3647119,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:59.278 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.892018+0000", 2024-09-13T10:11:59.278 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.303352058000002, 2024-09-13T10:11:59.278 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013334828, 2024-09-13T10:11:59.278 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.278 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.278 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.278 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.278 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.278 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.278 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2817 2024-09-13T10:11:59.279 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.279 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.279 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.279 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.279 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892018+0000", 2024-09-13T10:11:59.279 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.279 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.279 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.279 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.279 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892018+0000", 2024-09-13T10:11:59.279 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.279 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.279 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.280 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.280 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892019+0000", 2024-09-13T10:11:59.280 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.747e-06 2024-09-13T10:11:59.280 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.280 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.280 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.280 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892031+0000", 2024-09-13T10:11:59.280 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.149e-05 2024-09-13T10:11:59.280 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.280 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.280 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.280 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892053+0000", 2024-09-13T10:11:59.280 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2069000000000002e-05 2024-09-13T10:11:59.281 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.281 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.281 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.281 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892073+0000", 2024-09-13T10:11:59.281 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0591999999999999e-05 2024-09-13T10:11:59.281 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.281 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.281 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.281 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892825+0000", 2024-09-13T10:11:59.281 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00075120299999999998 2024-09-13T10:11:59.281 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.281 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.281 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.282 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893031+0000", 2024-09-13T10:11:59.282 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000206579 2024-09-13T10:11:59.282 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.282 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.282 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.282 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.905352+0000", 2024-09-13T10:11:59.282 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012321148000000001 2024-09-13T10:11:59.282 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.282 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.282 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.282 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.282 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.282 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2818 4.3 4:c426503f:::smithi19530714-11:head [sparse-read 0~2823261] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:59.283 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.892118+0000", 2024-09-13T10:11:59.283 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.303251268, 2024-09-13T10:11:59.283 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014982577, 2024-09-13T10:11:59.283 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.283 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.283 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.283 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.283 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.283 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.283 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2818 2024-09-13T10:11:59.283 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.283 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.283 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.283 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.284 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892118+0000", 2024-09-13T10:11:59.284 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.284 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.284 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.284 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.284 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892118+0000", 2024-09-13T10:11:59.284 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.284 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.284 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.284 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.284 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892120+0000", 2024-09-13T10:11:59.284 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5260000000000001e-06 2024-09-13T10:11:59.284 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.284 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.285 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.285 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892128+0000", 2024-09-13T10:11:59.285 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7640000000000005e-06 2024-09-13T10:11:59.285 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.285 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.285 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.285 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892172+0000", 2024-09-13T10:11:59.285 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4561000000000002e-05 2024-09-13T10:11:59.285 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.285 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.285 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.285 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892231+0000", 2024-09-13T10:11:59.286 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9194999999999998e-05 2024-09-13T10:11:59.286 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.286 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.286 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.286 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.903452+0000", 2024-09-13T10:11:59.286 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011221024 2024-09-13T10:11:59.286 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.286 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.286 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.286 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.903711+0000", 2024-09-13T10:11:59.286 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025891100000000002 2024-09-13T10:11:59.286 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.286 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.287 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.287 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.907101+0000", 2024-09-13T10:11:59.287 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0033895959999999999 2024-09-13T10:11:59.287 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.287 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.287 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.287 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.287 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.287 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2819 4.3 4:c426503f:::smithi19530714-11:head [sparse-read 1467121~1802604] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:59.287 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.892259+0000", 2024-09-13T10:11:59.287 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.30311026, 2024-09-13T10:11:59.287 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.018443480000000002, 2024-09-13T10:11:59.287 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.287 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.287 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.288 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.288 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.288 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.288 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2819 2024-09-13T10:11:59.288 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.288 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.288 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.288 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.288 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892259+0000", 2024-09-13T10:11:59.288 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.288 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.288 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.288 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.288 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892259+0000", 2024-09-13T10:11:59.289 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.289 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.289 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.289 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.289 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892261+0000", 2024-09-13T10:11:59.289 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5200000000000001e-06 2024-09-13T10:11:59.289 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.289 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.289 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.289 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892287+0000", 2024-09-13T10:11:59.289 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5976e-05 2024-09-13T10:11:59.289 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.289 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.290 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.290 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892306+0000", 2024-09-13T10:11:59.290 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9208999999999999e-05 2024-09-13T10:11:59.290 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.290 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.290 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.290 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892322+0000", 2024-09-13T10:11:59.290 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6426000000000002e-05 2024-09-13T10:11:59.290 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.290 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.290 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.290 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.908963+0000", 2024-09-13T10:11:59.290 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016640922999999998 2024-09-13T10:11:59.290 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.291 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.291 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.291 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.909202+0000", 2024-09-13T10:11:59.291 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023879500000000001 2024-09-13T10:11:59.291 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.291 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.291 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.291 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.910703+0000", 2024-09-13T10:11:59.291 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0015006310000000001 2024-09-13T10:11:59.291 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.291 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.291 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.291 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.292 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.292 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3078 4.3 4:c426503f:::smithi19530714-11:head [read 0~2487469,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T10:11:59.292 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:38.708725+0000", 2024-09-13T10:11:59.292 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 20.486644661, 2024-09-13T10:11:59.292 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013854750000000001, 2024-09-13T10:11:59.292 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.292 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.292 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.292 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.292 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.292 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.292 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3078 2024-09-13T10:11:59.292 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.293 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.293 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.293 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.293 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708725+0000", 2024-09-13T10:11:59.293 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.293 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.293 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.293 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.293 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708725+0000", 2024-09-13T10:11:59.293 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.293 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.293 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.294 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.294 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708726+0000", 2024-09-13T10:11:59.294 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.08e-06 2024-09-13T10:11:59.294 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.294 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.294 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.294 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708731+0000", 2024-09-13T10:11:59.294 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.3029999999999999e-06 2024-09-13T10:11:59.294 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.294 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.294 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.294 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708741+0000", 2024-09-13T10:11:59.295 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0077e-05 2024-09-13T10:11:59.295 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.295 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.295 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.295 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708750+0000", 2024-09-13T10:11:59.295 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5560000000000008e-06 2024-09-13T10:11:59.295 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.295 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.295 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.295 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.720444+0000", 2024-09-13T10:11:59.295 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011694334000000001 2024-09-13T10:11:59.295 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.295 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.296 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.296 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.720692+0000", 2024-09-13T10:11:59.296 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024811599999999999 2024-09-13T10:11:59.296 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.296 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.296 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.296 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.722580+0000", 2024-09-13T10:11:59.296 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0018872839999999999 2024-09-13T10:11:59.296 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.296 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.296 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.296 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.296 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.297 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3282 4.1 4:85f1e656:::smithi19530714-10:head [write 574872~695118,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:11:59.297 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:43.118594+0000", 2024-09-13T10:11:59.297 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 16.076775874999999, 2024-09-13T10:11:59.297 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015113895, 2024-09-13T10:11:59.297 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.297 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.297 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.297 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.297 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.297 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.297 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3282 2024-09-13T10:11:59.297 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.297 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.298 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.298 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.298 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.118594+0000", 2024-09-13T10:11:59.298 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.298 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.298 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.298 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.298 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.118594+0000", 2024-09-13T10:11:59.298 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.298 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.298 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.298 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.298 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.118597+0000", 2024-09-13T10:11:59.298 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7860000000000001e-06 2024-09-13T10:11:59.299 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.299 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.299 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.299 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.119871+0000", 2024-09-13T10:11:59.299 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0012741530000000001 2024-09-13T10:11:59.299 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.299 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.299 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.299 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.119904+0000", 2024-09-13T10:11:59.299 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3325999999999999e-05 2024-09-13T10:11:59.299 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.299 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.300 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.300 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.119927+0000", 2024-09-13T10:11:59.300 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2580000000000001e-05 2024-09-13T10:11:59.300 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.300 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.300 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.300 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.120067+0000", 2024-09-13T10:11:59.300 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014075200000000001 2024-09-13T10:11:59.300 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.300 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.300 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op must wait for map", 2024-09-13T10:11:59.300 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.120078+0000", 2024-09-13T10:11:59.300 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0776e-05 2024-09-13T10:11:59.301 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.301 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.301 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.301 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.124523+0000", 2024-09-13T10:11:59.301 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044453089999999997 2024-09-13T10:11:59.301 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.301 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.301 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.301 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.124748+0000", 2024-09-13T10:11:59.301 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022474100000000001 2024-09-13T10:11:59.301 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.301 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.301 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:11:59.302 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.125194+0000", 2024-09-13T10:11:59.302 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00044562699999999998 2024-09-13T10:11:59.302 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.302 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.302 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.302 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.132524+0000", 2024-09-13T10:11:59.302 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0073297850000000001 2024-09-13T10:11:59.302 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.302 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.302 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.302 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.132626+0000", 2024-09-13T10:11:59.302 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010255399999999999 2024-09-13T10:11:59.302 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.302 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.132778+0000", 2024-09-13T10:11:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000151459 2024-09-13T10:11:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.133577+0000", 2024-09-13T10:11:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00079910099999999998 2024-09-13T10:11:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.133708+0000", 2024-09-13T10:11:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000130946 2024-09-13T10:11:59.303 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.304 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.304 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.304 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.304 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.304 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3439 4.1 4:85f1e656:::smithi19530714-10:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e85)", 2024-09-13T10:11:59.304 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:47.513941+0000", 2024-09-13T10:11:59.304 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 11.681428308999999, 2024-09-13T10:11:59.304 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051734025000000003, 2024-09-13T10:11:59.304 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.304 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.304 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.304 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.304 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.305 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.305 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3439 2024-09-13T10:11:59.305 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.305 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.305 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.305 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.305 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.513941+0000", 2024-09-13T10:11:59.305 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.305 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.305 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.305 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.305 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.513941+0000", 2024-09-13T10:11:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.513945+0000", 2024-09-13T10:11:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.207e-06 2024-09-13T10:11:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.513956+0000", 2024-09-13T10:11:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1899000000000001e-05 2024-09-13T10:11:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.306 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.513996+0000", 2024-09-13T10:11:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9456000000000003e-05 2024-09-13T10:11:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.514014+0000", 2024-09-13T10:11:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8050000000000002e-05 2024-09-13T10:11:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.514184+0000", 2024-09-13T10:11:59.307 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017006500000000001 2024-09-13T10:11:59.308 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.308 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.308 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op must wait for map", 2024-09-13T10:11:59.308 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.514196+0000", 2024-09-13T10:11:59.308 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2223e-05 2024-09-13T10:11:59.308 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.308 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.308 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.308 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.517357+0000", 2024-09-13T10:11:59.308 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0031610919999999999 2024-09-13T10:11:59.308 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.309 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.309 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.309 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.517577+0000", 2024-09-13T10:11:59.309 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021954400000000001 2024-09-13T10:11:59.309 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.309 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.309 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.309 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.565675+0000", 2024-09-13T10:11:59.309 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048098489000000001 2024-09-13T10:11:59.309 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.309 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.309 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.309 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.310 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.310 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3529 4.3 4:c5b5ff75:::smithi19530714-20:head [write 494205~473908] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-13T10:11:59.310 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:48.974598+0000", 2024-09-13T10:11:59.310 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.220771485, 2024-09-13T10:11:59.310 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017452275, 2024-09-13T10:11:59.310 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.310 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.310 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.312 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.312 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.312 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.312 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3529 2024-09-13T10:11:59.313 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.313 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.313 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.313 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.313 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.974598+0000", 2024-09-13T10:11:59.313 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.313 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.313 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.313 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.313 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.974598+0000", 2024-09-13T10:11:59.313 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.313 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.313 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.314 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.314 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.974602+0000", 2024-09-13T10:11:59.314 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.393e-06 2024-09-13T10:11:59.314 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.314 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.314 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.314 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.975525+0000", 2024-09-13T10:11:59.314 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00092394999999999999 2024-09-13T10:11:59.314 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.314 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.314 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.314 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.975562+0000", 2024-09-13T10:11:59.314 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6949000000000001e-05 2024-09-13T10:11:59.315 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.315 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.315 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.315 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.975587+0000", 2024-09-13T10:11:59.315 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4805999999999999e-05 2024-09-13T10:11:59.315 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.315 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.315 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.315 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.975799+0000", 2024-09-13T10:11:59.315 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021148200000000001 2024-09-13T10:11:59.315 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.315 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.315 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op must wait for map", 2024-09-13T10:11:59.316 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.975813+0000", 2024-09-13T10:11:59.316 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4090000000000001e-05 2024-09-13T10:11:59.316 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.316 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.316 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.316 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.980728+0000", 2024-09-13T10:11:59.316 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0049153310000000002 2024-09-13T10:11:59.316 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.316 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.316 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.316 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.980922+0000", 2024-09-13T10:11:59.316 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000194372 2024-09-13T10:11:59.317 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.317 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.317 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:59.317 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.981273+0000", 2024-09-13T10:11:59.317 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00035047799999999999 2024-09-13T10:11:59.317 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.317 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.317 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.317 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.991703+0000", 2024-09-13T10:11:59.317 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010430406 2024-09-13T10:11:59.317 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.317 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.317 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.318 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.991802+0000", 2024-09-13T10:11:59.318 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.8664000000000003e-05 2024-09-13T10:11:59.318 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.318 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.318 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.318 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.991903+0000", 2024-09-13T10:11:59.318 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010054499999999999 2024-09-13T10:11:59.318 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.318 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.318 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.318 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.991977+0000", 2024-09-13T10:11:59.318 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.4017999999999995e-05 2024-09-13T10:11:59.318 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.319 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.319 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.319 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.992050+0000", 2024-09-13T10:11:59.319 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.3790999999999999e-05 2024-09-13T10:11:59.319 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.319 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.319 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.319 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.319 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.319 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3530 4.3 4:c5b5ff75:::smithi19530714-20:head [write 1433122~407102] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-13T10:11:59.319 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:48.979789+0000", 2024-09-13T10:11:59.319 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.215580507, 2024-09-13T10:11:59.319 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.01268818, 2024-09-13T10:11:59.320 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.320 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.320 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.320 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.320 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.320 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.320 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3530 2024-09-13T10:11:59.320 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.320 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.320 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.320 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.320 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.979789+0000", 2024-09-13T10:11:59.320 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.320 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.321 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.321 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.321 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.979789+0000", 2024-09-13T10:11:59.321 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.321 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.321 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.321 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.321 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.979793+0000", 2024-09-13T10:11:59.321 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1559999999999997e-06 2024-09-13T10:11:59.321 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.321 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.321 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.321 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.980499+0000", 2024-09-13T10:11:59.322 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00070596000000000005 2024-09-13T10:11:59.322 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.322 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.322 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.322 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.980523+0000", 2024-09-13T10:11:59.322 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3541999999999999e-05 2024-09-13T10:11:59.322 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.322 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.322 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.322 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.980543+0000", 2024-09-13T10:11:59.322 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0477000000000001e-05 2024-09-13T10:11:59.322 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.323 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.323 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.323 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.986322+0000", 2024-09-13T10:11:59.323 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.005778783 2024-09-13T10:11:59.323 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.323 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.323 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.323 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.986458+0000", 2024-09-13T10:11:59.323 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000135797 2024-09-13T10:11:59.323 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.323 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.323 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:11:59.323 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.986687+0000", 2024-09-13T10:11:59.324 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000228948 2024-09-13T10:11:59.324 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.324 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.324 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.324 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.992075+0000", 2024-09-13T10:11:59.324 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.005388103 2024-09-13T10:11:59.324 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.324 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.324 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.324 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.992239+0000", 2024-09-13T10:11:59.324 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000164203 2024-09-13T10:11:59.324 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.324 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.325 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.325 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.992344+0000", 2024-09-13T10:11:59.325 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000104731 2024-09-13T10:11:59.325 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.325 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.325 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.325 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.992419+0000", 2024-09-13T10:11:59.325 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5238e-05 2024-09-13T10:11:59.325 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.325 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.325 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.325 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.992477+0000", 2024-09-13T10:11:59.325 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8242000000000002e-05 2024-09-13T10:11:59.326 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.326 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.326 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.326 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.326 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.326 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:59.328 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T10:11:59.328 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:11:59.321+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:11:59.334 INFO:tasks.rados.rados.0.smithi195.stdout:1658: expect (ObjNum 339 snap 0 seq_num 339) 2024-09-13T10:11:59.408 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 730330 2024-09-13T10:11:59.429 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:11:59.426+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:11:59.454 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:59.454 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:59.454 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:11:59.454 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:59.468 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T10:11:59.476 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:59.530 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:11:59.527+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:11:59.554 INFO:tasks.rados.rados.0.smithi195.stdout:1654: done (6 left) 2024-09-13T10:11:59.554 INFO:tasks.rados.rados.0.smithi195.stdout:1655: done (5 left) 2024-09-13T10:11:59.554 INFO:tasks.rados.rados.0.smithi195.stdout:1656: done (4 left) 2024-09-13T10:11:59.554 INFO:tasks.rados.rados.0.smithi195.stdout:1657: done (3 left) 2024-09-13T10:11:59.554 INFO:tasks.rados.rados.0.smithi195.stdout:1658: done (2 left) 2024-09-13T10:11:59.554 INFO:tasks.rados.rados.0.smithi195.stdout:1659: done (1 left) 2024-09-13T10:11:59.554 INFO:tasks.rados.rados.0.smithi195.stdout:1660: done (0 left) 2024-09-13T10:11:59.554 INFO:tasks.rados.rados.0.smithi195.stdout:1661: set_chunk oid 4 target oid 2024-09-13T10:11:59.574 INFO:tasks.rados.rados.0.smithi195.stdout:1661: set_chunk oid 4 offset: 2571776 length: 1124864 target oid 610dedd959639fb30374cb8fcde17e38e3aa5b4d30d57eec82b5d2a7de4d845a offset: 0 snap -1 2024-09-13T10:11:59.575 INFO:tasks.rados.rados.0.smithi195.stdout:1662: read oid 49 snap -1 2024-09-13T10:11:59.575 INFO:tasks.rados.rados.0.smithi195.stdout:1662: expect deleted 2024-09-13T10:11:59.575 INFO:tasks.rados.rados.0.smithi195.stdout:1663: read oid 6 snap -1 2024-09-13T10:11:59.575 INFO:tasks.rados.rados.0.smithi195.stdout:1663: expect (ObjNum 337 snap 0 seq_num 337) 2024-09-13T10:11:59.575 INFO:tasks.rados.rados.0.smithi195.stdout:1664: set_chunk oid 9 target oid 2024-09-13T10:11:59.588 INFO:tasks.rados.rados.0.smithi195.stdout:1664: set_chunk oid 9 offset: 1769472 length: 499712 target oid 6b111e9676c2b9fa7ff581fef1dae96b01304db5e4a898a64b9602e7d6020af6 offset: 0 snap -1 2024-09-13T10:11:59.588 INFO:tasks.rados.rados.0.smithi195.stdout:1665: read oid 21 snap -1 2024-09-13T10:11:59.588 INFO:tasks.rados.rados.0.smithi195.stdout:1665: expect (ObjNum 235 snap 0 seq_num 235) 2024-09-13T10:11:59.588 INFO:tasks.rados.rados.0.smithi195.stdout:1666: write (excl) oid 10 current snap is 0 2024-09-13T10:11:59.588 INFO:tasks.rados.rados.0.smithi195.stdout:1661: finishing set_chunk to oid 4 2024-09-13T10:11:59.588 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 4 set_chunk returned error code -95 offset: 2571776 length: 1124864 2024-09-13T10:11:59.588 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 0 (ObjNum 352 snap 0 seq_num 352) dirty exists 2024-09-13T10:11:59.589 INFO:tasks.rados.rados.0.smithi195.stdout:1666: seq_num 355 ranges {766485=465633,1990637=647776,3299280=667205} 2024-09-13T10:11:59.608 INFO:tasks.rados.rados.0.smithi195.stdout:1666: writing smithi19530714-10 from 766485 to 1232118 tid 1 2024-09-13T10:11:59.630 INFO:tasks.rados.rados.0.smithi195.stdout:1666: writing smithi19530714-10 from 1990637 to 2638413 tid 2 2024-09-13T10:11:59.631 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:11:59.627+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:11:59.644 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:59.645 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:11:59.645 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:11:59.645 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:11:59.645 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:59.647 INFO:tasks.rados.rados.0.smithi195.stdout:1666: writing smithi19530714-10 from 3299280 to 3966485 tid 3 2024-09-13T10:11:59.647 INFO:tasks.rados.rados.0.smithi195.stdout:1661: done (5 left) 2024-09-13T10:11:59.647 INFO:tasks.rados.rados.0.smithi195.stdout:1662: done (4 left) 2024-09-13T10:11:59.647 INFO:tasks.rados.rados.0.smithi195.stdout:1667: read oid 44 snap -1 2024-09-13T10:11:59.647 INFO:tasks.rados.rados.0.smithi195.stdout:1667: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:11:59.647 INFO:tasks.rados.rados.0.smithi195.stdout:1668: write (excl) oid 36 current snap is 0 2024-09-13T10:11:59.648 INFO:tasks.rados.rados.0.smithi195.stdout:1668: seq_num 356 ranges {783292=540882,1932766=507787,3210350=572013} 2024-09-13T10:11:59.659 INFO:tasks.rados.rados.0.smithi195.stdout:1668: writing smithi19530714-36 from 783292 to 1324174 tid 1 2024-09-13T10:11:59.661 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T10:11:59.668 INFO:tasks.rados.rados.0.smithi195.stdout:1668: writing smithi19530714-36 from 1932766 to 2440553 tid 2 2024-09-13T10:11:59.678 INFO:tasks.rados.rados.0.smithi195.stdout:1668: writing smithi19530714-36 from 3210350 to 3782363 tid 3 2024-09-13T10:11:59.678 INFO:tasks.rados.rados.0.smithi195.stdout:1669: read oid 2 snap -1 2024-09-13T10:11:59.678 INFO:tasks.rados.rados.0.smithi195.stdout:1669: expect (ObjNum 280 snap 0 seq_num 280) 2024-09-13T10:11:59.678 INFO:tasks.rados.rados.0.smithi195.stdout:1670: write oid 1 current snap is 0 2024-09-13T10:11:59.679 INFO:tasks.rados.rados.0.smithi195.stdout:1670: seq_num 357 ranges {400099=616131,1591248=408851} 2024-09-13T10:11:59.685 INFO:tasks.rados.rados.0.smithi195.stdout:1670: writing smithi19530714-1 from 400099 to 1016230 tid 1 2024-09-13T10:11:59.692 INFO:tasks.rados.rados.0.smithi195.stdout:1670: writing smithi19530714-1 from 1591248 to 2000099 tid 2 2024-09-13T10:11:59.692 INFO:tasks.rados.rados.0.smithi195.stdout:1671: set_chunk oid 23 target oid 2024-09-13T10:11:59.698 INFO:tasks.rados.rados.0.smithi195.stdout:1671: set_chunk oid 23 offset: 708096 length: 92672 target oid 6e9b8f6ec325f569d4b92bbe2029af2f1759cee585a5c89e36fd76c74c159126 offset: 0 snap -1 2024-09-13T10:11:59.698 INFO:tasks.rados.rados.0.smithi195.stdout:1672: tier_flush oid 5 2024-09-13T10:11:59.698 INFO:tasks.rados.rados.0.smithi195.stdout:1663: expect (ObjNum 337 snap 0 seq_num 337) 2024-09-13T10:11:59.698 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:11:59.732 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:11:59.726+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:11:59.747 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-13T10:11:59.787 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:11:59.788 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:11:59.788 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:11:59.788 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:11:59.788 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.788 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:251 4.5 4:a71ca386:::smithi19530714-2:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:59.788 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:11:59.788 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 68.666300171000003, 2024-09-13T10:11:59.788 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041813326999999997, 2024-09-13T10:11:59.788 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.788 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.788 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.788 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.789 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.789 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.789 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 251 2024-09-13T10:11:59.789 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.789 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.789 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.789 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.789 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:11:59.789 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.789 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.789 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.789 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.789 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:11:59.790 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.790 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.790 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.790 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.790 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117990+0000", 2024-09-13T10:11:59.790 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.203e-06 2024-09-13T10:11:59.790 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.790 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.790 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.790 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117997+0000", 2024-09-13T10:11:59.790 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9750000000000001e-06 2024-09-13T10:11:59.790 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.791 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.791 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.791 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118009+0000", 2024-09-13T10:11:59.791 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2478e-05 2024-09-13T10:11:59.791 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.791 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.791 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.791 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118017+0000", 2024-09-13T10:11:59.791 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6909999999999997e-06 2024-09-13T10:11:59.791 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.791 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.791 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.791 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118106+0000", 2024-09-13T10:11:59.792 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.9331000000000002e-05 2024-09-13T10:11:59.792 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.792 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.792 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.792 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118184+0000", 2024-09-13T10:11:59.792 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7448999999999997e-05 2024-09-13T10:11:59.792 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.792 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.792 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,10", 2024-09-13T10:11:59.792 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158604+0000", 2024-09-13T10:11:59.792 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.040419954000000001 2024-09-13T10:11:59.793 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.793 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.793 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.793 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159306+0000", 2024-09-13T10:11:59.793 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00070229299999999999 2024-09-13T10:11:59.793 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.793 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.793 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.793 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159539+0000", 2024-09-13T10:11:59.793 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023353600000000001 2024-09-13T10:11:59.793 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.793 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.794 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.794 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159696+0000", 2024-09-13T10:11:59.794 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015616 2024-09-13T10:11:59.794 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.794 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.794 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.794 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159754+0000", 2024-09-13T10:11:59.794 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8513e-05 2024-09-13T10:11:59.794 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.794 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.794 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.794 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159802+0000", 2024-09-13T10:11:59.795 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.7744000000000002e-05 2024-09-13T10:11:59.795 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.795 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.795 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.795 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.795 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.795 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:270 4.5 4:a59e88f7:::smithi19530714-21:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:59.795 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:11:59.795 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 68.558254044999998, 2024-09-13T10:11:59.795 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041472594000000002, 2024-09-13T10:11:59.795 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.796 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.796 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.796 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.796 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.796 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.796 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 270 2024-09-13T10:11:59.796 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.796 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.796 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.796 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.796 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:11:59.796 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.796 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.797 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.797 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.797 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:11:59.797 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.797 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.797 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.797 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.797 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226037+0000", 2024-09-13T10:11:59.797 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6560000000000001e-06 2024-09-13T10:11:59.797 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.797 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.797 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.798 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226048+0000", 2024-09-13T10:11:59.798 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0893000000000001e-05 2024-09-13T10:11:59.798 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.798 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.798 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.798 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226079+0000", 2024-09-13T10:11:59.798 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0412999999999998e-05 2024-09-13T10:11:59.798 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.798 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.798 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.798 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226106+0000", 2024-09-13T10:11:59.798 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7160000000000001e-05 2024-09-13T10:11:59.798 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.799 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.799 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.799 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226234+0000", 2024-09-13T10:11:59.799 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000128393 2024-09-13T10:11:59.799 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.799 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.799 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.799 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226388+0000", 2024-09-13T10:11:59.799 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015424300000000001 2024-09-13T10:11:59.799 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.799 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.800 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,10", 2024-09-13T10:11:59.800 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262647+0000", 2024-09-13T10:11:59.800 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036258460999999999 2024-09-13T10:11:59.800 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.800 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.800 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.800 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264690+0000", 2024-09-13T10:11:59.800 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020429889999999998 2024-09-13T10:11:59.800 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.800 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.800 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.800 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.266720+0000", 2024-09-13T10:11:59.800 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002029669 2024-09-13T10:11:59.801 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.801 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.801 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.801 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267350+0000", 2024-09-13T10:11:59.801 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00063076699999999996 2024-09-13T10:11:59.801 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.801 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.801 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.801 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267409+0000", 2024-09-13T10:11:59.801 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8366000000000003e-05 2024-09-13T10:11:59.801 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.801 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.802 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.802 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267507+0000", 2024-09-13T10:11:59.802 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.8584000000000003e-05 2024-09-13T10:11:59.802 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.802 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.802 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.802 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.802 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.802 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:275 4.2 4:4ac406d9:::smithi19530714-26:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:59.802 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:11:59.802 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 68.554506982000007, 2024-09-13T10:11:59.802 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024603505000000001, 2024-09-13T10:11:59.803 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.803 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.803 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.803 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.803 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.803 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.803 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 275 2024-09-13T10:11:59.803 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.803 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.803 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.803 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.803 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:11:59.803 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.804 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.804 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.804 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.804 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:11:59.804 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.804 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.804 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.804 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.804 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229783+0000", 2024-09-13T10:11:59.804 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.44e-06 2024-09-13T10:11:59.804 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.804 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.805 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.805 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229793+0000", 2024-09-13T10:11:59.805 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0163000000000001e-05 2024-09-13T10:11:59.805 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.805 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.805 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.805 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229814+0000", 2024-09-13T10:11:59.805 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0564000000000002e-05 2024-09-13T10:11:59.805 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.805 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.805 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.806 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229831+0000", 2024-09-13T10:11:59.806 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7142e-05 2024-09-13T10:11:59.806 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.806 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.806 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.806 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229949+0000", 2024-09-13T10:11:59.806 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000118014 2024-09-13T10:11:59.806 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.806 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.806 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.806 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230081+0000", 2024-09-13T10:11:59.806 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013181299999999999 2024-09-13T10:11:59.807 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.807 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.807 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,14", 2024-09-13T10:11:59.807 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.236541+0000", 2024-09-13T10:11:59.807 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0064597680000000003 2024-09-13T10:11:59.807 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.807 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.807 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.807 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.238773+0000", 2024-09-13T10:11:59.807 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022326680000000002 2024-09-13T10:11:59.807 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.807 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.807 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.807 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.241978+0000", 2024-09-13T10:11:59.808 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00320455 2024-09-13T10:11:59.808 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.808 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.808 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.808 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254159+0000", 2024-09-13T10:11:59.808 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012181149 2024-09-13T10:11:59.808 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.808 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.808 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.808 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254259+0000", 2024-09-13T10:11:59.808 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000100184 2024-09-13T10:11:59.808 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.809 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.809 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.809 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254385+0000", 2024-09-13T10:11:59.809 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012605 2024-09-13T10:11:59.809 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.809 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.809 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.809 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.809 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.809 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:270 4.5 4:a59e88f7:::smithi19530714-21:head [assert-version v9,set-chunk] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:59.809 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:11:59.809 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 68.550852551999995, 2024-09-13T10:11:59.809 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033985986000000003, 2024-09-13T10:11:59.810 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.810 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.810 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "waiting for ondisk", 2024-09-13T10:11:59.810 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.810 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.810 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.810 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 270 2024-09-13T10:11:59.810 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.810 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.810 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.810 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.810 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:11:59.810 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.811 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.811 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.811 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.811 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:11:59.811 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.811 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.811 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.811 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.811 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233438+0000", 2024-09-13T10:11:59.811 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.187e-06 2024-09-13T10:11:59.811 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.811 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.811 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.811 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233449+0000", 2024-09-13T10:11:59.811 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0926e-05 2024-09-13T10:11:59.812 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.812 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.812 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.812 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233477+0000", 2024-09-13T10:11:59.812 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7515999999999999e-05 2024-09-13T10:11:59.812 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.812 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.812 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.812 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233499+0000", 2024-09-13T10:11:59.812 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1719999999999999e-05 2024-09-13T10:11:59.812 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.812 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.812 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.812 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233670+0000", 2024-09-13T10:11:59.813 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000171146 2024-09-13T10:11:59.813 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.813 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.813 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for blocked object", 2024-09-13T10:11:59.813 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233823+0000", 2024-09-13T10:11:59.813 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015309300000000001 2024-09-13T10:11:59.813 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.813 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.813 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.813 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.263191+0000", 2024-09-13T10:11:59.813 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029368157999999998 2024-09-13T10:11:59.813 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.813 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.814 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for ondisk", 2024-09-13T10:11:59.814 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.263301+0000", 2024-09-13T10:11:59.814 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110214 2024-09-13T10:11:59.814 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.814 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.814 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.814 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267422+0000", 2024-09-13T10:11:59.814 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0041210259999999999 2024-09-13T10:11:59.814 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.814 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.814 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.814 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.814 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.814 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:277 4.4 4:20c7201f:::smithi19530714-28:head [assert-version v9,set-chunk] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:59.814 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:11:59.815 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 68.550246466999994, 2024-09-13T10:11:59.815 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.042430906999999997, 2024-09-13T10:11:59.815 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.815 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.815 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.815 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.815 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.815 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.815 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 277 2024-09-13T10:11:59.815 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.815 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.815 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.815 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.815 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:11:59.815 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.816 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.816 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.816 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.816 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:11:59.816 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.816 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.816 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.816 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.816 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234043+0000", 2024-09-13T10:11:59.816 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.13e-06 2024-09-13T10:11:59.816 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.816 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.816 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.817 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234050+0000", 2024-09-13T10:11:59.817 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.263e-06 2024-09-13T10:11:59.817 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.817 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.817 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.817 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234065+0000", 2024-09-13T10:11:59.817 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5207e-05 2024-09-13T10:11:59.817 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.817 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.818 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.818 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234078+0000", 2024-09-13T10:11:59.818 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3643e-05 2024-09-13T10:11:59.818 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.818 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.818 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.818 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.235383+0000", 2024-09-13T10:11:59.818 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0013048269999999999 2024-09-13T10:11:59.818 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.818 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.818 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.818 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.235604+0000", 2024-09-13T10:11:59.819 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022116900000000001 2024-09-13T10:11:59.819 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.819 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.819 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:59.819 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.271455+0000", 2024-09-13T10:11:59.819 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035850173999999999 2024-09-13T10:11:59.819 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.819 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.819 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.819 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273083+0000", 2024-09-13T10:11:59.819 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0016287649999999999 2024-09-13T10:11:59.819 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.819 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.819 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.820 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.275756+0000", 2024-09-13T10:11:59.820 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0026721140000000002 2024-09-13T10:11:59.820 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.820 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.820 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.820 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276255+0000", 2024-09-13T10:11:59.820 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00049961600000000004 2024-09-13T10:11:59.820 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.820 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.820 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.820 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276401+0000", 2024-09-13T10:11:59.820 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014584099999999999 2024-09-13T10:11:59.821 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.821 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.821 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.821 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276473+0000", 2024-09-13T10:11:59.821 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2157999999999996e-05 2024-09-13T10:11:59.821 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.821 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.821 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.821 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.821 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.821 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:324 4.4 4:26b855f3:::smithi19530714-25:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:59.821 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:11:59.821 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 68.503584220999997, 2024-09-13T10:11:59.822 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033888489000000001, 2024-09-13T10:11:59.822 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.822 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.822 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.822 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.822 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.822 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.822 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 324 2024-09-13T10:11:59.822 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.822 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.822 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.822 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.822 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:11:59.823 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.823 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.823 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.823 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.823 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:11:59.823 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.823 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.823 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.823 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.823 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280706+0000", 2024-09-13T10:11:59.823 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6479999999999999e-06 2024-09-13T10:11:59.823 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.823 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.824 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.824 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280715+0000", 2024-09-13T10:11:59.824 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1500000000000005e-06 2024-09-13T10:11:59.824 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.824 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.824 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.824 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280733+0000", 2024-09-13T10:11:59.824 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8168e-05 2024-09-13T10:11:59.824 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.824 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.824 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.824 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280747+0000", 2024-09-13T10:11:59.824 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3661e-05 2024-09-13T10:11:59.824 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.825 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.825 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.825 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281013+0000", 2024-09-13T10:11:59.825 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026565099999999999 2024-09-13T10:11:59.825 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.825 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.825 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.825 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281124+0000", 2024-09-13T10:11:59.825 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000111715 2024-09-13T10:11:59.825 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.825 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.825 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:59.826 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286502+0000", 2024-09-13T10:11:59.826 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0053775259999999997 2024-09-13T10:11:59.826 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.826 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.826 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.826 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.289334+0000", 2024-09-13T10:11:59.826 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0028316779999999998 2024-09-13T10:11:59.826 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.826 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.826 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.826 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.290331+0000", 2024-09-13T10:11:59.826 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0009974929999999999 2024-09-13T10:11:59.826 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.826 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.827 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.827 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314453+0000", 2024-09-13T10:11:59.827 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024121417999999999 2024-09-13T10:11:59.827 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.827 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.827 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.827 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314531+0000", 2024-09-13T10:11:59.827 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8841e-05 2024-09-13T10:11:59.827 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.827 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.827 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.827 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314593+0000", 2024-09-13T10:11:59.827 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.1539999999999997e-05 2024-09-13T10:11:59.828 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.828 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.828 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.828 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.828 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.828 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:327 4.4 4:20c7201f:::smithi19530714-28:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:59.828 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:11:59.828 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 68.503116391999995, 2024-09-13T10:11:59.828 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035855512999999999, 2024-09-13T10:11:59.828 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.828 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.828 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.828 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.828 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.829 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.829 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 327 2024-09-13T10:11:59.829 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.829 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.829 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.829 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.829 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:11:59.829 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.829 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.829 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.829 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.829 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:11:59.829 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.830 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.830 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.830 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.830 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281173+0000", 2024-09-13T10:11:59.830 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0020000000000001e-06 2024-09-13T10:11:59.830 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.830 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.830 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.830 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281180+0000", 2024-09-13T10:11:59.830 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.8260000000000004e-06 2024-09-13T10:11:59.830 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.830 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.830 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.831 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281192+0000", 2024-09-13T10:11:59.831 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1725e-05 2024-09-13T10:11:59.831 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.831 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.831 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.831 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281202+0000", 2024-09-13T10:11:59.831 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0492000000000001e-05 2024-09-13T10:11:59.831 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.831 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.831 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.831 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281396+0000", 2024-09-13T10:11:59.831 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019404200000000001 2024-09-13T10:11:59.831 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.831 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.832 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.832 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281533+0000", 2024-09-13T10:11:59.832 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000136267 2024-09-13T10:11:59.832 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.832 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.832 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:59.832 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.313834+0000", 2024-09-13T10:11:59.832 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032301491000000002 2024-09-13T10:11:59.832 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.832 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.832 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.832 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314993+0000", 2024-09-13T10:11:59.832 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001158585 2024-09-13T10:11:59.832 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.833 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.833 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.833 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.315097+0000", 2024-09-13T10:11:59.833 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000104685 2024-09-13T10:11:59.833 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.833 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.833 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.833 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.316808+0000", 2024-09-13T10:11:59.833 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0017110529999999999 2024-09-13T10:11:59.833 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.833 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.834 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.834 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.316946+0000", 2024-09-13T10:11:59.834 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000137737 2024-09-13T10:11:59.834 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.834 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.834 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.834 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.317028+0000", 2024-09-13T10:11:59.834 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.1607999999999995e-05 2024-09-13T10:11:59.834 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.834 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.834 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.834 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.834 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.835 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:335 4.2 4:4a24ecac:::smithi19530714-36:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:59.835 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:11:59.835 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 68.501797646, 2024-09-13T10:11:59.835 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032260373000000002, 2024-09-13T10:11:59.835 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.835 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.835 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.835 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.835 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.835 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.835 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 335 2024-09-13T10:11:59.835 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.835 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.836 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.836 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.836 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:11:59.836 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.836 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.836 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.836 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.836 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:11:59.836 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.836 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.836 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.837 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.837 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282494+0000", 2024-09-13T10:11:59.837 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5390000000000002e-06 2024-09-13T10:11:59.837 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.837 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.837 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.837 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282506+0000", 2024-09-13T10:11:59.837 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2646000000000001e-05 2024-09-13T10:11:59.837 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.837 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.837 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.837 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282539+0000", 2024-09-13T10:11:59.837 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3176000000000002e-05 2024-09-13T10:11:59.838 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.838 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.838 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.838 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282558+0000", 2024-09-13T10:11:59.838 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9065000000000001e-05 2024-09-13T10:11:59.838 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.838 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.838 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.838 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282697+0000", 2024-09-13T10:11:59.838 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013857999999999999 2024-09-13T10:11:59.838 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.838 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.838 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.838 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282874+0000", 2024-09-13T10:11:59.839 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017656900000000001 2024-09-13T10:11:59.839 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.839 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.839 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,14", 2024-09-13T10:11:59.839 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.287911+0000", 2024-09-13T10:11:59.839 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0050369220000000001 2024-09-13T10:11:59.839 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.839 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.839 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.839 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.290491+0000", 2024-09-13T10:11:59.839 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025802500000000001 2024-09-13T10:11:59.839 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.839 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.839 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.840 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.291374+0000", 2024-09-13T10:11:59.840 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00088367800000000005 2024-09-13T10:11:59.840 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.840 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.840 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.840 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314557+0000", 2024-09-13T10:11:59.840 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023182094 2024-09-13T10:11:59.840 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.840 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.840 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.840 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314664+0000", 2024-09-13T10:11:59.840 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010751099999999999 2024-09-13T10:11:59.840 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.841 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.841 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.841 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314751+0000", 2024-09-13T10:11:59.841 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7342999999999994e-05 2024-09-13T10:11:59.841 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.841 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.841 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.841 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.841 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.841 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:337 4.4 4:24275569:::smithi19530714-38:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:11:59.841 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:11:59.841 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 68.501141094999994, 2024-09-13T10:11:59.841 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.031660147, 2024-09-13T10:11:59.842 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.842 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.842 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:11:59.842 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.842 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.842 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.842 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 337 2024-09-13T10:11:59.842 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.842 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.842 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.842 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.842 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:11:59.842 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.843 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.843 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.843 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.843 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:11:59.843 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.843 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.843 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.843 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.843 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283149+0000", 2024-09-13T10:11:59.843 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.655e-06 2024-09-13T10:11:59.843 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.843 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.843 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.844 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283159+0000", 2024-09-13T10:11:59.844 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.9529999999999994e-06 2024-09-13T10:11:59.844 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.844 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.844 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.844 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283177+0000", 2024-09-13T10:11:59.844 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8168e-05 2024-09-13T10:11:59.844 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.844 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.844 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.844 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283193+0000", 2024-09-13T10:11:59.844 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5563e-05 2024-09-13T10:11:59.845 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.845 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.845 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.845 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283317+0000", 2024-09-13T10:11:59.845 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012421700000000001 2024-09-13T10:11:59.845 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.845 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.845 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.845 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283478+0000", 2024-09-13T10:11:59.845 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000160378 2024-09-13T10:11:59.845 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.845 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.846 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:11:59.846 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.289593+0000", 2024-09-13T10:11:59.846 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0061152200000000002 2024-09-13T10:11:59.846 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.846 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.846 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.846 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.291636+0000", 2024-09-13T10:11:59.846 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020435430000000001 2024-09-13T10:11:59.846 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.846 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.846 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:11:59.846 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294186+0000", 2024-09-13T10:11:59.846 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025494760000000002 2024-09-13T10:11:59.846 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.847 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.847 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:11:59.847 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314672+0000", 2024-09-13T10:11:59.847 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020486392999999999 2024-09-13T10:11:59.847 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.847 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.847 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:11:59.847 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314741+0000", 2024-09-13T10:11:59.847 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9003000000000003e-05 2024-09-13T10:11:59.847 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.847 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.847 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.847 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314808+0000", 2024-09-13T10:11:59.848 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.6577999999999999e-05 2024-09-13T10:11:59.848 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.848 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.848 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.848 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.848 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.848 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1865 4.12 4:4ac406d9:::smithi19530714-26:head [read 0~2050454,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:59.848 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.728406+0000", 2024-09-13T10:11:59.848 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 48.055882519999997, 2024-09-13T10:11:59.848 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029384652000000001, 2024-09-13T10:11:59.848 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.848 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.848 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.849 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.849 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.849 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.849 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1865 2024-09-13T10:11:59.849 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.849 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.849 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.849 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.849 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728406+0000", 2024-09-13T10:11:59.849 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.849 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.849 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.850 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.850 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728406+0000", 2024-09-13T10:11:59.850 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.850 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.850 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.850 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.850 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728410+0000", 2024-09-13T10:11:59.850 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9129999999999996e-06 2024-09-13T10:11:59.850 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.850 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.850 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.850 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728423+0000", 2024-09-13T10:11:59.850 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2425999999999999e-05 2024-09-13T10:11:59.851 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.851 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.851 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.851 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728453+0000", 2024-09-13T10:11:59.851 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0673999999999998e-05 2024-09-13T10:11:59.851 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.851 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.851 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.851 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728478+0000", 2024-09-13T10:11:59.851 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5099000000000001e-05 2024-09-13T10:11:59.851 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.851 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.852 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.852 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730729+0000", 2024-09-13T10:11:59.852 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002251009 2024-09-13T10:11:59.852 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.852 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.852 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:11:59.852 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730758+0000", 2024-09-13T10:11:59.852 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8623e-05 2024-09-13T10:11:59.852 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.852 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.852 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.852 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.754517+0000", 2024-09-13T10:11:59.852 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023759199000000002 2024-09-13T10:11:59.853 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.853 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.853 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.853 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.754761+0000", 2024-09-13T10:11:59.853 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000243923 2024-09-13T10:11:59.853 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.853 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.853 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.853 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.757791+0000", 2024-09-13T10:11:59.853 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003029786 2024-09-13T10:11:59.854 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.854 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.854 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.854 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.854 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.854 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1866 4.12 4:4ac406d9:::smithi19530714-26:head [sparse-read 0~1216897] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:59.854 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.728515+0000", 2024-09-13T10:11:59.854 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 48.055773465000001, 2024-09-13T10:11:59.854 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.031206358999999999, 2024-09-13T10:11:59.854 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.854 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.854 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.854 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.855 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.855 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.855 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1866 2024-09-13T10:11:59.855 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.855 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.855 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.855 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.855 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728515+0000", 2024-09-13T10:11:59.855 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.855 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.855 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.855 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.855 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728515+0000", 2024-09-13T10:11:59.856 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.856 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.856 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.856 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.856 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728516+0000", 2024-09-13T10:11:59.856 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1429999999999999e-06 2024-09-13T10:11:59.856 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.856 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.856 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.856 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728522+0000", 2024-09-13T10:11:59.856 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5740000000000003e-06 2024-09-13T10:11:59.856 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.856 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.857 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.857 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728534+0000", 2024-09-13T10:11:59.857 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1539000000000001e-05 2024-09-13T10:11:59.857 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.857 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.857 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.857 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728543+0000", 2024-09-13T10:11:59.857 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6059999999999992e-06 2024-09-13T10:11:59.857 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.857 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.857 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.857 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730840+0000", 2024-09-13T10:11:59.857 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022966789999999998 2024-09-13T10:11:59.857 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.858 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.858 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:11:59.858 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730859+0000", 2024-09-13T10:11:59.858 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9613e-05 2024-09-13T10:11:59.858 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.858 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.858 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.858 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.758405+0000", 2024-09-13T10:11:59.858 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027545656000000002 2024-09-13T10:11:59.858 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.858 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.858 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.858 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.758574+0000", 2024-09-13T10:11:59.858 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000168808 2024-09-13T10:11:59.859 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.859 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.859 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.859 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759722+0000", 2024-09-13T10:11:59.859 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011477410000000001 2024-09-13T10:11:59.859 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.859 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.859 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.859 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.859 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.859 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1867 4.12 4:4ac406d9:::smithi19530714-26:head [read 116628~1933826,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:11:59.859 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.728557+0000", 2024-09-13T10:11:59.860 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 48.055731983000001, 2024-09-13T10:11:59.860 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033656772000000001, 2024-09-13T10:11:59.860 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.860 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.860 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.860 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.860 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.860 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.860 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1867 2024-09-13T10:11:59.860 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.860 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.860 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.861 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.861 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728557+0000", 2024-09-13T10:11:59.861 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.861 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.861 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.861 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.861 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728557+0000", 2024-09-13T10:11:59.861 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.861 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.861 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.861 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.861 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728558+0000", 2024-09-13T10:11:59.862 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8400000000000003e-07 2024-09-13T10:11:59.862 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.862 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.862 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.862 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728562+0000", 2024-09-13T10:11:59.862 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.8029999999999996e-06 2024-09-13T10:11:59.862 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.862 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.862 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.862 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728589+0000", 2024-09-13T10:11:59.862 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6536000000000001e-05 2024-09-13T10:11:59.862 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.862 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.863 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.863 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728603+0000", 2024-09-13T10:11:59.863 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4483e-05 2024-09-13T10:11:59.863 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.863 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.863 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.863 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730933+0000", 2024-09-13T10:11:59.863 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023295939999999999 2024-09-13T10:11:59.863 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.863 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.863 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:11:59.863 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730952+0000", 2024-09-13T10:11:59.864 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8614000000000002e-05 2024-09-13T10:11:59.864 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.864 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.864 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.864 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759881+0000", 2024-09-13T10:11:59.864 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.02892962 2024-09-13T10:11:59.864 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.864 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.864 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.864 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.760061+0000", 2024-09-13T10:11:59.864 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00018033 2024-09-13T10:11:59.864 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.864 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.864 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.865 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.762214+0000", 2024-09-13T10:11:59.865 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002152008 2024-09-13T10:11:59.865 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.865 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.865 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.865 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.865 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.865 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1932 4.5 4:a3098556:::smithi19530714-22:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:11:59.865 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.759351+0000", 2024-09-13T10:11:59.865 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 48.024937569999999, 2024-09-13T10:11:59.865 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.043979880999999998, 2024-09-13T10:11:59.865 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.866 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.866 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.866 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.866 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.866 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.866 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1932 2024-09-13T10:11:59.866 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.866 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.866 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.866 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.866 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759351+0000", 2024-09-13T10:11:59.866 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.866 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.866 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.867 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.867 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759351+0000", 2024-09-13T10:11:59.867 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.867 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.867 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.867 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.867 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759352+0000", 2024-09-13T10:11:59.867 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.28e-06 2024-09-13T10:11:59.867 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.867 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.867 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.867 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759358+0000", 2024-09-13T10:11:59.867 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8699999999999997e-06 2024-09-13T10:11:59.867 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.868 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.868 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.868 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759376+0000", 2024-09-13T10:11:59.868 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7532000000000001e-05 2024-09-13T10:11:59.868 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.868 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.868 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.868 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759391+0000", 2024-09-13T10:11:59.868 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5590000000000002e-05 2024-09-13T10:11:59.868 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.868 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.868 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.868 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759607+0000", 2024-09-13T10:11:59.868 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021536999999999999 2024-09-13T10:11:59.868 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.869 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.869 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.869 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759831+0000", 2024-09-13T10:11:59.869 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000224243 2024-09-13T10:11:59.869 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.869 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.869 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.869 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.803331+0000", 2024-09-13T10:11:59.869 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.043499995999999999 2024-09-13T10:11:59.869 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.869 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.869 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.869 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.869 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.870 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2189 4.4 4:26b855f3:::smithi19530714-25:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:11:59.870 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:17.462885+0000", 2024-09-13T10:11:59.870 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 42.321403863999997, 2024-09-13T10:11:59.870 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041082475, 2024-09-13T10:11:59.870 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.870 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.870 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.870 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.870 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.870 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.870 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2189 2024-09-13T10:11:59.870 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.870 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.870 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.871 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.871 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462885+0000", 2024-09-13T10:11:59.871 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.871 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.871 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.871 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.871 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462885+0000", 2024-09-13T10:11:59.871 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.871 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.871 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.871 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.871 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462888+0000", 2024-09-13T10:11:59.872 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.74e-06 2024-09-13T10:11:59.872 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.872 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.872 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.872 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462898+0000", 2024-09-13T10:11:59.872 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0618999999999999e-05 2024-09-13T10:11:59.872 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.872 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.872 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.872 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462928+0000", 2024-09-13T10:11:59.872 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9685000000000001e-05 2024-09-13T10:11:59.872 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.873 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.873 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.873 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462944+0000", 2024-09-13T10:11:59.873 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6175999999999999e-05 2024-09-13T10:11:59.873 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.873 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.873 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.873 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.463063+0000", 2024-09-13T10:11:59.873 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000119393 2024-09-13T10:11:59.873 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.873 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.873 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.874 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.463204+0000", 2024-09-13T10:11:59.874 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014020600000000001 2024-09-13T10:11:59.874 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.874 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.874 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.874 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.503967+0000", 2024-09-13T10:11:59.874 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.040763656000000002 2024-09-13T10:11:59.874 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.874 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.874 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.874 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.874 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.874 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2832 4.4 4:26b855f3:::smithi19530714-25:head [read 0~3613505,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:59.875 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.892960+0000", 2024-09-13T10:11:59.875 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.891328598000001, 2024-09-13T10:11:59.875 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.026786087, 2024-09-13T10:11:59.875 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.875 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.875 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.875 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.875 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.875 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.875 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2832 2024-09-13T10:11:59.875 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.875 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.875 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.876 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.876 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892960+0000", 2024-09-13T10:11:59.876 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.876 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.876 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.876 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.876 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892960+0000", 2024-09-13T10:11:59.876 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.876 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.876 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.876 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.876 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892962+0000", 2024-09-13T10:11:59.877 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.573e-06 2024-09-13T10:11:59.877 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.877 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.877 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.877 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892969+0000", 2024-09-13T10:11:59.877 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.277e-06 2024-09-13T10:11:59.877 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.877 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.877 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.877 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892984+0000", 2024-09-13T10:11:59.877 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5013e-05 2024-09-13T10:11:59.878 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.878 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.878 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.878 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892997+0000", 2024-09-13T10:11:59.878 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2692e-05 2024-09-13T10:11:59.878 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.878 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.878 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.878 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.917535+0000", 2024-09-13T10:11:59.878 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024538806 2024-09-13T10:11:59.878 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.878 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.879 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.879 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.917788+0000", 2024-09-13T10:11:59.879 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025253100000000001 2024-09-13T10:11:59.879 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.879 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.879 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.879 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.919746+0000", 2024-09-13T10:11:59.879 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019581949999999998 2024-09-13T10:11:59.879 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.879 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.879 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.879 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.879 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.879 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2833 4.4 4:26b855f3:::smithi19530714-25:head [read 2395497~1218008,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:59.880 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.893015+0000", 2024-09-13T10:11:59.880 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.891273793, 2024-09-13T10:11:59.880 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027607530000000002, 2024-09-13T10:11:59.880 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.880 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.880 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.880 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.880 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.880 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.880 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2833 2024-09-13T10:11:59.880 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.880 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.880 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.881 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.881 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893015+0000", 2024-09-13T10:11:59.881 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.881 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.881 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.881 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.881 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893015+0000", 2024-09-13T10:11:59.881 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.881 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.881 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.881 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.881 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893016+0000", 2024-09-13T10:11:59.881 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.156e-06 2024-09-13T10:11:59.882 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.882 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.882 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.882 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893024+0000", 2024-09-13T10:11:59.882 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5900000000000002e-06 2024-09-13T10:11:59.882 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.882 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.882 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.882 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893039+0000", 2024-09-13T10:11:59.882 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4949999999999999e-05 2024-09-13T10:11:59.882 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.882 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.882 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.883 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893052+0000", 2024-09-13T10:11:59.883 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3362e-05 2024-09-13T10:11:59.883 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.883 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.883 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.883 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.919791+0000", 2024-09-13T10:11:59.883 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.026739237999999999 2024-09-13T10:11:59.883 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.883 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.883 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.883 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.919930+0000", 2024-09-13T10:11:59.883 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000139202 2024-09-13T10:11:59.883 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.884 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.884 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.884 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.920622+0000", 2024-09-13T10:11:59.884 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00069203199999999998 2024-09-13T10:11:59.884 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.884 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.884 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.884 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.884 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.884 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2840 4.4 4:24da8222:::smithi19530714-49:head [sparse-read 2104672~451177,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:59.884 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.893098+0000", 2024-09-13T10:11:59.884 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.891190225999999, 2024-09-13T10:11:59.884 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029549518, 2024-09-13T10:11:59.885 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.885 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.885 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.885 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.885 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.885 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.885 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2840 2024-09-13T10:11:59.885 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.885 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.885 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.885 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.885 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893098+0000", 2024-09-13T10:11:59.886 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.886 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.886 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.886 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.886 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893098+0000", 2024-09-13T10:11:59.886 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.886 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.886 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.886 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.886 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893107+0000", 2024-09-13T10:11:59.886 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7430000000000008e-06 2024-09-13T10:11:59.886 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.886 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.886 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.886 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893117+0000", 2024-09-13T10:11:59.887 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7000000000000003e-06 2024-09-13T10:11:59.887 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.887 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.887 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.887 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893136+0000", 2024-09-13T10:11:59.887 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8705000000000001e-05 2024-09-13T10:11:59.887 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.887 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.887 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.887 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893151+0000", 2024-09-13T10:11:59.887 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5183e-05 2024-09-13T10:11:59.887 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.887 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.888 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.888 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.920721+0000", 2024-09-13T10:11:59.888 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027569917999999999 2024-09-13T10:11:59.888 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.888 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.888 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.888 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.920886+0000", 2024-09-13T10:11:59.888 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016508699999999999 2024-09-13T10:11:59.888 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.888 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.888 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.888 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.922648+0000", 2024-09-13T10:11:59.889 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0017621819999999999 2024-09-13T10:11:59.889 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.889 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.889 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.889 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.889 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.889 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2841 4.4 4:24da8222:::smithi19530714-49:head [read 0~3042067,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:59.889 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.893173+0000", 2024-09-13T10:11:59.889 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.891115804999998, 2024-09-13T10:11:59.889 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.037712765000000002, 2024-09-13T10:11:59.889 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.889 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.890 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.890 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.890 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.890 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.890 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2841 2024-09-13T10:11:59.890 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.890 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.890 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.890 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.890 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893173+0000", 2024-09-13T10:11:59.890 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.890 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.891 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.891 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.891 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893173+0000", 2024-09-13T10:11:59.891 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.891 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.891 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.891 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.891 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893174+0000", 2024-09-13T10:11:59.891 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4959999999999999e-06 2024-09-13T10:11:59.891 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.891 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.891 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.891 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893182+0000", 2024-09-13T10:11:59.891 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7060000000000004e-06 2024-09-13T10:11:59.892 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.892 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.892 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.892 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893201+0000", 2024-09-13T10:11:59.892 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8655999999999999e-05 2024-09-13T10:11:59.892 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.892 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.892 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.892 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893215+0000", 2024-09-13T10:11:59.892 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4513e-05 2024-09-13T10:11:59.892 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.892 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.893 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.893 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.922869+0000", 2024-09-13T10:11:59.893 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029653845000000002 2024-09-13T10:11:59.893 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.893 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.893 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.893 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.923431+0000", 2024-09-13T10:11:59.893 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00056212899999999997 2024-09-13T10:11:59.893 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.893 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.893 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.893 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.930886+0000", 2024-09-13T10:11:59.893 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0074544199999999998 2024-09-13T10:11:59.894 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.894 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.894 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.894 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.894 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.894 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2842 4.4 4:24da8222:::smithi19530714-49:head [sparse-read 0~1861544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:11:59.894 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.893233+0000", 2024-09-13T10:11:59.894 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.891055325, 2024-09-13T10:11:59.894 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.038772649999999999, 2024-09-13T10:11:59.894 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.894 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.894 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.894 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.894 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.895 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.895 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2842 2024-09-13T10:11:59.895 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.895 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.895 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.895 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.895 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893233+0000", 2024-09-13T10:11:59.895 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.895 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.895 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.895 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.895 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893233+0000", 2024-09-13T10:11:59.896 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.896 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.896 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.896 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.896 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893235+0000", 2024-09-13T10:11:59.896 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.217e-06 2024-09-13T10:11:59.896 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.896 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.896 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.896 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893240+0000", 2024-09-13T10:11:59.896 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.6999999999999996e-06 2024-09-13T10:11:59.896 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.897 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.897 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.897 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893254+0000", 2024-09-13T10:11:59.897 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.401e-05 2024-09-13T10:11:59.897 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.897 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.897 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.897 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893267+0000", 2024-09-13T10:11:59.897 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3135e-05 2024-09-13T10:11:59.897 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.897 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.897 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.897 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.930989+0000", 2024-09-13T10:11:59.897 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.037721452000000003 2024-09-13T10:11:59.898 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.898 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.898 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.898 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.931214+0000", 2024-09-13T10:11:59.898 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000225529 2024-09-13T10:11:59.898 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.898 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.898 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.898 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.932006+0000", 2024-09-13T10:11:59.898 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00079160700000000005 2024-09-13T10:11:59.898 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.898 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.898 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.899 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.899 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.899 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3003 4.c 4:3e6706e5:::smithi19530714-14:head [read 0~2084101,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e75)", 2024-09-13T10:11:59.899 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:37.319096+0000", 2024-09-13T10:11:59.899 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 22.465193014, 2024-09-13T10:11:59.899 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024544567, 2024-09-13T10:11:59.899 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:11:59.899 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:11:59.899 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:11:59.899 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:11:59.899 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:11:59.899 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:11:59.899 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3003 2024-09-13T10:11:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:11:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:11:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.319096+0000", 2024-09-13T10:11:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:11:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.319096+0000", 2024-09-13T10:11:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:11:59.900 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:11:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.319097+0000", 2024-09-13T10:11:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5999999999999999e-06 2024-09-13T10:11:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:11:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.319107+0000", 2024-09-13T10:11:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5429999999999999e-06 2024-09-13T10:11:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:11:59.901 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.319124+0000", 2024-09-13T10:11:59.902 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7436e-05 2024-09-13T10:11:59.902 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.902 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.902 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:11:59.902 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.319138+0000", 2024-09-13T10:11:59.902 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4209999999999999e-05 2024-09-13T10:11:59.902 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.902 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.902 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:11:59.902 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.340175+0000", 2024-09-13T10:11:59.902 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021036837999999999 2024-09-13T10:11:59.902 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.902 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.903 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:11:59.903 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.340370+0000", 2024-09-13T10:11:59.903 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019481000000000001 2024-09-13T10:11:59.903 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:11:59.903 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:11:59.903 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:11:59.903 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:37.343640+0000", 2024-09-13T10:11:59.903 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00327013 2024-09-13T10:11:59.903 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.903 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.903 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.903 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:11:59.903 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:11:59.904 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:11:59.905 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 825805 2024-09-13T10:11:59.905 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-13T10:11:59.906 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:11:59.899+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:11:59.977 INFO:tasks.rados.rados.0.smithi195.stdout:1665: expect (ObjNum 235 snap 0 seq_num 235) 2024-09-13T10:12:00.006 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:12:00.003+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:12:00.018 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3200152 2024-09-13T10:12:00.049 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2210252 2024-09-13T10:12:00.075 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:00.076 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:12:00.076 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:12:00.076 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:00.087 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:00.090 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-13T10:12:00.107 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:12:00.099+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:12:00.153 INFO:tasks.rados.rados.0.smithi195.stdout:1664: finishing set_chunk to oid 9 2024-09-13T10:12:00.153 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 123 (ObjNum 259 snap 0 seq_num 259) dirty exists 2024-09-13T10:12:00.153 INFO:tasks.rados.rados.0.smithi195.stdout:1666: finishing write tid 1 to smithi19530714-10 2024-09-13T10:12:00.153 INFO:tasks.rados.rados.0.smithi195.stdout:1666: finishing write tid 2 to smithi19530714-10 2024-09-13T10:12:00.153 INFO:tasks.rados.rados.0.smithi195.stdout:1667: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:12:00.206 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1963745 2024-09-13T10:12:00.208 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:12:00.203+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:12:00.213 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:00.213 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:12:00.213 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:12:00.213 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:12:00.213 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:00.220 INFO:tasks.rados.rados.0.smithi195.stdout:1666: finishing write tid 3 to smithi19530714-10 2024-09-13T10:12:00.220 INFO:tasks.rados.rados.0.smithi195.stdout:1666: finishing write tid 4 to smithi19530714-10 2024-09-13T10:12:00.220 INFO:tasks.rados.rados.0.smithi195.stdout:1666: finishing write tid 5 to smithi19530714-10 2024-09-13T10:12:00.221 INFO:tasks.rados.rados.0.smithi195.stdout:1666: oid 10 updating version 0 to 28 2024-09-13T10:12:00.221 INFO:tasks.rados.rados.0.smithi195.stdout:1666: oid 10 updating version 28 to 31 2024-09-13T10:12:00.221 INFO:tasks.rados.rados.0.smithi195.stdout:1666: oid 10 version 31 is already newer than 30 2024-09-13T10:12:00.221 INFO:tasks.rados.rados.0.smithi195.stdout:1666: oid 10 version 31 is already newer than 29 2024-09-13T10:12:00.221 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 31 (ObjNum 355 snap 0 seq_num 355) dirty exists 2024-09-13T10:12:00.221 INFO:tasks.rados.rados.0.smithi195.stdout:1666: left oid 10 (ObjNum 355 snap 0 seq_num 355) 2024-09-13T10:12:00.221 INFO:tasks.rados.rados.0.smithi195.stdout:1668: finishing write tid 1 to smithi19530714-36 2024-09-13T10:12:00.221 INFO:tasks.rados.rados.0.smithi195.stdout:1668: finishing write tid 2 to smithi19530714-36 2024-09-13T10:12:00.221 INFO:tasks.rados.rados.0.smithi195.stdout:1672: tier_flush oid 5 snap -1 2024-09-13T10:12:00.221 INFO:tasks.rados.rados.0.smithi195.stdout:1668: finishing write tid 3 to smithi19530714-36 2024-09-13T10:12:00.221 INFO:tasks.rados.rados.0.smithi195.stdout:1668: finishing write tid 4 to smithi19530714-36 2024-09-13T10:12:00.221 INFO:tasks.rados.rados.0.smithi195.stdout:1668: finishing write tid 5 to smithi19530714-36 2024-09-13T10:12:00.222 INFO:tasks.rados.rados.0.smithi195.stdout:1668: oid 36 updating version 0 to 49 2024-09-13T10:12:00.222 INFO:tasks.rados.rados.0.smithi195.stdout:1668: oid 36 version 49 is already newer than 46 2024-09-13T10:12:00.222 INFO:tasks.rados.rados.0.smithi195.stdout:1668: oid 36 version 49 is already newer than 47 2024-09-13T10:12:00.222 INFO:tasks.rados.rados.0.smithi195.stdout:1668: oid 36 version 49 is already newer than 48 2024-09-13T10:12:00.222 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 36 v 49 (ObjNum 356 snap 0 seq_num 356) dirty exists 2024-09-13T10:12:00.222 INFO:tasks.rados.rados.0.smithi195.stdout:1668: left oid 36 (ObjNum 356 snap 0 seq_num 356) 2024-09-13T10:12:00.222 INFO:tasks.rados.rados.0.smithi195.stdout:1669: expect (ObjNum 280 snap 0 seq_num 280) 2024-09-13T10:12:00.222 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:00.228 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-13T10:12:00.291 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1155093 2024-09-13T10:12:00.308 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:00.305+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:00.365 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:00.365 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:12:00.365 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:12:00.365 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:12:00.366 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.366 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:201 3.3 3:d4fb61be:::smithi19530714-2 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.366 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:12:00.366 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.367795285, 2024-09-13T10:12:00.366 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.052937043000000003, 2024-09-13T10:12:00.366 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.366 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.366 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.366 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.366 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.366 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.367 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 201 2024-09-13T10:12:00.367 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.367 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.367 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.367 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.367 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:12:00.367 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.367 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.367 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.367 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.367 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:12:00.367 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.368 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.368 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.368 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.368 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993192+0000", 2024-09-13T10:12:00.368 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8559999999999996e-06 2024-09-13T10:12:00.368 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.368 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.368 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.368 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993203+0000", 2024-09-13T10:12:00.368 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1113e-05 2024-09-13T10:12:00.368 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.368 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.368 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.369 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993225+0000", 2024-09-13T10:12:00.369 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2793e-05 2024-09-13T10:12:00.369 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.369 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.369 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.369 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993241+0000", 2024-09-13T10:12:00.369 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5806000000000001e-05 2024-09-13T10:12:00.369 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.369 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.369 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.369 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993401+0000", 2024-09-13T10:12:00.369 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000160149 2024-09-13T10:12:00.369 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.370 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.370 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.370 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994047+0000", 2024-09-13T10:12:00.370 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00064594100000000003 2024-09-13T10:12:00.370 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.370 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.370 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.370 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.011243+0000", 2024-09-13T10:12:00.370 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017196026999999999 2024-09-13T10:12:00.370 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.370 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.370 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.370 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029096+0000", 2024-09-13T10:12:00.370 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017852115000000002 2024-09-13T10:12:00.371 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.371 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.371 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.371 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029271+0000", 2024-09-13T10:12:00.371 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017589299999999999 2024-09-13T10:12:00.371 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.371 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.371 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.371 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.045867+0000", 2024-09-13T10:12:00.371 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016595625999999999 2024-09-13T10:12:00.371 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.371 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.371 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.371 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.046017+0000", 2024-09-13T10:12:00.372 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014974199999999999 2024-09-13T10:12:00.372 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.372 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.372 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.372 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.046120+0000", 2024-09-13T10:12:00.372 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000102982 2024-09-13T10:12:00.372 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.372 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.372 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.372 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.372 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.372 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:208 3.3 3:dbb0d8aa:::smithi19530714-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.373 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:12:00.373 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.367699474999995, 2024-09-13T10:12:00.373 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.071216475000000001, 2024-09-13T10:12:00.373 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.373 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.373 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.373 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.373 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.373 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.373 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 208 2024-09-13T10:12:00.373 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.373 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.373 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.374 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.374 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:12:00.374 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.374 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.374 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.374 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.374 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:12:00.374 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.374 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.374 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.374 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.374 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993280+0000", 2024-09-13T10:12:00.374 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1999999999999999e-06 2024-09-13T10:12:00.375 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.375 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.375 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.375 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993289+0000", 2024-09-13T10:12:00.375 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8200000000000003e-06 2024-09-13T10:12:00.375 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.375 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.375 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.375 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994000+0000", 2024-09-13T10:12:00.375 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00071192499999999997 2024-09-13T10:12:00.375 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.375 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.375 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.375 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994024+0000", 2024-09-13T10:12:00.376 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3578999999999999e-05 2024-09-13T10:12:00.376 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.376 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.376 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.376 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994468+0000", 2024-09-13T10:12:00.376 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00044410599999999999 2024-09-13T10:12:00.376 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.376 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.376 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.376 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994842+0000", 2024-09-13T10:12:00.376 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037380300000000003 2024-09-13T10:12:00.376 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.376 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.377 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.377 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031572+0000", 2024-09-13T10:12:00.377 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036730479000000003 2024-09-13T10:12:00.377 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.377 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.377 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.377 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.039007+0000", 2024-09-13T10:12:00.377 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.007434871 2024-09-13T10:12:00.377 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.377 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.377 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.377 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.041001+0000", 2024-09-13T10:12:00.377 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001994104 2024-09-13T10:12:00.378 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.378 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.378 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.378 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064330+0000", 2024-09-13T10:12:00.378 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023328912 2024-09-13T10:12:00.378 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.378 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.378 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.378 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064401+0000", 2024-09-13T10:12:00.378 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.1123000000000003e-05 2024-09-13T10:12:00.378 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.378 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.378 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.379 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064495+0000", 2024-09-13T10:12:00.379 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.3553000000000004e-05 2024-09-13T10:12:00.379 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.379 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.379 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.379 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.379 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.379 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:215 3.3 3:ed6c0a17:::smithi19530714-16 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.379 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:12:00.379 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.366919221000003, 2024-09-13T10:12:00.379 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.055806517999999999, 2024-09-13T10:12:00.379 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.379 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.380 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.380 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.380 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.380 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.380 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 215 2024-09-13T10:12:00.380 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.380 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.380 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.380 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.380 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:12:00.380 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.381 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.381 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.381 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:12:00.381 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.381 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.381 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.381 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994060+0000", 2024-09-13T10:12:00.381 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5030000000000001e-06 2024-09-13T10:12:00.381 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.381 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.382 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.382 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994070+0000", 2024-09-13T10:12:00.382 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0046e-05 2024-09-13T10:12:00.382 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.382 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.382 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.382 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994087+0000", 2024-09-13T10:12:00.382 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6283999999999999e-05 2024-09-13T10:12:00.382 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.382 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.382 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.382 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994096+0000", 2024-09-13T10:12:00.382 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.969e-06 2024-09-13T10:12:00.382 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.383 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.383 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.383 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.995226+0000", 2024-09-13T10:12:00.383 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011299540000000001 2024-09-13T10:12:00.383 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.383 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.383 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.383 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.995548+0000", 2024-09-13T10:12:00.383 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00032266199999999998 2024-09-13T10:12:00.383 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.383 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.383 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.383 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.018922+0000", 2024-09-13T10:12:00.384 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023373485999999999 2024-09-13T10:12:00.384 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.384 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.384 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.384 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029400+0000", 2024-09-13T10:12:00.384 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010478721 2024-09-13T10:12:00.384 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.384 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.384 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.384 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038932+0000", 2024-09-13T10:12:00.384 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0095316859999999993 2024-09-13T10:12:00.384 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.385 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.385 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.385 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049620+0000", 2024-09-13T10:12:00.385 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010687688000000001 2024-09-13T10:12:00.385 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.385 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.385 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.385 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049754+0000", 2024-09-13T10:12:00.385 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013457700000000001 2024-09-13T10:12:00.385 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.385 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.385 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.385 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049865+0000", 2024-09-13T10:12:00.386 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110942 2024-09-13T10:12:00.386 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.386 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.386 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.386 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.386 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.386 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:224 3.3 3:f941658c:::smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.386 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:12:00.386 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.310411184000003, 2024-09-13T10:12:00.386 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.025014027000000001, 2024-09-13T10:12:00.386 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.386 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.386 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.386 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.387 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.387 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.387 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 224 2024-09-13T10:12:00.387 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.387 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.387 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.387 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.387 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:12:00.387 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.387 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.387 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.387 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.388 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:12:00.388 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.388 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.388 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.388 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.388 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050570+0000", 2024-09-13T10:12:00.388 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9799999999999998e-06 2024-09-13T10:12:00.388 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.388 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.388 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.388 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050582+0000", 2024-09-13T10:12:00.388 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2170999999999999e-05 2024-09-13T10:12:00.389 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.389 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.389 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.389 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050605+0000", 2024-09-13T10:12:00.389 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3543000000000001e-05 2024-09-13T10:12:00.389 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.389 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.389 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.389 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050619+0000", 2024-09-13T10:12:00.389 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3478999999999999e-05 2024-09-13T10:12:00.389 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.389 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.390 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.390 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050725+0000", 2024-09-13T10:12:00.390 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000105585 2024-09-13T10:12:00.390 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.390 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.390 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.390 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.051086+0000", 2024-09-13T10:12:00.390 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036162100000000001 2024-09-13T10:12:00.390 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.390 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.390 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.390 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.058628+0000", 2024-09-13T10:12:00.390 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075415569999999999 2024-09-13T10:12:00.391 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.391 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.391 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.391 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065344+0000", 2024-09-13T10:12:00.391 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0067161740000000001 2024-09-13T10:12:00.391 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.391 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.391 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.391 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.066222+0000", 2024-09-13T10:12:00.391 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00087765599999999999 2024-09-13T10:12:00.391 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.391 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.392 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.392 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075372+0000", 2024-09-13T10:12:00.392 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0091499609999999999 2024-09-13T10:12:00.392 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.392 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.392 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.392 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075489+0000", 2024-09-13T10:12:00.392 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011718300000000001 2024-09-13T10:12:00.392 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.392 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.392 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.392 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075581+0000", 2024-09-13T10:12:00.393 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.2116999999999999e-05 2024-09-13T10:12:00.393 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.393 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.393 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.393 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.393 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.393 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:226 3.3 3:fe5bfa3a:::smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.393 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:12:00.393 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.300878174999994, 2024-09-13T10:12:00.393 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051515848000000003, 2024-09-13T10:12:00.393 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.393 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.394 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.394 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.394 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.394 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.394 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 226 2024-09-13T10:12:00.394 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.394 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.394 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.394 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.394 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:12:00.394 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.394 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:12:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060102+0000", 2024-09-13T10:12:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0480000000000001e-06 2024-09-13T10:12:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.395 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060112+0000", 2024-09-13T10:12:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6649999999999998e-06 2024-09-13T10:12:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060132+0000", 2024-09-13T10:12:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0262e-05 2024-09-13T10:12:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060143+0000", 2024-09-13T10:12:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1420000000000001e-05 2024-09-13T10:12:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.396 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064563+0000", 2024-09-13T10:12:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044199119999999998 2024-09-13T10:12:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064882+0000", 2024-09-13T10:12:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000318735 2024-09-13T10:12:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.078858+0000", 2024-09-13T10:12:00.397 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013976455 2024-09-13T10:12:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088977+0000", 2024-09-13T10:12:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010118544 2024-09-13T10:12:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.090451+0000", 2024-09-13T10:12:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001474026 2024-09-13T10:12:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.398 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111539+0000", 2024-09-13T10:12:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.02108786 2024-09-13T10:12:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111580+0000", 2024-09-13T10:12:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1245000000000002e-05 2024-09-13T10:12:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111616+0000", 2024-09-13T10:12:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5676000000000002e-05 2024-09-13T10:12:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.399 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:235 3.3 3:e217e96e:::smithi19530714-36 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:12:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.290100546000005, 2024-09-13T10:12:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051184682000000002, 2024-09-13T10:12:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.400 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 235 2024-09-13T10:12:00.401 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.401 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.401 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.401 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.401 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:12:00.401 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.401 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.401 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.401 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.401 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:12:00.401 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.401 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.402 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.402 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.402 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070881+0000", 2024-09-13T10:12:00.402 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0579999999999999e-06 2024-09-13T10:12:00.402 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.402 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.402 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.402 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070897+0000", 2024-09-13T10:12:00.402 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5967000000000001e-05 2024-09-13T10:12:00.402 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.402 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.402 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.402 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070942+0000", 2024-09-13T10:12:00.403 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4947000000000002e-05 2024-09-13T10:12:00.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.403 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.403 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.403 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070963+0000", 2024-09-13T10:12:00.403 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0098e-05 2024-09-13T10:12:00.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.403 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.403 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.403 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.071099+0000", 2024-09-13T10:12:00.403 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013697199999999999 2024-09-13T10:12:00.403 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.403 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.404 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.404 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.071330+0000", 2024-09-13T10:12:00.404 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000230881 2024-09-13T10:12:00.404 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.404 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.404 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.404 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.084668+0000", 2024-09-13T10:12:00.404 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013337663 2024-09-13T10:12:00.404 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.404 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.404 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.404 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.092656+0000", 2024-09-13T10:12:00.404 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0079878880000000003 2024-09-13T10:12:00.405 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.405 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.405 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.405 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.100238+0000", 2024-09-13T10:12:00.405 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075819449999999997 2024-09-13T10:12:00.405 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.405 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.405 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.405 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121877+0000", 2024-09-13T10:12:00.405 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021639104999999999 2024-09-13T10:12:00.405 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.405 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.406 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.406 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121978+0000", 2024-09-13T10:12:00.406 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000101363 2024-09-13T10:12:00.406 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.406 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.406 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.406 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.122062+0000", 2024-09-13T10:12:00.406 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.3795000000000007e-05 2024-09-13T10:12:00.407 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.407 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.407 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.407 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.407 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.407 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:237 3.3 3:de40747e:::smithi19530714-38 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.407 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:12:00.407 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.281911342000001, 2024-09-13T10:12:00.407 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.073341331999999995, 2024-09-13T10:12:00.407 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.407 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.407 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.407 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.408 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.408 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.408 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 237 2024-09-13T10:12:00.408 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.408 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.408 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.408 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.408 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:12:00.408 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.408 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.408 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.408 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.408 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:12:00.409 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.409 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.409 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.409 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.409 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079069+0000", 2024-09-13T10:12:00.409 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6979999999999998e-06 2024-09-13T10:12:00.409 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.409 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.409 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.409 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079083+0000", 2024-09-13T10:12:00.409 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.384e-05 2024-09-13T10:12:00.409 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.410 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.410 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.410 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079105+0000", 2024-09-13T10:12:00.410 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1736e-05 2024-09-13T10:12:00.410 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.410 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.410 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.410 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079123+0000", 2024-09-13T10:12:00.410 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7889000000000001e-05 2024-09-13T10:12:00.410 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.410 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.410 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.410 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088459+0000", 2024-09-13T10:12:00.411 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0093359700000000007 2024-09-13T10:12:00.411 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.411 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.411 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.411 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088726+0000", 2024-09-13T10:12:00.411 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026732299999999997 2024-09-13T10:12:00.411 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.411 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.411 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.411 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.097522+0000", 2024-09-13T10:12:00.411 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0087962630000000003 2024-09-13T10:12:00.411 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.411 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.411 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.412 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.103972+0000", 2024-09-13T10:12:00.412 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0064494840000000001 2024-09-13T10:12:00.412 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.412 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.412 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.412 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.104160+0000", 2024-09-13T10:12:00.412 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00018782200000000001 2024-09-13T10:12:00.412 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.412 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.412 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.412 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152294+0000", 2024-09-13T10:12:00.412 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048134458999999997 2024-09-13T10:12:00.412 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.412 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.413 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.413 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152373+0000", 2024-09-13T10:12:00.413 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8574999999999999e-05 2024-09-13T10:12:00.413 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.413 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.413 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.413 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152408+0000", 2024-09-13T10:12:00.413 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5272999999999999e-05 2024-09-13T10:12:00.413 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.413 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.413 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.413 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.413 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.413 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:238 3.3 3:c43dd60b:::smithi19530714-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.414 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:12:00.414 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.272325695000006, 2024-09-13T10:12:00.414 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.069426360000000006, 2024-09-13T10:12:00.414 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.414 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.414 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.414 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.414 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.414 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.414 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 238 2024-09-13T10:12:00.414 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.414 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.414 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.415 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.415 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:12:00.415 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.415 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.415 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.415 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:12:00.415 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.415 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.415 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.415 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.415 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088655+0000", 2024-09-13T10:12:00.415 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2359999999999999e-06 2024-09-13T10:12:00.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.416 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.416 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.416 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088663+0000", 2024-09-13T10:12:00.416 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7800000000000006e-06 2024-09-13T10:12:00.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.416 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.416 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.416 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088688+0000", 2024-09-13T10:12:00.416 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4507000000000001e-05 2024-09-13T10:12:00.416 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.416 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.416 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.417 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088717+0000", 2024-09-13T10:12:00.417 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9204e-05 2024-09-13T10:12:00.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.417 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.417 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.417 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.089038+0000", 2024-09-13T10:12:00.417 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00032087199999999999 2024-09-13T10:12:00.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.417 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.417 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.417 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.089435+0000", 2024-09-13T10:12:00.417 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039756600000000002 2024-09-13T10:12:00.417 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.417 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.418 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.418 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.101672+0000", 2024-09-13T10:12:00.418 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012236528 2024-09-13T10:12:00.418 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.418 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.418 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.418 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.105256+0000", 2024-09-13T10:12:00.418 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0035837159999999998 2024-09-13T10:12:00.418 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.418 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.418 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.418 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111504+0000", 2024-09-13T10:12:00.418 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.006248577 2024-09-13T10:12:00.418 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.418 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.419 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.419 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.157950+0000", 2024-09-13T10:12:00.419 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.046446031999999998 2024-09-13T10:12:00.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.419 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.419 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.419 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158027+0000", 2024-09-13T10:12:00.419 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6402999999999993e-05 2024-09-13T10:12:00.419 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.419 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.419 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.419 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158079+0000", 2024-09-13T10:12:00.419 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.1938999999999999e-05 2024-09-13T10:12:00.419 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.420 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.420 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.420 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.420 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.420 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:246 3.3 3:f1be28e7:::smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.420 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:12:00.420 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.247614948000006, 2024-09-13T10:12:00.420 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051630810999999999, 2024-09-13T10:12:00.420 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.420 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.420 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.420 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.420 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.421 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.421 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 246 2024-09-13T10:12:00.421 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.421 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.421 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.421 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.421 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:12:00.421 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.421 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.421 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.421 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.421 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:12:00.421 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.422 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.422 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.422 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.422 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113365+0000", 2024-09-13T10:12:00.422 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.762e-06 2024-09-13T10:12:00.422 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.422 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.422 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.422 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113372+0000", 2024-09-13T10:12:00.422 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5020000000000003e-06 2024-09-13T10:12:00.422 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.422 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.422 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.423 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113390+0000", 2024-09-13T10:12:00.423 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8056000000000001e-05 2024-09-13T10:12:00.423 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.423 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.423 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.423 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113400+0000", 2024-09-13T10:12:00.423 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5119999999999997e-06 2024-09-13T10:12:00.423 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.423 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.423 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.423 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113456+0000", 2024-09-13T10:12:00.423 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.6221999999999998e-05 2024-09-13T10:12:00.423 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.424 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.424 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.424 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113576+0000", 2024-09-13T10:12:00.424 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000119466 2024-09-13T10:12:00.424 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.424 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.424 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.424 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.129127+0000", 2024-09-13T10:12:00.424 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015551003000000001 2024-09-13T10:12:00.424 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.424 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.424 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.424 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142482+0000", 2024-09-13T10:12:00.425 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013355104 2024-09-13T10:12:00.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.425 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.425 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.425 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142518+0000", 2024-09-13T10:12:00.425 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6532000000000002e-05 2024-09-13T10:12:00.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.425 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.425 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.425 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164919+0000", 2024-09-13T10:12:00.425 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022401038000000002 2024-09-13T10:12:00.425 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.426 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.426 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.426 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164962+0000", 2024-09-13T10:12:00.426 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.2329000000000001e-05 2024-09-13T10:12:00.426 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.426 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.426 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.426 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164994+0000", 2024-09-13T10:12:00.426 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2285e-05 2024-09-13T10:12:00.426 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.426 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.426 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.427 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.427 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.427 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:249 3.3 3:f565152a:::smithi19530714-50 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.427 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:12:00.427 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.246012471, 2024-09-13T10:12:00.427 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051233251, 2024-09-13T10:12:00.427 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.427 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.427 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.427 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.427 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.427 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.428 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 249 2024-09-13T10:12:00.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.428 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.428 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.428 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.428 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:12:00.428 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.428 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.428 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.428 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:12:00.428 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.428 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114967+0000", 2024-09-13T10:12:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.313e-06 2024-09-13T10:12:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114972+0000", 2024-09-13T10:12:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5620000000000003e-06 2024-09-13T10:12:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.429 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114997+0000", 2024-09-13T10:12:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4845999999999999e-05 2024-09-13T10:12:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115007+0000", 2024-09-13T10:12:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7200000000000001e-06 2024-09-13T10:12:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115076+0000", 2024-09-13T10:12:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9473000000000004e-05 2024-09-13T10:12:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.430 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.431 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.431 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115221+0000", 2024-09-13T10:12:00.431 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000144604 2024-09-13T10:12:00.431 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.431 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.431 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.431 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.136862+0000", 2024-09-13T10:12:00.431 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021641365999999999 2024-09-13T10:12:00.431 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.431 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.431 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.431 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.145775+0000", 2024-09-13T10:12:00.431 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0089130649999999995 2024-09-13T10:12:00.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.432 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.432 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.432 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.150578+0000", 2024-09-13T10:12:00.432 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0048025999999999998 2024-09-13T10:12:00.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.432 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.432 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.432 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166133+0000", 2024-09-13T10:12:00.432 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015554853 2024-09-13T10:12:00.432 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.432 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.432 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.433 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166167+0000", 2024-09-13T10:12:00.433 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3973999999999997e-05 2024-09-13T10:12:00.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.433 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.433 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.433 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166199+0000", 2024-09-13T10:12:00.433 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1875000000000002e-05 2024-09-13T10:12:00.433 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.433 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.433 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.433 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.433 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.434 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.6.23:1 3.3 3:d4fb61be:::smithi19530714-2 low_tier:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.434 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:12:00.434 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.240849421999997, 2024-09-13T10:12:00.434 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.038168746000000003, 2024-09-13T10:12:00.434 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.434 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.434 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.434 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.434 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.434 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.434 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:12:00.434 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.434 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:12:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120131+0000", 2024-09-13T10:12:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5620000000000002e-06 2024-09-13T10:12:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.435 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120140+0000", 2024-09-13T10:12:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1889999999999996e-06 2024-09-13T10:12:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120161+0000", 2024-09-13T10:12:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0485000000000001e-05 2024-09-13T10:12:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120200+0000", 2024-09-13T10:12:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9648999999999999e-05 2024-09-13T10:12:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.436 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.437 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.437 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120285+0000", 2024-09-13T10:12:00.437 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.4488999999999997e-05 2024-09-13T10:12:00.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.437 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.437 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.437 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120399+0000", 2024-09-13T10:12:00.437 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000113857 2024-09-13T10:12:00.437 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.437 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.437 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.437 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120640+0000", 2024-09-13T10:12:00.438 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024112600000000001 2024-09-13T10:12:00.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.438 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.438 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.438 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121715+0000", 2024-09-13T10:12:00.438 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001074885 2024-09-13T10:12:00.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.438 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.438 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.438 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.122870+0000", 2024-09-13T10:12:00.438 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00115504 2024-09-13T10:12:00.438 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.438 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.439 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.439 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158160+0000", 2024-09-13T10:12:00.439 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035289760000000003 2024-09-13T10:12:00.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.439 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.439 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.439 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158211+0000", 2024-09-13T10:12:00.439 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.1005999999999999e-05 2024-09-13T10:12:00.439 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.439 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.439 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.439 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158297+0000", 2024-09-13T10:12:00.440 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.6698000000000002e-05 2024-09-13T10:12:00.440 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.440 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.440 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.440 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.440 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.440 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.14.23:42 3.3 3:ce7ec0f0:::302c6a3bdf6d249958d305a37c5742c0de5d7cad722f2dabd19f570d2f795527:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:00.440 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:10.780924+0000", 2024-09-13T10:12:00.440 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 49.580053722000002, 2024-09-13T10:12:00.440 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013436228, 2024-09-13T10:12:00.440 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.440 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.780924+0000", 2024-09-13T10:12:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.780924+0000", 2024-09-13T10:12:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.441 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.780927+0000", 2024-09-13T10:12:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3860000000000001e-06 2024-09-13T10:12:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781100+0000", 2024-09-13T10:12:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000172967 2024-09-13T10:12:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781123+0000", 2024-09-13T10:12:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.302e-05 2024-09-13T10:12:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.442 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781141+0000", 2024-09-13T10:12:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8692999999999999e-05 2024-09-13T10:12:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.785827+0000", 2024-09-13T10:12:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0046856850000000002 2024-09-13T10:12:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.786033+0000", 2024-09-13T10:12:00.443 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00020591000000000001 2024-09-13T10:12:00.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.444 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.444 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.444 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.786402+0000", 2024-09-13T10:12:00.444 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036895 2024-09-13T10:12:00.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.444 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.444 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.444 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.793957+0000", 2024-09-13T10:12:00.444 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075548940000000004 2024-09-13T10:12:00.444 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.444 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.444 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.445 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794103+0000", 2024-09-13T10:12:00.445 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014615799999999999 2024-09-13T10:12:00.445 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.445 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.445 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.445 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794217+0000", 2024-09-13T10:12:00.445 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011437799999999999 2024-09-13T10:12:00.445 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.445 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.445 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.445 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794297+0000", 2024-09-13T10:12:00.445 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9431999999999994e-05 2024-09-13T10:12:00.446 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.446 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.446 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.446 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794360+0000", 2024-09-13T10:12:00.446 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.3755000000000002e-05 2024-09-13T10:12:00.446 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.446 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.446 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.446 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.446 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.446 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.14.23:44 3.3 3:e2816a99:::d4bb749789c90c884ffd20ac62f804c7a6b80502d14432fa627e1239249ee2da:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:00.446 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:10.781168+0000", 2024-09-13T10:12:00.446 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 49.579809812999997, 2024-09-13T10:12:00.447 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013562833, 2024-09-13T10:12:00.447 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.447 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.447 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.447 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.447 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.447 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.447 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781168+0000", 2024-09-13T10:12:00.447 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.447 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.447 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.447 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781168+0000", 2024-09-13T10:12:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781170+0000", 2024-09-13T10:12:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5400000000000001e-06 2024-09-13T10:12:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781470+0000", 2024-09-13T10:12:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00030077100000000002 2024-09-13T10:12:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.448 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.449 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.449 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781508+0000", 2024-09-13T10:12:00.449 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.7517e-05 2024-09-13T10:12:00.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.449 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.449 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.449 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781527+0000", 2024-09-13T10:12:00.449 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8766e-05 2024-09-13T10:12:00.449 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.449 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.449 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.450 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.788166+0000", 2024-09-13T10:12:00.450 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0066389450000000003 2024-09-13T10:12:00.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.450 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.450 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.450 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.788515+0000", 2024-09-13T10:12:00.450 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00034976099999999997 2024-09-13T10:12:00.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.450 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.450 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.450 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.789180+0000", 2024-09-13T10:12:00.450 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00066460200000000003 2024-09-13T10:12:00.450 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794399+0000", 2024-09-13T10:12:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0052185340000000004 2024-09-13T10:12:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794487+0000", 2024-09-13T10:12:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8660999999999997e-05 2024-09-13T10:12:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.451 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794601+0000", 2024-09-13T10:12:00.452 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000114188 2024-09-13T10:12:00.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.452 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.452 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.452 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794678+0000", 2024-09-13T10:12:00.452 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6058999999999998e-05 2024-09-13T10:12:00.452 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.452 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.452 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.452 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794731+0000", 2024-09-13T10:12:00.452 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.3489000000000002e-05 2024-09-13T10:12:00.452 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.453 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.453 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.453 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.453 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.453 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.14.23:47 3.3 3:ffec9690:::b0d2dba9f365561c17cd4cce8c8a8f4d8f931d05ca998f81ae8f5caa986fcf1e:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:00.453 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:10.781559+0000", 2024-09-13T10:12:00.453 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 49.579418515999997, 2024-09-13T10:12:00.453 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015576100000000001, 2024-09-13T10:12:00.453 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.453 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.453 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.454 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.454 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.454 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.454 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781559+0000", 2024-09-13T10:12:00.454 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.454 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.454 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.454 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781559+0000", 2024-09-13T10:12:00.454 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.454 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.454 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.454 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781561+0000", 2024-09-13T10:12:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5460000000000001e-06 2024-09-13T10:12:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781708+0000", 2024-09-13T10:12:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000147291 2024-09-13T10:12:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781735+0000", 2024-09-13T10:12:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7175999999999999e-05 2024-09-13T10:12:00.455 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.456 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.456 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.456 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781755+0000", 2024-09-13T10:12:00.456 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9653e-05 2024-09-13T10:12:00.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.456 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.456 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.456 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.790861+0000", 2024-09-13T10:12:00.456 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0091056920000000003 2024-09-13T10:12:00.456 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.456 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.456 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.457 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.791072+0000", 2024-09-13T10:12:00.457 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000211103 2024-09-13T10:12:00.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.457 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.457 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.457 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.791511+0000", 2024-09-13T10:12:00.457 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00043908899999999998 2024-09-13T10:12:00.457 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.457 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.457 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.457 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794857+0000", 2024-09-13T10:12:00.457 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0033462940000000001 2024-09-13T10:12:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794945+0000", 2024-09-13T10:12:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7626000000000004e-05 2024-09-13T10:12:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.797001+0000", 2024-09-13T10:12:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020557280000000002 2024-09-13T10:12:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.458 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.797080+0000", 2024-09-13T10:12:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9169e-05 2024-09-13T10:12:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.797136+0000", 2024-09-13T10:12:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5733000000000003e-05 2024-09-13T10:12:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.6.23:159 3.3 3:d330ea58:::b1e0a8f1f87dd65498385354f65ab6aa4dbeb3cb948fa25b5d7b0975437eef68:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:12:00.459 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:18.186115+0000", 2024-09-13T10:12:00.460 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 42.174862515000001, 2024-09-13T10:12:00.460 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.01270717, 2024-09-13T10:12:00.460 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.460 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.460 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.460 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.460 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.460 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.460 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186115+0000", 2024-09-13T10:12:00.460 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.460 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.460 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.460 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186115+0000", 2024-09-13T10:12:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186116+0000", 2024-09-13T10:12:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5599999999999995e-07 2024-09-13T10:12:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186174+0000", 2024-09-13T10:12:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8397999999999999e-05 2024-09-13T10:12:00.461 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.462 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.462 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.462 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186182+0000", 2024-09-13T10:12:00.462 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.729e-06 2024-09-13T10:12:00.462 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.462 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.462 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.462 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.186189+0000", 2024-09-13T10:12:00.462 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4629999999999998e-06 2024-09-13T10:12:00.462 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.462 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.462 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.463 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.195175+0000", 2024-09-13T10:12:00.463 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0089862550000000003 2024-09-13T10:12:00.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.463 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.463 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.463 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.195287+0000", 2024-09-13T10:12:00.463 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011233700000000001 2024-09-13T10:12:00.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.463 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.463 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.463 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.195457+0000", 2024-09-13T10:12:00.463 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016981799999999999 2024-09-13T10:12:00.463 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.196713+0000", 2024-09-13T10:12:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001256463 2024-09-13T10:12:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.197341+0000", 2024-09-13T10:12:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00062791400000000003 2024-09-13T10:12:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.464 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.198733+0000", 2024-09-13T10:12:00.465 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001391403 2024-09-13T10:12:00.465 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.465 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.465 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.465 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.198781+0000", 2024-09-13T10:12:00.465 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.8680000000000001e-05 2024-09-13T10:12:00.465 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.465 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.465 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.465 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:18.198823+0000", 2024-09-13T10:12:00.465 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1153999999999999e-05 2024-09-13T10:12:00.465 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.466 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.466 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.466 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.466 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.466 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.13.21:65 3.3 3:febf9037:::86029bdee449ecbe1f721f6a717c52067baf9f2cd6735940d23b17d31604c51c:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:12:00.466 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:29.419053+0000", 2024-09-13T10:12:00.466 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 30.941925381000001, 2024-09-13T10:12:00.466 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013463819, 2024-09-13T10:12:00.466 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.466 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.466 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.466 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.466 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.467 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.467 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.419053+0000", 2024-09-13T10:12:00.467 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.467 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.467 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.467 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.467 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.419053+0000", 2024-09-13T10:12:00.467 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.467 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.467 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.467 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.467 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.419054+0000", 2024-09-13T10:12:00.467 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.657e-06 2024-09-13T10:12:00.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.468 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.468 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.468 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.419978+0000", 2024-09-13T10:12:00.468 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00092341999999999999 2024-09-13T10:12:00.468 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.468 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.468 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.468 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.419998+0000", 2024-09-13T10:12:00.469 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0809e-05 2024-09-13T10:12:00.469 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.469 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.469 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.469 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.420011+0000", 2024-09-13T10:12:00.469 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2163e-05 2024-09-13T10:12:00.469 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.469 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.469 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.469 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.426538+0000", 2024-09-13T10:12:00.469 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0065269639999999997 2024-09-13T10:12:00.469 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.469 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.469 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.426917+0000", 2024-09-13T10:12:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037961499999999998 2024-09-13T10:12:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.427518+0000", 2024-09-13T10:12:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00060034900000000002 2024-09-13T10:12:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.432102+0000", 2024-09-13T10:12:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0045840079999999997 2024-09-13T10:12:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.470 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.471 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.471 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.432144+0000", 2024-09-13T10:12:00.471 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1922000000000003e-05 2024-09-13T10:12:00.471 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.471 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.471 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.471 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.432268+0000", 2024-09-13T10:12:00.471 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012485900000000001 2024-09-13T10:12:00.471 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.471 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.471 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.471 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.432417+0000", 2024-09-13T10:12:00.471 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014855399999999999 2024-09-13T10:12:00.471 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.471 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.472 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.472 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.432516+0000", 2024-09-13T10:12:00.472 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.9499000000000004e-05 2024-09-13T10:12:00.472 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.472 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.472 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.472 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.472 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.472 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.13.21:66 3.3 3:ec59e195:::89902e50bfc3ea21737900836f42c22669bb4be539f50b5d1ee5d1c721c9a0f8:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:12:00.472 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:29.420033+0000", 2024-09-13T10:12:00.472 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 30.940944683000001, 2024-09-13T10:12:00.472 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013349420000000001, 2024-09-13T10:12:00.472 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.473 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.473 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.473 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.473 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.473 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.473 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.420033+0000", 2024-09-13T10:12:00.473 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.473 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.473 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.473 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.473 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.420033+0000", 2024-09-13T10:12:00.473 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.473 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.474 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.474 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.474 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.420035+0000", 2024-09-13T10:12:00.474 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4160000000000001e-06 2024-09-13T10:12:00.474 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.474 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.474 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.474 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.420254+0000", 2024-09-13T10:12:00.474 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021894500000000001 2024-09-13T10:12:00.474 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.474 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.474 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.474 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.420316+0000", 2024-09-13T10:12:00.475 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.2757999999999998e-05 2024-09-13T10:12:00.475 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.475 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.475 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.475 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.420335+0000", 2024-09-13T10:12:00.475 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9072e-05 2024-09-13T10:12:00.475 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.475 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.475 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.475 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.430124+0000", 2024-09-13T10:12:00.475 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0097888520000000007 2024-09-13T10:12:00.475 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.475 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.475 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.476 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.430481+0000", 2024-09-13T10:12:00.476 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00035628899999999998 2024-09-13T10:12:00.476 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.476 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.476 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.476 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.430877+0000", 2024-09-13T10:12:00.476 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039646100000000002 2024-09-13T10:12:00.476 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.476 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.476 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.476 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.432557+0000", 2024-09-13T10:12:00.476 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00168023 2024-09-13T10:12:00.476 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.477 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.477 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.477 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.433129+0000", 2024-09-13T10:12:00.477 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00057135699999999998 2024-09-13T10:12:00.477 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.477 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.477 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.477 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.433245+0000", 2024-09-13T10:12:00.477 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011629900000000001 2024-09-13T10:12:00.477 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.477 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.477 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.477 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.433321+0000", 2024-09-13T10:12:00.477 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6054000000000001e-05 2024-09-13T10:12:00.478 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.478 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.478 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.478 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:29.433383+0000", 2024-09-13T10:12:00.478 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.1686999999999994e-05 2024-09-13T10:12:00.478 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.478 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.478 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.478 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.478 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.478 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3772 4.18 4:1de9da87:::smithi19530714-5:head [read 359834~3052020,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e91)", 2024-09-13T10:12:00.478 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:55.340676+0000", 2024-09-13T10:12:00.478 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 5.0203017179999998, 2024-09-13T10:12:00.478 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014691111, 2024-09-13T10:12:00.479 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.479 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.479 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:00.479 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.479 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.479 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.479 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3772 2024-09-13T10:12:00.479 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.479 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.479 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.479 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.479 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:55.340676+0000", 2024-09-13T10:12:00.479 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.480 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.480 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.480 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.480 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:55.340676+0000", 2024-09-13T10:12:00.480 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.480 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.480 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.480 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.480 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:55.340678+0000", 2024-09-13T10:12:00.480 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3659999999999999e-06 2024-09-13T10:12:00.480 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.480 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.480 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.481 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:55.340687+0000", 2024-09-13T10:12:00.481 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1600000000000004e-06 2024-09-13T10:12:00.481 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.481 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.481 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.481 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:55.340707+0000", 2024-09-13T10:12:00.481 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0296999999999999e-05 2024-09-13T10:12:00.481 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.481 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.481 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.481 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:55.340724+0000", 2024-09-13T10:12:00.481 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7112000000000002e-05 2024-09-13T10:12:00.482 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.482 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.482 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.482 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:55.340922+0000", 2024-09-13T10:12:00.482 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019769899999999999 2024-09-13T10:12:00.482 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.482 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.482 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting_for_map not empty", 2024-09-13T10:12:00.482 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:55.340937+0000", 2024-09-13T10:12:00.482 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4919999999999999e-05 2024-09-13T10:12:00.482 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.482 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.482 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.482 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:55.351836+0000", 2024-09-13T10:12:00.483 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010899057 2024-09-13T10:12:00.483 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.483 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.483 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.483 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:55.351974+0000", 2024-09-13T10:12:00.483 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013781700000000001 2024-09-13T10:12:00.483 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.483 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.483 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.483 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:55.355367+0000", 2024-09-13T10:12:00.483 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0033936830000000002 2024-09-13T10:12:00.483 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.483 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.483 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.484 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.484 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.484 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3970 4.18 4:1de9da87:::smithi19530714-5:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e94)", 2024-09-13T10:12:00.484 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:00.220914+0000", 2024-09-13T10:12:00.484 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 0.14006448099999999, 2024-09-13T10:12:00.484 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048456570999999997, 2024-09-13T10:12:00.484 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.484 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.484 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:00.484 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.484 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.484 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.484 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3970 2024-09-13T10:12:00.485 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.485 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.485 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.485 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.485 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.220914+0000", 2024-09-13T10:12:00.485 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.485 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.485 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.485 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.485 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.220914+0000", 2024-09-13T10:12:00.485 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.485 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.485 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.485 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.486 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.220917+0000", 2024-09-13T10:12:00.486 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.36e-06 2024-09-13T10:12:00.486 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.486 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.486 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.486 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.220934+0000", 2024-09-13T10:12:00.486 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6748999999999998e-05 2024-09-13T10:12:00.486 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.486 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.486 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.486 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.220973+0000", 2024-09-13T10:12:00.486 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9693000000000001e-05 2024-09-13T10:12:00.486 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.486 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.487 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.487 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.221001+0000", 2024-09-13T10:12:00.487 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7209e-05 2024-09-13T10:12:00.487 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.487 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.487 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.487 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.221170+0000", 2024-09-13T10:12:00.487 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016995700000000001 2024-09-13T10:12:00.487 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.487 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.487 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.487 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.221316+0000", 2024-09-13T10:12:00.487 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000145981 2024-09-13T10:12:00.488 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.488 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.488 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.488 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.269370+0000", 2024-09-13T10:12:00.488 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048053621999999997 2024-09-13T10:12:00.488 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.488 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.488 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.488 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.488 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.488 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.10.22:27 3.3 3:cad3271e:::676486c06ed048ca5257a41d3d81840c3f0cf43506417db4c50de7b6efe59a25:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e95)", 2024-09-13T10:12:00.488 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:00.254542+0000", 2024-09-13T10:12:00.488 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 0.106436107, 2024-09-13T10:12:00.488 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013258137999999999, 2024-09-13T10:12:00.489 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.489 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.489 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.489 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.489 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.489 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.489 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.254542+0000", 2024-09-13T10:12:00.489 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.489 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.489 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.489 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.254542+0000", 2024-09-13T10:12:00.489 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.490 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.490 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.490 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.254543+0000", 2024-09-13T10:12:00.490 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.136e-06 2024-09-13T10:12:00.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.490 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.490 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.490 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.254732+0000", 2024-09-13T10:12:00.490 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00018898999999999999 2024-09-13T10:12:00.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.490 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.490 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.490 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.254743+0000", 2024-09-13T10:12:00.491 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1287999999999999e-05 2024-09-13T10:12:00.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.491 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.491 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.491 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.254752+0000", 2024-09-13T10:12:00.491 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5699999999999993e-06 2024-09-13T10:12:00.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.491 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.491 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.491 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.263132+0000", 2024-09-13T10:12:00.491 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0083803190000000007 2024-09-13T10:12:00.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.492 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.492 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.492 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.263256+0000", 2024-09-13T10:12:00.492 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012354099999999999 2024-09-13T10:12:00.492 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.492 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.492 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:00.492 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.263489+0000", 2024-09-13T10:12:00.492 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023289299999999999 2024-09-13T10:12:00.492 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.492 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.492 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.492 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.264985+0000", 2024-09-13T10:12:00.492 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0014966529999999999 2024-09-13T10:12:00.493 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.493 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.493 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.493 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.266132+0000", 2024-09-13T10:12:00.493 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001147035 2024-09-13T10:12:00.493 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.493 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.493 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.493 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.267748+0000", 2024-09-13T10:12:00.493 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001615561 2024-09-13T10:12:00.493 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.493 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.493 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.494 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.267777+0000", 2024-09-13T10:12:00.494 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8979000000000002e-05 2024-09-13T10:12:00.494 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.494 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.494 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.494 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.267800+0000", 2024-09-13T10:12:00.494 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3173e-05 2024-09-13T10:12:00.494 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.494 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.494 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.494 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.494 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.494 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:00.495 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:12:00.498 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-13T10:12:00.498 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:00.490+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:00.499 INFO:tasks.rados.rados.0.smithi195.stdout:1670: finishing write tid 1 to smithi19530714-1 2024-09-13T10:12:00.499 INFO:tasks.rados.rados.0.smithi195.stdout:1670: finishing write tid 2 to smithi19530714-1 2024-09-13T10:12:00.500 INFO:tasks.rados.rados.0.smithi195.stdout:1670: finishing write tid 3 to smithi19530714-1 2024-09-13T10:12:00.500 INFO:tasks.rados.rados.0.smithi195.stdout:1670: finishing write tid 4 to smithi19530714-1 2024-09-13T10:12:00.500 INFO:tasks.rados.rados.0.smithi195.stdout:1670: oid 1 updating version 0 to 100 2024-09-13T10:12:00.500 INFO:tasks.rados.rados.0.smithi195.stdout:1670: oid 1 version 100 is already newer than 98 2024-09-13T10:12:00.500 INFO:tasks.rados.rados.0.smithi195.stdout:1670: oid 1 version 100 is already newer than 99 2024-09-13T10:12:00.500 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 100 (ObjNum 357 snap 0 seq_num 357) dirty exists 2024-09-13T10:12:00.500 INFO:tasks.rados.rados.0.smithi195.stdout:1670: left oid 1 (ObjNum 357 snap 0 seq_num 357) 2024-09-13T10:12:00.500 INFO:tasks.rados.rados.0.smithi195.stdout:1671: finishing set_chunk to oid 23 2024-09-13T10:12:00.500 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 42 (ObjNum 286 snap 0 seq_num 286) dirty exists 2024-09-13T10:12:00.500 INFO:tasks.rados.rados.0.smithi195.stdout:1672: got (0) Success 2024-09-13T10:12:00.500 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 0 (ObjNum 322 snap 0 seq_num 322) dirty exists 2024-09-13T10:12:00.500 INFO:tasks.rados.rados.0.smithi195.stdout:1663: done (9 left) 2024-09-13T10:12:00.501 INFO:tasks.rados.rados.0.smithi195.stdout:1664: done (8 left) 2024-09-13T10:12:00.501 INFO:tasks.rados.rados.0.smithi195.stdout:1665: done (7 left) 2024-09-13T10:12:00.501 INFO:tasks.rados.rados.0.smithi195.stdout:1666: done (6 left) 2024-09-13T10:12:00.501 INFO:tasks.rados.rados.0.smithi195.stdout:1667: done (5 left) 2024-09-13T10:12:00.501 INFO:tasks.rados.rados.0.smithi195.stdout:1668: done (4 left) 2024-09-13T10:12:00.501 INFO:tasks.rados.rados.0.smithi195.stdout:1669: done (3 left) 2024-09-13T10:12:00.501 INFO:tasks.rados.rados.0.smithi195.stdout:1670: done (2 left) 2024-09-13T10:12:00.501 INFO:tasks.rados.rados.0.smithi195.stdout:1671: done (1 left) 2024-09-13T10:12:00.501 INFO:tasks.rados.rados.0.smithi195.stdout:1672: done (0 left) 2024-09-13T10:12:00.501 INFO:tasks.rados.rados.0.smithi195.stdout:1673: read oid 7 snap -1 2024-09-13T10:12:00.501 INFO:tasks.rados.rados.0.smithi195.stdout:1673: expect (ObjNum 276 snap 0 seq_num 276) 2024-09-13T10:12:00.501 INFO:tasks.rados.rados.0.smithi195.stdout:1674: read oid 22 snap -1 2024-09-13T10:12:00.501 INFO:tasks.rados.rados.0.smithi195.stdout:1674: expect (ObjNum 341 snap 0 seq_num 341) 2024-09-13T10:12:00.502 INFO:tasks.rados.rados.0.smithi195.stdout:1675: read oid 6 snap -1 2024-09-13T10:12:00.502 INFO:tasks.rados.rados.0.smithi195.stdout:1675: expect (ObjNum 337 snap 0 seq_num 337) 2024-09-13T10:12:00.502 INFO:tasks.rados.rados.0.smithi195.stdout:1676: read oid 17 snap -1 2024-09-13T10:12:00.502 INFO:tasks.rados.rados.0.smithi195.stdout:1676: expect (ObjNum 347 snap 0 seq_num 347) 2024-09-13T10:12:00.502 INFO:tasks.rados.rados.0.smithi195.stdout:1677: read oid 44 snap -1 2024-09-13T10:12:00.502 INFO:tasks.rados.rados.0.smithi195.stdout:1677: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:12:00.502 INFO:tasks.rados.rados.0.smithi195.stdout:1678: tier_evict oid 13 2024-09-13T10:12:00.502 INFO:tasks.rados.rados.0.smithi195.stdout:1678: tier_evict oid 13 snap -1 2024-09-13T10:12:00.502 INFO:tasks.rados.rados.0.smithi195.stdout:1679: read oid 27 snap -1 2024-09-13T10:12:00.502 INFO:tasks.rados.rados.0.smithi195.stdout:1679: expect (ObjNum 221 snap 0 seq_num 221) 2024-09-13T10:12:00.502 INFO:tasks.rados.rados.0.smithi195.stdout:1680: write oid 35 current snap is 0 2024-09-13T10:12:00.502 INFO:tasks.rados.rados.0.smithi195.stdout:1680: seq_num 358 ranges {416906=407733,1366083=511456} 2024-09-13T10:12:00.502 INFO:tasks.rados.rados.0.smithi195.stdout:1680: writing smithi19530714-35 from 416906 to 824639 tid 1 2024-09-13T10:12:00.503 INFO:tasks.rados.rados.0.smithi195.stdout:1680: writing smithi19530714-35 from 1366083 to 1877539 tid 2 2024-09-13T10:12:00.503 INFO:tasks.rados.rados.0.smithi195.stdout:1681: read oid 24 snap -1 2024-09-13T10:12:00.503 INFO:tasks.rados.rados.0.smithi195.stdout:1681: expect deleted 2024-09-13T10:12:00.503 INFO:tasks.rados.rados.0.smithi195.stdout:1682: write oid 11 current snap is 0 2024-09-13T10:12:00.503 INFO:tasks.rados.rados.0.smithi195.stdout:1677: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:12:00.503 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3341251 2024-09-13T10:12:00.503 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:00.536 INFO:tasks.rados.rados.0.smithi195.stdout:1678: got (22) Invalid argument 2024-09-13T10:12:00.536 INFO:tasks.rados.rados.0.smithi195.stdout:1674: expect (ObjNum 341 snap 0 seq_num 341) 2024-09-13T10:12:00.539 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1823383 2024-09-13T10:12:00.564 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:00.596 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:12:00.591+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:12:00.625 INFO:tasks.rados.rados.0.smithi195.stdout:1673: expect (ObjNum 276 snap 0 seq_num 276) 2024-09-13T10:12:00.625 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:00.653 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:12:00.653 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:12:00.668 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:00.668 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:12:00.668 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:12:00.669 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:00.674 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:12:00.674 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:12:00.683 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-13T10:12:00.697 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:00.691+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:00.697 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:12:00.698 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:12:00.720 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:12:00.720 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:12:00.735 INFO:tasks.rados.rados.0.smithi195.stdout:1676: expect (ObjNum 347 snap 0 seq_num 347) 2024-09-13T10:12:00.744 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:12:00.744 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:12:00.765 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:12:00.765 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:12:00.788 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:12:00.788 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:12:00.797 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:00.791+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:00.805 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:00.805 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:12:00.806 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:12:00.806 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:12:00.806 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:00.810 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:12:00.810 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:12:00.820 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-13T10:12:00.832 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:12:00.833 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:12:00.835 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1329677 2024-09-13T10:12:00.855 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:12:00.855 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:12:00.877 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:12:00.877 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:12:00.898 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:12:00.895+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:12:00.900 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:12:00.900 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:12:00.920 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:12:00.921 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:12:00.945 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:12:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:12:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:12:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:202 3.2 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.991310+0000", 2024-09-13T10:12:00.945 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.949192608000004, 2024-09-13T10:12:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.040945251000000002, 2024-09-13T10:12:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 202 2024-09-13T10:12:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.946 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991310+0000", 2024-09-13T10:12:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991310+0000", 2024-09-13T10:12:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991314+0000", 2024-09-13T10:12:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.7100000000000001e-06 2024-09-13T10:12:00.947 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991328+0000", 2024-09-13T10:12:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.413e-05 2024-09-13T10:12:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991361+0000", 2024-09-13T10:12:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2512999999999999e-05 2024-09-13T10:12:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.948 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991377+0000", 2024-09-13T10:12:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6478999999999999e-05 2024-09-13T10:12:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991546+0000", 2024-09-13T10:12:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016856600000000001 2024-09-13T10:12:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991850+0000", 2024-09-13T10:12:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00030478800000000001 2024-09-13T10:12:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.949 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.001416+0000", 2024-09-13T10:12:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0095659279999999996 2024-09-13T10:12:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031502+0000", 2024-09-13T10:12:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.030085526000000001 2024-09-13T10:12:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031585+0000", 2024-09-13T10:12:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.3175000000000002e-05 2024-09-13T10:12:00.950 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031775+0000", 2024-09-13T10:12:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000190161 2024-09-13T10:12:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031982+0000", 2024-09-13T10:12:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000207245 2024-09-13T10:12:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032255+0000", 2024-09-13T10:12:00.951 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00027303000000000001 2024-09-13T10:12:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:206 3.2 3:4cbdbe9e:::smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.991649+0000", 2024-09-13T10:12:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.948854280000006, 2024-09-13T10:12:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041234240999999998, 2024-09-13T10:12:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.952 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 206 2024-09-13T10:12:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991649+0000", 2024-09-13T10:12:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.953 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991649+0000", 2024-09-13T10:12:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991650+0000", 2024-09-13T10:12:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.483e-06 2024-09-13T10:12:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991657+0000", 2024-09-13T10:12:00.954 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.0899999999999999e-06 2024-09-13T10:12:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991672+0000", 2024-09-13T10:12:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4514e-05 2024-09-13T10:12:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991681+0000", 2024-09-13T10:12:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6360000000000006e-06 2024-09-13T10:12:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.955 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992159+0000", 2024-09-13T10:12:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00047820099999999998 2024-09-13T10:12:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992411+0000", 2024-09-13T10:12:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025199700000000001 2024-09-13T10:12:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.009775+0000", 2024-09-13T10:12:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017363841000000001 2024-09-13T10:12:00.956 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032446+0000", 2024-09-13T10:12:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022670490000000001 2024-09-13T10:12:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032482+0000", 2024-09-13T10:12:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5902000000000003e-05 2024-09-13T10:12:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.957 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032612+0000", 2024-09-13T10:12:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013022700000000001 2024-09-13T10:12:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032767+0000", 2024-09-13T10:12:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015477000000000001 2024-09-13T10:12:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032883+0000", 2024-09-13T10:12:00.958 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011608999999999999 2024-09-13T10:12:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:210 3.2 3:46ab5247:::smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.992337+0000", 2024-09-13T10:12:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.948166137000001, 2024-09-13T10:12:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.057985696000000003, 2024-09-13T10:12:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.959 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 210 2024-09-13T10:12:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992337+0000", 2024-09-13T10:12:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.960 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992337+0000", 2024-09-13T10:12:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992339+0000", 2024-09-13T10:12:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2400000000000002e-06 2024-09-13T10:12:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992350+0000", 2024-09-13T10:12:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1259e-05 2024-09-13T10:12:00.961 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992368+0000", 2024-09-13T10:12:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8133000000000001e-05 2024-09-13T10:12:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992382+0000", 2024-09-13T10:12:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3837e-05 2024-09-13T10:12:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.962 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992752+0000", 2024-09-13T10:12:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037008000000000001 2024-09-13T10:12:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992980+0000", 2024-09-13T10:12:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022799100000000001 2024-09-13T10:12:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027651+0000", 2024-09-13T10:12:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.034671027 2024-09-13T10:12:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.963 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.034868+0000", 2024-09-13T10:12:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0072169920000000002 2024-09-13T10:12:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038033+0000", 2024-09-13T10:12:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003164966 2024-09-13T10:12:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050187+0000", 2024-09-13T10:12:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012153815 2024-09-13T10:12:00.964 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050266+0000", 2024-09-13T10:12:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9359999999999999e-05 2024-09-13T10:12:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050322+0000", 2024-09-13T10:12:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5995999999999997e-05 2024-09-13T10:12:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.965 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:214 3.2 3:68ba95ba:::smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.992479+0000", 2024-09-13T10:12:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.948024063999995, 2024-09-13T10:12:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.045987867000000002, 2024-09-13T10:12:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 214 2024-09-13T10:12:00.966 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992479+0000", 2024-09-13T10:12:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992479+0000", 2024-09-13T10:12:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.967 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992481+0000", 2024-09-13T10:12:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2960000000000001e-06 2024-09-13T10:12:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992493+0000", 2024-09-13T10:12:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2077e-05 2024-09-13T10:12:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992519+0000", 2024-09-13T10:12:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6282e-05 2024-09-13T10:12:00.968 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992538+0000", 2024-09-13T10:12:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9083e-05 2024-09-13T10:12:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993393+0000", 2024-09-13T10:12:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00085445200000000001 2024-09-13T10:12:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.969 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993682+0000", 2024-09-13T10:12:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000288895 2024-09-13T10:12:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.019700+0000", 2024-09-13T10:12:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.02601794 2024-09-13T10:12:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032498+0000", 2024-09-13T10:12:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012798535999999999 2024-09-13T10:12:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.970 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033641+0000", 2024-09-13T10:12:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001142569 2024-09-13T10:12:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038259+0000", 2024-09-13T10:12:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0046180939999999997 2024-09-13T10:12:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038379+0000", 2024-09-13T10:12:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000119914 2024-09-13T10:12:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.971 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038467+0000", 2024-09-13T10:12:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7729e-05 2024-09-13T10:12:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:217 3.2 3:554c406d:::smithi19530714-18 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.027515+0000", 2024-09-13T10:12:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.912988229999996, 2024-09-13T10:12:00.972 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032787180999999999, 2024-09-13T10:12:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 217 2024-09-13T10:12:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027515+0000", 2024-09-13T10:12:00.973 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027515+0000", 2024-09-13T10:12:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027518+0000", 2024-09-13T10:12:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.4429999999999999e-06 2024-09-13T10:12:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.974 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027532+0000", 2024-09-13T10:12:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3517e-05 2024-09-13T10:12:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027563+0000", 2024-09-13T10:12:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1158999999999998e-05 2024-09-13T10:12:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027585+0000", 2024-09-13T10:12:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.228e-05 2024-09-13T10:12:00.975 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033014+0000", 2024-09-13T10:12:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0054288649999999997 2024-09-13T10:12:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033269+0000", 2024-09-13T10:12:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000255054 2024-09-13T10:12:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.976 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.044618+0000", 2024-09-13T10:12:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011349391 2024-09-13T10:12:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.054627+0000", 2024-09-13T10:12:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010008992 2024-09-13T10:12:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.058047+0000", 2024-09-13T10:12:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0034193000000000001 2024-09-13T10:12:00.977 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060139+0000", 2024-09-13T10:12:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020925240000000001 2024-09-13T10:12:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060222+0000", 2024-09-13T10:12:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.2917999999999995e-05 2024-09-13T10:12:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060302+0000", 2024-09-13T10:12:00.978 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9738000000000001e-05 2024-09-13T10:12:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:218 3.2 3:5915c927:::smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.032754+0000", 2024-09-13T10:12:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.907748961999999, 2024-09-13T10:12:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023630241, 2024-09-13T10:12:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.979 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 218 2024-09-13T10:12:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032754+0000", 2024-09-13T10:12:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032754+0000", 2024-09-13T10:12:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.980 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032757+0000", 2024-09-13T10:12:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6330000000000002e-06 2024-09-13T10:12:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032772+0000", 2024-09-13T10:12:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4636e-05 2024-09-13T10:12:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032796+0000", 2024-09-13T10:12:00.981 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3963000000000001e-05 2024-09-13T10:12:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032815+0000", 2024-09-13T10:12:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9159000000000001e-05 2024-09-13T10:12:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033742+0000", 2024-09-13T10:12:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00092672700000000002 2024-09-13T10:12:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.982 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.034124+0000", 2024-09-13T10:12:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00038220399999999997 2024-09-13T10:12:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.040963+0000", 2024-09-13T10:12:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0068392230000000002 2024-09-13T10:12:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050362+0000", 2024-09-13T10:12:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0093986060000000003 2024-09-13T10:12:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.983 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.054585+0000", 2024-09-13T10:12:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0042225429999999996 2024-09-13T10:12:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.056250+0000", 2024-09-13T10:12:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001665251 2024-09-13T10:12:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.984 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.056320+0000", 2024-09-13T10:12:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9913999999999995e-05 2024-09-13T10:12:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.056384+0000", 2024-09-13T10:12:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4381999999999994e-05 2024-09-13T10:12:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:220 3.2 3:753d5785:::smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.985 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.035716+0000", 2024-09-13T10:12:00.986 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.904786469000001, 2024-09-13T10:12:00.986 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.030171203000000001, 2024-09-13T10:12:00.986 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.986 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.986 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.986 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.986 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.986 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.986 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 220 2024-09-13T10:12:00.986 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.986 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.986 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.987 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.987 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035716+0000", 2024-09-13T10:12:00.987 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.987 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.987 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.987 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.987 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035716+0000", 2024-09-13T10:12:00.987 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.987 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.987 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.987 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.987 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035720+0000", 2024-09-13T10:12:00.987 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5310000000000002e-06 2024-09-13T10:12:00.987 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035735+0000", 2024-09-13T10:12:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5517000000000001e-05 2024-09-13T10:12:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035765+0000", 2024-09-13T10:12:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9771e-05 2024-09-13T10:12:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.988 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035786+0000", 2024-09-13T10:12:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0928e-05 2024-09-13T10:12:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035945+0000", 2024-09-13T10:12:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015871900000000001 2024-09-13T10:12:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.036337+0000", 2024-09-13T10:12:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039231299999999998 2024-09-13T10:12:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.989 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.051063+0000", 2024-09-13T10:12:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014725607 2024-09-13T10:12:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.055802+0000", 2024-09-13T10:12:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0047387929999999998 2024-09-13T10:12:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064353+0000", 2024-09-13T10:12:00.990 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0085516629999999993 2024-09-13T10:12:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065750+0000", 2024-09-13T10:12:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001396748 2024-09-13T10:12:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065811+0000", 2024-09-13T10:12:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.0770000000000003e-05 2024-09-13T10:12:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065888+0000", 2024-09-13T10:12:00.991 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6842999999999996e-05 2024-09-13T10:12:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:227 3.2 3:6ac7800c:::smithi19530714-28 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.060091+0000", 2024-09-13T10:12:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.880411404, 2024-09-13T10:12:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.034993362, 2024-09-13T10:12:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.992 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 227 2024-09-13T10:12:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060091+0000", 2024-09-13T10:12:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060091+0000", 2024-09-13T10:12:00.993 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060094+0000", 2024-09-13T10:12:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5330000000000001e-06 2024-09-13T10:12:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060103+0000", 2024-09-13T10:12:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7020000000000007e-06 2024-09-13T10:12:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.994 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060121+0000", 2024-09-13T10:12:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.791e-05 2024-09-13T10:12:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060135+0000", 2024-09-13T10:12:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4059999999999999e-05 2024-09-13T10:12:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060346+0000", 2024-09-13T10:12:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021175600000000001 2024-09-13T10:12:00.995 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060559+0000", 2024-09-13T10:12:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021239499999999999 2024-09-13T10:12:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.073003+0000", 2024-09-13T10:12:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012444255 2024-09-13T10:12:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:00.996 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.081388+0000", 2024-09-13T10:12:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0083852119999999995 2024-09-13T10:12:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094289+0000", 2024-09-13T10:12:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012900271 2024-09-13T10:12:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094913+0000", 2024-09-13T10:12:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00062429200000000003 2024-09-13T10:12:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.997 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094972+0000", 2024-09-13T10:12:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9457999999999999e-05 2024-09-13T10:12:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095085+0000", 2024-09-13T10:12:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000112518 2024-09-13T10:12:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:00.998 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:230 3.2 3:5d108080:::smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.065002+0000", 2024-09-13T10:12:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.875500959999997, 2024-09-13T10:12:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.049477431000000002, 2024-09-13T10:12:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 230 2024-09-13T10:12:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:00.999 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065002+0000", 2024-09-13T10:12:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065002+0000", 2024-09-13T10:12:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.000 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065004+0000", 2024-09-13T10:12:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4269999999999998e-06 2024-09-13T10:12:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065014+0000", 2024-09-13T10:12:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.9799999999999993e-06 2024-09-13T10:12:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:01.001 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065038+0000", 2024-09-13T10:12:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4229000000000001e-05 2024-09-13T10:12:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065054+0000", 2024-09-13T10:12:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5764999999999999e-05 2024-09-13T10:12:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065171+0000", 2024-09-13T10:12:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117032 2024-09-13T10:12:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.002 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065513+0000", 2024-09-13T10:12:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00034131 2024-09-13T10:12:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.087042+0000", 2024-09-13T10:12:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021529534999999999 2024-09-13T10:12:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.097903+0000", 2024-09-13T10:12:01.003 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010861282999999999 2024-09-13T10:12:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.099922+0000", 2024-09-13T10:12:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020186539999999999 2024-09-13T10:12:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114391+0000", 2024-09-13T10:12:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014468880999999999 2024-09-13T10:12:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.004 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114434+0000", 2024-09-13T10:12:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.3511999999999999e-05 2024-09-13T10:12:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114479+0000", 2024-09-13T10:12:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4823000000000001e-05 2024-09-13T10:12:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.005 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:232 3.2 3:6538713c:::smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.065891+0000", 2024-09-13T10:12:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.874612088999996, 2024-09-13T10:12:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048804489999999999, 2024-09-13T10:12:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:01.006 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 232 2024-09-13T10:12:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065891+0000", 2024-09-13T10:12:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065891+0000", 2024-09-13T10:12:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.007 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065892+0000", 2024-09-13T10:12:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6249999999999999e-06 2024-09-13T10:12:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065902+0000", 2024-09-13T10:12:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.4150000000000005e-06 2024-09-13T10:12:01.008 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065920+0000", 2024-09-13T10:12:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7907e-05 2024-09-13T10:12:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065932+0000", 2024-09-13T10:12:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2469999999999999e-05 2024-09-13T10:12:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.066072+0000", 2024-09-13T10:12:01.009 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013935999999999999 2024-09-13T10:12:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.066435+0000", 2024-09-13T10:12:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036323300000000002 2024-09-13T10:12:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091133+0000", 2024-09-13T10:12:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024698494000000001 2024-09-13T10:12:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.010 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.101407+0000", 2024-09-13T10:12:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010273835 2024-09-13T10:12:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114342+0000", 2024-09-13T10:12:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012934826 2024-09-13T10:12:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114548+0000", 2024-09-13T10:12:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000206243 2024-09-13T10:12:01.011 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114630+0000", 2024-09-13T10:12:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.2127999999999997e-05 2024-09-13T10:12:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114695+0000", 2024-09-13T10:12:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4954000000000002e-05 2024-09-13T10:12:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:01.012 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:236 3.2 3:73c2893b:::smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.076316+0000", 2024-09-13T10:12:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.864186967999999, 2024-09-13T10:12:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041217193999999999, 2024-09-13T10:12:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:01.013 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 236 2024-09-13T10:12:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076316+0000", 2024-09-13T10:12:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076316+0000", 2024-09-13T10:12:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.014 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076319+0000", 2024-09-13T10:12:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1750000000000001e-06 2024-09-13T10:12:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076333+0000", 2024-09-13T10:12:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3862e-05 2024-09-13T10:12:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.015 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076359+0000", 2024-09-13T10:12:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5641999999999999e-05 2024-09-13T10:12:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076378+0000", 2024-09-13T10:12:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9406999999999999e-05 2024-09-13T10:12:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.078831+0000", 2024-09-13T10:12:01.016 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0024532590000000002 2024-09-13T10:12:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079137+0000", 2024-09-13T10:12:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00030614500000000002 2024-09-13T10:12:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.102393+0000", 2024-09-13T10:12:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023255175999999999 2024-09-13T10:12:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.017 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114728+0000", 2024-09-13T10:12:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012335376 2024-09-13T10:12:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114735+0000", 2024-09-13T10:12:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2969999999999998e-06 2024-09-13T10:12:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.018 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117465+0000", 2024-09-13T10:12:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0027298019999999999 2024-09-13T10:12:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117501+0000", 2024-09-13T10:12:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5837999999999998e-05 2024-09-13T10:12:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117533+0000", 2024-09-13T10:12:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2215000000000002e-05 2024-09-13T10:12:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.019 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:240 3.2 3:7eee2f4a:::smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.091046+0000", 2024-09-13T10:12:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.849457126999994, 2024-09-13T10:12:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048376970999999998, 2024-09-13T10:12:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:01.020 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 240 2024-09-13T10:12:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091046+0000", 2024-09-13T10:12:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091046+0000", 2024-09-13T10:12:01.021 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091048+0000", 2024-09-13T10:12:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6299999999999998e-06 2024-09-13T10:12:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091060+0000", 2024-09-13T10:12:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1229999999999999e-05 2024-09-13T10:12:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.022 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091075+0000", 2024-09-13T10:12:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5313000000000001e-05 2024-09-13T10:12:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091085+0000", 2024-09-13T10:12:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0284e-05 2024-09-13T10:12:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094402+0000", 2024-09-13T10:12:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00331708 2024-09-13T10:12:01.023 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094649+0000", 2024-09-13T10:12:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024695300000000001 2024-09-13T10:12:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.107510+0000", 2024-09-13T10:12:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012860778999999999 2024-09-13T10:12:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.024 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114739+0000", 2024-09-13T10:12:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0072289700000000004 2024-09-13T10:12:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118693+0000", 2024-09-13T10:12:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0039540560000000001 2024-09-13T10:12:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139282+0000", 2024-09-13T10:12:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020589423999999999 2024-09-13T10:12:01.025 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139357+0000", 2024-09-13T10:12:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.4276999999999994e-05 2024-09-13T10:12:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139423+0000", 2024-09-13T10:12:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.5975000000000004e-05 2024-09-13T10:12:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.026 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:242 3.2 3:501d22a1:::smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.095846+0000", 2024-09-13T10:12:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.844656923000002, 2024-09-13T10:12:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.043813831999999997, 2024-09-13T10:12:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 242 2024-09-13T10:12:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.027 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095846+0000", 2024-09-13T10:12:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095846+0000", 2024-09-13T10:12:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.028 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095848+0000", 2024-09-13T10:12:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3760000000000002e-06 2024-09-13T10:12:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095859+0000", 2024-09-13T10:12:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0429e-05 2024-09-13T10:12:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095876+0000", 2024-09-13T10:12:01.029 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7504e-05 2024-09-13T10:12:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095886+0000", 2024-09-13T10:12:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0182e-05 2024-09-13T10:12:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.096010+0000", 2024-09-13T10:12:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012336700000000001 2024-09-13T10:12:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.030 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.096252+0000", 2024-09-13T10:12:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024189200000000001 2024-09-13T10:12:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111492+0000", 2024-09-13T10:12:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015239920000000001 2024-09-13T10:12:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115691+0000", 2024-09-13T10:12:01.031 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0041992410000000003 2024-09-13T10:12:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139492+0000", 2024-09-13T10:12:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023801495999999998 2024-09-13T10:12:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139560+0000", 2024-09-13T10:12:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.8163000000000005e-05 2024-09-13T10:12:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.032 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139619+0000", 2024-09-13T10:12:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8678999999999998e-05 2024-09-13T10:12:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139660+0000", 2024-09-13T10:12:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0583000000000001e-05 2024-09-13T10:12:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.033 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:244 3.2 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.112158+0000", 2024-09-13T10:12:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.828344459999997, 2024-09-13T10:12:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.030058202999999999, 2024-09-13T10:12:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 244 2024-09-13T10:12:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.034 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112158+0000", 2024-09-13T10:12:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112158+0000", 2024-09-13T10:12:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:01.035 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112161+0000", 2024-09-13T10:12:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.384e-06 2024-09-13T10:12:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112168+0000", 2024-09-13T10:12:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.7780000000000002e-06 2024-09-13T10:12:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112183+0000", 2024-09-13T10:12:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5917999999999999e-05 2024-09-13T10:12:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.036 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112195+0000", 2024-09-13T10:12:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1578e-05 2024-09-13T10:12:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114766+0000", 2024-09-13T10:12:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025710659999999999 2024-09-13T10:12:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114995+0000", 2024-09-13T10:12:01.037 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022844 2024-09-13T10:12:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.125677+0000", 2024-09-13T10:12:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010682677999999999 2024-09-13T10:12:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.140355+0000", 2024-09-13T10:12:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014677773999999999 2024-09-13T10:12:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.038 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.141368+0000", 2024-09-13T10:12:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0010134040000000001 2024-09-13T10:12:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142112+0000", 2024-09-13T10:12:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00074347599999999997 2024-09-13T10:12:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142172+0000", 2024-09-13T10:12:01.039 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9749e-05 2024-09-13T10:12:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142217+0000", 2024-09-13T10:12:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4957999999999999e-05 2024-09-13T10:12:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:245 3.2 3:7837e849:::smithi19530714-46 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.112398+0000", 2024-09-13T10:12:01.040 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.828104402999998, 2024-09-13T10:12:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.039535731999999997, 2024-09-13T10:12:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 245 2024-09-13T10:12:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:01.041 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112398+0000", 2024-09-13T10:12:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112398+0000", 2024-09-13T10:12:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112399+0000", 2024-09-13T10:12:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0470000000000001e-06 2024-09-13T10:12:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.042 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112405+0000", 2024-09-13T10:12:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.6899999999999997e-06 2024-09-13T10:12:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112417+0000", 2024-09-13T10:12:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2334000000000001e-05 2024-09-13T10:12:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112427+0000", 2024-09-13T10:12:01.043 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.0929999999999992e-06 2024-09-13T10:12:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115182+0000", 2024-09-13T10:12:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0027552739999999998 2024-09-13T10:12:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115305+0000", 2024-09-13T10:12:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012364200000000001 2024-09-13T10:12:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.044 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.136435+0000", 2024-09-13T10:12:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021129479999999999 2024-09-13T10:12:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143376+0000", 2024-09-13T10:12:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0069408600000000001 2024-09-13T10:12:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.150416+0000", 2024-09-13T10:12:01.045 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0070403669999999996 2024-09-13T10:12:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.151872+0000", 2024-09-13T10:12:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001456286 2024-09-13T10:12:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.151906+0000", 2024-09-13T10:12:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3167999999999998e-05 2024-09-13T10:12:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.046 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.151934+0000", 2024-09-13T10:12:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8490999999999999e-05 2024-09-13T10:12:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:250 3.2 3:78d40eb9:::smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.115099+0000", 2024-09-13T10:12:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.825404160000005, 2024-09-13T10:12:01.047 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.028750412999999999, 2024-09-13T10:12:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 250 2024-09-13T10:12:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:01.048 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115099+0000", 2024-09-13T10:12:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115099+0000", 2024-09-13T10:12:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115101+0000", 2024-09-13T10:12:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1789999999999998e-06 2024-09-13T10:12:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.049 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115132+0000", 2024-09-13T10:12:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1163e-05 2024-09-13T10:12:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115150+0000", 2024-09-13T10:12:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7864999999999999e-05 2024-09-13T10:12:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115159+0000", 2024-09-13T10:12:01.050 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1719999999999996e-06 2024-09-13T10:12:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115440+0000", 2024-09-13T10:12:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028068199999999998 2024-09-13T10:12:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115579+0000", 2024-09-13T10:12:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013910800000000001 2024-09-13T10:12:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.051 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.132849+0000", 2024-09-13T10:12:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017270127999999999 2024-09-13T10:12:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142332+0000", 2024-09-13T10:12:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0094831859999999994 2024-09-13T10:12:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143034+0000", 2024-09-13T10:12:01.052 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00070226499999999999 2024-09-13T10:12:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143793+0000", 2024-09-13T10:12:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00075876700000000004 2024-09-13T10:12:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143825+0000", 2024-09-13T10:12:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2017999999999997e-05 2024-09-13T10:12:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.053 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143849+0000", 2024-09-13T10:12:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3879999999999998e-05 2024-09-13T10:12:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.7.22:3 3.2 3:5915c927:::smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.216433+0000", 2024-09-13T10:12:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.724069782000001, 2024-09-13T10:12:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.037069469000000001, 2024-09-13T10:12:01.054 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216433+0000", 2024-09-13T10:12:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216433+0000", 2024-09-13T10:12:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.055 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216436+0000", 2024-09-13T10:12:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7800000000000001e-06 2024-09-13T10:12:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216473+0000", 2024-09-13T10:12:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6931999999999998e-05 2024-09-13T10:12:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.056 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216512+0000", 2024-09-13T10:12:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.8831999999999997e-05 2024-09-13T10:12:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216539+0000", 2024-09-13T10:12:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7705999999999999e-05 2024-09-13T10:12:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216732+0000", 2024-09-13T10:12:01.057 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019224500000000001 2024-09-13T10:12:01.058 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.058 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.058 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:01.058 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217000+0000", 2024-09-13T10:12:01.058 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026813100000000001 2024-09-13T10:12:01.058 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.058 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.058 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:01.058 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217638+0000", 2024-09-13T10:12:01.058 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00063874400000000001 2024-09-13T10:12:01.058 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.058 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.059 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:01.059 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.219940+0000", 2024-09-13T10:12:01.059 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023015890000000002 2024-09-13T10:12:01.059 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.059 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.059 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.059 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.220304+0000", 2024-09-13T10:12:01.059 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036399099999999998 2024-09-13T10:12:01.059 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.059 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.059 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.059 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253232+0000", 2024-09-13T10:12:01.060 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032927640000000001 2024-09-13T10:12:01.060 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.060 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.060 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:01.060 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253393+0000", 2024-09-13T10:12:01.060 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000161142 2024-09-13T10:12:01.060 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.060 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.060 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:01.060 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253503+0000", 2024-09-13T10:12:01.060 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000109737 2024-09-13T10:12:01.060 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.060 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:01.061 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.061 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.061 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.061 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.2.23:2 3.2 3:46ab5247:::smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:01.061 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.217995+0000", 2024-09-13T10:12:01.061 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 69.722508310999999, 2024-09-13T10:12:01.061 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036081690999999999, 2024-09-13T10:12:01.061 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:01.061 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:01.061 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:01.061 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:01.061 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.061 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:01.062 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217995+0000", 2024-09-13T10:12:01.062 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.062 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.062 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.062 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:01.062 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217995+0000", 2024-09-13T10:12:01.062 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.062 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.062 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.062 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:01.062 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217997+0000", 2024-09-13T10:12:01.062 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3429999999999998e-06 2024-09-13T10:12:01.063 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.063 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.063 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:01.063 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218007+0000", 2024-09-13T10:12:01.063 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.91e-06 2024-09-13T10:12:01.063 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.063 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.063 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:01.063 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218024+0000", 2024-09-13T10:12:01.063 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7549000000000001e-05 2024-09-13T10:12:01.063 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.063 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.064 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:01.064 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218038+0000", 2024-09-13T10:12:01.064 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3509999999999999e-05 2024-09-13T10:12:01.064 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.064 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.064 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:01.064 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218384+0000", 2024-09-13T10:12:01.064 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00034618499999999999 2024-09-13T10:12:01.064 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.064 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.064 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:01.064 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218605+0000", 2024-09-13T10:12:01.064 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022059400000000001 2024-09-13T10:12:01.065 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.065 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.065 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:01.065 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.219124+0000", 2024-09-13T10:12:01.065 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00051934699999999997 2024-09-13T10:12:01.065 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.065 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.065 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.065 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.222285+0000", 2024-09-13T10:12:01.065 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0031605790000000002 2024-09-13T10:12:01.065 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.065 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.066 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:01.066 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.224618+0000", 2024-09-13T10:12:01.066 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023338209999999998 2024-09-13T10:12:01.066 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.066 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.066 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.066 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253766+0000", 2024-09-13T10:12:01.066 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029147890999999999 2024-09-13T10:12:01.066 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.066 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.066 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:01.067 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253920+0000", 2024-09-13T10:12:01.067 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015327299999999999 2024-09-13T10:12:01.067 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.067 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.067 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:01.067 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254076+0000", 2024-09-13T10:12:01.067 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000156689 2024-09-13T10:12:01.067 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.067 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:01.067 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.067 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.067 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.067 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1854 4.b 4:d8e9927b:::smithi19530714-44:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:01.068 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:10.749110+0000", 2024-09-13T10:12:01.068 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 50.191393220999998, 2024-09-13T10:12:01.068 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.049047407000000001, 2024-09-13T10:12:01.068 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:01.068 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:01.068 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:01.068 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:01.068 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:01.068 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:01.068 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1854 2024-09-13T10:12:01.068 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.068 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:01.068 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.069 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:01.069 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749110+0000", 2024-09-13T10:12:01.069 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.069 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.069 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.069 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:01.069 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749110+0000", 2024-09-13T10:12:01.069 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.069 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.069 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.069 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:01.069 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749113+0000", 2024-09-13T10:12:01.070 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0970000000000002e-06 2024-09-13T10:12:01.070 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.070 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.070 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:01.070 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749125+0000", 2024-09-13T10:12:01.070 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2048999999999999e-05 2024-09-13T10:12:01.070 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.070 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.070 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:01.070 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749164+0000", 2024-09-13T10:12:01.070 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9579999999999997e-05 2024-09-13T10:12:01.070 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.071 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.071 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:01.071 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749188+0000", 2024-09-13T10:12:01.071 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4026e-05 2024-09-13T10:12:01.071 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.071 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.071 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:01.071 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749320+0000", 2024-09-13T10:12:01.071 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013178500000000001 2024-09-13T10:12:01.071 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.071 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.071 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:01.071 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749506+0000", 2024-09-13T10:12:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00018623700000000001 2024-09-13T10:12:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.798157+0000", 2024-09-13T10:12:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048650632999999999 2024-09-13T10:12:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3445 3.2 3:7ba83ace:::4c246a1df6a0d5acaf0a3a85249ff707b21f90a5f3db763dc6d544e5b7cbec73:head [write 0~3552768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86)", 2024-09-13T10:12:01.072 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:48.123137+0000", 2024-09-13T10:12:01.073 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 12.81736538, 2024-09-13T10:12:01.073 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021450407000000001, 2024-09-13T10:12:01.073 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:01.073 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:01.073 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:01.073 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:01.073 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:01.073 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:01.073 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3445 2024-09-13T10:12:01.073 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.073 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:01.073 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.073 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:01.074 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.123137+0000", 2024-09-13T10:12:01.074 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.074 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.074 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.074 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:01.074 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.123137+0000", 2024-09-13T10:12:01.074 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:01.074 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.074 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.074 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:01.074 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.123142+0000", 2024-09-13T10:12:01.074 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1999999999999996e-06 2024-09-13T10:12:01.074 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.074 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.074 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:01.075 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.128749+0000", 2024-09-13T10:12:01.075 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0056071949999999997 2024-09-13T10:12:01.075 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.075 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.075 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:01.075 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.128778+0000", 2024-09-13T10:12:01.075 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9209000000000001e-05 2024-09-13T10:12:01.075 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.075 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.075 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:01.075 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.128802+0000", 2024-09-13T10:12:01.075 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4389999999999999e-05 2024-09-13T10:12:01.075 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.075 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.076 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:01.076 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.128982+0000", 2024-09-13T10:12:01.076 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017994500000000001 2024-09-13T10:12:01.076 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.076 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.076 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:01.076 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.129246+0000", 2024-09-13T10:12:01.076 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000263224 2024-09-13T10:12:01.076 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.076 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.076 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:01.076 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.129498+0000", 2024-09-13T10:12:01.076 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025232999999999999 2024-09-13T10:12:01.077 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.077 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.077 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.077 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.140550+0000", 2024-09-13T10:12:01.077 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011051983 2024-09-13T10:12:01.077 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.077 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.077 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:01.077 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.141160+0000", 2024-09-13T10:12:01.077 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00061049200000000002 2024-09-13T10:12:01.077 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.077 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.078 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:01.078 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.144404+0000", 2024-09-13T10:12:01.078 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00324355 2024-09-13T10:12:01.078 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.078 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.078 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:01.078 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.144495+0000", 2024-09-13T10:12:01.078 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1038000000000005e-05 2024-09-13T10:12:01.078 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:01.078 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:01.078 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:01.078 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.144588+0000", 2024-09-13T10:12:01.079 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.2850999999999996e-05 2024-09-13T10:12:01.079 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.079 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:01.079 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.079 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:01.079 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:01.079 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:01.080 INFO:tasks.rados.rados.0.smithi195.stdout:1675: expect (ObjNum 337 snap 0 seq_num 337) 2024-09-13T10:12:01.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:12:00.939+0000 7f21e3fff640 0 -- 172.21.15.73:0/4058432432 >> v2:172.21.15.136:6812/4083982131 conn(0x7f21e421c530 msgr2=0x7f21e421e920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:12:01.081 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:12:01.081 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:12:01.082 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:12:01.082 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T10:12:01.083 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:12:01.083 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:12:01.084 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:12:01.075+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:12:01.182 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:01.178+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:01.211 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:01.211 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:01.224 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T10:12:01.267 INFO:tasks.rados.rados.0.smithi195.stdout:1680: finishing write tid 1 to smithi19530714-35 2024-09-13T10:12:01.267 INFO:tasks.rados.rados.0.smithi195.stdout:1679: expect (ObjNum 221 snap 0 seq_num 221) 2024-09-13T10:12:01.274 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:12:01.274 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:12:01.274 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:12:01.274 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-13T10:12:01.274 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-13T10:12:01.282 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:01.282+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:01.337 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:01.337 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:01.350 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T10:12:01.383 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:12:01.378+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:12:01.456 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2868776 2024-09-13T10:12:01.475 INFO:tasks.rados.rados.0.smithi195.stdout:1680: finishing write tid 2 to smithi19530714-35 2024-09-13T10:12:01.475 INFO:tasks.rados.rados.0.smithi195.stdout:1680: finishing write tid 3 to smithi19530714-35 2024-09-13T10:12:01.475 INFO:tasks.rados.rados.0.smithi195.stdout:1680: finishing write tid 4 to smithi19530714-35 2024-09-13T10:12:01.475 INFO:tasks.rados.rados.0.smithi195.stdout:1680: oid 35 updating version 0 to 88 2024-09-13T10:12:01.475 INFO:tasks.rados.rados.0.smithi195.stdout:1680: oid 35 version 88 is already newer than 87 2024-09-13T10:12:01.475 INFO:tasks.rados.rados.0.smithi195.stdout:1680: oid 35 version 88 is already newer than 86 2024-09-13T10:12:01.475 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 88 (ObjNum 358 snap 0 seq_num 358) dirty exists 2024-09-13T10:12:01.475 INFO:tasks.rados.rados.0.smithi195.stdout:1680: left oid 35 (ObjNum 358 snap 0 seq_num 358) 2024-09-13T10:12:01.475 INFO:tasks.rados.rados.0.smithi195.stdout:1682: seq_num 359 ranges {433713=482982,1424565=609148} 2024-09-13T10:12:01.484 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:01.479+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:01.489 INFO:tasks.rados.rados.0.smithi195.stdout:1682: writing smithi19530714-11 from 433713 to 916695 tid 1 2024-09-13T10:12:01.508 INFO:tasks.rados.rados.0.smithi195.stdout:1682: writing smithi19530714-11 from 1424565 to 2033713 tid 2 2024-09-13T10:12:01.508 INFO:tasks.rados.rados.0.smithi195.stdout:1673: done (9 left) 2024-09-13T10:12:01.508 INFO:tasks.rados.rados.0.smithi195.stdout:1674: done (8 left) 2024-09-13T10:12:01.508 INFO:tasks.rados.rados.0.smithi195.stdout:1675: done (7 left) 2024-09-13T10:12:01.508 INFO:tasks.rados.rados.0.smithi195.stdout:1676: done (6 left) 2024-09-13T10:12:01.508 INFO:tasks.rados.rados.0.smithi195.stdout:1677: done (5 left) 2024-09-13T10:12:01.508 INFO:tasks.rados.rados.0.smithi195.stdout:1678: done (4 left) 2024-09-13T10:12:01.508 INFO:tasks.rados.rados.0.smithi195.stdout:1679: done (3 left) 2024-09-13T10:12:01.508 INFO:tasks.rados.rados.0.smithi195.stdout:1680: done (2 left) 2024-09-13T10:12:01.508 INFO:tasks.rados.rados.0.smithi195.stdout:1681: done (1 left) 2024-09-13T10:12:01.509 INFO:tasks.rados.rados.0.smithi195.stdout:1683: write (excl) oid 5 current snap is 0 2024-09-13T10:12:01.509 INFO:tasks.rados.rados.0.smithi195.stdout:1682: finishing write tid 1 to smithi19530714-11 2024-09-13T10:12:01.509 INFO:tasks.rados.rados.0.smithi195.stdout:1683: seq_num 360 ranges {450520=558231,1483047=515125} 2024-09-13T10:12:01.512 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:01.512 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:01.523 INFO:tasks.rados.rados.0.smithi195.stdout:1683: writing smithi19530714-5 from 450520 to 1008751 tid 1 2024-09-13T10:12:01.526 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T10:12:01.537 INFO:tasks.rados.rados.0.smithi195.stdout:1683: writing smithi19530714-5 from 1483047 to 1998172 tid 2 2024-09-13T10:12:01.537 INFO:tasks.rados.rados.0.smithi195.stdout:1684: set_chunk oid 47 target oid 2024-09-13T10:12:01.543 INFO:tasks.rados.rados.0.smithi195.stdout:1684: set_chunk oid 47 offset: 14848 length: 180224 target oid 04db6bc901fd5824d01b77b0923abf5d0a1b25c8bc32dcf1d569f2b5dfe609e3 offset: 0 snap -1 2024-09-13T10:12:01.544 INFO:tasks.rados.rados.0.smithi195.stdout:1685: read oid 33 snap -1 2024-09-13T10:12:01.544 INFO:tasks.rados.rados.0.smithi195.stdout:1685: expect deleted 2024-09-13T10:12:01.544 INFO:tasks.rados.rados.0.smithi195.stdout:1686: set_chunk oid 25 target oid 2024-09-13T10:12:01.560 INFO:tasks.rados.rados.0.smithi195.stdout:1686: set_chunk oid 25 offset: 1329664 length: 356864 target oid 06cd10300b0fd6d314a6cd3b936c1143537ac69c6e35d271db6f99afaa44d5e7 offset: 0 snap -1 2024-09-13T10:12:01.560 INFO:tasks.rados.rados.0.smithi195.stdout:1687: read oid 12 snap -1 2024-09-13T10:12:01.560 INFO:tasks.rados.rados.0.smithi195.stdout:1687: expect (ObjNum 334 snap 0 seq_num 334) 2024-09-13T10:12:01.560 INFO:tasks.rados.rados.0.smithi195.stdout:1688: read oid 23 snap -1 2024-09-13T10:12:01.560 INFO:tasks.rados.rados.0.smithi195.stdout:1688: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:12:01.560 INFO:tasks.rados.rados.0.smithi195.stdout:1689: read oid 22 snap -1 2024-09-13T10:12:01.560 INFO:tasks.rados.rados.0.smithi195.stdout:1689: expect (ObjNum 341 snap 0 seq_num 341) 2024-09-13T10:12:01.560 INFO:tasks.rados.rados.0.smithi195.stdout:1690: read oid 45 snap -1 2024-09-13T10:12:01.560 INFO:tasks.rados.rados.0.smithi195.stdout:1690: expect (ObjNum 311 snap 0 seq_num 311) 2024-09-13T10:12:01.561 INFO:tasks.rados.rados.0.smithi195.stdout:1691: read oid 4 snap -1 2024-09-13T10:12:01.561 INFO:tasks.rados.rados.0.smithi195.stdout:1691: expect (ObjNum 352 snap 0 seq_num 352) 2024-09-13T10:12:01.561 INFO:tasks.rados.rados.0.smithi195.stdout:1692: read oid 32 snap -1 2024-09-13T10:12:01.561 INFO:tasks.rados.rados.0.smithi195.stdout:1692: expect (ObjNum 320 snap 0 seq_num 320) 2024-09-13T10:12:01.561 INFO:tasks.rados.rados.0.smithi195.stdout:1693: read oid 18 snap -1 2024-09-13T10:12:01.561 INFO:tasks.rados.rados.0.smithi195.stdout:1693: expect (ObjNum 338 snap 0 seq_num 338) 2024-09-13T10:12:01.561 INFO:tasks.rados.rados.0.smithi195.stdout:1694: write (excl) oid 15 current snap is 0 2024-09-13T10:12:01.561 INFO:tasks.rados.rados.0.smithi195.stdout:1682: finishing write tid 2 to smithi19530714-11 2024-09-13T10:12:01.561 INFO:tasks.rados.rados.0.smithi195.stdout:1682: finishing write tid 3 to smithi19530714-11 2024-09-13T10:12:01.561 INFO:tasks.rados.rados.0.smithi195.stdout:1694: seq_num 361 ranges {467327=633480,1825176=242151} 2024-09-13T10:12:01.568 INFO:tasks.rados.rados.0.smithi195.stdout:1694: writing smithi19530714-15 from 467327 to 1100807 tid 1 2024-09-13T10:12:01.577 INFO:tasks.rados.rados.0.smithi195.stdout:1694: writing smithi19530714-15 from 1825176 to 2067327 tid 2 2024-09-13T10:12:01.578 INFO:tasks.rados.rados.0.smithi195.stdout:1695: read oid 34 snap -1 2024-09-13T10:12:01.578 INFO:tasks.rados.rados.0.smithi195.stdout:1695: expect deleted 2024-09-13T10:12:01.578 INFO:tasks.rados.rados.0.smithi195.stdout:1696: read oid 2 snap -1 2024-09-13T10:12:01.578 INFO:tasks.rados.rados.0.smithi195.stdout:1696: expect (ObjNum 280 snap 0 seq_num 280) 2024-09-13T10:12:01.578 INFO:tasks.rados.rados.0.smithi195.stdout:1697: tier_evict oid 14 2024-09-13T10:12:01.578 INFO:tasks.rados.rados.0.smithi195.stdout:1697: tier_evict oid 14 snap -1 2024-09-13T10:12:01.578 INFO:tasks.rados.rados.0.smithi195.stdout:1682: finishing write tid 4 to smithi19530714-11 2024-09-13T10:12:01.578 INFO:tasks.rados.rados.0.smithi195.stdout:1682: oid 11 updating version 0 to 96 2024-09-13T10:12:01.578 INFO:tasks.rados.rados.0.smithi195.stdout:1682: oid 11 version 96 is already newer than 95 2024-09-13T10:12:01.578 INFO:tasks.rados.rados.0.smithi195.stdout:1682: oid 11 version 96 is already newer than 94 2024-09-13T10:12:01.578 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 96 (ObjNum 359 snap 0 seq_num 359) dirty exists 2024-09-13T10:12:01.578 INFO:tasks.rados.rados.0.smithi195.stdout:1682: left oid 11 (ObjNum 359 snap 0 seq_num 359) 2024-09-13T10:12:01.579 INFO:tasks.rados.rados.0.smithi195.stdout:1683: finishing write tid 1 to smithi19530714-5 2024-09-13T10:12:01.579 INFO:tasks.rados.rados.0.smithi195.stdout:1682: done (15 left) 2024-09-13T10:12:01.579 INFO:tasks.rados.rados.0.smithi195.stdout:1698: write oid 40 current snap is 0 2024-09-13T10:12:01.579 INFO:tasks.rados.rados.0.smithi195.stdout:1683: finishing write tid 2 to smithi19530714-5 2024-09-13T10:12:01.579 INFO:tasks.rados.rados.0.smithi195.stdout:1683: finishing write tid 3 to smithi19530714-5 2024-09-13T10:12:01.579 INFO:tasks.rados.rados.0.smithi195.stdout:1698: seq_num 362 ranges {484134=708729,1883658=200476} 2024-09-13T10:12:01.584 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:01.579+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:01.588 INFO:tasks.rados.rados.0.smithi195.stdout:1698: writing smithi19530714-40 from 484134 to 1192863 tid 1 2024-09-13T10:12:01.597 INFO:tasks.rados.rados.0.smithi195.stdout:1698: writing smithi19530714-40 from 1883658 to 2084134 tid 2 2024-09-13T10:12:01.597 INFO:tasks.rados.rados.0.smithi195.stdout:1685: done (15 left) 2024-09-13T10:12:01.597 INFO:tasks.rados.rados.0.smithi195.stdout:1699: read oid 3 snap -1 2024-09-13T10:12:01.598 INFO:tasks.rados.rados.0.smithi195.stdout:1699: expect (ObjNum 328 snap 0 seq_num 328) 2024-09-13T10:12:01.598 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:12:01.598 INFO:tasks.rados.rados.0.smithi195.stdout:1683: finishing write tid 4 to smithi19530714-5 2024-09-13T10:12:01.598 INFO:tasks.rados.rados.0.smithi195.stdout:1683: oid 5 updating version 0 to 30 2024-09-13T10:12:01.598 INFO:tasks.rados.rados.0.smithi195.stdout:1683: oid 5 version 30 is already newer than 29 2024-09-13T10:12:01.598 INFO:tasks.rados.rados.0.smithi195.stdout:1683: oid 5 version 30 is already newer than 28 2024-09-13T10:12:01.598 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 30 (ObjNum 360 snap 0 seq_num 360) dirty exists 2024-09-13T10:12:01.598 INFO:tasks.rados.rados.0.smithi195.stdout:1683: left oid 5 (ObjNum 360 snap 0 seq_num 360) 2024-09-13T10:12:01.598 INFO:tasks.rados.rados.0.smithi195.stdout:1684: finishing set_chunk to oid 47 2024-09-13T10:12:01.598 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 54 (ObjNum 260 snap 0 seq_num 260) dirty exists 2024-09-13T10:12:01.598 INFO:tasks.rados.rados.0.smithi195.stdout:1683: done (15 left) 2024-09-13T10:12:01.598 INFO:tasks.rados.rados.0.smithi195.stdout:1684: done (14 left) 2024-09-13T10:12:01.598 INFO:tasks.rados.rados.0.smithi195.stdout:1700: tier_promote oid 21 2024-09-13T10:12:01.598 INFO:tasks.rados.rados.0.smithi195.stdout:1701: read oid 37 snap -1 2024-09-13T10:12:01.599 INFO:tasks.rados.rados.0.smithi195.stdout:1701: expect deleted 2024-09-13T10:12:01.599 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:12:01.599 INFO:tasks.rados.rados.0.smithi195.stdout:1688: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:12:01.599 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:01.601 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-09-13T10:12:01.597+0000 7fc6a0050640 -1 mgr.server reply reply (4) Interrupted system call no change 2024-09-13T10:12:01.601 INFO:teuthology.orchestra.run.smithi073.stdout:moved 10 / 117 (8.54701%) 2024-09-13T10:12:01.601 INFO:teuthology.orchestra.run.smithi073.stdout:avg 7.3125 2024-09-13T10:12:01.601 INFO:teuthology.orchestra.run.smithi073.stdout:stddev 2.36428 -> 2.31081 (expected baseline 2.61829) 2024-09-13T10:12:01.602 INFO:teuthology.orchestra.run.smithi073.stdout:min osd.11 with 4 -> 4 pgs (0.547009 -> 0.547009 * mean) 2024-09-13T10:12:01.602 INFO:teuthology.orchestra.run.smithi073.stdout:max osd.3 with 12 -> 10 pgs (1.64103 -> 1.36752 * mean) 2024-09-13T10:12:01.602 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:12:01.602 INFO:teuthology.orchestra.run.smithi073.stdout:oload 110 2024-09-13T10:12:01.602 INFO:teuthology.orchestra.run.smithi073.stdout:max_change 1 2024-09-13T10:12:01.602 INFO:teuthology.orchestra.run.smithi073.stdout:max_change_osds 4 2024-09-13T10:12:01.602 INFO:teuthology.orchestra.run.smithi073.stdout:average_utilization 0.0007 2024-09-13T10:12:01.602 INFO:teuthology.orchestra.run.smithi073.stdout:overload_utilization 0.0007 2024-09-13T10:12:01.602 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4 weight 1.0000 -> 0.6034 2024-09-13T10:12:01.602 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3 weight 1.0000 -> 0.6842 2024-09-13T10:12:01.602 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14 weight 1.0000 -> 0.7048 2024-09-13T10:12:01.602 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7 weight 1.0000 -> 0.8202 2024-09-13T10:12:01.603 INFO:teuthology.orchestra.run.smithi073.stderr:no change 2024-09-13T10:12:01.621 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-13T10:12:01.621 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-09-13T10:12:01.658 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:01.658 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:01.671 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T10:12:01.685 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:01.682+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:01.770 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T10:12:01.780 INFO:tasks.rados.rados.0.smithi195.stdout:1688: done (15 left) 2024-09-13T10:12:01.780 INFO:tasks.rados.rados.0.smithi195.stdout:1695: done (14 left) 2024-09-13T10:12:01.780 INFO:tasks.rados.rados.0.smithi195.stdout:1702: read oid 28 snap -1 2024-09-13T10:12:01.780 INFO:tasks.rados.rados.0.smithi195.stdout:1702: expect (ObjNum 180 snap 0 seq_num 180) 2024-09-13T10:12:01.780 INFO:tasks.rados.rados.0.smithi195.stdout:1703: tier_evict oid 43 2024-09-13T10:12:01.780 INFO:tasks.rados.rados.0.smithi195.stdout:1703: tier_evict oid 43 snap -1 2024-09-13T10:12:01.780 INFO:tasks.rados.rados.0.smithi195.stdout:1690: expect (ObjNum 311 snap 0 seq_num 311) 2024-09-13T10:12:01.786 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:01.782+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:01.829 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2554395 2024-09-13T10:12:01.837 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:01.838 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:01.851 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T10:12:01.896 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:01.893+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:01.962 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:01.962 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:01.977 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-13T10:12:01.978 INFO:tasks.rados.rados.0.smithi195.stdout:1692: expect (ObjNum 320 snap 0 seq_num 320) 2024-09-13T10:12:01.978 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout:moved 0 / 117 (0%) 2024-09-13T10:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout:avg 7.3125 2024-09-13T10:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout:stddev 2.36428 -> 2.36428 (expected baseline 2.61829) 2024-09-13T10:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout:min osd.11 with 4 -> 4 pgs (0.547009 -> 0.547009 * mean) 2024-09-13T10:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout:max osd.3 with 12 -> 12 pgs (1.64103 -> 1.64103 * mean) 2024-09-13T10:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout:oload 1000 2024-09-13T10:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout:max_change 3 2024-09-13T10:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout:max_change_osds 4 2024-09-13T10:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout:average_utilization 0.0007 2024-09-13T10:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout:overload_utilization 0.0068 2024-09-13T10:12:01.980 INFO:teuthology.orchestra.run.smithi073.stderr:no change 2024-09-13T10:12:01.999 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-13T10:12:01.999 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-13T10:12:02.020 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:02.017+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:02.089 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:02.085+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:02.125 INFO:tasks.rados.rados.0.smithi195.stdout:1691: expect (ObjNum 352 snap 0 seq_num 352) 2024-09-13T10:12:02.131 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:02.131 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:02.144 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-13T10:12:02.190 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:02.187+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:02.194 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:02.291 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:12:02.287+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:12:02.296 INFO:teuthology.orchestra.run.smithi073.stdout:moved 0 / 117 (0%) 2024-09-13T10:12:02.297 INFO:teuthology.orchestra.run.smithi073.stdout:avg 7.3125 2024-09-13T10:12:02.297 INFO:teuthology.orchestra.run.smithi073.stdout:stddev 2.36428 -> 2.36428 (expected baseline 2.61829) 2024-09-13T10:12:02.297 INFO:teuthology.orchestra.run.smithi073.stdout:min osd.11 with 4 -> 4 pgs (0.547009 -> 0.547009 * mean) 2024-09-13T10:12:02.297 INFO:teuthology.orchestra.run.smithi073.stdout:max osd.3 with 12 -> 12 pgs (1.64103 -> 1.64103 * mean) 2024-09-13T10:12:02.297 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:12:02.297 INFO:teuthology.orchestra.run.smithi073.stdout:oload 1000 2024-09-13T10:12:02.297 INFO:teuthology.orchestra.run.smithi073.stdout:max_change 3 2024-09-13T10:12:02.297 INFO:teuthology.orchestra.run.smithi073.stdout:max_change_osds 4 2024-09-13T10:12:02.297 INFO:teuthology.orchestra.run.smithi073.stdout:average_utilization 0.0007 2024-09-13T10:12:02.297 INFO:teuthology.orchestra.run.smithi073.stdout:overload_utilization 0.0068 2024-09-13T10:12:02.298 INFO:teuthology.orchestra.run.smithi073.stderr:no change 2024-09-13T10:12:02.304 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:02.305 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:02.317 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-13T10:12:02.317 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 1.0 2024-09-13T10:12:02.319 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-13T10:12:02.323 INFO:tasks.rados.rados.0.smithi195.stdout:1687: expect (ObjNum 334 snap 0 seq_num 334) 2024-09-13T10:12:02.365 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:02.410 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:12:02.407+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:12:02.417 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:02.440 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:02.441 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:02.454 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-13T10:12:02.468 INFO:tasks.rados.rados.0.smithi195.stdout:1689: expect (ObjNum 341 snap 0 seq_num 341) 2024-09-13T10:12:02.480 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2145153 2024-09-13T10:12:02.492 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:02.489+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:02.498 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1669278 2024-09-13T10:12:02.524 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:02.562 INFO:tasks.rados.rados.0.smithi195.stdout:1686: finishing set_chunk to oid 25 2024-09-13T10:12:02.562 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 149 (ObjNum 324 snap 0 seq_num 324) dirty exists 2024-09-13T10:12:02.562 INFO:tasks.rados.rados.0.smithi195.stdout:1694: finishing write tid 1 to smithi19530714-15 2024-09-13T10:12:02.562 INFO:tasks.rados.rados.0.smithi195.stdout:1686: done (15 left) 2024-09-13T10:12:02.562 INFO:tasks.rados.rados.0.smithi195.stdout:1687: done (14 left) 2024-09-13T10:12:02.562 INFO:tasks.rados.rados.0.smithi195.stdout:1689: done (13 left) 2024-09-13T10:12:02.562 INFO:tasks.rados.rados.0.smithi195.stdout:1690: done (12 left) 2024-09-13T10:12:02.562 INFO:tasks.rados.rados.0.smithi195.stdout:1691: done (11 left) 2024-09-13T10:12:02.563 INFO:tasks.rados.rados.0.smithi195.stdout:1692: done (10 left) 2024-09-13T10:12:02.563 INFO:tasks.rados.rados.0.smithi195.stdout:1704: tier_promote oid 7 2024-09-13T10:12:02.563 INFO:tasks.rados.rados.0.smithi195.stdout:1697: got (22) Invalid argument 2024-09-13T10:12:02.563 INFO:tasks.rados.rados.0.smithi195.stdout:1694: finishing write tid 2 to smithi19530714-15 2024-09-13T10:12:02.563 INFO:tasks.rados.rados.0.smithi195.stdout:1694: finishing write tid 3 to smithi19530714-15 2024-09-13T10:12:02.563 INFO:tasks.rados.rados.0.smithi195.stdout:1694: finishing write tid 4 to smithi19530714-15 2024-09-13T10:12:02.563 INFO:tasks.rados.rados.0.smithi195.stdout:1694: oid 15 updating version 0 to 111 2024-09-13T10:12:02.563 INFO:tasks.rados.rados.0.smithi195.stdout:1694: oid 15 version 111 is already newer than 109 2024-09-13T10:12:02.563 INFO:tasks.rados.rados.0.smithi195.stdout:1694: oid 15 version 111 is already newer than 110 2024-09-13T10:12:02.563 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 111 (ObjNum 361 snap 0 seq_num 361) dirty exists 2024-09-13T10:12:02.563 INFO:tasks.rados.rados.0.smithi195.stdout:1694: left oid 15 (ObjNum 361 snap 0 seq_num 361) 2024-09-13T10:12:02.563 INFO:tasks.rados.rados.0.smithi195.stdout:1693: expect (ObjNum 338 snap 0 seq_num 338) 2024-09-13T10:12:02.563 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:02.593 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:02.589+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:02.594 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:02.594 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:02.607 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-13T10:12:02.637 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1289498 2024-09-13T10:12:02.667 INFO:teuthology.orchestra.run.smithi073.stdout:moved 0 / 117 (0%) 2024-09-13T10:12:02.667 INFO:teuthology.orchestra.run.smithi073.stdout:avg 7.3125 2024-09-13T10:12:02.667 INFO:teuthology.orchestra.run.smithi073.stdout:stddev 2.36428 -> 2.36428 (expected baseline 2.61829) 2024-09-13T10:12:02.668 INFO:teuthology.orchestra.run.smithi073.stdout:min osd.11 with 4 -> 4 pgs (0.547009 -> 0.547009 * mean) 2024-09-13T10:12:02.668 INFO:teuthology.orchestra.run.smithi073.stdout:max osd.3 with 12 -> 12 pgs (1.64103 -> 1.64103 * mean) 2024-09-13T10:12:02.668 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:12:02.668 INFO:teuthology.orchestra.run.smithi073.stdout:oload 1000 2024-09-13T10:12:02.668 INFO:teuthology.orchestra.run.smithi073.stdout:max_change 1 2024-09-13T10:12:02.668 INFO:teuthology.orchestra.run.smithi073.stdout:max_change_osds 4 2024-09-13T10:12:02.668 INFO:teuthology.orchestra.run.smithi073.stdout:average_utilization 0.0007 2024-09-13T10:12:02.668 INFO:teuthology.orchestra.run.smithi073.stdout:overload_utilization 0.0068 2024-09-13T10:12:02.668 INFO:teuthology.orchestra.run.smithi073.stderr:no change 2024-09-13T10:12:02.685 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-13T10:12:02.686 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-13T10:12:02.694 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:02.689+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:02.708 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-13T10:12:02.750 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:02.750 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:02.753 INFO:tasks.rados.rados.0.smithi195.stdout:1698: finishing write tid 1 to smithi19530714-40 2024-09-13T10:12:02.753 INFO:tasks.rados.rados.0.smithi195.stdout:1698: finishing write tid 2 to smithi19530714-40 2024-09-13T10:12:02.753 INFO:tasks.rados.rados.0.smithi195.stdout:1698: finishing write tid 3 to smithi19530714-40 2024-09-13T10:12:02.753 INFO:tasks.rados.rados.0.smithi195.stdout:1698: finishing write tid 4 to smithi19530714-40 2024-09-13T10:12:02.753 INFO:tasks.rados.rados.0.smithi195.stdout:1698: oid 40 updating version 0 to 27 2024-09-13T10:12:02.753 INFO:tasks.rados.rados.0.smithi195.stdout:1698: oid 40 updating version 27 to 28 2024-09-13T10:12:02.753 INFO:tasks.rados.rados.0.smithi195.stdout:1698: oid 40 updating version 28 to 29 2024-09-13T10:12:02.753 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 29 (ObjNum 362 snap 0 seq_num 362) dirty exists 2024-09-13T10:12:02.753 INFO:tasks.rados.rados.0.smithi195.stdout:1698: left oid 40 (ObjNum 362 snap 0 seq_num 362) 2024-09-13T10:12:02.753 INFO:tasks.rados.rados.0.smithi195.stdout:1696: expect (ObjNum 280 snap 0 seq_num 280) 2024-09-13T10:12:02.764 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-13T10:12:02.797 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:12:02.794+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:12:02.861 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:02.897 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:02.893+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:02.917 INFO:tasks.rados.rados.0.smithi195.stdout:1699: expect (ObjNum 328 snap 0 seq_num 328) 2024-09-13T10:12:02.917 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:02.932 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:02.932 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:02.946 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T10:12:03.001 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:12:02.998+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:12:03.057 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:03.061 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:03.061 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:03.074 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T10:12:03.084 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:12:03.118 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:12:03.115+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:12:03.127 INFO:tasks.rados.rados.0.smithi195.stdout:1700: got (0) Success 2024-09-13T10:12:03.127 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 114 (ObjNum 235 snap 0 seq_num 235) dirty exists 2024-09-13T10:12:03.127 INFO:tasks.rados.rados.0.smithi195.stdout:1703: got (22) Invalid argument 2024-09-13T10:12:03.127 INFO:tasks.rados.rados.0.smithi195.stdout:1702: expect (ObjNum 180 snap 0 seq_num 180) 2024-09-13T10:12:03.152 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2231016 2024-09-13T10:12:03.171 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 402284 2024-09-13T10:12:03.188 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:03.188 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:03.205 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T10:12:03.206 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:03.201+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:03.256 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:12:03.256 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:12:03.274 INFO:tasks.rados.rados.0.smithi195.stdout:1704: got (0) Success 2024-09-13T10:12:03.274 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 99 (ObjNum 276 snap 0 seq_num 276) dirty exists 2024-09-13T10:12:03.274 INFO:tasks.rados.rados.0.smithi195.stdout:1693: done (10 left) 2024-09-13T10:12:03.275 INFO:tasks.rados.rados.0.smithi195.stdout:1694: done (9 left) 2024-09-13T10:12:03.275 INFO:tasks.rados.rados.0.smithi195.stdout:1696: done (8 left) 2024-09-13T10:12:03.275 INFO:tasks.rados.rados.0.smithi195.stdout:1697: done (7 left) 2024-09-13T10:12:03.275 INFO:tasks.rados.rados.0.smithi195.stdout:1698: done (6 left) 2024-09-13T10:12:03.275 INFO:tasks.rados.rados.0.smithi195.stdout:1699: done (5 left) 2024-09-13T10:12:03.275 INFO:tasks.rados.rados.0.smithi195.stdout:1700: done (4 left) 2024-09-13T10:12:03.275 INFO:tasks.rados.rados.0.smithi195.stdout:1701: done (3 left) 2024-09-13T10:12:03.275 INFO:tasks.rados.rados.0.smithi195.stdout:1702: done (2 left) 2024-09-13T10:12:03.275 INFO:tasks.rados.rados.0.smithi195.stdout:1703: done (1 left) 2024-09-13T10:12:03.275 INFO:tasks.rados.rados.0.smithi195.stdout:1704: done (0 left) 2024-09-13T10:12:03.275 INFO:tasks.rados.rados.0.smithi195.stdout:1705: set_chunk oid 46 target oid 2024-09-13T10:12:03.277 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:12:03.277 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:12:03.296 INFO:tasks.rados.rados.0.smithi195.stdout:1705: set_chunk oid 46 offset: 1402368 length: 1216512 target oid b7ab8263590e935cc5ec89e7e3adbb475e6e357571c8b15375b352a7d3ecfa16 offset: 0 snap -1 2024-09-13T10:12:03.296 INFO:tasks.rados.rados.0.smithi195.stdout:1706: read oid 49 snap -1 2024-09-13T10:12:03.297 INFO:tasks.rados.rados.0.smithi195.stdout:1706: expect deleted 2024-09-13T10:12:03.297 INFO:tasks.rados.rados.0.smithi195.stdout:1707: delete oid 1 current snap is 0 2024-09-13T10:12:03.299 INFO:tasks.rados.rados.0.smithi195.stdout:1706: done (2 left) 2024-09-13T10:12:03.299 INFO:tasks.rados.rados.0.smithi195.stdout:1707: done (1 left) 2024-09-13T10:12:03.299 INFO:tasks.rados.rados.0.smithi195.stdout:1708: delete oid 30 current snap is 0 2024-09-13T10:12:03.300 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:12:03.300 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:12:03.302 INFO:tasks.rados.rados.0.smithi195.stdout:1705: finishing set_chunk to oid 46 2024-09-13T10:12:03.302 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 66 (ObjNum 329 snap 0 seq_num 329) dirty exists 2024-09-13T10:12:03.303 INFO:tasks.rados.rados.0.smithi195.stdout:1705: done (1 left) 2024-09-13T10:12:03.303 INFO:tasks.rados.rados.0.smithi195.stdout:1708: done (0 left) 2024-09-13T10:12:03.303 INFO:tasks.rados.rados.0.smithi195.stdout:1709: delete oid 33 current snap is 0 2024-09-13T10:12:03.306 INFO:tasks.rados.rados.0.smithi195.stdout:1709: done (0 left) 2024-09-13T10:12:03.306 INFO:tasks.rados.rados.0.smithi195.stdout:1710: read oid 1 snap -1 2024-09-13T10:12:03.306 INFO:tasks.rados.rados.0.smithi195.stdout:1710: expect deleted 2024-09-13T10:12:03.306 INFO:tasks.rados.rados.0.smithi195.stdout:1711: write oid 31 current snap is 0 2024-09-13T10:12:03.306 INFO:tasks.rados.rados.0.smithi195.stdout:1711: seq_num 366 ranges {551362=726078,1717586=409779} 2024-09-13T10:12:03.307 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:12:03.299+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:12:03.318 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:03.318 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:12:03.318 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:12:03.318 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:12:03.318 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.318 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:202 3.2 e38/30 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'1, mlcod=0'0)", 2024-09-13T10:12:03.318 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:12:03.318 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.313282688000001, 2024-09-13T10:12:03.318 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016741669000000001, 2024-09-13T10:12:03.318 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.319 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.319 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.319 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.319 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.319 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:12:03.319 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.319 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.319 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:12:03.319 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002368+0000", 2024-09-13T10:12:03.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-13T10:12:03.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010047+0000", 2024-09-13T10:12:03.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007678748 2024-09-13T10:12:03.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010081+0000", 2024-09-13T10:12:03.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4011999999999999e-05 2024-09-13T10:12:03.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.321 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.321 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010098+0000", 2024-09-13T10:12:03.321 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7125999999999998e-05 2024-09-13T10:12:03.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.321 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.321 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010253+0000", 2024-09-13T10:12:03.321 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000154881 2024-09-13T10:12:03.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.322 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.322 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010294+0000", 2024-09-13T10:12:03.322 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1746000000000003e-05 2024-09-13T10:12:03.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.322 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.322 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.322 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.018922+0000", 2024-09-13T10:12:03.322 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0086279619999999994 2024-09-13T10:12:03.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.322 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.322 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.322 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.019109+0000", 2024-09-13T10:12:03.322 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000186057 2024-09-13T10:12:03.322 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.322 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.323 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.323 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.323 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:206 3.2 e38/30 3:4cbdbe9e:::smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:12:03.323 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:12:03.323 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.304897956999994, 2024-09-13T10:12:03.323 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011224615, 2024-09-13T10:12:03.323 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.323 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.323 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.323 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.323 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.323 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:12:03.324 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.324 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.324 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:12:03.324 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.324 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.324 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010754+0000", 2024-09-13T10:12:03.324 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9999999999999999e-06 2024-09-13T10:12:03.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.325 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.325 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.325 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016750+0000", 2024-09-13T10:12:03.325 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059966480000000003 2024-09-13T10:12:03.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.325 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.325 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.325 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016782+0000", 2024-09-13T10:12:03.325 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2218999999999998e-05 2024-09-13T10:12:03.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.325 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.325 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.325 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016799+0000", 2024-09-13T10:12:03.326 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6685999999999999e-05 2024-09-13T10:12:03.326 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.326 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.326 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.326 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016923+0000", 2024-09-13T10:12:03.326 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000124122 2024-09-13T10:12:03.326 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.326 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.326 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.326 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016962+0000", 2024-09-13T10:12:03.326 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8748999999999998e-05 2024-09-13T10:12:03.326 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.326 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.326 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.327 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021869+0000", 2024-09-13T10:12:03.327 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049073249999999997 2024-09-13T10:12:03.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.327 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.327 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021976+0000", 2024-09-13T10:12:03.327 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106866 2024-09-13T10:12:03.327 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.327 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.327 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.327 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:214 3.2 e38/30 3:68ba95ba:::smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'3, mlcod=0'0)", 2024-09-13T10:12:03.328 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:12:03.328 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.295195301000007, 2024-09-13T10:12:03.328 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011518149, 2024-09-13T10:12:03.328 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.328 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.328 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.328 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.328 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.328 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.328 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:12:03.328 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.328 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.329 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.329 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:12:03.329 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.329 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.329 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.329 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020456+0000", 2024-09-13T10:12:03.329 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0430000000000002e-06 2024-09-13T10:12:03.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.329 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.329 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.329 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026998+0000", 2024-09-13T10:12:03.329 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065420169999999998 2024-09-13T10:12:03.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.330 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.330 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.330 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027030+0000", 2024-09-13T10:12:03.330 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1976e-05 2024-09-13T10:12:03.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.330 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.330 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.330 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027048+0000", 2024-09-13T10:12:03.330 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7615e-05 2024-09-13T10:12:03.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.330 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.330 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.330 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027206+0000", 2024-09-13T10:12:03.331 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015788900000000001 2024-09-13T10:12:03.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.331 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.331 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.331 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027283+0000", 2024-09-13T10:12:03.331 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6942999999999998e-05 2024-09-13T10:12:03.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.331 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.331 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.331 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.031808+0000", 2024-09-13T10:12:03.331 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045256940000000002 2024-09-13T10:12:03.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.331 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.332 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.332 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.031972+0000", 2024-09-13T10:12:03.332 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016397199999999999 2024-09-13T10:12:03.332 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.332 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.332 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.332 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.332 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:210 3.2 e38/30 3:46ab5247:::smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:12:03.332 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:12:03.332 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.287329583000002, 2024-09-13T10:12:03.332 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015681245, 2024-09-13T10:12:03.332 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.333 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.333 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.333 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.333 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.333 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.333 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:12:03.333 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.333 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.333 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.333 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:12:03.333 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.334 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.334 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.334 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028322+0000", 2024-09-13T10:12:03.334 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0200000000000001e-06 2024-09-13T10:12:03.334 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.334 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.334 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.334 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037903+0000", 2024-09-13T10:12:03.334 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095811639999999997 2024-09-13T10:12:03.334 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.334 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.334 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.334 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037945+0000", 2024-09-13T10:12:03.335 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1891000000000003e-05 2024-09-13T10:12:03.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.335 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.335 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.335 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037971+0000", 2024-09-13T10:12:03.335 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5829e-05 2024-09-13T10:12:03.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.335 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.335 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.335 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038125+0000", 2024-09-13T10:12:03.335 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000153852 2024-09-13T10:12:03.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.335 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.336 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.336 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038193+0000", 2024-09-13T10:12:03.336 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7995000000000002e-05 2024-09-13T10:12:03.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.336 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.336 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.336 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.043904+0000", 2024-09-13T10:12:03.336 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0057110340000000002 2024-09-13T10:12:03.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.336 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.336 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.336 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.044001+0000", 2024-09-13T10:12:03.336 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7460000000000005e-05 2024-09-13T10:12:03.337 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.337 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.337 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.337 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.337 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:218 3.2 e38/30 3:5915c927:::smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'5, mlcod=38'2)", 2024-09-13T10:12:03.337 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:12:03.337 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.273966997000002, 2024-09-13T10:12:03.337 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012988942, 2024-09-13T10:12:03.337 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.337 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.337 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.338 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.338 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.338 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.338 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:12:03.338 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.338 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.338 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.338 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:12:03.338 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.338 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.339 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.339 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041685+0000", 2024-09-13T10:12:03.339 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7769999999999998e-06 2024-09-13T10:12:03.339 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.339 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.339 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.339 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046575+0000", 2024-09-13T10:12:03.339 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048899520000000004 2024-09-13T10:12:03.339 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.339 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.339 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.339 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046633+0000", 2024-09-13T10:12:03.339 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7561e-05 2024-09-13T10:12:03.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.340 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.340 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.340 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046657+0000", 2024-09-13T10:12:03.340 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4223000000000002e-05 2024-09-13T10:12:03.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.340 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.340 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.340 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046819+0000", 2024-09-13T10:12:03.340 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016167699999999999 2024-09-13T10:12:03.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.340 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.341 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.341 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046882+0000", 2024-09-13T10:12:03.341 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3467999999999996e-05 2024-09-13T10:12:03.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.341 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.341 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.341 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054529+0000", 2024-09-13T10:12:03.341 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0076467289999999997 2024-09-13T10:12:03.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.341 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.341 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.341 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054671+0000", 2024-09-13T10:12:03.341 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014255499999999999 2024-09-13T10:12:03.342 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.342 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.342 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.342 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.342 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.342 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:217 3.2 e38/30 3:554c406d:::smithi19530714-18 low_tier:head v 38'6, mlcod=38'2)", 2024-09-13T10:12:03.342 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:12:03.342 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.268920206999994, 2024-09-13T10:12:03.342 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013473855999999999, 2024-09-13T10:12:03.342 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.342 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.342 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.342 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.342 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.343 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.343 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:12:03.343 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.343 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.343 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.343 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:12:03.343 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.343 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.343 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.343 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046731+0000", 2024-09-13T10:12:03.343 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.813e-06 2024-09-13T10:12:03.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.344 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.344 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.344 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054994+0000", 2024-09-13T10:12:03.344 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0082631279999999998 2024-09-13T10:12:03.344 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.344 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.344 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.344 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055024+0000", 2024-09-13T10:12:03.344 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9839e-05 2024-09-13T10:12:03.344 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.344 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.344 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.344 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055044+0000", 2024-09-13T10:12:03.345 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9507000000000001e-05 2024-09-13T10:12:03.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.345 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.345 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.345 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055186+0000", 2024-09-13T10:12:03.345 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014243099999999999 2024-09-13T10:12:03.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.345 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.345 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.345 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055240+0000", 2024-09-13T10:12:03.345 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4444999999999998e-05 2024-09-13T10:12:03.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.345 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.346 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.346 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060135+0000", 2024-09-13T10:12:03.346 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048942949999999999 2024-09-13T10:12:03.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.346 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.346 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.346 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060203+0000", 2024-09-13T10:12:03.346 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8398000000000005e-05 2024-09-13T10:12:03.346 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.346 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.346 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.346 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.347 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:220 3.2 e38/30 3:753d5785:::smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'7, mlcod=38'3)", 2024-09-13T10:12:03.347 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:12:03.347 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.260538939, 2024-09-13T10:12:03.347 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.009287455, 2024-09-13T10:12:03.347 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.347 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.347 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.347 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.347 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.347 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.347 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:12:03.347 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.347 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.348 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.348 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:12:03.348 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.348 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.348 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.348 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.348 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055112+0000", 2024-09-13T10:12:03.348 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6589999999999999e-06 2024-09-13T10:12:03.348 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.348 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.348 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.348 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057327+0000", 2024-09-13T10:12:03.348 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0022145229999999999 2024-09-13T10:12:03.349 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.349 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.349 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.349 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057354+0000", 2024-09-13T10:12:03.349 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7642000000000001e-05 2024-09-13T10:12:03.349 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.349 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.349 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.349 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057373+0000", 2024-09-13T10:12:03.349 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8768999999999999e-05 2024-09-13T10:12:03.349 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.349 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.349 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.349 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057511+0000", 2024-09-13T10:12:03.350 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013793900000000001 2024-09-13T10:12:03.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.350 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.350 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057550+0000", 2024-09-13T10:12:03.350 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9301999999999997e-05 2024-09-13T10:12:03.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.350 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.350 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.064324+0000", 2024-09-13T10:12:03.350 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0067737429999999996 2024-09-13T10:12:03.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.350 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.350 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.064398+0000", 2024-09-13T10:12:03.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3878e-05 2024-09-13T10:12:03.351 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.351 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.351 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.351 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:227 3.2 e38/30 3:6ac7800c:::smithi19530714-28 low_tier:head v 38'8, mlcod=38'6)", 2024-09-13T10:12:03.351 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:12:03.351 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.240948289000002, 2024-09-13T10:12:03.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018812816, 2024-09-13T10:12:03.351 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.351 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.351 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.352 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.352 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.352 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.352 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:12:03.352 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.352 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.352 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.352 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:12:03.352 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.352 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.353 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.353 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074708+0000", 2024-09-13T10:12:03.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.968e-06 2024-09-13T10:12:03.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.353 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.353 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.353 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085262+0000", 2024-09-13T10:12:03.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010553842000000001 2024-09-13T10:12:03.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.353 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.353 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.353 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085289+0000", 2024-09-13T10:12:03.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7288999999999999e-05 2024-09-13T10:12:03.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.354 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.354 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085305+0000", 2024-09-13T10:12:03.354 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5925000000000001e-05 2024-09-13T10:12:03.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.354 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.354 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085418+0000", 2024-09-13T10:12:03.354 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112584 2024-09-13T10:12:03.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.354 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.354 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085451+0000", 2024-09-13T10:12:03.354 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3605000000000001e-05 2024-09-13T10:12:03.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.355 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.355 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.355 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093417+0000", 2024-09-13T10:12:03.355 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0079654249999999999 2024-09-13T10:12:03.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.355 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.355 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.355 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093514+0000", 2024-09-13T10:12:03.355 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7177999999999997e-05 2024-09-13T10:12:03.355 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.355 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.355 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.356 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:230 3.2 e38/30 3:5d108080:::smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'9, mlcod=38'6)", 2024-09-13T10:12:03.356 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:12:03.356 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.226665259000001, 2024-09-13T10:12:03.356 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01953365, 2024-09-13T10:12:03.356 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.356 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.356 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.356 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.356 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.356 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:12:03.356 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.357 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.357 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.357 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:12:03.357 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.357 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.357 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.357 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088987+0000", 2024-09-13T10:12:03.357 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4600000000000002e-06 2024-09-13T10:12:03.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.357 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.357 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.357 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095842+0000", 2024-09-13T10:12:03.358 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0068548760000000002 2024-09-13T10:12:03.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.358 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095872+0000", 2024-09-13T10:12:03.358 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0476000000000001e-05 2024-09-13T10:12:03.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.358 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095890+0000", 2024-09-13T10:12:03.358 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7554000000000002e-05 2024-09-13T10:12:03.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.359 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.096006+0000", 2024-09-13T10:12:03.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011597199999999999 2024-09-13T10:12:03.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.359 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.359 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.096058+0000", 2024-09-13T10:12:03.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2274000000000001e-05 2024-09-13T10:12:03.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.359 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.359 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.108451+0000", 2024-09-13T10:12:03.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012392815 2024-09-13T10:12:03.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.360 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.360 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.108518+0000", 2024-09-13T10:12:03.360 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7223000000000003e-05 2024-09-13T10:12:03.360 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.360 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.360 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.360 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.360 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.360 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:232 3.2 e38/30 3:6538713c:::smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'10, mlcod=38'6)", 2024-09-13T10:12:03.360 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:12:03.360 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.219714378000006, 2024-09-13T10:12:03.360 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015307497, 2024-09-13T10:12:03.360 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.360 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.361 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.361 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.361 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.361 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:12:03.361 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.361 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.361 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:12:03.361 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.361 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.361 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095937+0000", 2024-09-13T10:12:03.362 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.367e-06 2024-09-13T10:12:03.362 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.362 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.362 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.362 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099841+0000", 2024-09-13T10:12:03.362 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0039043960000000001 2024-09-13T10:12:03.362 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.362 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.362 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.362 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099865+0000", 2024-09-13T10:12:03.362 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4538000000000001e-05 2024-09-13T10:12:03.362 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.362 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.362 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.363 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099881+0000", 2024-09-13T10:12:03.363 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5809000000000001e-05 2024-09-13T10:12:03.363 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.363 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.363 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.363 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099995+0000", 2024-09-13T10:12:03.363 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113843 2024-09-13T10:12:03.363 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.363 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.363 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.363 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.100043+0000", 2024-09-13T10:12:03.363 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8291999999999998e-05 2024-09-13T10:12:03.363 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.363 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.364 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.364 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111200+0000", 2024-09-13T10:12:03.364 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011156291 2024-09-13T10:12:03.364 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.364 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.364 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.364 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111243+0000", 2024-09-13T10:12:03.364 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2960999999999997e-05 2024-09-13T10:12:03.364 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.364 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.364 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.364 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.365 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.365 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:236 3.2 e38/30 3:73c2893b:::smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'11, mlcod=38'7)", 2024-09-13T10:12:03.365 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:12:03.365 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.210214260000001, 2024-09-13T10:12:03.365 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0098707670000000008, 2024-09-13T10:12:03.365 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.365 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.365 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.365 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.365 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.365 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.365 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:12:03.365 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.366 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.366 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.366 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:12:03.366 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.366 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.366 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.366 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105438+0000", 2024-09-13T10:12:03.366 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3010000000000001e-06 2024-09-13T10:12:03.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.366 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.366 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.367 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112262+0000", 2024-09-13T10:12:03.367 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.006824793 2024-09-13T10:12:03.367 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.367 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.367 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.367 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112279+0000", 2024-09-13T10:12:03.367 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6829000000000001e-05 2024-09-13T10:12:03.367 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.367 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.367 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.367 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112293+0000", 2024-09-13T10:12:03.367 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3801000000000001e-05 2024-09-13T10:12:03.367 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.368 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.368 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.368 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112381+0000", 2024-09-13T10:12:03.368 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7825999999999996e-05 2024-09-13T10:12:03.368 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.368 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.368 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.368 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112411+0000", 2024-09-13T10:12:03.368 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0434e-05 2024-09-13T10:12:03.368 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.368 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.368 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.368 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.115246+0000", 2024-09-13T10:12:03.368 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028352360000000001 2024-09-13T10:12:03.369 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.369 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.369 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.369 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.115306+0000", 2024-09-13T10:12:03.369 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9546999999999998e-05 2024-09-13T10:12:03.369 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.369 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.369 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.369 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.369 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.369 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:240 3.2 e38/30 3:7eee2f4a:::smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'12, mlcod=38'7)", 2024-09-13T10:12:03.369 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:12:03.369 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.203304473000003, 2024-09-13T10:12:03.370 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011743134000000001, 2024-09-13T10:12:03.370 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.370 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.370 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.370 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.370 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.370 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.370 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:12:03.370 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.370 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.370 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.370 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:12:03.370 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.371 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.371 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.371 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112346+0000", 2024-09-13T10:12:03.371 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.037e-06 2024-09-13T10:12:03.371 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.371 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.371 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.371 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120152+0000", 2024-09-13T10:12:03.371 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0078054700000000001 2024-09-13T10:12:03.371 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.371 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.371 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.371 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120173+0000", 2024-09-13T10:12:03.371 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1498999999999999e-05 2024-09-13T10:12:03.372 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.372 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.372 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.372 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120188+0000", 2024-09-13T10:12:03.372 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5159e-05 2024-09-13T10:12:03.372 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.372 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.372 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.372 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120313+0000", 2024-09-13T10:12:03.372 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012440199999999999 2024-09-13T10:12:03.372 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.372 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.372 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.373 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120358+0000", 2024-09-13T10:12:03.373 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-13T10:12:03.373 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.373 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.373 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.373 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.123945+0000", 2024-09-13T10:12:03.373 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035865490000000001 2024-09-13T10:12:03.373 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.373 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.373 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.373 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.124088+0000", 2024-09-13T10:12:03.373 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014318999999999999 2024-09-13T10:12:03.373 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.373 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.374 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.374 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.374 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.374 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:242 3.2 e38/30 3:501d22a1:::smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'13, mlcod=38'7)", 2024-09-13T10:12:03.374 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:12:03.374 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.195411465000007, 2024-09-13T10:12:03.374 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0088394330000000007, 2024-09-13T10:12:03.374 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.374 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.374 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.374 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.374 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.374 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.374 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:12:03.375 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.375 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.375 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.375 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:12:03.375 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.375 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.375 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.375 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120240+0000", 2024-09-13T10:12:03.375 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.731e-06 2024-09-13T10:12:03.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.375 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.375 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.376 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125395+0000", 2024-09-13T10:12:03.376 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0051547490000000001 2024-09-13T10:12:03.376 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.376 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.376 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.376 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125412+0000", 2024-09-13T10:12:03.376 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7833e-05 2024-09-13T10:12:03.376 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.376 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.376 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.376 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125426+0000", 2024-09-13T10:12:03.376 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3237999999999999e-05 2024-09-13T10:12:03.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.377 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.377 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.377 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125541+0000", 2024-09-13T10:12:03.377 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011566299999999999 2024-09-13T10:12:03.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.377 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.377 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.377 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125578+0000", 2024-09-13T10:12:03.377 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7197000000000003e-05 2024-09-13T10:12:03.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.377 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.377 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.378 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.128885+0000", 2024-09-13T10:12:03.378 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0033067119999999998 2024-09-13T10:12:03.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.378 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.378 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.378 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.129077+0000", 2024-09-13T10:12:03.378 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019231000000000001 2024-09-13T10:12:03.378 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.378 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.378 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.378 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.378 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:244 3.2 e38/30 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'14, mlcod=38'10)", 2024-09-13T10:12:03.379 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:12:03.379 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.189392096999995, 2024-09-13T10:12:03.379 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01536599, 2024-09-13T10:12:03.379 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.379 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.379 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.379 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.379 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.379 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.379 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:12:03.379 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.379 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.379 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.380 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:12:03.380 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.380 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.380 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.380 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126259+0000", 2024-09-13T10:12:03.380 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.528e-06 2024-09-13T10:12:03.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.380 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.380 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.380 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134336+0000", 2024-09-13T10:12:03.380 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080767410000000001 2024-09-13T10:12:03.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.380 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.381 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134356+0000", 2024-09-13T10:12:03.381 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9936999999999999e-05 2024-09-13T10:12:03.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.381 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.381 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134367+0000", 2024-09-13T10:12:03.381 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1352000000000001e-05 2024-09-13T10:12:03.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.381 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.381 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134532+0000", 2024-09-13T10:12:03.381 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001651 2024-09-13T10:12:03.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.382 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.382 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.382 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134578+0000", 2024-09-13T10:12:03.382 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5442000000000003e-05 2024-09-13T10:12:03.382 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.382 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.382 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.382 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.141521+0000", 2024-09-13T10:12:03.382 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069432720000000003 2024-09-13T10:12:03.382 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.382 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.382 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.382 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.141623+0000", 2024-09-13T10:12:03.383 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102618 2024-09-13T10:12:03.383 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.383 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.383 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.383 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.383 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.383 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:250 3.2 e38/30 3:78d40eb9:::smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'15, mlcod=38'10)", 2024-09-13T10:12:03.383 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:12:03.383 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.181195626000004, 2024-09-13T10:12:03.383 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095447210000000008, 2024-09-13T10:12:03.383 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.383 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.383 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.383 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.384 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.384 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.384 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:12:03.384 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.384 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.384 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.384 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:12:03.384 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.384 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.384 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.384 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134455+0000", 2024-09-13T10:12:03.384 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2699999999999999e-07 2024-09-13T10:12:03.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.385 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.385 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139352+0000", 2024-09-13T10:12:03.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048977810000000004 2024-09-13T10:12:03.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.385 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.385 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139374+0000", 2024-09-13T10:12:03.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1339e-05 2024-09-13T10:12:03.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.385 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139384+0000", 2024-09-13T10:12:03.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0183999999999999e-05 2024-09-13T10:12:03.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139477+0000", 2024-09-13T10:12:03.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2590999999999995e-05 2024-09-13T10:12:03.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139515+0000", 2024-09-13T10:12:03.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8624999999999997e-05 2024-09-13T10:12:03.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.387 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.143939+0000", 2024-09-13T10:12:03.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0044241189999999998 2024-09-13T10:12:03.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.387 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.143999+0000", 2024-09-13T10:12:03.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9354999999999997e-05 2024-09-13T10:12:03.387 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.387 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.387 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.388 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:245 3.2 e38/30 3:7837e849:::smithi19530714-46 low_tier:head v 38'16, mlcod=38'10)", 2024-09-13T10:12:03.388 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:12:03.388 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.176239272000004, 2024-09-13T10:12:03.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012687423999999999, 2024-09-13T10:12:03.388 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.388 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.388 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.388 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.388 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.388 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.388 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:12:03.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:12:03.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139411+0000", 2024-09-13T10:12:03.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6300000000000005e-07 2024-09-13T10:12:03.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144635+0000", 2024-09-13T10:12:03.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0052242169999999997 2024-09-13T10:12:03.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144651+0000", 2024-09-13T10:12:03.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6371e-05 2024-09-13T10:12:03.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144662+0000", 2024-09-13T10:12:03.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0003000000000001e-05 2024-09-13T10:12:03.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144741+0000", 2024-09-13T10:12:03.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9614e-05 2024-09-13T10:12:03.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144768+0000", 2024-09-13T10:12:03.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6826999999999999e-05 2024-09-13T10:12:03.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152015+0000", 2024-09-13T10:12:03.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0072470879999999996 2024-09-13T10:12:03.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.392 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.392 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.392 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152098+0000", 2024-09-13T10:12:03.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2640999999999997e-05 2024-09-13T10:12:03.392 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.392 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.392 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.392 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.392 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:2811 3.2 e71/30 3:5a2483ca:::890eeb33aa7613d6935d7cb9ed4860e522183188e5dc23a51d7045443a8d23dd:head v 71'264, mlcod=70'262)", 2024-09-13T10:12:03.392 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:32.583079+0000", 2024-09-13T10:12:03.392 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.732570075000002, 2024-09-13T10:12:03.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0094005459999999992, 2024-09-13T10:12:03.393 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.393 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.393 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.393 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.583079+0000", 2024-09-13T10:12:03.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.583079+0000", 2024-09-13T10:12:03.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.583082+0000", 2024-09-13T10:12:03.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3070000000000001e-06 2024-09-13T10:12:03.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.587892+0000", 2024-09-13T10:12:03.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048105889999999997 2024-09-13T10:12:03.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.587924+0000", 2024-09-13T10:12:03.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1217999999999998e-05 2024-09-13T10:12:03.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.587938+0000", 2024-09-13T10:12:03.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4341999999999999e-05 2024-09-13T10:12:03.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.588076+0000", 2024-09-13T10:12:03.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013836600000000001 2024-09-13T10:12:03.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.588116+0000", 2024-09-13T10:12:03.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9480999999999997e-05 2024-09-13T10:12:03.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.592358+0000", 2024-09-13T10:12:03.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042423740000000001 2024-09-13T10:12:03.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.592480+0000", 2024-09-13T10:12:03.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012186899999999999 2024-09-13T10:12:03.396 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.396 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.396 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.397 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.397 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(osd.6.0:365 4.5 e82/41 4:a3098556:::smithi19530714-22:head v 82'129, mlcod=79'127)", 2024-09-13T10:12:03.397 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:44.595715+0000", 2024-09-13T10:12:03.397 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.719934431999999, 2024-09-13T10:12:03.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010604906000000001, 2024-09-13T10:12:03.397 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.397 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.397 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.397 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.397 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.397 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.397 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.595715+0000", 2024-09-13T10:12:03.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.595715+0000", 2024-09-13T10:12:03.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.595716+0000", 2024-09-13T10:12:03.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8400000000000004e-07 2024-09-13T10:12:03.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.601483+0000", 2024-09-13T10:12:03.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0057671789999999999 2024-09-13T10:12:03.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.601511+0000", 2024-09-13T10:12:03.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8218999999999999e-05 2024-09-13T10:12:03.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.601526+0000", 2024-09-13T10:12:03.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4956e-05 2024-09-13T10:12:03.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.601667+0000", 2024-09-13T10:12:03.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000140774 2024-09-13T10:12:03.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.601709+0000", 2024-09-13T10:12:03.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2688999999999997e-05 2024-09-13T10:12:03.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.606248+0000", 2024-09-13T10:12:03.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004538065 2024-09-13T10:12:03.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.401 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.401 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.606320+0000", 2024-09-13T10:12:03.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2440000000000004e-05 2024-09-13T10:12:03.401 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.401 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.401 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.401 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:3445 3.2 e86/30 3:7ba83ace:::4c246a1df6a0d5acaf0a3a85249ff707b21f90a5f3db763dc6d544e5b7cbec73:head v 86'330, mlcod=85'328)", 2024-09-13T10:12:03.401 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:48.129770+0000", 2024-09-13T10:12:03.401 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.185879736, 2024-09-13T10:12:03.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010414537999999999, 2024-09-13T10:12:03.402 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.402 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.402 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.402 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.129770+0000", 2024-09-13T10:12:03.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.402 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.129770+0000", 2024-09-13T10:12:03.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.129771+0000", 2024-09-13T10:12:03.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.59e-06 2024-09-13T10:12:03.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.136002+0000", 2024-09-13T10:12:03.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0062306380000000001 2024-09-13T10:12:03.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.136024+0000", 2024-09-13T10:12:03.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2106000000000001e-05 2024-09-13T10:12:03.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.136035+0000", 2024-09-13T10:12:03.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1293e-05 2024-09-13T10:12:03.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.136115+0000", 2024-09-13T10:12:03.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9294000000000003e-05 2024-09-13T10:12:03.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.136136+0000", 2024-09-13T10:12:03.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1495999999999999e-05 2024-09-13T10:12:03.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.140076+0000", 2024-09-13T10:12:03.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0039398849999999997 2024-09-13T10:12:03.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.140184+0000", 2024-09-13T10:12:03.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108236 2024-09-13T10:12:03.405 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.405 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.406 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.406 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:3917 3.2 e94/30 3:55300012:::a6332e51e016c24df8505825880ac4e703da29058ee795e9033d5e12bb724144:head v 94'352, mlcod=93'350)", 2024-09-13T10:12:03.406 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:58.912836+0000", 2024-09-13T10:12:03.406 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.4028134740000002, 2024-09-13T10:12:03.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0083088199999999997, 2024-09-13T10:12:03.406 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.406 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.406 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.406 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.912836+0000", 2024-09-13T10:12:03.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.407 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.912836+0000", 2024-09-13T10:12:03.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.407 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.407 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.912836+0000", 2024-09-13T10:12:03.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7499999999999999e-07 2024-09-13T10:12:03.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.917813+0000", 2024-09-13T10:12:03.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049764029999999999 2024-09-13T10:12:03.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.917847+0000", 2024-09-13T10:12:03.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4144999999999999e-05 2024-09-13T10:12:03.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.917869+0000", 2024-09-13T10:12:03.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.196e-05 2024-09-13T10:12:03.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.917959+0000", 2024-09-13T10:12:03.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0421999999999996e-05 2024-09-13T10:12:03.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.917978+0000", 2024-09-13T10:12:03.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8368000000000001e-05 2024-09-13T10:12:03.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.921076+0000", 2024-09-13T10:12:03.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030987689999999999 2024-09-13T10:12:03.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.921145+0000", 2024-09-13T10:12:03.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8578000000000006e-05 2024-09-13T10:12:03.410 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.410 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.410 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.410 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.410 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.410 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:03.412 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:12:03.412 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:12:03.414 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T10:12:03.414 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:12:03.414 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:12:03.415 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:12:03.415 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:12:03.416 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:12:03.416 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:12:03.417 INFO:tasks.rados.rados.0.smithi195.stdout:1711: writing smithi19530714-31 from 551362 to 1277440 tid 1 2024-09-13T10:12:03.417 INFO:tasks.rados.rados.0.smithi195.stdout:1711: writing smithi19530714-31 from 1717586 to 2127365 tid 2 2024-09-13T10:12:03.417 INFO:tasks.rados.rados.0.smithi195.stdout:1712: read oid 20 snap -1 2024-09-13T10:12:03.417 INFO:tasks.rados.rados.0.smithi195.stdout:1712: expect (ObjNum 315 snap 0 seq_num 315) 2024-09-13T10:12:03.417 INFO:tasks.rados.rados.0.smithi195.stdout:1713: tier_evict oid 17 2024-09-13T10:12:03.417 INFO:tasks.rados.rados.0.smithi195.stdout:1713: tier_evict oid 17 snap -1 2024-09-13T10:12:03.417 INFO:tasks.rados.rados.0.smithi195.stdout:1711: finishing write tid 1 to smithi19530714-31 2024-09-13T10:12:03.417 INFO:tasks.rados.rados.0.smithi195.stdout:1710: done (3 left) 2024-09-13T10:12:03.417 INFO:tasks.rados.rados.0.smithi195.stdout:1714: read oid 15 snap -1 2024-09-13T10:12:03.417 INFO:tasks.rados.rados.0.smithi195.stdout:1714: expect (ObjNum 361 snap 0 seq_num 361) 2024-09-13T10:12:03.417 INFO:tasks.rados.rados.0.smithi195.stdout:1715: write oid 10 current snap is 0 2024-09-13T10:12:03.417 INFO:tasks.rados.rados.0.smithi195.stdout:1715: seq_num 367 ranges {568169=401327,1376068=669790} 2024-09-13T10:12:03.418 INFO:tasks.rados.rados.0.smithi195.stdout:1715: writing smithi19530714-10 from 568169 to 969496 tid 1 2024-09-13T10:12:03.418 INFO:tasks.rados.rados.0.smithi195.stdout:1715: writing smithi19530714-10 from 1376068 to 2045858 tid 2 2024-09-13T10:12:03.418 INFO:tasks.rados.rados.0.smithi195.stdout:1716: write (excl) oid 37 current snap is 0 2024-09-13T10:12:03.418 INFO:tasks.rados.rados.0.smithi195.stdout:1716: seq_num 368 ranges {584976=476576,1834550=350426} 2024-09-13T10:12:03.418 INFO:tasks.rados.rados.0.smithi195.stdout:1716: writing smithi19530714-37 from 584976 to 1061552 tid 1 2024-09-13T10:12:03.418 INFO:tasks.rados.rados.0.smithi195.stdout:1716: writing smithi19530714-37 from 1834550 to 2184976 tid 2 2024-09-13T10:12:03.418 INFO:tasks.rados.rados.0.smithi195.stdout:1717: set_chunk oid 46 target oid 2024-09-13T10:12:03.418 INFO:tasks.rados.rados.0.smithi195.stdout:1717: set_chunk oid 46 offset: 2174976 length: 522752 target oid 20aac0d1b3bca85337656595b412e1edb15311d159ab7ec4d0044fad374c1d93 offset: 0 snap -1 2024-09-13T10:12:03.418 INFO:tasks.rados.rados.0.smithi195.stdout:1718: read oid 13 snap -1 2024-09-13T10:12:03.418 INFO:tasks.rados.rados.0.smithi195.stdout:1718: expect (ObjNum 342 snap 0 seq_num 342) 2024-09-13T10:12:03.418 INFO:tasks.rados.rados.0.smithi195.stdout:1719: write (excl) oid 28 current snap is 0 2024-09-13T10:12:03.418 INFO:tasks.rados.rados.0.smithi195.stdout:1719: seq_num 369 ranges {601783=551825,1776679=425104} 2024-09-13T10:12:03.418 INFO:tasks.rados.rados.0.smithi195.stdout:1719: writing smithi19530714-28 from 601783 to 1153608 tid 1 2024-09-13T10:12:03.419 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:12:03.419 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:12:03.419 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:03.407+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:03.419 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:12:03.420 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:12:03.420 INFO:tasks.rados.rados.0.smithi195.stdout:1719: writing smithi19530714-28 from 1776679 to 2201783 tid 2 2024-09-13T10:12:03.420 INFO:tasks.rados.rados.0.smithi195.stdout:1720: write (excl) oid 24 current snap is 0 2024-09-13T10:12:03.421 INFO:tasks.rados.rados.0.smithi195.stdout:1713: got (22) Invalid argument 2024-09-13T10:12:03.421 INFO:tasks.rados.rados.0.smithi195.stdout:1711: finishing write tid 2 to smithi19530714-31 2024-09-13T10:12:03.421 INFO:tasks.rados.rados.0.smithi195.stdout:1711: finishing write tid 3 to smithi19530714-31 2024-09-13T10:12:03.421 INFO:tasks.rados.rados.0.smithi195.stdout:1720: seq_num 370 ranges {618590=627074,1835161=383429} 2024-09-13T10:12:03.429 INFO:tasks.rados.rados.0.smithi195.stdout:1720: writing smithi19530714-24 from 618590 to 1245664 tid 1 2024-09-13T10:12:03.438 INFO:tasks.rados.rados.0.smithi195.stdout:1720: writing smithi19530714-24 from 1835161 to 2218590 tid 2 2024-09-13T10:12:03.438 INFO:tasks.rados.rados.0.smithi195.stdout:1713: done (9 left) 2024-09-13T10:12:03.438 INFO:tasks.rados.rados.0.smithi195.stdout:1721: delete oid 36 current snap is 0 2024-09-13T10:12:03.438 INFO:tasks.rados.rados.0.smithi195.stdout:1711: finishing write tid 4 to smithi19530714-31 2024-09-13T10:12:03.439 INFO:tasks.rados.rados.0.smithi195.stdout:1711: oid 31 updating version 0 to 104 2024-09-13T10:12:03.439 INFO:tasks.rados.rados.0.smithi195.stdout:1711: oid 31 version 104 is already newer than 103 2024-09-13T10:12:03.439 INFO:tasks.rados.rados.0.smithi195.stdout:1711: oid 31 version 104 is already newer than 102 2024-09-13T10:12:03.439 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 104 (ObjNum 366 snap 0 seq_num 366) dirty exists 2024-09-13T10:12:03.439 INFO:tasks.rados.rados.0.smithi195.stdout:1711: left oid 31 (ObjNum 366 snap 0 seq_num 366) 2024-09-13T10:12:03.439 INFO:tasks.rados.rados.0.smithi195.stdout:1712: expect (ObjNum 315 snap 0 seq_num 315) 2024-09-13T10:12:03.440 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:12:03.440 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:12:03.473 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:12:03.473 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:12:03.494 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:12:03.494 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:12:03.509 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:03.512 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:12:03.512 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:12:03.512 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:03.507+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:03.532 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:12:03.533 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:12:03.540 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:03.540 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:12:03.540 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:12:03.540 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:03.553 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T10:12:03.554 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:12:03.554 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:12:03.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:12:03.551+0000 7f5b07fff640 0 -- 172.21.15.73:0/4291746067 >> v2:172.21.15.195:6805/848686366 conn(0x7f5b08228040 msgr2=0x7f5b0822a430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-13T10:12:03.579 INFO:tasks.rados.rados.0.smithi195.stdout:1714: expect (ObjNum 361 snap 0 seq_num 361) 2024-09-13T10:12:03.579 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:03.614 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:12:03.610+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:12:03.646 INFO:tasks.rados.rados.0.smithi195.stdout:1715: finishing write tid 1 to smithi19530714-10 2024-09-13T10:12:03.646 INFO:tasks.rados.rados.0.smithi195.stdout:1715: finishing write tid 2 to smithi19530714-10 2024-09-13T10:12:03.646 INFO:tasks.rados.rados.0.smithi195.stdout:1715: finishing write tid 3 to smithi19530714-10 2024-09-13T10:12:03.646 INFO:tasks.rados.rados.0.smithi195.stdout:1715: finishing write tid 4 to smithi19530714-10 2024-09-13T10:12:03.646 INFO:tasks.rados.rados.0.smithi195.stdout:1715: oid 10 updating version 0 to 33 2024-09-13T10:12:03.646 INFO:tasks.rados.rados.0.smithi195.stdout:1715: oid 10 version 33 is already newer than 32 2024-09-13T10:12:03.647 INFO:tasks.rados.rados.0.smithi195.stdout:1715: oid 10 updating version 33 to 34 2024-09-13T10:12:03.647 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 34 (ObjNum 367 snap 0 seq_num 367) dirty exists 2024-09-13T10:12:03.647 INFO:tasks.rados.rados.0.smithi195.stdout:1715: left oid 10 (ObjNum 367 snap 0 seq_num 367) 2024-09-13T10:12:03.647 INFO:tasks.rados.rados.0.smithi195.stdout:1716: finishing write tid 1 to smithi19530714-37 2024-09-13T10:12:03.647 INFO:tasks.rados.rados.0.smithi195.stdout:1716: finishing write tid 2 to smithi19530714-37 2024-09-13T10:12:03.647 INFO:tasks.rados.rados.0.smithi195.stdout:1716: finishing write tid 3 to smithi19530714-37 2024-09-13T10:12:03.647 INFO:tasks.rados.rados.0.smithi195.stdout:1716: finishing write tid 4 to smithi19530714-37 2024-09-13T10:12:03.647 INFO:tasks.rados.rados.0.smithi195.stdout:1716: oid 37 updating version 0 to 23 2024-09-13T10:12:03.647 INFO:tasks.rados.rados.0.smithi195.stdout:1716: oid 37 updating version 23 to 25 2024-09-13T10:12:03.647 INFO:tasks.rados.rados.0.smithi195.stdout:1716: oid 37 version 25 is already newer than 24 2024-09-13T10:12:03.647 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 25 (ObjNum 368 snap 0 seq_num 368) dirty exists 2024-09-13T10:12:03.648 INFO:tasks.rados.rados.0.smithi195.stdout:1716: left oid 37 (ObjNum 368 snap 0 seq_num 368) 2024-09-13T10:12:03.648 INFO:tasks.rados.rados.0.smithi195.stdout:1717: finishing set_chunk to oid 46 2024-09-13T10:12:03.648 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 46 set_chunk returned error code -95 offset: 2174976 length: 522752 2024-09-13T10:12:03.648 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 0 (ObjNum 329 snap 0 seq_num 329) dirty exists 2024-09-13T10:12:03.648 INFO:tasks.rados.rados.0.smithi195.stdout:1718: expect (ObjNum 342 snap 0 seq_num 342) 2024-09-13T10:12:03.648 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:03.675 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:03.675 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:12:03.675 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:12:03.675 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:12:03.675 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:03.690 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T10:12:03.734 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:12:03.731+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:12:03.739 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:03.767 INFO:teuthology.orchestra.run.smithi073.stdout:moved 5 / 117 (4.2735%) 2024-09-13T10:12:03.767 INFO:teuthology.orchestra.run.smithi073.stdout:avg 7.3125 2024-09-13T10:12:03.768 INFO:teuthology.orchestra.run.smithi073.stdout:stddev 2.36428 -> 2.41658 (expected baseline 2.61829) 2024-09-13T10:12:03.768 INFO:teuthology.orchestra.run.smithi073.stdout:min osd.11 with 4 -> 4 pgs (0.547009 -> 0.547009 * mean) 2024-09-13T10:12:03.768 INFO:teuthology.orchestra.run.smithi073.stdout:max osd.3 with 12 -> 13 pgs (1.64103 -> 1.77778 * mean) 2024-09-13T10:12:03.768 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:12:03.768 INFO:teuthology.orchestra.run.smithi073.stdout:oload 110 2024-09-13T10:12:03.768 INFO:teuthology.orchestra.run.smithi073.stdout:max_change 0.05 2024-09-13T10:12:03.768 INFO:teuthology.orchestra.run.smithi073.stdout:max_change_osds 4 2024-09-13T10:12:03.768 INFO:teuthology.orchestra.run.smithi073.stdout:average_utilization 0.0007 2024-09-13T10:12:03.768 INFO:teuthology.orchestra.run.smithi073.stdout:overload_utilization 0.0008 2024-09-13T10:12:03.768 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4 weight 1.0000 -> 0.9500 2024-09-13T10:12:03.768 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3 weight 1.0000 -> 0.9500 2024-09-13T10:12:03.769 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14 weight 1.0000 -> 0.9500 2024-09-13T10:12:03.769 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13 weight 1.0000 -> 0.9500 2024-09-13T10:12:03.784 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:12:03.784 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:12:03.818 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:03.818 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:12:03.818 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:12:03.818 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:12:03.818 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.819 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:203 3.1 e38/30 3:b033e801:::smithi19530714-4 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:12:03.819 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.002504+0000", 2024-09-13T10:12:03.819 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.813861572999997, 2024-09-13T10:12:03.819 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016371101999999998, 2024-09-13T10:12:03.819 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.819 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.819 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.819 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.819 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.819 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.819 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002504+0000", 2024-09-13T10:12:03.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.820 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.820 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.820 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002504+0000", 2024-09-13T10:12:03.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.820 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.820 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.820 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002507+0000", 2024-09-13T10:12:03.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2699999999999999e-06 2024-09-13T10:12:03.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.821 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.821 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007079+0000", 2024-09-13T10:12:03.821 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045722200000000001 2024-09-13T10:12:03.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.821 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.821 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007114+0000", 2024-09-13T10:12:03.821 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5389000000000003e-05 2024-09-13T10:12:03.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.822 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.822 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007132+0000", 2024-09-13T10:12:03.822 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7489000000000001e-05 2024-09-13T10:12:03.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.822 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.822 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007266+0000", 2024-09-13T10:12:03.822 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013401900000000001 2024-09-13T10:12:03.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.822 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.822 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007299+0000", 2024-09-13T10:12:03.823 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3136000000000003e-05 2024-09-13T10:12:03.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.823 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.018741+0000", 2024-09-13T10:12:03.823 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011441957000000001 2024-09-13T10:12:03.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.823 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.018875+0000", 2024-09-13T10:12:03.823 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013462200000000001 2024-09-13T10:12:03.823 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.824 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.824 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.824 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.824 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:211 3.0 e38/30 3:0596651d:::smithi19530714-12 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:12:03.824 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.007453+0000", 2024-09-13T10:12:03.824 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.808912749000001, 2024-09-13T10:12:03.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018632359000000001, 2024-09-13T10:12:03.824 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.824 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.825 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.825 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.825 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.825 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007453+0000", 2024-09-13T10:12:03.825 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.825 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.825 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007453+0000", 2024-09-13T10:12:03.825 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.826 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.826 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007455+0000", 2024-09-13T10:12:03.826 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4030000000000001e-06 2024-09-13T10:12:03.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.826 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.826 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020746+0000", 2024-09-13T10:12:03.826 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013290219000000001 2024-09-13T10:12:03.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.827 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.827 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020788+0000", 2024-09-13T10:12:03.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2342000000000002e-05 2024-09-13T10:12:03.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.827 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.827 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020868+0000", 2024-09-13T10:12:03.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9519999999999998e-05 2024-09-13T10:12:03.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.828 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.828 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021080+0000", 2024-09-13T10:12:03.828 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021249600000000001 2024-09-13T10:12:03.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.828 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.828 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021113+0000", 2024-09-13T10:12:03.828 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2786000000000001e-05 2024-09-13T10:12:03.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.829 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.829 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.025974+0000", 2024-09-13T10:12:03.829 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048614929999999997 2024-09-13T10:12:03.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.829 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.829 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026085+0000", 2024-09-13T10:12:03.829 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001111 2024-09-13T10:12:03.829 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.829 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.829 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.830 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:205 3.1 e38/30 3:aec11a92:::smithi19530714-6 low_tier:head v 38'2, mlcod=0'0)", 2024-09-13T10:12:03.830 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.014358+0000", 2024-09-13T10:12:03.830 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.802008125, 2024-09-13T10:12:03.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033491923999999999, 2024-09-13T10:12:03.830 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.830 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.830 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.830 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.830 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.831 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.014358+0000", 2024-09-13T10:12:03.831 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.831 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.831 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.014358+0000", 2024-09-13T10:12:03.831 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.831 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.014360+0000", 2024-09-13T10:12:03.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.057e-06 2024-09-13T10:12:03.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026358+0000", 2024-09-13T10:12:03.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011998688 2024-09-13T10:12:03.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.833 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.833 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026392+0000", 2024-09-13T10:12:03.833 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3575999999999998e-05 2024-09-13T10:12:03.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.833 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.833 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.833 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027340+0000", 2024-09-13T10:12:03.833 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00094826100000000005 2024-09-13T10:12:03.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.833 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.833 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.833 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027449+0000", 2024-09-13T10:12:03.834 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108797 2024-09-13T10:12:03.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.834 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.834 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.834 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027500+0000", 2024-09-13T10:12:03.834 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0757999999999998e-05 2024-09-13T10:12:03.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.834 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.834 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.834 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047748+0000", 2024-09-13T10:12:03.834 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020247884000000001 2024-09-13T10:12:03.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.835 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.835 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047850+0000", 2024-09-13T10:12:03.835 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101903 2024-09-13T10:12:03.835 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.835 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.835 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.835 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:212 3.0 e38/30 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:12:03.836 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.021018+0000", 2024-09-13T10:12:03.836 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.795347316999994, 2024-09-13T10:12:03.836 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044503290000000001, 2024-09-13T10:12:03.836 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.836 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.836 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.836 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.836 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.836 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021018+0000", 2024-09-13T10:12:03.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.837 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021018+0000", 2024-09-13T10:12:03.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.837 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.838 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021019+0000", 2024-09-13T10:12:03.838 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7400000000000002e-07 2024-09-13T10:12:03.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.838 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.838 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038659+0000", 2024-09-13T10:12:03.838 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017639555000000001 2024-09-13T10:12:03.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.838 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.838 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038678+0000", 2024-09-13T10:12:03.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8912000000000001e-05 2024-09-13T10:12:03.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.839 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.839 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.839 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038693+0000", 2024-09-13T10:12:03.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5393e-05 2024-09-13T10:12:03.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.839 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.839 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.839 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038829+0000", 2024-09-13T10:12:03.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000135912 2024-09-13T10:12:03.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.840 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.840 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038887+0000", 2024-09-13T10:12:03.840 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7858000000000001e-05 2024-09-13T10:12:03.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.840 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.840 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065395+0000", 2024-09-13T10:12:03.840 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02650773 2024-09-13T10:12:03.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.841 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.841 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.841 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065522+0000", 2024-09-13T10:12:03.841 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012705600000000001 2024-09-13T10:12:03.841 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.841 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.841 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.841 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.841 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:213 3.1 e38/30 3:accf4e56:::smithi19530714-14 low_tier:head v 38'3, mlcod=0'0)", 2024-09-13T10:12:03.842 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.027392+0000", 2024-09-13T10:12:03.842 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.788973807999994, 2024-09-13T10:12:03.842 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025423449000000001, 2024-09-13T10:12:03.842 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.842 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.842 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.842 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.842 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.842 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.842 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027392+0000", 2024-09-13T10:12:03.843 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.843 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.843 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027392+0000", 2024-09-13T10:12:03.843 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.843 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.843 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027394+0000", 2024-09-13T10:12:03.843 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7370000000000001e-06 2024-09-13T10:12:03.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032354+0000", 2024-09-13T10:12:03.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049601589999999996 2024-09-13T10:12:03.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032413+0000", 2024-09-13T10:12:03.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9095000000000003e-05 2024-09-13T10:12:03.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.845 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.845 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032426+0000", 2024-09-13T10:12:03.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3465999999999999e-05 2024-09-13T10:12:03.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.845 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.845 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032805+0000", 2024-09-13T10:12:03.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000378704 2024-09-13T10:12:03.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.846 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.846 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.846 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032862+0000", 2024-09-13T10:12:03.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6878e-05 2024-09-13T10:12:03.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.846 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.846 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.846 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.052688+0000", 2024-09-13T10:12:03.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019826308000000001 2024-09-13T10:12:03.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.846 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.846 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.847 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.052815+0000", 2024-09-13T10:12:03.847 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127102 2024-09-13T10:12:03.847 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.847 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.847 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.847 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:209 3.1 e38/30 3:9798d58d:::smithi19530714-10 low_tier:head v 38'4, mlcod=0'0)", 2024-09-13T10:12:03.847 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.032594+0000", 2024-09-13T10:12:03.847 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.783771372999993, 2024-09-13T10:12:03.847 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028120108000000001, 2024-09-13T10:12:03.847 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.848 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.848 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.848 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.848 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.848 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.848 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032594+0000", 2024-09-13T10:12:03.848 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.848 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.848 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.848 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032594+0000", 2024-09-13T10:12:03.848 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.849 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.849 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032596+0000", 2024-09-13T10:12:03.849 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2330000000000001e-06 2024-09-13T10:12:03.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.849 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.849 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.035217+0000", 2024-09-13T10:12:03.849 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026212679999999999 2024-09-13T10:12:03.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.849 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.850 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.035250+0000", 2024-09-13T10:12:03.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3065000000000003e-05 2024-09-13T10:12:03.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.850 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.850 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.035272+0000", 2024-09-13T10:12:03.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2163000000000001e-05 2024-09-13T10:12:03.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.850 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.850 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038562+0000", 2024-09-13T10:12:03.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003290322 2024-09-13T10:12:03.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.851 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038642+0000", 2024-09-13T10:12:03.851 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9457999999999997e-05 2024-09-13T10:12:03.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.851 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060605+0000", 2024-09-13T10:12:03.851 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021962751999999999 2024-09-13T10:12:03.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.852 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060715+0000", 2024-09-13T10:12:03.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010984699999999999 2024-09-13T10:12:03.852 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.852 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.852 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.852 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.852 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:207 3.0 e38/30 3:332cd44b:::smithi19530714-8 low_tier:head v 38'3, mlcod=0'0)", 2024-09-13T10:12:03.852 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.038735+0000", 2024-09-13T10:12:03.852 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.777630794000004, 2024-09-13T10:12:03.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.026872200999999998, 2024-09-13T10:12:03.852 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.853 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.853 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.853 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038735+0000", 2024-09-13T10:12:03.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038735+0000", 2024-09-13T10:12:03.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.854 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.854 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038737+0000", 2024-09-13T10:12:03.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5430000000000001e-06 2024-09-13T10:12:03.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.854 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.854 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046860+0000", 2024-09-13T10:12:03.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081236629999999997 2024-09-13T10:12:03.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.855 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.855 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046895+0000", 2024-09-13T10:12:03.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4989e-05 2024-09-13T10:12:03.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.855 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.855 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046915+0000", 2024-09-13T10:12:03.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0026000000000001e-05 2024-09-13T10:12:03.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.856 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.856 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047057+0000", 2024-09-13T10:12:03.856 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014151800000000001 2024-09-13T10:12:03.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.856 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.856 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047111+0000", 2024-09-13T10:12:03.856 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3885000000000003e-05 2024-09-13T10:12:03.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065539+0000", 2024-09-13T10:12:03.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018428225999999999 2024-09-13T10:12:03.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065607+0000", 2024-09-13T10:12:03.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8350999999999997e-05 2024-09-13T10:12:03.857 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.857 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.858 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.858 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:216 3.0 e38/30 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:12:03.858 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.046965+0000", 2024-09-13T10:12:03.858 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.769400331, 2024-09-13T10:12:03.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.022018617000000001, 2024-09-13T10:12:03.858 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.858 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.858 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.858 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.859 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046965+0000", 2024-09-13T10:12:03.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.859 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046965+0000", 2024-09-13T10:12:03.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046967+0000", 2024-09-13T10:12:03.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3689999999999999e-06 2024-09-13T10:12:03.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054950+0000", 2024-09-13T10:12:03.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0079834910000000005 2024-09-13T10:12:03.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054967+0000", 2024-09-13T10:12:03.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6382e-05 2024-09-13T10:12:03.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054981+0000", 2024-09-13T10:12:03.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3974999999999999e-05 2024-09-13T10:12:03.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.862 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.862 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055057+0000", 2024-09-13T10:12:03.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5964000000000007e-05 2024-09-13T10:12:03.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.862 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.862 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055100+0000", 2024-09-13T10:12:03.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3522000000000001e-05 2024-09-13T10:12:03.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.862 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.068892+0000", 2024-09-13T10:12:03.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01379175 2024-09-13T10:12:03.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.068984+0000", 2024-09-13T10:12:03.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2163999999999994e-05 2024-09-13T10:12:03.863 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.863 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.863 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.863 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:219 3.1 e38/30 3:87bfc7b1:::smithi19530714-20 low_tier:head v 38'5, mlcod=0'0)", 2024-09-13T10:12:03.864 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.048853+0000", 2024-09-13T10:12:03.864 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.767512858999993, 2024-09-13T10:12:03.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017128925999999999, 2024-09-13T10:12:03.864 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.864 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.864 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.864 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.048853+0000", 2024-09-13T10:12:03.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.865 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.048853+0000", 2024-09-13T10:12:03.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.865 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.048855+0000", 2024-09-13T10:12:03.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.877e-06 2024-09-13T10:12:03.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.865 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053357+0000", 2024-09-13T10:12:03.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045020149999999998 2024-09-13T10:12:03.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053378+0000", 2024-09-13T10:12:03.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1639000000000001e-05 2024-09-13T10:12:03.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053396+0000", 2024-09-13T10:12:03.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7135000000000001e-05 2024-09-13T10:12:03.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053526+0000", 2024-09-13T10:12:03.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013083999999999999 2024-09-13T10:12:03.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053590+0000", 2024-09-13T10:12:03.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3355000000000006e-05 2024-09-13T10:12:03.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065909+0000", 2024-09-13T10:12:03.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012319583 2024-09-13T10:12:03.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065982+0000", 2024-09-13T10:12:03.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2482000000000002e-05 2024-09-13T10:12:03.868 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.868 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.868 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.868 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:221 3.0 e38/30 3:18ae852b:::smithi19530714-22 low_tier:head v 38'5, mlcod=0'0)", 2024-09-13T10:12:03.869 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.058895+0000", 2024-09-13T10:12:03.869 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.757470643999994, 2024-09-13T10:12:03.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019544733000000002, 2024-09-13T10:12:03.869 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.869 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.869 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.869 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.869 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.058895+0000", 2024-09-13T10:12:03.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.058895+0000", 2024-09-13T10:12:03.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.058897+0000", 2024-09-13T10:12:03.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0420000000000001e-06 2024-09-13T10:12:03.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.061589+0000", 2024-09-13T10:12:03.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026919869999999999 2024-09-13T10:12:03.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.061614+0000", 2024-09-13T10:12:03.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4984999999999999e-05 2024-09-13T10:12:03.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.061629+0000", 2024-09-13T10:12:03.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4501999999999999e-05 2024-09-13T10:12:03.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065668+0000", 2024-09-13T10:12:03.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040393260000000002 2024-09-13T10:12:03.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065711+0000", 2024-09-13T10:12:03.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2673999999999998e-05 2024-09-13T10:12:03.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.078336+0000", 2024-09-13T10:12:03.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012625113 2024-09-13T10:12:03.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.078440+0000", 2024-09-13T10:12:03.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000104104 2024-09-13T10:12:03.874 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.874 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.874 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.874 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:231 3.1 e38/30 3:9d247594:::smithi19530714-32 low_tier:head v 38'7, mlcod=38'4)", 2024-09-13T10:12:03.874 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.076614+0000", 2024-09-13T10:12:03.875 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.739751681000001, 2024-09-13T10:12:03.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017222986999999999, 2024-09-13T10:12:03.875 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.875 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.875 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.875 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.875 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.875 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.875 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.076614+0000", 2024-09-13T10:12:03.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.076614+0000", 2024-09-13T10:12:03.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.076616+0000", 2024-09-13T10:12:03.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5069999999999999e-06 2024-09-13T10:12:03.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082649+0000", 2024-09-13T10:12:03.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0060329499999999996 2024-09-13T10:12:03.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082674+0000", 2024-09-13T10:12:03.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5624000000000001e-05 2024-09-13T10:12:03.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082690+0000", 2024-09-13T10:12:03.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6152999999999999e-05 2024-09-13T10:12:03.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082812+0000", 2024-09-13T10:12:03.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012209499999999999 2024-09-13T10:12:03.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082863+0000", 2024-09-13T10:12:03.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0781000000000001e-05 2024-09-13T10:12:03.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093763+0000", 2024-09-13T10:12:03.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010899382000000001 2024-09-13T10:12:03.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.880 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.880 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093837+0000", 2024-09-13T10:12:03.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4494999999999997e-05 2024-09-13T10:12:03.880 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.880 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.880 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.881 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:234 3.1 e38/30 3:bdce9caa:::smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'8, mlcod=38'4)", 2024-09-13T10:12:03.881 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.086591+0000", 2024-09-13T10:12:03.881 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.729774567999996, 2024-09-13T10:12:03.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025699538000000001, 2024-09-13T10:12:03.881 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.881 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.881 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.881 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.086591+0000", 2024-09-13T10:12:03.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.086591+0000", 2024-09-13T10:12:03.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.086623+0000", 2024-09-13T10:12:03.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1335999999999999e-05 2024-09-13T10:12:03.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094695+0000", 2024-09-13T10:12:03.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080726919999999994 2024-09-13T10:12:03.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094716+0000", 2024-09-13T10:12:03.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1015000000000001e-05 2024-09-13T10:12:03.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094732+0000", 2024-09-13T10:12:03.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5532999999999998e-05 2024-09-13T10:12:03.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094807+0000", 2024-09-13T10:12:03.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4837000000000002e-05 2024-09-13T10:12:03.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.885 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094836+0000", 2024-09-13T10:12:03.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8912999999999999e-05 2024-09-13T10:12:03.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.885 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112239+0000", 2024-09-13T10:12:03.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017402995000000001 2024-09-13T10:12:03.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112291+0000", 2024-09-13T10:12:03.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2216999999999998e-05 2024-09-13T10:12:03.886 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.886 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.887 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.887 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:233 3.0 e38/30 3:24ead787:::smithi19530714-34 low_tier:head v 38'9, mlcod=38'4)", 2024-09-13T10:12:03.887 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.090071+0000", 2024-09-13T10:12:03.887 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.726294562000007, 2024-09-13T10:12:03.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.023396641999999999, 2024-09-13T10:12:03.887 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.887 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.887 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.887 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090071+0000", 2024-09-13T10:12:03.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090071+0000", 2024-09-13T10:12:03.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090073+0000", 2024-09-13T10:12:03.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0470000000000001e-06 2024-09-13T10:12:03.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097736+0000", 2024-09-13T10:12:03.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007662416 2024-09-13T10:12:03.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097761+0000", 2024-09-13T10:12:03.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4919e-05 2024-09-13T10:12:03.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.890 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097773+0000", 2024-09-13T10:12:03.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2182e-05 2024-09-13T10:12:03.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.890 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097877+0000", 2024-09-13T10:12:03.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010438900000000001 2024-09-13T10:12:03.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097910+0000", 2024-09-13T10:12:03.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2808999999999997e-05 2024-09-13T10:12:03.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113429+0000", 2024-09-13T10:12:03.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015518557000000001 2024-09-13T10:12:03.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113468+0000", 2024-09-13T10:12:03.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9323000000000003e-05 2024-09-13T10:12:03.893 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.893 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.893 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.893 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:223 3.1 e38/30 3:8759f4e0:::smithi19530714-24 low_tier:head v 38'9, mlcod=38'4)", 2024-09-13T10:12:03.893 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.094775+0000", 2024-09-13T10:12:03.893 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 72.721591044999997, 2024-09-13T10:12:03.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018476023000000001, 2024-09-13T10:12:03.893 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.893 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.893 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.894 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094775+0000", 2024-09-13T10:12:03.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094775+0000", 2024-09-13T10:12:03.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094776+0000", 2024-09-13T10:12:03.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.466e-06 2024-09-13T10:12:03.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.895 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.895 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097100+0000", 2024-09-13T10:12:03.895 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002323435 2024-09-13T10:12:03.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097123+0000", 2024-09-13T10:12:03.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3756000000000001e-05 2024-09-13T10:12:03.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.896 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.896 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.896 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097139+0000", 2024-09-13T10:12:03.896 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5424e-05 2024-09-13T10:12:03.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.098642+0000", 2024-09-13T10:12:03.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015034359999999999 2024-09-13T10:12:03.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.897 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.897 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.098683+0000", 2024-09-13T10:12:03.897 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0756000000000001e-05 2024-09-13T10:12:03.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113193+0000", 2024-09-13T10:12:03.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014509831000000001 2024-09-13T10:12:03.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.898 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.898 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113251+0000", 2024-09-13T10:12:03.898 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7918999999999999e-05 2024-09-13T10:12:03.899 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.899 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.899 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.899 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.899 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1959 4.13 4.e5457733 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:12:03.899 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.866379+0000", 2024-09-13T10:12:03.899 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.949986680000002, 2024-09-13T10:12:03.899 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.83266194900000001, 2024-09-13T10:12:03.900 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.900 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.900 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:12:03.900 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:03.900 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:03.900 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:03.900 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1959 2024-09-13T10:12:03.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.900 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.900 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.866379+0000", 2024-09-13T10:12:03.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.901 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.901 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.866379+0000", 2024-09-13T10:12:03.901 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.866381+0000", 2024-09-13T10:12:03.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.125e-06 2024-09-13T10:12:03.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.902 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.902 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.902 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867454+0000", 2024-09-13T10:12:03.902 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001072649 2024-09-13T10:12:03.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867475+0000", 2024-09-13T10:12:03.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1299999999999999e-05 2024-09-13T10:12:03.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.903 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.903 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867486+0000", 2024-09-13T10:12:03.903 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1167e-05 2024-09-13T10:12:03.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.903 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867581+0000", 2024-09-13T10:12:03.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4561999999999993e-05 2024-09-13T10:12:03.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:12:03.904 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867592+0000", 2024-09-13T10:12:03.904 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1031e-05 2024-09-13T10:12:03.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.904 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.904 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699018+0000", 2024-09-13T10:12:03.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.83142604099999995 2024-09-13T10:12:03.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.905 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.905 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.905 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699041+0000", 2024-09-13T10:12:03.905 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3074e-05 2024-09-13T10:12:03.905 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.905 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.905 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.906 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.906 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1960 4.13 4.e5457733 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:12:03.906 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.874165+0000", 2024-09-13T10:12:03.906 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.94220104, 2024-09-13T10:12:03.906 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82492870799999996, 2024-09-13T10:12:03.906 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.906 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.906 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:12:03.906 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:03.906 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:03.907 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:03.907 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1960 2024-09-13T10:12:03.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.907 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.907 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874165+0000", 2024-09-13T10:12:03.907 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.907 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.907 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874165+0000", 2024-09-13T10:12:03.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.908 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874166+0000", 2024-09-13T10:12:03.908 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.325e-06 2024-09-13T10:12:03.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.908 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874706+0000", 2024-09-13T10:12:03.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00053996899999999995 2024-09-13T10:12:03.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.909 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874719+0000", 2024-09-13T10:12:03.909 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2866e-05 2024-09-13T10:12:03.909 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.909 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.909 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874726+0000", 2024-09-13T10:12:03.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7560000000000002e-06 2024-09-13T10:12:03.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.910 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874801+0000", 2024-09-13T10:12:03.910 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5612999999999996e-05 2024-09-13T10:12:03.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.910 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.910 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:12:03.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874810+0000", 2024-09-13T10:12:03.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2619999999999998e-06 2024-09-13T10:12:03.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699082+0000", 2024-09-13T10:12:03.911 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82427217699999999 2024-09-13T10:12:03.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.911 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.911 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.911 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699093+0000", 2024-09-13T10:12:03.912 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1739999999999999e-05 2024-09-13T10:12:03.912 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.912 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.912 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.912 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.912 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1961 4.13 4.e5457733 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:12:03.912 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.877465+0000", 2024-09-13T10:12:03.912 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.938900789000002, 2024-09-13T10:12:03.913 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82170111099999998, 2024-09-13T10:12:03.913 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.913 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.913 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:12:03.913 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:03.913 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:03.913 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:03.913 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1961 2024-09-13T10:12:03.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.913 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877465+0000", 2024-09-13T10:12:03.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.914 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.914 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.914 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877465+0000", 2024-09-13T10:12:03.914 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877466+0000", 2024-09-13T10:12:03.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0100000000000004e-07 2024-09-13T10:12:03.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.915 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.915 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877640+0000", 2024-09-13T10:12:03.915 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000173795 2024-09-13T10:12:03.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.915 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877650+0000", 2024-09-13T10:12:03.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0465000000000001e-05 2024-09-13T10:12:03.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.916 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.916 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877656+0000", 2024-09-13T10:12:03.916 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6849999999999998e-06 2024-09-13T10:12:03.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.916 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877714+0000", 2024-09-13T10:12:03.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8097999999999998e-05 2024-09-13T10:12:03.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.917 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:12:03.917 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877726+0000", 2024-09-13T10:12:03.917 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1857e-05 2024-09-13T10:12:03.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.917 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699146+0000", 2024-09-13T10:12:03.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82142019700000002 2024-09-13T10:12:03.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.918 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.918 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.918 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699166+0000", 2024-09-13T10:12:03.918 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0213000000000001e-05 2024-09-13T10:12:03.919 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.919 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.919 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.919 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.919 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1962 4.13 4.e5457733 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:12:03.919 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.877669+0000", 2024-09-13T10:12:03.919 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.938696653999997, 2024-09-13T10:12:03.919 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82156359899999998, 2024-09-13T10:12:03.919 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.920 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.920 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:12:03.920 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:03.920 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:03.920 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:03.920 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1962 2024-09-13T10:12:03.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.920 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.920 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.920 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877669+0000", 2024-09-13T10:12:03.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.921 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877669+0000", 2024-09-13T10:12:03.921 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.921 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.921 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877670+0000", 2024-09-13T10:12:03.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9899999999999999e-07 2024-09-13T10:12:03.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877675+0000", 2024-09-13T10:12:03.922 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5510000000000001e-06 2024-09-13T10:12:03.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.922 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.922 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.922 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877684+0000", 2024-09-13T10:12:03.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5829999999999996e-06 2024-09-13T10:12:03.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877694+0000", 2024-09-13T10:12:03.923 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0207000000000001e-05 2024-09-13T10:12:03.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.923 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.923 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.923 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877784+0000", 2024-09-13T10:12:03.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9990999999999999e-05 2024-09-13T10:12:03.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:12:03.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877798+0000", 2024-09-13T10:12:03.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3444e-05 2024-09-13T10:12:03.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.924 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.924 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.924 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699222+0000", 2024-09-13T10:12:03.924 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82142437999999995 2024-09-13T10:12:03.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.925 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.925 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699233+0000", 2024-09-13T10:12:03.925 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0644e-05 2024-09-13T10:12:03.925 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.925 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.925 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.926 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1963 4.13 4.e5457733 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:12:03.926 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.877721+0000", 2024-09-13T10:12:03.926 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 51.938644396000001, 2024-09-13T10:12:03.926 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82158741499999999, 2024-09-13T10:12:03.926 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.926 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.926 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:12:03.926 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:03.926 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:03.926 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:03.926 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1963 2024-09-13T10:12:03.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.927 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.927 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877721+0000", 2024-09-13T10:12:03.927 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.927 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.927 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877721+0000", 2024-09-13T10:12:03.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.928 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877722+0000", 2024-09-13T10:12:03.928 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4099999999999998e-07 2024-09-13T10:12:03.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.928 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.928 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877728+0000", 2024-09-13T10:12:03.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1129999999999997e-06 2024-09-13T10:12:03.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877737+0000", 2024-09-13T10:12:03.929 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3969999999999994e-06 2024-09-13T10:12:03.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.929 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.929 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.929 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877743+0000", 2024-09-13T10:12:03.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7950000000000002e-06 2024-09-13T10:12:03.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877826+0000", 2024-09-13T10:12:03.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2213999999999996e-05 2024-09-13T10:12:03.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.930 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:12:03.930 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877831+0000", 2024-09-13T10:12:03.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.604e-06 2024-09-13T10:12:03.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699295+0000", 2024-09-13T10:12:03.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82146349299999999 2024-09-13T10:12:03.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699309+0000", 2024-09-13T10:12:03.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4058e-05 2024-09-13T10:12:03.931 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.931 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.932 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.932 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:3745 3.0 e90/30 3:161dd296:::f6d96d7a8dac6a7d56d3d9df9ef9c133dce9c515f90d85faf1e037f770fc9485:head v 90'300, mlcod=89'298)", 2024-09-13T10:12:03.932 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:54.612179+0000", 2024-09-13T10:12:03.932 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.2041864209999993, 2024-09-13T10:12:03.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017220320000000001, 2024-09-13T10:12:03.932 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:03.932 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:03.932 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:03.933 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:03.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:03.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.612179+0000", 2024-09-13T10:12:03.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:03.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.612179+0000", 2024-09-13T10:12:03.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:03.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:03.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.612181+0000", 2024-09-13T10:12:03.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.897e-06 2024-09-13T10:12:03.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:03.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.618107+0000", 2024-09-13T10:12:03.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059256550000000002 2024-09-13T10:12:03.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:03.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.618138+0000", 2024-09-13T10:12:03.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0715e-05 2024-09-13T10:12:03.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:03.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.618153+0000", 2024-09-13T10:12:03.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5187e-05 2024-09-13T10:12:03.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:03.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.618319+0000", 2024-09-13T10:12:03.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016641099999999999 2024-09-13T10:12:03.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:03.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.618384+0000", 2024-09-13T10:12:03.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5121000000000002e-05 2024-09-13T10:12:03.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.936 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.936 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:03.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.629302+0000", 2024-09-13T10:12:03.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010917237999999999 2024-09-13T10:12:03.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:03.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:03.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:03.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.629400+0000", 2024-09-13T10:12:03.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8096000000000004e-05 2024-09-13T10:12:03.937 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.937 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.937 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.938 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:03.938 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:03.938 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:03.939 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:03.814+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:03.941 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-13T10:12:03.941 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2080970 2024-09-13T10:12:03.942 INFO:tasks.rados.rados.0.smithi195.stdout:1719: finishing write tid 1 to smithi19530714-28 2024-09-13T10:12:03.942 INFO:tasks.rados.rados.0.smithi195.stdout:1719: finishing write tid 2 to smithi19530714-28 2024-09-13T10:12:03.942 INFO:tasks.rados.rados.0.smithi195.stdout:1719: finishing write tid 3 to smithi19530714-28 2024-09-13T10:12:03.942 INFO:tasks.rados.rados.0.smithi195.stdout:1719: finishing write tid 4 to smithi19530714-28 2024-09-13T10:12:03.942 INFO:tasks.rados.rados.0.smithi195.stdout:1719: oid 28 updating version 0 to 162 2024-09-13T10:12:03.942 INFO:tasks.rados.rados.0.smithi195.stdout:1719: oid 28 updating version 162 to 163 2024-09-13T10:12:03.942 INFO:tasks.rados.rados.0.smithi195.stdout:1719: oid 28 updating version 163 to 164 2024-09-13T10:12:03.942 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 164 (ObjNum 369 snap 0 seq_num 369) dirty exists 2024-09-13T10:12:03.942 INFO:tasks.rados.rados.0.smithi195.stdout:1719: left oid 28 (ObjNum 369 snap 0 seq_num 369) 2024-09-13T10:12:03.942 INFO:tasks.rados.rados.0.smithi195.stdout:1720: finishing write tid 1 to smithi19530714-24 2024-09-13T10:12:03.943 INFO:tasks.rados.rados.0.smithi195.stdout:1720: finishing write tid 2 to smithi19530714-24 2024-09-13T10:12:03.943 INFO:tasks.rados.rados.0.smithi195.stdout:1720: finishing write tid 3 to smithi19530714-24 2024-09-13T10:12:03.943 INFO:tasks.rados.rados.0.smithi195.stdout:1720: finishing write tid 4 to smithi19530714-24 2024-09-13T10:12:03.943 INFO:tasks.rados.rados.0.smithi195.stdout:1720: oid 24 updating version 0 to 166 2024-09-13T10:12:03.943 INFO:tasks.rados.rados.0.smithi195.stdout:1720: oid 24 updating version 166 to 167 2024-09-13T10:12:03.943 INFO:tasks.rados.rados.0.smithi195.stdout:1720: oid 24 version 167 is already newer than 165 2024-09-13T10:12:03.943 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 167 (ObjNum 370 snap 0 seq_num 370) dirty exists 2024-09-13T10:12:03.943 INFO:tasks.rados.rados.0.smithi195.stdout:1720: left oid 24 (ObjNum 370 snap 0 seq_num 370) 2024-09-13T10:12:03.943 INFO:tasks.rados.rados.0.smithi195.stdout:1711: done (9 left) 2024-09-13T10:12:03.943 INFO:tasks.rados.rados.0.smithi195.stdout:1712: done (8 left) 2024-09-13T10:12:03.944 INFO:tasks.rados.rados.0.smithi195.stdout:1714: done (7 left) 2024-09-13T10:12:03.944 INFO:tasks.rados.rados.0.smithi195.stdout:1715: done (6 left) 2024-09-13T10:12:03.944 INFO:tasks.rados.rados.0.smithi195.stdout:1716: done (5 left) 2024-09-13T10:12:03.944 INFO:tasks.rados.rados.0.smithi195.stdout:1717: done (4 left) 2024-09-13T10:12:03.944 INFO:tasks.rados.rados.0.smithi195.stdout:1718: done (3 left) 2024-09-13T10:12:03.944 INFO:tasks.rados.rados.0.smithi195.stdout:1719: done (2 left) 2024-09-13T10:12:03.944 INFO:tasks.rados.rados.0.smithi195.stdout:1720: done (1 left) 2024-09-13T10:12:03.944 INFO:tasks.rados.rados.0.smithi195.stdout:1721: done (0 left) 2024-09-13T10:12:03.944 INFO:tasks.rados.rados.0.smithi195.stdout:1722: set_chunk oid 39 target oid 2024-09-13T10:12:03.944 INFO:tasks.rados.rados.0.smithi195.stdout:1722: set_chunk oid 39 offset: 771072 length: 817664 target oid a883c24e7949982aaa702f95d2b60635ad1eee1f45311ad321d63986e57eaedf offset: 0 snap -1 2024-09-13T10:12:03.945 INFO:tasks.rados.rados.0.smithi195.stdout:1723: read oid 14 snap -1 2024-09-13T10:12:03.945 INFO:tasks.rados.rados.0.smithi195.stdout:1723: expect (ObjNum 243 snap 0 seq_num 243) 2024-09-13T10:12:03.945 INFO:tasks.rados.rados.0.smithi195.stdout:1724: write oid 16 current snap is 0 2024-09-13T10:12:03.945 INFO:tasks.rados.rados.0.smithi195.stdout:1724: seq_num 372 ranges {652204=777572,1952125=300079} 2024-09-13T10:12:03.945 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:03.934+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:03.951 INFO:tasks.rados.rados.0.smithi195.stdout:1724: writing smithi19530714-16 from 652204 to 1429776 tid 1 2024-09-13T10:12:03.967 INFO:tasks.rados.rados.0.smithi195.stdout:1724: writing smithi19530714-16 from 1952125 to 2252204 tid 2 2024-09-13T10:12:03.967 INFO:tasks.rados.rados.0.smithi195.stdout:1725: tier_evict oid 22 2024-09-13T10:12:03.968 INFO:tasks.rados.rados.0.smithi195.stdout:1725: tier_evict oid 22 snap -1 2024-09-13T10:12:03.968 INFO:tasks.rados.rados.0.smithi195.stdout:1722: finishing set_chunk to oid 39 2024-09-13T10:12:03.968 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 40 (ObjNum 316 snap 0 seq_num 316) dirty exists 2024-09-13T10:12:03.968 INFO:tasks.rados.rados.0.smithi195.stdout:1722: done (3 left) 2024-09-13T10:12:03.968 INFO:tasks.rados.rados.0.smithi195.stdout:1726: read oid 38 snap -1 2024-09-13T10:12:03.968 INFO:tasks.rados.rados.0.smithi195.stdout:1726: expect (ObjNum 298 snap 0 seq_num 298) 2024-09-13T10:12:03.969 INFO:tasks.rados.rados.0.smithi195.stdout:1727: read oid 8 snap -1 2024-09-13T10:12:03.969 INFO:tasks.rados.rados.0.smithi195.stdout:1727: expect (ObjNum 351 snap 0 seq_num 351) 2024-09-13T10:12:03.969 INFO:tasks.rados.rados.0.smithi195.stdout:1728: read oid 20 snap -1 2024-09-13T10:12:03.969 INFO:tasks.rados.rados.0.smithi195.stdout:1728: expect (ObjNum 315 snap 0 seq_num 315) 2024-09-13T10:12:03.969 INFO:tasks.rados.rados.0.smithi195.stdout:1729: read oid 2 snap -1 2024-09-13T10:12:03.970 INFO:tasks.rados.rados.0.smithi195.stdout:1729: expect (ObjNum 280 snap 0 seq_num 280) 2024-09-13T10:12:03.970 INFO:tasks.rados.rados.0.smithi195.stdout:1730: read oid 31 snap -1 2024-09-13T10:12:03.970 INFO:tasks.rados.rados.0.smithi195.stdout:1730: expect (ObjNum 366 snap 0 seq_num 366) 2024-09-13T10:12:03.970 INFO:tasks.rados.rados.0.smithi195.stdout:1731: write oid 15 current snap is 0 2024-09-13T10:12:03.970 INFO:tasks.rados.rados.0.smithi195.stdout:1723: expect (ObjNum 243 snap 0 seq_num 243) 2024-09-13T10:12:03.971 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 597860 2024-09-13T10:12:04.041 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:12:04.035+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:12:04.074 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 922714 2024-09-13T10:12:04.099 INFO:tasks.rados.rados.0.smithi195.stdout:1724: finishing write tid 1 to smithi19530714-16 2024-09-13T10:12:04.099 INFO:tasks.rados.rados.0.smithi195.stdout:1731: seq_num 373 ranges {669011=569174,2010607=258404} 2024-09-13T10:12:04.106 INFO:tasks.rados.rados.0.smithi195.stdout:1731: writing smithi19530714-15 from 669011 to 1238185 tid 1 2024-09-13T10:12:04.109 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:04.109 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:12:04.109 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:12:04.109 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:04.112 INFO:tasks.rados.rados.0.smithi195.stdout:1731: writing smithi19530714-15 from 2010607 to 2269011 tid 2 2024-09-13T10:12:04.112 INFO:tasks.rados.rados.0.smithi195.stdout:1723: done (8 left) 2024-09-13T10:12:04.112 INFO:tasks.rados.rados.0.smithi195.stdout:1732: read oid 17 snap -1 2024-09-13T10:12:04.112 INFO:tasks.rados.rados.0.smithi195.stdout:1732: expect (ObjNum 347 snap 0 seq_num 347) 2024-09-13T10:12:04.112 INFO:tasks.rados.rados.0.smithi195.stdout:1733: write (excl) oid 49 current snap is 0 2024-09-13T10:12:04.112 INFO:tasks.rados.rados.0.smithi195.stdout:1733: seq_num 374 ranges {685818=644423,2069089=216729} 2024-09-13T10:12:04.119 INFO:tasks.rados.rados.0.smithi195.stdout:1733: writing smithi19530714-49 from 685818 to 1330241 tid 1 2024-09-13T10:12:04.124 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-13T10:12:04.125 INFO:tasks.rados.rados.0.smithi195.stdout:1733: writing smithi19530714-49 from 2069089 to 2285818 tid 2 2024-09-13T10:12:04.125 INFO:tasks.rados.rados.0.smithi195.stdout:1734: read oid 36 snap -1 2024-09-13T10:12:04.125 INFO:tasks.rados.rados.0.smithi195.stdout:1734: expect deleted 2024-09-13T10:12:04.126 INFO:tasks.rados.rados.0.smithi195.stdout:1735: tier_evict oid 5 2024-09-13T10:12:04.126 INFO:tasks.rados.rados.0.smithi195.stdout:1735: tier_evict oid 5 snap -1 2024-09-13T10:12:04.126 INFO:tasks.rados.rados.0.smithi195.stdout:1725: got (22) Invalid argument 2024-09-13T10:12:04.126 INFO:tasks.rados.rados.0.smithi195.stdout:1724: finishing write tid 2 to smithi19530714-16 2024-09-13T10:12:04.126 INFO:tasks.rados.rados.0.smithi195.stdout:1724: finishing write tid 3 to smithi19530714-16 2024-09-13T10:12:04.126 INFO:tasks.rados.rados.0.smithi195.stdout:1724: finishing write tid 4 to smithi19530714-16 2024-09-13T10:12:04.126 INFO:tasks.rados.rados.0.smithi195.stdout:1724: oid 16 updating version 0 to 57 2024-09-13T10:12:04.126 INFO:tasks.rados.rados.0.smithi195.stdout:1724: oid 16 updating version 57 to 58 2024-09-13T10:12:04.126 INFO:tasks.rados.rados.0.smithi195.stdout:1724: oid 16 updating version 58 to 59 2024-09-13T10:12:04.126 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 59 (ObjNum 372 snap 0 seq_num 372) dirty exists 2024-09-13T10:12:04.126 INFO:tasks.rados.rados.0.smithi195.stdout:1724: left oid 16 (ObjNum 372 snap 0 seq_num 372) 2024-09-13T10:12:04.126 INFO:tasks.rados.rados.0.smithi195.stdout:1728: expect (ObjNum 315 snap 0 seq_num 315) 2024-09-13T10:12:04.141 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:04.138+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:04.232 INFO:tasks.rados.rados.0.smithi195.stdout:1730: expect (ObjNum 366 snap 0 seq_num 366) 2024-09-13T10:12:04.242 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:12:04.235+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:12:04.249 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 608479 2024-09-13T10:12:04.264 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 230355 2024-09-13T10:12:04.284 INFO:tasks.rados.rados.0.smithi195.stdout:1729: expect (ObjNum 280 snap 0 seq_num 280) 2024-09-13T10:12:04.285 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:04.285 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:04.285 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:12:04.285 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:12:04.285 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:12:04.286 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:04.298 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-13T10:12:04.343 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:04.337+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:04.344 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:04.444 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:04.442+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:04.458 INFO:tasks.rados.rados.0.smithi195.stdout:1727: expect (ObjNum 351 snap 0 seq_num 351) 2024-09-13T10:12:04.465 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:04.465 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:12:04.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:12:04.466 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:12:04.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.466 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:04.466 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:12:04.466 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 94.080070152999994, 2024-09-13T10:12:04.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034624456999999997, 2024-09-13T10:12:04.466 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.466 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.466 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.466 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.467 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:04.467 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:04.467 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 70 2024-09-13T10:12:04.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.467 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:12:04.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:12:04.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381723+0000", 2024-09-13T10:12:04.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.540000000000001e-07 2024-09-13T10:12:04.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381731+0000", 2024-09-13T10:12:04.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.486e-06 2024-09-13T10:12:04.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381742+0000", 2024-09-13T10:12:04.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1269e-05 2024-09-13T10:12:04.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381750+0000", 2024-09-13T10:12:04.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8830000000000006e-06 2024-09-13T10:12:04.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401255+0000", 2024-09-13T10:12:04.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019505110999999999 2024-09-13T10:12:04.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401388+0000", 2024-09-13T10:12:04.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013253200000000001 2024-09-13T10:12:04.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:04.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401609+0000", 2024-09-13T10:12:04.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000220981 2024-09-13T10:12:04.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407538+0000", 2024-09-13T10:12:04.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059291220000000002 2024-09-13T10:12:04.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407626+0000", 2024-09-13T10:12:04.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8361000000000003e-05 2024-09-13T10:12:04.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.471 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:04.471 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416154+0000", 2024-09-13T10:12:04.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0085281000000000003 2024-09-13T10:12:04.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416234+0000", 2024-09-13T10:12:04.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9594999999999999e-05 2024-09-13T10:12:04.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416347+0000", 2024-09-13T10:12:04.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113063 2024-09-13T10:12:04.472 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.472 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.472 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.473 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:71 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:04.473 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:12:04.473 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 94.080029570999997, 2024-09-13T10:12:04.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035004, 2024-09-13T10:12:04.473 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.473 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.473 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.473 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.473 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:04.473 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:04.473 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 71 2024-09-13T10:12:04.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.474 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:12:04.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:12:04.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381764+0000", 2024-09-13T10:12:04.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4600000000000004e-07 2024-09-13T10:12:04.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381769+0000", 2024-09-13T10:12:04.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.48e-06 2024-09-13T10:12:04.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381780+0000", 2024-09-13T10:12:04.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0893000000000001e-05 2024-09-13T10:12:04.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381788+0000", 2024-09-13T10:12:04.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6389999999999992e-06 2024-09-13T10:12:04.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407896+0000", 2024-09-13T10:12:04.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.026107807 2024-09-13T10:12:04.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.476 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.408029+0000", 2024-09-13T10:12:04.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013383899999999999 2024-09-13T10:12:04.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:04.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.408250+0000", 2024-09-13T10:12:04.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022105399999999999 2024-09-13T10:12:04.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416130+0000", 2024-09-13T10:12:04.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0078798500000000007 2024-09-13T10:12:04.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416488+0000", 2024-09-13T10:12:04.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00035819299999999998 2024-09-13T10:12:04.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:04.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416595+0000", 2024-09-13T10:12:04.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001064 2024-09-13T10:12:04.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416662+0000", 2024-09-13T10:12:04.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6775999999999999e-05 2024-09-13T10:12:04.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416767+0000", 2024-09-13T10:12:04.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000105423 2024-09-13T10:12:04.479 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.479 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.479 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.479 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:72 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:04.479 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:12:04.480 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 94.079992536999995, 2024-09-13T10:12:04.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039399762999999997, 2024-09-13T10:12:04.480 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.480 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.480 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.480 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.480 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:04.480 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:04.480 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 72 2024-09-13T10:12:04.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.480 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:12:04.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:12:04.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.481 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.481 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381801+0000", 2024-09-13T10:12:04.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7700000000000004e-07 2024-09-13T10:12:04.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381898+0000", 2024-09-13T10:12:04.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7066999999999997e-05 2024-09-13T10:12:04.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381908+0000", 2024-09-13T10:12:04.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0336000000000001e-05 2024-09-13T10:12:04.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381917+0000", 2024-09-13T10:12:04.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5169999999999999e-06 2024-09-13T10:12:04.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.411900+0000", 2024-09-13T10:12:04.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029983381999999999 2024-09-13T10:12:04.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.412028+0000", 2024-09-13T10:12:04.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127916 2024-09-13T10:12:04.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:04.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.412248+0000", 2024-09-13T10:12:04.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022014699999999999 2024-09-13T10:12:04.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416576+0000", 2024-09-13T10:12:04.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004327549 2024-09-13T10:12:04.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416860+0000", 2024-09-13T10:12:04.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028391000000000001 2024-09-13T10:12:04.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:04.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421014+0000", 2024-09-13T10:12:04.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041539330000000003 2024-09-13T10:12:04.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421093+0000", 2024-09-13T10:12:04.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9097999999999994e-05 2024-09-13T10:12:04.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421200+0000", 2024-09-13T10:12:04.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000107231 2024-09-13T10:12:04.486 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.486 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.486 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.486 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:73 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:04.486 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:12:04.486 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 94.079858825000002, 2024-09-13T10:12:04.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044034528000000003, 2024-09-13T10:12:04.486 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.486 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.486 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.487 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.487 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:04.487 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:04.487 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 73 2024-09-13T10:12:04.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.487 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:12:04.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:12:04.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381935+0000", 2024-09-13T10:12:04.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0999999999999997e-07 2024-09-13T10:12:04.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381940+0000", 2024-09-13T10:12:04.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7030000000000003e-06 2024-09-13T10:12:04.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381949+0000", 2024-09-13T10:12:04.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5359999999999992e-06 2024-09-13T10:12:04.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381961+0000", 2024-09-13T10:12:04.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2539e-05 2024-09-13T10:12:04.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416942+0000", 2024-09-13T10:12:04.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034980270000000001 2024-09-13T10:12:04.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.417074+0000", 2024-09-13T10:12:04.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013246199999999999 2024-09-13T10:12:04.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:04.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.417298+0000", 2024-09-13T10:12:04.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000223823 2024-09-13T10:12:04.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425459+0000", 2024-09-13T10:12:04.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081613750000000002 2024-09-13T10:12:04.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425575+0000", 2024-09-13T10:12:04.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115513 2024-09-13T10:12:04.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.491 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:04.491 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425774+0000", 2024-09-13T10:12:04.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019959399999999999 2024-09-13T10:12:04.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425854+0000", 2024-09-13T10:12:04.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9532999999999999e-05 2024-09-13T10:12:04.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425968+0000", 2024-09-13T10:12:04.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011437 2024-09-13T10:12:04.492 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.492 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.492 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.493 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:74 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:04.493 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:12:04.493 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 94.079815951, 2024-09-13T10:12:04.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052701496, 2024-09-13T10:12:04.493 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.493 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.493 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.494 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.494 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:04.494 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:04.494 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 74 2024-09-13T10:12:04.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.494 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:12:04.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:12:04.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:12:04.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7999999999999995e-07 2024-09-13T10:12:04.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381983+0000", 2024-09-13T10:12:04.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4430000000000002e-06 2024-09-13T10:12:04.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381991+0000", 2024-09-13T10:12:04.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8960000000000003e-06 2024-09-13T10:12:04.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381998+0000", 2024-09-13T10:12:04.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7729999999999999e-06 2024-09-13T10:12:04.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421291+0000", 2024-09-13T10:12:04.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039292753 2024-09-13T10:12:04.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421424+0000", 2024-09-13T10:12:04.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013291900000000001 2024-09-13T10:12:04.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:04.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421646+0000", 2024-09-13T10:12:04.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000221843 2024-09-13T10:12:04.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425667+0000", 2024-09-13T10:12:04.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040206779999999998 2024-09-13T10:12:04.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425755+0000", 2024-09-13T10:12:04.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8848e-05 2024-09-13T10:12:04.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:04.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434472+0000", 2024-09-13T10:12:04.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0087164879999999997 2024-09-13T10:12:04.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434544+0000", 2024-09-13T10:12:04.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2273000000000004e-05 2024-09-13T10:12:04.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434678+0000", 2024-09-13T10:12:04.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013390200000000001 2024-09-13T10:12:04.499 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.499 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.500 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.500 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:75 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:04.500 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:12:04.500 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 94.079779086000002, 2024-09-13T10:12:04.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052864585999999998, 2024-09-13T10:12:04.500 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.500 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.500 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.500 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.500 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:04.500 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:04.500 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 75 2024-09-13T10:12:04.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.501 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:12:04.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:12:04.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:12:04.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8299999999999996e-07 2024-09-13T10:12:04.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382120+0000", 2024-09-13T10:12:04.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010627400000000001 2024-09-13T10:12:04.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382131+0000", 2024-09-13T10:12:04.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0796e-05 2024-09-13T10:12:04.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382140+0000", 2024-09-13T10:12:04.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5560000000000008e-06 2024-09-13T10:12:04.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426014+0000", 2024-09-13T10:12:04.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.043873843000000003 2024-09-13T10:12:04.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426205+0000", 2024-09-13T10:12:04.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019154600000000001 2024-09-13T10:12:04.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:04.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426429+0000", 2024-09-13T10:12:04.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022344699999999999 2024-09-13T10:12:04.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434352+0000", 2024-09-13T10:12:04.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007923355 2024-09-13T10:12:04.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434452+0000", 2024-09-13T10:12:04.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9961000000000001e-05 2024-09-13T10:12:04.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:04.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434700+0000", 2024-09-13T10:12:04.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024790899999999999 2024-09-13T10:12:04.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434774+0000", 2024-09-13T10:12:04.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4457999999999998e-05 2024-09-13T10:12:04.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434878+0000", 2024-09-13T10:12:04.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010375800000000001 2024-09-13T10:12:04.506 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.506 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.506 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.506 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:76 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:04.506 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:12:04.506 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 94.079612105999999, 2024-09-13T10:12:04.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053104138000000002, 2024-09-13T10:12:04.507 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.507 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.507 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:04.507 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:04.507 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 76 2024-09-13T10:12:04.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.507 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:12:04.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:12:04.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:12:04.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4600000000000003e-07 2024-09-13T10:12:04.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382188+0000", 2024-09-13T10:12:04.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4869999999999999e-06 2024-09-13T10:12:04.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382197+0000", 2024-09-13T10:12:04.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5519999999999993e-06 2024-09-13T10:12:04.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382207+0000", 2024-09-13T10:12:04.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7240000000000004e-06 2024-09-13T10:12:04.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430285+0000", 2024-09-13T10:12:04.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.048078093000000002 2024-09-13T10:12:04.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430429+0000", 2024-09-13T10:12:04.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000143981 2024-09-13T10:12:04.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:04.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430717+0000", 2024-09-13T10:12:04.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028802 2024-09-13T10:12:04.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435015+0000", 2024-09-13T10:12:04.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042973150000000003 2024-09-13T10:12:04.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435083+0000", 2024-09-13T10:12:04.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8448999999999995e-05 2024-09-13T10:12:04.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:04.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435109+0000", 2024-09-13T10:12:04.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6064999999999999e-05 2024-09-13T10:12:04.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435176+0000", 2024-09-13T10:12:04.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7362999999999999e-05 2024-09-13T10:12:04.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435285+0000", 2024-09-13T10:12:04.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108243 2024-09-13T10:12:04.513 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.513 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.513 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.513 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:96 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:04.513 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:12:04.513 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 93.937980675999995, 2024-09-13T10:12:04.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032701419000000002, 2024-09-13T10:12:04.513 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.513 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.513 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:04.514 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:04.514 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 96 2024-09-13T10:12:04.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.514 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:12:04.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:12:04.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523813+0000", 2024-09-13T10:12:04.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.364e-06 2024-09-13T10:12:04.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523913+0000", 2024-09-13T10:12:04.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9827000000000005e-05 2024-09-13T10:12:04.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523927+0000", 2024-09-13T10:12:04.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4029000000000001e-05 2024-09-13T10:12:04.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523936+0000", 2024-09-13T10:12:04.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1030000000000008e-06 2024-09-13T10:12:04.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.544707+0000", 2024-09-13T10:12:04.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0207706 2024-09-13T10:12:04.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.544834+0000", 2024-09-13T10:12:04.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012683200000000001 2024-09-13T10:12:04.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:04.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.545057+0000", 2024-09-13T10:12:04.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022338100000000001 2024-09-13T10:12:04.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549279+0000", 2024-09-13T10:12:04.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042223080000000001 2024-09-13T10:12:04.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549556+0000", 2024-09-13T10:12:04.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000276307 2024-09-13T10:12:04.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:04.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556364+0000", 2024-09-13T10:12:04.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0068082510000000004 2024-09-13T10:12:04.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556445+0000", 2024-09-13T10:12:04.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1081000000000005e-05 2024-09-13T10:12:04.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556513+0000", 2024-09-13T10:12:04.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8336000000000004e-05 2024-09-13T10:12:04.519 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.519 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.519 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.519 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:97 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:04.520 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:12:04.520 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 93.937837791000007, 2024-09-13T10:12:04.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035410807000000002, 2024-09-13T10:12:04.520 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.520 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.520 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.520 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.520 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:04.520 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:04.520 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 97 2024-09-13T10:12:04.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.520 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:12:04.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:12:04.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523956+0000", 2024-09-13T10:12:04.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8599999999999997e-07 2024-09-13T10:12:04.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523962+0000", 2024-09-13T10:12:04.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1240000000000002e-06 2024-09-13T10:12:04.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523971+0000", 2024-09-13T10:12:04.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7320000000000004e-06 2024-09-13T10:12:04.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523978+0000", 2024-09-13T10:12:04.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0899999999999999e-06 2024-09-13T10:12:04.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549637+0000", 2024-09-13T10:12:04.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025658876000000001 2024-09-13T10:12:04.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549757+0000", 2024-09-13T10:12:04.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000120837 2024-09-13T10:12:04.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:04.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549971+0000", 2024-09-13T10:12:04.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021356000000000001 2024-09-13T10:12:04.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556335+0000", 2024-09-13T10:12:04.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0063639329999999996 2024-09-13T10:12:04.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556725+0000", 2024-09-13T10:12:04.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000390221 2024-09-13T10:12:04.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:04.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559219+0000", 2024-09-13T10:12:04.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0024943500000000002 2024-09-13T10:12:04.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559296+0000", 2024-09-13T10:12:04.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7065999999999997e-05 2024-09-13T10:12:04.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559366+0000", 2024-09-13T10:12:04.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9132000000000002e-05 2024-09-13T10:12:04.526 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.526 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.526 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.527 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:98 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:04.527 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:12:04.527 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 93.937803183, 2024-09-13T10:12:04.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035839604999999997, 2024-09-13T10:12:04.527 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.527 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.527 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.527 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.527 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:04.527 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:04.527 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 98 2024-09-13T10:12:04.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.527 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:12:04.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:12:04.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523990+0000", 2024-09-13T10:12:04.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6300000000000005e-07 2024-09-13T10:12:04.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523995+0000", 2024-09-13T10:12:04.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2669999999999997e-06 2024-09-13T10:12:04.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524007+0000", 2024-09-13T10:12:04.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2002e-05 2024-09-13T10:12:04.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524015+0000", 2024-09-13T10:12:04.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4869999999999996e-06 2024-09-13T10:12:04.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.553854+0000", 2024-09-13T10:12:04.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029839633000000001 2024-09-13T10:12:04.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.553985+0000", 2024-09-13T10:12:04.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013032899999999999 2024-09-13T10:12:04.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:04.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.554212+0000", 2024-09-13T10:12:04.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022723400000000001 2024-09-13T10:12:04.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559523+0000", 2024-09-13T10:12:04.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053113509999999997 2024-09-13T10:12:04.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559662+0000", 2024-09-13T10:12:04.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013833500000000001 2024-09-13T10:12:04.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:04.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559698+0000", 2024-09-13T10:12:04.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5982000000000002e-05 2024-09-13T10:12:04.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559764+0000", 2024-09-13T10:12:04.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6019000000000006e-05 2024-09-13T10:12:04.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559829+0000", 2024-09-13T10:12:04.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5302999999999995e-05 2024-09-13T10:12:04.533 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.533 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.533 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.533 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:99 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:04.533 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:12:04.533 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 93.937765489, 2024-09-13T10:12:04.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.037637084000000001, 2024-09-13T10:12:04.533 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.534 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.534 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.534 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.534 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:04.534 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:04.534 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 99 2024-09-13T10:12:04.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.534 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:12:04.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:12:04.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524028+0000", 2024-09-13T10:12:04.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7999999999999995e-07 2024-09-13T10:12:04.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524109+0000", 2024-09-13T10:12:04.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0834999999999994e-05 2024-09-13T10:12:04.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524119+0000", 2024-09-13T10:12:04.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0149999999999999e-05 2024-09-13T10:12:04.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524127+0000", 2024-09-13T10:12:04.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8660000000000006e-06 2024-09-13T10:12:04.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556868+0000", 2024-09-13T10:12:04.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.03274146 2024-09-13T10:12:04.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.557001+0000", 2024-09-13T10:12:04.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000133249 2024-09-13T10:12:04.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:04.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.557224+0000", 2024-09-13T10:12:04.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000222914 2024-09-13T10:12:04.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561387+0000", 2024-09-13T10:12:04.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041627019999999999 2024-09-13T10:12:04.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:04.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561419+0000", 2024-09-13T10:12:04.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1924999999999997e-05 2024-09-13T10:12:04.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561476+0000", 2024-09-13T10:12:04.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7346999999999998e-05 2024-09-13T10:12:04.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561552+0000", 2024-09-13T10:12:04.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5847999999999996e-05 2024-09-13T10:12:04.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561664+0000", 2024-09-13T10:12:04.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112108 2024-09-13T10:12:04.539 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.539 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.540 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.540 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:100 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:04.540 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:12:04.540 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 93.937589338999999, 2024-09-13T10:12:04.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.038825510000000001, 2024-09-13T10:12:04.540 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.540 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.540 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.540 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.540 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:04.540 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:04.540 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 100 2024-09-13T10:12:04.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.541 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:12:04.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:12:04.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524204+0000", 2024-09-13T10:12:04.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.54e-07 2024-09-13T10:12:04.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524211+0000", 2024-09-13T10:12:04.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6919999999999996e-06 2024-09-13T10:12:04.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524221+0000", 2024-09-13T10:12:04.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0223e-05 2024-09-13T10:12:04.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524229+0000", 2024-09-13T10:12:04.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9859999999999994e-06 2024-09-13T10:12:04.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559920+0000", 2024-09-13T10:12:04.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035690854000000001 2024-09-13T10:12:04.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.560052+0000", 2024-09-13T10:12:04.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013199899999999999 2024-09-13T10:12:04.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:04.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.560356+0000", 2024-09-13T10:12:04.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030450200000000001 2024-09-13T10:12:04.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:04.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562115+0000", 2024-09-13T10:12:04.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0017586430000000001 2024-09-13T10:12:04.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.545 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.545 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562512+0000", 2024-09-13T10:12:04.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00039728099999999998 2024-09-13T10:12:04.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.545 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.545 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562828+0000", 2024-09-13T10:12:04.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00031528799999999999 2024-09-13T10:12:04.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.545 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.545 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562913+0000", 2024-09-13T10:12:04.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5628000000000001e-05 2024-09-13T10:12:04.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.546 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.546 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.563029+0000", 2024-09-13T10:12:04.546 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011556 2024-09-13T10:12:04.546 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.546 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.546 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.546 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:122 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:04.546 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:12:04.546 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 93.820904858000006, 2024-09-13T10:12:04.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.027900708, 2024-09-13T10:12:04.547 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.547 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.547 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.547 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.547 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:04.547 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:04.547 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 122 2024-09-13T10:12:04.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.547 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:12:04.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640888+0000", 2024-09-13T10:12:04.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640889+0000", 2024-09-13T10:12:04.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4139999999999999e-06 2024-09-13T10:12:04.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640900+0000", 2024-09-13T10:12:04.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0559e-05 2024-09-13T10:12:04.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640916+0000", 2024-09-13T10:12:04.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6399e-05 2024-09-13T10:12:04.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640931+0000", 2024-09-13T10:12:04.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4659e-05 2024-09-13T10:12:04.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.550 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.550 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.550 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663602+0000", 2024-09-13T10:12:04.550 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.022671674999999999 2024-09-13T10:12:04.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.550 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.550 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.550 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663734+0000", 2024-09-13T10:12:04.550 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013189899999999999 2024-09-13T10:12:04.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.550 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.550 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:04.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.663946+0000", 2024-09-13T10:12:04.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021198699999999999 2024-09-13T10:12:04.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668523+0000", 2024-09-13T10:12:04.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045765789999999999 2024-09-13T10:12:04.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668638+0000", 2024-09-13T10:12:04.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011504 2024-09-13T10:12:04.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:04.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668657+0000", 2024-09-13T10:12:04.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9298999999999999e-05 2024-09-13T10:12:04.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668722+0000", 2024-09-13T10:12:04.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5126e-05 2024-09-13T10:12:04.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668788+0000", 2024-09-13T10:12:04.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6072e-05 2024-09-13T10:12:04.553 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.553 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.553 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.553 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:123 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:04.553 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:12:04.553 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 93.820838980999994, 2024-09-13T10:12:04.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028273795000000001, 2024-09-13T10:12:04.553 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.553 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.553 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.554 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.554 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:04.554 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:04.554 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 123 2024-09-13T10:12:04.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.554 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.554 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:12:04.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.554 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640954+0000", 2024-09-13T10:12:04.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.640955+0000", 2024-09-13T10:12:04.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.356e-06 2024-09-13T10:12:04.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641083+0000", 2024-09-13T10:12:04.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012763600000000001 2024-09-13T10:12:04.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641102+0000", 2024-09-13T10:12:04.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9055e-05 2024-09-13T10:12:04.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641117+0000", 2024-09-13T10:12:04.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5665999999999999e-05 2024-09-13T10:12:04.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.665747+0000", 2024-09-13T10:12:04.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.024629796999999998 2024-09-13T10:12:04.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.665876+0000", 2024-09-13T10:12:04.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012925600000000001 2024-09-13T10:12:04.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:04.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.666105+0000", 2024-09-13T10:12:04.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000228543 2024-09-13T10:12:04.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.557 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.557 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.668924+0000", 2024-09-13T10:12:04.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028194309999999998 2024-09-13T10:12:04.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669070+0000", 2024-09-13T10:12:04.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014560199999999999 2024-09-13T10:12:04.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:04.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669104+0000", 2024-09-13T10:12:04.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3654e-05 2024-09-13T10:12:04.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669173+0000", 2024-09-13T10:12:04.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9819000000000004e-05 2024-09-13T10:12:04.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669227+0000", 2024-09-13T10:12:04.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3980000000000002e-05 2024-09-13T10:12:04.559 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.559 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.559 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.559 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:124 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:04.559 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:12:04.560 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 93.820645940000006, 2024-09-13T10:12:04.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030852801999999999, 2024-09-13T10:12:04.560 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.560 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.560 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.560 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.560 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:04.560 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:04.560 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 124 2024-09-13T10:12:04.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.560 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.560 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.560 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.560 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:12:04.560 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641147+0000", 2024-09-13T10:12:04.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641148+0000", 2024-09-13T10:12:04.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4529999999999999e-06 2024-09-13T10:12:04.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641158+0000", 2024-09-13T10:12:04.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0003999999999999e-05 2024-09-13T10:12:04.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641199+0000", 2024-09-13T10:12:04.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0426999999999998e-05 2024-09-13T10:12:04.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.641214+0000", 2024-09-13T10:12:04.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5716e-05 2024-09-13T10:12:04.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669267+0000", 2024-09-13T10:12:04.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028053112000000002 2024-09-13T10:12:04.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669394+0000", 2024-09-13T10:12:04.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012656000000000001 2024-09-13T10:12:04.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:04.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.669614+0000", 2024-09-13T10:12:04.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021970400000000001 2024-09-13T10:12:04.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671267+0000", 2024-09-13T10:12:04.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001652976 2024-09-13T10:12:04.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671383+0000", 2024-09-13T10:12:04.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000116113 2024-09-13T10:12:04.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:04.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671828+0000", 2024-09-13T10:12:04.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000444871 2024-09-13T10:12:04.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671935+0000", 2024-09-13T10:12:04.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010703 2024-09-13T10:12:04.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.671999+0000", 2024-09-13T10:12:04.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4836000000000001e-05 2024-09-13T10:12:04.565 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.565 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.565 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.566 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:201 3.3 e38/30 3:d4fb61be:::smithi19530714-2 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:12:04.566 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:12:04.566 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.448195046999999, 2024-09-13T10:12:04.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032127035999999998, 2024-09-13T10:12:04.566 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.566 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.566 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.566 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:12:04.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:12:04.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:12:04.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2600000000000001e-07 2024-09-13T10:12:04.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030449+0000", 2024-09-13T10:12:04.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016850477999999999 2024-09-13T10:12:04.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030499+0000", 2024-09-13T10:12:04.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0212e-05 2024-09-13T10:12:04.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030524+0000", 2024-09-13T10:12:04.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5222e-05 2024-09-13T10:12:04.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030739+0000", 2024-09-13T10:12:04.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000214396 2024-09-13T10:12:04.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030818+0000", 2024-09-13T10:12:04.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8941000000000002e-05 2024-09-13T10:12:04.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.045512+0000", 2024-09-13T10:12:04.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014694248 2024-09-13T10:12:04.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.045725+0000", 2024-09-13T10:12:04.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021261300000000001 2024-09-13T10:12:04.570 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.570 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.570 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.570 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:226 3.3 e38/30 3:fe5bfa3a:::smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'5, mlcod=38'3)", 2024-09-13T10:12:04.570 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:12:04.570 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.381334804000005, 2024-09-13T10:12:04.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030943633000000002, 2024-09-13T10:12:04.570 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.570 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.570 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.571 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:12:04.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:12:04.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080461+0000", 2024-09-13T10:12:04.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7049999999999999e-06 2024-09-13T10:12:04.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097571+0000", 2024-09-13T10:12:04.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017110658000000001 2024-09-13T10:12:04.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097615+0000", 2024-09-13T10:12:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3404e-05 2024-09-13T10:12:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097635+0000", 2024-09-13T10:12:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0279e-05 2024-09-13T10:12:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097995+0000", 2024-09-13T10:12:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00036038699999999999 2024-09-13T10:12:04.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.098029+0000", 2024-09-13T10:12:04.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4162000000000003e-05 2024-09-13T10:12:04.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111330+0000", 2024-09-13T10:12:04.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013300268000000001 2024-09-13T10:12:04.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111401+0000", 2024-09-13T10:12:04.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1769999999999999e-05 2024-09-13T10:12:04.576 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.576 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.576 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.576 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.576 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:237 3.3 e38/30 3:de40747e:::smithi19530714-38 low_tier:head v 38'7, mlcod=38'3)", 2024-09-13T10:12:04.576 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:12:04.576 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.347102011999993, 2024-09-13T10:12:04.576 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.037747504000000001, 2024-09-13T10:12:04.576 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.576 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.576 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.576 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:12:04.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:12:04.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.577 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.577 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:12:04.577 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7299999999999995e-07 2024-09-13T10:12:04.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145311+0000", 2024-09-13T10:12:04.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030619621 2024-09-13T10:12:04.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145336+0000", 2024-09-13T10:12:04.578 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5009999999999999e-05 2024-09-13T10:12:04.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.578 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.578 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.578 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145349+0000", 2024-09-13T10:12:04.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2724999999999999e-05 2024-09-13T10:12:04.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145445+0000", 2024-09-13T10:12:04.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6108999999999997e-05 2024-09-13T10:12:04.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.579 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.579 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.579 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145470+0000", 2024-09-13T10:12:04.579 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5514e-05 2024-09-13T10:12:04.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152389+0000", 2024-09-13T10:12:04.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069190739999999999 2024-09-13T10:12:04.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.580 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.580 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.580 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152438+0000", 2024-09-13T10:12:04.580 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8778e-05 2024-09-13T10:12:04.580 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.580 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.581 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.581 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.581 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.581 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:286 4.a 4:5ad8883b:::smithi19530714-37:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:04.581 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:12:04.581 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.213505385000005, 2024-09-13T10:12:04.581 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032980435000000002, 2024-09-13T10:12:04.581 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.581 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.581 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:04.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:04.581 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:04.582 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 286 2024-09-13T10:12:04.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.582 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:12:04.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.582 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.582 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.582 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:12:04.582 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248290+0000", 2024-09-13T10:12:04.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0170000000000001e-06 2024-09-13T10:12:04.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248302+0000", 2024-09-13T10:12:04.583 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1602e-05 2024-09-13T10:12:04.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.583 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.583 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.583 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248328+0000", 2024-09-13T10:12:04.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.622e-05 2024-09-13T10:12:04.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248344+0000", 2024-09-13T10:12:04.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6192999999999999e-05 2024-09-13T10:12:04.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.584 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248466+0000", 2024-09-13T10:12:04.584 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012186899999999999 2024-09-13T10:12:04.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.584 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.584 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248602+0000", 2024-09-13T10:12:04.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013623500000000001 2024-09-13T10:12:04.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 6,13", 2024-09-13T10:12:04.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274091+0000", 2024-09-13T10:12:04.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025489050999999999 2024-09-13T10:12:04.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.585 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.585 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.585 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275976+0000", 2024-09-13T10:12:04.585 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018847289999999999 2024-09-13T10:12:04.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:04.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276801+0000", 2024-09-13T10:12:04.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00082477899999999999 2024-09-13T10:12:04.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:04.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281064+0000", 2024-09-13T10:12:04.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042625950000000001 2024-09-13T10:12:04.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.586 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.586 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:04.586 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281173+0000", 2024-09-13T10:12:04.586 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000109391 2024-09-13T10:12:04.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.587 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.587 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281268+0000", 2024-09-13T10:12:04.587 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4753999999999994e-05 2024-09-13T10:12:04.587 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.587 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.587 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.587 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.587 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1631 4.a 4:5ad8883b:::smithi19530714-37:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:12:04.587 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:06.099753+0000", 2024-09-13T10:12:04.587 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 58.362039811999999, 2024-09-13T10:12:04.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033125612999999998, 2024-09-13T10:12:04.588 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:04.588 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:04.588 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:12:04.588 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:04.588 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:04.588 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:04.588 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1631 2024-09-13T10:12:04.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.588 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:04.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.588 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:04.588 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099753+0000", 2024-09-13T10:12:04.588 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:04.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099753+0000", 2024-09-13T10:12:04.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:04.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:04.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099754+0000", 2024-09-13T10:12:04.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2699999999999999e-06 2024-09-13T10:12:04.589 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.589 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.589 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:04.589 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099758+0000", 2024-09-13T10:12:04.589 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6540000000000001e-06 2024-09-13T10:12:04.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:04.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099768+0000", 2024-09-13T10:12:04.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0412e-05 2024-09-13T10:12:04.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:04.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099777+0000", 2024-09-13T10:12:04.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4600000000000003e-06 2024-09-13T10:12:04.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.590 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.590 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:04.590 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099830+0000", 2024-09-13T10:12:04.590 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3192000000000002e-05 2024-09-13T10:12:04.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:04.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099889+0000", 2024-09-13T10:12:04.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9576e-05 2024-09-13T10:12:04.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:04.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:04.591 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:04.591 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.132878+0000", 2024-09-13T10:12:04.591 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032989048999999999 2024-09-13T10:12:04.591 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.591 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.591 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.591 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:04.591 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:04.591 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:04.593 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3757140 2024-09-13T10:12:04.593 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:04.593 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:04.589+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:04.594 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-13T10:12:04.608 INFO:tasks.rados.rados.0.smithi195.stdout:1726: expect (ObjNum 298 snap 0 seq_num 298) 2024-09-13T10:12:04.694 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:12:04.691+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:12:04.696 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 196463 2024-09-13T10:12:04.719 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:04.720 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:12:04.720 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:12:04.720 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:04.732 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T10:12:04.734 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-13T10:12:04.794 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:04.790+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:04.864 INFO:tasks.rados.rados.0.smithi195.stdout:1731: finishing write tid 1 to smithi19530714-15 2024-09-13T10:12:04.865 INFO:tasks.rados.rados.0.smithi195.stdout:1731: finishing write tid 2 to smithi19530714-15 2024-09-13T10:12:04.865 INFO:tasks.rados.rados.0.smithi195.stdout:1731: finishing write tid 3 to smithi19530714-15 2024-09-13T10:12:04.865 INFO:tasks.rados.rados.0.smithi195.stdout:1731: finishing write tid 4 to smithi19530714-15 2024-09-13T10:12:04.865 INFO:tasks.rados.rados.0.smithi195.stdout:1731: oid 15 updating version 0 to 114 2024-09-13T10:12:04.865 INFO:tasks.rados.rados.0.smithi195.stdout:1731: oid 15 version 114 is already newer than 112 2024-09-13T10:12:04.865 INFO:tasks.rados.rados.0.smithi195.stdout:1731: oid 15 version 114 is already newer than 113 2024-09-13T10:12:04.865 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 114 (ObjNum 373 snap 0 seq_num 373) dirty exists 2024-09-13T10:12:04.865 INFO:tasks.rados.rados.0.smithi195.stdout:1731: left oid 15 (ObjNum 373 snap 0 seq_num 373) 2024-09-13T10:12:04.865 INFO:tasks.rados.rados.0.smithi195.stdout:1735: got (22) Invalid argument 2024-09-13T10:12:04.865 INFO:tasks.rados.rados.0.smithi195.stdout:1732: expect (ObjNum 347 snap 0 seq_num 347) 2024-09-13T10:12:04.895 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:04.889+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:04.901 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:04.901 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:12:04.901 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:12:04.901 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:12:04.901 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:04.916 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-13T10:12:04.996 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:04.991+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:05.036 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:05.036 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:12:05.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:12:05.036 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:12:05.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.037 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:53 4.c 4:3e6706e5:::smithi19530714-14:head [write 618491~694590,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.037 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:50.494212+0000", 2024-09-13T10:12:05.037 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 74.538156505000003, 2024-09-13T10:12:05.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0086614210000000007, 2024-09-13T10:12:05.037 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.037 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.037 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.037 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.037 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.037 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.037 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53 2024-09-13T10:12:05.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.041 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.041 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.041 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494212+0000", 2024-09-13T10:12:05.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.041 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.041 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494212+0000", 2024-09-13T10:12:05.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494213+0000", 2024-09-13T10:12:05.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.84e-06 2024-09-13T10:12:05.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494889+0000", 2024-09-13T10:12:05.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00067531799999999995 2024-09-13T10:12:05.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494907+0000", 2024-09-13T10:12:05.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7884999999999999e-05 2024-09-13T10:12:05.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494917+0000", 2024-09-13T10:12:05.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0043e-05 2024-09-13T10:12:05.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494971+0000", 2024-09-13T10:12:05.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4775000000000003e-05 2024-09-13T10:12:05.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.495081+0000", 2024-09-13T10:12:05.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110055 2024-09-13T10:12:05.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:05.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.495184+0000", 2024-09-13T10:12:05.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010221 2024-09-13T10:12:05.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.044 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.496618+0000", 2024-09-13T10:12:05.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014343590000000001 2024-09-13T10:12:05.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.498806+0000", 2024-09-13T10:12:05.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021880269999999999 2024-09-13T10:12:05.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.502814+0000", 2024-09-13T10:12:05.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040077890000000003 2024-09-13T10:12:05.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.502844+0000", 2024-09-13T10:12:05.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0462000000000001e-05 2024-09-13T10:12:05.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.502873+0000", 2024-09-13T10:12:05.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8657999999999999e-05 2024-09-13T10:12:05.046 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.046 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.046 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.047 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:161 4.d 4:b19b932a:::smithi19530714-41:head [write 672280~791725,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.047 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:50.852563+0000", 2024-09-13T10:12:05.047 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 74.179804537999999, 2024-09-13T10:12:05.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011425286999999999, 2024-09-13T10:12:05.047 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.047 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.047 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.047 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.047 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.047 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.047 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 161 2024-09-13T10:12:05.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.047 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.852563+0000", 2024-09-13T10:12:05.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.852563+0000", 2024-09-13T10:12:05.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.852567+0000", 2024-09-13T10:12:05.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1269999999999999e-06 2024-09-13T10:12:05.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853363+0000", 2024-09-13T10:12:05.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00079620899999999998 2024-09-13T10:12:05.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853394+0000", 2024-09-13T10:12:05.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1192000000000002e-05 2024-09-13T10:12:05.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853410+0000", 2024-09-13T10:12:05.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5896000000000002e-05 2024-09-13T10:12:05.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853574+0000", 2024-09-13T10:12:05.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000164358 2024-09-13T10:12:05.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853920+0000", 2024-09-13T10:12:05.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034612500000000002 2024-09-13T10:12:05.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:12:05.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.854305+0000", 2024-09-13T10:12:05.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038458900000000001 2024-09-13T10:12:05.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.858018+0000", 2024-09-13T10:12:05.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0037128619999999999 2024-09-13T10:12:05.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.051 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.858905+0000", 2024-09-13T10:12:05.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00088737899999999999 2024-09-13T10:12:05.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.863681+0000", 2024-09-13T10:12:05.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004775311 2024-09-13T10:12:05.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.052 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.052 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.052 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.863813+0000", 2024-09-13T10:12:05.052 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013254900000000001 2024-09-13T10:12:05.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.053 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.053 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.863989+0000", 2024-09-13T10:12:05.053 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017568999999999999 2024-09-13T10:12:05.053 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.053 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.053 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.053 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.053 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(osd.14.23:4 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.053 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:50.996997+0000", 2024-09-13T10:12:05.053 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 74.035371131999995, 2024-09-13T10:12:05.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0083262170000000003, 2024-09-13T10:12:05.054 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.054 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.054 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:12:05.054 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.054 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.996997+0000", 2024-09-13T10:12:05.054 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.054 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.054 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.996997+0000", 2024-09-13T10:12:05.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997000+0000", 2024-09-13T10:12:05.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3560000000000001e-06 2024-09-13T10:12:05.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.055 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.055 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.055 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997015+0000", 2024-09-13T10:12:05.055 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.523e-05 2024-09-13T10:12:05.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997048+0000", 2024-09-13T10:12:05.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2162000000000001e-05 2024-09-13T10:12:05.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.056 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997071+0000", 2024-09-13T10:12:05.056 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2918999999999999e-05 2024-09-13T10:12:05.056 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.056 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.056 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.001996+0000", 2024-09-13T10:12:05.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049256559999999996 2024-09-13T10:12:05.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002198+0000", 2024-09-13T10:12:05.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020196600000000001 2024-09-13T10:12:05.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.057 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.057 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.057 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.005323+0000", 2024-09-13T10:12:05.057 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0031249279999999999 2024-09-13T10:12:05.058 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.058 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.058 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.058 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:261 4.c 4:3d56d753:::smithi19530714-12:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.058 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.215641+0000", 2024-09-13T10:12:05.058 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.816726814000006, 2024-09-13T10:12:05.058 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010509116000000001, 2024-09-13T10:12:05.058 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.058 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.058 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.058 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.059 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.059 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.059 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 261 2024-09-13T10:12:05.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.059 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215641+0000", 2024-09-13T10:12:05.059 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.059 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.059 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.059 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.059 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215641+0000", 2024-09-13T10:12:05.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215646+0000", 2024-09-13T10:12:05.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9300000000000002e-06 2024-09-13T10:12:05.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.060 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.060 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.060 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215666+0000", 2024-09-13T10:12:05.060 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0095999999999999e-05 2024-09-13T10:12:05.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215713+0000", 2024-09-13T10:12:05.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7125e-05 2024-09-13T10:12:05.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215739+0000", 2024-09-13T10:12:05.061 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5488999999999999e-05 2024-09-13T10:12:05.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.061 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.061 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.061 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215918+0000", 2024-09-13T10:12:05.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017948399999999999 2024-09-13T10:12:05.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216165+0000", 2024-09-13T10:12:05.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024625200000000003 2024-09-13T10:12:05.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.062 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:05.062 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.223467+0000", 2024-09-13T10:12:05.062 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0073023840000000003 2024-09-13T10:12:05.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.062 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.225020+0000", 2024-09-13T10:12:05.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015529809999999999 2024-09-13T10:12:05.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.225406+0000", 2024-09-13T10:12:05.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038614699999999999 2024-09-13T10:12:05.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.063 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.063 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.063 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.225903+0000", 2024-09-13T10:12:05.063 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00049721900000000002 2024-09-13T10:12:05.063 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.226026+0000", 2024-09-13T10:12:05.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000123163 2024-09-13T10:12:05.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.064 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.064 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.064 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.226150+0000", 2024-09-13T10:12:05.064 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012384599999999999 2024-09-13T10:12:05.064 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.064 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.064 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.065 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.065 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:263 4.c 4:3e6706e5:::smithi19530714-14:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.065 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.215802+0000", 2024-09-13T10:12:05.065 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.816565605999998, 2024-09-13T10:12:05.065 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015708573999999999, 2024-09-13T10:12:05.065 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.065 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.065 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.065 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.065 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.065 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.065 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 263 2024-09-13T10:12:05.065 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.066 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215802+0000", 2024-09-13T10:12:05.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.066 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215802+0000", 2024-09-13T10:12:05.066 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.066 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215804+0000", 2024-09-13T10:12:05.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.624e-06 2024-09-13T10:12:05.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215813+0000", 2024-09-13T10:12:05.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4159999999999993e-06 2024-09-13T10:12:05.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.067 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.067 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.067 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215832+0000", 2024-09-13T10:12:05.067 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8768999999999999e-05 2024-09-13T10:12:05.067 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215843+0000", 2024-09-13T10:12:05.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0336999999999999e-05 2024-09-13T10:12:05.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216612+0000", 2024-09-13T10:12:05.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00076899399999999995 2024-09-13T10:12:05.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.068 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.068 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.068 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216767+0000", 2024-09-13T10:12:05.068 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015566399999999999 2024-09-13T10:12:05.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:05.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.227138+0000", 2024-09-13T10:12:05.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010370701 2024-09-13T10:12:05.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.069 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.229299+0000", 2024-09-13T10:12:05.069 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021611870000000002 2024-09-13T10:12:05.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.069 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.069 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.229896+0000", 2024-09-13T10:12:05.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00059659700000000003 2024-09-13T10:12:05.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231283+0000", 2024-09-13T10:12:05.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001387133 2024-09-13T10:12:05.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.070 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.070 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231392+0000", 2024-09-13T10:12:05.070 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010878999999999999 2024-09-13T10:12:05.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.070 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.071 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.071 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231511+0000", 2024-09-13T10:12:05.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119362 2024-09-13T10:12:05.071 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.071 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.071 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.071 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.071 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:264 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.071 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.215936+0000", 2024-09-13T10:12:05.071 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.816431627, 2024-09-13T10:12:05.071 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054954390999999998, 2024-09-13T10:12:05.071 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.071 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.072 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.072 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 264 2024-09-13T10:12:05.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.072 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.072 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.072 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215936+0000", 2024-09-13T10:12:05.072 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.072 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215936+0000", 2024-09-13T10:12:05.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.073 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215938+0000", 2024-09-13T10:12:05.073 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9860000000000001e-06 2024-09-13T10:12:05.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.073 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215951+0000", 2024-09-13T10:12:05.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2449999999999999e-05 2024-09-13T10:12:05.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215982+0000", 2024-09-13T10:12:05.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1133000000000002e-05 2024-09-13T10:12:05.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.074 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.074 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216009+0000", 2024-09-13T10:12:05.074 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6652000000000002e-05 2024-09-13T10:12:05.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.074 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.217072+0000", 2024-09-13T10:12:05.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010634240000000001 2024-09-13T10:12:05.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.217268+0000", 2024-09-13T10:12:05.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019641 2024-09-13T10:12:05.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:05.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.265415+0000", 2024-09-13T10:12:05.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.048146807999999999 2024-09-13T10:12:05.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.267067+0000", 2024-09-13T10:12:05.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00165211 2024-09-13T10:12:05.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.267319+0000", 2024-09-13T10:12:05.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025160500000000001 2024-09-13T10:12:05.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270591+0000", 2024-09-13T10:12:05.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0032720959999999999 2024-09-13T10:12:05.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270753+0000", 2024-09-13T10:12:05.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016215199999999999 2024-09-13T10:12:05.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270891+0000", 2024-09-13T10:12:05.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000137565 2024-09-13T10:12:05.078 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.078 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.078 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.078 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:278 4.c 4:39f17ee9:::smithi19530714-29:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.078 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.231988+0000", 2024-09-13T10:12:05.078 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.800379761000002, 2024-09-13T10:12:05.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011030108, 2024-09-13T10:12:05.078 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.078 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.078 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.078 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.079 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.079 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.079 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 278 2024-09-13T10:12:05.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.079 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231988+0000", 2024-09-13T10:12:05.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231988+0000", 2024-09-13T10:12:05.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231990+0000", 2024-09-13T10:12:05.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0499999999999999e-06 2024-09-13T10:12:05.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232002+0000", 2024-09-13T10:12:05.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2189e-05 2024-09-13T10:12:05.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232029+0000", 2024-09-13T10:12:05.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6673e-05 2024-09-13T10:12:05.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232043+0000", 2024-09-13T10:12:05.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4188999999999999e-05 2024-09-13T10:12:05.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232239+0000", 2024-09-13T10:12:05.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019557 2024-09-13T10:12:05.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232472+0000", 2024-09-13T10:12:05.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00023310900000000001 2024-09-13T10:12:05.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:05.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.237292+0000", 2024-09-13T10:12:05.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048195069999999998 2024-09-13T10:12:05.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.238595+0000", 2024-09-13T10:12:05.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013033260000000001 2024-09-13T10:12:05.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.239056+0000", 2024-09-13T10:12:05.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00046106800000000002 2024-09-13T10:12:05.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.242820+0000", 2024-09-13T10:12:05.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003763897 2024-09-13T10:12:05.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.242905+0000", 2024-09-13T10:12:05.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5580000000000004e-05 2024-09-13T10:12:05.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.243018+0000", 2024-09-13T10:12:05.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011294999999999999 2024-09-13T10:12:05.084 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.084 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.084 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.085 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:290 4.d 4:b19b932a:::smithi19530714-41:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.085 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.255335+0000", 2024-09-13T10:12:05.085 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.777033239000005, 2024-09-13T10:12:05.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021335703000000001, 2024-09-13T10:12:05.085 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.085 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.085 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.085 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.085 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.085 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.085 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 290 2024-09-13T10:12:05.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.086 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255335+0000", 2024-09-13T10:12:05.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255335+0000", 2024-09-13T10:12:05.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255338+0000", 2024-09-13T10:12:05.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6940000000000001e-06 2024-09-13T10:12:05.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255353+0000", 2024-09-13T10:12:05.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4136e-05 2024-09-13T10:12:05.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255390+0000", 2024-09-13T10:12:05.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7364999999999999e-05 2024-09-13T10:12:05.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255413+0000", 2024-09-13T10:12:05.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2690000000000001e-05 2024-09-13T10:12:05.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255615+0000", 2024-09-13T10:12:05.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000202306 2024-09-13T10:12:05.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255813+0000", 2024-09-13T10:12:05.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019769300000000001 2024-09-13T10:12:05.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:12:05.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274190+0000", 2024-09-13T10:12:05.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01837714 2024-09-13T10:12:05.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275400+0000", 2024-09-13T10:12:05.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001210275 2024-09-13T10:12:05.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276346+0000", 2024-09-13T10:12:05.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00094586699999999998 2024-09-13T10:12:05.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276497+0000", 2024-09-13T10:12:05.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015150299999999999 2024-09-13T10:12:05.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276597+0000", 2024-09-13T10:12:05.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9092999999999995e-05 2024-09-13T10:12:05.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276670+0000", 2024-09-13T10:12:05.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3941000000000002e-05 2024-09-13T10:12:05.091 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.091 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.091 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.091 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:295 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.091 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.261467+0000", 2024-09-13T10:12:05.091 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.770901099, 2024-09-13T10:12:05.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020013564000000001, 2024-09-13T10:12:05.092 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.092 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.092 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.092 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.092 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.092 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.092 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 295 2024-09-13T10:12:05.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.092 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.092 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.092 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261467+0000", 2024-09-13T10:12:05.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261467+0000", 2024-09-13T10:12:05.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261470+0000", 2024-09-13T10:12:05.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8200000000000001e-06 2024-09-13T10:12:05.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261487+0000", 2024-09-13T10:12:05.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7646e-05 2024-09-13T10:12:05.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261523+0000", 2024-09-13T10:12:05.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5299000000000002e-05 2024-09-13T10:12:05.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261545+0000", 2024-09-13T10:12:05.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2436e-05 2024-09-13T10:12:05.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261717+0000", 2024-09-13T10:12:05.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017231000000000001 2024-09-13T10:12:05.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261980+0000", 2024-09-13T10:12:05.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000262959 2024-09-13T10:12:05.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:12:05.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277104+0000", 2024-09-13T10:12:05.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015123983000000001 2024-09-13T10:12:05.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.279042+0000", 2024-09-13T10:12:05.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001937338 2024-09-13T10:12:05.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.279396+0000", 2024-09-13T10:12:05.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00035408199999999998 2024-09-13T10:12:05.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281302+0000", 2024-09-13T10:12:05.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001906346 2024-09-13T10:12:05.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281400+0000", 2024-09-13T10:12:05.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7528000000000005e-05 2024-09-13T10:12:05.097 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281480+0000", 2024-09-13T10:12:05.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0816999999999996e-05 2024-09-13T10:12:05.097 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.098 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.098 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.098 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:297 4.c 4:381e677b:::smithi19530714-48:head [assert-version v15,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.098 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.262493+0000", 2024-09-13T10:12:05.098 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.76987493, 2024-09-13T10:12:05.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012458446, 2024-09-13T10:12:05.098 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.098 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.098 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.099 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.099 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.099 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.099 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 297 2024-09-13T10:12:05.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.099 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262493+0000", 2024-09-13T10:12:05.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262493+0000", 2024-09-13T10:12:05.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262496+0000", 2024-09-13T10:12:05.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4729999999999999e-06 2024-09-13T10:12:05.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262510+0000", 2024-09-13T10:12:05.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4205999999999999e-05 2024-09-13T10:12:05.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262545+0000", 2024-09-13T10:12:05.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4789000000000002e-05 2024-09-13T10:12:05.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.101 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262566+0000", 2024-09-13T10:12:05.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1789000000000001e-05 2024-09-13T10:12:05.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262748+0000", 2024-09-13T10:12:05.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000181848 2024-09-13T10:12:05.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262986+0000", 2024-09-13T10:12:05.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00023747500000000001 2024-09-13T10:12:05.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:05.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.267794+0000", 2024-09-13T10:12:05.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048086479999999996 2024-09-13T10:12:05.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.269878+0000", 2024-09-13T10:12:05.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0020833900000000001 2024-09-13T10:12:05.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270928+0000", 2024-09-13T10:12:05.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001049901 2024-09-13T10:12:05.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274724+0000", 2024-09-13T10:12:05.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0037964880000000002 2024-09-13T10:12:05.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.104 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274862+0000", 2024-09-13T10:12:05.104 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000138286 2024-09-13T10:12:05.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274952+0000", 2024-09-13T10:12:05.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9153000000000005e-05 2024-09-13T10:12:05.105 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.105 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.105 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.105 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:313 4.c 4:3e6706e5:::smithi19530714-14:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.105 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.275533+0000", 2024-09-13T10:12:05.105 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.756834713000003, 2024-09-13T10:12:05.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019582932000000001, 2024-09-13T10:12:05.105 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.106 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.106 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.106 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.106 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.106 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.106 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 313 2024-09-13T10:12:05.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.106 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.106 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275533+0000", 2024-09-13T10:12:05.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275533+0000", 2024-09-13T10:12:05.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275536+0000", 2024-09-13T10:12:05.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4899999999999999e-06 2024-09-13T10:12:05.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275549+0000", 2024-09-13T10:12:05.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3699e-05 2024-09-13T10:12:05.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275574+0000", 2024-09-13T10:12:05.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4586000000000001e-05 2024-09-13T10:12:05.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.108 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275588+0000", 2024-09-13T10:12:05.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3746e-05 2024-09-13T10:12:05.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275705+0000", 2024-09-13T10:12:05.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000117352 2024-09-13T10:12:05.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275898+0000", 2024-09-13T10:12:05.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019293000000000001 2024-09-13T10:12:05.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:05.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286176+0000", 2024-09-13T10:12:05.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010277578000000001 2024-09-13T10:12:05.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.287526+0000", 2024-09-13T10:12:05.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013505349999999999 2024-09-13T10:12:05.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.288025+0000", 2024-09-13T10:12:05.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00049897500000000003 2024-09-13T10:12:05.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294958+0000", 2024-09-13T10:12:05.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069327980000000004 2024-09-13T10:12:05.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.295047+0000", 2024-09-13T10:12:05.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9154999999999996e-05 2024-09-13T10:12:05.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.295116+0000", 2024-09-13T10:12:05.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9088e-05 2024-09-13T10:12:05.112 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.112 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.112 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.112 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:314 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.112 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.276956+0000", 2024-09-13T10:12:05.112 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.755412034000003, 2024-09-13T10:12:05.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017940078000000002, 2024-09-13T10:12:05.113 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.113 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.113 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.113 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.113 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.113 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.113 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 314 2024-09-13T10:12:05.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.113 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276956+0000", 2024-09-13T10:12:05.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276956+0000", 2024-09-13T10:12:05.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276959+0000", 2024-09-13T10:12:05.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5730000000000002e-06 2024-09-13T10:12:05.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276971+0000", 2024-09-13T10:12:05.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2680000000000001e-05 2024-09-13T10:12:05.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276997+0000", 2024-09-13T10:12:05.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5579e-05 2024-09-13T10:12:05.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277018+0000", 2024-09-13T10:12:05.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1409000000000001e-05 2024-09-13T10:12:05.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277136+0000", 2024-09-13T10:12:05.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000117353 2024-09-13T10:12:05.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277237+0000", 2024-09-13T10:12:05.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101412 2024-09-13T10:12:05.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:05.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.285332+0000", 2024-09-13T10:12:05.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080951770000000003 2024-09-13T10:12:05.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.287120+0000", 2024-09-13T10:12:05.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001787828 2024-09-13T10:12:05.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.287200+0000", 2024-09-13T10:12:05.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0279999999999997e-05 2024-09-13T10:12:05.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294808+0000", 2024-09-13T10:12:05.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0076072400000000004 2024-09-13T10:12:05.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294853+0000", 2024-09-13T10:12:05.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5438000000000001e-05 2024-09-13T10:12:05.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294896+0000", 2024-09-13T10:12:05.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3108999999999997e-05 2024-09-13T10:12:05.119 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.119 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.119 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.119 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:340 4.d 4:b19b932a:::smithi19530714-41:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.119 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.284107+0000", 2024-09-13T10:12:05.119 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.748260803999997, 2024-09-13T10:12:05.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010293416, 2024-09-13T10:12:05.120 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.120 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.120 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.120 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.120 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.120 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.120 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 340 2024-09-13T10:12:05.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.120 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284107+0000", 2024-09-13T10:12:05.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284107+0000", 2024-09-13T10:12:05.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284109+0000", 2024-09-13T10:12:05.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.007e-06 2024-09-13T10:12:05.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284120+0000", 2024-09-13T10:12:05.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0346999999999999e-05 2024-09-13T10:12:05.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284144+0000", 2024-09-13T10:12:05.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3998999999999998e-05 2024-09-13T10:12:05.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284157+0000", 2024-09-13T10:12:05.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3045e-05 2024-09-13T10:12:05.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284244+0000", 2024-09-13T10:12:05.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7882999999999999e-05 2024-09-13T10:12:05.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284360+0000", 2024-09-13T10:12:05.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115467 2024-09-13T10:12:05.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:12:05.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.292368+0000", 2024-09-13T10:12:05.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080080380000000003 2024-09-13T10:12:05.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.293486+0000", 2024-09-13T10:12:05.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001117776 2024-09-13T10:12:05.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294261+0000", 2024-09-13T10:12:05.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00077481100000000003 2024-09-13T10:12:05.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294309+0000", 2024-09-13T10:12:05.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8086e-05 2024-09-13T10:12:05.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294358+0000", 2024-09-13T10:12:05.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8853999999999997e-05 2024-09-13T10:12:05.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294401+0000", 2024-09-13T10:12:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3102999999999997e-05 2024-09-13T10:12:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:345 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.286309+0000", 2024-09-13T10:12:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.746058626000007, 2024-09-13T10:12:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010478568000000001, 2024-09-13T10:12:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.126 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 345 2024-09-13T10:12:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286309+0000", 2024-09-13T10:12:05.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286309+0000", 2024-09-13T10:12:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286311+0000", 2024-09-13T10:12:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-13T10:12:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286320+0000", 2024-09-13T10:12:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8370000000000003e-06 2024-09-13T10:12:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286342+0000", 2024-09-13T10:12:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1446000000000001e-05 2024-09-13T10:12:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286385+0000", 2024-09-13T10:12:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3083000000000001e-05 2024-09-13T10:12:05.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286549+0000", 2024-09-13T10:12:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016463 2024-09-13T10:12:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286739+0000", 2024-09-13T10:12:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018965400000000001 2024-09-13T10:12:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:12:05.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294808+0000", 2024-09-13T10:12:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080692609999999994 2024-09-13T10:12:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.295388+0000", 2024-09-13T10:12:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00057994000000000003 2024-09-13T10:12:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296321+0000", 2024-09-13T10:12:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00093279099999999998 2024-09-13T10:12:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296622+0000", 2024-09-13T10:12:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030084300000000002 2024-09-13T10:12:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296702+0000", 2024-09-13T10:12:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0547e-05 2024-09-13T10:12:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296788+0000", 2024-09-13T10:12:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5612999999999995e-05 2024-09-13T10:12:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.132 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:364 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:12:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.297175+0000", 2024-09-13T10:12:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.735193449999997, 2024-09-13T10:12:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0085499329999999991, 2024-09-13T10:12:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 364 2024-09-13T10:12:05.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297175+0000", 2024-09-13T10:12:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297175+0000", 2024-09-13T10:12:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297176+0000", 2024-09-13T10:12:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9199999999999999e-07 2024-09-13T10:12:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297181+0000", 2024-09-13T10:12:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5149999999999997e-06 2024-09-13T10:12:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297191+0000", 2024-09-13T10:12:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8260000000000005e-06 2024-09-13T10:12:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297198+0000", 2024-09-13T10:12:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6950000000000003e-06 2024-09-13T10:12:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297244+0000", 2024-09-13T10:12:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6397999999999999e-05 2024-09-13T10:12:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297291+0000", 2024-09-13T10:12:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7342999999999997e-05 2024-09-13T10:12:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.303658+0000", 2024-09-13T10:12:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0063669 2024-09-13T10:12:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.304437+0000", 2024-09-13T10:12:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00077861399999999996 2024-09-13T10:12:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.304902+0000", 2024-09-13T10:12:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00046505899999999998 2024-09-13T10:12:05.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.305534+0000", 2024-09-13T10:12:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00063224199999999998 2024-09-13T10:12:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.305643+0000", 2024-09-13T10:12:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108384 2024-09-13T10:12:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.305725+0000", 2024-09-13T10:12:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1965000000000006e-05 2024-09-13T10:12:05.138 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:378 4.c 4:39f17ee9:::smithi19530714-29:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:12:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.320710+0000", 2024-09-13T10:12:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.711658079000003, 2024-09-13T10:12:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0079049020000000001, 2024-09-13T10:12:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.139 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 378 2024-09-13T10:12:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320710+0000", 2024-09-13T10:12:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320710+0000", 2024-09-13T10:12:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320712+0000", 2024-09-13T10:12:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4099999999999998e-06 2024-09-13T10:12:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320724+0000", 2024-09-13T10:12:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.134e-05 2024-09-13T10:12:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320746+0000", 2024-09-13T10:12:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2342999999999999e-05 2024-09-13T10:12:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320758+0000", 2024-09-13T10:12:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1557e-05 2024-09-13T10:12:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320840+0000", 2024-09-13T10:12:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2274000000000005e-05 2024-09-13T10:12:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320933+0000", 2024-09-13T10:12:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3375999999999995e-05 2024-09-13T10:12:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.326291+0000", 2024-09-13T10:12:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053575089999999999 2024-09-13T10:12:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.327254+0000", 2024-09-13T10:12:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00096337099999999998 2024-09-13T10:12:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.327902+0000", 2024-09-13T10:12:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00064762100000000003 2024-09-13T10:12:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328501+0000", 2024-09-13T10:12:05.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000599232 2024-09-13T10:12:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328547+0000", 2024-09-13T10:12:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5973000000000002e-05 2024-09-13T10:12:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328615+0000", 2024-09-13T10:12:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7896000000000002e-05 2024-09-13T10:12:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:395 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:12:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.322105+0000", 2024-09-13T10:12:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.710263006999995, 2024-09-13T10:12:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0084827089999999997, 2024-09-13T10:12:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 395 2024-09-13T10:12:05.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322105+0000", 2024-09-13T10:12:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322105+0000", 2024-09-13T10:12:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322106+0000", 2024-09-13T10:12:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2099999999999995e-07 2024-09-13T10:12:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322110+0000", 2024-09-13T10:12:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.827e-06 2024-09-13T10:12:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322120+0000", 2024-09-13T10:12:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0227999999999999e-05 2024-09-13T10:12:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322160+0000", 2024-09-13T10:12:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9941999999999998e-05 2024-09-13T10:12:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322232+0000", 2024-09-13T10:12:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2451000000000001e-05 2024-09-13T10:12:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322314+0000", 2024-09-13T10:12:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2089000000000006e-05 2024-09-13T10:12:05.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:12:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328374+0000", 2024-09-13T10:12:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0060591350000000002 2024-09-13T10:12:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.329439+0000", 2024-09-13T10:12:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010654830000000001 2024-09-13T10:12:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330219+0000", 2024-09-13T10:12:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00078003700000000003 2024-09-13T10:12:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330488+0000", 2024-09-13T10:12:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000269411 2024-09-13T10:12:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330544+0000", 2024-09-13T10:12:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5704000000000001e-05 2024-09-13T10:12:05.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330588+0000", 2024-09-13T10:12:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3581000000000002e-05 2024-09-13T10:12:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:595 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:12:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.370445+0000", 2024-09-13T10:12:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.661922973000003, 2024-09-13T10:12:05.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0084586149999999992, 2024-09-13T10:12:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 595 2024-09-13T10:12:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370445+0000", 2024-09-13T10:12:05.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370445+0000", 2024-09-13T10:12:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370447+0000", 2024-09-13T10:12:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.66e-06 2024-09-13T10:12:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370455+0000", 2024-09-13T10:12:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3059999999999998e-06 2024-09-13T10:12:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370472+0000", 2024-09-13T10:12:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7479999999999999e-05 2024-09-13T10:12:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370484+0000", 2024-09-13T10:12:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1882000000000001e-05 2024-09-13T10:12:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370562+0000", 2024-09-13T10:12:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7531e-05 2024-09-13T10:12:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370738+0000", 2024-09-13T10:12:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017587200000000001 2024-09-13T10:12:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:12:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.377463+0000", 2024-09-13T10:12:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0067247829999999998 2024-09-13T10:12:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378047+0000", 2024-09-13T10:12:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00058493100000000001 2024-09-13T10:12:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378730+0000", 2024-09-13T10:12:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068235799999999999 2024-09-13T10:12:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378840+0000", 2024-09-13T10:12:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110296 2024-09-13T10:12:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378875+0000", 2024-09-13T10:12:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4552999999999999e-05 2024-09-13T10:12:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378904+0000", 2024-09-13T10:12:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8963e-05 2024-09-13T10:12:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:782 4.c 4:39f17ee9:::smithi19530714-29:head [write 1543557~550004] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:12:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.601833+0000", 2024-09-13T10:12:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 73.430535098999997, 2024-09-13T10:12:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081663250000000003, 2024-09-13T10:12:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.159 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 782 2024-09-13T10:12:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.601833+0000", 2024-09-13T10:12:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.601833+0000", 2024-09-13T10:12:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.601835+0000", 2024-09-13T10:12:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.576e-06 2024-09-13T10:12:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602350+0000", 2024-09-13T10:12:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00051449700000000004 2024-09-13T10:12:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602379+0000", 2024-09-13T10:12:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8598999999999998e-05 2024-09-13T10:12:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602394+0000", 2024-09-13T10:12:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4952e-05 2024-09-13T10:12:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602534+0000", 2024-09-13T10:12:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014037899999999999 2024-09-13T10:12:05.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602796+0000", 2024-09-13T10:12:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000262415 2024-09-13T10:12:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.603214+0000", 2024-09-13T10:12:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00041785599999999998 2024-09-13T10:12:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.606389+0000", 2024-09-13T10:12:05.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0031751549999999998 2024-09-13T10:12:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.608306+0000", 2024-09-13T10:12:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0019163540000000001 2024-09-13T10:12:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.609577+0000", 2024-09-13T10:12:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001271466 2024-09-13T10:12:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.609872+0000", 2024-09-13T10:12:05.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00029501400000000002 2024-09-13T10:12:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.609999+0000", 2024-09-13T10:12:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127062 2024-09-13T10:12:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1085 4.c 4:3e6706e5:::smithi19530714-14:head [sparse-read 0~906036] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:56.648267+0000", 2024-09-13T10:12:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.384100575999994, 2024-09-13T10:12:05.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0083262089999999993, 2024-09-13T10:12:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:12:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1085 2024-09-13T10:12:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648267+0000", 2024-09-13T10:12:05.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648267+0000", 2024-09-13T10:12:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648269+0000", 2024-09-13T10:12:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1769999999999999e-06 2024-09-13T10:12:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648276+0000", 2024-09-13T10:12:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0530000000000001e-06 2024-09-13T10:12:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648292+0000", 2024-09-13T10:12:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6209e-05 2024-09-13T10:12:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648307+0000", 2024-09-13T10:12:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4647000000000001e-05 2024-09-13T10:12:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.655806+0000", 2024-09-13T10:12:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0074996500000000001 2024-09-13T10:12:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.656060+0000", 2024-09-13T10:12:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000253639 2024-09-13T10:12:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.656594+0000", 2024-09-13T10:12:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00053383399999999996 2024-09-13T10:12:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.169 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:05.170 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:05.170 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:05.172 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T10:12:05.173 INFO:tasks.rados.rados.0.smithi195.stdout:1724: done (11 left) 2024-09-13T10:12:05.173 INFO:tasks.rados.rados.0.smithi195.stdout:1725: done (10 left) 2024-09-13T10:12:05.173 INFO:tasks.rados.rados.0.smithi195.stdout:1726: done (9 left) 2024-09-13T10:12:05.173 INFO:tasks.rados.rados.0.smithi195.stdout:1727: done (8 left) 2024-09-13T10:12:05.173 INFO:tasks.rados.rados.0.smithi195.stdout:1728: done (7 left) 2024-09-13T10:12:05.173 INFO:tasks.rados.rados.0.smithi195.stdout:1729: done (6 left) 2024-09-13T10:12:05.173 INFO:tasks.rados.rados.0.smithi195.stdout:1730: done (5 left) 2024-09-13T10:12:05.173 INFO:tasks.rados.rados.0.smithi195.stdout:1731: done (4 left) 2024-09-13T10:12:05.173 INFO:tasks.rados.rados.0.smithi195.stdout:1732: done (3 left) 2024-09-13T10:12:05.173 INFO:tasks.rados.rados.0.smithi195.stdout:1734: done (2 left) 2024-09-13T10:12:05.174 INFO:tasks.rados.rados.0.smithi195.stdout:1735: done (1 left) 2024-09-13T10:12:05.174 INFO:tasks.rados.rados.0.smithi195.stdout:1736: write (excl) oid 4 current snap is 0 2024-09-13T10:12:05.174 INFO:tasks.rados.rados.0.smithi195.stdout:1736: seq_num 375 ranges {702625=719672,2127571=175054} 2024-09-13T10:12:05.174 INFO:tasks.rados.rados.0.smithi195.stdout:1736: writing smithi19530714-4 from 702625 to 1422297 tid 1 2024-09-13T10:12:05.174 INFO:tasks.rados.rados.0.smithi195.stdout:1736: writing smithi19530714-4 from 2127571 to 2302625 tid 2 2024-09-13T10:12:05.174 INFO:tasks.rados.rados.0.smithi195.stdout:1737: read oid 20 snap -1 2024-09-13T10:12:05.174 INFO:tasks.rados.rados.0.smithi195.stdout:1737: expect (ObjNum 315 snap 0 seq_num 315) 2024-09-13T10:12:05.174 INFO:tasks.rados.rados.0.smithi195.stdout:1738: read oid 16 snap -1 2024-09-13T10:12:05.174 INFO:tasks.rados.rados.0.smithi195.stdout:1738: expect (ObjNum 372 snap 0 seq_num 372) 2024-09-13T10:12:05.174 INFO:tasks.rados.rados.0.smithi195.stdout:1739: read oid 38 snap -1 2024-09-13T10:12:05.174 INFO:tasks.rados.rados.0.smithi195.stdout:1739: expect (ObjNum 298 snap 0 seq_num 298) 2024-09-13T10:12:05.174 INFO:tasks.rados.rados.0.smithi195.stdout:1740: tier_flush oid 46 2024-09-13T10:12:05.174 INFO:tasks.rados.rados.0.smithi195.stdout:1740: tier_flush oid 46 snap -1 2024-09-13T10:12:05.174 INFO:tasks.rados.rados.0.smithi195.stdout:1733: finishing write tid 1 to smithi19530714-49 2024-09-13T10:12:05.174 INFO:tasks.rados.rados.0.smithi195.stdout:1733: finishing write tid 2 to smithi19530714-49 2024-09-13T10:12:05.175 INFO:tasks.rados.rados.0.smithi195.stdout:1733: finishing write tid 3 to smithi19530714-49 2024-09-13T10:12:05.175 INFO:tasks.rados.rados.0.smithi195.stdout:1733: finishing write tid 4 to smithi19530714-49 2024-09-13T10:12:05.175 INFO:tasks.rados.rados.0.smithi195.stdout:1733: oid 49 updating version 0 to 168 2024-09-13T10:12:05.175 INFO:tasks.rados.rados.0.smithi195.stdout:1733: oid 49 updating version 168 to 169 2024-09-13T10:12:05.175 INFO:tasks.rados.rados.0.smithi195.stdout:1733: oid 49 updating version 169 to 170 2024-09-13T10:12:05.175 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 170 (ObjNum 374 snap 0 seq_num 374) dirty exists 2024-09-13T10:12:05.175 INFO:tasks.rados.rados.0.smithi195.stdout:1733: left oid 49 (ObjNum 374 snap 0 seq_num 374) 2024-09-13T10:12:05.175 INFO:tasks.rados.rados.0.smithi195.stdout:1733: done (5 left) 2024-09-13T10:12:05.175 INFO:tasks.rados.rados.0.smithi195.stdout:1741: delete oid 41 current snap is 0 2024-09-13T10:12:05.175 INFO:tasks.rados.rados.0.smithi195.stdout:1736: finishing write tid 1 to smithi19530714-4 2024-09-13T10:12:05.175 INFO:tasks.rados.rados.0.smithi195.stdout:1736: finishing write tid 2 to smithi19530714-4 2024-09-13T10:12:05.175 INFO:tasks.rados.rados.0.smithi195.stdout:1736: finishing write tid 3 to smithi19530714-4 2024-09-13T10:12:05.175 INFO:tasks.rados.rados.0.smithi195.stdout:1737: expect (ObjNum 315 snap 0 seq_num 315) 2024-09-13T10:12:05.176 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:12:05.167+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:12:05.273 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:05.270+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:05.273 INFO:tasks.rados.rados.0.smithi195.stdout:1736: finishing write tid 4 to smithi19530714-4 2024-09-13T10:12:05.273 INFO:tasks.rados.rados.0.smithi195.stdout:1736: oid 4 updating version 0 to 125 2024-09-13T10:12:05.273 INFO:tasks.rados.rados.0.smithi195.stdout:1736: oid 4 version 125 is already newer than 124 2024-09-13T10:12:05.273 INFO:tasks.rados.rados.0.smithi195.stdout:1736: oid 4 updating version 125 to 126 2024-09-13T10:12:05.273 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 126 (ObjNum 375 snap 0 seq_num 375) dirty exists 2024-09-13T10:12:05.273 INFO:tasks.rados.rados.0.smithi195.stdout:1736: left oid 4 (ObjNum 375 snap 0 seq_num 375) 2024-09-13T10:12:05.274 INFO:tasks.rados.rados.0.smithi195.stdout:1739: expect (ObjNum 298 snap 0 seq_num 298) 2024-09-13T10:12:05.307 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:05.307 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:12:05.307 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:12:05.307 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:05.322 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T10:12:05.374 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:05.370+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:05.443 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:12:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:12:05.444 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:12:05.444 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:05.458 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T10:12:05.474 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:05.470+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:05.522 INFO:tasks.rados.rados.0.smithi195.stdout:1738: expect (ObjNum 372 snap 0 seq_num 372) 2024-09-13T10:12:05.565 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:05.575 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:05.569+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:05.580 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:05.580 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:12:05.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:12:05.580 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:12:05.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.580 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:253 4.7 4:e81d9ad0:::smithi19530714-4:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.134475+0000", 2024-09-13T10:12:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 74.443645790999994, 2024-09-13T10:12:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013417535, 2024-09-13T10:12:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 253 2024-09-13T10:12:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134475+0000", 2024-09-13T10:12:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134475+0000", 2024-09-13T10:12:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134477+0000", 2024-09-13T10:12:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1859999999999999e-06 2024-09-13T10:12:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134486+0000", 2024-09-13T10:12:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3139999999999993e-06 2024-09-13T10:12:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134506+0000", 2024-09-13T10:12:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9165999999999999e-05 2024-09-13T10:12:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134515+0000", 2024-09-13T10:12:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3670000000000003e-06 2024-09-13T10:12:05.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134579+0000", 2024-09-13T10:12:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4200000000000002e-05 2024-09-13T10:12:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134661+0000", 2024-09-13T10:12:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1632999999999996e-05 2024-09-13T10:12:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:05.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.144025+0000", 2024-09-13T10:12:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093644190000000006 2024-09-13T10:12:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.144574+0000", 2024-09-13T10:12:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054873699999999999 2024-09-13T10:12:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.146021+0000", 2024-09-13T10:12:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001446826 2024-09-13T10:12:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.147761+0000", 2024-09-13T10:12:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017406399999999999 2024-09-13T10:12:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.147835+0000", 2024-09-13T10:12:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3708999999999995e-05 2024-09-13T10:12:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.147893+0000", 2024-09-13T10:12:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7337999999999999e-05 2024-09-13T10:12:05.586 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:256 4.7 4:e8c2cbec:::smithi19530714-7:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.214859+0000", 2024-09-13T10:12:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 74.363261433000005, 2024-09-13T10:12:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049512370999999999, 2024-09-13T10:12:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.587 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 256 2024-09-13T10:12:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214859+0000", 2024-09-13T10:12:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214859+0000", 2024-09-13T10:12:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214863+0000", 2024-09-13T10:12:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.343e-06 2024-09-13T10:12:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214876+0000", 2024-09-13T10:12:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.308e-05 2024-09-13T10:12:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214909+0000", 2024-09-13T10:12:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2882999999999997e-05 2024-09-13T10:12:05.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214925+0000", 2024-09-13T10:12:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6002e-05 2024-09-13T10:12:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215088+0000", 2024-09-13T10:12:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016304199999999999 2024-09-13T10:12:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215261+0000", 2024-09-13T10:12:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000173804 2024-09-13T10:12:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258838+0000", 2024-09-13T10:12:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043576675000000002 2024-09-13T10:12:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.260096+0000", 2024-09-13T10:12:05.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012580410000000001 2024-09-13T10:12:05.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.263959+0000", 2024-09-13T10:12:05.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038630840000000001 2024-09-13T10:12:05.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264262+0000", 2024-09-13T10:12:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030279699999999999 2024-09-13T10:12:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264321+0000", 2024-09-13T10:12:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-13T10:12:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264372+0000", 2024-09-13T10:12:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0513000000000002e-05 2024-09-13T10:12:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.593 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:266 4.7 4:e55883d8:::smithi19530714-17:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.215454+0000", 2024-09-13T10:12:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 74.362666771999997, 2024-09-13T10:12:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020880112999999999, 2024-09-13T10:12:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 266 2024-09-13T10:12:05.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215454+0000", 2024-09-13T10:12:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215454+0000", 2024-09-13T10:12:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215457+0000", 2024-09-13T10:12:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6029999999999999e-06 2024-09-13T10:12:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215471+0000", 2024-09-13T10:12:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4629e-05 2024-09-13T10:12:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215503+0000", 2024-09-13T10:12:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1449000000000003e-05 2024-09-13T10:12:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215528+0000", 2024-09-13T10:12:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5273e-05 2024-09-13T10:12:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215632+0000", 2024-09-13T10:12:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010387699999999999 2024-09-13T10:12:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215898+0000", 2024-09-13T10:12:05.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026582100000000002 2024-09-13T10:12:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228364+0000", 2024-09-13T10:12:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012466672 2024-09-13T10:12:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.229463+0000", 2024-09-13T10:12:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001098344 2024-09-13T10:12:05.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.230814+0000", 2024-09-13T10:12:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013513959999999999 2024-09-13T10:12:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236141+0000", 2024-09-13T10:12:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053270160000000004 2024-09-13T10:12:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:05.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236223+0000", 2024-09-13T10:12:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1799999999999996e-05 2024-09-13T10:12:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236334+0000", 2024-09-13T10:12:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111233 2024-09-13T10:12:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:267 4.7 4:ea79a1ec:::smithi19530714-18:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.215583+0000", 2024-09-13T10:12:05.600 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 74.362537799999998, 2024-09-13T10:12:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.051600035000000002, 2024-09-13T10:12:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 267 2024-09-13T10:12:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215583+0000", 2024-09-13T10:12:05.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215583+0000", 2024-09-13T10:12:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215585+0000", 2024-09-13T10:12:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9099999999999999e-06 2024-09-13T10:12:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215594+0000", 2024-09-13T10:12:05.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8799999999999997e-06 2024-09-13T10:12:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215613+0000", 2024-09-13T10:12:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9473000000000001e-05 2024-09-13T10:12:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215630+0000", 2024-09-13T10:12:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6466000000000001e-05 2024-09-13T10:12:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.216360+0000", 2024-09-13T10:12:05.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073001200000000004 2024-09-13T10:12:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.216526+0000", 2024-09-13T10:12:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000166775 2024-09-13T10:12:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.260535+0000", 2024-09-13T10:12:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044008310000000002 2024-09-13T10:12:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:05.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261671+0000", 2024-09-13T10:12:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011362169999999999 2024-09-13T10:12:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.266819+0000", 2024-09-13T10:12:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051479940000000004 2024-09-13T10:12:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.267087+0000", 2024-09-13T10:12:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026847299999999998 2024-09-13T10:12:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.267140+0000", 2024-09-13T10:12:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2054e-05 2024-09-13T10:12:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.267183+0000", 2024-09-13T10:12:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3470999999999998e-05 2024-09-13T10:12:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:303 4.7 4:e81d9ad0:::smithi19530714-4:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:05.606 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.268636+0000", 2024-09-13T10:12:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 74.309485185, 2024-09-13T10:12:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018691609000000001, 2024-09-13T10:12:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 303 2024-09-13T10:12:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268636+0000", 2024-09-13T10:12:05.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268636+0000", 2024-09-13T10:12:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268638+0000", 2024-09-13T10:12:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8990000000000001e-06 2024-09-13T10:12:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268651+0000", 2024-09-13T10:12:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2556e-05 2024-09-13T10:12:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268682+0000", 2024-09-13T10:12:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0979000000000003e-05 2024-09-13T10:12:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268707+0000", 2024-09-13T10:12:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4680000000000001e-05 2024-09-13T10:12:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268787+0000", 2024-09-13T10:12:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9878999999999999e-05 2024-09-13T10:12:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268881+0000", 2024-09-13T10:12:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4242999999999999e-05 2024-09-13T10:12:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:05.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.283577+0000", 2024-09-13T10:12:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014695952999999999 2024-09-13T10:12:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.284599+0000", 2024-09-13T10:12:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001022329 2024-09-13T10:12:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285985+0000", 2024-09-13T10:12:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013861430000000001 2024-09-13T10:12:05.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.287150+0000", 2024-09-13T10:12:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001164382 2024-09-13T10:12:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.287245+0000", 2024-09-13T10:12:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5643999999999994e-05 2024-09-13T10:12:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.287327+0000", 2024-09-13T10:12:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1922000000000006e-05 2024-09-13T10:12:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:567 4.7 4:ea79a1ec:::smithi19530714-18:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:12:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:12:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 74.210820033000005, 2024-09-13T10:12:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012585694999999999, 2024-09-13T10:12:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.613 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 567 2024-09-13T10:12:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:12:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:12:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:12:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1200000000000003e-07 2024-09-13T10:12:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367306+0000", 2024-09-13T10:12:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0309999999999998e-06 2024-09-13T10:12:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367314+0000", 2024-09-13T10:12:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8960000000000003e-06 2024-09-13T10:12:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367321+0000", 2024-09-13T10:12:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4880000000000004e-06 2024-09-13T10:12:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367612+0000", 2024-09-13T10:12:05.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029163099999999999 2024-09-13T10:12:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367715+0000", 2024-09-13T10:12:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102837 2024-09-13T10:12:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.374040+0000", 2024-09-13T10:12:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063245539999999996 2024-09-13T10:12:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.374609+0000", 2024-09-13T10:12:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056937299999999997 2024-09-13T10:12:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379774+0000", 2024-09-13T10:12:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005164737 2024-09-13T10:12:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379810+0000", 2024-09-13T10:12:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6587000000000001e-05 2024-09-13T10:12:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379845+0000", 2024-09-13T10:12:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.468e-05 2024-09-13T10:12:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379886+0000", 2024-09-13T10:12:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1269000000000001e-05 2024-09-13T10:12:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1039 4.17 4:ea79a1ec:::smithi19530714-18:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:05.619 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.096166+0000", 2024-09-13T10:12:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 69.481954775999995, 2024-09-13T10:12:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.051040927, 2024-09-13T10:12:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1039 2024-09-13T10:12:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096166+0000", 2024-09-13T10:12:05.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096166+0000", 2024-09-13T10:12:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096169+0000", 2024-09-13T10:12:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8100000000000002e-06 2024-09-13T10:12:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096180+0000", 2024-09-13T10:12:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1537e-05 2024-09-13T10:12:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096212+0000", 2024-09-13T10:12:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2104999999999998e-05 2024-09-13T10:12:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096268+0000", 2024-09-13T10:12:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5127999999999998e-05 2024-09-13T10:12:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096480+0000", 2024-09-13T10:12:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021249100000000001 2024-09-13T10:12:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096692+0000", 2024-09-13T10:12:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021200400000000001 2024-09-13T10:12:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.147207+0000", 2024-09-13T10:12:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.050514851999999999 2024-09-13T10:12:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.623 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1574 4.7 4:e79f1505:::smithi19530714-32:head [write 1799494~402223] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:12:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.716930+0000", 2024-09-13T10:12:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 60.86119051, 2024-09-13T10:12:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017286329999999999, 2024-09-13T10:12:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.624 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1574 2024-09-13T10:12:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.716930+0000", 2024-09-13T10:12:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.716930+0000", 2024-09-13T10:12:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.716962+0000", 2024-09-13T10:12:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1279999999999999e-05 2024-09-13T10:12:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717453+0000", 2024-09-13T10:12:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049160399999999998 2024-09-13T10:12:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717480+0000", 2024-09-13T10:12:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7313000000000001e-05 2024-09-13T10:12:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717496+0000", 2024-09-13T10:12:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5435999999999999e-05 2024-09-13T10:12:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717592+0000", 2024-09-13T10:12:05.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5851000000000001e-05 2024-09-13T10:12:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717732+0000", 2024-09-13T10:12:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013985000000000001 2024-09-13T10:12:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717962+0000", 2024-09-13T10:12:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000230518 2024-09-13T10:12:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722934+0000", 2024-09-13T10:12:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049717099999999998 2024-09-13T10:12:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724179+0000", 2024-09-13T10:12:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012455389999999999 2024-09-13T10:12:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.733963+0000", 2024-09-13T10:12:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097835349999999995 2024-09-13T10:12:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.734101+0000", 2024-09-13T10:12:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013852 2024-09-13T10:12:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.734217+0000", 2024-09-13T10:12:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011517400000000001 2024-09-13T10:12:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.630 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1575 4.7 4:e79f1505:::smithi19530714-32:head [setxattr _header (56) in=7b,truncate 2201717] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:12:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.717530+0000", 2024-09-13T10:12:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 60.860590354000003, 2024-09-13T10:12:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018674682000000001, 2024-09-13T10:12:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.631 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1575 2024-09-13T10:12:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717530+0000", 2024-09-13T10:12:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717530+0000", 2024-09-13T10:12:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717532+0000", 2024-09-13T10:12:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.813e-06 2024-09-13T10:12:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717542+0000", 2024-09-13T10:12:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4930000000000004e-06 2024-09-13T10:12:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717559+0000", 2024-09-13T10:12:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7337e-05 2024-09-13T10:12:05.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717575+0000", 2024-09-13T10:12:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6385999999999999e-05 2024-09-13T10:12:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.719954+0000", 2024-09-13T10:12:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023782679999999998 2024-09-13T10:12:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.720308+0000", 2024-09-13T10:12:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035397100000000002 2024-09-13T10:12:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.720827+0000", 2024-09-13T10:12:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00051909999999999999 2024-09-13T10:12:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722944+0000", 2024-09-13T10:12:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021173210000000001 2024-09-13T10:12:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724274+0000", 2024-09-13T10:12:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013303099999999999 2024-09-13T10:12:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.735926+0000", 2024-09-13T10:12:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011651846 2024-09-13T10:12:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736061+0000", 2024-09-13T10:12:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013452000000000001 2024-09-13T10:12:05.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736205+0000", 2024-09-13T10:12:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014431700000000001 2024-09-13T10:12:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1576 4.7 4:e79f1505:::smithi19530714-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:12:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.717602+0000", 2024-09-13T10:12:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 60.860519169, 2024-09-13T10:12:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019134776999999999, 2024-09-13T10:12:05.637 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1576 2024-09-13T10:12:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717602+0000", 2024-09-13T10:12:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717602+0000", 2024-09-13T10:12:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717603+0000", 2024-09-13T10:12:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8899999999999999e-06 2024-09-13T10:12:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717610+0000", 2024-09-13T10:12:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8900000000000001e-06 2024-09-13T10:12:05.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717631+0000", 2024-09-13T10:12:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0741999999999999e-05 2024-09-13T10:12:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717644+0000", 2024-09-13T10:12:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2680000000000001e-05 2024-09-13T10:12:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722783+0000", 2024-09-13T10:12:05.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005139429 2024-09-13T10:12:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722912+0000", 2024-09-13T10:12:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012886399999999999 2024-09-13T10:12:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736301+0000", 2024-09-13T10:12:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013389082 2024-09-13T10:12:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736498+0000", 2024-09-13T10:12:05.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000197059 2024-09-13T10:12:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736736+0000", 2024-09-13T10:12:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023814100000000001 2024-09-13T10:12:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1580 4.17 4:e8c2cbec:::smithi19530714-7:head [write 618524~710869,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:12:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.724425+0000", 2024-09-13T10:12:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 60.853695428000002, 2024-09-13T10:12:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013470263999999999, 2024-09-13T10:12:05.642 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1580 2024-09-13T10:12:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724425+0000", 2024-09-13T10:12:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724425+0000", 2024-09-13T10:12:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724427+0000", 2024-09-13T10:12:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9530000000000002e-06 2024-09-13T10:12:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725799+0000", 2024-09-13T10:12:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001371466 2024-09-13T10:12:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725823+0000", 2024-09-13T10:12:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4332999999999999e-05 2024-09-13T10:12:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725844+0000", 2024-09-13T10:12:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0539000000000001e-05 2024-09-13T10:12:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725963+0000", 2024-09-13T10:12:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119524 2024-09-13T10:12:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.726194+0000", 2024-09-13T10:12:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023045800000000001 2024-09-13T10:12:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.726619+0000", 2024-09-13T10:12:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042517199999999997 2024-09-13T10:12:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:05.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.735852+0000", 2024-09-13T10:12:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0092335560000000004 2024-09-13T10:12:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.735959+0000", 2024-09-13T10:12:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106351 2024-09-13T10:12:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.737757+0000", 2024-09-13T10:12:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017985119999999999 2024-09-13T10:12:05.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.737838+0000", 2024-09-13T10:12:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0457999999999994e-05 2024-09-13T10:12:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.737896+0000", 2024-09-13T10:12:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7942000000000002e-05 2024-09-13T10:12:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1855 4.17 4:e81d9ad0:::smithi19530714-4:head [write 739541~709000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.758272+0000", 2024-09-13T10:12:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.819848724000003, 2024-09-13T10:12:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01875696, 2024-09-13T10:12:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1855 2024-09-13T10:12:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.649 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.758272+0000", 2024-09-13T10:12:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.758272+0000", 2024-09-13T10:12:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.758275+0000", 2024-09-13T10:12:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4429999999999999e-06 2024-09-13T10:12:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.759964+0000", 2024-09-13T10:12:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001688557 2024-09-13T10:12:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.759996+0000", 2024-09-13T10:12:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1705999999999998e-05 2024-09-13T10:12:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.760013+0000", 2024-09-13T10:12:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6846000000000001e-05 2024-09-13T10:12:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.760156+0000", 2024-09-13T10:12:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014331700000000001 2024-09-13T10:12:05.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-13T10:12:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.760167+0000", 2024-09-13T10:12:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.062e-05 2024-09-13T10:12:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.763490+0000", 2024-09-13T10:12:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033235309999999998 2024-09-13T10:12:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.763611+0000", 2024-09-13T10:12:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012099699999999999 2024-09-13T10:12:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.763830+0000", 2024-09-13T10:12:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000218519 2024-09-13T10:12:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773195+0000", 2024-09-13T10:12:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093654340000000006 2024-09-13T10:12:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.774701+0000", 2024-09-13T10:12:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00150588 2024-09-13T10:12:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.776817+0000", 2024-09-13T10:12:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021156180000000001 2024-09-13T10:12:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.776910+0000", 2024-09-13T10:12:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3194000000000003e-05 2024-09-13T10:12:05.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.777029+0000", 2024-09-13T10:12:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119298 2024-09-13T10:12:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1856 4.17 4:e81d9ad0:::smithi19530714-4:head [write 2121859~426376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.765194+0000", 2024-09-13T10:12:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.812926931, 2024-09-13T10:12:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013074073, 2024-09-13T10:12:05.656 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1856 2024-09-13T10:12:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765194+0000", 2024-09-13T10:12:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765194+0000", 2024-09-13T10:12:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765196+0000", 2024-09-13T10:12:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.317e-06 2024-09-13T10:12:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765559+0000", 2024-09-13T10:12:05.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036326199999999999 2024-09-13T10:12:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765581+0000", 2024-09-13T10:12:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1676e-05 2024-09-13T10:12:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765595+0000", 2024-09-13T10:12:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4025999999999999e-05 2024-09-13T10:12:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.767613+0000", 2024-09-13T10:12:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00201768 2024-09-13T10:12:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.767751+0000", 2024-09-13T10:12:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013809 2024-09-13T10:12:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.767983+0000", 2024-09-13T10:12:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023206499999999999 2024-09-13T10:12:05.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773203+0000", 2024-09-13T10:12:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052195599999999998 2024-09-13T10:12:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.777481+0000", 2024-09-13T10:12:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042782840000000003 2024-09-13T10:12:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.778120+0000", 2024-09-13T10:12:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063947299999999999 2024-09-13T10:12:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.778203+0000", 2024-09-13T10:12:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2857999999999998e-05 2024-09-13T10:12:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.778268+0000", 2024-09-13T10:12:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4782000000000004e-05 2024-09-13T10:12:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.662 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1857 4.17 4:e81d9ad0:::smithi19530714-4:head [setxattr _header (55) in=7b,truncate 2739541] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.765630+0000", 2024-09-13T10:12:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.812491223999999, 2024-09-13T10:12:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013989379999999999, 2024-09-13T10:12:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1857 2024-09-13T10:12:05.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765630+0000", 2024-09-13T10:12:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765630+0000", 2024-09-13T10:12:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765631+0000", 2024-09-13T10:12:05.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.64e-06 2024-09-13T10:12:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765641+0000", 2024-09-13T10:12:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0047000000000001e-05 2024-09-13T10:12:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765658+0000", 2024-09-13T10:12:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6909e-05 2024-09-13T10:12:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765673+0000", 2024-09-13T10:12:05.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4980000000000001e-05 2024-09-13T10:12:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.771667+0000", 2024-09-13T10:12:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059938209999999999 2024-09-13T10:12:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.771795+0000", 2024-09-13T10:12:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012816999999999999 2024-09-13T10:12:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:05.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.772028+0000", 2024-09-13T10:12:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023327800000000001 2024-09-13T10:12:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773657+0000", 2024-09-13T10:12:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016287039999999999 2024-09-13T10:12:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.777579+0000", 2024-09-13T10:12:05.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039218589999999998 2024-09-13T10:12:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779159+0000", 2024-09-13T10:12:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001580176 2024-09-13T10:12:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779343+0000", 2024-09-13T10:12:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000183769 2024-09-13T10:12:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779619+0000", 2024-09-13T10:12:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027602699999999998 2024-09-13T10:12:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1858 4.17 4:e81d9ad0:::smithi19530714-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.765692+0000", 2024-09-13T10:12:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.812429137999999, 2024-09-13T10:12:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014311129000000001, 2024-09-13T10:12:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.669 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1858 2024-09-13T10:12:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765692+0000", 2024-09-13T10:12:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765692+0000", 2024-09-13T10:12:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765693+0000", 2024-09-13T10:12:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-13T10:12:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765697+0000", 2024-09-13T10:12:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6770000000000003e-06 2024-09-13T10:12:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765707+0000", 2024-09-13T10:12:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0129000000000001e-05 2024-09-13T10:12:05.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765715+0000", 2024-09-13T10:12:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0469999999999994e-06 2024-09-13T10:12:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773032+0000", 2024-09-13T10:12:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073166680000000001 2024-09-13T10:12:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:05.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773169+0000", 2024-09-13T10:12:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136964 2024-09-13T10:12:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779726+0000", 2024-09-13T10:12:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065572499999999997 2024-09-13T10:12:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779851+0000", 2024-09-13T10:12:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012477500000000001 2024-09-13T10:12:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.780003+0000", 2024-09-13T10:12:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015165299999999999 2024-09-13T10:12:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2113 4.17 4:e81d9ad0:::smithi19530714-4:head [write 1410296~761389] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T10:12:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:15.709580+0000", 2024-09-13T10:12:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.868540756000002, 2024-09-13T10:12:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012946568, 2024-09-13T10:12:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.674 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2113 2024-09-13T10:12:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.709580+0000", 2024-09-13T10:12:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.709580+0000", 2024-09-13T10:12:05.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.709582+0000", 2024-09-13T10:12:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4530000000000001e-06 2024-09-13T10:12:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710399+0000", 2024-09-13T10:12:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081639799999999997 2024-09-13T10:12:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710428+0000", 2024-09-13T10:12:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8736000000000001e-05 2024-09-13T10:12:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710443+0000", 2024-09-13T10:12:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5216e-05 2024-09-13T10:12:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710572+0000", 2024-09-13T10:12:05.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012882799999999999 2024-09-13T10:12:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710808+0000", 2024-09-13T10:12:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023585500000000001 2024-09-13T10:12:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.711245+0000", 2024-09-13T10:12:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043798099999999999 2024-09-13T10:12:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.716561+0000", 2024-09-13T10:12:05.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053156760000000001 2024-09-13T10:12:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.718008+0000", 2024-09-13T10:12:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014466380000000001 2024-09-13T10:12:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.722191+0000", 2024-09-13T10:12:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041829229999999998 2024-09-13T10:12:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.722439+0000", 2024-09-13T10:12:05.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024823899999999998 2024-09-13T10:12:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.722527+0000", 2024-09-13T10:12:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7625000000000002e-05 2024-09-13T10:12:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2412 4.7 4:e55883d8:::smithi19530714-17:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e63)", 2024-09-13T10:12:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:22.233147+0000", 2024-09-13T10:12:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.344973789999997, 2024-09-13T10:12:05.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042005107999999999, 2024-09-13T10:12:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2412 2024-09-13T10:12:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233147+0000", 2024-09-13T10:12:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233147+0000", 2024-09-13T10:12:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233150+0000", 2024-09-13T10:12:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0029999999999999e-06 2024-09-13T10:12:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233162+0000", 2024-09-13T10:12:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2449000000000001e-05 2024-09-13T10:12:05.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233195+0000", 2024-09-13T10:12:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2455e-05 2024-09-13T10:12:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233213+0000", 2024-09-13T10:12:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8185999999999998e-05 2024-09-13T10:12:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233351+0000", 2024-09-13T10:12:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000137942 2024-09-13T10:12:05.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233497+0000", 2024-09-13T10:12:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014627100000000001 2024-09-13T10:12:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.275152+0000", 2024-09-13T10:12:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041654801999999998 2024-09-13T10:12:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.684 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2766 4.17 4:e81d9ad0:::smithi19530714-4:head [sparse-read 1791986~224887,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-13T10:12:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:32.706469+0000", 2024-09-13T10:12:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.871651391, 2024-09-13T10:12:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.186036496, 2024-09-13T10:12:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2766 2024-09-13T10:12:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706469+0000", 2024-09-13T10:12:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706469+0000", 2024-09-13T10:12:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706472+0000", 2024-09-13T10:12:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0800000000000002e-06 2024-09-13T10:12:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706483+0000", 2024-09-13T10:12:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0386e-05 2024-09-13T10:12:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706522+0000", 2024-09-13T10:12:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8927999999999997e-05 2024-09-13T10:12:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706544+0000", 2024-09-13T10:12:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2436e-05 2024-09-13T10:12:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706732+0000", 2024-09-13T10:12:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000187715 2024-09-13T10:12:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706767+0000", 2024-09-13T10:12:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5462e-05 2024-09-13T10:12:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.891841+0000", 2024-09-13T10:12:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.185074035 2024-09-13T10:12:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.892071+0000", 2024-09-13T10:12:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000229643 2024-09-13T10:12:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.892506+0000", 2024-09-13T10:12:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043481100000000001 2024-09-13T10:12:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.689 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2767 4.17 4:e81d9ad0:::smithi19530714-4:head [sparse-read 0~1606736] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-13T10:12:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:32.706596+0000", 2024-09-13T10:12:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.871524522999998, 2024-09-13T10:12:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18875979000000001, 2024-09-13T10:12:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2767 2024-09-13T10:12:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706596+0000", 2024-09-13T10:12:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706596+0000", 2024-09-13T10:12:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706598+0000", 2024-09-13T10:12:05.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5659999999999999e-06 2024-09-13T10:12:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706606+0000", 2024-09-13T10:12:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1899999999999995e-06 2024-09-13T10:12:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706624+0000", 2024-09-13T10:12:05.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7546000000000001e-05 2024-09-13T10:12:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706639+0000", 2024-09-13T10:12:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5126000000000001e-05 2024-09-13T10:12:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706840+0000", 2024-09-13T10:12:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020117099999999999 2024-09-13T10:12:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706860+0000", 2024-09-13T10:12:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0205999999999999e-05 2024-09-13T10:12:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.893234+0000", 2024-09-13T10:12:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18637414799999999 2024-09-13T10:12:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.893485+0000", 2024-09-13T10:12:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025044200000000003 2024-09-13T10:12:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.895356+0000", 2024-09-13T10:12:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018713950000000001 2024-09-13T10:12:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2768 4.17 4:e81d9ad0:::smithi19530714-4:head [sparse-read 0~1606736] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-13T10:12:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:32.706665+0000", 2024-09-13T10:12:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.871455789999999, 2024-09-13T10:12:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18904678599999999, 2024-09-13T10:12:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:05.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2768 2024-09-13T10:12:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706665+0000", 2024-09-13T10:12:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:05.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706665+0000", 2024-09-13T10:12:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706666+0000", 2024-09-13T10:12:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.297e-06 2024-09-13T10:12:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706676+0000", 2024-09-13T10:12:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6090000000000007e-06 2024-09-13T10:12:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706692+0000", 2024-09-13T10:12:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5758999999999999e-05 2024-09-13T10:12:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706706+0000", 2024-09-13T10:12:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.482e-05 2024-09-13T10:12:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706925+0000", 2024-09-13T10:12:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000218607 2024-09-13T10:12:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706945+0000", 2024-09-13T10:12:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9466e-05 2024-09-13T10:12:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.895401+0000", 2024-09-13T10:12:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18845690700000001 2024-09-13T10:12:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.895464+0000", 2024-09-13T10:12:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2434999999999994e-05 2024-09-13T10:12:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.895712+0000", 2024-09-13T10:12:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000247886 2024-09-13T10:12:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:05.700 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:05.700 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:05.702 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T10:12:05.703 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:05.695+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:05.703 INFO:tasks.rados.rados.0.smithi195.stdout:1736: done (5 left) 2024-09-13T10:12:05.704 INFO:tasks.rados.rados.0.smithi195.stdout:1737: done (4 left) 2024-09-13T10:12:05.704 INFO:tasks.rados.rados.0.smithi195.stdout:1738: done (3 left) 2024-09-13T10:12:05.704 INFO:tasks.rados.rados.0.smithi195.stdout:1739: done (2 left) 2024-09-13T10:12:05.704 INFO:tasks.rados.rados.0.smithi195.stdout:1741: done (1 left) 2024-09-13T10:12:05.704 INFO:tasks.rados.rados.0.smithi195.stdout:1742: read oid 36 snap -1 2024-09-13T10:12:05.704 INFO:tasks.rados.rados.0.smithi195.stdout:1742: expect deleted 2024-09-13T10:12:05.704 INFO:tasks.rados.rados.0.smithi195.stdout:1743: read oid 7 snap -1 2024-09-13T10:12:05.704 INFO:tasks.rados.rados.0.smithi195.stdout:1743: expect (ObjNum 276 snap 0 seq_num 276) 2024-09-13T10:12:05.704 INFO:tasks.rados.rados.0.smithi195.stdout:1744: tier_promote oid 13 2024-09-13T10:12:05.704 INFO:tasks.rados.rados.0.smithi195.stdout:1745: read oid 45 snap -1 2024-09-13T10:12:05.704 INFO:tasks.rados.rados.0.smithi195.stdout:1745: expect (ObjNum 311 snap 0 seq_num 311) 2024-09-13T10:12:05.704 INFO:tasks.rados.rados.0.smithi195.stdout:1746: tier_evict oid 26 2024-09-13T10:12:05.704 INFO:tasks.rados.rados.0.smithi195.stdout:1746: tier_evict oid 26 snap -1 2024-09-13T10:12:05.704 INFO:tasks.rados.rados.0.smithi195.stdout:1747: read oid 32 snap -1 2024-09-13T10:12:05.705 INFO:tasks.rados.rados.0.smithi195.stdout:1747: expect (ObjNum 320 snap 0 seq_num 320) 2024-09-13T10:12:05.705 INFO:tasks.rados.rados.0.smithi195.stdout:1748: set_chunk oid 47 target oid 2024-09-13T10:12:05.705 INFO:tasks.rados.rados.0.smithi195.stdout:1748: set_chunk oid 47 offset: 925184 length: 1443328 target oid d6f7942767ea5e89c8bb70a587acb9697791df46be7079926fb019db3cf130d7 offset: 0 snap -1 2024-09-13T10:12:05.705 INFO:tasks.rados.rados.0.smithi195.stdout:1749: write (excl) oid 39 current snap is 0 2024-09-13T10:12:05.705 INFO:tasks.rados.rados.0.smithi195.stdout:1749: seq_num 377 ranges {736239=470170,1728182=571782} 2024-09-13T10:12:05.705 INFO:tasks.rados.rados.0.smithi195.stdout:1749: writing smithi19530714-39 from 736239 to 1206409 tid 1 2024-09-13T10:12:05.706 INFO:tasks.rados.rados.0.smithi195.stdout:1749: writing smithi19530714-39 from 1728182 to 2299964 tid 2 2024-09-13T10:12:05.706 INFO:tasks.rados.rados.0.smithi195.stdout:1750: read oid 27 snap -1 2024-09-13T10:12:05.706 INFO:tasks.rados.rados.0.smithi195.stdout:1750: expect (ObjNum 221 snap 0 seq_num 221) 2024-09-13T10:12:05.706 INFO:tasks.rados.rados.0.smithi195.stdout:1751: read oid 23 snap -1 2024-09-13T10:12:05.706 INFO:tasks.rados.rados.0.smithi195.stdout:1751: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:12:05.706 INFO:tasks.rados.rados.0.smithi195.stdout:1752: read oid 41 snap -1 2024-09-13T10:12:05.706 INFO:tasks.rados.rados.0.smithi195.stdout:1752: expect deleted 2024-09-13T10:12:05.706 INFO:tasks.rados.rados.0.smithi195.stdout:1753: read oid 4 snap -1 2024-09-13T10:12:05.706 INFO:tasks.rados.rados.0.smithi195.stdout:1753: expect (ObjNum 375 snap 0 seq_num 375) 2024-09-13T10:12:05.706 INFO:tasks.rados.rados.0.smithi195.stdout:1754: set_chunk oid 16 target oid 2024-09-13T10:12:05.711 INFO:tasks.rados.rados.0.smithi195.stdout:1754: set_chunk oid 16 offset: 2191872 length: 21504 target oid 1a5f375b2a17c236449179a2e0af87dee3f30c4c0d296048a8e4dfd694da8977 offset: 0 snap -1 2024-09-13T10:12:05.711 INFO:tasks.rados.rados.0.smithi195.stdout:1755: set_chunk oid 6 target oid 2024-09-13T10:12:05.785 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-13T10:12:05.802 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:05.797+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:05.807 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:12:05.826 INFO:tasks.rados.rados.0.smithi195.stdout:1755: set_chunk oid 6 offset: 1512448 length: 1358336 target oid 69d093dcf06abe73aae417270401e679f2d8749c7503c646f8965884ab8aa28b offset: 0 snap -1 2024-09-13T10:12:05.826 INFO:tasks.rados.rados.0.smithi195.stdout:1756: delete oid 20 current snap is 0 2024-09-13T10:12:05.826 INFO:tasks.rados.rados.0.smithi195.stdout:1746: got (22) Invalid argument 2024-09-13T10:12:05.826 INFO:tasks.rados.rados.0.smithi195.stdout:1743: expect (ObjNum 276 snap 0 seq_num 276) 2024-09-13T10:12:05.884 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:12:05.885 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:12:05.885 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:05.900 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T10:12:05.903 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:05.897+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:05.959 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1436827 2024-09-13T10:12:05.959 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:12:05.960 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:12:05.979 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:12:05.980 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:12:05.986 INFO:tasks.rados.rados.0.smithi195.stdout:1747: expect (ObjNum 320 snap 0 seq_num 320) 2024-09-13T10:12:05.998 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1106415 2024-09-13T10:12:06.002 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:12:06.002 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:12:06.005 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:06.002+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:06.022 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:06.022 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:12:06.022 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:12:06.022 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:12:06.022 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:06.023 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:12:06.023 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:12:06.037 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T10:12:06.038 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:12:06.039 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:12:06.058 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:12:06.079 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:12:06.079 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:12:06.093 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1509231 2024-09-13T10:12:06.099 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:12:06.100 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:12:06.105 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:06.101+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:06.118 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:12:06.118 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:12:06.128 INFO:tasks.rados.rados.0.smithi195.stdout:1745: expect (ObjNum 311 snap 0 seq_num 311) 2024-09-13T10:12:06.131 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 456482 2024-09-13T10:12:06.136 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:12:06.137 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:12:06.158 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:12:06.158 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:12:06.166 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:06.166 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:06.179 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:12:06.180 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:12:06.181 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-13T10:12:06.197 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:12:06.197 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:12:06.204 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 777085 2024-09-13T10:12:06.215 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:12:06.216 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:12:06.227 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:12:06.223+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:12:06.237 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:12:06.238 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:12:06.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:12:06.230+0000 7fc07bfff640 0 -- 172.21.15.73:0/247627507 >> v2:172.21.15.195:6805/848686366 conn(0x7fc07c2207e0 msgr2=0x7fc07c222bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:12:06.304 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:06.305 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:06.307 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:06.306+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:06.315 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2433097 2024-09-13T10:12:06.318 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-13T10:12:06.329 INFO:tasks.rados.rados.0.smithi195.stdout:1748: finishing set_chunk to oid 47 2024-09-13T10:12:06.329 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 47 set_chunk returned error code -95 offset: 925184 length: 1443328 2024-09-13T10:12:06.330 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 0 (ObjNum 260 snap 0 seq_num 260) dirty exists 2024-09-13T10:12:06.330 INFO:tasks.rados.rados.0.smithi195.stdout:1749: finishing write tid 1 to smithi19530714-39 2024-09-13T10:12:06.330 INFO:tasks.rados.rados.0.smithi195.stdout:1751: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:12:06.358 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1690723 2024-09-13T10:12:06.408 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:06.405+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:06.462 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:12:06.462 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:12:06.481 INFO:tasks.rados.rados.0.smithi195.stdout:1749: finishing write tid 2 to smithi19530714-39 2024-09-13T10:12:06.481 INFO:tasks.rados.rados.0.smithi195.stdout:1749: finishing write tid 3 to smithi19530714-39 2024-09-13T10:12:06.481 INFO:tasks.rados.rados.0.smithi195.stdout:1749: finishing write tid 4 to smithi19530714-39 2024-09-13T10:12:06.481 INFO:tasks.rados.rados.0.smithi195.stdout:1749: oid 39 updating version 0 to 42 2024-09-13T10:12:06.481 INFO:tasks.rados.rados.0.smithi195.stdout:1749: oid 39 updating version 42 to 43 2024-09-13T10:12:06.481 INFO:tasks.rados.rados.0.smithi195.stdout:1749: oid 39 updating version 43 to 44 2024-09-13T10:12:06.481 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 44 (ObjNum 377 snap 0 seq_num 377) dirty exists 2024-09-13T10:12:06.481 INFO:tasks.rados.rados.0.smithi195.stdout:1749: left oid 39 (ObjNum 377 snap 0 seq_num 377) 2024-09-13T10:12:06.481 INFO:tasks.rados.rados.0.smithi195.stdout:1754: finishing set_chunk to oid 16 2024-09-13T10:12:06.481 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 59 (ObjNum 372 snap 0 seq_num 372) dirty exists 2024-09-13T10:12:06.482 INFO:tasks.rados.rados.0.smithi195.stdout:1753: expect (ObjNum 375 snap 0 seq_num 375) 2024-09-13T10:12:06.482 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:06.483 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:06.496 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-13T10:12:06.543 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:06.539+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:06.561 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:06.602 INFO:tasks.rados.rados.0.smithi195.stdout:1750: expect (ObjNum 221 snap 0 seq_num 221) 2024-09-13T10:12:06.602 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:06.611 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:12:06.606+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:06.633 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-13T10:12:06.711 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:06.705+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:06.756 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 661507 2024-09-13T10:12:06.801 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:06.801 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:06.812 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:06.806+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:06.816 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-13T10:12:06.819 INFO:tasks.rados.rados.0.smithi195.stdout:1755: finishing set_chunk to oid 6 2024-09-13T10:12:06.819 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 6 set_chunk returned error code -95 offset: 1512448 length: 1358336 2024-09-13T10:12:06.819 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 6 v 0 (ObjNum 337 snap 0 seq_num 337) dirty exists 2024-09-13T10:12:06.819 INFO:tasks.rados.rados.0.smithi195.stdout:1744: got (0) Success 2024-09-13T10:12:06.819 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 84 (ObjNum 342 snap 0 seq_num 342) dirty exists 2024-09-13T10:12:06.819 INFO:tasks.rados.rados.0.smithi195.stdout:1740: got (0) Success 2024-09-13T10:12:06.819 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 0 (ObjNum 329 snap 0 seq_num 329) dirty exists 2024-09-13T10:12:06.820 INFO:tasks.rados.rados.0.smithi195.stdout:1740: done (15 left) 2024-09-13T10:12:06.820 INFO:tasks.rados.rados.0.smithi195.stdout:1742: done (14 left) 2024-09-13T10:12:06.820 INFO:tasks.rados.rados.0.smithi195.stdout:1743: done (13 left) 2024-09-13T10:12:06.820 INFO:tasks.rados.rados.0.smithi195.stdout:1744: done (12 left) 2024-09-13T10:12:06.820 INFO:tasks.rados.rados.0.smithi195.stdout:1745: done (11 left) 2024-09-13T10:12:06.820 INFO:tasks.rados.rados.0.smithi195.stdout:1746: done (10 left) 2024-09-13T10:12:06.820 INFO:tasks.rados.rados.0.smithi195.stdout:1747: done (9 left) 2024-09-13T10:12:06.820 INFO:tasks.rados.rados.0.smithi195.stdout:1748: done (8 left) 2024-09-13T10:12:06.820 INFO:tasks.rados.rados.0.smithi195.stdout:1749: done (7 left) 2024-09-13T10:12:06.820 INFO:tasks.rados.rados.0.smithi195.stdout:1750: done (6 left) 2024-09-13T10:12:06.820 INFO:tasks.rados.rados.0.smithi195.stdout:1751: done (5 left) 2024-09-13T10:12:06.820 INFO:tasks.rados.rados.0.smithi195.stdout:1752: done (4 left) 2024-09-13T10:12:06.820 INFO:tasks.rados.rados.0.smithi195.stdout:1753: done (3 left) 2024-09-13T10:12:06.821 INFO:tasks.rados.rados.0.smithi195.stdout:1754: done (2 left) 2024-09-13T10:12:06.821 INFO:tasks.rados.rados.0.smithi195.stdout:1755: done (1 left) 2024-09-13T10:12:06.821 INFO:tasks.rados.rados.0.smithi195.stdout:1756: done (0 left) 2024-09-13T10:12:06.821 INFO:tasks.rados.rados.0.smithi195.stdout:1757: read oid 23 snap -1 2024-09-13T10:12:06.821 INFO:tasks.rados.rados.0.smithi195.stdout:1757: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:12:06.821 INFO:tasks.rados.rados.0.smithi195.stdout:1758: tier_promote oid 9 2024-09-13T10:12:06.821 INFO:tasks.rados.rados.0.smithi195.stdout:1759: read oid 49 snap -1 2024-09-13T10:12:06.821 INFO:tasks.rados.rados.0.smithi195.stdout:1759: expect (ObjNum 374 snap 0 seq_num 374) 2024-09-13T10:12:06.821 INFO:tasks.rados.rados.0.smithi195.stdout:1760: write oid 26 current snap is 0 2024-09-13T10:12:06.821 INFO:tasks.rados.rados.0.smithi195.stdout:1760: seq_num 379 ranges {769853=620668,1845146=524707} 2024-09-13T10:12:06.842 INFO:tasks.rados.rados.0.smithi195.stdout:1760: writing smithi19530714-26 from 769853 to 1390521 tid 1 2024-09-13T10:12:06.858 INFO:tasks.rados.rados.0.smithi195.stdout:1760: writing smithi19530714-26 from 1845146 to 2369853 tid 2 2024-09-13T10:12:06.858 INFO:tasks.rados.rados.0.smithi195.stdout:1761: write oid 21 current snap is 0 2024-09-13T10:12:06.858 INFO:tasks.rados.rados.0.smithi195.stdout:1761: seq_num 380 ranges {786660=695917,1903628=435462} 2024-09-13T10:12:06.878 INFO:tasks.rados.rados.0.smithi195.stdout:1761: writing smithi19530714-21 from 786660 to 1482577 tid 1 2024-09-13T10:12:06.887 INFO:tasks.rados.rados.0.smithi195.stdout:1761: writing smithi19530714-21 from 1903628 to 2339090 tid 2 2024-09-13T10:12:06.887 INFO:tasks.rados.rados.0.smithi195.stdout:1762: write (excl) oid 33 current snap is 0 2024-09-13T10:12:06.887 INFO:tasks.rados.rados.0.smithi195.stdout:1762: seq_num 381 ranges {403467=487519,1562110=579120} 2024-09-13T10:12:06.895 INFO:tasks.rados.rados.0.smithi195.stdout:1762: writing smithi19530714-33 from 403467 to 890986 tid 1 2024-09-13T10:12:06.906 INFO:tasks.rados.rados.0.smithi195.stdout:1762: writing smithi19530714-33 from 1562110 to 2141230 tid 2 2024-09-13T10:12:06.906 INFO:tasks.rados.rados.0.smithi195.stdout:1763: set_chunk oid 36 target oid 2024-09-13T10:12:06.906 INFO:tasks.rados.rados.0.smithi195.stdout:1763: set_chunk oid 36 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:12:06.906 INFO:tasks.rados.rados.0.smithi195.stdout:1764: write (excl) oid 40 current snap is 0 2024-09-13T10:12:06.906 INFO:tasks.rados.rados.0.smithi195.stdout:1764: seq_num 382 ranges {420274=562768,1620592=439131} 2024-09-13T10:12:06.913 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:12:06.907+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:12:06.913 INFO:tasks.rados.rados.0.smithi195.stdout:1764: writing smithi19530714-40 from 420274 to 983042 tid 1 2024-09-13T10:12:06.921 INFO:tasks.rados.rados.0.smithi195.stdout:1764: writing smithi19530714-40 from 1620592 to 2059723 tid 2 2024-09-13T10:12:06.921 INFO:tasks.rados.rados.0.smithi195.stdout:1765: write (excl) oid 47 current snap is 0 2024-09-13T10:12:06.921 INFO:tasks.rados.rados.0.smithi195.stdout:1765: seq_num 383 ranges {437081=638017,1679074=582789} 2024-09-13T10:12:06.927 INFO:tasks.rados.rados.0.smithi195.stdout:1765: writing smithi19530714-47 from 437081 to 1075098 tid 1 2024-09-13T10:12:06.935 INFO:tasks.rados.rados.0.smithi195.stdout:1765: writing smithi19530714-47 from 1679074 to 2261863 tid 2 2024-09-13T10:12:06.935 INFO:tasks.rados.rados.0.smithi195.stdout:1766: read oid 6 snap -1 2024-09-13T10:12:06.935 INFO:tasks.rados.rados.0.smithi195.stdout:1766: expect (ObjNum 337 snap 0 seq_num 337) 2024-09-13T10:12:06.935 INFO:tasks.rados.rados.0.smithi195.stdout:1767: tier_promote oid 4 2024-09-13T10:12:06.935 INFO:tasks.rados.rados.0.smithi195.stdout:1768: read oid 44 snap -1 2024-09-13T10:12:06.935 INFO:tasks.rados.rados.0.smithi195.stdout:1768: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:12:06.935 INFO:tasks.rados.rados.0.smithi195.stdout:1769: set_chunk oid 2 target oid 2024-09-13T10:12:06.943 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:06.943 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:06.952 INFO:tasks.rados.rados.0.smithi195.stdout:1769: set_chunk oid 2 offset: 662016 length: 1348608 target oid c69098b4200c8b647b813f281939782551304df32971bec17f8ad567c333ca84 offset: 0 snap -1 2024-09-13T10:12:06.952 INFO:tasks.rados.rados.0.smithi195.stdout:1757: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:12:06.952 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:06.958 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-13T10:12:07.014 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:07.009+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:07.063 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:07.115 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:12:07.110+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:12:07.121 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 323239 2024-09-13T10:12:07.122 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:07.122 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:07.136 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T10:12:07.177 INFO:tasks.rados.rados.0.smithi195.stdout:1759: expect (ObjNum 374 snap 0 seq_num 374) 2024-09-13T10:12:07.202 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:07.216 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:07.209+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:07.232 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 832225 2024-09-13T10:12:07.247 INFO:tasks.rados.rados.0.smithi195.stdout:1760: finishing write tid 1 to smithi19530714-26 2024-09-13T10:12:07.248 INFO:tasks.rados.rados.0.smithi195.stdout:1760: finishing write tid 2 to smithi19530714-26 2024-09-13T10:12:07.248 INFO:tasks.rados.rados.0.smithi195.stdout:1760: finishing write tid 3 to smithi19530714-26 2024-09-13T10:12:07.248 INFO:tasks.rados.rados.0.smithi195.stdout:1760: finishing write tid 4 to smithi19530714-26 2024-09-13T10:12:07.248 INFO:tasks.rados.rados.0.smithi195.stdout:1760: oid 26 updating version 0 to 51 2024-09-13T10:12:07.248 INFO:tasks.rados.rados.0.smithi195.stdout:1760: oid 26 updating version 51 to 52 2024-09-13T10:12:07.248 INFO:tasks.rados.rados.0.smithi195.stdout:1760: oid 26 updating version 52 to 53 2024-09-13T10:12:07.248 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 53 (ObjNum 379 snap 0 seq_num 379) dirty exists 2024-09-13T10:12:07.248 INFO:tasks.rados.rados.0.smithi195.stdout:1760: left oid 26 (ObjNum 379 snap 0 seq_num 379) 2024-09-13T10:12:07.248 INFO:tasks.rados.rados.0.smithi195.stdout:1761: finishing write tid 1 to smithi19530714-21 2024-09-13T10:12:07.248 INFO:tasks.rados.rados.0.smithi195.stdout:1761: finishing write tid 2 to smithi19530714-21 2024-09-13T10:12:07.248 INFO:tasks.rados.rados.0.smithi195.stdout:1761: finishing write tid 3 to smithi19530714-21 2024-09-13T10:12:07.248 INFO:tasks.rados.rados.0.smithi195.stdout:1761: finishing write tid 4 to smithi19530714-21 2024-09-13T10:12:07.249 INFO:tasks.rados.rados.0.smithi195.stdout:1761: oid 21 updating version 0 to 142 2024-09-13T10:12:07.249 INFO:tasks.rados.rados.0.smithi195.stdout:1761: oid 21 updating version 142 to 144 2024-09-13T10:12:07.249 INFO:tasks.rados.rados.0.smithi195.stdout:1761: oid 21 version 144 is already newer than 143 2024-09-13T10:12:07.249 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 144 (ObjNum 380 snap 0 seq_num 380) dirty exists 2024-09-13T10:12:07.249 INFO:tasks.rados.rados.0.smithi195.stdout:1761: left oid 21 (ObjNum 380 snap 0 seq_num 380) 2024-09-13T10:12:07.249 INFO:tasks.rados.rados.0.smithi195.stdout:1762: finishing write tid 1 to smithi19530714-33 2024-09-13T10:12:07.249 INFO:tasks.rados.rados.0.smithi195.stdout:1763: finishing set_chunk to oid 36 2024-09-13T10:12:07.249 INFO:tasks.rados.rados.0.smithi195.stdout:1763: got expected ENOENT (src dne) 2024-09-13T10:12:07.249 INFO:tasks.rados.rados.0.smithi195.stdout:1762: finishing write tid 2 to smithi19530714-33 2024-09-13T10:12:07.249 INFO:tasks.rados.rados.0.smithi195.stdout:1762: finishing write tid 3 to smithi19530714-33 2024-09-13T10:12:07.249 INFO:tasks.rados.rados.0.smithi195.stdout:1762: finishing write tid 4 to smithi19530714-33 2024-09-13T10:12:07.249 INFO:tasks.rados.rados.0.smithi195.stdout:1762: oid 33 updating version 0 to 60 2024-09-13T10:12:07.249 INFO:tasks.rados.rados.0.smithi195.stdout:1762: oid 33 updating version 60 to 62 2024-09-13T10:12:07.250 INFO:tasks.rados.rados.0.smithi195.stdout:1762: oid 33 version 62 is already newer than 61 2024-09-13T10:12:07.250 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 62 (ObjNum 381 snap 0 seq_num 381) dirty exists 2024-09-13T10:12:07.250 INFO:tasks.rados.rados.0.smithi195.stdout:1762: left oid 33 (ObjNum 381 snap 0 seq_num 381) 2024-09-13T10:12:07.250 INFO:tasks.rados.rados.0.smithi195.stdout:1764: finishing write tid 1 to smithi19530714-40 2024-09-13T10:12:07.250 INFO:tasks.rados.rados.0.smithi195.stdout:1764: finishing write tid 2 to smithi19530714-40 2024-09-13T10:12:07.250 INFO:tasks.rados.rados.0.smithi195.stdout:1764: finishing write tid 3 to smithi19530714-40 2024-09-13T10:12:07.250 INFO:tasks.rados.rados.0.smithi195.stdout:1764: finishing write tid 4 to smithi19530714-40 2024-09-13T10:12:07.250 INFO:tasks.rados.rados.0.smithi195.stdout:1764: oid 40 updating version 0 to 30 2024-09-13T10:12:07.250 INFO:tasks.rados.rados.0.smithi195.stdout:1764: oid 40 updating version 30 to 32 2024-09-13T10:12:07.250 INFO:tasks.rados.rados.0.smithi195.stdout:1764: oid 40 version 32 is already newer than 31 2024-09-13T10:12:07.250 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 32 (ObjNum 382 snap 0 seq_num 382) dirty exists 2024-09-13T10:12:07.250 INFO:tasks.rados.rados.0.smithi195.stdout:1764: left oid 40 (ObjNum 382 snap 0 seq_num 382) 2024-09-13T10:12:07.250 INFO:tasks.rados.rados.0.smithi195.stdout:1765: finishing write tid 1 to smithi19530714-47 2024-09-13T10:12:07.251 INFO:tasks.rados.rados.0.smithi195.stdout:1767: got (0) Success 2024-09-13T10:12:07.251 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 126 (ObjNum 375 snap 0 seq_num 375) dirty exists 2024-09-13T10:12:07.251 INFO:tasks.rados.rados.0.smithi195.stdout:1765: finishing write tid 2 to smithi19530714-47 2024-09-13T10:12:07.251 INFO:tasks.rados.rados.0.smithi195.stdout:1765: finishing write tid 3 to smithi19530714-47 2024-09-13T10:12:07.251 INFO:tasks.rados.rados.0.smithi195.stdout:1765: finishing write tid 4 to smithi19530714-47 2024-09-13T10:12:07.251 INFO:tasks.rados.rados.0.smithi195.stdout:1765: oid 47 updating version 0 to 63 2024-09-13T10:12:07.251 INFO:tasks.rados.rados.0.smithi195.stdout:1765: oid 47 updating version 63 to 65 2024-09-13T10:12:07.251 INFO:tasks.rados.rados.0.smithi195.stdout:1765: oid 47 version 65 is already newer than 64 2024-09-13T10:12:07.251 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 47 v 65 (ObjNum 383 snap 0 seq_num 383) dirty exists 2024-09-13T10:12:07.251 INFO:tasks.rados.rados.0.smithi195.stdout:1765: left oid 47 (ObjNum 383 snap 0 seq_num 383) 2024-09-13T10:12:07.251 INFO:tasks.rados.rados.0.smithi195.stdout:1766: expect (ObjNum 337 snap 0 seq_num 337) 2024-09-13T10:12:07.267 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:07.267 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:07.285 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T10:12:07.316 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:07.315+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:07.403 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:07.403 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:07.417 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T10:12:07.417 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:07.414+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:07.481 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3454935 2024-09-13T10:12:07.494 INFO:tasks.rados.rados.0.smithi195.stdout:1769: finishing set_chunk to oid 2 2024-09-13T10:12:07.494 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 2 set_chunk returned error code -95 offset: 662016 length: 1348608 2024-09-13T10:12:07.494 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 0 (ObjNum 280 snap 0 seq_num 280) dirty exists 2024-09-13T10:12:07.494 INFO:tasks.rados.rados.0.smithi195.stdout:1770: read oid 19 snap -1 2024-09-13T10:12:07.494 INFO:tasks.rados.rados.0.smithi195.stdout:1770: expect (ObjNum 346 snap 0 seq_num 346) 2024-09-13T10:12:07.494 INFO:tasks.rados.rados.0.smithi195.stdout:1757: done (13 left) 2024-09-13T10:12:07.494 INFO:tasks.rados.rados.0.smithi195.stdout:1759: done (12 left) 2024-09-13T10:12:07.494 INFO:tasks.rados.rados.0.smithi195.stdout:1760: done (11 left) 2024-09-13T10:12:07.494 INFO:tasks.rados.rados.0.smithi195.stdout:1761: done (10 left) 2024-09-13T10:12:07.495 INFO:tasks.rados.rados.0.smithi195.stdout:1762: done (9 left) 2024-09-13T10:12:07.495 INFO:tasks.rados.rados.0.smithi195.stdout:1763: done (8 left) 2024-09-13T10:12:07.495 INFO:tasks.rados.rados.0.smithi195.stdout:1764: done (7 left) 2024-09-13T10:12:07.495 INFO:tasks.rados.rados.0.smithi195.stdout:1765: done (6 left) 2024-09-13T10:12:07.495 INFO:tasks.rados.rados.0.smithi195.stdout:1766: done (5 left) 2024-09-13T10:12:07.495 INFO:tasks.rados.rados.0.smithi195.stdout:1767: done (4 left) 2024-09-13T10:12:07.495 INFO:tasks.rados.rados.0.smithi195.stdout:1769: done (3 left) 2024-09-13T10:12:07.495 INFO:tasks.rados.rados.0.smithi195.stdout:1771: read oid 29 snap -1 2024-09-13T10:12:07.495 INFO:tasks.rados.rados.0.smithi195.stdout:1771: expect (ObjNum 220 snap 0 seq_num 220) 2024-09-13T10:12:07.495 INFO:tasks.rados.rados.0.smithi195.stdout:1772: set_chunk oid 42 target oid 2024-09-13T10:12:07.509 INFO:tasks.rados.rados.0.smithi195.stdout:1772: set_chunk oid 42 offset: 1648640 length: 841728 target oid daf7c752a3a8485055352d00b78528157d40d06f7f865c1284509b2ca17ce078 offset: 0 snap -1 2024-09-13T10:12:07.509 INFO:tasks.rados.rados.0.smithi195.stdout:1773: tier_flush oid 25 2024-09-13T10:12:07.509 INFO:tasks.rados.rados.0.smithi195.stdout:1773: tier_flush oid 25 snap -1 2024-09-13T10:12:07.509 INFO:tasks.rados.rados.0.smithi195.stdout:1771: expect (ObjNum 220 snap 0 seq_num 220) 2024-09-13T10:12:07.517 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:07.514+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:07.526 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3335605 2024-09-13T10:12:07.530 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:07.575 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 900008 2024-09-13T10:12:07.593 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:07.593 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:07.603 INFO:tasks.rados.rados.0.smithi195.stdout:1770: expect (ObjNum 346 snap 0 seq_num 346) 2024-09-13T10:12:07.603 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:07.607 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T10:12:07.649 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:07.646+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:07.720 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:07.714+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:07.771 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:07.771 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:07.785 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T10:12:07.796 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1493598 2024-09-13T10:12:07.809 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T10:12:07.820 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:07.817+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:07.844 INFO:tasks.rados.rados.0.smithi195.stdout:1772: finishing set_chunk to oid 42 2024-09-13T10:12:07.844 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 26 (ObjNum 343 snap 0 seq_num 343) dirty exists 2024-09-13T10:12:07.844 INFO:tasks.rados.rados.0.smithi195.stdout:1758: got (0) Success 2024-09-13T10:12:07.844 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 123 (ObjNum 259 snap 0 seq_num 259) dirty exists 2024-09-13T10:12:07.844 INFO:tasks.rados.rados.0.smithi195.stdout:1773: got (0) Success 2024-09-13T10:12:07.844 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 0 (ObjNum 324 snap 0 seq_num 324) dirty exists 2024-09-13T10:12:07.844 INFO:tasks.rados.rados.0.smithi195.stdout:1768: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:12:07.855 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2127931 2024-09-13T10:12:07.868 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:07.902 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:07.921 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:07.917+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:07.937 INFO:tasks.rados.rados.0.smithi195.stdout:1758: done (5 left) 2024-09-13T10:12:07.937 INFO:tasks.rados.rados.0.smithi195.stdout:1768: done (4 left) 2024-09-13T10:12:07.938 INFO:tasks.rados.rados.0.smithi195.stdout:1770: done (3 left) 2024-09-13T10:12:07.938 INFO:tasks.rados.rados.0.smithi195.stdout:1771: done (2 left) 2024-09-13T10:12:07.938 INFO:tasks.rados.rados.0.smithi195.stdout:1772: done (1 left) 2024-09-13T10:12:07.938 INFO:tasks.rados.rados.0.smithi195.stdout:1773: done (0 left) 2024-09-13T10:12:07.938 INFO:tasks.rados.rados.0.smithi195.stdout:1774: write oid 49 current snap is 0 2024-09-13T10:12:07.938 INFO:tasks.rados.rados.0.smithi195.stdout:1774: seq_num 384 ranges {453888=713266,1737556=442800} 2024-09-13T10:12:07.953 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:07.953 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:07.956 INFO:tasks.rados.rados.0.smithi195.stdout:1774: writing smithi19530714-49 from 453888 to 1167154 tid 1 2024-09-13T10:12:07.970 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T10:12:07.974 INFO:tasks.rados.rados.0.smithi195.stdout:1774: writing smithi19530714-49 from 1737556 to 2180356 tid 2 2024-09-13T10:12:07.974 INFO:tasks.rados.rados.0.smithi195.stdout:1775: write oid 45 current snap is 0 2024-09-13T10:12:07.975 INFO:tasks.rados.rados.0.smithi195.stdout:1774: finishing write tid 1 to smithi19530714-49 2024-09-13T10:12:07.975 INFO:tasks.rados.rados.0.smithi195.stdout:1775: seq_num 385 ranges {470695=788515,1679685=586458} 2024-09-13T10:12:07.994 INFO:tasks.rados.rados.0.smithi195.stdout:1775: writing smithi19530714-45 from 470695 to 1259210 tid 1 2024-09-13T10:12:08.009 INFO:tasks.rados.rados.0.smithi195.stdout:1775: writing smithi19530714-45 from 1679685 to 2266143 tid 2 2024-09-13T10:12:08.009 INFO:tasks.rados.rados.0.smithi195.stdout:1776: read oid 12 snap -1 2024-09-13T10:12:08.009 INFO:tasks.rados.rados.0.smithi195.stdout:1776: expect (ObjNum 334 snap 0 seq_num 334) 2024-09-13T10:12:08.009 INFO:tasks.rados.rados.0.smithi195.stdout:1777: tier_promote oid 30 2024-09-13T10:12:08.009 INFO:tasks.rados.rados.0.smithi195.stdout:1774: finishing write tid 2 to smithi19530714-49 2024-09-13T10:12:08.010 INFO:tasks.rados.rados.0.smithi195.stdout:1774: finishing write tid 3 to smithi19530714-49 2024-09-13T10:12:08.010 INFO:tasks.rados.rados.0.smithi195.stdout:1774: finishing write tid 4 to smithi19530714-49 2024-09-13T10:12:08.010 INFO:tasks.rados.rados.0.smithi195.stdout:1774: oid 49 updating version 0 to 172 2024-09-13T10:12:08.010 INFO:tasks.rados.rados.0.smithi195.stdout:1774: oid 49 updating version 172 to 174 2024-09-13T10:12:08.010 INFO:tasks.rados.rados.0.smithi195.stdout:1774: oid 49 version 174 is already newer than 173 2024-09-13T10:12:08.010 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 174 (ObjNum 384 snap 0 seq_num 384) dirty exists 2024-09-13T10:12:08.010 INFO:tasks.rados.rados.0.smithi195.stdout:1774: left oid 49 (ObjNum 384 snap 0 seq_num 384) 2024-09-13T10:12:08.010 INFO:tasks.rados.rados.0.smithi195.stdout:1775: finishing write tid 1 to smithi19530714-45 2024-09-13T10:12:08.010 INFO:tasks.rados.rados.0.smithi195.stdout:1774: done (3 left) 2024-09-13T10:12:08.010 INFO:tasks.rados.rados.0.smithi195.stdout:1778: read oid 44 snap -1 2024-09-13T10:12:08.010 INFO:tasks.rados.rados.0.smithi195.stdout:1778: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:12:08.010 INFO:tasks.rados.rados.0.smithi195.stdout:1779: write (excl) oid 31 current snap is 0 2024-09-13T10:12:08.010 INFO:tasks.rados.rados.0.smithi195.stdout:1779: seq_num 386 ranges {487502=463764,1738167=446469} 2024-09-13T10:12:08.022 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:08.017+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:08.023 INFO:tasks.rados.rados.0.smithi195.stdout:1779: writing smithi19530714-31 from 487502 to 951266 tid 1 2024-09-13T10:12:08.038 INFO:tasks.rados.rados.0.smithi195.stdout:1779: writing smithi19530714-31 from 1738167 to 2184636 tid 2 2024-09-13T10:12:08.038 INFO:tasks.rados.rados.0.smithi195.stdout:1780: write (excl) oid 2 current snap is 0 2024-09-13T10:12:08.038 INFO:tasks.rados.rados.0.smithi195.stdout:1780: seq_num 387 ranges {504309=539013,1796649=590127} 2024-09-13T10:12:08.049 INFO:tasks.rados.rados.0.smithi195.stdout:1780: writing smithi19530714-2 from 504309 to 1043322 tid 1 2024-09-13T10:12:08.061 INFO:tasks.rados.rados.0.smithi195.stdout:1780: writing smithi19530714-2 from 1796649 to 2386776 tid 2 2024-09-13T10:12:08.062 INFO:tasks.rados.rados.0.smithi195.stdout:1781: tier_promote oid 9 2024-09-13T10:12:08.062 INFO:tasks.rados.rados.0.smithi195.stdout:1782: read oid 16 snap -1 2024-09-13T10:12:08.062 INFO:tasks.rados.rados.0.smithi195.stdout:1782: expect (ObjNum 372 snap 0 seq_num 372) 2024-09-13T10:12:08.062 INFO:tasks.rados.rados.0.smithi195.stdout:1783: read oid 14 snap -1 2024-09-13T10:12:08.062 INFO:tasks.rados.rados.0.smithi195.stdout:1783: expect (ObjNum 243 snap 0 seq_num 243) 2024-09-13T10:12:08.062 INFO:tasks.rados.rados.0.smithi195.stdout:1784: read oid 23 snap -1 2024-09-13T10:12:08.062 INFO:tasks.rados.rados.0.smithi195.stdout:1784: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:12:08.062 INFO:tasks.rados.rados.0.smithi195.stdout:1785: read oid 39 snap -1 2024-09-13T10:12:08.062 INFO:tasks.rados.rados.0.smithi195.stdout:1785: expect (ObjNum 377 snap 0 seq_num 377) 2024-09-13T10:12:08.062 INFO:tasks.rados.rados.0.smithi195.stdout:1786: read oid 11 snap -1 2024-09-13T10:12:08.062 INFO:tasks.rados.rados.0.smithi195.stdout:1786: expect (ObjNum 359 snap 0 seq_num 359) 2024-09-13T10:12:08.062 INFO:tasks.rados.rados.0.smithi195.stdout:1787: read oid 18 snap -1 2024-09-13T10:12:08.063 INFO:tasks.rados.rados.0.smithi195.stdout:1787: expect (ObjNum 338 snap 0 seq_num 338) 2024-09-13T10:12:08.063 INFO:tasks.rados.rados.0.smithi195.stdout:1788: write (excl) oid 17 current snap is 0 2024-09-13T10:12:08.063 INFO:tasks.rados.rados.0.smithi195.stdout:1788: seq_num 388 ranges {521116=730615,1971484=549632} 2024-09-13T10:12:08.071 INFO:tasks.rados.rados.0.smithi195.stdout:1788: writing smithi19530714-17 from 521116 to 1251731 tid 1 2024-09-13T10:12:08.080 INFO:tasks.rados.rados.0.smithi195.stdout:1788: writing smithi19530714-17 from 1971484 to 2521116 tid 2 2024-09-13T10:12:08.080 INFO:tasks.rados.rados.0.smithi195.stdout:1789: read oid 28 snap -1 2024-09-13T10:12:08.080 INFO:tasks.rados.rados.0.smithi195.stdout:1789: expect (ObjNum 369 snap 0 seq_num 369) 2024-09-13T10:12:08.080 INFO:tasks.rados.rados.0.smithi195.stdout:1790: read oid 15 snap -1 2024-09-13T10:12:08.080 INFO:tasks.rados.rados.0.smithi195.stdout:1790: expect (ObjNum 373 snap 0 seq_num 373) 2024-09-13T10:12:08.080 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:12:08.080 INFO:tasks.rados.rados.0.smithi195.stdout:1777: got (2) No such file or directory 2024-09-13T10:12:08.080 INFO:tasks.rados.rados.0.smithi195.stdout:1777: got expected ENOENT (src dne) 2024-09-13T10:12:08.081 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 42 (ObjNum 96578784 snap 32644 seq_num 304) clean dne 2024-09-13T10:12:08.081 INFO:tasks.rados.rados.0.smithi195.stdout:1778: expect (ObjNum 205 snap 0 seq_num 205) 2024-09-13T10:12:08.101 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2828758 2024-09-13T10:12:08.112 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:08.112 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:08.123 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:08.119+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:08.127 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-13T10:12:08.200 INFO:tasks.rados.rados.0.smithi195.stdout:1775: finishing write tid 2 to smithi19530714-45 2024-09-13T10:12:08.200 INFO:tasks.rados.rados.0.smithi195.stdout:1775: finishing write tid 3 to smithi19530714-45 2024-09-13T10:12:08.200 INFO:tasks.rados.rados.0.smithi195.stdout:1775: finishing write tid 4 to smithi19530714-45 2024-09-13T10:12:08.201 INFO:tasks.rados.rados.0.smithi195.stdout:1775: oid 45 updating version 0 to 175 2024-09-13T10:12:08.201 INFO:tasks.rados.rados.0.smithi195.stdout:1775: oid 45 updating version 175 to 177 2024-09-13T10:12:08.201 INFO:tasks.rados.rados.0.smithi195.stdout:1775: oid 45 version 177 is already newer than 176 2024-09-13T10:12:08.201 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 177 (ObjNum 385 snap 0 seq_num 385) dirty exists 2024-09-13T10:12:08.201 INFO:tasks.rados.rados.0.smithi195.stdout:1775: left oid 45 (ObjNum 385 snap 0 seq_num 385) 2024-09-13T10:12:08.201 INFO:tasks.rados.rados.0.smithi195.stdout:1776: expect (ObjNum 334 snap 0 seq_num 334) 2024-09-13T10:12:08.224 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:12:08.219+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:12:08.227 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2899494 2024-09-13T10:12:08.245 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:08.245 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:08.249 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1334299 2024-09-13T10:12:08.260 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-13T10:12:08.297 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1322229 2024-09-13T10:12:08.325 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:08.321+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:08.338 INFO:tasks.rados.rados.0.smithi195.stdout:1779: finishing write tid 1 to smithi19530714-31 2024-09-13T10:12:08.338 INFO:tasks.rados.rados.0.smithi195.stdout:1779: finishing write tid 2 to smithi19530714-31 2024-09-13T10:12:08.338 INFO:tasks.rados.rados.0.smithi195.stdout:1779: finishing write tid 3 to smithi19530714-31 2024-09-13T10:12:08.339 INFO:tasks.rados.rados.0.smithi195.stdout:1779: finishing write tid 4 to smithi19530714-31 2024-09-13T10:12:08.339 INFO:tasks.rados.rados.0.smithi195.stdout:1779: oid 31 updating version 0 to 107 2024-09-13T10:12:08.339 INFO:tasks.rados.rados.0.smithi195.stdout:1779: oid 31 version 107 is already newer than 105 2024-09-13T10:12:08.339 INFO:tasks.rados.rados.0.smithi195.stdout:1779: oid 31 version 107 is already newer than 106 2024-09-13T10:12:08.339 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 107 (ObjNum 386 snap 0 seq_num 386) dirty exists 2024-09-13T10:12:08.339 INFO:tasks.rados.rados.0.smithi195.stdout:1779: left oid 31 (ObjNum 386 snap 0 seq_num 386) 2024-09-13T10:12:08.339 INFO:tasks.rados.rados.0.smithi195.stdout:1780: finishing write tid 1 to smithi19530714-2 2024-09-13T10:12:08.339 INFO:tasks.rados.rados.0.smithi195.stdout:1784: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:12:08.339 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:08.383 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:08.384 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:08.398 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-13T10:12:08.399 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 435835 2024-09-13T10:12:08.426 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:08.422+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:08.489 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:12:08.509 INFO:tasks.rados.rados.0.smithi195.stdout:1780: finishing write tid 2 to smithi19530714-2 2024-09-13T10:12:08.511 INFO:tasks.rados.rados.0.smithi195.stdout:1780: finishing write tid 3 to smithi19530714-2 2024-09-13T10:12:08.512 INFO:tasks.rados.rados.0.smithi195.stdout:1780: finishing write tid 4 to smithi19530714-2 2024-09-13T10:12:08.512 INFO:tasks.rados.rados.0.smithi195.stdout:1780: oid 2 updating version 0 to 149 2024-09-13T10:12:08.512 INFO:tasks.rados.rados.0.smithi195.stdout:1780: oid 2 version 149 is already newer than 148 2024-09-13T10:12:08.512 INFO:tasks.rados.rados.0.smithi195.stdout:1780: oid 2 version 149 is already newer than 147 2024-09-13T10:12:08.512 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 149 (ObjNum 387 snap 0 seq_num 387) dirty exists 2024-09-13T10:12:08.513 INFO:tasks.rados.rados.0.smithi195.stdout:1780: left oid 2 (ObjNum 387 snap 0 seq_num 387) 2024-09-13T10:12:08.513 INFO:tasks.rados.rados.0.smithi195.stdout:1783: expect (ObjNum 243 snap 0 seq_num 243) 2024-09-13T10:12:08.520 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:08.520 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:08.533 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2025584 2024-09-13T10:12:08.534 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-13T10:12:08.535 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:08.530+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:08.556 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1703086 2024-09-13T10:12:08.560 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:08.594 INFO:tasks.rados.rados.0.smithi195.stdout:1781: got (0) Success 2024-09-13T10:12:08.594 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 123 (ObjNum 259 snap 0 seq_num 259) dirty exists 2024-09-13T10:12:08.595 INFO:tasks.rados.rados.0.smithi195.stdout:1785: expect (ObjNum 377 snap 0 seq_num 377) 2024-09-13T10:12:08.595 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 378303 2024-09-13T10:12:08.627 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:12:08.623+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:12:08.644 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:12:08.644 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:12:08.652 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:08.652 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:08.665 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:12:08.666 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:12:08.667 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-13T10:12:08.684 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:12:08.684 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:12:08.692 INFO:tasks.rados.rados.0.smithi195.stdout:1786: expect (ObjNum 359 snap 0 seq_num 359) 2024-09-13T10:12:08.692 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:08.705 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:12:08.705 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:12:08.712 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:08.724 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:12:08.724 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:12:08.728 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:08.725+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:08.731 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:08.745 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:12:08.745 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:12:08.751 INFO:tasks.rados.rados.0.smithi195.stdout:1788: finishing write tid 1 to smithi19530714-17 2024-09-13T10:12:08.751 INFO:tasks.rados.rados.0.smithi195.stdout:1782: expect (ObjNum 372 snap 0 seq_num 372) 2024-09-13T10:12:08.751 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:08.764 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:12:08.765 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:12:08.786 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:12:08.786 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:12:08.791 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:12:08.791 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:12:08.791 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:12:08.792 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_0 2024-09-13T10:12:08.792 INFO:tasks.ceph.ceph_manager.ceph:contract_pool unique_pool_0 by 10 min 16 2024-09-13T10:12:08.792 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:12:08.793 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:08.793 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:08.804 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:12:08.804 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:12:08.805 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-13T10:12:08.807 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-13T10:12:08.829 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:12:08.829 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:12:08.838 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:08.847 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:12:08.847 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:12:08.856 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:12:08.854+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:12:08.866 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:12:08.866 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:12:08.882 INFO:tasks.rados.rados.0.smithi195.stdout:1787: expect (ObjNum 338 snap 0 seq_num 338) 2024-09-13T10:12:08.882 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:12:08.883 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:12:08.903 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:12:08.903 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:12:08.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:12:08.898+0000 7f582affd640 0 -- 172.21.15.73:0/2969119073 >> v2:172.21.15.181:6804/1173379175 conn(0x7f582c21f880 msgr2=0x7f582c221cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-13T10:12:08.928 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:08.928 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:08.930 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:08.922+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:08.942 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T10:12:08.969 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2274005 2024-09-13T10:12:09.009 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 834895 2024-09-13T10:12:09.032 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:09.030+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:09.063 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:09.064 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:12:09.064 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:12:09.064 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:09.065 INFO:tasks.rados.rados.0.smithi195.stdout:1789: expect (ObjNum 369 snap 0 seq_num 369) 2024-09-13T10:12:09.070 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 626914 2024-09-13T10:12:09.077 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T10:12:09.093 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:12:09.095 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:12:09.111 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-09-13T10:12:08.956959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":231090971,"num_objects":169,"num_object_clones":0,"num_object_copies":507,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":17,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":5123,"num_read_kb":3519189,"num_write":3135,"num_write_kb":706915,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":39659497,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":18,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":117,"acting":117,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":1067716,"kb_used_data":571576,"kb_used_omap":24,"kb_used_meta":486183,"kb_avail":1508881724,"statfs":{"total":1546188226560,"available":1545094885376,"internally_reserved":0,"allocated":585293824,"data_stored":576704548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":497851471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9912651,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":15,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":271,"num_read_kb":192374,"num_write":153,"num_write_kb":28302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":12383428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":-2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001227"},"pg_stats":[{"pgid":"4.19","version":"100'65","reported_seq":326,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.816125+0000","last_change":"2024-09-13T10:11:45.089925+0000","last_active":"2024-09-13T10:12:07.816125+0000","last_peered":"2024-09-13T10:12:07.816125+0000","last_clean":"2024-09-13T10:12:07.816125+0000","last_became_active":"2024-09-13T10:11:45.087574+0000","last_became_peered":"2024-09-13T10:11:45.087574+0000","last_unstale":"2024-09-13T10:12:07.816125+0000","last_undegraded":"2024-09-13T10:12:07.816125+0000","last_fullsized":"2024-09-13T10:12:07.816125+0000","mapping_epoch":82,"log_start":"100'62","ondisk_log_start":"100'62","created":41,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'28","last_scrub_stamp":"2024-09-13T10:11:00.388132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.388132+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":62,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:22.382648+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4840548,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":142,"num_read_kb":115217,"num_write":40,"num_write_kb":10178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2376704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,2],"acting":[3,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"95'30","reported_seq":249,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.811755+0000","last_change":"2024-09-13T10:11:43.078139+0000","last_active":"2024-09-13T10:12:07.811755+0000","last_peered":"2024-09-13T10:12:07.811755+0000","last_clean":"2024-09-13T10:12:07.811755+0000","last_became_active":"2024-09-13T10:11:43.076166+0000","last_became_peered":"2024-09-13T10:11:43.076166+0000","last_unstale":"2024-09-13T10:12:07.811755+0000","last_undegraded":"2024-09-13T10:12:07.811755+0000","last_fullsized":"2024-09-13T10:12:07.811755+0000","mapping_epoch":80,"log_start":"95'27","ondisk_log_start":"95'27","created":41,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'5","last_scrub_stamp":"2024-09-13T10:11:01.783496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.783496+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":27,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:12.427296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2050520,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":85,"num_read_kb":73224,"num_write":28,"num_write_kb":7995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":2436277,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,13],"acting":[10,8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.17","version":"100'127","reported_seq":457,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.834673+0000","last_change":"2024-09-13T10:11:34.723291+0000","last_active":"2024-09-13T10:12:07.834673+0000","last_peered":"2024-09-13T10:12:07.834673+0000","last_clean":"2024-09-13T10:12:07.834673+0000","last_became_active":"2024-09-13T10:11:34.720812+0000","last_became_peered":"2024-09-13T10:11:34.720812+0000","last_unstale":"2024-09-13T10:12:07.834673+0000","last_undegraded":"2024-09-13T10:12:07.834673+0000","last_fullsized":"2024-09-13T10:12:07.834673+0000","mapping_epoch":73,"log_start":"98'124","ondisk_log_start":"98'124","created":41,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'34","last_scrub_stamp":"2024-09-13T10:11:00.442413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.442413+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":124,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:13.441116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8622123,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":226,"num_read_kb":198315,"num_write":118,"num_write_kb":34536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3911454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,1],"acting":[5,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"93'78","reported_seq":344,"reported_epoch":96,"state":"peering","last_fresh":"2024-09-13T10:12:03.112281+0000","last_change":"2024-09-13T10:12:03.776934+0000","last_active":"2024-09-13T10:12:03.776934+0000","last_peered":"2024-09-13T10:12:03.112281+0000","last_clean":"2024-09-13T10:12:03.112281+0000","last_became_active":"2024-09-13T10:11:25.825236+0000","last_became_peered":"2024-09-13T10:11:25.825236+0000","last_unstale":"2024-09-13T10:12:03.112281+0000","last_undegraded":"2024-09-13T10:12:03.112281+0000","last_fullsized":"2024-09-13T10:12:03.112281+0000","mapping_epoch":97,"log_start":"93'75","ondisk_log_start":"93'75","created":41,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":5,"last_scrub":"46'44","last_scrub_stamp":"2024-09-13T10:11:00.456457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.456457+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":72,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7327918,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":141054,"num_write":39,"num_write_kb":10677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3338234,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,5,1],"acting":[14,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,5,13],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.15","version":"101'45","reported_seq":284,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.823300+0000","last_change":"2024-09-13T10:11:23.706313+0000","last_active":"2024-09-13T10:12:07.823300+0000","last_peered":"2024-09-13T10:12:07.823300+0000","last_clean":"2024-09-13T10:12:07.823300+0000","last_became_active":"2024-09-13T10:11:23.705370+0000","last_became_peered":"2024-09-13T10:11:23.705370+0000","last_unstale":"2024-09-13T10:12:07.823300+0000","last_undegraded":"2024-09-13T10:12:07.823300+0000","last_fullsized":"2024-09-13T10:12:07.823300+0000","mapping_epoch":64,"log_start":"87'42","ondisk_log_start":"87'42","created":41,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'36","last_scrub_stamp":"2024-09-13T10:11:05.611551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.611551+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":40,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:14.570465+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":37704,"num_write":6,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1702851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,13],"acting":[6,7,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"39'40","reported_seq":205,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.807641+0000","last_change":"2024-09-13T10:12:00.310899+0000","last_active":"2024-09-13T10:12:07.807641+0000","last_peered":"2024-09-13T10:12:07.807641+0000","last_clean":"2024-09-13T10:12:07.807641+0000","last_became_active":"2024-09-13T10:11:47.715516+0000","last_became_peered":"2024-09-13T10:11:47.715516+0000","last_unstale":"2024-09-13T10:12:07.807641+0000","last_undegraded":"2024-09-13T10:12:07.807641+0000","last_fullsized":"2024-09-13T10:12:07.807641+0000","mapping_epoch":85,"log_start":"39'37","ondisk_log_start":"39'37","created":41,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'40","last_scrub_stamp":"2024-09-13T10:12:00.310748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.310748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":37,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:07.940440+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.13","version":"98'44","reported_seq":287,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.816085+0000","last_change":"2024-09-13T10:11:15.761185+0000","last_active":"2024-09-13T10:12:07.816085+0000","last_peered":"2024-09-13T10:12:07.816085+0000","last_clean":"2024-09-13T10:12:07.816085+0000","last_became_active":"2024-09-13T10:11:15.757364+0000","last_became_peered":"2024-09-13T10:11:15.757364+0000","last_unstale":"2024-09-13T10:12:07.816085+0000","last_undegraded":"2024-09-13T10:12:07.816085+0000","last_fullsized":"2024-09-13T10:12:07.816085+0000","mapping_epoch":58,"log_start":"97'41","ondisk_log_start":"97'41","created":41,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'17","last_scrub_stamp":"2024-09-13T10:11:03.220846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:03.220846+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":39,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:12.668032+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2336239,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":58,"num_read_kb":46084,"num_write":31,"num_write_kb":7734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1493592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"100'53","reported_seq":341,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T10:12:06.864413+0000","last_change":"2024-09-13T10:11:55.737652+0000","last_active":"2024-09-13T10:12:06.864413+0000","last_peered":"2024-09-13T10:12:06.864413+0000","last_clean":"2024-09-13T10:12:06.864413+0000","last_became_active":"2024-09-13T10:11:55.736402+0000","last_became_peered":"2024-09-13T10:11:55.736402+0000","last_unstale":"2024-09-13T10:12:06.864413+0000","last_undegraded":"2024-09-13T10:12:06.864413+0000","last_fullsized":"2024-09-13T10:12:06.864413+0000","mapping_epoch":91,"log_start":"96'50","ondisk_log_start":"96'50","created":41,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'17","last_scrub_stamp":"2024-09-13T10:11:02.582829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:02.582829+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":50,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:09.726149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2369853,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":160,"num_read_kb":132115,"num_write":36,"num_write_kb":10155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":7321937,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,13],"acting":[1,4,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"92'26","reported_seq":254,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:12:05.792309+0000","last_change":"2024-09-13T10:11:40.065958+0000","last_active":"2024-09-13T10:12:05.792309+0000","last_peered":"2024-09-13T10:12:05.792309+0000","last_clean":"2024-09-13T10:12:05.792309+0000","last_became_active":"2024-09-13T10:11:40.064608+0000","last_became_peered":"2024-09-13T10:11:40.064608+0000","last_unstale":"2024-09-13T10:12:05.792309+0000","last_undegraded":"2024-09-13T10:12:05.792309+0000","last_fullsized":"2024-09-13T10:12:05.792309+0000","mapping_epoch":77,"log_start":"92'23","ondisk_log_start":"92'23","created":41,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":5,"last_scrub":"40'13","last_scrub_stamp":"2024-09-13T10:11:04.242723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.242723+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":20,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:11.098336+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3764801,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":66,"num_read_kb":54303,"num_write":16,"num_write_kb":5802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1805029,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,12],"acting":[13,4,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.10","version":"98'60","reported_seq":347,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T10:12:06.790643+0000","last_change":"2024-09-13T10:11:40.068589+0000","last_active":"2024-09-13T10:12:06.790643+0000","last_peered":"2024-09-13T10:12:06.790643+0000","last_clean":"2024-09-13T10:12:06.790643+0000","last_became_active":"2024-09-13T10:11:40.063380+0000","last_became_peered":"2024-09-13T10:11:40.063380+0000","last_unstale":"2024-09-13T10:12:06.790643+0000","last_undegraded":"2024-09-13T10:12:06.790643+0000","last_fullsized":"2024-09-13T10:12:06.790643+0000","mapping_epoch":77,"log_start":"97'57","ondisk_log_start":"97'57","created":41,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'16","last_scrub_stamp":"2024-09-13T10:11:00.706874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.706874+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":55,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:17.975637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5344725,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":169,"num_read_kb":121930,"num_write":51,"num_write_kb":14184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2889991,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,14,3],"acting":[9,14,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":111,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.816728+0000","last_change":"2024-09-13T10:12:03.961952+0000","last_active":"2024-09-13T10:12:07.816728+0000","last_peered":"2024-09-13T10:12:07.816728+0000","last_clean":"2024-09-13T10:12:07.816728+0000","last_became_active":"2024-09-13T10:11:12.744090+0000","last_became_peered":"2024-09-13T10:11:12.744090+0000","last_unstale":"2024-09-13T10:12:07.816728+0000","last_undegraded":"2024-09-13T10:12:07.816728+0000","last_fullsized":"2024-09-13T10:12:07.816728+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:03.961779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:03.961779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:12.481787+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,12],"acting":[3,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"39'12","reported_seq":206,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.822973+0000","last_change":"2024-09-13T10:11:01.537080+0000","last_active":"2024-09-13T10:12:07.822973+0000","last_peered":"2024-09-13T10:12:07.822973+0000","last_clean":"2024-09-13T10:12:07.822973+0000","last_became_active":"2024-09-13T10:10:54.318519+0000","last_became_peered":"2024-09-13T10:10:54.318519+0000","last_unstale":"2024-09-13T10:12:07.822973+0000","last_undegraded":"2024-09-13T10:12:07.822973+0000","last_fullsized":"2024-09-13T10:12:07.822973+0000","mapping_epoch":31,"log_start":"38'9","ondisk_log_start":"38'9","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'12","last_scrub_stamp":"2024-09-13T10:11:01.536963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.536963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":9,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,14],"acting":[6,8,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":167,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.816667+0000","last_change":"2024-09-13T10:10:32.932079+0000","last_active":"2024-09-13T10:12:07.816667+0000","last_peered":"2024-09-13T10:12:07.816667+0000","last_clean":"2024-09-13T10:12:07.816667+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:12:07.816667+0000","last_undegraded":"2024-09-13T10:12:07.816667+0000","last_fullsized":"2024-09-13T10:12:07.816667+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:10:29.101645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"95'96","reported_seq":353,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T10:12:03.968138+0000","last_change":"2024-09-13T10:11:01.219554+0000","last_active":"2024-09-13T10:12:03.968138+0000","last_peered":"2024-09-13T10:12:03.968138+0000","last_clean":"2024-09-13T10:12:03.968138+0000","last_became_active":"2024-09-13T10:10:54.314545+0000","last_became_peered":"2024-09-13T10:10:54.314545+0000","last_unstale":"2024-09-13T10:12:03.968138+0000","last_undegraded":"2024-09-13T10:12:03.968138+0000","last_fullsized":"2024-09-13T10:12:03.968138+0000","mapping_epoch":31,"log_start":"87'93","ondisk_log_start":"87'93","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'27","last_scrub_stamp":"2024-09-13T10:11:01.219486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.219486+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":92,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:20.542560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5327918,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":133,"num_read_kb":109523,"num_write":86,"num_write_kb":23904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,8],"acting":[2,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":173,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.157188+0000","last_change":"2024-09-13T10:12:01.407368+0000","last_active":"2024-09-13T10:12:07.157188+0000","last_peered":"2024-09-13T10:12:07.157188+0000","last_clean":"2024-09-13T10:12:07.157188+0000","last_became_active":"2024-09-13T10:10:30.128658+0000","last_became_peered":"2024-09-13T10:10:30.128658+0000","last_unstale":"2024-09-13T10:12:07.157188+0000","last_undegraded":"2024-09-13T10:12:07.157188+0000","last_fullsized":"2024-09-13T10:12:07.157188+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:01.407274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:01.407274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:11.291191+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.0","version":"100'32","reported_seq":259,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.834591+0000","last_change":"2024-09-13T10:11:05.681812+0000","last_active":"2024-09-13T10:12:07.834591+0000","last_peered":"2024-09-13T10:12:07.834591+0000","last_clean":"2024-09-13T10:12:07.834591+0000","last_became_active":"2024-09-13T10:10:54.320014+0000","last_became_peered":"2024-09-13T10:10:54.320014+0000","last_unstale":"2024-09-13T10:12:07.834591+0000","last_undegraded":"2024-09-13T10:12:07.834591+0000","last_fullsized":"2024-09-13T10:12:07.834591+0000","mapping_epoch":31,"log_start":"95'29","ondisk_log_start":"95'29","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'14","last_scrub_stamp":"2024-09-13T10:11:05.681653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.681653+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":28,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:21.666096+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2420274,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":43458,"num_write":19,"num_write_kb":5241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":19,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.816589+0000","last_change":"2024-09-13T10:12:05.204517+0000","last_active":"2024-09-13T10:12:07.816589+0000","last_peered":"2024-09-13T10:12:07.816589+0000","last_clean":"2024-09-13T10:12:07.816589+0000","last_became_active":"2024-09-13T10:12:05.203738+0000","last_became_peered":"2024-09-13T10:12:05.203738+0000","last_unstale":"2024-09-13T10:12:07.816589+0000","last_undegraded":"2024-09-13T10:12:07.816589+0000","last_fullsized":"2024-09-13T10:12:07.816589+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:11:41.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:41.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:49.027751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12,1],"acting":[3,12,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"96'34","reported_seq":248,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T10:12:03.770603+0000","last_change":"2024-09-13T10:11:02.190704+0000","last_active":"2024-09-13T10:12:03.770603+0000","last_peered":"2024-09-13T10:12:03.770603+0000","last_clean":"2024-09-13T10:12:03.770603+0000","last_became_active":"2024-09-13T10:10:54.317600+0000","last_became_peered":"2024-09-13T10:10:54.317600+0000","last_unstale":"2024-09-13T10:12:03.770603+0000","last_undegraded":"2024-09-13T10:12:03.770603+0000","last_fullsized":"2024-09-13T10:12:03.770603+0000","mapping_epoch":31,"log_start":"94'31","ondisk_log_start":"94'31","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'18","last_scrub_stamp":"2024-09-13T10:11:02.190506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:02.190506+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":31,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:06.358145+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2168169,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":67,"num_read_kb":73516,"num_write":18,"num_write_kb":5584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,12],"acting":[2,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"99'296","reported_seq":501,"reported_epoch":99,"state":"active+remapped+backfilling","last_fresh":"2024-09-13T10:12:05.994373+0000","last_change":"2024-09-13T10:12:05.854310+0000","last_active":"2024-09-13T10:12:05.994373+0000","last_peered":"2024-09-13T10:12:05.994373+0000","last_clean":"2024-09-13T10:12:03.427993+0000","last_became_active":"2024-09-13T10:12:05.846735+0000","last_became_peered":"2024-09-13T10:12:05.846735+0000","last_unstale":"2024-09-13T10:12:05.994373+0000","last_undegraded":"2024-09-13T10:12:05.994373+0000","last_fullsized":"2024-09-13T10:12:05.994373+0000","mapping_epoch":98,"log_start":"99'293","ondisk_log_start":"99'293","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":308,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26621319,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":273,"num_read_kb":4732,"num_write":383,"num_write_kb":73574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":12383428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,10],"acting":[13,14,4],"avail_no_missing":["13","4","14"],"object_location_counts":[{"shards":"4,13,14","objects":28}],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":258,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.815772+0000","last_change":"2024-09-13T10:11:48.946651+0000","last_active":"2024-09-13T10:12:07.815772+0000","last_peered":"2024-09-13T10:12:07.815772+0000","last_clean":"2024-09-13T10:12:07.815772+0000","last_became_active":"2024-09-13T10:11:12.699043+0000","last_became_peered":"2024-09-13T10:11:12.699043+0000","last_unstale":"2024-09-13T10:12:07.815772+0000","last_undegraded":"2024-09-13T10:12:07.815772+0000","last_fullsized":"2024-09-13T10:12:07.815772+0000","mapping_epoch":55,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'112","last_scrub_stamp":"2024-09-13T10:11:48.946478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:48.946478+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:12.399266+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,7],"acting":[3,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":79,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:12:05.792413+0000","last_change":"2024-09-13T10:11:38.808286+0000","last_active":"2024-09-13T10:12:05.792413+0000","last_peered":"2024-09-13T10:12:05.792413+0000","last_clean":"2024-09-13T10:12:05.792413+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:12:05.792413+0000","last_undegraded":"2024-09-13T10:12:05.792413+0000","last_fullsized":"2024-09-13T10:12:05.792413+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T10:11:38.808119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:38.808119+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:43.158000+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.6","version":"95'88","reported_seq":358,"reported_epoch":100,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:12:06.942104+0000","last_change":"2024-09-13T10:12:05.805129+0000","last_active":"2024-09-13T10:12:06.942104+0000","last_peered":"2024-09-13T10:12:06.942104+0000","last_clean":"2024-09-13T10:12:00.406498+0000","last_became_active":"2024-09-13T10:12:05.800127+0000","last_became_peered":"2024-09-13T10:12:05.800127+0000","last_unstale":"2024-09-13T10:12:06.942104+0000","last_undegraded":"2024-09-13T10:12:06.942104+0000","last_fullsized":"2024-09-13T10:12:06.942104+0000","mapping_epoch":98,"log_start":"94'85","ondisk_log_start":"94'85","created":31,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'44","last_scrub_stamp":"2024-09-13T10:11:05.396838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.396838+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":85,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5680865,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":114016,"num_write":51,"num_write_kb":15352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,4],"acting":[1,7,13],"avail_no_missing":["1","7","13"],"object_location_counts":[{"shards":"1,7,13","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"101'337","reported_seq":516,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.837538+0000","last_change":"2024-09-13T10:12:00.967692+0000","last_active":"2024-09-13T10:12:07.837538+0000","last_peered":"2024-09-13T10:12:07.837538+0000","last_clean":"2024-09-13T10:12:07.837538+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:12:07.837538+0000","last_undegraded":"2024-09-13T10:12:07.837538+0000","last_fullsized":"2024-09-13T10:12:07.837538+0000","mapping_epoch":30,"log_start":"100'334","ondisk_log_start":"100'334","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'314","last_scrub_stamp":"2024-09-13T10:12:00.967585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.967585+0000","objects_scrubbed":33,"log_size":3,"log_dups_size":346,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:19.050419+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31681660,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":310,"num_read_kb":12010,"num_write":446,"num_write_kb":92707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":171,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T10:12:03.770620+0000","last_change":"2024-09-13T10:11:30.291380+0000","last_active":"2024-09-13T10:12:03.770620+0000","last_peered":"2024-09-13T10:12:03.770620+0000","last_clean":"2024-09-13T10:12:03.770620+0000","last_became_active":"2024-09-13T10:10:30.127232+0000","last_became_peered":"2024-09-13T10:10:30.127232+0000","last_unstale":"2024-09-13T10:12:03.770620+0000","last_undegraded":"2024-09-13T10:12:03.770620+0000","last_fullsized":"2024-09-13T10:12:03.770620+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-13T10:11:30.291157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:30.291157+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:31.417661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0034622780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"93'96","reported_seq":365,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T10:12:06.845478+0000","last_change":"2024-09-13T10:11:05.436449+0000","last_active":"2024-09-13T10:12:06.845478+0000","last_peered":"2024-09-13T10:12:06.845478+0000","last_clean":"2024-09-13T10:12:06.845478+0000","last_became_active":"2024-09-13T10:10:54.314465+0000","last_became_peered":"2024-09-13T10:10:54.314465+0000","last_unstale":"2024-09-13T10:12:06.845478+0000","last_undegraded":"2024-09-13T10:12:06.845478+0000","last_fullsized":"2024-09-13T10:12:06.845478+0000","mapping_epoch":31,"log_start":"93'93","ondisk_log_start":"93'93","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'40","last_scrub_stamp":"2024-09-13T10:11:05.436272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.436272+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":90,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:08.571155+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7210269,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":107174,"num_write":66,"num_write_kb":19748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,10],"acting":[1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"101'315","reported_seq":483,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.812001+0000","last_change":"2024-09-13T10:12:02.384819+0000","last_active":"2024-09-13T10:12:07.812001+0000","last_peered":"2024-09-13T10:12:07.812001+0000","last_clean":"2024-09-13T10:12:07.812001+0000","last_became_active":"2024-09-13T10:10:43.220074+0000","last_became_peered":"2024-09-13T10:10:43.220074+0000","last_unstale":"2024-09-13T10:12:07.812001+0000","last_undegraded":"2024-09-13T10:12:07.812001+0000","last_fullsized":"2024-09-13T10:12:07.812001+0000","mapping_epoch":30,"log_start":"100'312","ondisk_log_start":"100'312","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'292","last_scrub_stamp":"2024-09-13T10:12:02.384617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:02.384617+0000","objects_scrubbed":28,"log_size":3,"log_dups_size":324,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:26.698118+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23871252,"num_objects":32,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":283,"num_read_kb":4417,"num_write":414,"num_write_kb":77058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.157098+0000","last_change":"2024-09-13T10:11:53.335859+0000","last_active":"2024-09-13T10:12:07.157098+0000","last_peered":"2024-09-13T10:12:07.157098+0000","last_clean":"2024-09-13T10:12:07.157098+0000","last_became_active":"2024-09-13T10:11:39.046090+0000","last_became_peered":"2024-09-13T10:11:39.046090+0000","last_unstale":"2024-09-13T10:12:07.157098+0000","last_undegraded":"2024-09-13T10:12:07.157098+0000","last_fullsized":"2024-09-13T10:12:07.157098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:11:53.335756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:53.335756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:17.649356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,7,9],"acting":[15,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.4","version":"101'177","reported_seq":629,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:08.085865+0000","last_change":"2024-09-13T10:11:03.633738+0000","last_active":"2024-09-13T10:12:08.085865+0000","last_peered":"2024-09-13T10:12:08.085865+0000","last_clean":"2024-09-13T10:12:08.085865+0000","last_became_active":"2024-09-13T10:10:54.316518+0000","last_became_peered":"2024-09-13T10:10:54.316518+0000","last_unstale":"2024-09-13T10:12:08.085865+0000","last_undegraded":"2024-09-13T10:12:08.085865+0000","last_fullsized":"2024-09-13T10:12:08.085865+0000","mapping_epoch":31,"log_start":"101'174","ondisk_log_start":"101'174","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'66","last_scrub_stamp":"2024-09-13T10:11:03.633499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:03.633499+0000","objects_scrubbed":5,"log_size":3,"log_dups_size":174,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:14.513606+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15798910,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":505,"num_read_kb":477122,"num_write":124,"num_write_kb":34713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10],"acting":[6,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"95'361","reported_seq":527,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T10:12:03.775962+0000","last_change":"2024-09-13T10:12:00.627361+0000","last_active":"2024-09-13T10:12:03.775962+0000","last_peered":"2024-09-13T10:12:03.775962+0000","last_clean":"2024-09-13T10:12:03.775962+0000","last_became_active":"2024-09-13T10:10:43.229961+0000","last_became_peered":"2024-09-13T10:10:43.229961+0000","last_unstale":"2024-09-13T10:12:03.775962+0000","last_undegraded":"2024-09-13T10:12:03.775962+0000","last_fullsized":"2024-09-13T10:12:03.775962+0000","mapping_epoch":30,"log_start":"94'358","ondisk_log_start":"94'358","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'359","last_scrub_stamp":"2024-09-13T10:12:00.627252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.627252+0000","objects_scrubbed":25,"log_size":3,"log_dups_size":367,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:24.042469+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18725530,"num_objects":23,"num_object_clones":0,"num_object_copies":69,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":330,"num_read_kb":5913,"num_write":446,"num_write_kb":88658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":176,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.823101+0000","last_change":"2024-09-13T10:11:51.041068+0000","last_active":"2024-09-13T10:12:07.823101+0000","last_peered":"2024-09-13T10:12:07.823101+0000","last_clean":"2024-09-13T10:12:07.823101+0000","last_became_active":"2024-09-13T10:10:30.272007+0000","last_became_peered":"2024-09-13T10:10:30.272007+0000","last_unstale":"2024-09-13T10:12:07.823101+0000","last_undegraded":"2024-09-13T10:12:07.823101+0000","last_fullsized":"2024-09-13T10:12:07.823101+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:11:51.040975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:51.040975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:13.227058+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.000987604,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"101'151","reported_seq":551,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:08.079909+0000","last_change":"2024-09-13T10:11:00.541038+0000","last_active":"2024-09-13T10:12:08.079909+0000","last_peered":"2024-09-13T10:12:08.079909+0000","last_clean":"2024-09-13T10:12:08.079909+0000","last_became_active":"2024-09-13T10:10:54.315196+0000","last_became_peered":"2024-09-13T10:10:54.315196+0000","last_unstale":"2024-09-13T10:12:08.079909+0000","last_undegraded":"2024-09-13T10:12:08.079909+0000","last_fullsized":"2024-09-13T10:12:08.079909+0000","mapping_epoch":31,"log_start":"101'148","ondisk_log_start":"101'148","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'48","last_scrub_stamp":"2024-09-13T10:11:00.540905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.540905+0000","objects_scrubbed":5,"log_size":3,"log_dups_size":147,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:12.038831+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14689516,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":384,"num_read_kb":353516,"num_write":121,"num_write_kb":38068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,0],"acting":[6,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"39'5","reported_seq":189,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.815455+0000","last_change":"2024-09-13T10:11:00.786261+0000","last_active":"2024-09-13T10:12:07.815455+0000","last_peered":"2024-09-13T10:12:07.815455+0000","last_clean":"2024-09-13T10:12:07.815455+0000","last_became_active":"2024-09-13T10:10:54.314935+0000","last_became_peered":"2024-09-13T10:10:54.314935+0000","last_unstale":"2024-09-13T10:12:07.815455+0000","last_undegraded":"2024-09-13T10:12:07.815455+0000","last_fullsized":"2024-09-13T10:12:07.815455+0000","mapping_epoch":31,"log_start":"38'2","ondisk_log_start":"38'2","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'5","last_scrub_stamp":"2024-09-13T10:11:00.786110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.786110+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,0],"acting":[3,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"94'43","reported_seq":280,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.807613+0000","last_change":"2024-09-13T10:11:04.446852+0000","last_active":"2024-09-13T10:12:07.807613+0000","last_peered":"2024-09-13T10:12:07.807613+0000","last_clean":"2024-09-13T10:12:07.807613+0000","last_became_active":"2024-09-13T10:10:54.316057+0000","last_became_peered":"2024-09-13T10:10:54.316057+0000","last_unstale":"2024-09-13T10:12:07.807613+0000","last_undegraded":"2024-09-13T10:12:07.807613+0000","last_fullsized":"2024-09-13T10:12:07.807613+0000","mapping_epoch":31,"log_start":"79'40","ondisk_log_start":"79'40","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'29","last_scrub_stamp":"2024-09-13T10:11:04.446717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.446717+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":37,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:17.702973+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2806802,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":102,"num_read_kb":85289,"num_write":13,"num_write_kb":3657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,9],"acting":[11,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.a","version":"96'25","reported_seq":222,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.822900+0000","last_change":"2024-09-13T10:11:12.708135+0000","last_active":"2024-09-13T10:12:07.822900+0000","last_peered":"2024-09-13T10:12:07.822900+0000","last_clean":"2024-09-13T10:12:07.822900+0000","last_became_active":"2024-09-13T10:11:12.707306+0000","last_became_peered":"2024-09-13T10:11:12.707306+0000","last_unstale":"2024-09-13T10:12:07.822900+0000","last_undegraded":"2024-09-13T10:12:07.822900+0000","last_fullsized":"2024-09-13T10:12:07.822900+0000","mapping_epoch":55,"log_start":"69'22","ondisk_log_start":"69'22","created":31,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":22,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2184976,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":52,"num_read_kb":54208,"num_write":25,"num_write_kb":8053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,13],"acting":[6,8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"92'84","reported_seq":361,"reported_epoch":97,"state":"unknown","last_fresh":"2024-09-13T10:12:03.771944+0000","last_change":"2024-09-13T10:12:03.980424+0000","last_active":"2024-09-13T10:12:03.771944+0000","last_peered":"2024-09-13T10:11:56.004145+0000","last_clean":"2024-09-13T10:11:56.004145+0000","last_became_active":"2024-09-13T10:10:44.226956+0000","last_became_peered":"2024-09-13T10:10:44.226956+0000","last_unstale":"2024-09-13T10:12:03.771944+0000","last_undegraded":"2024-09-13T10:12:03.771944+0000","last_fullsized":"2024-09-13T10:12:03.771944+0000","mapping_epoch":97,"log_start":"92'81","ondisk_log_start":"92'81","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":81,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7193429,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":316,"num_read_kb":286358,"num_write":82,"num_write_kb":22727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"97'114","reported_seq":489,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-13T10:12:08.092186+0000","last_change":"2024-09-13T10:10:59.373229+0000","last_active":"2024-09-13T10:12:08.092186+0000","last_peered":"2024-09-13T10:12:08.092186+0000","last_clean":"2024-09-13T10:12:08.092186+0000","last_became_active":"2024-09-13T10:10:59.366341+0000","last_became_peered":"2024-09-13T10:10:59.366341+0000","last_unstale":"2024-09-13T10:12:08.092186+0000","last_undegraded":"2024-09-13T10:12:08.092186+0000","last_fullsized":"2024-09-13T10:12:08.092186+0000","mapping_epoch":44,"log_start":"95'111","ondisk_log_start":"95'111","created":31,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":111,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:11.024361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14101238,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":512,"num_read_kb":480138,"num_write":117,"num_write_kb":32118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,12,3],"acting":[6,12,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"98'69","reported_seq":305,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-13T10:12:07.157222+0000","last_change":"2024-09-13T10:12:03.448785+0000","last_active":"2024-09-13T10:12:07.157222+0000","last_peered":"2024-09-13T10:12:07.157222+0000","last_clean":"2024-09-13T10:12:07.157222+0000","last_became_active":"2024-09-13T10:10:59.364443+0000","last_became_peered":"2024-09-13T10:10:59.364443+0000","last_unstale":"2024-09-13T10:12:07.157222+0000","last_undegraded":"2024-09-13T10:12:07.157222+0000","last_fullsized":"2024-09-13T10:12:07.157222+0000","mapping_epoch":44,"log_start":"89'65","ondisk_log_start":"89'65","created":31,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'67","last_scrub_stamp":"2024-09-13T10:12:03.448609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:03.448609+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":65,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:13.860860+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3529503,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":120980,"num_write":68,"num_write_kb":19128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,12,7],"acting":[15,12,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.e","version":"93'29","reported_seq":186,"reported_epoch":97,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:12:03.972502+0000","last_change":"2024-09-13T10:11:50.878103+0000","last_active":"2024-09-13T10:12:03.972502+0000","last_peered":"2024-09-13T10:12:03.972502+0000","last_clean":"2024-09-13T10:11:03.155929+0000","last_became_active":"2024-09-13T10:11:20.874865+0000","last_became_peered":"2024-09-13T10:11:20.874865+0000","last_unstale":"2024-09-13T10:12:03.972502+0000","last_undegraded":"2024-09-13T10:12:03.972502+0000","last_fullsized":"2024-09-13T10:12:03.972502+0000","mapping_epoch":61,"log_start":"93'26","ondisk_log_start":"93'26","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":26,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3899257,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":48,"num_read_kb":39513,"num_write":30,"num_write_kb":9155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,15],"acting":[2,8,4],"avail_no_missing":["2","4","8"],"object_location_counts":[{"shards":"2,4,8","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"96'104","reported_seq":378,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-09-13T10:12:05.792436+0000","last_change":"2024-09-13T10:11:53.668346+0000","last_active":"2024-09-13T10:12:05.792436+0000","last_peered":"2024-09-13T10:12:05.792436+0000","last_clean":"2024-09-13T10:12:05.792436+0000","last_became_active":"2024-09-13T10:10:44.223700+0000","last_became_peered":"2024-09-13T10:10:44.223700+0000","last_unstale":"2024-09-13T10:12:05.792436+0000","last_undegraded":"2024-09-13T10:12:05.792436+0000","last_fullsized":"2024-09-13T10:12:05.792436+0000","mapping_epoch":31,"log_start":"96'101","ondisk_log_start":"96'101","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'87","last_scrub_stamp":"2024-09-13T10:11:53.668179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:53.668179+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":101,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:07.663032+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5932970,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":227287,"num_write":114,"num_write_kb":32067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,7],"acting":[13,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":26,"stat_sum":{"num_bytes":129600823,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":3841,"num_read_kb":3492044,"num_write":1299,"num_write_kb":372244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":27276069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":18,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":261656576,"data_stored":260872918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":78,"acting":78,"num_store_stats":16},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":100899761,"num_objects":120,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":13,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":0,"num_read":1196,"num_read_kb":27072,"num_write":1689,"num_write_kb":331997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":12383428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":277643264,"data_stored":277033544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":13}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247830,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93748,"kb_used_data":62564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94278092,"statfs":{"total":96636764160,"available":96540766208,"internally_reserved":0,"allocated":64065536,"data_stored":63520361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247832,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48564,"kb_used_data":17380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323276,"statfs":{"total":96636764160,"available":96587034624,"internally_reserved":0,"allocated":17797120,"data_stored":17329487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247832,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63512,"kb_used_data":32328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308328,"statfs":{"total":96636764160,"available":96571727872,"internally_reserved":0,"allocated":33103872,"data_stored":32589907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313240,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33384,"kb_used_data":6360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338456,"statfs":{"total":96636764160,"available":96602578944,"internally_reserved":0,"allocated":6512640,"data_stored":6055320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,5,6,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313239,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83248,"kb_used_data":52064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94288592,"statfs":{"total":96636764160,"available":96551518208,"internally_reserved":0,"allocated":53313536,"data_stored":52800791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,6,7,8,9,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247830,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62508,"kb_used_data":31320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94309332,"statfs":{"total":96636764160,"available":96572755968,"internally_reserved":0,"allocated":32071680,"data_stored":31516044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,7,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280535,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63984,"kb_used_data":31596,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94307856,"statfs":{"total":96636764160,"available":96571244544,"internally_reserved":0,"allocated":32354304,"data_stored":31763844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280535,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101940,"kb_used_data":70756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94269900,"statfs":{"total":96636764160,"available":96532377600,"internally_reserved":0,"allocated":72454144,"data_stored":71841113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280536,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104264,"kb_used_data":73080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94267576,"statfs":{"total":96636764160,"available":96529997824,"internally_reserved":0,"allocated":74833920,"data_stored":74244308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280534,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55232,"kb_used_data":28208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94316608,"statfs":{"total":96636764160,"available":96580206592,"internally_reserved":0,"allocated":28884992,"data_stored":28384229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280535,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73820,"kb_used_data":42636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94298020,"statfs":{"total":96636764160,"available":96561172480,"internally_reserved":0,"allocated":43659264,"data_stored":43130723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280535,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94716,"kb_used_data":61156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94277124,"statfs":{"total":96636764160,"available":96539774976,"internally_reserved":0,"allocated":62623744,"data_stored":61956228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280535,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35080,"kb_used_data":3896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336760,"statfs":{"total":96636764160,"available":96600842240,"internally_reserved":0,"allocated":3989504,"data_stored":3533363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,5,6,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280535,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30852,"kb_used_data":3828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94340988,"statfs":{"total":96636764160,"available":96605171712,"internally_reserved":0,"allocated":3919872,"data_stored":3467296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247830,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48324,"kb_used_data":17140,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323516,"statfs":{"total":96636764160,"available":96587280384,"internally_reserved":0,"allocated":17551360,"data_stored":17071178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247830,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":74540,"kb_used_data":37264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94297300,"statfs":{"total":96636764160,"available":96560435200,"internally_reserved":0,"allocated":38158336,"data_stored":37500356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14188544,"data_stored":14159972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15228928,"data_stored":15179963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49909760,"data_stored":49785092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":31748096,"data_stored":31681660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":31748096,"data_stored":31681660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":23207936,"data_stored":23154079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":35909632,"data_stored":35818171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":26664960,"data_stored":26621319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":49037312,"data_stored":48951628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16781312,"data_stored":16689792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16912384,"data_stored":16813558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16527360,"data_stored":16486480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20787200,"data_stored":20757762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15204352,"data_stored":15178418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28299264,"data_stored":28249828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":35983360,"data_stored":35798220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":11554816,"data_stored":11540941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":9019392,"data_stored":8996844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":5369856,"data_stored":5363936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":27602944,"data_stored":27468601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":2838528,"data_stored":2833714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":16695296,"data_stored":16663207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":21917696,"data_stored":21891652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":13393920,"data_stored":13372318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":2768896,"data_stored":2767647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:12:09.114 INFO:tasks.ceph.ceph_manager.ceph:decrease pool size by 10 2024-09-13T10:12:09.114 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pg_num 16 2024-09-13T10:12:09.128 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:12:09.128 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:12:09.130 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1476055 2024-09-13T10:12:09.132 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:12:09.131+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:12:09.157 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:12:09.157 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:12:09.183 INFO:tasks.rados.rados.0.smithi195.stdout:1788: finishing write tid 2 to smithi19530714-17 2024-09-13T10:12:09.183 INFO:tasks.rados.rados.0.smithi195.stdout:1788: finishing write tid 3 to smithi19530714-17 2024-09-13T10:12:09.183 INFO:tasks.rados.rados.0.smithi195.stdout:1788: finishing write tid 4 to smithi19530714-17 2024-09-13T10:12:09.183 INFO:tasks.rados.rados.0.smithi195.stdout:1788: oid 17 updating version 0 to 98 2024-09-13T10:12:09.183 INFO:tasks.rados.rados.0.smithi195.stdout:1788: oid 17 version 98 is already newer than 97 2024-09-13T10:12:09.183 INFO:tasks.rados.rados.0.smithi195.stdout:1788: oid 17 updating version 98 to 99 2024-09-13T10:12:09.183 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 99 (ObjNum 388 snap 0 seq_num 388) dirty exists 2024-09-13T10:12:09.184 INFO:tasks.rados.rados.0.smithi195.stdout:1788: left oid 17 (ObjNum 388 snap 0 seq_num 388) 2024-09-13T10:12:09.184 INFO:tasks.rados.rados.0.smithi195.stdout:1790: expect (ObjNum 373 snap 0 seq_num 373) 2024-09-13T10:12:09.184 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:09.216 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:09.233 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:09.230+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:09.238 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:09.238 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:12:09.238 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:12:09.238 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:12:09.238 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:09.252 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T10:12:09.282 INFO:tasks.rados.rados.0.smithi195.stdout:1775: done (15 left) 2024-09-13T10:12:09.282 INFO:tasks.rados.rados.0.smithi195.stdout:1776: done (14 left) 2024-09-13T10:12:09.282 INFO:tasks.rados.rados.0.smithi195.stdout:1777: done (13 left) 2024-09-13T10:12:09.283 INFO:tasks.rados.rados.0.smithi195.stdout:1778: done (12 left) 2024-09-13T10:12:09.283 INFO:tasks.rados.rados.0.smithi195.stdout:1779: done (11 left) 2024-09-13T10:12:09.283 INFO:tasks.rados.rados.0.smithi195.stdout:1780: done (10 left) 2024-09-13T10:12:09.283 INFO:tasks.rados.rados.0.smithi195.stdout:1781: done (9 left) 2024-09-13T10:12:09.283 INFO:tasks.rados.rados.0.smithi195.stdout:1782: done (8 left) 2024-09-13T10:12:09.283 INFO:tasks.rados.rados.0.smithi195.stdout:1783: done (7 left) 2024-09-13T10:12:09.283 INFO:tasks.rados.rados.0.smithi195.stdout:1784: done (6 left) 2024-09-13T10:12:09.283 INFO:tasks.rados.rados.0.smithi195.stdout:1785: done (5 left) 2024-09-13T10:12:09.283 INFO:tasks.rados.rados.0.smithi195.stdout:1786: done (4 left) 2024-09-13T10:12:09.283 INFO:tasks.rados.rados.0.smithi195.stdout:1787: done (3 left) 2024-09-13T10:12:09.283 INFO:tasks.rados.rados.0.smithi195.stdout:1788: done (2 left) 2024-09-13T10:12:09.283 INFO:tasks.rados.rados.0.smithi195.stdout:1789: done (1 left) 2024-09-13T10:12:09.283 INFO:tasks.rados.rados.0.smithi195.stdout:1790: done (0 left) 2024-09-13T10:12:09.284 INFO:tasks.rados.rados.0.smithi195.stdout:1791: write oid 20 current snap is 0 2024-09-13T10:12:09.284 INFO:tasks.rados.rados.0.smithi195.stdout:1791: seq_num 389 ranges {537923=405864,1513613=593796} 2024-09-13T10:12:09.289 INFO:tasks.rados.rados.0.smithi195.stdout:1791: writing smithi19530714-20 from 537923 to 943787 tid 1 2024-09-13T10:12:09.299 INFO:tasks.rados.rados.0.smithi195.stdout:1791: writing smithi19530714-20 from 1513613 to 2107409 tid 2 2024-09-13T10:12:09.299 INFO:tasks.rados.rados.0.smithi195.stdout:1792: set_chunk oid 15 target oid 2024-09-13T10:12:09.317 INFO:tasks.rados.rados.0.smithi195.stdout:1792: set_chunk oid 15 offset: 985600 length: 1182720 target oid bee1586f37a6f692be499ea076d101634d4d699605eb6640f0d71e63781d1f99 offset: 0 snap -1 2024-09-13T10:12:09.317 INFO:tasks.rados.rados.0.smithi195.stdout:1793: delete oid 44 current snap is 0 2024-09-13T10:12:09.317 INFO:tasks.rados.rados.0.smithi195.stdout:1791: finishing write tid 1 to smithi19530714-20 2024-09-13T10:12:09.317 INFO:tasks.rados.rados.0.smithi195.stdout:1791: finishing write tid 2 to smithi19530714-20 2024-09-13T10:12:09.317 INFO:tasks.rados.rados.0.smithi195.stdout:1791: finishing write tid 3 to smithi19530714-20 2024-09-13T10:12:09.317 INFO:tasks.rados.rados.0.smithi195.stdout:1791: finishing write tid 4 to smithi19530714-20 2024-09-13T10:12:09.317 INFO:tasks.rados.rados.0.smithi195.stdout:1791: oid 20 updating version 0 to 100 2024-09-13T10:12:09.318 INFO:tasks.rados.rados.0.smithi195.stdout:1791: oid 20 version 100 is already newer than 99 2024-09-13T10:12:09.318 INFO:tasks.rados.rados.0.smithi195.stdout:1791: oid 20 version 100 is already newer than 98 2024-09-13T10:12:09.318 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 20 v 100 (ObjNum 389 snap 0 seq_num 389) dirty exists 2024-09-13T10:12:09.318 INFO:tasks.rados.rados.0.smithi195.stdout:1791: left oid 20 (ObjNum 389 snap 0 seq_num 389) 2024-09-13T10:12:09.321 INFO:tasks.rados.rados.0.smithi195.stdout:1791: done (2 left) 2024-09-13T10:12:09.321 INFO:tasks.rados.rados.0.smithi195.stdout:1793: done (1 left) 2024-09-13T10:12:09.321 INFO:tasks.rados.rados.0.smithi195.stdout:1794: write oid 48 current snap is 0 2024-09-13T10:12:09.322 INFO:tasks.rados.rados.0.smithi195.stdout:1794: seq_num 391 ranges {571537=556362,1630577=597465} 2024-09-13T10:12:09.329 INFO:tasks.rados.rados.0.smithi195.stdout:1794: writing smithi19530714-48 from 571537 to 1127899 tid 1 2024-09-13T10:12:09.333 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:09.329+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:09.337 INFO:tasks.rados.rados.0.smithi195.stdout:1794: writing smithi19530714-48 from 1630577 to 2228042 tid 2 2024-09-13T10:12:09.337 INFO:tasks.rados.rados.0.smithi195.stdout:1795: read oid 31 snap -1 2024-09-13T10:12:09.337 INFO:tasks.rados.rados.0.smithi195.stdout:1795: expect (ObjNum 386 snap 0 seq_num 386) 2024-09-13T10:12:09.337 INFO:tasks.rados.rados.0.smithi195.stdout:1796: write oid 2 current snap is 0 2024-09-13T10:12:09.337 INFO:tasks.rados.rados.0.smithi195.stdout:1792: finishing set_chunk to oid 15 2024-09-13T10:12:09.337 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 114 (ObjNum 373 snap 0 seq_num 373) dirty exists 2024-09-13T10:12:09.337 INFO:tasks.rados.rados.0.smithi195.stdout:1796: seq_num 392 ranges {588344=631611,1689059=741123} 2024-09-13T10:12:09.345 INFO:tasks.rados.rados.0.smithi195.stdout:1796: writing smithi19530714-2 from 588344 to 1219955 tid 1 2024-09-13T10:12:09.353 INFO:tasks.rados.rados.0.smithi195.stdout:1796: writing smithi19530714-2 from 1689059 to 2430182 tid 2 2024-09-13T10:12:09.353 INFO:tasks.rados.rados.0.smithi195.stdout:1792: done (3 left) 2024-09-13T10:12:09.353 INFO:tasks.rados.rados.0.smithi195.stdout:1797: write (excl) oid 10 current snap is 0 2024-09-13T10:12:09.353 INFO:tasks.rados.rados.0.smithi195.stdout:1797: seq_num 393 ranges {605151=706860,2031188=573963} 2024-09-13T10:12:09.360 INFO:tasks.rados.rados.0.smithi195.stdout:1797: writing smithi19530714-10 from 605151 to 1312011 tid 1 2024-09-13T10:12:09.368 INFO:tasks.rados.rados.0.smithi195.stdout:1797: writing smithi19530714-10 from 2031188 to 2605151 tid 2 2024-09-13T10:12:09.368 INFO:tasks.rados.rados.0.smithi195.stdout:1794: finishing write tid 1 to smithi19530714-48 2024-09-13T10:12:09.368 INFO:tasks.rados.rados.0.smithi195.stdout:1794: finishing write tid 2 to smithi19530714-48 2024-09-13T10:12:09.368 INFO:tasks.rados.rados.0.smithi195.stdout:1794: finishing write tid 3 to smithi19530714-48 2024-09-13T10:12:09.368 INFO:tasks.rados.rados.0.smithi195.stdout:1794: finishing write tid 4 to smithi19530714-48 2024-09-13T10:12:09.368 INFO:tasks.rados.rados.0.smithi195.stdout:1794: oid 48 updating version 0 to 118 2024-09-13T10:12:09.368 INFO:tasks.rados.rados.0.smithi195.stdout:1794: oid 48 version 118 is already newer than 116 2024-09-13T10:12:09.368 INFO:tasks.rados.rados.0.smithi195.stdout:1794: oid 48 version 118 is already newer than 117 2024-09-13T10:12:09.368 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 118 (ObjNum 391 snap 0 seq_num 391) dirty exists 2024-09-13T10:12:09.368 INFO:tasks.rados.rados.0.smithi195.stdout:1794: left oid 48 (ObjNum 391 snap 0 seq_num 391) 2024-09-13T10:12:09.369 INFO:tasks.rados.rados.0.smithi195.stdout:1795: expect (ObjNum 386 snap 0 seq_num 386) 2024-09-13T10:12:09.404 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:09.414 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:09.414 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:12:09.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:12:09.414 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:12:09.414 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.414 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:207 3.0 3:332cd44b:::smithi19530714-8 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:09.414 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:12:09.414 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.417838134999997, 2024-09-13T10:12:09.414 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074034500000000003, 2024-09-13T10:12:09.414 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.414 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.415 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.415 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.415 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.415 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.415 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 207 2024-09-13T10:12:09.415 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.415 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.415 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.415 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.415 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:12:09.415 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:12:09.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.416 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.416 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991271+0000", 2024-09-13T10:12:09.416 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9289999999999998e-06 2024-09-13T10:12:09.416 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.416 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991284+0000", 2024-09-13T10:12:09.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2597e-05 2024-09-13T10:12:09.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991313+0000", 2024-09-13T10:12:09.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8982000000000001e-05 2024-09-13T10:12:09.417 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.417 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.417 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.417 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991334+0000", 2024-09-13T10:12:09.417 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1046000000000001e-05 2024-09-13T10:12:09.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991416+0000", 2024-09-13T10:12:09.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.153e-05 2024-09-13T10:12:09.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.418 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.418 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991504+0000", 2024-09-13T10:12:09.418 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8220999999999994e-05 2024-09-13T10:12:09.418 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.418 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:09.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017232+0000", 2024-09-13T10:12:09.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025728328000000002 2024-09-13T10:12:09.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:09.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.031588+0000", 2024-09-13T10:12:09.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014355557 2024-09-13T10:12:09.419 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.419 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.419 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.419 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034211+0000", 2024-09-13T10:12:09.419 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026232790000000001 2024-09-13T10:12:09.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065168+0000", 2024-09-13T10:12:09.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030956833999999999 2024-09-13T10:12:09.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.420 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.420 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.420 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065260+0000", 2024-09-13T10:12:09.420 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2044000000000002e-05 2024-09-13T10:12:09.420 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.421 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.421 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065303+0000", 2024-09-13T10:12:09.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3152999999999998e-05 2024-09-13T10:12:09.421 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.421 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.421 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.421 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.421 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.421 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:211 3.0 3:0596651d:::smithi19530714-12 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:09.421 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:12:09.421 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.417114756999993, 2024-09-13T10:12:09.421 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.042923122000000001, 2024-09-13T10:12:09.422 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.422 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.422 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.422 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.422 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.422 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.422 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 211 2024-09-13T10:12:09.422 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.422 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.422 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.422 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.422 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:12:09.422 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:12:09.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.423 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991995+0000", 2024-09-13T10:12:09.423 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7640000000000001e-06 2024-09-13T10:12:09.423 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.423 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.423 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992007+0000", 2024-09-13T10:12:09.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2058999999999999e-05 2024-09-13T10:12:09.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992040+0000", 2024-09-13T10:12:09.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3466000000000001e-05 2024-09-13T10:12:09.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.424 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.424 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.424 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992059+0000", 2024-09-13T10:12:09.424 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8485999999999999e-05 2024-09-13T10:12:09.424 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992136+0000", 2024-09-13T10:12:09.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6994000000000002e-05 2024-09-13T10:12:09.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992207+0000", 2024-09-13T10:12:09.425 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1357000000000001e-05 2024-09-13T10:12:09.425 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.425 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.425 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:09.425 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.006377+0000", 2024-09-13T10:12:09.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014170455 2024-09-13T10:12:09.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:09.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.022844+0000", 2024-09-13T10:12:09.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016466275999999998 2024-09-13T10:12:09.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.426 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.426 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.426 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023031+0000", 2024-09-13T10:12:09.426 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018686000000000001 2024-09-13T10:12:09.426 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034788+0000", 2024-09-13T10:12:09.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011756967 2024-09-13T10:12:09.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034857+0000", 2024-09-13T10:12:09.427 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9169000000000001e-05 2024-09-13T10:12:09.427 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.427 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.427 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.427 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034915+0000", 2024-09-13T10:12:09.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8269000000000001e-05 2024-09-13T10:12:09.428 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.428 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.428 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.428 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.428 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.428 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:212 3.0 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:09.428 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:12:09.428 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.417008424000002, 2024-09-13T10:12:09.428 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.073011809999999996, 2024-09-13T10:12:09.428 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.428 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.429 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.429 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.429 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.429 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.429 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 212 2024-09-13T10:12:09.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.429 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.429 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.429 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.429 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:12:09.429 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.429 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:12:09.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992099+0000", 2024-09-13T10:12:09.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4899999999999995e-07 2024-09-13T10:12:09.430 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.430 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.430 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.430 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992103+0000", 2024-09-13T10:12:09.430 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.33e-06 2024-09-13T10:12:09.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992110+0000", 2024-09-13T10:12:09.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8399999999999997e-06 2024-09-13T10:12:09.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992116+0000", 2024-09-13T10:12:09.431 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6790000000000002e-06 2024-09-13T10:12:09.431 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.431 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.431 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.431 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992292+0000", 2024-09-13T10:12:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017659199999999999 2024-09-13T10:12:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992385+0000", 2024-09-13T10:12:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2608000000000005e-05 2024-09-13T10:12:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011428+0000", 2024-09-13T10:12:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.019042631000000001 2024-09-13T10:12:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.432 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.022852+0000", 2024-09-13T10:12:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011424528999999999 2024-09-13T10:12:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.031660+0000", 2024-09-13T10:12:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088079899999999999 2024-09-13T10:12:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065032+0000", 2024-09-13T10:12:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033372247000000001 2024-09-13T10:12:09.433 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065073+0000", 2024-09-13T10:12:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0598e-05 2024-09-13T10:12:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065110+0000", 2024-09-13T10:12:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7116999999999997e-05 2024-09-13T10:12:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.434 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:202 3.2 e38/30 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'1, mlcod=0'0)", 2024-09-13T10:12:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:12:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.407517818000002, 2024-09-13T10:12:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017280298999999999, 2024-09-13T10:12:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:12:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.435 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:12:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001594+0000", 2024-09-13T10:12:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5550000000000002e-06 2024-09-13T10:12:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.436 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009486+0000", 2024-09-13T10:12:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078916799999999999 2024-09-13T10:12:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009519+0000", 2024-09-13T10:12:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2898999999999998e-05 2024-09-13T10:12:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009540+0000", 2024-09-13T10:12:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1559000000000001e-05 2024-09-13T10:12:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.437 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009655+0000", 2024-09-13T10:12:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114271 2024-09-13T10:12:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009683+0000", 2024-09-13T10:12:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8379000000000001e-05 2024-09-13T10:12:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.438 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.018766+0000", 2024-09-13T10:12:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0090829589999999998 2024-09-13T10:12:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.018869+0000", 2024-09-13T10:12:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102997 2024-09-13T10:12:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:216 3.0 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:12:09.439 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.404505282000002, 2024-09-13T10:12:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.063936142000000001, 2024-09-13T10:12:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 216 2024-09-13T10:12:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.440 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:12:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:12:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004603+0000", 2024-09-13T10:12:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5540000000000001e-06 2024-09-13T10:12:09.441 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004615+0000", 2024-09-13T10:12:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2597e-05 2024-09-13T10:12:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004637+0000", 2024-09-13T10:12:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1713e-05 2024-09-13T10:12:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.442 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004651+0000", 2024-09-13T10:12:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3878e-05 2024-09-13T10:12:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004985+0000", 2024-09-13T10:12:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033347299999999998 2024-09-13T10:12:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.005706+0000", 2024-09-13T10:12:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00072122199999999999 2024-09-13T10:12:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.443 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023801+0000", 2024-09-13T10:12:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018095466000000001 2024-09-13T10:12:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.032153+0000", 2024-09-13T10:12:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083516010000000002 2024-09-13T10:12:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.035250+0000", 2024-09-13T10:12:09.444 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030974409999999998 2024-09-13T10:12:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068439+0000", 2024-09-13T10:12:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033189042000000002 2024-09-13T10:12:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068499+0000", 2024-09-13T10:12:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9969000000000001e-05 2024-09-13T10:12:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.445 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068537+0000", 2024-09-13T10:12:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8186000000000003e-05 2024-09-13T10:12:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:221 3.0 3:18ae852b:::smithi19530714-22 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:12:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.370580774999993, 2024-09-13T10:12:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039809298, 2024-09-13T10:12:09.446 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 221 2024-09-13T10:12:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:12:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.447 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:12:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038528+0000", 2024-09-13T10:12:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8819999999999999e-06 2024-09-13T10:12:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.448 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038534+0000", 2024-09-13T10:12:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0730000000000001e-06 2024-09-13T10:12:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038546+0000", 2024-09-13T10:12:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.219e-05 2024-09-13T10:12:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038566+0000", 2024-09-13T10:12:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9867000000000001e-05 2024-09-13T10:12:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.449 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038608+0000", 2024-09-13T10:12:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1977999999999997e-05 2024-09-13T10:12:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038707+0000", 2024-09-13T10:12:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8708000000000004e-05 2024-09-13T10:12:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.057374+0000", 2024-09-13T10:12:09.450 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018667652999999999 2024-09-13T10:12:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061983+0000", 2024-09-13T10:12:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0046087089999999999 2024-09-13T10:12:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.063027+0000", 2024-09-13T10:12:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001044239 2024-09-13T10:12:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.451 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078233+0000", 2024-09-13T10:12:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015205639999999999 2024-09-13T10:12:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078298+0000", 2024-09-13T10:12:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5180999999999998e-05 2024-09-13T10:12:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078335+0000", 2024-09-13T10:12:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7178000000000002e-05 2024-09-13T10:12:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.452 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:225 3.0 3:26960251:::smithi19530714-26 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:12:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.350654500999994, 2024-09-13T10:12:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030704875999999999, 2024-09-13T10:12:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.453 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 225 2024-09-13T10:12:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:12:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:12:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.454 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058455+0000", 2024-09-13T10:12:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3220000000000001e-06 2024-09-13T10:12:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058467+0000", 2024-09-13T10:12:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1987999999999999e-05 2024-09-13T10:12:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058502+0000", 2024-09-13T10:12:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4873000000000003e-05 2024-09-13T10:12:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.455 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058520+0000", 2024-09-13T10:12:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7977000000000001e-05 2024-09-13T10:12:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060066+0000", 2024-09-13T10:12:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001545261 2024-09-13T10:12:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060680+0000", 2024-09-13T10:12:09.456 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00061431799999999998 2024-09-13T10:12:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070650+0000", 2024-09-13T10:12:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0099703099999999996 2024-09-13T10:12:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078192+0000", 2024-09-13T10:12:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0075419709999999997 2024-09-13T10:12:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.457 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084486+0000", 2024-09-13T10:12:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0062943779999999998 2024-09-13T10:12:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089029+0000", 2024-09-13T10:12:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045422379999999997 2024-09-13T10:12:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089092+0000", 2024-09-13T10:12:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3435999999999994e-05 2024-09-13T10:12:09.458 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089157+0000", 2024-09-13T10:12:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4803999999999998e-05 2024-09-13T10:12:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:228 3.0 3:0168a995:::smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:12:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.348853543000004, 2024-09-13T10:12:09.459 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030139446, 2024-09-13T10:12:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 228 2024-09-13T10:12:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:12:09.460 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:12:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060255+0000", 2024-09-13T10:12:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.787e-06 2024-09-13T10:12:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.461 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060262+0000", 2024-09-13T10:12:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8630000000000002e-06 2024-09-13T10:12:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060285+0000", 2024-09-13T10:12:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2935e-05 2024-09-13T10:12:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060294+0000", 2024-09-13T10:12:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3940000000000002e-06 2024-09-13T10:12:09.462 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061041+0000", 2024-09-13T10:12:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00074706700000000002 2024-09-13T10:12:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061148+0000", 2024-09-13T10:12:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107364 2024-09-13T10:12:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074199+0000", 2024-09-13T10:12:09.463 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013050447999999999 2024-09-13T10:12:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084506+0000", 2024-09-13T10:12:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010306789 2024-09-13T10:12:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089222+0000", 2024-09-13T10:12:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047163480000000004 2024-09-13T10:12:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.464 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090310+0000", 2024-09-13T10:12:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010880919999999999 2024-09-13T10:12:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090356+0000", 2024-09-13T10:12:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5522000000000002e-05 2024-09-13T10:12:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090393+0000", 2024-09-13T10:12:09.465 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6837e-05 2024-09-13T10:12:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:229 3.0 3:1e6cc58c:::smithi19530714-30 low_tier:head [copy-from ver 18] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:12:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.347547168000006, 2024-09-13T10:12:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033410479, 2024-09-13T10:12:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.466 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 229 2024-09-13T10:12:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:12:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.467 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:12:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061562+0000", 2024-09-13T10:12:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9730000000000002e-06 2024-09-13T10:12:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061577+0000", 2024-09-13T10:12:09.468 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5044e-05 2024-09-13T10:12:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061602+0000", 2024-09-13T10:12:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4494999999999998e-05 2024-09-13T10:12:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061615+0000", 2024-09-13T10:12:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3207000000000001e-05 2024-09-13T10:12:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.469 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061701+0000", 2024-09-13T10:12:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5694999999999999e-05 2024-09-13T10:12:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061871+0000", 2024-09-13T10:12:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017004100000000001 2024-09-13T10:12:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:09.470 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.079511+0000", 2024-09-13T10:12:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017639708 2024-09-13T10:12:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089295+0000", 2024-09-13T10:12:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0097846359999999993 2024-09-13T10:12:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089316+0000", 2024-09-13T10:12:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0288999999999998e-05 2024-09-13T10:12:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.471 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094875+0000", 2024-09-13T10:12:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0055598389999999996 2024-09-13T10:12:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094931+0000", 2024-09-13T10:12:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5556999999999997e-05 2024-09-13T10:12:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094970+0000", 2024-09-13T10:12:09.472 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8995000000000002e-05 2024-09-13T10:12:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:233 3.0 3:24ead787:::smithi19530714-34 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:12:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.343370664000005, 2024-09-13T10:12:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047200234000000001, 2024-09-13T10:12:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.473 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 233 2024-09-13T10:12:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:12:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:12:09.474 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065737+0000", 2024-09-13T10:12:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.283e-06 2024-09-13T10:12:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065745+0000", 2024-09-13T10:12:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3130000000000001e-06 2024-09-13T10:12:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.475 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065757+0000", 2024-09-13T10:12:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2627e-05 2024-09-13T10:12:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065765+0000", 2024-09-13T10:12:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4669999999999998e-06 2024-09-13T10:12:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065816+0000", 2024-09-13T10:12:09.476 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1545000000000002e-05 2024-09-13T10:12:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065940+0000", 2024-09-13T10:12:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012366699999999999 2024-09-13T10:12:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.086064+0000", 2024-09-13T10:12:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.020124487 2024-09-13T10:12:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.477 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090747+0000", 2024-09-13T10:12:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004682795 2024-09-13T10:12:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.091235+0000", 2024-09-13T10:12:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048779900000000002 2024-09-13T10:12:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112809+0000", 2024-09-13T10:12:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.021573967999999999 2024-09-13T10:12:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.478 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112878+0000", 2024-09-13T10:12:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9413999999999996e-05 2024-09-13T10:12:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112936+0000", 2024-09-13T10:12:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7868999999999998e-05 2024-09-13T10:12:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.479 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:227 3.2 e38/30 3:6ac7800c:::smithi19530714-28 low_tier:head v 38'8, mlcod=38'6)", 2024-09-13T10:12:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:12:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.335007601000001, 2024-09-13T10:12:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014433425999999999, 2024-09-13T10:12:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:12:09.480 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:12:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074101+0000", 2024-09-13T10:12:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.511e-06 2024-09-13T10:12:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.481 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083754+0000", 2024-09-13T10:12:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096533799999999996 2024-09-13T10:12:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083772+0000", 2024-09-13T10:12:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8241e-05 2024-09-13T10:12:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083782+0000", 2024-09-13T10:12:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.004e-05 2024-09-13T10:12:09.482 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083875+0000", 2024-09-13T10:12:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.268e-05 2024-09-13T10:12:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083918+0000", 2024-09-13T10:12:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-13T10:12:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.088456+0000", 2024-09-13T10:12:09.483 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045385540000000002 2024-09-13T10:12:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.088532+0000", 2024-09-13T10:12:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6069000000000006e-05 2024-09-13T10:12:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:239 3.0 3:078df4f4:::smithi19530714-40 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:09.484 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:12:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.319804089000002, 2024-09-13T10:12:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025192791999999999, 2024-09-13T10:12:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 239 2024-09-13T10:12:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.485 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:12:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:12:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089305+0000", 2024-09-13T10:12:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1710000000000001e-06 2024-09-13T10:12:09.486 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089312+0000", 2024-09-13T10:12:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6860000000000006e-06 2024-09-13T10:12:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089323+0000", 2024-09-13T10:12:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0467e-05 2024-09-13T10:12:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089330+0000", 2024-09-13T10:12:09.487 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3939999999999998e-06 2024-09-13T10:12:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089379+0000", 2024-09-13T10:12:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8990999999999999e-05 2024-09-13T10:12:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089543+0000", 2024-09-13T10:12:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016351700000000001 2024-09-13T10:12:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.488 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.098886+0000", 2024-09-13T10:12:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0093434980000000004 2024-09-13T10:12:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.105855+0000", 2024-09-13T10:12:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.006968279 2024-09-13T10:12:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:09.489 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.105868+0000", 2024-09-13T10:12:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3560999999999999e-05 2024-09-13T10:12:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114382+0000", 2024-09-13T10:12:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0085134730000000006 2024-09-13T10:12:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114458+0000", 2024-09-13T10:12:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6143999999999995e-05 2024-09-13T10:12:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.490 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114495+0000", 2024-09-13T10:12:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7611000000000003e-05 2024-09-13T10:12:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:241 3.0 3:388ca573:::smithi19530714-42 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:12:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.313930366999998, 2024-09-13T10:12:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057595649999999998, 2024-09-13T10:12:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.491 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 241 2024-09-13T10:12:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:12:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.492 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:12:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095178+0000", 2024-09-13T10:12:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.345e-06 2024-09-13T10:12:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095184+0000", 2024-09-13T10:12:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4239999999999998e-06 2024-09-13T10:12:09.493 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095198+0000", 2024-09-13T10:12:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4037e-05 2024-09-13T10:12:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095206+0000", 2024-09-13T10:12:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1629999999999996e-06 2024-09-13T10:12:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.494 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095274+0000", 2024-09-13T10:12:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7948999999999996e-05 2024-09-13T10:12:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095485+0000", 2024-09-13T10:12:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021050100000000001 2024-09-13T10:12:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.134651+0000", 2024-09-13T10:12:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039166200999999998 2024-09-13T10:12:09.495 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.140536+0000", 2024-09-13T10:12:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058849719999999996 2024-09-13T10:12:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.140980+0000", 2024-09-13T10:12:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044394200000000001 2024-09-13T10:12:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.496 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152700+0000", 2024-09-13T10:12:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011720431999999999 2024-09-13T10:12:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152739+0000", 2024-09-13T10:12:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8448999999999997e-05 2024-09-13T10:12:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152772+0000", 2024-09-13T10:12:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3235000000000003e-05 2024-09-13T10:12:09.497 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:247 3.0 3:0b30fd85:::smithi19530714-48 low_tier:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:12:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.296030716000004, 2024-09-13T10:12:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032448926000000003, 2024-09-13T10:12:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.498 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 247 2024-09-13T10:12:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:12:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:12:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.499 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113078+0000", 2024-09-13T10:12:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8670000000000001e-06 2024-09-13T10:12:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113086+0000", 2024-09-13T10:12:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2260000000000005e-06 2024-09-13T10:12:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113104+0000", 2024-09-13T10:12:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7887e-05 2024-09-13T10:12:09.500 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113115+0000", 2024-09-13T10:12:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0763e-05 2024-09-13T10:12:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113163+0000", 2024-09-13T10:12:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8541999999999997e-05 2024-09-13T10:12:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.501 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113322+0000", 2024-09-13T10:12:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000159121 2024-09-13T10:12:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.125165+0000", 2024-09-13T10:12:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011842320999999999 2024-09-13T10:12:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.131189+0000", 2024-09-13T10:12:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.006024728 2024-09-13T10:12:09.502 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.131203+0000", 2024-09-13T10:12:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3551e-05 2024-09-13T10:12:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145435+0000", 2024-09-13T10:12:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014232273 2024-09-13T10:12:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.503 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145481+0000", 2024-09-13T10:12:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5918000000000003e-05 2024-09-13T10:12:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145525+0000", 2024-09-13T10:12:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3729000000000001e-05 2024-09-13T10:12:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:244 3.2 e38/30 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'14, mlcod=38'10)", 2024-09-13T10:12:09.504 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:12:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.282564913000002, 2024-09-13T10:12:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014967827, 2024-09-13T10:12:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:12:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.505 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:12:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126544+0000", 2024-09-13T10:12:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.03e-06 2024-09-13T10:12:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133374+0000", 2024-09-13T10:12:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068299119999999996 2024-09-13T10:12:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.506 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133398+0000", 2024-09-13T10:12:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4209000000000001e-05 2024-09-13T10:12:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133404+0000", 2024-09-13T10:12:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3570000000000003e-06 2024-09-13T10:12:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.507 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133457+0000", 2024-09-13T10:12:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2320000000000001e-05 2024-09-13T10:12:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133477+0000", 2024-09-13T10:12:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0721999999999999e-05 2024-09-13T10:12:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.141475+0000", 2024-09-13T10:12:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0079973649999999993 2024-09-13T10:12:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.508 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.141510+0000", 2024-09-13T10:12:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4912e-05 2024-09-13T10:12:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:254 4.8 4:1de9da87:::smithi19530714-5:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.214446+0000", 2024-09-13T10:12:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.194660575, 2024-09-13T10:12:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014408274, 2024-09-13T10:12:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.509 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 254 2024-09-13T10:12:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214446+0000", 2024-09-13T10:12:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214446+0000", 2024-09-13T10:12:09.510 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214449+0000", 2024-09-13T10:12:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7860000000000001e-06 2024-09-13T10:12:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214464+0000", 2024-09-13T10:12:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5290000000000001e-05 2024-09-13T10:12:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.511 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214499+0000", 2024-09-13T10:12:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5119e-05 2024-09-13T10:12:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214517+0000", 2024-09-13T10:12:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7563000000000001e-05 2024-09-13T10:12:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214680+0000", 2024-09-13T10:12:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016348799999999999 2024-09-13T10:12:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.512 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214937+0000", 2024-09-13T10:12:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025682599999999998 2024-09-13T10:12:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 0,11", 2024-09-13T10:12:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225660+0000", 2024-09-13T10:12:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010722768000000001 2024-09-13T10:12:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.227197+0000", 2024-09-13T10:12:09.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015373749999999999 2024-09-13T10:12:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228428+0000", 2024-09-13T10:12:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001230863 2024-09-13T10:12:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228646+0000", 2024-09-13T10:12:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021769700000000001 2024-09-13T10:12:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228755+0000", 2024-09-13T10:12:09.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109551 2024-09-13T10:12:09.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.228854+0000", 2024-09-13T10:12:09.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8948000000000002e-05 2024-09-13T10:12:09.515 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.515 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.515 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.515 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1836 3.0 3:0cf17fa4:::e22ed036f41f6a390924cf1e5e56fa98e0edea9e6c33080c396679f1537fbc7a:head [write 0~2655744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-13T10:12:09.515 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:10.675934+0000", 2024-09-13T10:12:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 58.733172279000001, 2024-09-13T10:12:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015481818, 2024-09-13T10:12:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1836 2024-09-13T10:12:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.675934+0000", 2024-09-13T10:12:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.675934+0000", 2024-09-13T10:12:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.675939+0000", 2024-09-13T10:12:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2830000000000003e-06 2024-09-13T10:12:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677769+0000", 2024-09-13T10:12:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0018299830000000001 2024-09-13T10:12:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677796+0000", 2024-09-13T10:12:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7858999999999999e-05 2024-09-13T10:12:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677809+0000", 2024-09-13T10:12:09.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2291e-05 2024-09-13T10:12:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677895+0000", 2024-09-13T10:12:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6581000000000003e-05 2024-09-13T10:12:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.678079+0000", 2024-09-13T10:12:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000183802 2024-09-13T10:12:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:09.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.678268+0000", 2024-09-13T10:12:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000188383 2024-09-13T10:12:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.683389+0000", 2024-09-13T10:12:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0051215840000000002 2024-09-13T10:12:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.684549+0000", 2024-09-13T10:12:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0011600169999999999 2024-09-13T10:12:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.691333+0000", 2024-09-13T10:12:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0067840510000000001 2024-09-13T10:12:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.691377+0000", 2024-09-13T10:12:09.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4313000000000001e-05 2024-09-13T10:12:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.691416+0000", 2024-09-13T10:12:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8671000000000003e-05 2024-09-13T10:12:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3640 4.12 4:4ac406d9:::smithi19530714-26:head [tier-promote] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e89)", 2024-09-13T10:12:09.522 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:53.723202+0000", 2024-09-13T10:12:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.685904765, 2024-09-13T10:12:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084555116999999999, 2024-09-13T10:12:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3640 2024-09-13T10:12:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723202+0000", 2024-09-13T10:12:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723202+0000", 2024-09-13T10:12:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723204+0000", 2024-09-13T10:12:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0420000000000001e-06 2024-09-13T10:12:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723212+0000", 2024-09-13T10:12:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2169999999999994e-06 2024-09-13T10:12:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723232+0000", 2024-09-13T10:12:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9419000000000002e-05 2024-09-13T10:12:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723245+0000", 2024-09-13T10:12:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3073e-05 2024-09-13T10:12:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723889+0000", 2024-09-13T10:12:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064398000000000005 2024-09-13T10:12:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-13T10:12:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723926+0000", 2024-09-13T10:12:09.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7685999999999998e-05 2024-09-13T10:12:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.805162+0000", 2024-09-13T10:12:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.081235561999999997 2024-09-13T10:12:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.805279+0000", 2024-09-13T10:12:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116883 2024-09-13T10:12:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.807757+0000", 2024-09-13T10:12:09.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002478255 2024-09-13T10:12:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3710 4.12 4:4a24ecac:::smithi19530714-36:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-13T10:12:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:53.723265+0000", 2024-09-13T10:12:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.685841719000001, 2024-09-13T10:12:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085310183999999997, 2024-09-13T10:12:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.528 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3710 2024-09-13T10:12:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723265+0000", 2024-09-13T10:12:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723265+0000", 2024-09-13T10:12:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723266+0000", 2024-09-13T10:12:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7300000000000004e-07 2024-09-13T10:12:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723269+0000", 2024-09-13T10:12:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5130000000000001e-06 2024-09-13T10:12:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723279+0000", 2024-09-13T10:12:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3049999999999998e-06 2024-09-13T10:12:09.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723286+0000", 2024-09-13T10:12:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6990000000000003e-06 2024-09-13T10:12:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.724004+0000", 2024-09-13T10:12:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00071808899999999999 2024-09-13T10:12:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-13T10:12:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.724020+0000", 2024-09-13T10:12:09.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5586999999999998e-05 2024-09-13T10:12:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.805788+0000", 2024-09-13T10:12:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.081768020999999996 2024-09-13T10:12:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.805868+0000", 2024-09-13T10:12:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9469999999999996e-05 2024-09-13T10:12:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.808575+0000", 2024-09-13T10:12:09.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027075269999999999 2024-09-13T10:12:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3745 3.0 3:161dd296:::f6d96d7a8dac6a7d56d3d9df9ef9c133dce9c515f90d85faf1e037f770fc9485:head [write 0~3512832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-13T10:12:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:54.607585+0000", 2024-09-13T10:12:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 14.801521474999999, 2024-09-13T10:12:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.021860623999999999, 2024-09-13T10:12:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:09.533 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3745 2024-09-13T10:12:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.607585+0000", 2024-09-13T10:12:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.607585+0000", 2024-09-13T10:12:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.607588+0000", 2024-09-13T10:12:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5790000000000002e-06 2024-09-13T10:12:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.610509+0000", 2024-09-13T10:12:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029216020000000001 2024-09-13T10:12:09.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.610542+0000", 2024-09-13T10:12:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2848000000000001e-05 2024-09-13T10:12:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.610559+0000", 2024-09-13T10:12:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7170999999999999e-05 2024-09-13T10:12:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:09.536 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.610699+0000", 2024-09-13T10:12:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000140032 2024-09-13T10:12:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.611058+0000", 2024-09-13T10:12:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000358246 2024-09-13T10:12:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.611462+0000", 2024-09-13T10:12:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000404165 2024-09-13T10:12:09.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.622881+0000", 2024-09-13T10:12:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011419558999999999 2024-09-13T10:12:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.624371+0000", 2024-09-13T10:12:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001489331 2024-09-13T10:12:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.629336+0000", 2024-09-13T10:12:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0049654850000000004 2024-09-13T10:12:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.629395+0000", 2024-09-13T10:12:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8405000000000001e-05 2024-09-13T10:12:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.629446+0000", 2024-09-13T10:12:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1201e-05 2024-09-13T10:12:09.539 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.540 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.540 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.540 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:09.540 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:09.540 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:09.542 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T10:12:09.543 INFO:tasks.rados.rados.0.smithi195.stdout:1796: finishing write tid 1 to smithi19530714-2 2024-09-13T10:12:09.543 INFO:tasks.rados.rados.0.smithi195.stdout:1796: finishing write tid 2 to smithi19530714-2 2024-09-13T10:12:09.543 INFO:tasks.rados.rados.0.smithi195.stdout:1796: finishing write tid 3 to smithi19530714-2 2024-09-13T10:12:09.543 INFO:tasks.rados.rados.0.smithi195.stdout:1796: finishing write tid 4 to smithi19530714-2 2024-09-13T10:12:09.543 INFO:tasks.rados.rados.0.smithi195.stdout:1796: oid 2 updating version 0 to 154 2024-09-13T10:12:09.543 INFO:tasks.rados.rados.0.smithi195.stdout:1796: oid 2 version 154 is already newer than 152 2024-09-13T10:12:09.543 INFO:tasks.rados.rados.0.smithi195.stdout:1796: oid 2 version 154 is already newer than 153 2024-09-13T10:12:09.543 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 154 (ObjNum 392 snap 0 seq_num 392) dirty exists 2024-09-13T10:12:09.543 INFO:tasks.rados.rados.0.smithi195.stdout:1796: left oid 2 (ObjNum 392 snap 0 seq_num 392) 2024-09-13T10:12:09.543 INFO:tasks.rados.rados.0.smithi195.stdout:1797: finishing write tid 1 to smithi19530714-10 2024-09-13T10:12:09.543 INFO:tasks.rados.rados.0.smithi195.stdout:1797: finishing write tid 2 to smithi19530714-10 2024-09-13T10:12:09.544 INFO:tasks.rados.rados.0.smithi195.stdout:1798: read oid 15 snap -1 2024-09-13T10:12:09.544 INFO:tasks.rados.rados.0.smithi195.stdout:1798: expect (ObjNum 373 snap 0 seq_num 373) 2024-09-13T10:12:09.544 INFO:tasks.rados.rados.0.smithi195.stdout:1794: done (4 left) 2024-09-13T10:12:09.544 INFO:tasks.rados.rados.0.smithi195.stdout:1795: done (3 left) 2024-09-13T10:12:09.544 INFO:tasks.rados.rados.0.smithi195.stdout:1796: done (2 left) 2024-09-13T10:12:09.544 INFO:tasks.rados.rados.0.smithi195.stdout:1799: read oid 42 snap -1 2024-09-13T10:12:09.544 INFO:tasks.rados.rados.0.smithi195.stdout:1799: expect (ObjNum 343 snap 0 seq_num 343) 2024-09-13T10:12:09.544 INFO:tasks.rados.rados.0.smithi195.stdout:1800: tier_evict oid 8 2024-09-13T10:12:09.544 INFO:tasks.rados.rados.0.smithi195.stdout:1800: tier_evict oid 8 snap -1 2024-09-13T10:12:09.544 INFO:tasks.rados.rados.0.smithi195.stdout:1797: finishing write tid 3 to smithi19530714-10 2024-09-13T10:12:09.544 INFO:tasks.rados.rados.0.smithi195.stdout:1797: finishing write tid 4 to smithi19530714-10 2024-09-13T10:12:09.544 INFO:tasks.rados.rados.0.smithi195.stdout:1797: oid 10 updating version 0 to 35 2024-09-13T10:12:09.544 INFO:tasks.rados.rados.0.smithi195.stdout:1797: oid 10 updating version 35 to 36 2024-09-13T10:12:09.544 INFO:tasks.rados.rados.0.smithi195.stdout:1797: oid 10 updating version 36 to 37 2024-09-13T10:12:09.545 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 37 (ObjNum 393 snap 0 seq_num 393) dirty exists 2024-09-13T10:12:09.545 INFO:tasks.rados.rados.0.smithi195.stdout:1797: left oid 10 (ObjNum 393 snap 0 seq_num 393) 2024-09-13T10:12:09.545 INFO:tasks.rados.rados.0.smithi195.stdout:1797: done (3 left) 2024-09-13T10:12:09.545 INFO:tasks.rados.rados.0.smithi195.stdout:1801: write (excl) oid 29 current snap is 0 2024-09-13T10:12:09.545 INFO:tasks.rados.rados.0.smithi195.stdout:1801: seq_num 394 ranges {621958=782109,2089670=532288} 2024-09-13T10:12:09.545 INFO:tasks.rados.rados.0.smithi195.stdout:1801: writing smithi19530714-29 from 621958 to 1404067 tid 1 2024-09-13T10:12:09.545 INFO:tasks.rados.rados.0.smithi195.stdout:1801: writing smithi19530714-29 from 2089670 to 2621958 tid 2 2024-09-13T10:12:09.545 INFO:tasks.rados.rados.0.smithi195.stdout:1802: write (excl) oid 12 current snap is 0 2024-09-13T10:12:09.545 INFO:tasks.rados.rados.0.smithi195.stdout:1800: got (22) Invalid argument 2024-09-13T10:12:09.545 INFO:tasks.rados.rados.0.smithi195.stdout:1798: expect (ObjNum 373 snap 0 seq_num 373) 2024-09-13T10:12:09.545 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:09.545 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1666875 2024-09-13T10:12:09.546 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:09.538+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:09.558 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 353249 2024-09-13T10:12:09.591 INFO:tasks.rados.rados.0.smithi195.stdout:1799: expect (ObjNum 343 snap 0 seq_num 343) 2024-09-13T10:12:09.642 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:09.637+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:09.675 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:09.675 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:12:09.676 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:12:09.676 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:09.688 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T10:12:09.744 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:12:09.742+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:12:09.817 INFO:teuthology.orchestra.run.smithi073.stderr:set pool 4 pg_num to 16 2024-09-13T10:12:09.828 INFO:tasks.rados.rados.0.smithi195.stdout:1802: seq_num 395 ranges {638765=457358,1748152=604803} 2024-09-13T10:12:09.834 INFO:tasks.rados.rados.0.smithi195.stdout:1802: writing smithi19530714-12 from 638765 to 1096123 tid 1 2024-09-13T10:12:09.841 INFO:tasks.rados.rados.0.smithi195.stdout:1802: writing smithi19530714-12 from 1748152 to 2352955 tid 2 2024-09-13T10:12:09.841 INFO:tasks.rados.rados.0.smithi195.stdout:1798: done (4 left) 2024-09-13T10:12:09.842 INFO:tasks.rados.rados.0.smithi195.stdout:1799: done (3 left) 2024-09-13T10:12:09.842 INFO:tasks.rados.rados.0.smithi195.stdout:1800: done (2 left) 2024-09-13T10:12:09.842 INFO:tasks.rados.rados.0.smithi195.stdout:1803: tier_evict oid 35 2024-09-13T10:12:09.842 INFO:tasks.rados.rados.0.smithi195.stdout:1803: tier_evict oid 35 snap -1 2024-09-13T10:12:09.842 INFO:tasks.rados.rados.0.smithi195.stdout:1801: finishing write tid 1 to smithi19530714-29 2024-09-13T10:12:09.842 INFO:tasks.rados.rados.0.smithi195.stdout:1801: finishing write tid 2 to smithi19530714-29 2024-09-13T10:12:09.842 INFO:tasks.rados.rados.0.smithi195.stdout:1801: finishing write tid 3 to smithi19530714-29 2024-09-13T10:12:09.842 INFO:tasks.rados.rados.0.smithi195.stdout:1801: finishing write tid 4 to smithi19530714-29 2024-09-13T10:12:09.842 INFO:tasks.rados.rados.0.smithi195.stdout:1801: oid 29 updating version 0 to 121 2024-09-13T10:12:09.842 INFO:tasks.rados.rados.0.smithi195.stdout:1801: oid 29 version 121 is already newer than 120 2024-09-13T10:12:09.842 INFO:tasks.rados.rados.0.smithi195.stdout:1801: oid 29 version 121 is already newer than 119 2024-09-13T10:12:09.842 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 121 (ObjNum 394 snap 0 seq_num 394) dirty exists 2024-09-13T10:12:09.843 INFO:tasks.rados.rados.0.smithi195.stdout:1801: left oid 29 (ObjNum 394 snap 0 seq_num 394) 2024-09-13T10:12:09.843 INFO:tasks.rados.rados.0.smithi195.stdout:1802: finishing write tid 1 to smithi19530714-12 2024-09-13T10:12:09.843 INFO:tasks.rados.rados.0.smithi195.stdout:1801: done (2 left) 2024-09-13T10:12:09.843 INFO:tasks.rados.rados.0.smithi195.stdout:1804: write (excl) oid 10 current snap is 0 2024-09-13T10:12:09.843 INFO:tasks.rados.rados.0.smithi195.stdout:1804: seq_num 396 ranges {655572=648960,1922987=732585} 2024-09-13T10:12:09.844 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:09.842+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:09.848 INFO:tasks.rados.rados.0.smithi195.stdout:1804: writing smithi19530714-10 from 655572 to 1304532 tid 1 2024-09-13T10:12:09.858 INFO:tasks.rados.rados.0.smithi195.stdout:1804: writing smithi19530714-10 from 1922987 to 2655572 tid 2 2024-09-13T10:12:09.858 INFO:tasks.rados.rados.0.smithi195.stdout:1805: read oid 25 snap -1 2024-09-13T10:12:09.858 INFO:tasks.rados.rados.0.smithi195.stdout:1805: expect (ObjNum 324 snap 0 seq_num 324) 2024-09-13T10:12:09.858 INFO:tasks.rados.rados.0.smithi195.stdout:1806: read oid 11 snap -1 2024-09-13T10:12:09.858 INFO:tasks.rados.rados.0.smithi195.stdout:1806: expect (ObjNum 359 snap 0 seq_num 359) 2024-09-13T10:12:09.858 INFO:tasks.rados.rados.0.smithi195.stdout:1807: read oid 24 snap -1 2024-09-13T10:12:09.858 INFO:tasks.rados.rados.0.smithi195.stdout:1807: expect (ObjNum 370 snap 0 seq_num 370) 2024-09-13T10:12:09.858 INFO:tasks.rados.rados.0.smithi195.stdout:1808: read oid 43 snap -1 2024-09-13T10:12:09.858 INFO:tasks.rados.rados.0.smithi195.stdout:1808: expect (ObjNum 344 snap 0 seq_num 344) 2024-09-13T10:12:09.858 INFO:tasks.rados.rados.0.smithi195.stdout:1809: read oid 46 snap -1 2024-09-13T10:12:09.859 INFO:tasks.rados.rados.0.smithi195.stdout:1809: expect (ObjNum 329 snap 0 seq_num 329) 2024-09-13T10:12:09.859 INFO:tasks.rados.rados.0.smithi195.stdout:1810: write oid 45 current snap is 0 2024-09-13T10:12:09.859 INFO:tasks.rados.rados.0.smithi195.stdout:1803: got (22) Invalid argument 2024-09-13T10:12:09.859 INFO:tasks.rados.rados.0.smithi195.stdout:1802: finishing write tid 2 to smithi19530714-12 2024-09-13T10:12:09.859 INFO:tasks.rados.rados.0.smithi195.stdout:1802: finishing write tid 3 to smithi19530714-12 2024-09-13T10:12:09.859 INFO:tasks.rados.rados.0.smithi195.stdout:1802: finishing write tid 4 to smithi19530714-12 2024-09-13T10:12:09.859 INFO:tasks.rados.rados.0.smithi195.stdout:1802: oid 12 updating version 0 to 124 2024-09-13T10:12:09.859 INFO:tasks.rados.rados.0.smithi195.stdout:1802: oid 12 version 124 is already newer than 122 2024-09-13T10:12:09.859 INFO:tasks.rados.rados.0.smithi195.stdout:1802: oid 12 version 124 is already newer than 123 2024-09-13T10:12:09.859 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 124 (ObjNum 395 snap 0 seq_num 395) dirty exists 2024-09-13T10:12:09.859 INFO:tasks.rados.rados.0.smithi195.stdout:1802: left oid 12 (ObjNum 395 snap 0 seq_num 395) 2024-09-13T10:12:09.859 INFO:tasks.rados.rados.0.smithi195.stdout:1810: seq_num 397 ranges {672379=724209,1865116=608472} 2024-09-13T10:12:09.860 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:09.860 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:12:09.860 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:12:09.860 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:12:09.860 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:09.867 INFO:tasks.rados.rados.0.smithi195.stdout:1810: writing smithi19530714-45 from 672379 to 1396588 tid 1 2024-09-13T10:12:09.876 INFO:tasks.rados.rados.0.smithi195.stdout:1810: writing smithi19530714-45 from 1865116 to 2473588 tid 2 2024-09-13T10:12:09.876 INFO:tasks.rados.rados.0.smithi195.stdout:1802: done (8 left) 2024-09-13T10:12:09.876 INFO:tasks.rados.rados.0.smithi195.stdout:1803: done (7 left) 2024-09-13T10:12:09.876 INFO:tasks.rados.rados.0.smithi195.stdout:1811: read oid 3 snap -1 2024-09-13T10:12:09.876 INFO:tasks.rados.rados.0.smithi195.stdout:1811: expect (ObjNum 328 snap 0 seq_num 328) 2024-09-13T10:12:09.876 INFO:tasks.rados.rados.0.smithi195.stdout:1812: read oid 19 snap -1 2024-09-13T10:12:09.876 INFO:tasks.rados.rados.0.smithi195.stdout:1812: expect (ObjNum 346 snap 0 seq_num 346) 2024-09-13T10:12:09.876 INFO:tasks.rados.rados.0.smithi195.stdout:1813: tier_evict oid 50 2024-09-13T10:12:09.876 INFO:tasks.rados.rados.0.smithi195.stdout:1813: tier_evict oid 50 snap -1 2024-09-13T10:12:09.876 INFO:tasks.rados.rados.0.smithi195.stdout:1809: expect (ObjNum 329 snap 0 seq_num 329) 2024-09-13T10:12:09.878 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T10:12:09.945 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:12:09.938+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:12:09.957 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:10.001 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:12:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:12:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:12:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:211 3.0 e38/30 3:0596651d:::smithi19530714-12 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:12:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:12:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.988874343999996, 2024-09-13T10:12:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013316957000000001, 2024-09-13T10:12:10.001 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:12:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:12:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.002 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008321+0000", 2024-09-13T10:12:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-13T10:12:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011069+0000", 2024-09-13T10:12:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002747893 2024-09-13T10:12:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011113+0000", 2024-09-13T10:12:10.003 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3921000000000002e-05 2024-09-13T10:12:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011140+0000", 2024-09-13T10:12:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7662999999999999e-05 2024-09-13T10:12:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011313+0000", 2024-09-13T10:12:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017232200000000001 2024-09-13T10:12:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.004 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011366+0000", 2024-09-13T10:12:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3888000000000003e-05 2024-09-13T10:12:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021577+0000", 2024-09-13T10:12:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010210472999999999 2024-09-13T10:12:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021636+0000", 2024-09-13T10:12:10.005 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8953999999999998e-05 2024-09-13T10:12:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:212 3.0 e38/30 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:12:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:12:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.985557237999998, 2024-09-13T10:12:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014217904, 2024-09-13T10:12:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.006 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:12:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:12:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011638+0000", 2024-09-13T10:12:10.007 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.672e-06 2024-09-13T10:12:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013289+0000", 2024-09-13T10:12:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001651511 2024-09-13T10:12:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013310+0000", 2024-09-13T10:12:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0917999999999999e-05 2024-09-13T10:12:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.008 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013322+0000", 2024-09-13T10:12:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2067e-05 2024-09-13T10:12:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.016882+0000", 2024-09-13T10:12:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035599130000000001 2024-09-13T10:12:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.016934+0000", 2024-09-13T10:12:10.009 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1715999999999997e-05 2024-09-13T10:12:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025787+0000", 2024-09-13T10:12:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088531809999999999 2024-09-13T10:12:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025854+0000", 2024-09-13T10:12:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6926000000000003e-05 2024-09-13T10:12:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.010 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:207 3.0 e38/30 3:332cd44b:::smithi19530714-8 low_tier:head v 38'3, mlcod=0'0)", 2024-09-13T10:12:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:12:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.979763797000004, 2024-09-13T10:12:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016660892, 2024-09-13T10:12:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:12:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.011 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:12:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017432+0000", 2024-09-13T10:12:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1490000000000001e-06 2024-09-13T10:12:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.012 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021310+0000", 2024-09-13T10:12:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003877985 2024-09-13T10:12:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021351+0000", 2024-09-13T10:12:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1105e-05 2024-09-13T10:12:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021376+0000", 2024-09-13T10:12:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5185999999999999e-05 2024-09-13T10:12:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.013 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021727+0000", 2024-09-13T10:12:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035131300000000002 2024-09-13T10:12:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021792+0000", 2024-09-13T10:12:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4368000000000004e-05 2024-09-13T10:12:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034027+0000", 2024-09-13T10:12:10.014 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012235582 2024-09-13T10:12:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034090+0000", 2024-09-13T10:12:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3204e-05 2024-09-13T10:12:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:216 3.0 e38/30 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:12:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:12:10.015 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.973198226999997, 2024-09-13T10:12:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011044523000000001, 2024-09-13T10:12:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:12:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.016 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:12:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023997+0000", 2024-09-13T10:12:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.533e-06 2024-09-13T10:12:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024872+0000", 2024-09-13T10:12:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087517699999999999 2024-09-13T10:12:10.017 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024894+0000", 2024-09-13T10:12:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2571000000000001e-05 2024-09-13T10:12:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024909+0000", 2024-09-13T10:12:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5158e-05 2024-09-13T10:12:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025979+0000", 2024-09-13T10:12:10.018 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010692329999999999 2024-09-13T10:12:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.026027+0000", 2024-09-13T10:12:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8158000000000002e-05 2024-09-13T10:12:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034992+0000", 2024-09-13T10:12:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0089650960000000005 2024-09-13T10:12:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.019 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.035040+0000", 2024-09-13T10:12:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7596999999999998e-05 2024-09-13T10:12:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:225 3.0 e38/30 3:26960251:::smithi19530714-26 low_tier:head v 38'6, mlcod=38'3)", 2024-09-13T10:12:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:12:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.92630115, 2024-09-13T10:12:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013205889, 2024-09-13T10:12:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.020 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:12:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:12:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.021 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070894+0000", 2024-09-13T10:12:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5e-06 2024-09-13T10:12:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072562+0000", 2024-09-13T10:12:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016683469999999999 2024-09-13T10:12:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.022 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072632+0000", 2024-09-13T10:12:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9665000000000005e-05 2024-09-13T10:12:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072661+0000", 2024-09-13T10:12:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9714999999999999e-05 2024-09-13T10:12:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072800+0000", 2024-09-13T10:12:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013886000000000001 2024-09-13T10:12:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.023 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072828+0000", 2024-09-13T10:12:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7886e-05 2024-09-13T10:12:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084041+0000", 2024-09-13T10:12:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011213064999999999 2024-09-13T10:12:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.024 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084098+0000", 2024-09-13T10:12:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6851000000000002e-05 2024-09-13T10:12:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:228 3.0 e38/30 3:0168a995:::smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'7, mlcod=38'3)", 2024-09-13T10:12:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:12:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.922517569999997, 2024-09-13T10:12:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094647819999999997, 2024-09-13T10:12:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.025 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:12:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:12:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.026 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074677+0000", 2024-09-13T10:12:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7099999999999999e-06 2024-09-13T10:12:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076599+0000", 2024-09-13T10:12:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019212459999999999 2024-09-13T10:12:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076616+0000", 2024-09-13T10:12:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7008999999999999e-05 2024-09-13T10:12:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.027 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076641+0000", 2024-09-13T10:12:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5380000000000001e-05 2024-09-13T10:12:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076772+0000", 2024-09-13T10:12:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013101599999999999 2024-09-13T10:12:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076867+0000", 2024-09-13T10:12:10.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5262999999999999e-05 2024-09-13T10:12:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084106+0000", 2024-09-13T10:12:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0072390859999999996 2024-09-13T10:12:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084140+0000", 2024-09-13T10:12:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4072000000000002e-05 2024-09-13T10:12:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.029 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:252 4.6 4:6ddb2b23:::smithi19530714-3:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:12:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.875129672, 2024-09-13T10:12:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.03110711, 2024-09-13T10:12:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.030 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 252 2024-09-13T10:12:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:12:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:12:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122066+0000", 2024-09-13T10:12:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4619999999999999e-06 2024-09-13T10:12:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122097+0000", 2024-09-13T10:12:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0488999999999999e-05 2024-09-13T10:12:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122120+0000", 2024-09-13T10:12:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3649999999999999e-05 2024-09-13T10:12:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122130+0000", 2024-09-13T10:12:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7950000000000002e-06 2024-09-13T10:12:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122212+0000", 2024-09-13T10:12:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2090999999999997e-05 2024-09-13T10:12:10.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122317+0000", 2024-09-13T10:12:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104954 2024-09-13T10:12:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.151952+0000", 2024-09-13T10:12:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.029634908000000001 2024-09-13T10:12:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152580+0000", 2024-09-13T10:12:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062796499999999997 2024-09-13T10:12:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152891+0000", 2024-09-13T10:12:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000310747 2024-09-13T10:12:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153077+0000", 2024-09-13T10:12:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018588200000000001 2024-09-13T10:12:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153130+0000", 2024-09-13T10:12:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3016000000000002e-05 2024-09-13T10:12:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153171+0000", 2024-09-13T10:12:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1151e-05 2024-09-13T10:12:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:255 4.6 4:6150ba38:::smithi19530714-6:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:12:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.782715468000006, 2024-09-13T10:12:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011708387000000001, 2024-09-13T10:12:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.037 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 255 2024-09-13T10:12:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:12:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:12:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214482+0000", 2024-09-13T10:12:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8139999999999998e-06 2024-09-13T10:12:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214498+0000", 2024-09-13T10:12:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6636000000000001e-05 2024-09-13T10:12:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214537+0000", 2024-09-13T10:12:10.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8668999999999999e-05 2024-09-13T10:12:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214562+0000", 2024-09-13T10:12:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5242999999999999e-05 2024-09-13T10:12:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214770+0000", 2024-09-13T10:12:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020771299999999999 2024-09-13T10:12:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.040 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214996+0000", 2024-09-13T10:12:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022615999999999999 2024-09-13T10:12:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.223455+0000", 2024-09-13T10:12:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0084591960000000004 2024-09-13T10:12:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225042+0000", 2024-09-13T10:12:10.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001586896 2024-09-13T10:12:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225719+0000", 2024-09-13T10:12:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00067701700000000003 2024-09-13T10:12:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225845+0000", 2024-09-13T10:12:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012626299999999999 2024-09-13T10:12:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226070+0000", 2024-09-13T10:12:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022411600000000001 2024-09-13T10:12:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226186+0000", 2024-09-13T10:12:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011666400000000001 2024-09-13T10:12:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:268 4.6 4:6fb28ed4:::smithi19530714-19:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:10.043 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:12:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.781796908000004, 2024-09-13T10:12:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041608707000000002, 2024-09-13T10:12:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 268 2024-09-13T10:12:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.044 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:12:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:12:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215399+0000", 2024-09-13T10:12:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8669999999999998e-06 2024-09-13T10:12:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215416+0000", 2024-09-13T10:12:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6792000000000001e-05 2024-09-13T10:12:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215480+0000", 2024-09-13T10:12:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3805000000000003e-05 2024-09-13T10:12:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215499+0000", 2024-09-13T10:12:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9349999999999999e-05 2024-09-13T10:12:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215600+0000", 2024-09-13T10:12:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010119 2024-09-13T10:12:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215715+0000", 2024-09-13T10:12:10.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011491 2024-09-13T10:12:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.253967+0000", 2024-09-13T10:12:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038251470000000003 2024-09-13T10:12:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.255748+0000", 2024-09-13T10:12:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017815089999999999 2024-09-13T10:12:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256188+0000", 2024-09-13T10:12:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00043959799999999997 2024-09-13T10:12:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256840+0000", 2024-09-13T10:12:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000652591 2024-09-13T10:12:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256932+0000", 2024-09-13T10:12:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1691e-05 2024-09-13T10:12:10.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.257005+0000", 2024-09-13T10:12:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2934000000000003e-05 2024-09-13T10:12:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:305 4.6 4:6150ba38:::smithi19530714-6:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:12:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.728566215000001, 2024-09-13T10:12:10.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010323297, 2024-09-13T10:12:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 305 2024-09-13T10:12:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:12:10.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:12:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268630+0000", 2024-09-13T10:12:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.548e-06 2024-09-13T10:12:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268644+0000", 2024-09-13T10:12:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.394e-05 2024-09-13T10:12:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268672+0000", 2024-09-13T10:12:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8242000000000002e-05 2024-09-13T10:12:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268689+0000", 2024-09-13T10:12:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7062e-05 2024-09-13T10:12:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268789+0000", 2024-09-13T10:12:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010026600000000001 2024-09-13T10:12:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268905+0000", 2024-09-13T10:12:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115908 2024-09-13T10:12:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.276776+0000", 2024-09-13T10:12:10.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078712000000000001 2024-09-13T10:12:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278412+0000", 2024-09-13T10:12:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016358670000000001 2024-09-13T10:12:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278664+0000", 2024-09-13T10:12:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025162600000000002 2024-09-13T10:12:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278785+0000", 2024-09-13T10:12:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012162100000000001 2024-09-13T10:12:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278895+0000", 2024-09-13T10:12:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109774 2024-09-13T10:12:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278950+0000", 2024-09-13T10:12:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5242999999999999e-05 2024-09-13T10:12:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.056 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1023 4.16 4:6fb28ed4:::smithi19530714-19:head [write 1798066~477679] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.078200+0000", 2024-09-13T10:12:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.918992854999999, 2024-09-13T10:12:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0095557869999999996, 2024-09-13T10:12:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.057 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1023 2024-09-13T10:12:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078200+0000", 2024-09-13T10:12:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078200+0000", 2024-09-13T10:12:10.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078202+0000", 2024-09-13T10:12:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2529999999999999e-06 2024-09-13T10:12:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078505+0000", 2024-09-13T10:12:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000303228 2024-09-13T10:12:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078519+0000", 2024-09-13T10:12:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3918e-05 2024-09-13T10:12:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078526+0000", 2024-09-13T10:12:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.311e-06 2024-09-13T10:12:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078577+0000", 2024-09-13T10:12:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1125999999999998e-05 2024-09-13T10:12:10.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078682+0000", 2024-09-13T10:12:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104671 2024-09-13T10:12:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078864+0000", 2024-09-13T10:12:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018166999999999999 2024-09-13T10:12:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.080583+0000", 2024-09-13T10:12:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001719138 2024-09-13T10:12:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.085104+0000", 2024-09-13T10:12:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045216950000000001 2024-09-13T10:12:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.087686+0000", 2024-09-13T10:12:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025817499999999998 2024-09-13T10:12:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.087721+0000", 2024-09-13T10:12:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5009999999999999e-05 2024-09-13T10:12:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.087756+0000", 2024-09-13T10:12:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5017e-05 2024-09-13T10:12:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1040 4.6 4:61ec6a5a:::smithi19530714-35:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.095747+0000", 2024-09-13T10:12:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.901445878000004, 2024-09-13T10:12:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036815318999999999, 2024-09-13T10:12:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1040 2024-09-13T10:12:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.064 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095747+0000", 2024-09-13T10:12:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095747+0000", 2024-09-13T10:12:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095749+0000", 2024-09-13T10:12:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9039999999999999e-06 2024-09-13T10:12:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095757+0000", 2024-09-13T10:12:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6660000000000008e-06 2024-09-13T10:12:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095784+0000", 2024-09-13T10:12:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6860999999999999e-05 2024-09-13T10:12:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095794+0000", 2024-09-13T10:12:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7149999999999993e-06 2024-09-13T10:12:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095860+0000", 2024-09-13T10:12:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6699999999999995e-05 2024-09-13T10:12:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095984+0000", 2024-09-13T10:12:10.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123676 2024-09-13T10:12:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.132563+0000", 2024-09-13T10:12:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036578797000000003 2024-09-13T10:12:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1045 4.6 4:6150ba38:::smithi19530714-6:head [write 527753~708201,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.313424+0000", 2024-09-13T10:12:10.068 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.683768943999993, 2024-09-13T10:12:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011098433, 2024-09-13T10:12:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.069 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1045 2024-09-13T10:12:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313424+0000", 2024-09-13T10:12:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313424+0000", 2024-09-13T10:12:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313427+0000", 2024-09-13T10:12:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0230000000000001e-06 2024-09-13T10:12:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313756+0000", 2024-09-13T10:12:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032905500000000003 2024-09-13T10:12:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313789+0000", 2024-09-13T10:12:10.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2359e-05 2024-09-13T10:12:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313807+0000", 2024-09-13T10:12:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8130000000000001e-05 2024-09-13T10:12:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313953+0000", 2024-09-13T10:12:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014562200000000001 2024-09-13T10:12:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.314185+0000", 2024-09-13T10:12:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023246399999999999 2024-09-13T10:12:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.314442+0000", 2024-09-13T10:12:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000257023 2024-09-13T10:12:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.317258+0000", 2024-09-13T10:12:10.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0028160149999999998 2024-09-13T10:12:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.319440+0000", 2024-09-13T10:12:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021817149999999999 2024-09-13T10:12:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.324214+0000", 2024-09-13T10:12:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047739189999999997 2024-09-13T10:12:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.074 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.324440+0000", 2024-09-13T10:12:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000225944 2024-09-13T10:12:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.324523+0000", 2024-09-13T10:12:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3164000000000005e-05 2024-09-13T10:12:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.075 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2603 4.6 4:6150ba38:::smithi19530714-6:head [setxattr _header (55) in=7b,truncate 3848803] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T10:12:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:26.522327+0000", 2024-09-13T10:12:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 43.474866136999999, 2024-09-13T10:12:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0099854200000000001, 2024-09-13T10:12:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2603 2024-09-13T10:12:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522327+0000", 2024-09-13T10:12:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522327+0000", 2024-09-13T10:12:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522329+0000", 2024-09-13T10:12:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.813e-06 2024-09-13T10:12:10.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522338+0000", 2024-09-13T10:12:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1269999999999992e-06 2024-09-13T10:12:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522359+0000", 2024-09-13T10:12:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1242000000000001e-05 2024-09-13T10:12:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522374+0000", 2024-09-13T10:12:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5085999999999999e-05 2024-09-13T10:12:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.523648+0000", 2024-09-13T10:12:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0012740049999999999 2024-09-13T10:12:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.523917+0000", 2024-09-13T10:12:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026876100000000001 2024-09-13T10:12:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.524319+0000", 2024-09-13T10:12:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040155200000000001 2024-09-13T10:12:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.525257+0000", 2024-09-13T10:12:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00093831600000000002 2024-09-13T10:12:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532117+0000", 2024-09-13T10:12:10.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068596530000000003 2024-09-13T10:12:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532185+0000", 2024-09-13T10:12:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8785e-05 2024-09-13T10:12:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532243+0000", 2024-09-13T10:12:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7627999999999998e-05 2024-09-13T10:12:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532313+0000", 2024-09-13T10:12:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9451999999999998e-05 2024-09-13T10:12:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2604 4.6 4:6150ba38:::smithi19530714-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-09-13T10:12:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:26.522408+0000", 2024-09-13T10:12:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 43.474785670000003, 2024-09-13T10:12:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010226071999999999, 2024-09-13T10:12:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.082 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2604 2024-09-13T10:12:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522408+0000", 2024-09-13T10:12:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522408+0000", 2024-09-13T10:12:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522409+0000", 2024-09-13T10:12:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3999999999999999e-06 2024-09-13T10:12:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522415+0000", 2024-09-13T10:12:10.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7790000000000001e-06 2024-09-13T10:12:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522433+0000", 2024-09-13T10:12:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7972999999999999e-05 2024-09-13T10:12:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522449+0000", 2024-09-13T10:12:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5899000000000001e-05 2024-09-13T10:12:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.524993+0000", 2024-09-13T10:12:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002544499 2024-09-13T10:12:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.525196+0000", 2024-09-13T10:12:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202721 2024-09-13T10:12:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532374+0000", 2024-09-13T10:12:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0071786259999999996 2024-09-13T10:12:10.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532498+0000", 2024-09-13T10:12:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012336899999999999 2024-09-13T10:12:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532634+0000", 2024-09-13T10:12:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013580600000000001 2024-09-13T10:12:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2991 4.6 4:61ec6a5a:::smithi19530714-35:head [write 453855~696987,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T10:12:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:37.300931+0000", 2024-09-13T10:12:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.696262650000001, 2024-09-13T10:12:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012633428, 2024-09-13T10:12:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2991 2024-09-13T10:12:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.088 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.300931+0000", 2024-09-13T10:12:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.300931+0000", 2024-09-13T10:12:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.300934+0000", 2024-09-13T10:12:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.856e-06 2024-09-13T10:12:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.301789+0000", 2024-09-13T10:12:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00085478099999999999 2024-09-13T10:12:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.301821+0000", 2024-09-13T10:12:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1301999999999999e-05 2024-09-13T10:12:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.301838+0000", 2024-09-13T10:12:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7229999999999999e-05 2024-09-13T10:12:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.302009+0000", 2024-09-13T10:12:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017093699999999999 2024-09-13T10:12:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op must wait for map", 2024-09-13T10:12:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.302027+0000", 2024-09-13T10:12:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8487000000000001e-05 2024-09-13T10:12:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.304197+0000", 2024-09-13T10:12:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002169706 2024-09-13T10:12:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.304321+0000", 2024-09-13T10:12:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012425899999999999 2024-09-13T10:12:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.304430+0000", 2024-09-13T10:12:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109323 2024-09-13T10:12:10.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.307797+0000", 2024-09-13T10:12:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033668000000000001 2024-09-13T10:12:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313063+0000", 2024-09-13T10:12:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.005265947 2024-09-13T10:12:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313113+0000", 2024-09-13T10:12:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9592000000000002e-05 2024-09-13T10:12:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313513+0000", 2024-09-13T10:12:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040035 2024-09-13T10:12:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313564+0000", 2024-09-13T10:12:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0858e-05 2024-09-13T10:12:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.094 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2992 4.6 4:61ec6a5a:::smithi19530714-35:head [write 1890787~548083] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T10:12:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:37.308827+0000", 2024-09-13T10:12:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.688366203999998, 2024-09-13T10:12:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011134115, 2024-09-13T10:12:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.095 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2992 2024-09-13T10:12:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.308827+0000", 2024-09-13T10:12:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.308827+0000", 2024-09-13T10:12:10.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.308830+0000", 2024-09-13T10:12:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4000000000000001e-06 2024-09-13T10:12:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309309+0000", 2024-09-13T10:12:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000478901 2024-09-13T10:12:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309348+0000", 2024-09-13T10:12:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8182000000000001e-05 2024-09-13T10:12:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309376+0000", 2024-09-13T10:12:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8082e-05 2024-09-13T10:12:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309433+0000", 2024-09-13T10:12:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6981000000000002e-05 2024-09-13T10:12:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309508+0000", 2024-09-13T10:12:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5378999999999998e-05 2024-09-13T10:12:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309603+0000", 2024-09-13T10:12:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4871999999999996e-05 2024-09-13T10:12:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313952+0000", 2024-09-13T10:12:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0043492139999999997 2024-09-13T10:12:10.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.317432+0000", 2024-09-13T10:12:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034796969999999999 2024-09-13T10:12:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.318860+0000", 2024-09-13T10:12:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001428627 2024-09-13T10:12:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.319830+0000", 2024-09-13T10:12:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00096928100000000001 2024-09-13T10:12:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.319961+0000", 2024-09-13T10:12:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013149900000000001 2024-09-13T10:12:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2993 4.6 4:61ec6a5a:::smithi19530714-35:head [setxattr _header (56) in=7b,truncate 2453855] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T10:12:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:37.309438+0000", 2024-09-13T10:12:10.101 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.687755363999997, 2024-09-13T10:12:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010978862000000001, 2024-09-13T10:12:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2993 2024-09-13T10:12:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309438+0000", 2024-09-13T10:12:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309438+0000", 2024-09-13T10:12:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309439+0000", 2024-09-13T10:12:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3370000000000001e-06 2024-09-13T10:12:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309449+0000", 2024-09-13T10:12:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0159e-05 2024-09-13T10:12:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309463+0000", 2024-09-13T10:12:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3305999999999999e-05 2024-09-13T10:12:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309473+0000", 2024-09-13T10:12:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8169999999999994e-06 2024-09-13T10:12:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.310756+0000", 2024-09-13T10:12:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0012835100000000001 2024-09-13T10:12:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.311065+0000", 2024-09-13T10:12:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030920900000000001 2024-09-13T10:12:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.311478+0000", 2024-09-13T10:12:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000413106 2024-09-13T10:12:10.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313969+0000", 2024-09-13T10:12:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002490216 2024-09-13T10:12:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.318098+0000", 2024-09-13T10:12:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0041298020000000001 2024-09-13T10:12:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320032+0000", 2024-09-13T10:12:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001933349 2024-09-13T10:12:10.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320356+0000", 2024-09-13T10:12:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032416699999999999 2024-09-13T10:12:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320417+0000", 2024-09-13T10:12:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0884000000000002e-05 2024-09-13T10:12:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.107 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2994 4.6 4:61ec6a5a:::smithi19530714-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e75)", 2024-09-13T10:12:10.108 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:37.309490+0000", 2024-09-13T10:12:10.108 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 32.687703046000003, 2024-09-13T10:12:10.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011140761000000001, 2024-09-13T10:12:10.108 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.108 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.108 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:10.108 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.108 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.108 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.108 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2994 2024-09-13T10:12:10.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.108 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309490+0000", 2024-09-13T10:12:10.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309490+0000", 2024-09-13T10:12:10.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309491+0000", 2024-09-13T10:12:10.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2699999999999998e-07 2024-09-13T10:12:10.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309495+0000", 2024-09-13T10:12:10.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7459999999999998e-06 2024-09-13T10:12:10.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309505+0000", 2024-09-13T10:12:10.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7969999999999995e-06 2024-09-13T10:12:10.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309516+0000", 2024-09-13T10:12:10.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1222e-05 2024-09-13T10:12:10.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313661+0000", 2024-09-13T10:12:10.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0041448309999999999 2024-09-13T10:12:10.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:10.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313920+0000", 2024-09-13T10:12:10.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025941099999999997 2024-09-13T10:12:10.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320456+0000", 2024-09-13T10:12:10.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0065354030000000004 2024-09-13T10:12:10.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320539+0000", 2024-09-13T10:12:10.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3935000000000002e-05 2024-09-13T10:12:10.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320631+0000", 2024-09-13T10:12:10.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1489000000000004e-05 2024-09-13T10:12:10.112 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.112 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.113 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.113 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:3745 3.0 e90/30 3:161dd296:::f6d96d7a8dac6a7d56d3d9df9ef9c133dce9c515f90d85faf1e037f770fc9485:head v 90'300, mlcod=89'298)", 2024-09-13T10:12:10.113 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:54.612303+0000", 2024-09-13T10:12:10.113 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 15.384890115999999, 2024-09-13T10:12:10.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011846107999999999, 2024-09-13T10:12:10.113 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.113 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.113 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.113 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.612303+0000", 2024-09-13T10:12:10.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.612303+0000", 2024-09-13T10:12:10.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.114 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.114 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.612305+0000", 2024-09-13T10:12:10.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.866e-06 2024-09-13T10:12:10.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.614283+0000", 2024-09-13T10:12:10.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001977707 2024-09-13T10:12:10.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.614313+0000", 2024-09-13T10:12:10.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0443999999999999e-05 2024-09-13T10:12:10.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.614327+0000", 2024-09-13T10:12:10.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3506000000000001e-05 2024-09-13T10:12:10.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.614496+0000", 2024-09-13T10:12:10.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016885500000000001 2024-09-13T10:12:10.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.614561+0000", 2024-09-13T10:12:10.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5612999999999997e-05 2024-09-13T10:12:10.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.624063+0000", 2024-09-13T10:12:10.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0095017829999999998 2024-09-13T10:12:10.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.624149+0000", 2024-09-13T10:12:10.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6334000000000004e-05 2024-09-13T10:12:10.117 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.117 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.117 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.117 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.117 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.117 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:10.119 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-13T10:12:10.119 INFO:tasks.rados.rados.0.smithi195.stdout:1806: expect (ObjNum 359 snap 0 seq_num 359) 2024-09-13T10:12:10.120 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:10.120 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:10.114+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:10.156 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 236754 2024-09-13T10:12:10.174 INFO:tasks.rados.rados.0.smithi195.stdout:1804: finishing write tid 1 to smithi19530714-10 2024-09-13T10:12:10.174 INFO:tasks.rados.rados.0.smithi195.stdout:1806: done (9 left) 2024-09-13T10:12:10.174 INFO:tasks.rados.rados.0.smithi195.stdout:1809: done (8 left) 2024-09-13T10:12:10.174 INFO:tasks.rados.rados.0.smithi195.stdout:1814: write oid 14 current snap is 0 2024-09-13T10:12:10.174 INFO:tasks.rados.rados.0.smithi195.stdout:1804: finishing write tid 2 to smithi19530714-10 2024-09-13T10:12:10.174 INFO:tasks.rados.rados.0.smithi195.stdout:1814: seq_num 398 ranges {689186=799458,1923598=752130} 2024-09-13T10:12:10.182 INFO:tasks.rados.rados.0.smithi195.stdout:1814: writing smithi19530714-14 from 689186 to 1488644 tid 1 2024-09-13T10:12:10.189 INFO:tasks.rados.rados.0.smithi195.stdout:1814: writing smithi19530714-14 from 1923598 to 2675728 tid 2 2024-09-13T10:12:10.189 INFO:tasks.rados.rados.0.smithi195.stdout:1815: tier_evict oid 37 2024-09-13T10:12:10.189 INFO:tasks.rados.rados.0.smithi195.stdout:1815: tier_evict oid 37 snap -1 2024-09-13T10:12:10.189 INFO:tasks.rados.rados.0.smithi195.stdout:1804: finishing write tid 3 to smithi19530714-10 2024-09-13T10:12:10.189 INFO:tasks.rados.rados.0.smithi195.stdout:1804: finishing write tid 4 to smithi19530714-10 2024-09-13T10:12:10.189 INFO:tasks.rados.rados.0.smithi195.stdout:1804: oid 10 updating version 0 to 40 2024-09-13T10:12:10.189 INFO:tasks.rados.rados.0.smithi195.stdout:1804: oid 10 version 40 is already newer than 39 2024-09-13T10:12:10.189 INFO:tasks.rados.rados.0.smithi195.stdout:1804: oid 10 version 40 is already newer than 38 2024-09-13T10:12:10.190 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 40 (ObjNum 396 snap 0 seq_num 396) dirty exists 2024-09-13T10:12:10.190 INFO:tasks.rados.rados.0.smithi195.stdout:1804: left oid 10 (ObjNum 396 snap 0 seq_num 396) 2024-09-13T10:12:10.190 INFO:tasks.rados.rados.0.smithi195.stdout:1813: got (22) Invalid argument 2024-09-13T10:12:10.190 INFO:tasks.rados.rados.0.smithi195.stdout:1808: expect (ObjNum 344 snap 0 seq_num 344) 2024-09-13T10:12:10.190 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:10.219 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:12:10.214+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:12:10.238 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:10.238 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:12:10.238 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:12:10.238 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:10.251 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-13T10:12:10.256 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 373615 2024-09-13T10:12:10.320 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:10.314+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:10.368 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:10.368 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:12:10.368 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:12:10.369 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:12:10.369 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:10.382 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-13T10:12:10.392 INFO:tasks.rados.rados.0.smithi195.stdout:1805: expect (ObjNum 324 snap 0 seq_num 324) 2024-09-13T10:12:10.421 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:10.417+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:10.460 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:10.511 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:10.511 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:12:10.511 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:12:10.511 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:12:10.511 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.511 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:89 4.9 4:96fcf081:::smithi19530714-23:head [write 769754~688184,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:10.512 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.809155+0000", 2024-09-13T10:12:10.512 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.698250168000001, 2024-09-13T10:12:10.512 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011562566, 2024-09-13T10:12:10.512 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.512 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.512 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.512 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.512 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.512 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.512 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 89 2024-09-13T10:12:10.512 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.512 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.809155+0000", 2024-09-13T10:12:10.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.809155+0000", 2024-09-13T10:12:10.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.513 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.513 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.513 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.513 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.809158+0000", 2024-09-13T10:12:10.513 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5770000000000001e-06 2024-09-13T10:12:10.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810210+0000", 2024-09-13T10:12:10.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010519889999999999 2024-09-13T10:12:10.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.514 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810238+0000", 2024-09-13T10:12:10.514 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8449000000000002e-05 2024-09-13T10:12:10.514 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.514 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.514 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810265+0000", 2024-09-13T10:12:10.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6832e-05 2024-09-13T10:12:10.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810374+0000", 2024-09-13T10:12:10.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109274 2024-09-13T10:12:10.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.515 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810519+0000", 2024-09-13T10:12:10.515 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014449299999999999 2024-09-13T10:12:10.515 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.515 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.515 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:12:10.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810684+0000", 2024-09-13T10:12:10.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000165744 2024-09-13T10:12:10.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816442+0000", 2024-09-13T10:12:10.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057578990000000004 2024-09-13T10:12:10.516 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.516 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.516 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.516 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816685+0000", 2024-09-13T10:12:10.516 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024254999999999999 2024-09-13T10:12:10.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820614+0000", 2024-09-13T10:12:10.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0039290970000000003 2024-09-13T10:12:10.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820668+0000", 2024-09-13T10:12:10.517 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3567999999999999e-05 2024-09-13T10:12:10.517 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.517 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.517 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.517 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820718+0000", 2024-09-13T10:12:10.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0093999999999998e-05 2024-09-13T10:12:10.518 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.518 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.518 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.518 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.518 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.518 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:90 4.9 4:96fcf081:::smithi19530714-23:head [write 2021192~348562] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:10.518 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810322+0000", 2024-09-13T10:12:10.518 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.697082933000004, 2024-09-13T10:12:10.518 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01166355, 2024-09-13T10:12:10.518 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.518 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.518 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.518 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.518 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.519 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.519 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 90 2024-09-13T10:12:10.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.519 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810322+0000", 2024-09-13T10:12:10.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.519 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.519 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.519 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.519 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810322+0000", 2024-09-13T10:12:10.519 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810324+0000", 2024-09-13T10:12:10.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.097e-06 2024-09-13T10:12:10.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.520 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810494+0000", 2024-09-13T10:12:10.520 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016961499999999999 2024-09-13T10:12:10.520 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.520 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.520 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810515+0000", 2024-09-13T10:12:10.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0939999999999999e-05 2024-09-13T10:12:10.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810533+0000", 2024-09-13T10:12:10.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7909000000000001e-05 2024-09-13T10:12:10.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.521 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.521 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.814429+0000", 2024-09-13T10:12:10.521 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0038957409999999999 2024-09-13T10:12:10.521 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.521 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.814603+0000", 2024-09-13T10:12:10.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000174294 2024-09-13T10:12:10.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:12:10.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.814804+0000", 2024-09-13T10:12:10.522 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020149799999999999 2024-09-13T10:12:10.522 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.522 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.522 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.522 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820569+0000", 2024-09-13T10:12:10.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0057648389999999999 2024-09-13T10:12:10.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.523 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820761+0000", 2024-09-13T10:12:10.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019123 2024-09-13T10:12:10.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.523 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821893+0000", 2024-09-13T10:12:10.523 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001132051 2024-09-13T10:12:10.523 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.523 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.523 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821949+0000", 2024-09-13T10:12:10.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6832000000000001e-05 2024-09-13T10:12:10.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.524 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.524 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821986+0000", 2024-09-13T10:12:10.524 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6504000000000002e-05 2024-09-13T10:12:10.524 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.524 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.524 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.524 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.524 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.524 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:91 4.9 4:96fcf081:::smithi19530714-23:head [setxattr _header (56) in=7b,truncate 2369754] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:10.524 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810564+0000", 2024-09-13T10:12:10.524 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.696841723000006, 2024-09-13T10:12:10.525 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01178625, 2024-09-13T10:12:10.525 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.525 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.525 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.525 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.525 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.525 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.525 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 91 2024-09-13T10:12:10.525 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.525 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.525 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.525 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.525 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810564+0000", 2024-09-13T10:12:10.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810564+0000", 2024-09-13T10:12:10.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.526 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.526 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810565+0000", 2024-09-13T10:12:10.526 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3960000000000001e-06 2024-09-13T10:12:10.526 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.526 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810574+0000", 2024-09-13T10:12:10.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9339999999999995e-06 2024-09-13T10:12:10.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810590+0000", 2024-09-13T10:12:10.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5858999999999998e-05 2024-09-13T10:12:10.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.527 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.527 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.527 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810604+0000", 2024-09-13T10:12:10.527 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4533e-05 2024-09-13T10:12:10.527 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.528 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816017+0000", 2024-09-13T10:12:10.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0054122390000000001 2024-09-13T10:12:10.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.528 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816092+0000", 2024-09-13T10:12:10.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5537999999999994e-05 2024-09-13T10:12:10.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.528 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.528 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:12:10.528 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816206+0000", 2024-09-13T10:12:10.528 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114298 2024-09-13T10:12:10.528 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820805+0000", 2024-09-13T10:12:10.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004598347 2024-09-13T10:12:10.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821735+0000", 2024-09-13T10:12:10.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000930348 2024-09-13T10:12:10.529 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.529 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.529 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.529 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822118+0000", 2024-09-13T10:12:10.529 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00038271099999999999 2024-09-13T10:12:10.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822208+0000", 2024-09-13T10:12:10.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0271000000000004e-05 2024-09-13T10:12:10.530 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.530 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.530 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.530 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822350+0000", 2024-09-13T10:12:10.530 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014177600000000001 2024-09-13T10:12:10.530 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.530 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.530 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.531 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:92 4.9 4:96fcf081:::smithi19530714-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:10.531 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810624+0000", 2024-09-13T10:12:10.531 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.696781298000005, 2024-09-13T10:12:10.531 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012449517, 2024-09-13T10:12:10.531 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.531 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.531 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:10.531 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.531 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.531 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.531 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 92 2024-09-13T10:12:10.531 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.531 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.531 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810624+0000", 2024-09-13T10:12:10.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810624+0000", 2024-09-13T10:12:10.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.532 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.532 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810625+0000", 2024-09-13T10:12:10.532 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3459999999999999e-06 2024-09-13T10:12:10.532 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.532 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810638+0000", 2024-09-13T10:12:10.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2853e-05 2024-09-13T10:12:10.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810655+0000", 2024-09-13T10:12:10.533 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6370000000000001e-05 2024-09-13T10:12:10.533 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.533 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.533 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.533 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810668+0000", 2024-09-13T10:12:10.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3906e-05 2024-09-13T10:12:10.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816513+0000", 2024-09-13T10:12:10.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058443369999999998 2024-09-13T10:12:10.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:10.534 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816657+0000", 2024-09-13T10:12:10.534 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014419199999999999 2024-09-13T10:12:10.534 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.534 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.534 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822492+0000", 2024-09-13T10:12:10.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.005835107 2024-09-13T10:12:10.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822740+0000", 2024-09-13T10:12:10.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024784600000000003 2024-09-13T10:12:10.535 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.535 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.535 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.535 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823074+0000", 2024-09-13T10:12:10.535 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033356 2024-09-13T10:12:10.535 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.535 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.536 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.536 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.536 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.536 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:129 4.9 4:998d9bf1:::smithi19530714-33:head [write 537824~473380,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:10.536 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.810689+0000", 2024-09-13T10:12:10.536 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.696716523000006, 2024-09-13T10:12:10.536 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01301894, 2024-09-13T10:12:10.536 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.536 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.536 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.536 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.536 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.536 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.537 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 129 2024-09-13T10:12:10.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.537 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810689+0000", 2024-09-13T10:12:10.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.537 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.537 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810689+0000", 2024-09-13T10:12:10.537 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.537 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.537 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.810690+0000", 2024-09-13T10:12:10.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2330000000000001e-06 2024-09-13T10:12:10.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811247+0000", 2024-09-13T10:12:10.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00055654999999999995 2024-09-13T10:12:10.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.538 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.538 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811264+0000", 2024-09-13T10:12:10.538 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7813e-05 2024-09-13T10:12:10.538 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.538 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811280+0000", 2024-09-13T10:12:10.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5676e-05 2024-09-13T10:12:10.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816755+0000", 2024-09-13T10:12:10.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0054754139999999996 2024-09-13T10:12:10.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.539 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.539 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.816940+0000", 2024-09-13T10:12:10.539 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184091 2024-09-13T10:12:10.539 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.539 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.540 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:12:10.540 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.817141+0000", 2024-09-13T10:12:10.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020179099999999999 2024-09-13T10:12:10.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.540 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.540 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821746+0000", 2024-09-13T10:12:10.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004604543 2024-09-13T10:12:10.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.540 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.540 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.540 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823211+0000", 2024-09-13T10:12:10.540 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001464824 2024-09-13T10:12:10.540 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823536+0000", 2024-09-13T10:12:10.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032508599999999998 2024-09-13T10:12:10.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823620+0000", 2024-09-13T10:12:10.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4524999999999995e-05 2024-09-13T10:12:10.541 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.541 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.541 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.541 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823708+0000", 2024-09-13T10:12:10.541 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.7393999999999997e-05 2024-09-13T10:12:10.542 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.542 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.542 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.542 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.542 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.542 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:130 4.9 4:998d9bf1:::smithi19530714-33:head [write 1806012~625998] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:10.542 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.811308+0000", 2024-09-13T10:12:10.542 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.696097558999995, 2024-09-13T10:12:10.542 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012775038000000001, 2024-09-13T10:12:10.542 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.542 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.542 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.542 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.543 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.543 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.543 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 130 2024-09-13T10:12:10.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.543 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811308+0000", 2024-09-13T10:12:10.543 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.543 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.543 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.543 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.543 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811308+0000", 2024-09-13T10:12:10.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811310+0000", 2024-09-13T10:12:10.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9400000000000001e-06 2024-09-13T10:12:10.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.544 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.544 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.544 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.811989+0000", 2024-09-13T10:12:10.544 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00067975999999999996 2024-09-13T10:12:10.544 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812006+0000", 2024-09-13T10:12:10.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6175999999999999e-05 2024-09-13T10:12:10.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812024+0000", 2024-09-13T10:12:10.545 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8813000000000001e-05 2024-09-13T10:12:10.545 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.545 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.545 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.545 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.818639+0000", 2024-09-13T10:12:10.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0066148639999999998 2024-09-13T10:12:10.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.546 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.546 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.818763+0000", 2024-09-13T10:12:10.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012372300000000001 2024-09-13T10:12:10.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.546 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.546 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:12:10.546 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.818947+0000", 2024-09-13T10:12:10.546 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000184394 2024-09-13T10:12:10.546 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.822370+0000", 2024-09-13T10:12:10.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034224989999999999 2024-09-13T10:12:10.547 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.547 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.547 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.547 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823413+0000", 2024-09-13T10:12:10.547 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001042739 2024-09-13T10:12:10.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823829+0000", 2024-09-13T10:12:10.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000416477 2024-09-13T10:12:10.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.548 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823982+0000", 2024-09-13T10:12:10.548 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015238599999999999 2024-09-13T10:12:10.548 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.548 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.548 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.549 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824083+0000", 2024-09-13T10:12:10.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000101267 2024-09-13T10:12:10.549 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.549 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.549 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.549 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.549 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.549 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:131 4.9 4:998d9bf1:::smithi19530714-33:head [setxattr _header (56) in=7b,truncate 2537824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:10.549 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.812049+0000", 2024-09-13T10:12:10.549 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.695355801000005, 2024-09-13T10:12:10.549 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012575461, 2024-09-13T10:12:10.549 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.549 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.550 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.550 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.550 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.550 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.550 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 131 2024-09-13T10:12:10.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.550 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.550 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.550 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812049+0000", 2024-09-13T10:12:10.550 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.550 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.550 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812049+0000", 2024-09-13T10:12:10.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812051+0000", 2024-09-13T10:12:10.551 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4270000000000001e-06 2024-09-13T10:12:10.551 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.551 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.551 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.551 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812060+0000", 2024-09-13T10:12:10.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9930000000000001e-06 2024-09-13T10:12:10.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812078+0000", 2024-09-13T10:12:10.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8266000000000001e-05 2024-09-13T10:12:10.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.552 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.552 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812093+0000", 2024-09-13T10:12:10.552 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5153e-05 2024-09-13T10:12:10.552 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.552 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.820907+0000", 2024-09-13T10:12:10.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088138650000000006 2024-09-13T10:12:10.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821046+0000", 2024-09-13T10:12:10.553 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013850900000000001 2024-09-13T10:12:10.553 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.553 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.553 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:12:10.553 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821338+0000", 2024-09-13T10:12:10.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029241800000000002 2024-09-13T10:12:10.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.823249+0000", 2024-09-13T10:12:10.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019105610000000001 2024-09-13T10:12:10.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824157+0000", 2024-09-13T10:12:10.554 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00090782899999999997 2024-09-13T10:12:10.554 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.554 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.554 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.554 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824422+0000", 2024-09-13T10:12:10.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026587900000000001 2024-09-13T10:12:10.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824497+0000", 2024-09-13T10:12:10.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4443999999999994e-05 2024-09-13T10:12:10.555 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.555 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.555 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.555 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824625+0000", 2024-09-13T10:12:10.555 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012811699999999999 2024-09-13T10:12:10.555 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.555 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.555 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.556 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.556 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.556 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:132 4.9 4:998d9bf1:::smithi19530714-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:10.556 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.812113+0000", 2024-09-13T10:12:10.556 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.695291788999995, 2024-09-13T10:12:10.556 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.01281326, 2024-09-13T10:12:10.556 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.556 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.556 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:10.556 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.556 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.556 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.557 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 132 2024-09-13T10:12:10.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.557 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.557 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.557 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812113+0000", 2024-09-13T10:12:10.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.557 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.557 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.557 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812113+0000", 2024-09-13T10:12:10.557 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.557 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.558 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.558 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812115+0000", 2024-09-13T10:12:10.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.826e-06 2024-09-13T10:12:10.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.558 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.558 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812121+0000", 2024-09-13T10:12:10.558 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9540000000000001e-06 2024-09-13T10:12:10.558 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.558 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.558 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812158+0000", 2024-09-13T10:12:10.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6619000000000003e-05 2024-09-13T10:12:10.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.812175+0000", 2024-09-13T10:12:10.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6662000000000001e-05 2024-09-13T10:12:10.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.559 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821791+0000", 2024-09-13T10:12:10.559 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.009616187 2024-09-13T10:12:10.559 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.559 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.559 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:10.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.821855+0000", 2024-09-13T10:12:10.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4183999999999994e-05 2024-09-13T10:12:10.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824680+0000", 2024-09-13T10:12:10.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0028252749999999999 2024-09-13T10:12:10.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.560 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.560 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.560 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824784+0000", 2024-09-13T10:12:10.560 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010365 2024-09-13T10:12:10.560 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.561 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.561 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.824927+0000", 2024-09-13T10:12:10.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000142903 2024-09-13T10:12:10.561 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.561 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.561 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.561 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.561 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.561 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:272 4.9 4:96fcf081:::smithi19530714-23:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:10.561 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.226803+0000", 2024-09-13T10:12:10.561 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.280602075999994, 2024-09-13T10:12:10.561 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0087344910000000005, 2024-09-13T10:12:10.562 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.562 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.562 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.562 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.562 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.562 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.562 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 272 2024-09-13T10:12:10.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.562 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.562 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.562 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.562 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226803+0000", 2024-09-13T10:12:10.562 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.562 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226803+0000", 2024-09-13T10:12:10.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226807+0000", 2024-09-13T10:12:10.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8829999999999999e-06 2024-09-13T10:12:10.563 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.563 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.563 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.563 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226823+0000", 2024-09-13T10:12:10.563 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5979999999999999e-05 2024-09-13T10:12:10.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226859+0000", 2024-09-13T10:12:10.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6331999999999997e-05 2024-09-13T10:12:10.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226882+0000", 2024-09-13T10:12:10.564 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2452000000000001e-05 2024-09-13T10:12:10.564 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.564 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.564 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.564 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.227037+0000", 2024-09-13T10:12:10.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000155563 2024-09-13T10:12:10.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.227232+0000", 2024-09-13T10:12:10.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00019476500000000001 2024-09-13T10:12:10.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.565 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:12:10.565 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.233676+0000", 2024-09-13T10:12:10.565 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0064435719999999998 2024-09-13T10:12:10.565 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.565 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235081+0000", 2024-09-13T10:12:10.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001404953 2024-09-13T10:12:10.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235114+0000", 2024-09-13T10:12:10.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2996000000000001e-05 2024-09-13T10:12:10.566 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.566 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.566 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.566 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235286+0000", 2024-09-13T10:12:10.566 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000171965 2024-09-13T10:12:10.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235431+0000", 2024-09-13T10:12:10.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000145205 2024-09-13T10:12:10.567 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.567 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.567 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.567 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.235538+0000", 2024-09-13T10:12:10.567 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010682499999999999 2024-09-13T10:12:10.567 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.567 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.567 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.568 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.568 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:282 4.9 4:998d9bf1:::smithi19530714-33:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:10.568 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.240459+0000", 2024-09-13T10:12:10.568 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.266946421, 2024-09-13T10:12:10.568 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.022877255999999999, 2024-09-13T10:12:10.568 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.568 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.568 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.568 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.568 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.568 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.568 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 282 2024-09-13T10:12:10.568 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.569 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240459+0000", 2024-09-13T10:12:10.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.569 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.569 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240459+0000", 2024-09-13T10:12:10.569 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.569 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.569 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240462+0000", 2024-09-13T10:12:10.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.971e-06 2024-09-13T10:12:10.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240479+0000", 2024-09-13T10:12:10.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6937000000000001e-05 2024-09-13T10:12:10.570 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.570 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.570 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.570 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240512+0000", 2024-09-13T10:12:10.570 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3012999999999997e-05 2024-09-13T10:12:10.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240533+0000", 2024-09-13T10:12:10.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1138999999999999e-05 2024-09-13T10:12:10.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.571 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240687+0000", 2024-09-13T10:12:10.571 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000154275 2024-09-13T10:12:10.571 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.571 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.571 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.240829+0000", 2024-09-13T10:12:10.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000141619 2024-09-13T10:12:10.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:12:10.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.260691+0000", 2024-09-13T10:12:10.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.019861706 2024-09-13T10:12:10.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.572 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.262189+0000", 2024-09-13T10:12:10.572 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0014983069999999999 2024-09-13T10:12:10.572 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.572 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.572 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.262975+0000", 2024-09-13T10:12:10.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00078655900000000004 2024-09-13T10:12:10.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.263100+0000", 2024-09-13T10:12:10.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012431899999999999 2024-09-13T10:12:10.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.573 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.573 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.263232+0000", 2024-09-13T10:12:10.573 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013227399999999999 2024-09-13T10:12:10.573 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.573 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.574 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.574 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.263336+0000", 2024-09-13T10:12:10.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104137 2024-09-13T10:12:10.574 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.574 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.574 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.574 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.574 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.574 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:296 4.9 4:9bec18ef:::smithi19530714-47:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:10.574 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.261575+0000", 2024-09-13T10:12:10.574 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.245830479000006, 2024-09-13T10:12:10.574 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0097761170000000008, 2024-09-13T10:12:10.574 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.574 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.575 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.575 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.575 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.575 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.575 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 296 2024-09-13T10:12:10.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.575 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.575 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.575 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.575 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261575+0000", 2024-09-13T10:12:10.575 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.575 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261575+0000", 2024-09-13T10:12:10.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.576 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261579+0000", 2024-09-13T10:12:10.576 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7740000000000002e-06 2024-09-13T10:12:10.576 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.576 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.576 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261594+0000", 2024-09-13T10:12:10.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4973999999999999e-05 2024-09-13T10:12:10.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261625+0000", 2024-09-13T10:12:10.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1519000000000001e-05 2024-09-13T10:12:10.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.577 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.577 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.577 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261645+0000", 2024-09-13T10:12:10.577 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9568e-05 2024-09-13T10:12:10.577 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261758+0000", 2024-09-13T10:12:10.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011301700000000001 2024-09-13T10:12:10.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.261902+0000", 2024-09-13T10:12:10.578 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014435300000000001 2024-09-13T10:12:10.578 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.578 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.578 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:12:10.578 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.269548+0000", 2024-09-13T10:12:10.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0076456850000000002 2024-09-13T10:12:10.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.270867+0000", 2024-09-13T10:12:10.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0013193930000000001 2024-09-13T10:12:10.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.579 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.270884+0000", 2024-09-13T10:12:10.579 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7116999999999999e-05 2024-09-13T10:12:10.579 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.579 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.579 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.271218+0000", 2024-09-13T10:12:10.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033386800000000002 2024-09-13T10:12:10.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.271305+0000", 2024-09-13T10:12:10.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6738999999999997e-05 2024-09-13T10:12:10.580 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.580 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.580 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.580 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.271351+0000", 2024-09-13T10:12:10.580 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6109999999999997e-05 2024-09-13T10:12:10.580 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.580 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.580 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.581 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.581 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.581 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:346 4.9 4:9bec18ef:::smithi19530714-47:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:10.581 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.285952+0000", 2024-09-13T10:12:10.581 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 79.221453629999999, 2024-09-13T10:12:10.581 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0081934079999999992, 2024-09-13T10:12:10.581 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.581 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.581 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.581 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.581 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.581 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.582 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 346 2024-09-13T10:12:10.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.582 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285952+0000", 2024-09-13T10:12:10.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.582 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.582 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.582 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.582 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285952+0000", 2024-09-13T10:12:10.582 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285954+0000", 2024-09-13T10:12:10.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3530000000000002e-06 2024-09-13T10:12:10.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.583 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285965+0000", 2024-09-13T10:12:10.583 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0718e-05 2024-09-13T10:12:10.583 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.583 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.583 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.285988+0000", 2024-09-13T10:12:10.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3095000000000002e-05 2024-09-13T10:12:10.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286005+0000", 2024-09-13T10:12:10.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7147999999999999e-05 2024-09-13T10:12:10.584 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.584 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.584 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.584 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286119+0000", 2024-09-13T10:12:10.584 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000113999 2024-09-13T10:12:10.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.286281+0000", 2024-09-13T10:12:10.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000161948 2024-09-13T10:12:10.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:12:10.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.291713+0000", 2024-09-13T10:12:10.585 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0054324810000000003 2024-09-13T10:12:10.585 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.585 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.585 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.585 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293743+0000", 2024-09-13T10:12:10.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020298439999999998 2024-09-13T10:12:10.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293904+0000", 2024-09-13T10:12:10.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000160853 2024-09-13T10:12:10.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.586 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.586 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.586 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.293952+0000", 2024-09-13T10:12:10.586 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7991999999999997e-05 2024-09-13T10:12:10.586 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294052+0000", 2024-09-13T10:12:10.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9893000000000001e-05 2024-09-13T10:12:10.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.587 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.587 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.587 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.294145+0000", 2024-09-13T10:12:10.587 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3084000000000005e-05 2024-09-13T10:12:10.587 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.587 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.587 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.587 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.588 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:801 4.9 4:96fcf081:::smithi19530714-23:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:12:10.588 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.798334+0000", 2024-09-13T10:12:10.588 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 78.709071272000003, 2024-09-13T10:12:10.588 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056549887, 2024-09-13T10:12:10.588 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.588 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.588 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:10.588 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.588 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.588 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.588 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 801 2024-09-13T10:12:10.588 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.588 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.588 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798334+0000", 2024-09-13T10:12:10.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798334+0000", 2024-09-13T10:12:10.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.589 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.589 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.589 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798338+0000", 2024-09-13T10:12:10.589 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3239999999999996e-06 2024-09-13T10:12:10.589 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798355+0000", 2024-09-13T10:12:10.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7149000000000001e-05 2024-09-13T10:12:10.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798403+0000", 2024-09-13T10:12:10.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7132000000000002e-05 2024-09-13T10:12:10.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.590 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.590 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798431+0000", 2024-09-13T10:12:10.590 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8365999999999999e-05 2024-09-13T10:12:10.590 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.590 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798534+0000", 2024-09-13T10:12:10.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102545 2024-09-13T10:12:10.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.798715+0000", 2024-09-13T10:12:10.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018161800000000001 2024-09-13T10:12:10.591 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.591 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.591 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.591 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.854884+0000", 2024-09-13T10:12:10.591 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.056168753000000002 2024-09-13T10:12:10.591 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.591 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.592 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.592 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.592 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.592 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1065 4.19 4:998d9bf1:::smithi19530714-33:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:10.592 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.618527+0000", 2024-09-13T10:12:10.592 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 73.888878555999995, 2024-09-13T10:12:10.592 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010820649999999999, 2024-09-13T10:12:10.592 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.592 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.592 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:10.592 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.592 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.593 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.593 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1065 2024-09-13T10:12:10.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.593 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618527+0000", 2024-09-13T10:12:10.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.593 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.593 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.593 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618527+0000", 2024-09-13T10:12:10.593 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.593 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618531+0000", 2024-09-13T10:12:10.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9899999999999999e-06 2024-09-13T10:12:10.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618546+0000", 2024-09-13T10:12:10.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5733e-05 2024-09-13T10:12:10.594 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.594 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.594 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.594 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618603+0000", 2024-09-13T10:12:10.594 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6075000000000001e-05 2024-09-13T10:12:10.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618635+0000", 2024-09-13T10:12:10.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2053e-05 2024-09-13T10:12:10.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618731+0000", 2024-09-13T10:12:10.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.6514000000000005e-05 2024-09-13T10:12:10.595 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.595 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.595 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.595 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.618887+0000", 2024-09-13T10:12:10.595 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000155415 2024-09-13T10:12:10.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.596 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.596 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.629347+0000", 2024-09-13T10:12:10.596 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010460870000000001 2024-09-13T10:12:10.596 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.596 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.596 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.596 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.596 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.596 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(osd.2.0:47 4.3 e46/41 4:c5b5ff75:::smithi19530714-20:head v 46'28, mlcod=43'26)", 2024-09-13T10:12:10.596 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:01.500294+0000", 2024-09-13T10:12:10.596 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 69.007111394000006, 2024-09-13T10:12:10.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010309196, 2024-09-13T10:12:10.597 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.597 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.597 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.597 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.500294+0000", 2024-09-13T10:12:10.597 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.597 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.597 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.597 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.597 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.500294+0000", 2024-09-13T10:12:10.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.500296+0000", 2024-09-13T10:12:10.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2800000000000002e-06 2024-09-13T10:12:10.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.598 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.598 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505222+0000", 2024-09-13T10:12:10.598 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0049261269999999998 2024-09-13T10:12:10.598 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.598 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505252+0000", 2024-09-13T10:12:10.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9272999999999999e-05 2024-09-13T10:12:10.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505266+0000", 2024-09-13T10:12:10.599 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4459e-05 2024-09-13T10:12:10.599 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.599 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.599 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.599 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505445+0000", 2024-09-13T10:12:10.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017880899999999999 2024-09-13T10:12:10.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.505501+0000", 2024-09-13T10:12:10.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5926e-05 2024-09-13T10:12:10.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.600 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.600 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.600 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.510523+0000", 2024-09-13T10:12:10.600 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0050222440000000004 2024-09-13T10:12:10.600 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.601 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.601 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:01.510603+0000", 2024-09-13T10:12:10.601 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.0078000000000001e-05 2024-09-13T10:12:10.601 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.601 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.601 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.601 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.601 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.601 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1619 4.19 4:998d9bf1:::smithi19530714-33:head [write 1750386~495032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:12:10.601 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.073959+0000", 2024-09-13T10:12:10.601 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.433446387000004, 2024-09-13T10:12:10.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0081975420000000004, 2024-09-13T10:12:10.602 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.602 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.602 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.602 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.602 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.602 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.602 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1619 2024-09-13T10:12:10.602 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.602 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.602 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.602 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.602 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.073959+0000", 2024-09-13T10:12:10.602 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.073959+0000", 2024-09-13T10:12:10.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.603 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.073961+0000", 2024-09-13T10:12:10.603 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3829999999999999e-06 2024-09-13T10:12:10.603 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.603 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.603 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074180+0000", 2024-09-13T10:12:10.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000218352 2024-09-13T10:12:10.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074208+0000", 2024-09-13T10:12:10.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7880000000000001e-05 2024-09-13T10:12:10.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.604 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.604 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074222+0000", 2024-09-13T10:12:10.604 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4250000000000001e-05 2024-09-13T10:12:10.604 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.604 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074360+0000", 2024-09-13T10:12:10.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000138023 2024-09-13T10:12:10.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074587+0000", 2024-09-13T10:12:10.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000226789 2024-09-13T10:12:10.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.605 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.605 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:12:10.605 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074984+0000", 2024-09-13T10:12:10.605 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000397031 2024-09-13T10:12:10.605 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.078275+0000", 2024-09-13T10:12:10.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032917250000000001 2024-09-13T10:12:10.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.080241+0000", 2024-09-13T10:12:10.606 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019660910000000001 2024-09-13T10:12:10.606 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.606 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.606 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.606 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.081869+0000", 2024-09-13T10:12:10.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001627325 2024-09-13T10:12:10.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082041+0000", 2024-09-13T10:12:10.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017206599999999999 2024-09-13T10:12:10.607 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.607 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.607 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.607 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082156+0000", 2024-09-13T10:12:10.607 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115627 2024-09-13T10:12:10.607 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.607 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.607 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.608 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.608 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.608 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1620 4.19 4:998d9bf1:::smithi19530714-33:head [setxattr _header (56) in=7b,truncate 2302559] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:12:10.608 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.074286+0000", 2024-09-13T10:12:10.608 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.433119020999996, 2024-09-13T10:12:10.608 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0086728220000000002, 2024-09-13T10:12:10.608 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.608 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.608 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.608 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.608 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.608 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.608 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1620 2024-09-13T10:12:10.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.609 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074286+0000", 2024-09-13T10:12:10.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.609 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.609 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074286+0000", 2024-09-13T10:12:10.609 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.609 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.609 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074288+0000", 2024-09-13T10:12:10.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.677e-06 2024-09-13T10:12:10.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074298+0000", 2024-09-13T10:12:10.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0020000000000001e-05 2024-09-13T10:12:10.610 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.610 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.610 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.610 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074316+0000", 2024-09-13T10:12:10.610 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7546000000000001e-05 2024-09-13T10:12:10.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074330+0000", 2024-09-13T10:12:10.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4399999999999999e-05 2024-09-13T10:12:10.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.076843+0000", 2024-09-13T10:12:10.611 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025133289999999999 2024-09-13T10:12:10.611 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.611 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.611 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.611 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.076976+0000", 2024-09-13T10:12:10.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013269000000000001 2024-09-13T10:12:10.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:12:10.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.077195+0000", 2024-09-13T10:12:10.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000219278 2024-09-13T10:12:10.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.612 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.078998+0000", 2024-09-13T10:12:10.612 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0018027379999999999 2024-09-13T10:12:10.612 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.612 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.612 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082240+0000", 2024-09-13T10:12:10.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032419929999999999 2024-09-13T10:12:10.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082653+0000", 2024-09-13T10:12:10.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041340499999999999 2024-09-13T10:12:10.613 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.613 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.613 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.613 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082806+0000", 2024-09-13T10:12:10.613 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015261300000000001 2024-09-13T10:12:10.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.614 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.614 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.082959+0000", 2024-09-13T10:12:10.614 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015313300000000001 2024-09-13T10:12:10.614 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.614 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.614 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.614 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.614 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.614 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1621 4.19 4:998d9bf1:::smithi19530714-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:12:10.614 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.074359+0000", 2024-09-13T10:12:10.614 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.433046242000003, 2024-09-13T10:12:10.615 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0092381359999999992, 2024-09-13T10:12:10.615 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.615 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.615 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:10.615 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.615 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.615 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.615 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1621 2024-09-13T10:12:10.615 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.615 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.615 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.615 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.615 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074359+0000", 2024-09-13T10:12:10.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074359+0000", 2024-09-13T10:12:10.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.616 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.616 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.616 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074361+0000", 2024-09-13T10:12:10.616 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4699999999999999e-06 2024-09-13T10:12:10.616 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074387+0000", 2024-09-13T10:12:10.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6472e-05 2024-09-13T10:12:10.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074402+0000", 2024-09-13T10:12:10.617 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5437000000000001e-05 2024-09-13T10:12:10.617 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.617 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.617 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.617 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.074411+0000", 2024-09-13T10:12:10.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8699999999999998e-06 2024-09-13T10:12:10.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.078356+0000", 2024-09-13T10:12:10.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003944894 2024-09-13T10:12:10.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:10.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.078565+0000", 2024-09-13T10:12:10.618 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020851800000000001 2024-09-13T10:12:10.618 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.618 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.618 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.618 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.083093+0000", 2024-09-13T10:12:10.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045284660000000001 2024-09-13T10:12:10.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.083319+0000", 2024-09-13T10:12:10.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022566200000000001 2024-09-13T10:12:10.619 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.619 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.619 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.619 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.083597+0000", 2024-09-13T10:12:10.619 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027834700000000002 2024-09-13T10:12:10.619 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.619 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.619 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.620 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.620 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1989 4.19 4:998d9bf1:::smithi19530714-33:head [write 574839~795192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-09-13T10:12:10.620 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:13.093772+0000", 2024-09-13T10:12:10.620 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 57.413633644000001, 2024-09-13T10:12:10.620 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.008334045, 2024-09-13T10:12:10.620 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.620 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.620 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.620 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.620 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.620 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.620 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1989 2024-09-13T10:12:10.620 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.621 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.093772+0000", 2024-09-13T10:12:10.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.621 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.093772+0000", 2024-09-13T10:12:10.621 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.621 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.621 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.621 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.093774+0000", 2024-09-13T10:12:10.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1519999999999999e-06 2024-09-13T10:12:10.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094142+0000", 2024-09-13T10:12:10.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00036777400000000002 2024-09-13T10:12:10.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.622 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.622 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094158+0000", 2024-09-13T10:12:10.622 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6574000000000001e-05 2024-09-13T10:12:10.622 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.622 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094167+0000", 2024-09-13T10:12:10.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.9299999999999992e-06 2024-09-13T10:12:10.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094284+0000", 2024-09-13T10:12:10.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116895 2024-09-13T10:12:10.623 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.623 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.623 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.623 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094428+0000", 2024-09-13T10:12:10.623 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014431099999999999 2024-09-13T10:12:10.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:12:10.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.094658+0000", 2024-09-13T10:12:10.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000230106 2024-09-13T10:12:10.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.098293+0000", 2024-09-13T10:12:10.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003634918 2024-09-13T10:12:10.624 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.624 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.624 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.624 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.100887+0000", 2024-09-13T10:12:10.624 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025936309999999999 2024-09-13T10:12:10.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.101823+0000", 2024-09-13T10:12:10.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00093637600000000005 2024-09-13T10:12:10.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.101980+0000", 2024-09-13T10:12:10.625 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015709400000000001 2024-09-13T10:12:10.625 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.625 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.625 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.625 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:13.102106+0000", 2024-09-13T10:12:10.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000125284 2024-09-13T10:12:10.626 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.626 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.626 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.626 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.626 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.626 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2416 4.19 4:998d9bf1:::smithi19530714-33:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e63)", 2024-09-13T10:12:10.626 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:22.232954+0000", 2024-09-13T10:12:10.626 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 48.274451507000002, 2024-09-13T10:12:10.626 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010811217999999999, 2024-09-13T10:12:10.626 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:10.626 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:10.626 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:10.627 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:10.627 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:10.627 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:10.627 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2416 2024-09-13T10:12:10.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.627 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:10.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:10.627 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.232954+0000", 2024-09-13T10:12:10.627 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.627 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.627 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.627 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:10.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.232954+0000", 2024-09-13T10:12:10.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:10.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:10.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.232957+0000", 2024-09-13T10:12:10.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0240000000000002e-06 2024-09-13T10:12:10.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.628 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:10.628 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.232969+0000", 2024-09-13T10:12:10.628 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2602000000000001e-05 2024-09-13T10:12:10.628 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.628 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:10.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.233001+0000", 2024-09-13T10:12:10.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1486000000000003e-05 2024-09-13T10:12:10.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:10.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.233017+0000", 2024-09-13T10:12:10.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6565000000000001e-05 2024-09-13T10:12:10.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.629 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.629 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.233160+0000", 2024-09-13T10:12:10.629 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014236000000000001 2024-09-13T10:12:10.629 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.629 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op must wait for map", 2024-09-13T10:12:10.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.233170+0000", 2024-09-13T10:12:10.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9599999999999995e-06 2024-09-13T10:12:10.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:10.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.236247+0000", 2024-09-13T10:12:10.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030776990000000001 2024-09-13T10:12:10.630 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.630 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.630 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:10.630 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.236509+0000", 2024-09-13T10:12:10.630 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026104399999999999 2024-09-13T10:12:10.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 5,9", 2024-09-13T10:12:10.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.236883+0000", 2024-09-13T10:12:10.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00037440500000000002 2024-09-13T10:12:10.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:10.631 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.240643+0000", 2024-09-13T10:12:10.631 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003760354 2024-09-13T10:12:10.631 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.631 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.631 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.241525+0000", 2024-09-13T10:12:10.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00088204699999999995 2024-09-13T10:12:10.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:10.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.243591+0000", 2024-09-13T10:12:10.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020659039999999999 2024-09-13T10:12:10.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.632 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:10.632 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.243723+0000", 2024-09-13T10:12:10.632 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013196 2024-09-13T10:12:10.632 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:10.632 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:10.633 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:10.633 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:22.243765+0000", 2024-09-13T10:12:10.633 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1807999999999997e-05 2024-09-13T10:12:10.633 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.633 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.633 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.633 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:10.633 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:10.633 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:10.635 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-13T10:12:10.635 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2642705 2024-09-13T10:12:10.635 INFO:tasks.rados.rados.0.smithi195.stdout:1807: expect (ObjNum 370 snap 0 seq_num 370) 2024-09-13T10:12:10.636 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 323723 2024-09-13T10:12:10.636 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 813972 2024-09-13T10:12:10.636 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:10.630+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:10.647 INFO:tasks.rados.rados.0.smithi195.stdout:1810: finishing write tid 1 to smithi19530714-45 2024-09-13T10:12:10.647 INFO:tasks.rados.rados.0.smithi195.stdout:1811: expect (ObjNum 328 snap 0 seq_num 328) 2024-09-13T10:12:10.735 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:10.735+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:10.766 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:10.766 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:12:10.766 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:12:10.766 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:10.779 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-13T10:12:10.828 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T10:12:10.836 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:10.830+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:10.846 INFO:tasks.rados.rados.0.smithi195.stdout:1810: finishing write tid 2 to smithi19530714-45 2024-09-13T10:12:10.846 INFO:tasks.rados.rados.0.smithi195.stdout:1810: finishing write tid 3 to smithi19530714-45 2024-09-13T10:12:10.846 INFO:tasks.rados.rados.0.smithi195.stdout:1804: done (9 left) 2024-09-13T10:12:10.846 INFO:tasks.rados.rados.0.smithi195.stdout:1805: done (8 left) 2024-09-13T10:12:10.846 INFO:tasks.rados.rados.0.smithi195.stdout:1807: done (7 left) 2024-09-13T10:12:10.846 INFO:tasks.rados.rados.0.smithi195.stdout:1808: done (6 left) 2024-09-13T10:12:10.847 INFO:tasks.rados.rados.0.smithi195.stdout:1811: done (5 left) 2024-09-13T10:12:10.847 INFO:tasks.rados.rados.0.smithi195.stdout:1813: done (4 left) 2024-09-13T10:12:10.847 INFO:tasks.rados.rados.0.smithi195.stdout:1816: delete oid 33 current snap is 0 2024-09-13T10:12:10.847 INFO:tasks.rados.rados.0.smithi195.stdout:1810: finishing write tid 4 to smithi19530714-45 2024-09-13T10:12:10.847 INFO:tasks.rados.rados.0.smithi195.stdout:1810: oid 45 updating version 0 to 179 2024-09-13T10:12:10.847 INFO:tasks.rados.rados.0.smithi195.stdout:1810: oid 45 updating version 179 to 180 2024-09-13T10:12:10.847 INFO:tasks.rados.rados.0.smithi195.stdout:1810: oid 45 version 180 is already newer than 178 2024-09-13T10:12:10.847 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 180 (ObjNum 397 snap 0 seq_num 397) dirty exists 2024-09-13T10:12:10.847 INFO:tasks.rados.rados.0.smithi195.stdout:1810: left oid 45 (ObjNum 397 snap 0 seq_num 397) 2024-09-13T10:12:10.847 INFO:tasks.rados.rados.0.smithi195.stdout:1812: expect (ObjNum 346 snap 0 seq_num 346) 2024-09-13T10:12:10.847 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:10.897 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:12:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:12:10.897 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:12:10.898 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:10.910 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-13T10:12:10.937 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:12:10.930+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:12:11.013 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:11.028 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:11.028 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:12:11.028 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:12:11.028 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:12:11.028 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.029 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:161 4.d e38/31 4:b19b932a:::smithi19530714-41:head v 38'1, mlcod=0'0)", 2024-09-13T10:12:11.029 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:12:11.029 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 80.170050441000001, 2024-09-13T10:12:11.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056705289999999997, 2024-09-13T10:12:11.029 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.029 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.029 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:11.029 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.029 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.029 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:12:11.029 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.029 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.029 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:12:11.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.854012+0000", 2024-09-13T10:12:11.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.0999999999999998e-07 2024-09-13T10:12:11.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.030 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.030 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856051+0000", 2024-09-13T10:12:11.030 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0020382130000000001 2024-09-13T10:12:11.030 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.030 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856079+0000", 2024-09-13T10:12:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8345999999999999e-05 2024-09-13T10:12:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856093+0000", 2024-09-13T10:12:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4353e-05 2024-09-13T10:12:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856232+0000", 2024-09-13T10:12:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013922199999999999 2024-09-13T10:12:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.031 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.856267+0000", 2024-09-13T10:12:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4823000000000002e-05 2024-09-13T10:12:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.859505+0000", 2024-09-13T10:12:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0032373269999999999 2024-09-13T10:12:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.859682+0000", 2024-09-13T10:12:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017753499999999999 2024-09-13T10:12:11.032 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1665 4.d 4:b19b932a:::smithi19530714-41:head [write 769787~704463,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-13T10:12:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.752922+0000", 2024-09-13T10:12:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.271140021999997, 2024-09-13T10:12:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014152090000000001, 2024-09-13T10:12:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.033 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1665 2024-09-13T10:12:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.752922+0000", 2024-09-13T10:12:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.752922+0000", 2024-09-13T10:12:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.034 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.752927+0000", 2024-09-13T10:12:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7430000000000002e-06 2024-09-13T10:12:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.753904+0000", 2024-09-13T10:12:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00097707799999999993 2024-09-13T10:12:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.753959+0000", 2024-09-13T10:12:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5099000000000002e-05 2024-09-13T10:12:11.035 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.753987+0000", 2024-09-13T10:12:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7736999999999999e-05 2024-09-13T10:12:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.754289+0000", 2024-09-13T10:12:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000301926 2024-09-13T10:12:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.036 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.755253+0000", 2024-09-13T10:12:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00096452500000000004 2024-09-13T10:12:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:12:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.756279+0000", 2024-09-13T10:12:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010256919999999999 2024-09-13T10:12:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.765950+0000", 2024-09-13T10:12:11.037 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096707410000000001 2024-09-13T10:12:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766011+0000", 2024-09-13T10:12:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1826000000000001e-05 2024-09-13T10:12:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766431+0000", 2024-09-13T10:12:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041941400000000001 2024-09-13T10:12:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.038 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.767009+0000", 2024-09-13T10:12:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00057826500000000001 2024-09-13T10:12:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.767074+0000", 2024-09-13T10:12:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5043999999999996e-05 2024-09-13T10:12:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.039 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1666 4.d 4:b19b932a:::smithi19530714-41:head [write 2267961~101826] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-13T10:12:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.762495+0000", 2024-09-13T10:12:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.261567490999994, 2024-09-13T10:12:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0061889800000000002, 2024-09-13T10:12:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1666 2024-09-13T10:12:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.040 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762495+0000", 2024-09-13T10:12:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762495+0000", 2024-09-13T10:12:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.041 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762542+0000", 2024-09-13T10:12:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7382999999999997e-05 2024-09-13T10:12:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762850+0000", 2024-09-13T10:12:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030803999999999998 2024-09-13T10:12:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762888+0000", 2024-09-13T10:12:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7645999999999998e-05 2024-09-13T10:12:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.042 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762919+0000", 2024-09-13T10:12:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0879e-05 2024-09-13T10:12:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763057+0000", 2024-09-13T10:12:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000138672 2024-09-13T10:12:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763221+0000", 2024-09-13T10:12:11.043 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016337200000000001 2024-09-13T10:12:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:12:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763457+0000", 2024-09-13T10:12:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023669399999999999 2024-09-13T10:12:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766139+0000", 2024-09-13T10:12:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026813819999999999 2024-09-13T10:12:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.044 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.767138+0000", 2024-09-13T10:12:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0009992200000000001 2024-09-13T10:12:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768189+0000", 2024-09-13T10:12:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001051614 2024-09-13T10:12:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768616+0000", 2024-09-13T10:12:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00042670000000000002 2024-09-13T10:12:11.045 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768684+0000", 2024-09-13T10:12:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7378000000000004e-05 2024-09-13T10:12:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1667 4.d 4:b19b932a:::smithi19530714-41:head [setxattr _header (56) in=7b,truncate 2369787] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49)", 2024-09-13T10:12:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.762978+0000", 2024-09-13T10:12:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.261083928000005, 2024-09-13T10:12:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0077634310000000003, 2024-09-13T10:12:11.046 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1667 2024-09-13T10:12:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762978+0000", 2024-09-13T10:12:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.047 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762978+0000", 2024-09-13T10:12:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762980+0000", 2024-09-13T10:12:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.654e-06 2024-09-13T10:12:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.048 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.762990+0000", 2024-09-13T10:12:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9629999999999993e-06 2024-09-13T10:12:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763008+0000", 2024-09-13T10:12:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8122e-05 2024-09-13T10:12:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763026+0000", 2024-09-13T10:12:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7747000000000001e-05 2024-09-13T10:12:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.049 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.764724+0000", 2024-09-13T10:12:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001698726 2024-09-13T10:12:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.764905+0000", 2024-09-13T10:12:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018053200000000001 2024-09-13T10:12:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:12:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.765167+0000", 2024-09-13T10:12:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026248100000000001 2024-09-13T10:12:11.050 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.767098+0000", 2024-09-13T10:12:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019303949999999999 2024-09-13T10:12:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768108+0000", 2024-09-13T10:12:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010101820000000001 2024-09-13T10:12:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.051 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.768766+0000", 2024-09-13T10:12:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00065837299999999997 2024-09-13T10:12:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770530+0000", 2024-09-13T10:12:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00176384 2024-09-13T10:12:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770742+0000", 2024-09-13T10:12:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000211416 2024-09-13T10:12:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.052 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1668 4.d 4:b19b932a:::smithi19530714-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e49)", 2024-09-13T10:12:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:06.763054+0000", 2024-09-13T10:12:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 64.261007829999997, 2024-09-13T10:12:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0080519860000000006, 2024-09-13T10:12:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:11.053 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1668 2024-09-13T10:12:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763054+0000", 2024-09-13T10:12:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763054+0000", 2024-09-13T10:12:11.054 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763056+0000", 2024-09-13T10:12:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.404e-06 2024-09-13T10:12:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763062+0000", 2024-09-13T10:12:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1260000000000003e-06 2024-09-13T10:12:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763080+0000", 2024-09-13T10:12:11.055 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7864e-05 2024-09-13T10:12:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.763093+0000", 2024-09-13T10:12:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3702e-05 2024-09-13T10:12:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766050+0000", 2024-09-13T10:12:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029562400000000002 2024-09-13T10:12:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.056 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.766127+0000", 2024-09-13T10:12:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.7469e-05 2024-09-13T10:12:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770802+0000", 2024-09-13T10:12:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0046747309999999997 2024-09-13T10:12:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.770954+0000", 2024-09-13T10:12:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015184000000000001 2024-09-13T10:12:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.057 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:06.771106+0000", 2024-09-13T10:12:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015260999999999999 2024-09-13T10:12:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1921 4.d 4:b19b932a:::smithi19530714-41:head [read 1405629~964158,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:12:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:11.758460+0000", 2024-09-13T10:12:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.265602473999998, 2024-09-13T10:12:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068984509999999999, 2024-09-13T10:12:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.058 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1921 2024-09-13T10:12:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758460+0000", 2024-09-13T10:12:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.059 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758460+0000", 2024-09-13T10:12:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758461+0000", 2024-09-13T10:12:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.31e-06 2024-09-13T10:12:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758469+0000", 2024-09-13T10:12:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8029999999999997e-06 2024-09-13T10:12:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.060 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758490+0000", 2024-09-13T10:12:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1365999999999998e-05 2024-09-13T10:12:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758503+0000", 2024-09-13T10:12:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3200000000000001e-05 2024-09-13T10:12:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.763779+0000", 2024-09-13T10:12:11.061 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0052761079999999998 2024-09-13T10:12:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.764101+0000", 2024-09-13T10:12:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032119700000000001 2024-09-13T10:12:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.765358+0000", 2024-09-13T10:12:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001257467 2024-09-13T10:12:11.062 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1922 4.d 4:b19b932a:::smithi19530714-41:head [sparse-read 1834913~534874] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:12:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:11.758578+0000", 2024-09-13T10:12:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.265483850000003, 2024-09-13T10:12:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.007449954, 2024-09-13T10:12:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.063 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1922 2024-09-13T10:12:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758578+0000", 2024-09-13T10:12:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758578+0000", 2024-09-13T10:12:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.064 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758580+0000", 2024-09-13T10:12:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.79e-06 2024-09-13T10:12:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758587+0000", 2024-09-13T10:12:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5599999999999999e-06 2024-09-13T10:12:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758620+0000", 2024-09-13T10:12:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3649000000000003e-05 2024-09-13T10:12:11.065 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.758628+0000", 2024-09-13T10:12:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2400000000000007e-06 2024-09-13T10:12:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.765429+0000", 2024-09-13T10:12:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068003600000000001 2024-09-13T10:12:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.066 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.765651+0000", 2024-09-13T10:12:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022221900000000001 2024-09-13T10:12:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:11.766028+0000", 2024-09-13T10:12:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00037713599999999998 2024-09-13T10:12:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2712 4.d 4:b19b932a:::smithi19530714-41:head [write 1860221~469412] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:12:11.067 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:29.375231+0000", 2024-09-13T10:12:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 41.648830566000001, 2024-09-13T10:12:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.005573349, 2024-09-13T10:12:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2712 2024-09-13T10:12:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375231+0000", 2024-09-13T10:12:11.068 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375231+0000", 2024-09-13T10:12:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375233+0000", 2024-09-13T10:12:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.517e-06 2024-09-13T10:12:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.069 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375610+0000", 2024-09-13T10:12:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00037666799999999998 2024-09-13T10:12:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375628+0000", 2024-09-13T10:12:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7918e-05 2024-09-13T10:12:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375639+0000", 2024-09-13T10:12:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1345000000000001e-05 2024-09-13T10:12:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.070 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375690+0000", 2024-09-13T10:12:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0603000000000003e-05 2024-09-13T10:12:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375755+0000", 2024-09-13T10:12:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5331000000000002e-05 2024-09-13T10:12:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:12:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.375864+0000", 2024-09-13T10:12:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000108695 2024-09-13T10:12:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.071 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.377367+0000", 2024-09-13T10:12:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0015037920000000001 2024-09-13T10:12:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.378220+0000", 2024-09-13T10:12:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00085309600000000002 2024-09-13T10:12:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.380433+0000", 2024-09-13T10:12:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0022124760000000001 2024-09-13T10:12:11.072 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.380726+0000", 2024-09-13T10:12:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029259799999999998 2024-09-13T10:12:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:29.380805+0000", 2024-09-13T10:12:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9309999999999998e-05 2024-09-13T10:12:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.073 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3259 4.d 4:ba7d2bc5:::smithi19530714-46:head [write 558065~619869,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-13T10:12:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:43.057116+0000", 2024-09-13T10:12:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 27.966945753000001, 2024-09-13T10:12:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0075474399999999999, 2024-09-13T10:12:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3259 2024-09-13T10:12:11.074 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.057116+0000", 2024-09-13T10:12:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.057116+0000", 2024-09-13T10:12:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.057119+0000", 2024-09-13T10:12:11.075 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0630000000000002e-06 2024-09-13T10:12:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.057773+0000", 2024-09-13T10:12:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00065407799999999999 2024-09-13T10:12:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.057806+0000", 2024-09-13T10:12:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2781999999999999e-05 2024-09-13T10:12:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.076 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.057823+0000", 2024-09-13T10:12:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7039999999999999e-05 2024-09-13T10:12:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.058151+0000", 2024-09-13T10:12:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032804399999999999 2024-09-13T10:12:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.058757+0000", 2024-09-13T10:12:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00060580900000000001 2024-09-13T10:12:11.077 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:12:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.059346+0000", 2024-09-13T10:12:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00058853599999999996 2024-09-13T10:12:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.062924+0000", 2024-09-13T10:12:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035781150000000002 2024-09-13T10:12:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.063034+0000", 2024-09-13T10:12:11.078 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109943 2024-09-13T10:12:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.064323+0000", 2024-09-13T10:12:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0012888369999999999 2024-09-13T10:12:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.064505+0000", 2024-09-13T10:12:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000182766 2024-09-13T10:12:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.079 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.064664+0000", 2024-09-13T10:12:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00015842699999999999 2024-09-13T10:12:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3260 4.d 4:ba7d2bc5:::smithi19530714-46:head [write 1579835~486765] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-09-13T10:12:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:43.070681+0000", 2024-09-13T10:12:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 27.953380942999999, 2024-09-13T10:12:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0056142930000000002, 2024-09-13T10:12:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.080 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3260 2024-09-13T10:12:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.070681+0000", 2024-09-13T10:12:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.081 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.070681+0000", 2024-09-13T10:12:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.070684+0000", 2024-09-13T10:12:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6869999999999999e-06 2024-09-13T10:12:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.070967+0000", 2024-09-13T10:12:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000282766 2024-09-13T10:12:11.082 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.070986+0000", 2024-09-13T10:12:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9521000000000002e-05 2024-09-13T10:12:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.070997+0000", 2024-09-13T10:12:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.06e-05 2024-09-13T10:12:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.083 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.071274+0000", 2024-09-13T10:12:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027780099999999999 2024-09-13T10:12:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.071763+0000", 2024-09-13T10:12:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000488665 2024-09-13T10:12:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:12:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.072179+0000", 2024-09-13T10:12:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00041585899999999998 2024-09-13T10:12:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.084 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.075091+0000", 2024-09-13T10:12:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029115349999999998 2024-09-13T10:12:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.075125+0000", 2024-09-13T10:12:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4442e-05 2024-09-13T10:12:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.076114+0000", 2024-09-13T10:12:11.085 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00098851899999999999 2024-09-13T10:12:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.076209+0000", 2024-09-13T10:12:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5087000000000006e-05 2024-09-13T10:12:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.076295+0000", 2024-09-13T10:12:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6811000000000006e-05 2024-09-13T10:12:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.086 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3261 4.d 4:ba7d2bc5:::smithi19530714-46:head [write 2468371~489694] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:43.082814+0000", 2024-09-13T10:12:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 27.941247687000001, 2024-09-13T10:12:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0079547009999999998, 2024-09-13T10:12:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.087 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3261 2024-09-13T10:12:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.082814+0000", 2024-09-13T10:12:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.082814+0000", 2024-09-13T10:12:11.088 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.082817+0000", 2024-09-13T10:12:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9670000000000001e-06 2024-09-13T10:12:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083161+0000", 2024-09-13T10:12:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00034351199999999997 2024-09-13T10:12:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.089 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083186+0000", 2024-09-13T10:12:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5194e-05 2024-09-13T10:12:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083199+0000", 2024-09-13T10:12:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2614e-05 2024-09-13T10:12:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083318+0000", 2024-09-13T10:12:11.090 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011906299999999999 2024-09-13T10:12:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op must wait for map", 2024-09-13T10:12:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083330+0000", 2024-09-13T10:12:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2558000000000001e-05 2024-09-13T10:12:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.085615+0000", 2024-09-13T10:12:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0022843540000000002 2024-09-13T10:12:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.091 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.085700+0000", 2024-09-13T10:12:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5177000000000001e-05 2024-09-13T10:12:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:12:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.085838+0000", 2024-09-13T10:12:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013857499999999999 2024-09-13T10:12:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.089886+0000", 2024-09-13T10:12:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0040475290000000002 2024-09-13T10:12:11.092 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.090273+0000", 2024-09-13T10:12:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000387439 2024-09-13T10:12:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.090564+0000", 2024-09-13T10:12:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00029072100000000002 2024-09-13T10:12:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:11.093 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.090655+0000", 2024-09-13T10:12:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.0823999999999996e-05 2024-09-13T10:12:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.090769+0000", 2024-09-13T10:12:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011417399999999999 2024-09-13T10:12:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.094 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3262 4.d 4:ba7d2bc5:::smithi19530714-46:head [setxattr _header (56) in=7b,truncate 2958065] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:43.083231+0000", 2024-09-13T10:12:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 27.940830738999999, 2024-09-13T10:12:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0079301509999999999, 2024-09-13T10:12:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3262 2024-09-13T10:12:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.095 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083231+0000", 2024-09-13T10:12:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083231+0000", 2024-09-13T10:12:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.096 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083233+0000", 2024-09-13T10:12:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.595e-06 2024-09-13T10:12:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083240+0000", 2024-09-13T10:12:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7340000000000002e-06 2024-09-13T10:12:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083248+0000", 2024-09-13T10:12:11.097 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.8970000000000008e-06 2024-09-13T10:12:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083254+0000", 2024-09-13T10:12:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3300000000000004e-06 2024-09-13T10:12:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083526+0000", 2024-09-13T10:12:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027166799999999997 2024-09-13T10:12:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.098 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting_for_map not empty", 2024-09-13T10:12:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083546+0000", 2024-09-13T10:12:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0092999999999999e-05 2024-09-13T10:12:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.087668+0000", 2024-09-13T10:12:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0041219719999999998 2024-09-13T10:12:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.087976+0000", 2024-09-13T10:12:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030828399999999998 2024-09-13T10:12:11.099 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:12:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.088483+0000", 2024-09-13T10:12:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00050680600000000005 2024-09-13T10:12:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.090406+0000", 2024-09-13T10:12:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001922864 2024-09-13T10:12:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.090523+0000", 2024-09-13T10:12:11.100 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000117627 2024-09-13T10:12:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.090794+0000", 2024-09-13T10:12:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00027118500000000001 2024-09-13T10:12:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.090881+0000", 2024-09-13T10:12:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6930999999999998e-05 2024-09-13T10:12:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.101 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.091161+0000", 2024-09-13T10:12:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00028016500000000003 2024-09-13T10:12:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3263 4.d 4:ba7d2bc5:::smithi19530714-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:43.083269+0000", 2024-09-13T10:12:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 27.940793016000001, 2024-09-13T10:12:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0081338560000000001, 2024-09-13T10:12:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.102 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3263 2024-09-13T10:12:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083269+0000", 2024-09-13T10:12:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.103 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083269+0000", 2024-09-13T10:12:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083270+0000", 2024-09-13T10:12:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9800000000000003e-07 2024-09-13T10:12:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083273+0000", 2024-09-13T10:12:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.416e-06 2024-09-13T10:12:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.104 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083283+0000", 2024-09-13T10:12:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.9520000000000006e-06 2024-09-13T10:12:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083292+0000", 2024-09-13T10:12:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8680000000000005e-06 2024-09-13T10:12:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083595+0000", 2024-09-13T10:12:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030329100000000002 2024-09-13T10:12:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.105 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting_for_map not empty", 2024-09-13T10:12:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.083599+0000", 2024-09-13T10:12:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9920000000000001e-06 2024-09-13T10:12:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.089551+0000", 2024-09-13T10:12:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0059521210000000003 2024-09-13T10:12:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.089770+0000", 2024-09-13T10:12:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021871400000000001 2024-09-13T10:12:11.106 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.091250+0000", 2024-09-13T10:12:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001479389 2024-09-13T10:12:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.091314+0000", 2024-09-13T10:12:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4888999999999995e-05 2024-09-13T10:12:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.107 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:43.091403+0000", 2024-09-13T10:12:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8425999999999997e-05 2024-09-13T10:12:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3521 4.d 4:ba7d2bc5:::smithi19530714-46:head [write 477398~798659] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-13T10:12:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:48.956271+0000", 2024-09-13T10:12:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 22.067791238000002, 2024-09-13T10:12:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0063851139999999999, 2024-09-13T10:12:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.108 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3521 2024-09-13T10:12:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.956271+0000", 2024-09-13T10:12:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.956271+0000", 2024-09-13T10:12:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.109 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.956275+0000", 2024-09-13T10:12:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.9330000000000003e-06 2024-09-13T10:12:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.957235+0000", 2024-09-13T10:12:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00095993900000000004 2024-09-13T10:12:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.957270+0000", 2024-09-13T10:12:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5194999999999998e-05 2024-09-13T10:12:11.110 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.957287+0000", 2024-09-13T10:12:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7453e-05 2024-09-13T10:12:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.957359+0000", 2024-09-13T10:12:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1314000000000002e-05 2024-09-13T10:12:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.957427+0000", 2024-09-13T10:12:11.111 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8090999999999996e-05 2024-09-13T10:12:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:12:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.957534+0000", 2024-09-13T10:12:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107031 2024-09-13T10:12:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.959738+0000", 2024-09-13T10:12:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002204484 2024-09-13T10:12:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.112 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.961174+0000", 2024-09-13T10:12:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001435644 2024-09-13T10:12:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.962581+0000", 2024-09-13T10:12:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0014072640000000001 2024-09-13T10:12:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.962619+0000", 2024-09-13T10:12:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7904999999999997e-05 2024-09-13T10:12:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.113 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.962656+0000", 2024-09-13T10:12:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6860999999999998e-05 2024-09-13T10:12:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3522 4.d 4:ba7d2bc5:::smithi19530714-46:head [write 1774640~663444] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-13T10:12:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:48.963159+0000", 2024-09-13T10:12:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 22.060902669000001, 2024-09-13T10:12:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0055737549999999997, 2024-09-13T10:12:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.114 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3522 2024-09-13T10:12:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.963159+0000", 2024-09-13T10:12:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.963159+0000", 2024-09-13T10:12:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.115 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.963162+0000", 2024-09-13T10:12:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9869999999999999e-06 2024-09-13T10:12:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.963664+0000", 2024-09-13T10:12:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00050161699999999999 2024-09-13T10:12:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.116 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.963691+0000", 2024-09-13T10:12:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7158999999999999e-05 2024-09-13T10:12:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.963711+0000", 2024-09-13T10:12:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0196000000000001e-05 2024-09-13T10:12:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.963823+0000", 2024-09-13T10:12:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000111841 2024-09-13T10:12:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.117 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.963991+0000", 2024-09-13T10:12:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000168085 2024-09-13T10:12:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:12:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.964239+0000", 2024-09-13T10:12:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00024774399999999999 2024-09-13T10:12:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.967726+0000", 2024-09-13T10:12:11.118 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003487244 2024-09-13T10:12:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.967978+0000", 2024-09-13T10:12:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025163600000000001 2024-09-13T10:12:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.968662+0000", 2024-09-13T10:12:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00068363600000000005 2024-09-13T10:12:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:11.119 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.968696+0000", 2024-09-13T10:12:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4764000000000001e-05 2024-09-13T10:12:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:48.968733+0000", 2024-09-13T10:12:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6845999999999999e-05 2024-09-13T10:12:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3759 4.d 4:b19b932a:::smithi19530714-41:head [read 0~3126135,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e90)", 2024-09-13T10:12:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:54.660700+0000", 2024-09-13T10:12:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 16.363361754, 2024-09-13T10:12:11.120 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0062921970000000002, 2024-09-13T10:12:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3759 2024-09-13T10:12:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.660700+0000", 2024-09-13T10:12:11.121 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.660700+0000", 2024-09-13T10:12:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.660701+0000", 2024-09-13T10:12:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4300000000000002e-07 2024-09-13T10:12:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.122 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.660706+0000", 2024-09-13T10:12:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6739999999999996e-06 2024-09-13T10:12:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.660714+0000", 2024-09-13T10:12:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.4200000000000007e-06 2024-09-13T10:12:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.660721+0000", 2024-09-13T10:12:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1339999999999998e-06 2024-09-13T10:12:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.123 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.662455+0000", 2024-09-13T10:12:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017337800000000001 2024-09-13T10:12:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.662562+0000", 2024-09-13T10:12:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010692399999999999 2024-09-13T10:12:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.666992+0000", 2024-09-13T10:12:11.124 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0044304219999999998 2024-09-13T10:12:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:4162 4.d 4:ba7d2bc5:::smithi19530714-46:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:12:05.151653+0000", 2024-09-13T10:12:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.8724093719999999, 2024-09-13T10:12:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.72483114100000001, 2024-09-13T10:12:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:11.125 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4162 2024-09-13T10:12:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.151653+0000", 2024-09-13T10:12:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.126 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.151653+0000", 2024-09-13T10:12:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.151656+0000", 2024-09-13T10:12:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4860000000000002e-06 2024-09-13T10:12:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.151684+0000", 2024-09-13T10:12:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.783e-05 2024-09-13T10:12:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.127 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.151715+0000", 2024-09-13T10:12:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1142000000000001e-05 2024-09-13T10:12:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.151732+0000", 2024-09-13T10:12:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.736e-05 2024-09-13T10:12:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.151872+0000", 2024-09-13T10:12:11.128 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013973800000000001 2024-09-13T10:12:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op must wait for map", 2024-09-13T10:12:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.151883+0000", 2024-09-13T10:12:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0623e-05 2024-09-13T10:12:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.155343+0000", 2024-09-13T10:12:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034601359999999999 2024-09-13T10:12:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.129 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.155489+0000", 2024-09-13T10:12:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000145921 2024-09-13T10:12:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.876484+0000", 2024-09-13T10:12:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.72099490499999996 2024-09-13T10:12:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.130 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:4163 4.d 4:b19b932a:::smithi19530714-41:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e98)", 2024-09-13T10:12:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:12:05.151788+0000", 2024-09-13T10:12:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 5.8722742910000001, 2024-09-13T10:12:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047290068999999997, 2024-09-13T10:12:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4163 2024-09-13T10:12:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.131 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.151788+0000", 2024-09-13T10:12:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.151788+0000", 2024-09-13T10:12:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.151789+0000", 2024-09-13T10:12:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.316e-06 2024-09-13T10:12:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.132 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.151796+0000", 2024-09-13T10:12:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2740000000000002e-06 2024-09-13T10:12:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.151815+0000", 2024-09-13T10:12:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8249000000000001e-05 2024-09-13T10:12:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.151831+0000", 2024-09-13T10:12:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6310000000000001e-05 2024-09-13T10:12:11.133 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.152066+0000", 2024-09-13T10:12:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023493900000000001 2024-09-13T10:12:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting_for_map not empty", 2024-09-13T10:12:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.152080+0000", 2024-09-13T10:12:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3873000000000001e-05 2024-09-13T10:12:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.134 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.192931+0000", 2024-09-13T10:12:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.040851650000000003 2024-09-13T10:12:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.193046+0000", 2024-09-13T10:12:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011479100000000001 2024-09-13T10:12:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 7,12", 2024-09-13T10:12:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.193217+0000", 2024-09-13T10:12:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017118999999999999 2024-09-13T10:12:11.135 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.197334+0000", 2024-09-13T10:12:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004117084 2024-09-13T10:12:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.197652+0000", 2024-09-13T10:12:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00031781300000000001 2024-09-13T10:12:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:11.136 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.198788+0000", 2024-09-13T10:12:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0011355250000000001 2024-09-13T10:12:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.198955+0000", 2024-09-13T10:12:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000166854 2024-09-13T10:12:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:05.199078+0000", 2024-09-13T10:12:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123201 2024-09-13T10:12:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.137 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:4360 4.d 4:ba7d2bc5:::smithi19530714-46:head [read 0~3529503,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-13T10:12:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:12:09.857539+0000", 2024-09-13T10:12:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.1665229180000001, 2024-09-13T10:12:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094979009999999996, 2024-09-13T10:12:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.138 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4360 2024-09-13T10:12:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.857539+0000", 2024-09-13T10:12:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.857539+0000", 2024-09-13T10:12:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.139 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.857540+0000", 2024-09-13T10:12:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7400000000000003e-07 2024-09-13T10:12:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.857543+0000", 2024-09-13T10:12:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5599999999999998e-06 2024-09-13T10:12:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.140 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.857551+0000", 2024-09-13T10:12:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1249999999999993e-06 2024-09-13T10:12:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.857557+0000", 2024-09-13T10:12:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.4709999999999998e-06 2024-09-13T10:12:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.857649+0000", 2024-09-13T10:12:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2305999999999993e-05 2024-09-13T10:12:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.141 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting_for_map not empty", 2024-09-13T10:12:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.857655+0000", 2024-09-13T10:12:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5339999999999999e-06 2024-09-13T10:12:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.862694+0000", 2024-09-13T10:12:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0050393030000000002 2024-09-13T10:12:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.862920+0000", 2024-09-13T10:12:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022587 2024-09-13T10:12:11.142 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.867037+0000", 2024-09-13T10:12:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0041171580000000001 2024-09-13T10:12:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:4361 4.d 4:ba7d2bc5:::smithi19530714-46:head [sparse-read 0~2767647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-13T10:12:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:12:09.857604+0000", 2024-09-13T10:12:11.143 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 1.1664584119999999, 2024-09-13T10:12:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012206473000000001, 2024-09-13T10:12:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4361 2024-09-13T10:12:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.144 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.857604+0000", 2024-09-13T10:12:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.857604+0000", 2024-09-13T10:12:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.857606+0000", 2024-09-13T10:12:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2230000000000001e-06 2024-09-13T10:12:11.145 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.857616+0000", 2024-09-13T10:12:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0386e-05 2024-09-13T10:12:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.857646+0000", 2024-09-13T10:12:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.97e-05 2024-09-13T10:12:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:11.146 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.857667+0000", 2024-09-13T10:12:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1060000000000002e-05 2024-09-13T10:12:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.857729+0000", 2024-09-13T10:12:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.1809000000000004e-05 2024-09-13T10:12:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting_for_map not empty", 2024-09-13T10:12:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.857734+0000", 2024-09-13T10:12:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.6990000000000002e-06 2024-09-13T10:12:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.147 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.867126+0000", 2024-09-13T10:12:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0093929270000000006 2024-09-13T10:12:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.867459+0000", 2024-09-13T10:12:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033235799999999999 2024-09-13T10:12:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:09.869810+0000", 2024-09-13T10:12:11.148 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002351311 2024-09-13T10:12:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:11.149 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:11.149 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:11.151 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T10:12:11.152 INFO:tasks.rados.rados.0.smithi195.stdout:1814: finishing write tid 1 to smithi19530714-14 2024-09-13T10:12:11.152 INFO:tasks.rados.rados.0.smithi195.stdout:1815: got (22) Invalid argument 2024-09-13T10:12:11.153 INFO:tasks.rados.rados.0.smithi195.stdout:1814: finishing write tid 2 to smithi19530714-14 2024-09-13T10:12:11.153 INFO:tasks.rados.rados.0.smithi195.stdout:1810: done (4 left) 2024-09-13T10:12:11.153 INFO:tasks.rados.rados.0.smithi195.stdout:1812: done (3 left) 2024-09-13T10:12:11.153 INFO:tasks.rados.rados.0.smithi195.stdout:1815: done (2 left) 2024-09-13T10:12:11.153 INFO:tasks.rados.rados.0.smithi195.stdout:1816: done (1 left) 2024-09-13T10:12:11.153 INFO:tasks.rados.rados.0.smithi195.stdout:1817: read oid 37 snap -1 2024-09-13T10:12:11.153 INFO:tasks.rados.rados.0.smithi195.stdout:1817: expect (ObjNum 368 snap 0 seq_num 368) 2024-09-13T10:12:11.153 INFO:tasks.rados.rados.0.smithi195.stdout:1818: read oid 5 snap -1 2024-09-13T10:12:11.153 INFO:tasks.rados.rados.0.smithi195.stdout:1818: expect (ObjNum 360 snap 0 seq_num 360) 2024-09-13T10:12:11.153 INFO:tasks.rados.rados.0.smithi195.stdout:1819: write (excl) oid 49 current snap is 0 2024-09-13T10:12:11.153 INFO:tasks.rados.rados.0.smithi195.stdout:1814: finishing write tid 3 to smithi19530714-14 2024-09-13T10:12:11.153 INFO:tasks.rados.rados.0.smithi195.stdout:1814: finishing write tid 4 to smithi19530714-14 2024-09-13T10:12:11.154 INFO:tasks.rados.rados.0.smithi195.stdout:1814: oid 14 updating version 0 to 126 2024-09-13T10:12:11.154 INFO:tasks.rados.rados.0.smithi195.stdout:1814: oid 14 version 126 is already newer than 125 2024-09-13T10:12:11.154 INFO:tasks.rados.rados.0.smithi195.stdout:1814: oid 14 updating version 126 to 127 2024-09-13T10:12:11.154 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 127 (ObjNum 398 snap 0 seq_num 398) dirty exists 2024-09-13T10:12:11.154 INFO:tasks.rados.rados.0.smithi195.stdout:1814: left oid 14 (ObjNum 398 snap 0 seq_num 398) 2024-09-13T10:12:11.154 INFO:tasks.rados.rados.0.smithi195.stdout:1819: seq_num 400 ranges {722800=549956,2040562=682238} 2024-09-13T10:12:11.154 INFO:tasks.rados.rados.0.smithi195.stdout:1819: writing smithi19530714-49 from 722800 to 1272756 tid 1 2024-09-13T10:12:11.154 INFO:tasks.rados.rados.0.smithi195.stdout:1819: writing smithi19530714-49 from 2040562 to 2722800 tid 2 2024-09-13T10:12:11.154 INFO:tasks.rados.rados.0.smithi195.stdout:1814: done (3 left) 2024-09-13T10:12:11.154 INFO:tasks.rados.rados.0.smithi195.stdout:1820: write oid 29 current snap is 0 2024-09-13T10:12:11.154 INFO:tasks.rados.rados.0.smithi195.stdout:1820: seq_num 401 ranges {739607=625205,1982691=499457} 2024-09-13T10:12:11.154 INFO:tasks.rados.rados.0.smithi195.stdout:1820: writing smithi19530714-29 from 739607 to 1364812 tid 1 2024-09-13T10:12:11.154 INFO:tasks.rados.rados.0.smithi195.stdout:1820: writing smithi19530714-29 from 1982691 to 2482148 tid 2 2024-09-13T10:12:11.155 INFO:tasks.rados.rados.0.smithi195.stdout:1821: read oid 38 snap -1 2024-09-13T10:12:11.155 INFO:tasks.rados.rados.0.smithi195.stdout:1821: expect (ObjNum 298 snap 0 seq_num 298) 2024-09-13T10:12:11.155 INFO:tasks.rados.rados.0.smithi195.stdout:1822: write (excl) oid 31 current snap is 0 2024-09-13T10:12:11.155 INFO:tasks.rados.rados.0.smithi195.stdout:1822: seq_num 402 ranges {756414=700454,2041173=715241} 2024-09-13T10:12:11.155 INFO:tasks.rados.rados.0.smithi195.stdout:1822: writing smithi19530714-31 from 756414 to 1456868 tid 1 2024-09-13T10:12:11.155 INFO:tasks.rados.rados.0.smithi195.stdout:1822: writing smithi19530714-31 from 2041173 to 2756414 tid 2 2024-09-13T10:12:11.155 INFO:tasks.rados.rados.0.smithi195.stdout:1823: write (excl) oid 48 current snap is 0 2024-09-13T10:12:11.155 INFO:tasks.rados.rados.0.smithi195.stdout:1823: seq_num 403 ranges {773221=492056,1816008=503126} 2024-09-13T10:12:11.156 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:12:11.147+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:12:11.156 INFO:tasks.rados.rados.0.smithi195.stdout:1823: writing smithi19530714-48 from 773221 to 1265277 tid 1 2024-09-13T10:12:11.161 INFO:tasks.rados.rados.0.smithi195.stdout:1823: writing smithi19530714-48 from 1816008 to 2319134 tid 2 2024-09-13T10:12:11.161 INFO:tasks.rados.rados.0.smithi195.stdout:1824: read oid 8 snap -1 2024-09-13T10:12:11.161 INFO:tasks.rados.rados.0.smithi195.stdout:1824: expect (ObjNum 351 snap 0 seq_num 351) 2024-09-13T10:12:11.161 INFO:tasks.rados.rados.0.smithi195.stdout:1825: read oid 36 snap -1 2024-09-13T10:12:11.162 INFO:tasks.rados.rados.0.smithi195.stdout:1825: expect deleted 2024-09-13T10:12:11.162 INFO:tasks.rados.rados.0.smithi195.stdout:1826: set_chunk oid 44 target oid 2024-09-13T10:12:11.162 INFO:tasks.rados.rados.0.smithi195.stdout:1826: set_chunk oid 44 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:12:11.162 INFO:tasks.rados.rados.0.smithi195.stdout:1827: read oid 11 snap -1 2024-09-13T10:12:11.162 INFO:tasks.rados.rados.0.smithi195.stdout:1827: expect (ObjNum 359 snap 0 seq_num 359) 2024-09-13T10:12:11.162 INFO:tasks.rados.rados.0.smithi195.stdout:1828: tier_promote oid 18 2024-09-13T10:12:11.162 INFO:tasks.rados.rados.0.smithi195.stdout:1829: read oid 24 snap -1 2024-09-13T10:12:11.162 INFO:tasks.rados.rados.0.smithi195.stdout:1829: expect (ObjNum 370 snap 0 seq_num 370) 2024-09-13T10:12:11.162 INFO:tasks.rados.rados.0.smithi195.stdout:1830: delete oid 35 current snap is 0 2024-09-13T10:12:11.162 INFO:tasks.rados.rados.0.smithi195.stdout:1817: expect (ObjNum 368 snap 0 seq_num 368) 2024-09-13T10:12:11.181 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:12:11.251 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:11.246+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:11.279 INFO:tasks.rados.rados.0.smithi195.stdout:1818: expect (ObjNum 360 snap 0 seq_num 360) 2024-09-13T10:12:11.279 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:11.279 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:12:11.279 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:12:11.279 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:11.289 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1382618 2024-09-13T10:12:11.294 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T10:12:11.338 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:12:11.338 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:12:11.344 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:11.352 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:11.346+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:11.359 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:12:11.359 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:12:11.380 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:12:11.380 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:12:11.383 INFO:tasks.rados.rados.0.smithi195.stdout:1819: finishing write tid 1 to smithi19530714-49 2024-09-13T10:12:11.383 INFO:tasks.rados.rados.0.smithi195.stdout:1819: finishing write tid 2 to smithi19530714-49 2024-09-13T10:12:11.383 INFO:tasks.rados.rados.0.smithi195.stdout:1819: finishing write tid 3 to smithi19530714-49 2024-09-13T10:12:11.383 INFO:tasks.rados.rados.0.smithi195.stdout:1819: finishing write tid 4 to smithi19530714-49 2024-09-13T10:12:11.383 INFO:tasks.rados.rados.0.smithi195.stdout:1819: oid 49 updating version 0 to 182 2024-09-13T10:12:11.383 INFO:tasks.rados.rados.0.smithi195.stdout:1819: oid 49 version 182 is already newer than 181 2024-09-13T10:12:11.384 INFO:tasks.rados.rados.0.smithi195.stdout:1819: oid 49 updating version 182 to 183 2024-09-13T10:12:11.384 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 183 (ObjNum 400 snap 0 seq_num 400) dirty exists 2024-09-13T10:12:11.384 INFO:tasks.rados.rados.0.smithi195.stdout:1819: left oid 49 (ObjNum 400 snap 0 seq_num 400) 2024-09-13T10:12:11.384 INFO:tasks.rados.rados.0.smithi195.stdout:1820: finishing write tid 1 to smithi19530714-29 2024-09-13T10:12:11.384 INFO:tasks.rados.rados.0.smithi195.stdout:1820: finishing write tid 2 to smithi19530714-29 2024-09-13T10:12:11.384 INFO:tasks.rados.rados.0.smithi195.stdout:1820: finishing write tid 3 to smithi19530714-29 2024-09-13T10:12:11.384 INFO:tasks.rados.rados.0.smithi195.stdout:1820: finishing write tid 4 to smithi19530714-29 2024-09-13T10:12:11.384 INFO:tasks.rados.rados.0.smithi195.stdout:1820: oid 29 updating version 0 to 130 2024-09-13T10:12:11.384 INFO:tasks.rados.rados.0.smithi195.stdout:1820: oid 29 version 130 is already newer than 129 2024-09-13T10:12:11.384 INFO:tasks.rados.rados.0.smithi195.stdout:1820: oid 29 version 130 is already newer than 128 2024-09-13T10:12:11.385 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 130 (ObjNum 401 snap 0 seq_num 401) dirty exists 2024-09-13T10:12:11.385 INFO:tasks.rados.rados.0.smithi195.stdout:1820: left oid 29 (ObjNum 401 snap 0 seq_num 401) 2024-09-13T10:12:11.385 INFO:tasks.rados.rados.0.smithi195.stdout:1817: done (13 left) 2024-09-13T10:12:11.385 INFO:tasks.rados.rados.0.smithi195.stdout:1818: done (12 left) 2024-09-13T10:12:11.385 INFO:tasks.rados.rados.0.smithi195.stdout:1819: done (11 left) 2024-09-13T10:12:11.385 INFO:tasks.rados.rados.0.smithi195.stdout:1820: done (10 left) 2024-09-13T10:12:11.385 INFO:tasks.rados.rados.0.smithi195.stdout:1830: done (9 left) 2024-09-13T10:12:11.385 INFO:tasks.rados.rados.0.smithi195.stdout:1831: set_chunk oid 21 target oid 2024-09-13T10:12:11.392 INFO:tasks.rados.rados.0.smithi195.stdout:1831: set_chunk oid 21 offset: 61952 length: 288256 target oid be3608e547e9a7268a63d1d086aa58756904f4bd13e4dddfacc46c62eb9fd92d offset: 0 snap -1 2024-09-13T10:12:11.392 INFO:tasks.rados.rados.0.smithi195.stdout:1821: expect (ObjNum 298 snap 0 seq_num 298) 2024-09-13T10:12:11.393 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:11.401 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:12:11.401 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:12:11.418 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:12:11.418 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:12:11.439 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:12:11.439 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:12:11.453 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:11.449+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:11.453 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:11.453 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:11.460 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:12:11.460 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:12:11.467 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T10:12:11.480 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:12:11.481 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:12:11.498 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:12:11.498 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:12:11.519 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:12:11.519 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:12:11.528 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2541888 2024-09-13T10:12:11.538 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:12:11.538 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:12:11.554 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:11.550+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:11.559 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:12:11.559 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:12:11.576 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:12:11.576 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:12:11.596 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:12:11.596 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:12:11.603 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:11.603 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:11.616 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T10:12:11.617 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:12:11.618 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:12:11.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:12:11.610+0000 7ff9bf7fe640 0 -- 172.21.15.73:0/2266954940 >> v2:172.21.15.195:6805/848686366 conn(0x7ff9c0225940 msgr2=0x7ff9c0227d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-13T10:12:11.629 INFO:tasks.rados.rados.0.smithi195.stdout:1822: finishing write tid 1 to smithi19530714-31 2024-09-13T10:12:11.629 INFO:tasks.rados.rados.0.smithi195.stdout:1822: finishing write tid 2 to smithi19530714-31 2024-09-13T10:12:11.629 INFO:tasks.rados.rados.0.smithi195.stdout:1822: finishing write tid 3 to smithi19530714-31 2024-09-13T10:12:11.630 INFO:tasks.rados.rados.0.smithi195.stdout:1822: finishing write tid 4 to smithi19530714-31 2024-09-13T10:12:11.630 INFO:tasks.rados.rados.0.smithi195.stdout:1822: oid 31 updating version 0 to 108 2024-09-13T10:12:11.630 INFO:tasks.rados.rados.0.smithi195.stdout:1822: oid 31 updating version 108 to 110 2024-09-13T10:12:11.630 INFO:tasks.rados.rados.0.smithi195.stdout:1822: oid 31 version 110 is already newer than 109 2024-09-13T10:12:11.630 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 110 (ObjNum 402 snap 0 seq_num 402) dirty exists 2024-09-13T10:12:11.630 INFO:tasks.rados.rados.0.smithi195.stdout:1822: left oid 31 (ObjNum 402 snap 0 seq_num 402) 2024-09-13T10:12:11.630 INFO:tasks.rados.rados.0.smithi195.stdout:1823: finishing write tid 1 to smithi19530714-48 2024-09-13T10:12:11.630 INFO:tasks.rados.rados.0.smithi195.stdout:1826: finishing set_chunk to oid 44 2024-09-13T10:12:11.630 INFO:tasks.rados.rados.0.smithi195.stdout:1826: got expected ENOENT (src dne) 2024-09-13T10:12:11.630 INFO:tasks.rados.rados.0.smithi195.stdout:1828: got (0) Success 2024-09-13T10:12:11.630 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 117 (ObjNum 338 snap 0 seq_num 338) dirty exists 2024-09-13T10:12:11.630 INFO:tasks.rados.rados.0.smithi195.stdout:1832: read oid 6 snap -1 2024-09-13T10:12:11.630 INFO:tasks.rados.rados.0.smithi195.stdout:1832: expect (ObjNum 337 snap 0 seq_num 337) 2024-09-13T10:12:11.630 INFO:tasks.rados.rados.0.smithi195.stdout:1821: done (10 left) 2024-09-13T10:12:11.631 INFO:tasks.rados.rados.0.smithi195.stdout:1822: done (9 left) 2024-09-13T10:12:11.631 INFO:tasks.rados.rados.0.smithi195.stdout:1825: done (8 left) 2024-09-13T10:12:11.631 INFO:tasks.rados.rados.0.smithi195.stdout:1826: done (7 left) 2024-09-13T10:12:11.631 INFO:tasks.rados.rados.0.smithi195.stdout:1828: done (6 left) 2024-09-13T10:12:11.631 INFO:tasks.rados.rados.0.smithi195.stdout:1833: read oid 32 snap -1 2024-09-13T10:12:11.631 INFO:tasks.rados.rados.0.smithi195.stdout:1833: expect (ObjNum 320 snap 0 seq_num 320) 2024-09-13T10:12:11.631 INFO:tasks.rados.rados.0.smithi195.stdout:1834: read oid 7 snap -1 2024-09-13T10:12:11.631 INFO:tasks.rados.rados.0.smithi195.stdout:1834: expect (ObjNum 276 snap 0 seq_num 276) 2024-09-13T10:12:11.631 INFO:tasks.rados.rados.0.smithi195.stdout:1835: read oid 35 snap -1 2024-09-13T10:12:11.631 INFO:tasks.rados.rados.0.smithi195.stdout:1835: expect deleted 2024-09-13T10:12:11.631 INFO:tasks.rados.rados.0.smithi195.stdout:1836: read oid 20 snap -1 2024-09-13T10:12:11.631 INFO:tasks.rados.rados.0.smithi195.stdout:1836: expect (ObjNum 389 snap 0 seq_num 389) 2024-09-13T10:12:11.631 INFO:tasks.rados.rados.0.smithi195.stdout:1837: read oid 36 snap -1 2024-09-13T10:12:11.631 INFO:tasks.rados.rados.0.smithi195.stdout:1837: expect deleted 2024-09-13T10:12:11.631 INFO:tasks.rados.rados.0.smithi195.stdout:1838: write oid 1 current snap is 0 2024-09-13T10:12:11.632 INFO:tasks.rados.rados.0.smithi195.stdout:1823: finishing write tid 2 to smithi19530714-48 2024-09-13T10:12:11.632 INFO:tasks.rados.rados.0.smithi195.stdout:1838: seq_num 405 ranges {406835=642554,1532972=506795,2772312=34523} 2024-09-13T10:12:11.646 INFO:tasks.rados.rados.0.smithi195.stdout:1838: writing smithi19530714-1 from 406835 to 1049389 tid 1 2024-09-13T10:12:11.655 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:12:11.655+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:12:11.663 INFO:tasks.rados.rados.0.smithi195.stdout:1838: writing smithi19530714-1 from 1532972 to 2039767 tid 2 2024-09-13T10:12:11.675 INFO:tasks.rados.rados.0.smithi195.stdout:1838: writing smithi19530714-1 from 2772312 to 2806835 tid 3 2024-09-13T10:12:11.675 INFO:tasks.rados.rados.0.smithi195.stdout:1839: set_chunk oid 13 target oid 2024-09-13T10:12:11.707 INFO:tasks.rados.rados.0.smithi195.stdout:1839: set_chunk oid 13 offset: 548352 length: 1741824 target oid d2907f63dd40e9c47bd41bc685ca7ed8c0ac5bf670c91972c1b4c2d6aa49c70c offset: 0 snap -1 2024-09-13T10:12:11.707 INFO:tasks.rados.rados.0.smithi195.stdout:1840: set_chunk oid 12 target oid 2024-09-13T10:12:11.719 INFO:tasks.rados.rados.0.smithi195.stdout:1840: set_chunk oid 12 offset: 587776 length: 273920 target oid 444ce06168b86ebc7b135b2d1b08c48a1bc621dd139977f8839f16befd38cd20 offset: 0 snap -1 2024-09-13T10:12:11.719 INFO:tasks.rados.rados.0.smithi195.stdout:1841: read oid 50 snap -1 2024-09-13T10:12:11.719 INFO:tasks.rados.rados.0.smithi195.stdout:1841: expect (ObjNum 303 snap 0 seq_num 303) 2024-09-13T10:12:11.719 INFO:tasks.rados.rados.0.smithi195.stdout:1842: read oid 23 snap -1 2024-09-13T10:12:11.719 INFO:tasks.rados.rados.0.smithi195.stdout:1842: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:12:11.719 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:12:11.719 INFO:tasks.rados.rados.0.smithi195.stdout:1823: finishing write tid 3 to smithi19530714-48 2024-09-13T10:12:11.719 INFO:tasks.rados.rados.0.smithi195.stdout:1823: finishing write tid 4 to smithi19530714-48 2024-09-13T10:12:11.719 INFO:tasks.rados.rados.0.smithi195.stdout:1823: oid 48 updating version 0 to 132 2024-09-13T10:12:11.719 INFO:tasks.rados.rados.0.smithi195.stdout:1823: oid 48 version 132 is already newer than 131 2024-09-13T10:12:11.719 INFO:tasks.rados.rados.0.smithi195.stdout:1823: oid 48 updating version 132 to 133 2024-09-13T10:12:11.719 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 133 (ObjNum 403 snap 0 seq_num 403) dirty exists 2024-09-13T10:12:11.720 INFO:tasks.rados.rados.0.smithi195.stdout:1823: left oid 48 (ObjNum 403 snap 0 seq_num 403) 2024-09-13T10:12:11.720 INFO:tasks.rados.rados.0.smithi195.stdout:1829: expect (ObjNum 370 snap 0 seq_num 370) 2024-09-13T10:12:11.720 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:11.731 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:11.731 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:11.745 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T10:12:11.756 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:11.754+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:11.770 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:11.795 INFO:tasks.rados.rados.0.smithi195.stdout:1827: expect (ObjNum 359 snap 0 seq_num 359) 2024-09-13T10:12:11.795 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:11.815 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:11.830 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-13T10:12:11.854 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:12:11.854 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:12:11.857 INFO:tasks.rados.rados.0.smithi195.stdout:1824: expect (ObjNum 351 snap 0 seq_num 351) 2024-09-13T10:12:11.857 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:11.850+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:11.860 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 641322 2024-09-13T10:12:11.884 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:11.884 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:11.897 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T10:12:11.958 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:11.959 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:11.953+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:12.011 INFO:tasks.rados.rados.0.smithi195.stdout:1831: finishing set_chunk to oid 21 2024-09-13T10:12:12.011 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 144 (ObjNum 380 snap 0 seq_num 380) dirty exists 2024-09-13T10:12:12.011 INFO:tasks.rados.rados.0.smithi195.stdout:1836: expect (ObjNum 389 snap 0 seq_num 389) 2024-09-13T10:12:12.018 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2181579 2024-09-13T10:12:12.056 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:12.056 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:12.060 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:12.053+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:12.069 INFO:tasks.rados.rados.0.smithi195.stdout:1832: expect (ObjNum 337 snap 0 seq_num 337) 2024-09-13T10:12:12.070 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-13T10:12:12.161 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:12.157+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:12.195 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:12.195 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:12.209 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-13T10:12:12.262 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:12.254+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:12.307 INFO:tasks.rados.rados.0.smithi195.stdout:1833: expect (ObjNum 320 snap 0 seq_num 320) 2024-09-13T10:12:12.333 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:12.333 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:12.345 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:12.347 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-13T10:12:12.363 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:12.357+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:12.446 INFO:tasks.rados.rados.0.smithi195.stdout:1834: expect (ObjNum 276 snap 0 seq_num 276) 2024-09-13T10:12:12.464 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:12.458+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:12.491 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1212126 2024-09-13T10:12:12.513 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:12.513 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:12.518 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:12.527 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-13T10:12:12.557 INFO:tasks.rados.rados.0.smithi195.stdout:1838: finishing write tid 1 to smithi19530714-1 2024-09-13T10:12:12.557 INFO:tasks.rados.rados.0.smithi195.stdout:1838: finishing write tid 2 to smithi19530714-1 2024-09-13T10:12:12.557 INFO:tasks.rados.rados.0.smithi195.stdout:1838: finishing write tid 3 to smithi19530714-1 2024-09-13T10:12:12.558 INFO:tasks.rados.rados.0.smithi195.stdout:1838: finishing write tid 4 to smithi19530714-1 2024-09-13T10:12:12.558 INFO:tasks.rados.rados.0.smithi195.stdout:1838: finishing write tid 5 to smithi19530714-1 2024-09-13T10:12:12.558 INFO:tasks.rados.rados.0.smithi195.stdout:1838: oid 1 updating version 0 to 111 2024-09-13T10:12:12.558 INFO:tasks.rados.rados.0.smithi195.stdout:1838: oid 1 updating version 111 to 113 2024-09-13T10:12:12.558 INFO:tasks.rados.rados.0.smithi195.stdout:1838: oid 1 version 113 is already newer than 112 2024-09-13T10:12:12.558 INFO:tasks.rados.rados.0.smithi195.stdout:1838: oid 1 updating version 113 to 114 2024-09-13T10:12:12.558 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 114 (ObjNum 405 snap 0 seq_num 405) dirty exists 2024-09-13T10:12:12.558 INFO:tasks.rados.rados.0.smithi195.stdout:1838: left oid 1 (ObjNum 405 snap 0 seq_num 405) 2024-09-13T10:12:12.558 INFO:tasks.rados.rados.0.smithi195.stdout:1839: finishing set_chunk to oid 13 2024-09-13T10:12:12.558 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 84 (ObjNum 342 snap 0 seq_num 342) dirty exists 2024-09-13T10:12:12.558 INFO:tasks.rados.rados.0.smithi195.stdout:1840: finishing set_chunk to oid 12 2024-09-13T10:12:12.558 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 124 (ObjNum 395 snap 0 seq_num 395) dirty exists 2024-09-13T10:12:12.558 INFO:tasks.rados.rados.0.smithi195.stdout:1842: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:12:12.558 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:12.565 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:12.562+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:12.644 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:12.644 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:12.657 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-13T10:12:12.666 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:12:12.663+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:12:12.713 INFO:tasks.rados.rados.0.smithi195.stdout:1841: expect (ObjNum 303 snap 0 seq_num 303) 2024-09-13T10:12:12.713 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:12.768 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:12:12.767+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:12:12.779 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:12.779 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:12.780 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:12.793 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-13T10:12:12.845 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:12.869 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:12.862+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:12.913 INFO:tasks.rados.rados.0.smithi195.stdout:1823: done (15 left) 2024-09-13T10:12:12.914 INFO:tasks.rados.rados.0.smithi195.stdout:1824: done (14 left) 2024-09-13T10:12:12.914 INFO:tasks.rados.rados.0.smithi195.stdout:1827: done (13 left) 2024-09-13T10:12:12.914 INFO:tasks.rados.rados.0.smithi195.stdout:1829: done (12 left) 2024-09-13T10:12:12.914 INFO:tasks.rados.rados.0.smithi195.stdout:1831: done (11 left) 2024-09-13T10:12:12.914 INFO:tasks.rados.rados.0.smithi195.stdout:1832: done (10 left) 2024-09-13T10:12:12.914 INFO:tasks.rados.rados.0.smithi195.stdout:1833: done (9 left) 2024-09-13T10:12:12.914 INFO:tasks.rados.rados.0.smithi195.stdout:1834: done (8 left) 2024-09-13T10:12:12.914 INFO:tasks.rados.rados.0.smithi195.stdout:1835: done (7 left) 2024-09-13T10:12:12.914 INFO:tasks.rados.rados.0.smithi195.stdout:1836: done (6 left) 2024-09-13T10:12:12.914 INFO:tasks.rados.rados.0.smithi195.stdout:1837: done (5 left) 2024-09-13T10:12:12.914 INFO:tasks.rados.rados.0.smithi195.stdout:1838: done (4 left) 2024-09-13T10:12:12.914 INFO:tasks.rados.rados.0.smithi195.stdout:1839: done (3 left) 2024-09-13T10:12:12.915 INFO:tasks.rados.rados.0.smithi195.stdout:1840: done (2 left) 2024-09-13T10:12:12.915 INFO:tasks.rados.rados.0.smithi195.stdout:1841: done (1 left) 2024-09-13T10:12:12.915 INFO:tasks.rados.rados.0.smithi195.stdout:1842: done (0 left) 2024-09-13T10:12:12.915 INFO:tasks.rados.rados.0.smithi195.stdout:1843: set_chunk oid 14 target oid 2024-09-13T10:12:12.917 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:12.917 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:12.931 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T10:12:12.942 INFO:tasks.rados.rados.0.smithi195.stdout:1843: set_chunk oid 14 offset: 632320 length: 1436672 target oid 11f1ef5e6f3e3d83a70283a48bc0c15e233aa2463ec2fa45095f617740b932f2 offset: 0 snap -1 2024-09-13T10:12:12.942 INFO:tasks.rados.rados.0.smithi195.stdout:1844: tier_evict oid 2 2024-09-13T10:12:12.942 INFO:tasks.rados.rados.0.smithi195.stdout:1844: tier_evict oid 2 snap -1 2024-09-13T10:12:12.942 INFO:tasks.rados.rados.0.smithi195.stdout:1845: read oid 20 snap -1 2024-09-13T10:12:12.942 INFO:tasks.rados.rados.0.smithi195.stdout:1845: expect (ObjNum 389 snap 0 seq_num 389) 2024-09-13T10:12:12.942 INFO:tasks.rados.rados.0.smithi195.stdout:1846: read oid 9 snap -1 2024-09-13T10:12:12.942 INFO:tasks.rados.rados.0.smithi195.stdout:1846: expect (ObjNum 259 snap 0 seq_num 259) 2024-09-13T10:12:12.942 INFO:tasks.rados.rados.0.smithi195.stdout:1847: read oid 25 snap -1 2024-09-13T10:12:12.943 INFO:tasks.rados.rados.0.smithi195.stdout:1847: expect (ObjNum 324 snap 0 seq_num 324) 2024-09-13T10:12:12.943 INFO:tasks.rados.rados.0.smithi195.stdout:1848: write (excl) oid 42 current snap is 0 2024-09-13T10:12:12.943 INFO:tasks.rados.rados.0.smithi195.stdout:1848: seq_num 406 ranges {423642=717803,1875101=766806} 2024-09-13T10:12:12.948 INFO:tasks.rados.rados.0.smithi195.stdout:1848: writing smithi19530714-42 from 423642 to 1141445 tid 1 2024-09-13T10:12:12.956 INFO:tasks.rados.rados.0.smithi195.stdout:1848: writing smithi19530714-42 from 1875101 to 2641907 tid 2 2024-09-13T10:12:12.957 INFO:tasks.rados.rados.0.smithi195.stdout:1849: write oid 7 current snap is 0 2024-09-13T10:12:12.957 INFO:tasks.rados.rados.0.smithi195.stdout:1849: seq_num 407 ranges {440449=793052,1933583=510464} 2024-09-13T10:12:12.963 INFO:tasks.rados.rados.0.smithi195.stdout:1849: writing smithi19530714-7 from 440449 to 1233501 tid 1 2024-09-13T10:12:12.974 INFO:tasks.rados.rados.0.smithi195.stdout:1849: writing smithi19530714-7 from 1933583 to 2444047 tid 2 2024-09-13T10:12:12.974 INFO:tasks.rados.rados.0.smithi195.stdout:1850: read oid 12 snap -1 2024-09-13T10:12:12.974 INFO:tasks.rados.rados.0.smithi195.stdout:1850: expect (ObjNum 395 snap 0 seq_num 395) 2024-09-13T10:12:12.975 INFO:tasks.rados.rados.0.smithi195.stdout:1851: write (excl) oid 33 current snap is 0 2024-09-13T10:12:12.975 INFO:tasks.rados.rados.0.smithi195.stdout:1851: seq_num 408 ranges {457256=468301,1592065=770475,2854581=2675} 2024-09-13T10:12:12.975 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:12:12.975+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:12:12.982 INFO:tasks.rados.rados.0.smithi195.stdout:1851: writing smithi19530714-33 from 457256 to 925557 tid 1 2024-09-13T10:12:12.993 INFO:tasks.rados.rados.0.smithi195.stdout:1851: writing smithi19530714-33 from 1592065 to 2362540 tid 2 2024-09-13T10:12:12.996 INFO:tasks.rados.rados.0.smithi195.stdout:1851: writing smithi19530714-33 from 2854581 to 2857256 tid 3 2024-09-13T10:12:12.996 INFO:tasks.rados.rados.0.smithi195.stdout:1852: write (excl) oid 35 current snap is 0 2024-09-13T10:12:12.996 INFO:tasks.rados.rados.0.smithi195.stdout:1852: seq_num 409 ranges {474063=543550,1650547=514133,2765651=108412} 2024-09-13T10:12:13.002 INFO:tasks.rados.rados.0.smithi195.stdout:1852: writing smithi19530714-35 from 474063 to 1017613 tid 1 2024-09-13T10:12:13.010 INFO:tasks.rados.rados.0.smithi195.stdout:1852: writing smithi19530714-35 from 1650547 to 2164680 tid 2 2024-09-13T10:12:13.014 INFO:tasks.rados.rados.0.smithi195.stdout:1852: writing smithi19530714-35 from 2765651 to 2874063 tid 3 2024-09-13T10:12:13.014 INFO:tasks.rados.rados.0.smithi195.stdout:1853: read oid 28 snap -1 2024-09-13T10:12:13.014 INFO:tasks.rados.rados.0.smithi195.stdout:1853: expect (ObjNum 369 snap 0 seq_num 369) 2024-09-13T10:12:13.014 INFO:tasks.rados.rados.0.smithi195.stdout:1854: set_chunk oid 43 target oid 2024-09-13T10:12:13.027 INFO:tasks.rados.rados.0.smithi195.stdout:1854: set_chunk oid 43 offset: 1236480 length: 512512 target oid 8620267f0b8a44fbc1cbbe1c0c949fd19d9d5e3129d16729e99bda668200270e offset: 0 snap -1 2024-09-13T10:12:13.027 INFO:tasks.rados.rados.0.smithi195.stdout:1855: write (excl) oid 48 current snap is 0 2024-09-13T10:12:13.027 INFO:tasks.rados.rados.0.smithi195.stdout:1844: got (22) Invalid argument 2024-09-13T10:12:13.027 INFO:tasks.rados.rados.0.smithi195.stdout:1855: seq_num 410 ranges {490870=618799,1592676=774144,2793074=97796} 2024-09-13T10:12:13.033 INFO:tasks.rados.rados.0.smithi195.stdout:1855: writing smithi19530714-48 from 490870 to 1109669 tid 1 2024-09-13T10:12:13.041 INFO:tasks.rados.rados.0.smithi195.stdout:1855: writing smithi19530714-48 from 1592676 to 2366820 tid 2 2024-09-13T10:12:13.045 INFO:tasks.rados.rados.0.smithi195.stdout:1855: writing smithi19530714-48 from 2793074 to 2890870 tid 3 2024-09-13T10:12:13.045 INFO:tasks.rados.rados.0.smithi195.stdout:1844: done (12 left) 2024-09-13T10:12:13.045 INFO:tasks.rados.rados.0.smithi195.stdout:1856: read oid 45 snap -1 2024-09-13T10:12:13.045 INFO:tasks.rados.rados.0.smithi195.stdout:1856: expect (ObjNum 397 snap 0 seq_num 397) 2024-09-13T10:12:13.045 INFO:tasks.rados.rados.0.smithi195.stdout:1857: set_chunk oid 34 target oid 2024-09-13T10:12:13.045 INFO:tasks.rados.rados.0.smithi195.stdout:1857: set_chunk oid 34 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:12:13.045 INFO:tasks.rados.rados.0.smithi195.stdout:1858: read oid 8 snap -1 2024-09-13T10:12:13.045 INFO:tasks.rados.rados.0.smithi195.stdout:1858: expect (ObjNum 351 snap 0 seq_num 351) 2024-09-13T10:12:13.046 INFO:tasks.rados.rados.0.smithi195.stdout:1859: read oid 37 snap -1 2024-09-13T10:12:13.046 INFO:tasks.rados.rados.0.smithi195.stdout:1859: expect (ObjNum 368 snap 0 seq_num 368) 2024-09-13T10:12:13.046 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:12:13.046 INFO:tasks.rados.rados.0.smithi195.stdout:1845: expect (ObjNum 389 snap 0 seq_num 389) 2024-09-13T10:12:13.070 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:13.065+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:13.073 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:13.073 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:13.088 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T10:12:13.105 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:13.160 INFO:tasks.rados.rados.0.smithi195.stdout:1846: expect (ObjNum 259 snap 0 seq_num 259) 2024-09-13T10:12:13.164 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1922901 2024-09-13T10:12:13.171 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:13.165+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:13.218 INFO:tasks.rados.rados.0.smithi195.stdout:1848: finishing write tid 1 to smithi19530714-42 2024-09-13T10:12:13.218 INFO:tasks.rados.rados.0.smithi195.stdout:1848: finishing write tid 2 to smithi19530714-42 2024-09-13T10:12:13.218 INFO:tasks.rados.rados.0.smithi195.stdout:1848: finishing write tid 3 to smithi19530714-42 2024-09-13T10:12:13.218 INFO:tasks.rados.rados.0.smithi195.stdout:1848: finishing write tid 4 to smithi19530714-42 2024-09-13T10:12:13.218 INFO:tasks.rados.rados.0.smithi195.stdout:1848: oid 42 updating version 0 to 29 2024-09-13T10:12:13.218 INFO:tasks.rados.rados.0.smithi195.stdout:1848: oid 42 updating version 29 to 30 2024-09-13T10:12:13.218 INFO:tasks.rados.rados.0.smithi195.stdout:1848: oid 42 version 30 is already newer than 28 2024-09-13T10:12:13.218 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 30 (ObjNum 406 snap 0 seq_num 406) dirty exists 2024-09-13T10:12:13.219 INFO:tasks.rados.rados.0.smithi195.stdout:1848: left oid 42 (ObjNum 406 snap 0 seq_num 406) 2024-09-13T10:12:13.219 INFO:tasks.rados.rados.0.smithi195.stdout:1843: finishing set_chunk to oid 14 2024-09-13T10:12:13.219 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 127 (ObjNum 398 snap 0 seq_num 398) dirty exists 2024-09-13T10:12:13.219 INFO:tasks.rados.rados.0.smithi195.stdout:1849: finishing write tid 1 to smithi19530714-7 2024-09-13T10:12:13.219 INFO:tasks.rados.rados.0.smithi195.stdout:1847: expect (ObjNum 324 snap 0 seq_num 324) 2024-09-13T10:12:13.219 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:13.256 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:13.256 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:13.271 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T10:12:13.273 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:12:13.266+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:12:13.294 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 521689 2024-09-13T10:12:13.374 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:13.369+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:13.392 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:13.392 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:13.406 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T10:12:13.427 INFO:tasks.rados.rados.0.smithi195.stdout:1849: finishing write tid 2 to smithi19530714-7 2024-09-13T10:12:13.427 INFO:tasks.rados.rados.0.smithi195.stdout:1849: finishing write tid 3 to smithi19530714-7 2024-09-13T10:12:13.427 INFO:tasks.rados.rados.0.smithi195.stdout:1849: finishing write tid 4 to smithi19530714-7 2024-09-13T10:12:13.428 INFO:tasks.rados.rados.0.smithi195.stdout:1849: oid 7 updating version 0 to 131 2024-09-13T10:12:13.428 INFO:tasks.rados.rados.0.smithi195.stdout:1849: oid 7 version 131 is already newer than 129 2024-09-13T10:12:13.428 INFO:tasks.rados.rados.0.smithi195.stdout:1849: oid 7 version 131 is already newer than 130 2024-09-13T10:12:13.428 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 7 v 131 (ObjNum 407 snap 0 seq_num 407) dirty exists 2024-09-13T10:12:13.428 INFO:tasks.rados.rados.0.smithi195.stdout:1849: left oid 7 (ObjNum 407 snap 0 seq_num 407) 2024-09-13T10:12:13.428 INFO:tasks.rados.rados.0.smithi195.stdout:1850: expect (ObjNum 395 snap 0 seq_num 395) 2024-09-13T10:12:13.474 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:13.469+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:13.489 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2413477 2024-09-13T10:12:13.501 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 297962 2024-09-13T10:12:13.528 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:13.529 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:13.542 INFO:tasks.rados.rados.0.smithi195.stdout:1852: finishing write tid 1 to smithi19530714-35 2024-09-13T10:12:13.542 INFO:tasks.rados.rados.0.smithi195.stdout:1852: finishing write tid 2 to smithi19530714-35 2024-09-13T10:12:13.542 INFO:tasks.rados.rados.0.smithi195.stdout:1852: finishing write tid 3 to smithi19530714-35 2024-09-13T10:12:13.543 INFO:tasks.rados.rados.0.smithi195.stdout:1852: finishing write tid 4 to smithi19530714-35 2024-09-13T10:12:13.543 INFO:tasks.rados.rados.0.smithi195.stdout:1852: finishing write tid 5 to smithi19530714-35 2024-09-13T10:12:13.543 INFO:tasks.rados.rados.0.smithi195.stdout:1852: oid 35 updating version 0 to 91 2024-09-13T10:12:13.543 INFO:tasks.rados.rados.0.smithi195.stdout:1852: oid 35 version 91 is already newer than 90 2024-09-13T10:12:13.543 INFO:tasks.rados.rados.0.smithi195.stdout:1852: oid 35 updating version 91 to 93 2024-09-13T10:12:13.543 INFO:tasks.rados.rados.0.smithi195.stdout:1852: oid 35 version 93 is already newer than 92 2024-09-13T10:12:13.543 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 93 (ObjNum 409 snap 0 seq_num 409) dirty exists 2024-09-13T10:12:13.543 INFO:tasks.rados.rados.0.smithi195.stdout:1852: left oid 35 (ObjNum 409 snap 0 seq_num 409) 2024-09-13T10:12:13.543 INFO:tasks.rados.rados.0.smithi195.stdout:1853: expect (ObjNum 369 snap 0 seq_num 369) 2024-09-13T10:12:13.544 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T10:12:13.548 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 674872 2024-09-13T10:12:13.575 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:13.569+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:13.583 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:13.666 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:13.666 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:13.667 INFO:tasks.rados.rados.0.smithi195.stdout:1854: finishing set_chunk to oid 43 2024-09-13T10:12:13.667 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 91 (ObjNum 344 snap 0 seq_num 344) dirty exists 2024-09-13T10:12:13.667 INFO:tasks.rados.rados.0.smithi195.stdout:1855: finishing write tid 1 to smithi19530714-48 2024-09-13T10:12:13.667 INFO:tasks.rados.rados.0.smithi195.stdout:1857: finishing set_chunk to oid 34 2024-09-13T10:12:13.667 INFO:tasks.rados.rados.0.smithi195.stdout:1857: got expected ENOENT (src dne) 2024-09-13T10:12:13.667 INFO:tasks.rados.rados.0.smithi195.stdout:1859: expect (ObjNum 368 snap 0 seq_num 368) 2024-09-13T10:12:13.676 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:12:13.675+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:12:13.680 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T10:12:13.689 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 18668 2024-09-13T10:12:13.714 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1327781 2024-09-13T10:12:13.728 INFO:tasks.rados.rados.0.smithi195.stdout:1855: finishing write tid 2 to smithi19530714-48 2024-09-13T10:12:13.728 INFO:tasks.rados.rados.0.smithi195.stdout:1855: finishing write tid 3 to smithi19530714-48 2024-09-13T10:12:13.728 INFO:tasks.rados.rados.0.smithi195.stdout:1855: finishing write tid 4 to smithi19530714-48 2024-09-13T10:12:13.728 INFO:tasks.rados.rados.0.smithi195.stdout:1855: finishing write tid 5 to smithi19530714-48 2024-09-13T10:12:13.728 INFO:tasks.rados.rados.0.smithi195.stdout:1855: oid 48 updating version 0 to 136 2024-09-13T10:12:13.728 INFO:tasks.rados.rados.0.smithi195.stdout:1855: oid 48 updating version 136 to 139 2024-09-13T10:12:13.728 INFO:tasks.rados.rados.0.smithi195.stdout:1855: oid 48 version 139 is already newer than 137 2024-09-13T10:12:13.728 INFO:tasks.rados.rados.0.smithi195.stdout:1855: oid 48 version 139 is already newer than 138 2024-09-13T10:12:13.729 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 48 v 139 (ObjNum 410 snap 0 seq_num 410) dirty exists 2024-09-13T10:12:13.729 INFO:tasks.rados.rados.0.smithi195.stdout:1855: left oid 48 (ObjNum 410 snap 0 seq_num 410) 2024-09-13T10:12:13.729 INFO:tasks.rados.rados.0.smithi195.stdout:1858: expect (ObjNum 351 snap 0 seq_num 351) 2024-09-13T10:12:13.729 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:13.777 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:12:13.775+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:12:13.847 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:13.847 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:13.854 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T10:12:13.862 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-13T10:12:13.874 INFO:tasks.rados.rados.0.smithi195.stdout:1856: expect (ObjNum 397 snap 0 seq_num 397) 2024-09-13T10:12:13.875 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:12:13.878 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:13.873+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:13.930 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 869235 2024-09-13T10:12:13.979 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:13.974+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:14.003 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2000246 2024-09-13T10:12:14.014 INFO:tasks.rados.rados.0.smithi195.stdout:1851: finishing write tid 1 to smithi19530714-33 2024-09-13T10:12:14.014 INFO:tasks.rados.rados.0.smithi195.stdout:1851: finishing write tid 2 to smithi19530714-33 2024-09-13T10:12:14.014 INFO:tasks.rados.rados.0.smithi195.stdout:1851: finishing write tid 3 to smithi19530714-33 2024-09-13T10:12:14.015 INFO:tasks.rados.rados.0.smithi195.stdout:1851: finishing write tid 4 to smithi19530714-33 2024-09-13T10:12:14.015 INFO:tasks.rados.rados.0.smithi195.stdout:1851: finishing write tid 5 to smithi19530714-33 2024-09-13T10:12:14.015 INFO:tasks.rados.rados.0.smithi195.stdout:1851: oid 33 updating version 0 to 67 2024-09-13T10:12:14.015 INFO:tasks.rados.rados.0.smithi195.stdout:1851: oid 33 updating version 67 to 68 2024-09-13T10:12:14.015 INFO:tasks.rados.rados.0.smithi195.stdout:1851: oid 33 updating version 68 to 70 2024-09-13T10:12:14.015 INFO:tasks.rados.rados.0.smithi195.stdout:1851: oid 33 version 70 is already newer than 69 2024-09-13T10:12:14.015 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 33 v 70 (ObjNum 408 snap 0 seq_num 408) dirty exists 2024-09-13T10:12:14.015 INFO:tasks.rados.rados.0.smithi195.stdout:1851: left oid 33 (ObjNum 408 snap 0 seq_num 408) 2024-09-13T10:12:14.015 INFO:tasks.rados.rados.0.smithi195.stdout:1843: done (15 left) 2024-09-13T10:12:14.015 INFO:tasks.rados.rados.0.smithi195.stdout:1845: done (14 left) 2024-09-13T10:12:14.015 INFO:tasks.rados.rados.0.smithi195.stdout:1846: done (13 left) 2024-09-13T10:12:14.015 INFO:tasks.rados.rados.0.smithi195.stdout:1847: done (12 left) 2024-09-13T10:12:14.015 INFO:tasks.rados.rados.0.smithi195.stdout:1848: done (11 left) 2024-09-13T10:12:14.016 INFO:tasks.rados.rados.0.smithi195.stdout:1849: done (10 left) 2024-09-13T10:12:14.016 INFO:tasks.rados.rados.0.smithi195.stdout:1850: done (9 left) 2024-09-13T10:12:14.016 INFO:tasks.rados.rados.0.smithi195.stdout:1851: done (8 left) 2024-09-13T10:12:14.016 INFO:tasks.rados.rados.0.smithi195.stdout:1852: done (7 left) 2024-09-13T10:12:14.016 INFO:tasks.rados.rados.0.smithi195.stdout:1853: done (6 left) 2024-09-13T10:12:14.016 INFO:tasks.rados.rados.0.smithi195.stdout:1854: done (5 left) 2024-09-13T10:12:14.016 INFO:tasks.rados.rados.0.smithi195.stdout:1855: done (4 left) 2024-09-13T10:12:14.016 INFO:tasks.rados.rados.0.smithi195.stdout:1856: done (3 left) 2024-09-13T10:12:14.016 INFO:tasks.rados.rados.0.smithi195.stdout:1857: done (2 left) 2024-09-13T10:12:14.016 INFO:tasks.rados.rados.0.smithi195.stdout:1858: done (1 left) 2024-09-13T10:12:14.016 INFO:tasks.rados.rados.0.smithi195.stdout:1859: done (0 left) 2024-09-13T10:12:14.016 INFO:tasks.rados.rados.0.smithi195.stdout:1860: read oid 47 snap -1 2024-09-13T10:12:14.016 INFO:tasks.rados.rados.0.smithi195.stdout:1860: expect (ObjNum 383 snap 0 seq_num 383) 2024-09-13T10:12:14.017 INFO:tasks.rados.rados.0.smithi195.stdout:1861: read oid 34 snap -1 2024-09-13T10:12:14.017 INFO:tasks.rados.rados.0.smithi195.stdout:1861: expect deleted 2024-09-13T10:12:14.017 INFO:tasks.rados.rados.0.smithi195.stdout:1862: read oid 11 snap -1 2024-09-13T10:12:14.017 INFO:tasks.rados.rados.0.smithi195.stdout:1862: expect (ObjNum 359 snap 0 seq_num 359) 2024-09-13T10:12:14.017 INFO:tasks.rados.rados.0.smithi195.stdout:1863: read oid 5 snap -1 2024-09-13T10:12:14.017 INFO:tasks.rados.rados.0.smithi195.stdout:1863: expect (ObjNum 360 snap 0 seq_num 360) 2024-09-13T10:12:14.017 INFO:tasks.rados.rados.0.smithi195.stdout:1864: write oid 24 current snap is 0 2024-09-13T10:12:14.017 INFO:tasks.rados.rados.0.smithi195.stdout:1864: seq_num 411 ranges {507677=410401,1367511=517802,2420497=487180} 2024-09-13T10:12:14.024 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:14.024 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:14.031 INFO:tasks.rados.rados.0.smithi195.stdout:1864: writing smithi19530714-24 from 507677 to 918078 tid 1 2024-09-13T10:12:14.038 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-13T10:12:14.044 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:12:14.045 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:12:14.045 INFO:tasks.rados.rados.0.smithi195.stdout:1864: writing smithi19530714-24 from 1367511 to 1885313 tid 2 2024-09-13T10:12:14.061 INFO:tasks.rados.rados.0.smithi195.stdout:1864: writing smithi19530714-24 from 2420497 to 2907677 tid 3 2024-09-13T10:12:14.062 INFO:tasks.rados.rados.0.smithi195.stdout:1865: set_chunk oid 31 target oid 2024-09-13T10:12:14.063 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:12:14.064 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:12:14.068 INFO:tasks.rados.rados.0.smithi195.stdout:1865: set_chunk oid 31 offset: 640000 length: 97280 target oid 431c93370df1d1ea01b1424dc7082f7d902f5097aa68390722333683478d9a0d offset: 0 snap -1 2024-09-13T10:12:14.068 INFO:tasks.rados.rados.0.smithi195.stdout:1866: write oid 2 current snap is 0 2024-09-13T10:12:14.068 INFO:tasks.rados.rados.0.smithi195.stdout:1860: expect (ObjNum 383 snap 0 seq_num 383) 2024-09-13T10:12:14.087 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:12:14.087 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:12:14.088 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2201055 2024-09-13T10:12:14.090 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:12:14.087+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:12:14.095 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:14.106 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:12:14.107 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:12:14.123 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:12:14.123 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:12:14.140 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:12:14.141 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:12:14.160 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:12:14.161 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:12:14.161 INFO:tasks.rados.rados.0.smithi195.stdout:1862: expect (ObjNum 359 snap 0 seq_num 359) 2024-09-13T10:12:14.161 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:14.163 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:14.163 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:14.177 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-13T10:12:14.179 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:12:14.179 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:12:14.182 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:14.178+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:14.183 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:14.196 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:12:14.196 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:12:14.209 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 741527 2024-09-13T10:12:14.216 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:12:14.217 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:12:14.222 INFO:tasks.rados.rados.0.smithi195.stdout:1864: finishing write tid 1 to smithi19530714-24 2024-09-13T10:12:14.222 INFO:tasks.rados.rados.0.smithi195.stdout:1864: finishing write tid 2 to smithi19530714-24 2024-09-13T10:12:14.222 INFO:tasks.rados.rados.0.smithi195.stdout:1864: finishing write tid 3 to smithi19530714-24 2024-09-13T10:12:14.222 INFO:tasks.rados.rados.0.smithi195.stdout:1864: finishing write tid 4 to smithi19530714-24 2024-09-13T10:12:14.222 INFO:tasks.rados.rados.0.smithi195.stdout:1864: finishing write tid 5 to smithi19530714-24 2024-09-13T10:12:14.222 INFO:tasks.rados.rados.0.smithi195.stdout:1864: oid 24 updating version 0 to 185 2024-09-13T10:12:14.222 INFO:tasks.rados.rados.0.smithi195.stdout:1864: oid 24 updating version 185 to 187 2024-09-13T10:12:14.222 INFO:tasks.rados.rados.0.smithi195.stdout:1864: oid 24 version 187 is already newer than 184 2024-09-13T10:12:14.222 INFO:tasks.rados.rados.0.smithi195.stdout:1864: oid 24 version 187 is already newer than 186 2024-09-13T10:12:14.222 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 24 v 187 (ObjNum 411 snap 0 seq_num 411) dirty exists 2024-09-13T10:12:14.223 INFO:tasks.rados.rados.0.smithi195.stdout:1864: left oid 24 (ObjNum 411 snap 0 seq_num 411) 2024-09-13T10:12:14.223 INFO:tasks.rados.rados.0.smithi195.stdout:1865: finishing set_chunk to oid 31 2024-09-13T10:12:14.223 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 110 (ObjNum 402 snap 0 seq_num 402) dirty exists 2024-09-13T10:12:14.223 INFO:tasks.rados.rados.0.smithi195.stdout:1866: seq_num 412 ranges {524484=485650,1425993=661460,2847920=76564} 2024-09-13T10:12:14.240 INFO:tasks.rados.rados.0.smithi195.stdout:1866: writing smithi19530714-2 from 524484 to 1010134 tid 1 2024-09-13T10:12:14.241 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:12:14.241 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:12:14.256 INFO:tasks.rados.rados.0.smithi195.stdout:1866: writing smithi19530714-2 from 1425993 to 2087453 tid 2 2024-09-13T10:12:14.258 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:12:14.258 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:12:14.269 INFO:tasks.rados.rados.0.smithi195.stdout:1866: writing smithi19530714-2 from 2847920 to 2924484 tid 3 2024-09-13T10:12:14.270 INFO:tasks.rados.rados.0.smithi195.stdout:1860: done (6 left) 2024-09-13T10:12:14.270 INFO:tasks.rados.rados.0.smithi195.stdout:1861: done (5 left) 2024-09-13T10:12:14.270 INFO:tasks.rados.rados.0.smithi195.stdout:1862: done (4 left) 2024-09-13T10:12:14.270 INFO:tasks.rados.rados.0.smithi195.stdout:1864: done (3 left) 2024-09-13T10:12:14.270 INFO:tasks.rados.rados.0.smithi195.stdout:1865: done (2 left) 2024-09-13T10:12:14.270 INFO:tasks.rados.rados.0.smithi195.stdout:1867: read oid 27 snap -1 2024-09-13T10:12:14.270 INFO:tasks.rados.rados.0.smithi195.stdout:1867: expect (ObjNum 221 snap 0 seq_num 221) 2024-09-13T10:12:14.270 INFO:tasks.rados.rados.0.smithi195.stdout:1868: write oid 49 current snap is 0 2024-09-13T10:12:14.270 INFO:tasks.rados.rados.0.smithi195.stdout:1866: finishing write tid 1 to smithi19530714-2 2024-09-13T10:12:14.270 INFO:tasks.rados.rados.0.smithi195.stdout:1866: finishing write tid 2 to smithi19530714-2 2024-09-13T10:12:14.270 INFO:tasks.rados.rados.0.smithi195.stdout:1868: seq_num 413 ranges {541291=560899,1884475=521471} 2024-09-13T10:12:14.276 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:12:14.276 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:12:14.283 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:14.277+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:14.284 INFO:tasks.rados.rados.0.smithi195.stdout:1868: writing smithi19530714-49 from 541291 to 1102190 tid 1 2024-09-13T10:12:14.294 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:12:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:12:14.303 INFO:tasks.rados.rados.0.smithi195.stdout:1868: writing smithi19530714-49 from 1884475 to 2405946 tid 2 2024-09-13T10:12:14.303 INFO:tasks.rados.rados.0.smithi195.stdout:1869: read oid 28 snap -1 2024-09-13T10:12:14.303 INFO:tasks.rados.rados.0.smithi195.stdout:1869: expect (ObjNum 369 snap 0 seq_num 369) 2024-09-13T10:12:14.303 INFO:tasks.rados.rados.0.smithi195.stdout:1870: tier_promote oid 9 2024-09-13T10:12:14.303 INFO:tasks.rados.rados.0.smithi195.stdout:1866: finishing write tid 3 to smithi19530714-2 2024-09-13T10:12:14.303 INFO:tasks.rados.rados.0.smithi195.stdout:1866: finishing write tid 4 to smithi19530714-2 2024-09-13T10:12:14.303 INFO:tasks.rados.rados.0.smithi195.stdout:1866: finishing write tid 5 to smithi19530714-2 2024-09-13T10:12:14.303 INFO:tasks.rados.rados.0.smithi195.stdout:1866: oid 2 updating version 0 to 156 2024-09-13T10:12:14.303 INFO:tasks.rados.rados.0.smithi195.stdout:1866: oid 2 updating version 156 to 159 2024-09-13T10:12:14.303 INFO:tasks.rados.rados.0.smithi195.stdout:1866: oid 2 version 159 is already newer than 157 2024-09-13T10:12:14.303 INFO:tasks.rados.rados.0.smithi195.stdout:1866: oid 2 version 159 is already newer than 158 2024-09-13T10:12:14.304 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 159 (ObjNum 412 snap 0 seq_num 412) dirty exists 2024-09-13T10:12:14.304 INFO:tasks.rados.rados.0.smithi195.stdout:1866: left oid 2 (ObjNum 412 snap 0 seq_num 412) 2024-09-13T10:12:14.304 INFO:tasks.rados.rados.0.smithi195.stdout:1867: expect (ObjNum 221 snap 0 seq_num 221) 2024-09-13T10:12:14.317 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:12:14.318 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:12:14.339 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:12:14.340 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:12:14.346 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:12:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:12:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:12:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1600 4.10 4.31420bd0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:12:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:05.990716+0000", 2024-09-13T10:12:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.353767356999995, 2024-09-13T10:12:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.412517299, 2024-09-13T10:12:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:12:14.347 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1600 2024-09-13T10:12:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.990716+0000", 2024-09-13T10:12:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.990716+0000", 2024-09-13T10:12:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.990720+0000", 2024-09-13T10:12:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9689999999999996e-06 2024-09-13T10:12:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991825+0000", 2024-09-13T10:12:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001104764 2024-09-13T10:12:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991856+0000", 2024-09-13T10:12:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1473000000000002e-05 2024-09-13T10:12:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991872+0000", 2024-09-13T10:12:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6212999999999999e-05 2024-09-13T10:12:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991974+0000", 2024-09-13T10:12:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010160799999999999 2024-09-13T10:12:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991988+0000", 2024-09-13T10:12:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4106e-05 2024-09-13T10:12:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403179+0000", 2024-09-13T10:12:14.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41119127100000002 2024-09-13T10:12:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403233+0000", 2024-09-13T10:12:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3894999999999998e-05 2024-09-13T10:12:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1601 4.10 4.31420bd0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:12:14.352 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.008020+0000", 2024-09-13T10:12:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.336463180999999, 2024-09-13T10:12:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39533983, 2024-09-13T10:12:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:12:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.353 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1601 2024-09-13T10:12:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.008020+0000", 2024-09-13T10:12:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.008020+0000", 2024-09-13T10:12:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.008023+0000", 2024-09-13T10:12:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.807e-06 2024-09-13T10:12:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009268+0000", 2024-09-13T10:12:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001245257 2024-09-13T10:12:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009298+0000", 2024-09-13T10:12:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0171999999999998e-05 2024-09-13T10:12:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009313+0000", 2024-09-13T10:12:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.471e-05 2024-09-13T10:12:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009469+0000", 2024-09-13T10:12:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015595600000000001 2024-09-13T10:12:14.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009481+0000", 2024-09-13T10:12:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2454e-05 2024-09-13T10:12:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403334+0000", 2024-09-13T10:12:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39385227499999997 2024-09-13T10:12:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403360+0000", 2024-09-13T10:12:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6199000000000001e-05 2024-09-13T10:12:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1602 4.10 4.31420bd0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:12:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.009344+0000", 2024-09-13T10:12:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.335139349000002, 2024-09-13T10:12:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39420750999999998, 2024-09-13T10:12:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.358 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:12:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1602 2024-09-13T10:12:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009344+0000", 2024-09-13T10:12:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009344+0000", 2024-09-13T10:12:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009345+0000", 2024-09-13T10:12:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1930000000000001e-06 2024-09-13T10:12:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009351+0000", 2024-09-13T10:12:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1399999999999997e-06 2024-09-13T10:12:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009361+0000", 2024-09-13T10:12:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9569999999999997e-06 2024-09-13T10:12:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009370+0000", 2024-09-13T10:12:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.619e-06 2024-09-13T10:12:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009570+0000", 2024-09-13T10:12:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019990599999999999 2024-09-13T10:12:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009583+0000", 2024-09-13T10:12:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2886e-05 2024-09-13T10:12:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403522+0000", 2024-09-13T10:12:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39393976000000003 2024-09-13T10:12:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403551+0000", 2024-09-13T10:12:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9048999999999999e-05 2024-09-13T10:12:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1603 4.10 4.31420bd0 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:12:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.009383+0000", 2024-09-13T10:12:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.335100456999996, 2024-09-13T10:12:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39425412900000001, 2024-09-13T10:12:14.363 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:12:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1603 2024-09-13T10:12:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009383+0000", 2024-09-13T10:12:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009383+0000", 2024-09-13T10:12:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009384+0000", 2024-09-13T10:12:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8700000000000005e-07 2024-09-13T10:12:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009389+0000", 2024-09-13T10:12:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1669999999999998e-06 2024-09-13T10:12:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009398+0000", 2024-09-13T10:12:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6260000000000007e-06 2024-09-13T10:12:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009407+0000", 2024-09-13T10:12:14.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1960000000000008e-06 2024-09-13T10:12:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009623+0000", 2024-09-13T10:12:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000216842 2024-09-13T10:12:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009632+0000", 2024-09-13T10:12:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3529999999999995e-06 2024-09-13T10:12:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403609+0000", 2024-09-13T10:12:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39397721600000002 2024-09-13T10:12:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403637+0000", 2024-09-13T10:12:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7942000000000001e-05 2024-09-13T10:12:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3264 4.10 4:0b4c105e:::smithi19530714-16:head [read 264340~2357585,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.083141+0000", 2024-09-13T10:12:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.261342690999999, 2024-09-13T10:12:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093235160000000004, 2024-09-13T10:12:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3264 2024-09-13T10:12:14.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083141+0000", 2024-09-13T10:12:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083141+0000", 2024-09-13T10:12:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083144+0000", 2024-09-13T10:12:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1269999999999999e-06 2024-09-13T10:12:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083156+0000", 2024-09-13T10:12:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1956e-05 2024-09-13T10:12:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083181+0000", 2024-09-13T10:12:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4856000000000001e-05 2024-09-13T10:12:14.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083196+0000", 2024-09-13T10:12:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5593000000000001e-05 2024-09-13T10:12:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083340+0000", 2024-09-13T10:12:14.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014363200000000001 2024-09-13T10:12:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083595+0000", 2024-09-13T10:12:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025489700000000002 2024-09-13T10:12:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.092464+0000", 2024-09-13T10:12:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008869455 2024-09-13T10:12:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.373 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3265 4.10 4:0b4c105e:::smithi19530714-16:head [read 0~2621925,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.083253+0000", 2024-09-13T10:12:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.261230221000002, 2024-09-13T10:12:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01421477, 2024-09-13T10:12:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.374 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3265 2024-09-13T10:12:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083253+0000", 2024-09-13T10:12:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083253+0000", 2024-09-13T10:12:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083255+0000", 2024-09-13T10:12:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.717e-06 2024-09-13T10:12:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083264+0000", 2024-09-13T10:12:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3989999999999993e-06 2024-09-13T10:12:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083281+0000", 2024-09-13T10:12:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6833e-05 2024-09-13T10:12:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083300+0000", 2024-09-13T10:12:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8972000000000001e-05 2024-09-13T10:12:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.091405+0000", 2024-09-13T10:12:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0081045509999999998 2024-09-13T10:12:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.091609+0000", 2024-09-13T10:12:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020448299999999999 2024-09-13T10:12:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.097468+0000", 2024-09-13T10:12:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058588149999999999 2024-09-13T10:12:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.378 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3266 4.10 4:0b4c105e:::smithi19530714-16:head [sparse-read 0~1476046] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.083328+0000", 2024-09-13T10:12:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.261155547000001, 2024-09-13T10:12:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015205013, 2024-09-13T10:12:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:14.379 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3266 2024-09-13T10:12:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083328+0000", 2024-09-13T10:12:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083328+0000", 2024-09-13T10:12:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083329+0000", 2024-09-13T10:12:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.313e-06 2024-09-13T10:12:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083336+0000", 2024-09-13T10:12:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2024-09-13T10:12:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083390+0000", 2024-09-13T10:12:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3662000000000002e-05 2024-09-13T10:12:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083405+0000", 2024-09-13T10:12:14.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5163e-05 2024-09-13T10:12:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.097580+0000", 2024-09-13T10:12:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014174755000000001 2024-09-13T10:12:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.097822+0000", 2024-09-13T10:12:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024186499999999999 2024-09-13T10:12:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.098533+0000", 2024-09-13T10:12:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00071110200000000002 2024-09-13T10:12:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:3234 4.12 e81/80 4:4ac406d9:::smithi19530714-26:head v 81'40, mlcod=0'0)", 2024-09-13T10:12:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.085189+0000", 2024-09-13T10:12:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.259294186999998, 2024-09-13T10:12:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017250062, 2024-09-13T10:12:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.384 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.085189+0000", 2024-09-13T10:12:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.085189+0000", 2024-09-13T10:12:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.085191+0000", 2024-09-13T10:12:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0829999999999998e-06 2024-09-13T10:12:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.087261+0000", 2024-09-13T10:12:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002069417 2024-09-13T10:12:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.087291+0000", 2024-09-13T10:12:14.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0222e-05 2024-09-13T10:12:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.087317+0000", 2024-09-13T10:12:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6003000000000002e-05 2024-09-13T10:12:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.087564+0000", 2024-09-13T10:12:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024710800000000001 2024-09-13T10:12:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.087662+0000", 2024-09-13T10:12:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8153999999999996e-05 2024-09-13T10:12:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101938+0000", 2024-09-13T10:12:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014275738 2024-09-13T10:12:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.102439+0000", 2024-09-13T10:12:14.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050133700000000003 2024-09-13T10:12:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:3235 4.12 e81/80 4:4ac406d9:::smithi19530714-26:head v 81'41, mlcod=0'0)", 2024-09-13T10:12:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.091873+0000", 2024-09-13T10:12:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.252610389000001, 2024-09-13T10:12:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011472080000000001, 2024-09-13T10:12:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.389 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.091873+0000", 2024-09-13T10:12:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.091873+0000", 2024-09-13T10:12:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.091875+0000", 2024-09-13T10:12:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2199999999999999e-06 2024-09-13T10:12:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.092906+0000", 2024-09-13T10:12:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031105 2024-09-13T10:12:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.092937+0000", 2024-09-13T10:12:14.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0326000000000001e-05 2024-09-13T10:12:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.092957+0000", 2024-09-13T10:12:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0225999999999999e-05 2024-09-13T10:12:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.099823+0000", 2024-09-13T10:12:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0068664319999999996 2024-09-13T10:12:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.099873+0000", 2024-09-13T10:12:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9838e-05 2024-09-13T10:12:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.103240+0000", 2024-09-13T10:12:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003367395 2024-09-13T10:12:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.103345+0000", 2024-09-13T10:12:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010453799999999999 2024-09-13T10:12:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:3236 4.12 e81/80 4:4ac406d9:::smithi19530714-26:head v 81'42, mlcod=0'0)", 2024-09-13T10:12:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.096873+0000", 2024-09-13T10:12:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.247610637000001, 2024-09-13T10:12:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072886340000000004, 2024-09-13T10:12:14.394 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096873+0000", 2024-09-13T10:12:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096873+0000", 2024-09-13T10:12:14.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096875+0000", 2024-09-13T10:12:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.193e-06 2024-09-13T10:12:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096890+0000", 2024-09-13T10:12:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5243e-05 2024-09-13T10:12:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096918+0000", 2024-09-13T10:12:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7985999999999999e-05 2024-09-13T10:12:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096943+0000", 2024-09-13T10:12:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4549000000000002e-05 2024-09-13T10:12:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.102522+0000", 2024-09-13T10:12:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055797579999999998 2024-09-13T10:12:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.102586+0000", 2024-09-13T10:12:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3748e-05 2024-09-13T10:12:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.104035+0000", 2024-09-13T10:12:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001448438 2024-09-13T10:12:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.104161+0000", 2024-09-13T10:12:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012671899999999999 2024-09-13T10:12:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3298 4.10 4:0bd0428c:::smithi19530714-50:head [read 1545508~1059610,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e82)", 2024-09-13T10:12:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:44.587079+0000", 2024-09-13T10:12:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.757404426000001, 2024-09-13T10:12:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0069565820000000002, 2024-09-13T10:12:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:14.399 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3298 2024-09-13T10:12:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:44.587079+0000", 2024-09-13T10:12:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:44.587079+0000", 2024-09-13T10:12:14.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:44.587080+0000", 2024-09-13T10:12:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2699999999999999e-06 2024-09-13T10:12:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:44.587092+0000", 2024-09-13T10:12:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1623e-05 2024-09-13T10:12:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:44.587109+0000", 2024-09-13T10:12:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7016000000000001e-05 2024-09-13T10:12:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:44.587124+0000", 2024-09-13T10:12:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4735999999999999e-05 2024-09-13T10:12:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:44.592406+0000", 2024-09-13T10:12:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052820619999999997 2024-09-13T10:12:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:44.592550+0000", 2024-09-13T10:12:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000144435 2024-09-13T10:12:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:44.594035+0000", 2024-09-13T10:12:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00148544 2024-09-13T10:12:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.403 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3732 4.10 4:0b4c105e:::smithi19530714-16:head [assert-version v55,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e90)", 2024-09-13T10:12:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:53.879799+0000", 2024-09-13T10:12:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.464684258999998, 2024-09-13T10:12:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078265209999999995, 2024-09-13T10:12:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3732 2024-09-13T10:12:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.879799+0000", 2024-09-13T10:12:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.879799+0000", 2024-09-13T10:12:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.879803+0000", 2024-09-13T10:12:14.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6799999999999999e-06 2024-09-13T10:12:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.879821+0000", 2024-09-13T10:12:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8559e-05 2024-09-13T10:12:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.879858+0000", 2024-09-13T10:12:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6294999999999997e-05 2024-09-13T10:12:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.406 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.879875+0000", 2024-09-13T10:12:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7399999999999999e-05 2024-09-13T10:12:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.880046+0000", 2024-09-13T10:12:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017084899999999999 2024-09-13T10:12:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.880308+0000", 2024-09-13T10:12:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026218299999999998 2024-09-13T10:12:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.407 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3,14", 2024-09-13T10:12:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.885684+0000", 2024-09-13T10:12:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053760800000000001 2024-09-13T10:12:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.886927+0000", 2024-09-13T10:12:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00124244 2024-09-13T10:12:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.408 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.887269+0000", 2024-09-13T10:12:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034287800000000002 2024-09-13T10:12:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.887471+0000", 2024-09-13T10:12:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000201864 2024-09-13T10:12:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.409 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.887556+0000", 2024-09-13T10:12:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4777999999999994e-05 2024-09-13T10:12:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.887626+0000", 2024-09-13T10:12:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9515000000000001e-05 2024-09-13T10:12:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.410 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4114 4.10 4:0b4c105e:::smithi19530714-16:head [write 652204~777572] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T10:12:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:03.951781+0000", 2024-09-13T10:12:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.392701895, 2024-09-13T10:12:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01227927, 2024-09-13T10:12:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4114 2024-09-13T10:12:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.411 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.951781+0000", 2024-09-13T10:12:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.951781+0000", 2024-09-13T10:12:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.412 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.951785+0000", 2024-09-13T10:12:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2024-09-13T10:12:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.953481+0000", 2024-09-13T10:12:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001696173 2024-09-13T10:12:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.953515+0000", 2024-09-13T10:12:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3929e-05 2024-09-13T10:12:14.413 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.953532+0000", 2024-09-13T10:12:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7224e-05 2024-09-13T10:12:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.953672+0000", 2024-09-13T10:12:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014041300000000001 2024-09-13T10:12:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-13T10:12:14.414 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.953684+0000", 2024-09-13T10:12:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1337e-05 2024-09-13T10:12:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.956654+0000", 2024-09-13T10:12:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029703249999999998 2024-09-13T10:12:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.956845+0000", 2024-09-13T10:12:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000190776 2024-09-13T10:12:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.415 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3,14", 2024-09-13T10:12:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.957082+0000", 2024-09-13T10:12:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023739799999999999 2024-09-13T10:12:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.963207+0000", 2024-09-13T10:12:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061252269999999996 2024-09-13T10:12:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.963683+0000", 2024-09-13T10:12:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047594599999999999 2024-09-13T10:12:14.416 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.963783+0000", 2024-09-13T10:12:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9281e-05 2024-09-13T10:12:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.963989+0000", 2024-09-13T10:12:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020614200000000001 2024-09-13T10:12:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.417 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964061+0000", 2024-09-13T10:12:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1896000000000004e-05 2024-09-13T10:12:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4115 4.10 4:0b4c105e:::smithi19530714-16:head [write 1952125~300079] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T10:12:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:03.965581+0000", 2024-09-13T10:12:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.378902614999999, 2024-09-13T10:12:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077346350000000001, 2024-09-13T10:12:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.418 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4115 2024-09-13T10:12:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.965581+0000", 2024-09-13T10:12:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.419 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.965581+0000", 2024-09-13T10:12:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.965584+0000", 2024-09-13T10:12:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0369999999999999e-06 2024-09-13T10:12:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966073+0000", 2024-09-13T10:12:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00048940500000000005 2024-09-13T10:12:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.420 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966108+0000", 2024-09-13T10:12:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4396000000000001e-05 2024-09-13T10:12:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966129+0000", 2024-09-13T10:12:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1263e-05 2024-09-13T10:12:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966282+0000", 2024-09-13T10:12:14.421 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015279000000000001 2024-09-13T10:12:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966498+0000", 2024-09-13T10:12:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021653199999999999 2024-09-13T10:12:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3,14", 2024-09-13T10:12:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966820+0000", 2024-09-13T10:12:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032185299999999998 2024-09-13T10:12:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:14.422 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.970736+0000", 2024-09-13T10:12:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039162950000000002 2024-09-13T10:12:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.970925+0000", 2024-09-13T10:12:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018865200000000001 2024-09-13T10:12:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.973125+0000", 2024-09-13T10:12:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021998859999999999 2024-09-13T10:12:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.423 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.973227+0000", 2024-09-13T10:12:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101912 2024-09-13T10:12:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.973315+0000", 2024-09-13T10:12:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8614000000000002e-05 2024-09-13T10:12:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.424 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4116 4.10 4:0b4c105e:::smithi19530714-16:head [setxattr _header (56) in=7b,truncate 2252204] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T10:12:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:03.966168+0000", 2024-09-13T10:12:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.378314825, 2024-09-13T10:12:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083925149999999997, 2024-09-13T10:12:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4116 2024-09-13T10:12:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.425 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966168+0000", 2024-09-13T10:12:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966168+0000", 2024-09-13T10:12:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966170+0000", 2024-09-13T10:12:14.426 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.544e-06 2024-09-13T10:12:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966177+0000", 2024-09-13T10:12:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6699999999999997e-06 2024-09-13T10:12:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966195+0000", 2024-09-13T10:12:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8529000000000002e-05 2024-09-13T10:12:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.427 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966205+0000", 2024-09-13T10:12:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9499999999999996e-06 2024-09-13T10:12:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.968990+0000", 2024-09-13T10:12:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027848059999999999 2024-09-13T10:12:14.428 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.969173+0000", 2024-09-13T10:12:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018300599999999999 2024-09-13T10:12:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3,14", 2024-09-13T10:12:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.969399+0000", 2024-09-13T10:12:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022624900000000001 2024-09-13T10:12:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.429 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.972501+0000", 2024-09-13T10:12:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031013619999999999 2024-09-13T10:12:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.973436+0000", 2024-09-13T10:12:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093499299999999996 2024-09-13T10:12:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.974390+0000", 2024-09-13T10:12:14.430 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095395699999999996 2024-09-13T10:12:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.974467+0000", 2024-09-13T10:12:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7594000000000003e-05 2024-09-13T10:12:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.974561+0000", 2024-09-13T10:12:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3855000000000002e-05 2024-09-13T10:12:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.431 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4117 4.10 4:0b4c105e:::smithi19530714-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e97)", 2024-09-13T10:12:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:03.966260+0000", 2024-09-13T10:12:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.378223753, 2024-09-13T10:12:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008682492, 2024-09-13T10:12:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.432 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4117 2024-09-13T10:12:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966260+0000", 2024-09-13T10:12:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966260+0000", 2024-09-13T10:12:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.433 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966260+0000", 2024-09-13T10:12:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9599999999999998e-07 2024-09-13T10:12:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966265+0000", 2024-09-13T10:12:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8169999999999998e-06 2024-09-13T10:12:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966277+0000", 2024-09-13T10:12:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1523e-05 2024-09-13T10:12:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.434 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966286+0000", 2024-09-13T10:12:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9700000000000005e-06 2024-09-13T10:12:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.970534+0000", 2024-09-13T10:12:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042480770000000003 2024-09-13T10:12:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.970706+0000", 2024-09-13T10:12:14.435 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000172456 2024-09-13T10:12:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.974672+0000", 2024-09-13T10:12:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039656650000000002 2024-09-13T10:12:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.974791+0000", 2024-09-13T10:12:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118714 2024-09-13T10:12:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.436 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.974942+0000", 2024-09-13T10:12:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015137399999999999 2024-09-13T10:12:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4156 4.10 4:0b4c105e:::smithi19530714-16:head [sparse-read 0~1588652,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.151897+0000", 2024-09-13T10:12:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1925862249999994, 2024-09-13T10:12:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082248189999999995, 2024-09-13T10:12:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.437 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4156 2024-09-13T10:12:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.151897+0000", 2024-09-13T10:12:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.438 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.151897+0000", 2024-09-13T10:12:14.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.151901+0000", 2024-09-13T10:12:14.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3699999999999997e-06 2024-09-13T10:12:14.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.439 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.439 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.151923+0000", 2024-09-13T10:12:14.439 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1691999999999998e-05 2024-09-13T10:12:14.439 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.439 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.151967+0000", 2024-09-13T10:12:14.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4279000000000001e-05 2024-09-13T10:12:14.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.151992+0000", 2024-09-13T10:12:14.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4267e-05 2024-09-13T10:12:14.440 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.440 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.440 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.440 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152163+0000", 2024-09-13T10:12:14.440 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017141300000000001 2024-09-13T10:12:14.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-13T10:12:14.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152180+0000", 2024-09-13T10:12:14.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7376000000000001e-05 2024-09-13T10:12:14.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.441 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.156082+0000", 2024-09-13T10:12:14.441 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039017190000000001 2024-09-13T10:12:14.441 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.441 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.441 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.156298+0000", 2024-09-13T10:12:14.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021607199999999999 2024-09-13T10:12:14.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.442 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.442 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.160122+0000", 2024-09-13T10:12:14.442 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038236310000000001 2024-09-13T10:12:14.442 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.442 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.442 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.442 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.442 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.442 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4157 4.10 4:0b4c105e:::smithi19530714-16:head [read 0~2252204,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:14.443 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.152052+0000", 2024-09-13T10:12:14.443 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1924309750000006, 2024-09-13T10:12:14.443 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010542621, 2024-09-13T10:12:14.443 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.443 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.443 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:14.443 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.443 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.443 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.443 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4157 2024-09-13T10:12:14.443 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.443 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.443 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152052+0000", 2024-09-13T10:12:14.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152052+0000", 2024-09-13T10:12:14.444 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.444 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.444 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.444 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.444 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152054+0000", 2024-09-13T10:12:14.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7230000000000001e-06 2024-09-13T10:12:14.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152061+0000", 2024-09-13T10:12:14.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1359999999999999e-06 2024-09-13T10:12:14.445 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.445 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.445 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.445 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152082+0000", 2024-09-13T10:12:14.445 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0339999999999998e-05 2024-09-13T10:12:14.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152098+0000", 2024-09-13T10:12:14.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6952999999999998e-05 2024-09-13T10:12:14.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.446 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152367+0000", 2024-09-13T10:12:14.446 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026855800000000001 2024-09-13T10:12:14.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.446 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.446 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-13T10:12:14.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152382+0000", 2024-09-13T10:12:14.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5393e-05 2024-09-13T10:12:14.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.160202+0000", 2024-09-13T10:12:14.447 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078198550000000006 2024-09-13T10:12:14.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.447 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.447 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.447 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.160407+0000", 2024-09-13T10:12:14.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000204248 2024-09-13T10:12:14.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.448 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.448 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.162595+0000", 2024-09-13T10:12:14.448 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002188415 2024-09-13T10:12:14.448 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.448 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.448 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.448 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.448 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.448 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4158 4.10 4:0b4c105e:::smithi19530714-16:head [sparse-read 965971~1163353] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:14.449 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.152124+0000", 2024-09-13T10:12:14.449 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1923593369999992, 2024-09-13T10:12:14.449 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011258710999999999, 2024-09-13T10:12:14.449 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.449 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.449 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:14.449 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.449 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.449 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.449 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4158 2024-09-13T10:12:14.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.449 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152124+0000", 2024-09-13T10:12:14.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152124+0000", 2024-09-13T10:12:14.450 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.450 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.450 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.450 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152125+0000", 2024-09-13T10:12:14.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2470000000000001e-06 2024-09-13T10:12:14.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152133+0000", 2024-09-13T10:12:14.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3170000000000004e-06 2024-09-13T10:12:14.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.451 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.451 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152149+0000", 2024-09-13T10:12:14.451 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6526000000000001e-05 2024-09-13T10:12:14.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.451 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152165+0000", 2024-09-13T10:12:14.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5569e-05 2024-09-13T10:12:14.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152495+0000", 2024-09-13T10:12:14.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032994300000000002 2024-09-13T10:12:14.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.452 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.452 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-13T10:12:14.452 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152511+0000", 2024-09-13T10:12:14.452 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6160000000000001e-05 2024-09-13T10:12:14.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.162726+0000", 2024-09-13T10:12:14.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010215307999999999 2024-09-13T10:12:14.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.453 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.453 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.162896+0000", 2024-09-13T10:12:14.453 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016949099999999999 2024-09-13T10:12:14.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.453 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.454 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.454 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.163383+0000", 2024-09-13T10:12:14.454 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00048715 2024-09-13T10:12:14.454 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.454 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.454 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.454 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.454 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.454 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4199 4.10 4:0b4c105e:::smithi19530714-16:head [assert-version v59,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:14.454 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.711426+0000", 2024-09-13T10:12:14.454 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 8.6330577210000001, 2024-09-13T10:12:14.455 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086774999999999994, 2024-09-13T10:12:14.455 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.455 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.455 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.455 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.455 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.455 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.455 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4199 2024-09-13T10:12:14.455 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.455 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.455 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.455 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.455 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.711426+0000", 2024-09-13T10:12:14.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.711426+0000", 2024-09-13T10:12:14.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.456 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.456 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.711428+0000", 2024-09-13T10:12:14.456 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6000000000000001e-06 2024-09-13T10:12:14.456 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.456 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.711440+0000", 2024-09-13T10:12:14.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2036e-05 2024-09-13T10:12:14.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.711470+0000", 2024-09-13T10:12:14.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9298999999999998e-05 2024-09-13T10:12:14.457 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.457 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.457 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.457 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.711486+0000", 2024-09-13T10:12:14.457 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6336000000000001e-05 2024-09-13T10:12:14.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.711629+0000", 2024-09-13T10:12:14.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000143151 2024-09-13T10:12:14.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.458 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.458 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.711872+0000", 2024-09-13T10:12:14.458 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024269499999999999 2024-09-13T10:12:14.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.458 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3,14", 2024-09-13T10:12:14.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.717340+0000", 2024-09-13T10:12:14.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054679359999999996 2024-09-13T10:12:14.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:14.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.719241+0000", 2024-09-13T10:12:14.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001901678 2024-09-13T10:12:14.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.459 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.459 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.459 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.719874+0000", 2024-09-13T10:12:14.459 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063235699999999995 2024-09-13T10:12:14.459 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.719952+0000", 2024-09-13T10:12:14.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8634999999999995e-05 2024-09-13T10:12:14.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.460 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.720035+0000", 2024-09-13T10:12:14.460 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2768000000000004e-05 2024-09-13T10:12:14.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.460 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.460 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.461 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.720103+0000", 2024-09-13T10:12:14.461 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8009000000000006e-05 2024-09-13T10:12:14.461 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.461 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.461 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.461 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.461 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.461 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:14.463 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-13T10:12:14.464 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:14.457+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:14.526 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3187635 2024-09-13T10:12:14.543 INFO:tasks.rados.rados.0.smithi195.stdout:1868: finishing write tid 1 to smithi19530714-49 2024-09-13T10:12:14.543 INFO:tasks.rados.rados.0.smithi195.stdout:1870: got (0) Success 2024-09-13T10:12:14.543 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 9 v 123 (ObjNum 259 snap 0 seq_num 259) dirty exists 2024-09-13T10:12:14.543 INFO:tasks.rados.rados.0.smithi195.stdout:1868: finishing write tid 2 to smithi19530714-49 2024-09-13T10:12:14.543 INFO:tasks.rados.rados.0.smithi195.stdout:1868: finishing write tid 3 to smithi19530714-49 2024-09-13T10:12:14.543 INFO:tasks.rados.rados.0.smithi195.stdout:1868: finishing write tid 4 to smithi19530714-49 2024-09-13T10:12:14.543 INFO:tasks.rados.rados.0.smithi195.stdout:1868: oid 49 updating version 0 to 190 2024-09-13T10:12:14.543 INFO:tasks.rados.rados.0.smithi195.stdout:1868: oid 49 version 190 is already newer than 188 2024-09-13T10:12:14.543 INFO:tasks.rados.rados.0.smithi195.stdout:1868: oid 49 version 190 is already newer than 189 2024-09-13T10:12:14.544 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 190 (ObjNum 413 snap 0 seq_num 413) dirty exists 2024-09-13T10:12:14.544 INFO:tasks.rados.rados.0.smithi195.stdout:1868: left oid 49 (ObjNum 413 snap 0 seq_num 413) 2024-09-13T10:12:14.544 INFO:tasks.rados.rados.0.smithi195.stdout:1869: expect (ObjNum 369 snap 0 seq_num 369) 2024-09-13T10:12:14.563 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:14.558+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:14.593 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:14.593 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:12:14.593 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:12:14.593 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:14.607 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-13T10:12:14.643 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 739851 2024-09-13T10:12:14.664 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:12:14.663+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:12:14.682 INFO:tasks.rados.rados.0.smithi195.stdout:1866: done (5 left) 2024-09-13T10:12:14.682 INFO:tasks.rados.rados.0.smithi195.stdout:1867: done (4 left) 2024-09-13T10:12:14.682 INFO:tasks.rados.rados.0.smithi195.stdout:1868: done (3 left) 2024-09-13T10:12:14.682 INFO:tasks.rados.rados.0.smithi195.stdout:1869: done (2 left) 2024-09-13T10:12:14.682 INFO:tasks.rados.rados.0.smithi195.stdout:1870: done (1 left) 2024-09-13T10:12:14.682 INFO:tasks.rados.rados.0.smithi195.stdout:1871: read oid 11 snap -1 2024-09-13T10:12:14.682 INFO:tasks.rados.rados.0.smithi195.stdout:1871: expect (ObjNum 359 snap 0 seq_num 359) 2024-09-13T10:12:14.682 INFO:tasks.rados.rados.0.smithi195.stdout:1872: set_chunk oid 34 target oid 2024-09-13T10:12:14.682 INFO:tasks.rados.rados.0.smithi195.stdout:1872: set_chunk oid 34 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:12:14.682 INFO:tasks.rados.rados.0.smithi195.stdout:1873: tier_flush oid 1 2024-09-13T10:12:14.682 INFO:tasks.rados.rados.0.smithi195.stdout:1873: tier_flush oid 1 snap -1 2024-09-13T10:12:14.682 INFO:tasks.rados.rados.0.smithi195.stdout:1874: read oid 19 snap -1 2024-09-13T10:12:14.683 INFO:tasks.rados.rados.0.smithi195.stdout:1874: expect (ObjNum 346 snap 0 seq_num 346) 2024-09-13T10:12:14.683 INFO:tasks.rados.rados.0.smithi195.stdout:1875: set_chunk oid 36 target oid 2024-09-13T10:12:14.683 INFO:tasks.rados.rados.0.smithi195.stdout:1875: set_chunk oid 36 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:12:14.683 INFO:tasks.rados.rados.0.smithi195.stdout:1876: read oid 33 snap -1 2024-09-13T10:12:14.683 INFO:tasks.rados.rados.0.smithi195.stdout:1876: expect (ObjNum 408 snap 0 seq_num 408) 2024-09-13T10:12:14.683 INFO:tasks.rados.rados.0.smithi195.stdout:1877: write (excl) oid 40 current snap is 0 2024-09-13T10:12:14.683 INFO:tasks.rados.rados.0.smithi195.stdout:1877: seq_num 414 ranges {558098=636148,1826604=665129} 2024-09-13T10:12:14.701 INFO:tasks.rados.rados.0.smithi195.stdout:1877: writing smithi19530714-40 from 558098 to 1194246 tid 1 2024-09-13T10:12:14.716 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-13T10:12:14.723 INFO:tasks.rados.rados.0.smithi195.stdout:1877: writing smithi19530714-40 from 1826604 to 2491733 tid 2 2024-09-13T10:12:14.723 INFO:tasks.rados.rados.0.smithi195.stdout:1878: read oid 44 snap -1 2024-09-13T10:12:14.723 INFO:tasks.rados.rados.0.smithi195.stdout:1878: expect deleted 2024-09-13T10:12:14.723 INFO:tasks.rados.rados.0.smithi195.stdout:1879: read oid 14 snap -1 2024-09-13T10:12:14.723 INFO:tasks.rados.rados.0.smithi195.stdout:1879: expect (ObjNum 398 snap 0 seq_num 398) 2024-09-13T10:12:14.724 INFO:tasks.rados.rados.0.smithi195.stdout:1880: read oid 20 snap -1 2024-09-13T10:12:14.724 INFO:tasks.rados.rados.0.smithi195.stdout:1880: expect (ObjNum 389 snap 0 seq_num 389) 2024-09-13T10:12:14.724 INFO:tasks.rados.rados.0.smithi195.stdout:1881: write oid 32 current snap is 0 2024-09-13T10:12:14.724 INFO:tasks.rados.rados.0.smithi195.stdout:1875: finishing set_chunk to oid 36 2024-09-13T10:12:14.724 INFO:tasks.rados.rados.0.smithi195.stdout:1875: got expected ENOENT (src dne) 2024-09-13T10:12:14.724 INFO:tasks.rados.rados.0.smithi195.stdout:1872: finishing set_chunk to oid 34 2024-09-13T10:12:14.724 INFO:tasks.rados.rados.0.smithi195.stdout:1872: got expected ENOENT (src dne) 2024-09-13T10:12:14.724 INFO:tasks.rados.rados.0.smithi195.stdout:1871: expect (ObjNum 359 snap 0 seq_num 359) 2024-09-13T10:12:14.739 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:14.739 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:12:14.739 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:12:14.739 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:12:14.739 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:14.742 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1495468 2024-09-13T10:12:14.754 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-13T10:12:14.765 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:14.761+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:14.791 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:14.817 INFO:tasks.rados.rados.0.smithi195.stdout:1874: expect (ObjNum 346 snap 0 seq_num 346) 2024-09-13T10:12:14.850 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:12:14.850 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:12:14.850 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:12:14.850 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T10:12:14.850 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T10:12:14.894 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:14.890+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:14.915 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:12:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:12:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:12:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:203 3.1 3:b033e801:::smithi19530714-4 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:12:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 83.919636311999994, 2024-09-13T10:12:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035547764000000003, 2024-09-13T10:12:14.915 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 203 2024-09-13T10:12:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:12:14.916 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:12:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991467+0000", 2024-09-13T10:12:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5030000000000002e-06 2024-09-13T10:12:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.917 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991483+0000", 2024-09-13T10:12:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5503e-05 2024-09-13T10:12:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991522+0000", 2024-09-13T10:12:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9399e-05 2024-09-13T10:12:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991544+0000", 2024-09-13T10:12:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2408999999999998e-05 2024-09-13T10:12:14.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991646+0000", 2024-09-13T10:12:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101798 2024-09-13T10:12:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991818+0000", 2024-09-13T10:12:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017204300000000001 2024-09-13T10:12:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.001618+0000", 2024-09-13T10:12:14.919 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097993490000000006 2024-09-13T10:12:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026785+0000", 2024-09-13T10:12:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025167446 2024-09-13T10:12:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026856+0000", 2024-09-13T10:12:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0575999999999996e-05 2024-09-13T10:12:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026915+0000", 2024-09-13T10:12:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9258000000000001e-05 2024-09-13T10:12:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026964+0000", 2024-09-13T10:12:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000003e-05 2024-09-13T10:12:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027011+0000", 2024-09-13T10:12:14.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7200000000000002e-05 2024-09-13T10:12:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:204 3.1 3:babcb876:::smithi19530714-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:12:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 83.919511795999995, 2024-09-13T10:12:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.22268644700000001, 2024-09-13T10:12:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.922 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 204 2024-09-13T10:12:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:12:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:12:14.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991589+0000", 2024-09-13T10:12:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3859999999999999e-06 2024-09-13T10:12:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991598+0000", 2024-09-13T10:12:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7669999999999992e-06 2024-09-13T10:12:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991616+0000", 2024-09-13T10:12:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7589e-05 2024-09-13T10:12:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991631+0000", 2024-09-13T10:12:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.47e-05 2024-09-13T10:12:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992050+0000", 2024-09-13T10:12:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041913399999999999 2024-09-13T10:12:14.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992229+0000", 2024-09-13T10:12:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017956499999999999 2024-09-13T10:12:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.203594+0000", 2024-09-13T10:12:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.211364303 2024-09-13T10:12:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.208526+0000", 2024-09-13T10:12:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049325020000000001 2024-09-13T10:12:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214019+0000", 2024-09-13T10:12:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054933270000000001 2024-09-13T10:12:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214108+0000", 2024-09-13T10:12:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8517000000000006e-05 2024-09-13T10:12:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214202+0000", 2024-09-13T10:12:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3763000000000003e-05 2024-09-13T10:12:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214275+0000", 2024-09-13T10:12:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2893999999999997e-05 2024-09-13T10:12:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:205 3.1 3:aec11a92:::smithi19530714-6 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:12:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 83.919436141999995, 2024-09-13T10:12:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056950259000000003, 2024-09-13T10:12:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 205 2024-09-13T10:12:14.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:12:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:12:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991665+0000", 2024-09-13T10:12:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1400000000000001e-06 2024-09-13T10:12:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991673+0000", 2024-09-13T10:12:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5730000000000008e-06 2024-09-13T10:12:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991689+0000", 2024-09-13T10:12:14.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5956000000000001e-05 2024-09-13T10:12:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991705+0000", 2024-09-13T10:12:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5539000000000001e-05 2024-09-13T10:12:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992393+0000", 2024-09-13T10:12:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068836600000000004 2024-09-13T10:12:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992509+0000", 2024-09-13T10:12:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115563 2024-09-13T10:12:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.011894+0000", 2024-09-13T10:12:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019385594999999999 2024-09-13T10:12:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:14.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027029+0000", 2024-09-13T10:12:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015134664000000001 2024-09-13T10:12:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.031277+0000", 2024-09-13T10:12:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004247804 2024-09-13T10:12:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048542+0000", 2024-09-13T10:12:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017265401999999999 2024-09-13T10:12:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048579+0000", 2024-09-13T10:12:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6238000000000001e-05 2024-09-13T10:12:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048614+0000", 2024-09-13T10:12:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5419000000000001e-05 2024-09-13T10:12:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:209 3.1 3:9798d58d:::smithi19530714-10 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:12:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 83.919307927000006, 2024-09-13T10:12:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.069824676000000002, 2024-09-13T10:12:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 209 2024-09-13T10:12:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.936 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:12:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:12:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991793+0000", 2024-09-13T10:12:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.057e-06 2024-09-13T10:12:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991803+0000", 2024-09-13T10:12:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7860000000000008e-06 2024-09-13T10:12:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991817+0000", 2024-09-13T10:12:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4066000000000001e-05 2024-09-13T10:12:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991828+0000", 2024-09-13T10:12:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1097000000000001e-05 2024-09-13T10:12:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992666+0000", 2024-09-13T10:12:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083755799999999997 2024-09-13T10:12:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992792+0000", 2024-09-13T10:12:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000126334 2024-09-13T10:12:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027675+0000", 2024-09-13T10:12:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034883299 2024-09-13T10:12:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.035648+0000", 2024-09-13T10:12:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007972972 2024-09-13T10:12:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.045764+0000", 2024-09-13T10:12:14.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010115832 2024-09-13T10:12:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061460+0000", 2024-09-13T10:12:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015695563999999999 2024-09-13T10:12:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061544+0000", 2024-09-13T10:12:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4866999999999999e-05 2024-09-13T10:12:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061617+0000", 2024-09-13T10:12:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2243999999999995e-05 2024-09-13T10:12:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:213 3.1 3:accf4e56:::smithi19530714-14 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:12:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 83.918533984000007, 2024-09-13T10:12:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.060593045999999998, 2024-09-13T10:12:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 213 2024-09-13T10:12:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:12:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:12:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992568+0000", 2024-09-13T10:12:14.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3640000000000002e-06 2024-09-13T10:12:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992580+0000", 2024-09-13T10:12:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1796e-05 2024-09-13T10:12:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992662+0000", 2024-09-13T10:12:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1814e-05 2024-09-13T10:12:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992683+0000", 2024-09-13T10:12:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0746000000000001e-05 2024-09-13T10:12:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.993024+0000", 2024-09-13T10:12:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034141299999999999 2024-09-13T10:12:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.993182+0000", 2024-09-13T10:12:14.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015817699999999999 2024-09-13T10:12:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.019852+0000", 2024-09-13T10:12:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026669470000000001 2024-09-13T10:12:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.032551+0000", 2024-09-13T10:12:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012699729999999999 2024-09-13T10:12:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.034351+0000", 2024-09-13T10:12:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017998269999999999 2024-09-13T10:12:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053106+0000", 2024-09-13T10:12:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018755236000000002 2024-09-13T10:12:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053131+0000", 2024-09-13T10:12:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4729e-05 2024-09-13T10:12:14.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053159+0000", 2024-09-13T10:12:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7744000000000001e-05 2024-09-13T10:12:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:219 3.1 3:87bfc7b1:::smithi19530714-20 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:12:14.949 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 83.878084911000002, 2024-09-13T10:12:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.037494069999999997, 2024-09-13T10:12:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 219 2024-09-13T10:12:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:12:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:12:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033019+0000", 2024-09-13T10:12:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.726e-06 2024-09-13T10:12:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033036+0000", 2024-09-13T10:12:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7329999999999998e-05 2024-09-13T10:12:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033060+0000", 2024-09-13T10:12:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4286000000000001e-05 2024-09-13T10:12:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033079+0000", 2024-09-13T10:12:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8872000000000002e-05 2024-09-13T10:12:14.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033155+0000", 2024-09-13T10:12:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6118999999999994e-05 2024-09-13T10:12:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033297+0000", 2024-09-13T10:12:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014154700000000001 2024-09-13T10:12:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:14.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.041666+0000", 2024-09-13T10:12:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083696180000000005 2024-09-13T10:12:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046621+0000", 2024-09-13T10:12:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049545759999999996 2024-09-13T10:12:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.052333+0000", 2024-09-13T10:12:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057120160000000003 2024-09-13T10:12:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070340+0000", 2024-09-13T10:12:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018006859 2024-09-13T10:12:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070447+0000", 2024-09-13T10:12:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106637 2024-09-13T10:12:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070509+0000", 2024-09-13T10:12:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2483999999999993e-05 2024-09-13T10:12:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.955 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:222 3.1 3:bdac4e61:::smithi19530714-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:12:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 83.864396936999995, 2024-09-13T10:12:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041434034000000002, 2024-09-13T10:12:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.956 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 222 2024-09-13T10:12:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:12:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:12:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046706+0000", 2024-09-13T10:12:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0089999999999999e-06 2024-09-13T10:12:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046719+0000", 2024-09-13T10:12:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2565000000000001e-05 2024-09-13T10:12:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046740+0000", 2024-09-13T10:12:14.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0979e-05 2024-09-13T10:12:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046752+0000", 2024-09-13T10:12:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2917e-05 2024-09-13T10:12:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046803+0000", 2024-09-13T10:12:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0417000000000002e-05 2024-09-13T10:12:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046908+0000", 2024-09-13T10:12:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105039 2024-09-13T10:12:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.066097+0000", 2024-09-13T10:12:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019189277000000001 2024-09-13T10:12:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070969+0000", 2024-09-13T10:12:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048721429999999998 2024-09-13T10:12:14.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.087852+0000", 2024-09-13T10:12:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016882489000000001 2024-09-13T10:12:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.087942+0000", 2024-09-13T10:12:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9738e-05 2024-09-13T10:12:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088076+0000", 2024-09-13T10:12:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013400700000000001 2024-09-13T10:12:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088137+0000", 2024-09-13T10:12:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1453999999999998e-05 2024-09-13T10:12:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:223 3.1 3:8759f4e0:::smithi19530714-24 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:12:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 83.851720068000006, 2024-09-13T10:12:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053691092000000003, 2024-09-13T10:12:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.963 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 223 2024-09-13T10:12:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:12:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:12:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059383+0000", 2024-09-13T10:12:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.847e-06 2024-09-13T10:12:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059399+0000", 2024-09-13T10:12:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6739e-05 2024-09-13T10:12:14.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059433+0000", 2024-09-13T10:12:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3946000000000003e-05 2024-09-13T10:12:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059456+0000", 2024-09-13T10:12:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2163000000000001e-05 2024-09-13T10:12:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059621+0000", 2024-09-13T10:12:14.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016546700000000001 2024-09-13T10:12:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.060003+0000", 2024-09-13T10:12:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000381982 2024-09-13T10:12:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.085339+0000", 2024-09-13T10:12:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025335610000000001 2024-09-13T10:12:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.089896+0000", 2024-09-13T10:12:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045575930000000004 2024-09-13T10:12:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.102515+0000", 2024-09-13T10:12:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012618664999999999 2024-09-13T10:12:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113009+0000", 2024-09-13T10:12:14.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010493784000000001 2024-09-13T10:12:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113044+0000", 2024-09-13T10:12:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5543999999999997e-05 2024-09-13T10:12:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113071+0000", 2024-09-13T10:12:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6752000000000001e-05 2024-09-13T10:12:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.969 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:234 3.1 3:bdce9caa:::smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:12:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 83.842000807000005, 2024-09-13T10:12:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043032263000000001, 2024-09-13T10:12:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.970 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 234 2024-09-13T10:12:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:12:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:12:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069102+0000", 2024-09-13T10:12:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.543e-06 2024-09-13T10:12:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069112+0000", 2024-09-13T10:12:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0091e-05 2024-09-13T10:12:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069133+0000", 2024-09-13T10:12:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1188000000000001e-05 2024-09-13T10:12:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069143+0000", 2024-09-13T10:12:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0456e-05 2024-09-13T10:12:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070588+0000", 2024-09-13T10:12:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001445046 2024-09-13T10:12:14.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070801+0000", 2024-09-13T10:12:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000212151 2024-09-13T10:12:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.082362+0000", 2024-09-13T10:12:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561781 2024-09-13T10:12:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088161+0000", 2024-09-13T10:12:14.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057986119999999999 2024-09-13T10:12:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.101972+0000", 2024-09-13T10:12:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013811498 2024-09-13T10:12:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112028+0000", 2024-09-13T10:12:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010055530999999999 2024-09-13T10:12:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112082+0000", 2024-09-13T10:12:14.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3974999999999997e-05 2024-09-13T10:12:14.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112131+0000", 2024-09-13T10:12:14.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9391000000000002e-05 2024-09-13T10:12:14.976 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.976 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.976 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.976 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:243 3.1 3:a4dd4282:::smithi19530714-44 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:12:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 83.813793102000005, 2024-09-13T10:12:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036759694000000002, 2024-09-13T10:12:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 243 2024-09-13T10:12:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:12:14.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:12:14.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097308+0000", 2024-09-13T10:12:14.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2099999999999995e-07 2024-09-13T10:12:14.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097314+0000", 2024-09-13T10:12:14.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9989999999999997e-06 2024-09-13T10:12:14.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097326+0000", 2024-09-13T10:12:14.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1787000000000001e-05 2024-09-13T10:12:14.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097332+0000", 2024-09-13T10:12:14.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6950000000000003e-06 2024-09-13T10:12:14.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097378+0000", 2024-09-13T10:12:14.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5744000000000001e-05 2024-09-13T10:12:14.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097489+0000", 2024-09-13T10:12:14.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111352 2024-09-13T10:12:14.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:14.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.118571+0000", 2024-09-13T10:12:14.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021081249999999999 2024-09-13T10:12:14.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:14.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.128494+0000", 2024-09-13T10:12:14.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099237270000000002 2024-09-13T10:12:14.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.128775+0000", 2024-09-13T10:12:14.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028027299999999999 2024-09-13T10:12:14.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:14.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.133963+0000", 2024-09-13T10:12:14.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005188191 2024-09-13T10:12:14.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:14.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134015+0000", 2024-09-13T10:12:14.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2438000000000002e-05 2024-09-13T10:12:14.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134066+0000", 2024-09-13T10:12:14.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1316999999999997e-05 2024-09-13T10:12:14.983 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.983 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.983 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.983 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1272 4.f 4:f7ad6ef6:::smithi19530714-1:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e45)", 2024-09-13T10:12:14.983 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:12:14.983 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 75.035564926999996, 2024-09-13T10:12:14.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.050041173000000001, 2024-09-13T10:12:14.984 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.984 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.984 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:14.984 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.984 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.984 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.984 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1272 2024-09-13T10:12:14.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.984 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:12:14.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:12:14.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875537+0000", 2024-09-13T10:12:14.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2689999999999998e-06 2024-09-13T10:12:14.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875548+0000", 2024-09-13T10:12:14.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0291e-05 2024-09-13T10:12:14.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875567+0000", 2024-09-13T10:12:14.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9284999999999999e-05 2024-09-13T10:12:14.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875578+0000", 2024-09-13T10:12:14.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1256e-05 2024-09-13T10:12:14.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875763+0000", 2024-09-13T10:12:14.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000185396 2024-09-13T10:12:14.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875908+0000", 2024-09-13T10:12:14.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014412299999999999 2024-09-13T10:12:14.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.987 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.925576+0000", 2024-09-13T10:12:14.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049668552999999997 2024-09-13T10:12:14.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.988 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.988 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.988 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2719 4.f 4:f7ad6ef6:::smithi19530714-1:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:12:14.988 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:29.386743+0000", 2024-09-13T10:12:14.988 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.524356844000003, 2024-09-13T10:12:14.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047467252000000001, 2024-09-13T10:12:14.988 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.988 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.988 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:14.988 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:14.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:14.989 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:14.989 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2719 2024-09-13T10:12:14.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.989 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386743+0000", 2024-09-13T10:12:14.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386743+0000", 2024-09-13T10:12:14.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386746+0000", 2024-09-13T10:12:14.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2729999999999998e-06 2024-09-13T10:12:14.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386758+0000", 2024-09-13T10:12:14.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1703e-05 2024-09-13T10:12:14.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386789+0000", 2024-09-13T10:12:14.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0815000000000002e-05 2024-09-13T10:12:14.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386806+0000", 2024-09-13T10:12:14.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7595999999999999e-05 2024-09-13T10:12:14.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386996+0000", 2024-09-13T10:12:14.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000189521 2024-09-13T10:12:14.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:14.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.387151+0000", 2024-09-13T10:12:14.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015501600000000001 2024-09-13T10:12:14.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.434210+0000", 2024-09-13T10:12:14.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047059327999999997 2024-09-13T10:12:14.992 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.992 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.992 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.993 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.9.21:5 3.1 3.b06a67a1 (undecoded) ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-13T10:12:14.993 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:03.964065+0000", 2024-09-13T10:12:14.993 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.947034984, 2024-09-13T10:12:14.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82246186799999998, 2024-09-13T10:12:14.993 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.993 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.993 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:12:14.993 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964065+0000", 2024-09-13T10:12:14.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964065+0000", 2024-09-13T10:12:14.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964069+0000", 2024-09-13T10:12:14.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9060000000000004e-06 2024-09-13T10:12:14.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:14.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964092+0000", 2024-09-13T10:12:14.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2892999999999999e-05 2024-09-13T10:12:14.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:14.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964133+0000", 2024-09-13T10:12:14.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1576000000000003e-05 2024-09-13T10:12:14.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:14.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964152+0000", 2024-09-13T10:12:14.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9165999999999999e-05 2024-09-13T10:12:14.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964289+0000", 2024-09-13T10:12:14.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136694 2024-09-13T10:12:14.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:14.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964310+0000", 2024-09-13T10:12:14.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1169e-05 2024-09-13T10:12:14.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:14.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:04.786399+0000", 2024-09-13T10:12:14.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82208862999999999 2024-09-13T10:12:14.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:14.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:04.786527+0000", 2024-09-13T10:12:14.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012783400000000001 2024-09-13T10:12:14.997 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.997 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:14.998 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:14.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.998 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.9.21:5 3.1 3:85e6560d:::989de9f18158d38e3c2846cd6c5625b2e4406737a04dca59570c2ce866cf967c:head [call cas.chunk_put_ref in=16b] snapc 0=[] RETRY=1 ondisk+retry+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:14.998 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.793509+0000", 2024-09-13T10:12:14.998 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1175912690000001, 2024-09-13T10:12:14.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.060644814999999998, 2024-09-13T10:12:14.998 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:14.998 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:14.998 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:14.998 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:14.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:14.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793509+0000", 2024-09-13T10:12:14.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:14.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793509+0000", 2024-09-13T10:12:14.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:14.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:14.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:14.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:14.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793511+0000", 2024-09-13T10:12:14.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2800000000000002e-06 2024-09-13T10:12:15.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:15.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793521+0000", 2024-09-13T10:12:15.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8090000000000004e-06 2024-09-13T10:12:15.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:15.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793539+0000", 2024-09-13T10:12:15.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8546000000000002e-05 2024-09-13T10:12:15.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793562+0000", 2024-09-13T10:12:15.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2490999999999999e-05 2024-09-13T10:12:15.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:15.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794585+0000", 2024-09-13T10:12:15.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010233519999999999 2024-09-13T10:12:15.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:15.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794602+0000", 2024-09-13T10:12:15.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6680999999999998e-05 2024-09-13T10:12:15.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:15.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.847842+0000", 2024-09-13T10:12:15.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053240290000000003 2024-09-13T10:12:15.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:15.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.848265+0000", 2024-09-13T10:12:15.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042276500000000002 2024-09-13T10:12:15.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,10,14", 2024-09-13T10:12:15.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.848806+0000", 2024-09-13T10:12:15.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054073499999999998 2024-09-13T10:12:15.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.851591+0000", 2024-09-13T10:12:15.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027856589999999998 2024-09-13T10:12:15.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.851712+0000", 2024-09-13T10:12:15.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120321 2024-09-13T10:12:15.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:15.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.851735+0000", 2024-09-13T10:12:15.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3825999999999999e-05 2024-09-13T10:12:15.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.853601+0000", 2024-09-13T10:12:15.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018660879999999999 2024-09-13T10:12:15.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:15.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.853776+0000", 2024-09-13T10:12:15.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017476300000000001 2024-09-13T10:12:15.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:15.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.854153+0000", 2024-09-13T10:12:15.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037720899999999999 2024-09-13T10:12:15.005 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:15.005 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:15.005 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:15.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.005 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.15.22:44 3.1 3:9ec9087f:::5d8d6b7076a8d108632e41519fcb6bd459cf76343846e25ee0c5e911722fca24:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:15.005 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.793735+0000", 2024-09-13T10:12:15.005 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1173646490000007, 2024-09-13T10:12:15.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.067146312, 2024-09-13T10:12:15.006 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:15.006 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:15.006 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.006 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:15.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:15.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793735+0000", 2024-09-13T10:12:15.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:15.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:15.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793735+0000", 2024-09-13T10:12:15.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:15.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:15.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793737+0000", 2024-09-13T10:12:15.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.404e-06 2024-09-13T10:12:15.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:15.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794023+0000", 2024-09-13T10:12:15.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028664599999999999 2024-09-13T10:12:15.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:15.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794045+0000", 2024-09-13T10:12:15.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1789000000000001e-05 2024-09-13T10:12:15.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794057+0000", 2024-09-13T10:12:15.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2061e-05 2024-09-13T10:12:15.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:15.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794659+0000", 2024-09-13T10:12:15.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060190099999999996 2024-09-13T10:12:15.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:15.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794670+0000", 2024-09-13T10:12:15.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0761000000000001e-05 2024-09-13T10:12:15.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:15.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.854930+0000", 2024-09-13T10:12:15.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.060260237000000001 2024-09-13T10:12:15.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:15.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.855350+0000", 2024-09-13T10:12:15.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042033800000000003 2024-09-13T10:12:15.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,10,14", 2024-09-13T10:12:15.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.856082+0000", 2024-09-13T10:12:15.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073115799999999998 2024-09-13T10:12:15.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.860193+0000", 2024-09-13T10:12:15.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041111079999999996 2024-09-13T10:12:15.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.860338+0000", 2024-09-13T10:12:15.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014509700000000001 2024-09-13T10:12:15.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.860470+0000", 2024-09-13T10:12:15.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132304 2024-09-13T10:12:15.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:15.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.860603+0000", 2024-09-13T10:12:15.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133137 2024-09-13T10:12:15.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:15.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.860686+0000", 2024-09-13T10:12:15.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2315e-05 2024-09-13T10:12:15.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:15.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.860882+0000", 2024-09-13T10:12:15.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019605600000000001 2024-09-13T10:12:15.012 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:15.013 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:15.013 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:15.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.013 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.15.22:45 3.1 3:974a72fa:::beb33e09e1c5ba22d3db3e2cd707b299c6694e34eb9bd9ac42f66c64167565c7:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:15.013 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.794078+0000", 2024-09-13T10:12:15.013 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1170217509999993, 2024-09-13T10:12:15.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.075604088999999999, 2024-09-13T10:12:15.013 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:15.013 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:15.013 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.013 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:15.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:15.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794078+0000", 2024-09-13T10:12:15.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:15.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:15.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794078+0000", 2024-09-13T10:12:15.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:15.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:15.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794079+0000", 2024-09-13T10:12:15.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0189999999999999e-06 2024-09-13T10:12:15.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:15.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794467+0000", 2024-09-13T10:12:15.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038798499999999998 2024-09-13T10:12:15.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:15.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794484+0000", 2024-09-13T10:12:15.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6388999999999998e-05 2024-09-13T10:12:15.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794495+0000", 2024-09-13T10:12:15.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.118e-05 2024-09-13T10:12:15.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:15.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794708+0000", 2024-09-13T10:12:15.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000213546 2024-09-13T10:12:15.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:15.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794718+0000", 2024-09-13T10:12:15.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9620000000000005e-06 2024-09-13T10:12:15.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:15.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.860986+0000", 2024-09-13T10:12:15.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.066268218000000004 2024-09-13T10:12:15.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:15.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.861385+0000", 2024-09-13T10:12:15.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039855799999999997 2024-09-13T10:12:15.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,10,14", 2024-09-13T10:12:15.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.862131+0000", 2024-09-13T10:12:15.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074556799999999995 2024-09-13T10:12:15.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.864957+0000", 2024-09-13T10:12:15.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028266950000000002 2024-09-13T10:12:15.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.865115+0000", 2024-09-13T10:12:15.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015747299999999999 2024-09-13T10:12:15.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:15.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.866537+0000", 2024-09-13T10:12:15.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014227409999999999 2024-09-13T10:12:15.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.869419+0000", 2024-09-13T10:12:15.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028810770000000001 2024-09-13T10:12:15.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:15.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.869518+0000", 2024-09-13T10:12:15.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8997000000000001e-05 2024-09-13T10:12:15.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:15.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.869682+0000", 2024-09-13T10:12:15.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016468099999999999 2024-09-13T10:12:15.020 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:15.020 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:15.020 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:15.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.020 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.15.22:51 3.1 3:845d12bf:::28848c4777c1cb2908d895c4117b5dc4ef8bdb4529e24915ac4b533e48aa206d:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:15.020 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.794512+0000", 2024-09-13T10:12:15.021 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1165878750000005, 2024-09-13T10:12:15.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.079145091000000001, 2024-09-13T10:12:15.021 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:15.021 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:15.021 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.021 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:15.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:15.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794512+0000", 2024-09-13T10:12:15.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:15.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:15.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794512+0000", 2024-09-13T10:12:15.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:15.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:15.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794513+0000", 2024-09-13T10:12:15.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0699999999999996e-07 2024-09-13T10:12:15.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:15.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794755+0000", 2024-09-13T10:12:15.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241955 2024-09-13T10:12:15.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:15.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794769+0000", 2024-09-13T10:12:15.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3784999999999999e-05 2024-09-13T10:12:15.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794778+0000", 2024-09-13T10:12:15.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6970000000000005e-06 2024-09-13T10:12:15.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:15.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794903+0000", 2024-09-13T10:12:15.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012476499999999999 2024-09-13T10:12:15.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:15.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794920+0000", 2024-09-13T10:12:15.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6393999999999999e-05 2024-09-13T10:12:15.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:15.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.866605+0000", 2024-09-13T10:12:15.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.071685663999999996 2024-09-13T10:12:15.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:15.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.867044+0000", 2024-09-13T10:12:15.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043897700000000001 2024-09-13T10:12:15.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,10,14", 2024-09-13T10:12:15.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.867828+0000", 2024-09-13T10:12:15.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078432999999999997 2024-09-13T10:12:15.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.870457+0000", 2024-09-13T10:12:15.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026287229999999999 2024-09-13T10:12:15.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.870558+0000", 2024-09-13T10:12:15.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010079800000000001 2024-09-13T10:12:15.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:15.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.872235+0000", 2024-09-13T10:12:15.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016767780000000001 2024-09-13T10:12:15.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.873474+0000", 2024-09-13T10:12:15.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001239448 2024-09-13T10:12:15.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:15.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.873539+0000", 2024-09-13T10:12:15.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4655999999999999e-05 2024-09-13T10:12:15.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:15.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.873657+0000", 2024-09-13T10:12:15.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118214 2024-09-13T10:12:15.028 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:15.028 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:15.028 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:15.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.028 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.15.22:59 3.1 3:964c391a:::c9c0cf4a00ae29c03b283d9cbb01cfb175b42402a28680dcd93389b12aeeb768:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:15.028 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.794801+0000", 2024-09-13T10:12:15.028 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.1162988160000005, 2024-09-13T10:12:15.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.080623212999999999, 2024-09-13T10:12:15.028 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:15.028 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:15.029 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.029 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:15.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:15.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794801+0000", 2024-09-13T10:12:15.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:15.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:15.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794801+0000", 2024-09-13T10:12:15.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:15.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:15.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794802+0000", 2024-09-13T10:12:15.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.029e-06 2024-09-13T10:12:15.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:15.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794839+0000", 2024-09-13T10:12:15.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6442000000000001e-05 2024-09-13T10:12:15.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:15.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794907+0000", 2024-09-13T10:12:15.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8913999999999998e-05 2024-09-13T10:12:15.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794923+0000", 2024-09-13T10:12:15.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5393e-05 2024-09-13T10:12:15.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:15.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794993+0000", 2024-09-13T10:12:15.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9777000000000006e-05 2024-09-13T10:12:15.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:15.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.795005+0000", 2024-09-13T10:12:15.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2536e-05 2024-09-13T10:12:15.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:15.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.870632+0000", 2024-09-13T10:12:15.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.075626342999999999 2024-09-13T10:12:15.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:15.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.870939+0000", 2024-09-13T10:12:15.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000306994 2024-09-13T10:12:15.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,10,14", 2024-09-13T10:12:15.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.871368+0000", 2024-09-13T10:12:15.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042931399999999998 2024-09-13T10:12:15.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.873711+0000", 2024-09-13T10:12:15.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023427159999999999 2024-09-13T10:12:15.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.873802+0000", 2024-09-13T10:12:15.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1730000000000004e-05 2024-09-13T10:12:15.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:15.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.873834+0000", 2024-09-13T10:12:15.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2106000000000001e-05 2024-09-13T10:12:15.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.875254+0000", 2024-09-13T10:12:15.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014194279999999999 2024-09-13T10:12:15.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:15.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.875313+0000", 2024-09-13T10:12:15.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9660000000000001e-05 2024-09-13T10:12:15.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:15.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.875424+0000", 2024-09-13T10:12:15.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110831 2024-09-13T10:12:15.035 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:15.035 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:15.035 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:15.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.036 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.23:171 3.1 3:ae1afb27:::0b4da3b2f5b021545e17c9fa615b6bae2f6acc3b51ee6b20bac582de1c7125d8:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:12:15.036 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.834554+0000", 2024-09-13T10:12:15.036 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.0765456449999995, 2024-09-13T10:12:15.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035802232000000003, 2024-09-13T10:12:15.036 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:15.036 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:15.036 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.036 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:15.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:15.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.834554+0000", 2024-09-13T10:12:15.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:15.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:15.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.834554+0000", 2024-09-13T10:12:15.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:15.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:15.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.834555+0000", 2024-09-13T10:12:15.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4399999999999999e-07 2024-09-13T10:12:15.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:15.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.834560+0000", 2024-09-13T10:12:15.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7600000000000002e-06 2024-09-13T10:12:15.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:15.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.834569+0000", 2024-09-13T10:12:15.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3279999999999995e-06 2024-09-13T10:12:15.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.834578+0000", 2024-09-13T10:12:15.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4179999999999997e-06 2024-09-13T10:12:15.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:15.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.834759+0000", 2024-09-13T10:12:15.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000181029 2024-09-13T10:12:15.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:15.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.834771+0000", 2024-09-13T10:12:15.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2660000000000001e-05 2024-09-13T10:12:15.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:15.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.865214+0000", 2024-09-13T10:12:15.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030443006000000002 2024-09-13T10:12:15.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:15.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.865441+0000", 2024-09-13T10:12:15.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022639900000000001 2024-09-13T10:12:15.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,10,14", 2024-09-13T10:12:15.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.865775+0000", 2024-09-13T10:12:15.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033417000000000001 2024-09-13T10:12:15.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.869809+0000", 2024-09-13T10:12:15.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040337819999999996 2024-09-13T10:12:15.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.869955+0000", 2024-09-13T10:12:15.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014625400000000001 2024-09-13T10:12:15.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:15.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.869996+0000", 2024-09-13T10:12:15.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1219e-05 2024-09-13T10:12:15.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.870086+0000", 2024-09-13T10:12:15.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0101999999999999e-05 2024-09-13T10:12:15.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:15.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.870184+0000", 2024-09-13T10:12:15.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7861000000000004e-05 2024-09-13T10:12:15.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:15.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.870356+0000", 2024-09-13T10:12:15.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017239999999999999 2024-09-13T10:12:15.042 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:15.042 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:15.042 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:15.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.043 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4552 4.f 4:f7ad6ef6:::smithi19530714-1:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e106)", 2024-09-13T10:12:15.043 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:14.681527+0000", 2024-09-13T10:12:15.043 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.22957290699999999, 2024-09-13T10:12:15.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040766716000000001, 2024-09-13T10:12:15.043 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:15.043 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:15.043 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:15.043 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:15.043 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:15.043 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.043 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4552 2024-09-13T10:12:15.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.044 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:15.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:15.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.681527+0000", 2024-09-13T10:12:15.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:15.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:15.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.681527+0000", 2024-09-13T10:12:15.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:15.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:15.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.681530+0000", 2024-09-13T10:12:15.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8289999999999999e-06 2024-09-13T10:12:15.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:15.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.681543+0000", 2024-09-13T10:12:15.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2747e-05 2024-09-13T10:12:15.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:15.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.681570+0000", 2024-09-13T10:12:15.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7492e-05 2024-09-13T10:12:15.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.681587+0000", 2024-09-13T10:12:15.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6650000000000002e-05 2024-09-13T10:12:15.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:15.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.681691+0000", 2024-09-13T10:12:15.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104751 2024-09-13T10:12:15.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:15.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.681791+0000", 2024-09-13T10:12:15.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9263000000000002e-05 2024-09-13T10:12:15.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:15.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:15.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:15.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.722294+0000", 2024-09-13T10:12:15.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040502983999999999 2024-09-13T10:12:15.047 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:15.047 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:15.047 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:15.047 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:15.047 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:15.047 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:15.049 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T10:12:15.050 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:15.042+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:15.060 INFO:tasks.rados.rados.0.smithi195.stdout:1876: expect (ObjNum 408 snap 0 seq_num 408) 2024-09-13T10:12:15.140 INFO:tasks.rados.rados.0.smithi195.stdout:1877: finishing write tid 1 to smithi19530714-40 2024-09-13T10:12:15.141 INFO:tasks.rados.rados.0.smithi195.stdout:1873: got (0) Success 2024-09-13T10:12:15.141 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 0 (ObjNum 405 snap 0 seq_num 405) dirty exists 2024-09-13T10:12:15.141 INFO:tasks.rados.rados.0.smithi195.stdout:1880: expect (ObjNum 389 snap 0 seq_num 389) 2024-09-13T10:12:15.149 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:12:15.147+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:12:15.172 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:12:15.172 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:15.173 INFO:teuthology.orchestra.run.smithi073.stdout: "epoch": 107, 2024-09-13T10:12:15.173 INFO:teuthology.orchestra.run.smithi073.stdout: "fsid": "c1e91d47-9492-4461-a043-8698fd74ccf3", 2024-09-13T10:12:15.173 INFO:teuthology.orchestra.run.smithi073.stdout: "created": "2024-09-13T10:10:14.019912+0000", 2024-09-13T10:12:15.173 INFO:teuthology.orchestra.run.smithi073.stdout: "modified": "2024-09-13T10:12:14.713308+0000", 2024-09-13T10:12:15.173 INFO:teuthology.orchestra.run.smithi073.stdout: "last_up_change": "2024-09-13T10:10:28.089885+0000", 2024-09-13T10:12:15.173 INFO:teuthology.orchestra.run.smithi073.stdout: "last_in_change": "2024-09-13T10:10:18.489546+0000", 2024-09-13T10:12:15.173 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T10:12:15.173 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_num": 5799936, 2024-09-13T10:12:15.173 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_set": [ 2024-09-13T10:12:15.173 INFO:teuthology.orchestra.run.smithi073.stdout: "pglog_hardlimit", 2024-09-13T10:12:15.173 INFO:teuthology.orchestra.run.smithi073.stdout: "purged_snapdirs", 2024-09-13T10:12:15.174 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_deletes", 2024-09-13T10:12:15.174 INFO:teuthology.orchestra.run.smithi073.stdout: "sortbitwise" 2024-09-13T10:12:15.174 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:12:15.174 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_version": 6, 2024-09-13T10:12:15.174 INFO:teuthology.orchestra.run.smithi073.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T10:12:15.174 INFO:teuthology.orchestra.run.smithi073.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T10:12:15.174 INFO:teuthology.orchestra.run.smithi073.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T10:12:15.174 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_snapshot": "", 2024-09-13T10:12:15.174 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_max": 4, 2024-09-13T10:12:15.174 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd": 16, 2024-09-13T10:12:15.174 INFO:teuthology.orchestra.run.smithi073.stdout: "require_min_compat_client": "luminous", 2024-09-13T10:12:15.174 INFO:teuthology.orchestra.run.smithi073.stdout: "min_compat_client": "luminous", 2024-09-13T10:12:15.174 INFO:teuthology.orchestra.run.smithi073.stdout: "require_osd_release": "squid", 2024-09-13T10:12:15.174 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_crimson": false, 2024-09-13T10:12:15.175 INFO:teuthology.orchestra.run.smithi073.stdout: "pools": [ 2024-09-13T10:12:15.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.175 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 1, 2024-09-13T10:12:15.175 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "rbd", 2024-09-13T10:12:15.175 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:28.924860+0000", 2024-09-13T10:12:15.175 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 8193, 2024-09-13T10:12:15.175 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T10:12:15.175 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:12:15.175 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:12:15.175 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:12:15.175 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:12:15.175 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:12:15.175 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:12:15.175 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:12:15.176 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:12:15.176 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:12:15.176 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:12:15.176 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:12:15.176 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 8, 2024-09-13T10:12:15.176 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 8, 2024-09-13T10:12:15.176 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 8, 2024-09-13T10:12:15.176 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 8, 2024-09-13T10:12:15.176 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 8, 2024-09-13T10:12:15.176 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:12:15.176 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:12:15.176 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:12:15.176 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:12:15.176 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:12:15.177 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:12:15.177 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:12:15.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.177 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "27", 2024-09-13T10:12:15.177 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:12:15.177 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:12:15.177 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:12:15.177 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:12:15.177 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:12:15.177 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 2, 2024-09-13T10:12:15.177 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 27, 2024-09-13T10:12:15.177 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:12:15.178 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:12:15.178 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:12:15.178 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:12:15.178 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:12:15.178 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:12:15.178 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:12:15.178 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:12:15.178 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:12:15.178 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:12:15.178 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:12:15.179 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:12:15.179 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:12:15.179 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:12:15.179 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:12:15.179 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:12:15.179 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:12:15.179 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:12:15.179 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:12:15.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.179 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:12:15.179 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:12:15.179 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:12:15.179 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:12:15.179 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:12:15.180 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:12:15.180 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:12:15.180 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:12:15.180 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:12:15.180 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:12:15.180 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:12:15.180 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-13T10:12:15.180 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-13T10:12:15.180 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd": {} 2024-09-13T10:12:15.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.180 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:12:15.180 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:12:15.181 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 5.2300000190734863, 2024-09-13T10:12:15.181 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 5.2300000190734863, 2024-09-13T10:12:15.181 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.79000002145767212, 2024-09-13T10:12:15.181 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 4.130000114440918, 2024-09-13T10:12:15.181 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 4.130000114440918, 2024-09-13T10:12:15.181 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8372650146484375, 2024-09-13T10:12:15.181 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:12:15.181 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.8399999737739563 2024-09-13T10:12:15.181 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.181 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-13T10:12:15.181 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": ".mgr", 2024-09-13T10:12:15.182 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:28.972519+0000", 2024-09-13T10:12:15.182 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-13T10:12:15.182 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-13T10:12:15.182 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:12:15.182 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:12:15.182 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:12:15.182 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:12:15.182 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:12:15.182 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:12:15.182 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:12:15.182 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:12:15.182 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:12:15.182 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:12:15.183 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:12:15.183 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 1, 2024-09-13T10:12:15.183 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 1, 2024-09-13T10:12:15.183 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 1, 2024-09-13T10:12:15.183 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 1, 2024-09-13T10:12:15.183 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 1, 2024-09-13T10:12:15.183 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:12:15.183 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:12:15.183 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:12:15.183 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:12:15.183 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:12:15.183 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:12:15.184 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:12:15.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.184 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "26", 2024-09-13T10:12:15.184 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:12:15.184 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:12:15.184 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:12:15.184 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:12:15.184 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:12:15.184 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-13T10:12:15.184 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-13T10:12:15.184 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:12:15.184 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:12:15.185 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:12:15.185 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:12:15.185 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:12:15.185 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:12:15.185 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:12:15.185 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:12:15.185 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:12:15.185 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:12:15.185 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:12:15.185 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:12:15.185 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:12:15.185 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:12:15.185 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:12:15.185 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:12:15.186 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:12:15.186 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:12:15.186 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:12:15.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.186 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:12:15.186 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:12:15.186 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:12:15.186 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:12:15.186 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:12:15.186 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:12:15.186 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:12:15.186 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:12:15.186 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:12:15.187 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:12:15.187 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:12:15.187 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-13T10:12:15.187 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_max": 32, 2024-09-13T10:12:15.187 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_min": 1 2024-09-13T10:12:15.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.187 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-13T10:12:15.187 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": {} 2024-09-13T10:12:15.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.187 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:12:15.187 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:12:15.187 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 14.289999961853027, 2024-09-13T10:12:15.187 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 14.289999961853027, 2024-09-13T10:12:15.187 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.20999999344348907, 2024-09-13T10:12:15.188 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 3, 2024-09-13T10:12:15.188 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 3, 2024-09-13T10:12:15.188 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8372650146484375, 2024-09-13T10:12:15.188 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:12:15.188 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.8399999737739563 2024-09-13T10:12:15.188 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.188 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 3, 2024-09-13T10:12:15.188 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "low_tier", 2024-09-13T10:12:15.188 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:41.320719+0000", 2024-09-13T10:12:15.189 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-13T10:12:15.189 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-13T10:12:15.189 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:12:15.189 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:12:15.189 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:12:15.189 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:12:15.189 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:12:15.189 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:12:15.189 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:12:15.189 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:12:15.189 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:12:15.189 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:12:15.190 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:12:15.190 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 4, 2024-09-13T10:12:15.190 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 4, 2024-09-13T10:12:15.190 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 4, 2024-09-13T10:12:15.190 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 4, 2024-09-13T10:12:15.190 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 4, 2024-09-13T10:12:15.190 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:12:15.190 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:12:15.190 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:12:15.190 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:12:15.190 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:12:15.190 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:12:15.191 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:12:15.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.191 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "32", 2024-09-13T10:12:15.191 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:12:15.191 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:12:15.191 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T10:12:15.191 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:12:15.191 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:12:15.191 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-13T10:12:15.191 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-13T10:12:15.192 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:12:15.192 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:12:15.192 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:12:15.192 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:12:15.192 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:12:15.192 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:12:15.192 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:12:15.192 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:12:15.192 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:12:15.192 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:12:15.192 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:12:15.193 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:12:15.193 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:12:15.193 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:12:15.193 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:12:15.193 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:12:15.193 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:12:15.193 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:12:15.193 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:12:15.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.193 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:12:15.193 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:12:15.193 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:12:15.194 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:12:15.194 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:12:15.194 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:12:15.194 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:12:15.194 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:12:15.194 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:12:15.194 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:12:15.194 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:12:15.194 INFO:teuthology.orchestra.run.smithi073.stdout: "options": {}, 2024-09-13T10:12:15.194 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": {}, 2024-09-13T10:12:15.194 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:12:15.194 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:12:15.194 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 3.5199999809265137, 2024-09-13T10:12:15.195 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 3.5199999809265137, 2024-09-13T10:12:15.195 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 0.63999998569488525, 2024-09-13T10:12:15.195 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 2.25, 2024-09-13T10:12:15.195 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 2.25, 2024-09-13T10:12:15.195 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8372650146484375, 2024-09-13T10:12:15.195 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:12:15.195 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.8399999737739563 2024-09-13T10:12:15.195 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.195 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 4, 2024-09-13T10:12:15.195 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": "unique_pool_0", 2024-09-13T10:12:15.195 INFO:teuthology.orchestra.run.smithi073.stdout: "create_time": "2024-09-13T10:10:42.575395+0000", 2024-09-13T10:12:15.195 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-09-13T10:12:15.196 INFO:teuthology.orchestra.run.smithi073.stdout: "flags_names": "hashpspool", 2024-09-13T10:12:15.196 INFO:teuthology.orchestra.run.smithi073.stdout: "type": 1, 2024-09-13T10:12:15.196 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 3, 2024-09-13T10:12:15.196 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": 2, 2024-09-13T10:12:15.196 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_rule": 0, 2024-09-13T10:12:15.196 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_count": 0, 2024-09-13T10:12:15.196 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_target": 0, 2024-09-13T10:12:15.196 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T10:12:15.196 INFO:teuthology.orchestra.run.smithi073.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T10:12:15.196 INFO:teuthology.orchestra.run.smithi073.stdout: "is_stretch_pool": false, 2024-09-13T10:12:15.196 INFO:teuthology.orchestra.run.smithi073.stdout: "object_hash": 2, 2024-09-13T10:12:15.196 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscale_mode": "off", 2024-09-13T10:12:15.197 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": 26, 2024-09-13T10:12:15.197 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num": 24, 2024-09-13T10:12:15.197 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_placement_num_target": 16, 2024-09-13T10:12:15.197 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_target": 16, 2024-09-13T10:12:15.197 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num_pending": 26, 2024-09-13T10:12:15.197 INFO:teuthology.orchestra.run.smithi073.stdout: "last_pg_merge_meta": { 2024-09-13T10:12:15.197 INFO:teuthology.orchestra.run.smithi073.stdout: "source_pgid": "0.0", 2024-09-13T10:12:15.197 INFO:teuthology.orchestra.run.smithi073.stdout: "ready_epoch": 0, 2024-09-13T10:12:15.197 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_started": 0, 2024-09-13T10:12:15.197 INFO:teuthology.orchestra.run.smithi073.stdout: "last_epoch_clean": 0, 2024-09-13T10:12:15.197 INFO:teuthology.orchestra.run.smithi073.stdout: "source_version": "0'0", 2024-09-13T10:12:15.198 INFO:teuthology.orchestra.run.smithi073.stdout: "target_version": "0'0" 2024-09-13T10:12:15.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.198 INFO:teuthology.orchestra.run.smithi073.stdout: "last_change": "106", 2024-09-13T10:12:15.198 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend": "0", 2024-09-13T10:12:15.198 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T10:12:15.198 INFO:teuthology.orchestra.run.smithi073.stdout: "last_force_op_resend_preluminous": "41", 2024-09-13T10:12:15.198 INFO:teuthology.orchestra.run.smithi073.stdout: "auid": 0, 2024-09-13T10:12:15.198 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_mode": "selfmanaged", 2024-09-13T10:12:15.198 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_seq": 0, 2024-09-13T10:12:15.198 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_epoch": 0, 2024-09-13T10:12:15.198 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_snaps": [], 2024-09-13T10:12:15.198 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps": "[]", 2024-09-13T10:12:15.198 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_bytes": 0, 2024-09-13T10:12:15.199 INFO:teuthology.orchestra.run.smithi073.stdout: "quota_max_objects": 0, 2024-09-13T10:12:15.199 INFO:teuthology.orchestra.run.smithi073.stdout: "tiers": [], 2024-09-13T10:12:15.199 INFO:teuthology.orchestra.run.smithi073.stdout: "tier_of": -1, 2024-09-13T10:12:15.199 INFO:teuthology.orchestra.run.smithi073.stdout: "read_tier": -1, 2024-09-13T10:12:15.199 INFO:teuthology.orchestra.run.smithi073.stdout: "write_tier": -1, 2024-09-13T10:12:15.199 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_mode": "none", 2024-09-13T10:12:15.199 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_bytes": 0, 2024-09-13T10:12:15.199 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_objects": 0, 2024-09-13T10:12:15.199 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T10:12:15.199 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T10:12:15.199 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T10:12:15.199 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_flush_age": 0, 2024-09-13T10:12:15.199 INFO:teuthology.orchestra.run.smithi073.stdout: "cache_min_evict_age": 0, 2024-09-13T10:12:15.199 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profile": "", 2024-09-13T10:12:15.200 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_params": { 2024-09-13T10:12:15.200 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "none" 2024-09-13T10:12:15.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.200 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_period": 0, 2024-09-13T10:12:15.200 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_count": 0, 2024-09-13T10:12:15.200 INFO:teuthology.orchestra.run.smithi073.stdout: "use_gmt_hitset": true, 2024-09-13T10:12:15.200 INFO:teuthology.orchestra.run.smithi073.stdout: "min_read_recency_for_promote": 0, 2024-09-13T10:12:15.200 INFO:teuthology.orchestra.run.smithi073.stdout: "min_write_recency_for_promote": 0, 2024-09-13T10:12:15.200 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T10:12:15.200 INFO:teuthology.orchestra.run.smithi073.stdout: "hit_set_search_last_n": 0, 2024-09-13T10:12:15.200 INFO:teuthology.orchestra.run.smithi073.stdout: "grade_table": [], 2024-09-13T10:12:15.200 INFO:teuthology.orchestra.run.smithi073.stdout: "stripe_width": 0, 2024-09-13T10:12:15.200 INFO:teuthology.orchestra.run.smithi073.stdout: "expected_num_objects": 0, 2024-09-13T10:12:15.201 INFO:teuthology.orchestra.run.smithi073.stdout: "fast_read": false, 2024-09-13T10:12:15.201 INFO:teuthology.orchestra.run.smithi073.stdout: "options": { 2024-09-13T10:12:15.201 INFO:teuthology.orchestra.run.smithi073.stdout: "dedup_cdc_chunk_size": 131072, 2024-09-13T10:12:15.201 INFO:teuthology.orchestra.run.smithi073.stdout: "dedup_chunk_algorithm": "fastcdc", 2024-09-13T10:12:15.201 INFO:teuthology.orchestra.run.smithi073.stdout: "dedup_tier": 3, 2024-09-13T10:12:15.201 INFO:teuthology.orchestra.run.smithi073.stdout: "fingerprint_algorithm": "sha256" 2024-09-13T10:12:15.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.201 INFO:teuthology.orchestra.run.smithi073.stdout: "application_metadata": { 2024-09-13T10:12:15.201 INFO:teuthology.orchestra.run.smithi073.stdout: "rados": {} 2024-09-13T10:12:15.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.201 INFO:teuthology.orchestra.run.smithi073.stdout: "read_balance": { 2024-09-13T10:12:15.201 INFO:teuthology.orchestra.run.smithi073.stdout: "score_type": "Fair distribution", 2024-09-13T10:12:15.201 INFO:teuthology.orchestra.run.smithi073.stdout: "score_acting": 3.2400000095367432, 2024-09-13T10:12:15.202 INFO:teuthology.orchestra.run.smithi073.stdout: "score_stable": 3.2400000095367432, 2024-09-13T10:12:15.202 INFO:teuthology.orchestra.run.smithi073.stdout: "optimal_score": 1.1399999856948853, 2024-09-13T10:12:15.202 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_acting": 3.690000057220459, 2024-09-13T10:12:15.202 INFO:teuthology.orchestra.run.smithi073.stdout: "raw_score_stable": 3.690000057220459, 2024-09-13T10:12:15.202 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity_weighted": 0.8372650146484375, 2024-09-13T10:12:15.202 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity": 0.95999997854232788, 2024-09-13T10:12:15.202 INFO:teuthology.orchestra.run.smithi073.stdout: "average_primary_affinity_weighted": 0.8399999737739563 2024-09-13T10:12:15.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.202 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:12:15.202 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:12:15.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.203 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-13T10:12:15.203 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "f7d92188-c27c-4744-a330-4b7105e0f2d6", 2024-09-13T10:12:15.203 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:12:15.203 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:12:15.203 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:12:15.203 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:12:15.203 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:12:15.203 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:12:15.203 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:12:15.203 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 79, 2024-09-13T10:12:15.203 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:12:15.203 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:12:15.204 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:12:15.204 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.204 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.204 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6812", 2024-09-13T10:12:15.204 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:12:15.204 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.204 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.204 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:12:15.204 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.204 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.205 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6813", 2024-09-13T10:12:15.205 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:12:15.205 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.205 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.205 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:12:15.205 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.205 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.205 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6815", 2024-09-13T10:12:15.205 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:12:15.205 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.206 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.206 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:12:15.206 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.206 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.206 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6814", 2024-09-13T10:12:15.206 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1977392321 2024-09-13T10:12:15.206 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.206 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.206 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6812/1977392321", 2024-09-13T10:12:15.206 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6813/1977392321", 2024-09-13T10:12:15.207 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6815/1977392321", 2024-09-13T10:12:15.207 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6814/1977392321", 2024-09-13T10:12:15.207 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:12:15.207 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:12:15.207 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:12:15.207 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.207 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-13T10:12:15.207 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e900b771-7c96-4b5d-93d7-35a8ea6c026a", 2024-09-13T10:12:15.207 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:12:15.207 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:12:15.207 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:12:15.207 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:12:15.208 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:12:15.208 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:12:15.208 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:12:15.208 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 98, 2024-09-13T10:12:15.208 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:12:15.208 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:12:15.208 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:12:15.208 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.208 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.208 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6808", 2024-09-13T10:12:15.208 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:12:15.209 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.209 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.209 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:12:15.209 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.209 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.209 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6809", 2024-09-13T10:12:15.209 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:12:15.209 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.209 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.210 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:12:15.210 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.210 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.210 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6811", 2024-09-13T10:12:15.210 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:12:15.210 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.210 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.210 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:12:15.210 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.210 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.211 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6810", 2024-09-13T10:12:15.211 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1228102864 2024-09-13T10:12:15.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.211 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.211 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6808/1228102864", 2024-09-13T10:12:15.211 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6809/1228102864", 2024-09-13T10:12:15.211 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6811/1228102864", 2024-09-13T10:12:15.211 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6810/1228102864", 2024-09-13T10:12:15.211 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:12:15.211 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:12:15.211 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:12:15.212 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.212 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-13T10:12:15.212 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "28b7aa59-33b2-4e24-8f3d-758f90e25f68", 2024-09-13T10:12:15.212 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:12:15.212 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:12:15.212 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:12:15.212 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:12:15.213 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:12:15.213 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:12:15.213 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:12:15.213 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 105, 2024-09-13T10:12:15.213 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:12:15.213 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:12:15.213 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:12:15.213 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.213 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.213 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6800", 2024-09-13T10:12:15.213 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:12:15.213 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.214 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.214 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:12:15.214 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.214 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.214 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6801", 2024-09-13T10:12:15.214 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:12:15.214 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.214 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.214 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:12:15.214 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.215 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.215 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6803", 2024-09-13T10:12:15.215 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:12:15.215 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.215 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.215 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:12:15.215 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.215 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.215 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6802", 2024-09-13T10:12:15.215 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3314820936 2024-09-13T10:12:15.216 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.216 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.216 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6800/3314820936", 2024-09-13T10:12:15.216 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6801/3314820936", 2024-09-13T10:12:15.216 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6803/3314820936", 2024-09-13T10:12:15.216 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6802/3314820936", 2024-09-13T10:12:15.216 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:12:15.216 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:12:15.216 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:12:15.216 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.217 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-13T10:12:15.217 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "a93d2df5-781e-49e1-a507-1718b6f954ed", 2024-09-13T10:12:15.217 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:12:15.217 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:12:15.217 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0.95001220703125, 2024-09-13T10:12:15.217 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:12:15.217 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:12:15.217 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:12:15.217 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:12:15.217 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 106, 2024-09-13T10:12:15.217 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:12:15.218 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:12:15.218 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:12:15.218 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.218 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.218 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6801", 2024-09-13T10:12:15.218 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:12:15.218 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.218 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.218 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:12:15.218 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.219 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.219 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6802", 2024-09-13T10:12:15.219 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:12:15.219 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.219 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.219 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:12:15.219 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.219 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.219 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6804", 2024-09-13T10:12:15.219 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:12:15.219 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.220 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.220 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:12:15.220 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.220 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.220 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6803", 2024-09-13T10:12:15.220 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2558565460 2024-09-13T10:12:15.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.220 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.220 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6801/2558565460", 2024-09-13T10:12:15.220 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6802/2558565460", 2024-09-13T10:12:15.221 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6804/2558565460", 2024-09-13T10:12:15.221 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6803/2558565460", 2024-09-13T10:12:15.221 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:12:15.221 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:12:15.221 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:12:15.221 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.221 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-13T10:12:15.221 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "e69053fd-f6d9-437e-a9d0-8896de6b927f", 2024-09-13T10:12:15.221 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:12:15.221 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:12:15.222 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0.95001220703125, 2024-09-13T10:12:15.222 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:12:15.222 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:12:15.222 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:12:15.222 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:12:15.222 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 76, 2024-09-13T10:12:15.222 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:12:15.222 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:12:15.222 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:12:15.222 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.222 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.222 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6800", 2024-09-13T10:12:15.223 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:12:15.223 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.223 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.223 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:12:15.223 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.223 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.223 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6801", 2024-09-13T10:12:15.223 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:12:15.223 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.223 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.224 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:12:15.224 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.224 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.224 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6803", 2024-09-13T10:12:15.224 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:12:15.224 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.224 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.224 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:12:15.224 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.224 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.225 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6802", 2024-09-13T10:12:15.225 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 858330493 2024-09-13T10:12:15.225 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.225 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.225 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6800/858330493", 2024-09-13T10:12:15.225 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6801/858330493", 2024-09-13T10:12:15.225 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6803/858330493", 2024-09-13T10:12:15.225 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6802/858330493", 2024-09-13T10:12:15.225 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:12:15.225 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:12:15.225 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:12:15.225 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.226 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-13T10:12:15.226 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "3667ec10-561e-4a23-8197-6357db4af315", 2024-09-13T10:12:15.226 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:12:15.226 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:12:15.226 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:12:15.226 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:12:15.226 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:12:15.226 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:12:15.226 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:12:15.226 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 84, 2024-09-13T10:12:15.226 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:12:15.227 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:12:15.227 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:12:15.227 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.227 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.227 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6804", 2024-09-13T10:12:15.227 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:12:15.227 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.227 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.227 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:12:15.227 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.228 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.228 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6805", 2024-09-13T10:12:15.228 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:12:15.228 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.228 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.228 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:12:15.228 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.228 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.228 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6807", 2024-09-13T10:12:15.228 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:12:15.228 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.229 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.229 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:12:15.229 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.229 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.229 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6806", 2024-09-13T10:12:15.229 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 984313481 2024-09-13T10:12:15.229 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.229 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.229 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6804/984313481", 2024-09-13T10:12:15.230 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6805/984313481", 2024-09-13T10:12:15.230 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6807/984313481", 2024-09-13T10:12:15.230 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6806/984313481", 2024-09-13T10:12:15.230 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:12:15.230 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:12:15.230 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:12:15.230 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.230 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-13T10:12:15.230 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "30d4d63c-f04f-4423-91a6-816a7cbc0287", 2024-09-13T10:12:15.230 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:12:15.231 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:12:15.231 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:12:15.231 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:12:15.231 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:12:15.231 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:12:15.231 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:12:15.231 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 79, 2024-09-13T10:12:15.231 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:12:15.231 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:12:15.231 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:12:15.231 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.232 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.232 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6812", 2024-09-13T10:12:15.232 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:12:15.232 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.232 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.232 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:12:15.232 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.232 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.232 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6813", 2024-09-13T10:12:15.232 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:12:15.232 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.233 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.233 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:12:15.233 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.233 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.233 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6815", 2024-09-13T10:12:15.233 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:12:15.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.233 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.234 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:12:15.234 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.234 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.234 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6814", 2024-09-13T10:12:15.234 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1505939978 2024-09-13T10:12:15.234 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.234 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.235 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6812/1505939978", 2024-09-13T10:12:15.235 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6813/1505939978", 2024-09-13T10:12:15.235 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6815/1505939978", 2024-09-13T10:12:15.235 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6814/1505939978", 2024-09-13T10:12:15.235 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:12:15.235 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:12:15.235 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:12:15.235 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.235 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-13T10:12:15.236 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "48812f05-82f1-4263-8ec5-4861a6e91b61", 2024-09-13T10:12:15.236 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:12:15.236 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:12:15.236 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:12:15.236 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0, 2024-09-13T10:12:15.236 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:12:15.236 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:12:15.236 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:12:15.236 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 41, 2024-09-13T10:12:15.236 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:12:15.236 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:12:15.236 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:12:15.237 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.237 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.237 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6809", 2024-09-13T10:12:15.237 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:12:15.237 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.237 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.237 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:12:15.237 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.237 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.238 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6810", 2024-09-13T10:12:15.238 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:12:15.238 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.238 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.238 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:12:15.238 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.238 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.238 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6812", 2024-09-13T10:12:15.238 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:12:15.238 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.239 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.239 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:12:15.239 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.239 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.239 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6811", 2024-09-13T10:12:15.239 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3679292757 2024-09-13T10:12:15.239 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.239 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.239 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6809/3679292757", 2024-09-13T10:12:15.239 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6810/3679292757", 2024-09-13T10:12:15.240 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6812/3679292757", 2024-09-13T10:12:15.240 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6811/3679292757", 2024-09-13T10:12:15.240 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:12:15.240 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:12:15.240 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:12:15.240 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.240 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 8, 2024-09-13T10:12:15.240 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "72056e61-d04b-4fd3-94bf-57ee3df4b466", 2024-09-13T10:12:15.240 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:12:15.240 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:12:15.241 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:12:15.241 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0.396240234375, 2024-09-13T10:12:15.241 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:12:15.241 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:12:15.241 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:12:15.241 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 31, 2024-09-13T10:12:15.241 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:12:15.241 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:12:15.241 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:12:15.241 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.241 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.242 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6804", 2024-09-13T10:12:15.242 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:12:15.242 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.242 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.242 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:12:15.242 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.242 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.242 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6805", 2024-09-13T10:12:15.242 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:12:15.242 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.243 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.243 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:12:15.243 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.243 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.243 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6807", 2024-09-13T10:12:15.243 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:12:15.243 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.243 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.243 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:12:15.244 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.244 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.244 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6806", 2024-09-13T10:12:15.244 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4052106885 2024-09-13T10:12:15.244 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.244 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.244 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6804/4052106885", 2024-09-13T10:12:15.244 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6805/4052106885", 2024-09-13T10:12:15.244 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6807/4052106885", 2024-09-13T10:12:15.245 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6806/4052106885", 2024-09-13T10:12:15.245 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:12:15.245 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:12:15.245 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:12:15.245 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.245 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 9, 2024-09-13T10:12:15.245 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "20e7f118-7148-4c27-9ec4-a33954709515", 2024-09-13T10:12:15.245 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:12:15.245 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:12:15.245 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:12:15.246 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:12:15.246 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:12:15.246 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:12:15.246 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 21, 2024-09-13T10:12:15.246 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 77, 2024-09-13T10:12:15.246 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:12:15.246 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:12:15.246 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:12:15.246 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.246 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.246 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6800", 2024-09-13T10:12:15.247 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:12:15.247 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.247 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.247 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:12:15.247 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.247 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.247 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6801", 2024-09-13T10:12:15.247 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:12:15.247 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.247 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.248 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:12:15.248 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.248 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.248 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6803", 2024-09-13T10:12:15.248 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:12:15.248 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.248 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.248 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:12:15.248 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.249 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.249 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6802", 2024-09-13T10:12:15.249 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4266571820 2024-09-13T10:12:15.249 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.249 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.249 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6800/4266571820", 2024-09-13T10:12:15.249 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6801/4266571820", 2024-09-13T10:12:15.249 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6803/4266571820", 2024-09-13T10:12:15.249 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6802/4266571820", 2024-09-13T10:12:15.249 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:12:15.250 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:12:15.250 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:12:15.250 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.250 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 10, 2024-09-13T10:12:15.250 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "012409e8-51c2-4584-aba7-95996f0d72ad", 2024-09-13T10:12:15.250 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:12:15.250 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:12:15.250 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:12:15.250 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:12:15.250 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:12:15.250 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:12:15.251 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:12:15.251 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 80, 2024-09-13T10:12:15.251 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:12:15.251 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:12:15.251 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:12:15.251 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.251 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.251 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6808", 2024-09-13T10:12:15.251 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:12:15.251 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.251 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.251 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:12:15.252 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.252 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.252 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6809", 2024-09-13T10:12:15.252 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:12:15.252 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.252 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.252 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:12:15.252 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.252 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.253 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6811", 2024-09-13T10:12:15.253 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:12:15.253 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.253 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.253 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:12:15.253 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.253 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.253 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6810", 2024-09-13T10:12:15.253 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 958255259 2024-09-13T10:12:15.253 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.254 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.254 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6808/958255259", 2024-09-13T10:12:15.254 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6809/958255259", 2024-09-13T10:12:15.254 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6811/958255259", 2024-09-13T10:12:15.254 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6810/958255259", 2024-09-13T10:12:15.254 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:12:15.254 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:12:15.254 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:12:15.254 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.254 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 11, 2024-09-13T10:12:15.254 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "88b137b1-7d30-45f7-ac5c-ba026704ef24", 2024-09-13T10:12:15.255 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:12:15.255 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:12:15.255 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:12:15.255 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:12:15.255 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:12:15.255 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:12:15.255 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:12:15.255 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 104, 2024-09-13T10:12:15.255 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:12:15.255 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:12:15.255 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:12:15.255 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.256 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.256 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6813", 2024-09-13T10:12:15.256 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:12:15.256 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.256 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.256 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:12:15.256 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.256 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.256 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6814", 2024-09-13T10:12:15.256 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:12:15.256 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.257 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.257 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:12:15.257 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.257 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.257 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6816", 2024-09-13T10:12:15.257 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:12:15.257 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.257 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.257 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:12:15.257 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.258 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.258 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6815", 2024-09-13T10:12:15.258 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 281884267 2024-09-13T10:12:15.258 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.258 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.258 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6813/281884267", 2024-09-13T10:12:15.258 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6814/281884267", 2024-09-13T10:12:15.258 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6816/281884267", 2024-09-13T10:12:15.258 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6815/281884267", 2024-09-13T10:12:15.258 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:12:15.258 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:12:15.259 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:12:15.259 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.259 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 12, 2024-09-13T10:12:15.259 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "b54732a2-9dff-42e1-b4ef-281cbca3ad7c", 2024-09-13T10:12:15.259 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:12:15.259 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:12:15.259 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:12:15.259 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 0, 2024-09-13T10:12:15.259 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:12:15.259 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:12:15.259 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:12:15.259 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 31, 2024-09-13T10:12:15.260 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:12:15.260 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:12:15.260 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:12:15.260 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.260 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.260 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6808", 2024-09-13T10:12:15.260 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:12:15.260 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.260 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.260 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:12:15.260 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.261 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.261 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6809", 2024-09-13T10:12:15.261 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:12:15.261 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.261 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.261 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:12:15.261 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.261 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.261 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6811", 2024-09-13T10:12:15.262 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:12:15.262 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.262 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.262 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:12:15.262 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.262 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.262 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6810", 2024-09-13T10:12:15.262 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3532043806 2024-09-13T10:12:15.262 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.262 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.263 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.73:6808/3532043806", 2024-09-13T10:12:15.263 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.73:6809/3532043806", 2024-09-13T10:12:15.263 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.73:6811/3532043806", 2024-09-13T10:12:15.263 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.73:6810/3532043806", 2024-09-13T10:12:15.263 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:12:15.263 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:12:15.263 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:12:15.263 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.263 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 13, 2024-09-13T10:12:15.263 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "20b48864-44dc-4732-8dff-d0d21cc08032", 2024-09-13T10:12:15.263 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:12:15.264 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:12:15.264 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0.95001220703125, 2024-09-13T10:12:15.264 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:12:15.264 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:12:15.264 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:12:15.264 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 21, 2024-09-13T10:12:15.264 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 100, 2024-09-13T10:12:15.264 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:12:15.264 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:12:15.264 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:12:15.264 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.265 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.265 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6812", 2024-09-13T10:12:15.265 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:12:15.265 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.265 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.265 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:12:15.265 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.265 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.265 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6813", 2024-09-13T10:12:15.265 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:12:15.265 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.266 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.266 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:12:15.266 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.266 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.266 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6815", 2024-09-13T10:12:15.266 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:12:15.266 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.266 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.267 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:12:15.267 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.267 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.267 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.136:6814", 2024-09-13T10:12:15.267 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4083982131 2024-09-13T10:12:15.267 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.267 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.267 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.136:6812/4083982131", 2024-09-13T10:12:15.267 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.136:6813/4083982131", 2024-09-13T10:12:15.267 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.136:6815/4083982131", 2024-09-13T10:12:15.267 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.136:6814/4083982131", 2024-09-13T10:12:15.267 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:12:15.268 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:12:15.268 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:12:15.268 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.268 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 14, 2024-09-13T10:12:15.268 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "9fac3ef4-e9be-4e8b-9a21-0de35b543d65", 2024-09-13T10:12:15.268 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:12:15.268 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:12:15.268 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 0.95001220703125, 2024-09-13T10:12:15.268 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:12:15.268 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:12:15.268 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:12:15.269 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 23, 2024-09-13T10:12:15.269 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 98, 2024-09-13T10:12:15.269 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:12:15.269 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:12:15.269 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:12:15.269 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.269 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.269 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6804", 2024-09-13T10:12:15.269 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:12:15.269 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.269 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.270 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:12:15.270 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.270 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.270 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6805", 2024-09-13T10:12:15.270 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:12:15.270 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.270 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.270 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:12:15.270 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.270 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.271 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6807", 2024-09-13T10:12:15.271 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:12:15.271 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.271 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.271 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:12:15.271 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.271 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.271 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.181:6806", 2024-09-13T10:12:15.271 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1173379175 2024-09-13T10:12:15.271 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.271 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.272 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.181:6804/1173379175", 2024-09-13T10:12:15.272 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.181:6805/1173379175", 2024-09-13T10:12:15.272 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.181:6807/1173379175", 2024-09-13T10:12:15.272 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.181:6806/1173379175", 2024-09-13T10:12:15.272 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:12:15.272 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:12:15.272 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:12:15.272 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.272 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 15, 2024-09-13T10:12:15.272 INFO:teuthology.orchestra.run.smithi073.stdout: "uuid": "16b34f90-3148-4d41-a403-ba0f900991e1", 2024-09-13T10:12:15.273 INFO:teuthology.orchestra.run.smithi073.stdout: "up": 1, 2024-09-13T10:12:15.273 INFO:teuthology.orchestra.run.smithi073.stdout: "in": 1, 2024-09-13T10:12:15.273 INFO:teuthology.orchestra.run.smithi073.stdout: "weight": 1, 2024-09-13T10:12:15.273 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_affinity": 1, 2024-09-13T10:12:15.273 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_begin": 0, 2024-09-13T10:12:15.273 INFO:teuthology.orchestra.run.smithi073.stdout: "last_clean_end": 0, 2024-09-13T10:12:15.273 INFO:teuthology.orchestra.run.smithi073.stdout: "up_from": 22, 2024-09-13T10:12:15.273 INFO:teuthology.orchestra.run.smithi073.stdout: "up_thru": 76, 2024-09-13T10:12:15.273 INFO:teuthology.orchestra.run.smithi073.stdout: "down_at": 0, 2024-09-13T10:12:15.273 INFO:teuthology.orchestra.run.smithi073.stdout: "lost_at": 0, 2024-09-13T10:12:15.273 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrs": { 2024-09-13T10:12:15.273 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.274 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.274 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6805", 2024-09-13T10:12:15.274 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:12:15.274 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.274 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.274 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addrs": { 2024-09-13T10:12:15.274 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.274 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.274 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6806", 2024-09-13T10:12:15.274 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:12:15.274 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.275 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.275 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addrs": { 2024-09-13T10:12:15.275 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.275 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.275 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6808", 2024-09-13T10:12:15.275 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:12:15.275 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.275 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.275 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addrs": { 2024-09-13T10:12:15.275 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-09-13T10:12:15.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.276 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-09-13T10:12:15.276 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.195:6807", 2024-09-13T10:12:15.276 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 848686366 2024-09-13T10:12:15.276 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.276 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.276 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "172.21.15.195:6805/848686366", 2024-09-13T10:12:15.276 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "172.21.15.195:6806/848686366", 2024-09-13T10:12:15.276 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_back_addr": "172.21.15.195:6808/848686366", 2024-09-13T10:12:15.276 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_front_addr": "172.21.15.195:6807/848686366", 2024-09-13T10:12:15.276 INFO:teuthology.orchestra.run.smithi073.stdout: "state": [ 2024-09-13T10:12:15.276 INFO:teuthology.orchestra.run.smithi073.stdout: "exists", 2024-09-13T10:12:15.276 INFO:teuthology.orchestra.run.smithi073.stdout: "up" 2024-09-13T10:12:15.277 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.277 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.277 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:12:15.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_xinfo": [ 2024-09-13T10:12:15.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.277 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 0, 2024-09-13T10:12:15.277 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:12:15.277 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:12:15.277 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:12:15.277 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:12:15.277 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:12:15.277 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:52.724148+0000", 2024-09-13T10:12:15.277 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:12:15.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.278 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 1, 2024-09-13T10:12:15.278 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:12:15.278 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:12:15.278 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:12:15.278 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:12:15.278 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:12:15.278 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:45.986851+0000", 2024-09-13T10:12:15.278 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:12:15.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.278 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 2, 2024-09-13T10:12:15.278 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:12:15.279 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:12:15.279 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:12:15.279 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:12:15.279 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:12:15.279 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:47.777103+0000", 2024-09-13T10:12:15.279 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:12:15.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.279 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 3, 2024-09-13T10:12:15.279 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:12:15.279 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:12:15.279 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:12:15.279 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:12:15.280 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:12:15.280 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:47.179172+0000", 2024-09-13T10:12:15.280 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:12:15.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.280 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 4, 2024-09-13T10:12:15.280 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:12:15.280 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:12:15.280 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:12:15.280 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:12:15.280 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:12:15.280 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:54.012724+0000", 2024-09-13T10:12:15.280 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:12:15.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 5, 2024-09-13T10:12:15.281 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:12:15.281 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:12:15.281 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:12:15.281 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:12:15.281 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:12:15.281 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:46.147756+0000", 2024-09-13T10:12:15.281 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:12:15.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 6, 2024-09-13T10:12:15.282 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:12:15.282 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:12:15.282 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:12:15.282 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:12:15.282 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:12:15.282 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:47.809723+0000", 2024-09-13T10:12:15.282 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:12:15.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.282 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 7, 2024-09-13T10:12:15.282 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:12:15.282 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:12:15.282 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:12:15.283 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:12:15.283 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:12:15.283 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:38.606440+0000", 2024-09-13T10:12:15.283 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:12:15.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.283 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 8, 2024-09-13T10:12:15.283 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:12:15.283 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:12:15.283 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:12:15.283 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:12:15.283 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:12:15.283 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:37.606540+0000", 2024-09-13T10:12:15.284 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:12:15.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.284 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 9, 2024-09-13T10:12:15.284 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:12:15.284 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:12:15.284 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:12:15.284 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:12:15.284 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:12:15.284 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:25.726157+0000", 2024-09-13T10:12:15.284 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:12:15.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 10, 2024-09-13T10:12:15.285 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:12:15.285 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:12:15.285 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:12:15.285 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:12:15.285 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:12:15.285 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:27.329800+0000", 2024-09-13T10:12:15.285 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:12:15.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.285 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 11, 2024-09-13T10:12:15.285 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:12:15.286 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:12:15.286 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:12:15.286 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:12:15.286 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:12:15.286 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:26.722524+0000", 2024-09-13T10:12:15.286 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:12:15.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.286 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 12, 2024-09-13T10:12:15.286 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:12:15.286 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:12:15.286 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:12:15.286 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:12:15.286 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:12:15.287 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:25.954484+0000", 2024-09-13T10:12:15.287 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:12:15.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.287 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 13, 2024-09-13T10:12:15.287 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:12:15.287 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:12:15.287 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:12:15.287 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:12:15.287 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:12:15.287 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:25.900950+0000", 2024-09-13T10:12:15.287 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:12:15.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 14, 2024-09-13T10:12:15.288 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:12:15.288 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:12:15.288 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:12:15.288 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:12:15.288 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:12:15.288 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:40.864138+0000", 2024-09-13T10:12:15.288 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:12:15.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.288 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": 15, 2024-09-13T10:12:15.288 INFO:teuthology.orchestra.run.smithi073.stdout: "down_stamp": "0.000000", 2024-09-13T10:12:15.288 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_probability": 0, 2024-09-13T10:12:15.289 INFO:teuthology.orchestra.run.smithi073.stdout: "laggy_interval": 0, 2024-09-13T10:12:15.289 INFO:teuthology.orchestra.run.smithi073.stdout: "features": 4540719139924082687, 2024-09-13T10:12:15.289 INFO:teuthology.orchestra.run.smithi073.stdout: "old_weight": 0, 2024-09-13T10:12:15.289 INFO:teuthology.orchestra.run.smithi073.stdout: "last_purged_snaps_scrub": "2024-09-13T10:11:48.003986+0000", 2024-09-13T10:12:15.289 INFO:teuthology.orchestra.run.smithi073.stdout: "dead_epoch": 0 2024-09-13T10:12:15.289 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.289 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:12:15.289 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap": [ 2024-09-13T10:12:15.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.289 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.e", 2024-09-13T10:12:15.289 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:12:15.289 INFO:teuthology.orchestra.run.smithi073.stdout: 8, 2024-09-13T10:12:15.289 INFO:teuthology.orchestra.run.smithi073.stdout: 3, 2024-09-13T10:12:15.290 INFO:teuthology.orchestra.run.smithi073.stdout: 15 2024-09-13T10:12:15.290 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.290 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.12", 2024-09-13T10:12:15.290 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:12:15.290 INFO:teuthology.orchestra.run.smithi073.stdout: 1, 2024-09-13T10:12:15.290 INFO:teuthology.orchestra.run.smithi073.stdout: 4, 2024-09-13T10:12:15.290 INFO:teuthology.orchestra.run.smithi073.stdout: 13 2024-09-13T10:12:15.290 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.290 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.290 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:12:15.290 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_items": [], 2024-09-13T10:12:15.291 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_upmap_primaries": [], 2024-09-13T10:12:15.291 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_temp": [ 2024-09-13T10:12:15.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.291 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.6", 2024-09-13T10:12:15.291 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:12:15.291 INFO:teuthology.orchestra.run.smithi073.stdout: 1, 2024-09-13T10:12:15.291 INFO:teuthology.orchestra.run.smithi073.stdout: 7, 2024-09-13T10:12:15.291 INFO:teuthology.orchestra.run.smithi073.stdout: 13 2024-09-13T10:12:15.291 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.291 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.e", 2024-09-13T10:12:15.292 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:12:15.292 INFO:teuthology.orchestra.run.smithi073.stdout: 2, 2024-09-13T10:12:15.292 INFO:teuthology.orchestra.run.smithi073.stdout: 8, 2024-09-13T10:12:15.292 INFO:teuthology.orchestra.run.smithi073.stdout: 4 2024-09-13T10:12:15.292 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.292 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.16", 2024-09-13T10:12:15.292 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:12:15.292 INFO:teuthology.orchestra.run.smithi073.stdout: 14, 2024-09-13T10:12:15.292 INFO:teuthology.orchestra.run.smithi073.stdout: 5, 2024-09-13T10:12:15.292 INFO:teuthology.orchestra.run.smithi073.stdout: 13 2024-09-13T10:12:15.293 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.293 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.18", 2024-09-13T10:12:15.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:12:15.293 INFO:teuthology.orchestra.run.smithi073.stdout: 8, 2024-09-13T10:12:15.293 INFO:teuthology.orchestra.run.smithi073.stdout: 10, 2024-09-13T10:12:15.293 INFO:teuthology.orchestra.run.smithi073.stdout: 13 2024-09-13T10:12:15.293 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:15.293 INFO:teuthology.orchestra.run.smithi073.stdout: "pgid": "4.19", 2024-09-13T10:12:15.293 INFO:teuthology.orchestra.run.smithi073.stdout: "osds": [ 2024-09-13T10:12:15.294 INFO:teuthology.orchestra.run.smithi073.stdout: 2, 2024-09-13T10:12:15.294 INFO:teuthology.orchestra.run.smithi073.stdout: 5, 2024-09-13T10:12:15.294 INFO:teuthology.orchestra.run.smithi073.stdout: 3 2024-09-13T10:12:15.294 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:15.294 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.294 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-13T10:12:15.294 INFO:teuthology.orchestra.run.smithi073.stdout: "primary_temp": [], 2024-09-13T10:12:15.294 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklist": {}, 2024-09-13T10:12:15.294 INFO:teuthology.orchestra.run.smithi073.stdout: "range_blocklist": {}, 2024-09-13T10:12:15.294 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_profiles": { 2024-09-13T10:12:15.294 INFO:teuthology.orchestra.run.smithi073.stdout: "default": { 2024-09-13T10:12:15.294 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-failure-domain": "osd", 2024-09-13T10:12:15.294 INFO:teuthology.orchestra.run.smithi073.stdout: "k": "2", 2024-09-13T10:12:15.295 INFO:teuthology.orchestra.run.smithi073.stdout: "m": "1", 2024-09-13T10:12:15.295 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin": "jerasure", 2024-09-13T10:12:15.295 INFO:teuthology.orchestra.run.smithi073.stdout: "technique": "reed_sol_van" 2024-09-13T10:12:15.295 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:15.295 INFO:teuthology.orchestra.run.smithi073.stdout: "removed_snaps_queue": [], 2024-09-13T10:12:15.295 INFO:teuthology.orchestra.run.smithi073.stdout: "new_removed_snaps": [], 2024-09-13T10:12:15.295 INFO:teuthology.orchestra.run.smithi073.stdout: "new_purged_snaps": [], 2024-09-13T10:12:15.295 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_node_flags": {}, 2024-09-13T10:12:15.295 INFO:teuthology.orchestra.run.smithi073.stdout: "device_class_flags": {}, 2024-09-13T10:12:15.295 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode": { 2024-09-13T10:12:15.295 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_enabled": false, 2024-09-13T10:12:15.295 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_bucket_count": 0, 2024-09-13T10:12:15.295 INFO:teuthology.orchestra.run.smithi073.stdout: "degraded_stretch_mode": 0, 2024-09-13T10:12:15.295 INFO:teuthology.orchestra.run.smithi073.stdout: "recovering_stretch_mode": 0, 2024-09-13T10:12:15.296 INFO:teuthology.orchestra.run.smithi073.stdout: "stretch_mode_bucket": 0 2024-09-13T10:12:15.296 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:15.296 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:15.297 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1945357 2024-09-13T10:12:15.298 INFO:tasks.rados.rados.0.smithi195.stdout:1877: finishing write tid 2 to smithi19530714-40 2024-09-13T10:12:15.298 INFO:tasks.rados.rados.0.smithi195.stdout:1877: finishing write tid 3 to smithi19530714-40 2024-09-13T10:12:15.298 INFO:tasks.rados.rados.0.smithi195.stdout:1877: finishing write tid 4 to smithi19530714-40 2024-09-13T10:12:15.298 INFO:tasks.rados.rados.0.smithi195.stdout:1877: oid 40 updating version 0 to 35 2024-09-13T10:12:15.298 INFO:tasks.rados.rados.0.smithi195.stdout:1877: oid 40 version 35 is already newer than 34 2024-09-13T10:12:15.298 INFO:tasks.rados.rados.0.smithi195.stdout:1877: oid 40 version 35 is already newer than 33 2024-09-13T10:12:15.298 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 35 (ObjNum 414 snap 0 seq_num 414) dirty exists 2024-09-13T10:12:15.298 INFO:tasks.rados.rados.0.smithi195.stdout:1877: left oid 40 (ObjNum 414 snap 0 seq_num 414) 2024-09-13T10:12:15.298 INFO:tasks.rados.rados.0.smithi195.stdout:1879: expect (ObjNum 398 snap 0 seq_num 398) 2024-09-13T10:12:15.298 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:15.299 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:15.293+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:15.299 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:15.300 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:12:15.300 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:12:15.300 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:15.301 INFO:tasks.thrashosds.thrasher:j is {'epoch': 107, 'fsid': 'c1e91d47-9492-4461-a043-8698fd74ccf3', 'created': '2024-09-13T10:10:14.019912+0000', 'modified': '2024-09-13T10:12:14.713308+0000', 'last_up_change': '2024-09-13T10:10:28.089885+0000', 'last_in_change': '2024-09-13T10:10:18.489546+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 16, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T10:10:28.924860+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 27, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5.230000019073486, 'score_stable': 5.230000019073486, 'optimal_score': 0.7900000214576721, 'raw_score_acting': 4.130000114440918, 'raw_score_stable': 4.130000114440918, 'primary_affinity_weighted': 0.8372650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8399999737739563}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T10:10:28.972519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 14.289999961853027, 'score_stable': 14.289999961853027, 'optimal_score': 0.20999999344348907, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.8372650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8399999737739563}}, {'pool': 3, 'pool_name': 'low_tier', 'create_time': '2024-09-13T10:10:41.320719+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.5199999809265137, 'score_stable': 3.5199999809265137, 'optimal_score': 0.6399999856948853, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.8372650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8399999737739563}}, {'pool': 4, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-13T10:10:42.575395+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 24, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '41', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'dedup_cdc_chunk_size': 131072, 'dedup_chunk_algorithm': 'fastcdc', 'dedup_tier': 3, 'fingerprint_algorithm': 'sha256'}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.240000009536743, 'score_stable': 3.240000009536743, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 3.690000057220459, 'raw_score_stable': 3.690000057220459, 'primary_affinity_weighted': 0.8372650146484375, 'average_primary_affinity': 0.9599999785423279, 'average_primary_affinity_weighted': 0.8399999737739563}}], 'osds': [{'osd': 0, 'uuid': 'f7d92188-c27c-4744-a330-4b7105e0f2d6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 79, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6812', 'nonce': 1977392321}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6813', 'nonce': 1977392321}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6815', 'nonce': 1977392321}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6814', 'nonce': 1977392321}]}, 'public_addr': '172.21.15.73:6812/1977392321', 'cluster_addr': '172.21.15.73:6813/1977392321', 'heartbeat_back_addr': '172.21.15.73:6815/1977392321', 'heartbeat_front_addr': '172.21.15.73:6814/1977392321', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'e900b771-7c96-4b5d-93d7-35a8ea6c026a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 98, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6808', 'nonce': 1228102864}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6809', 'nonce': 1228102864}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6811', 'nonce': 1228102864}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6810', 'nonce': 1228102864}]}, 'public_addr': '172.21.15.136:6808/1228102864', 'cluster_addr': '172.21.15.136:6809/1228102864', 'heartbeat_back_addr': '172.21.15.136:6811/1228102864', 'heartbeat_front_addr': '172.21.15.136:6810/1228102864', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '28b7aa59-33b2-4e24-8f3d-758f90e25f68', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 105, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6800', 'nonce': 3314820936}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6801', 'nonce': 3314820936}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6803', 'nonce': 3314820936}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6802', 'nonce': 3314820936}]}, 'public_addr': '172.21.15.181:6800/3314820936', 'cluster_addr': '172.21.15.181:6801/3314820936', 'heartbeat_back_addr': '172.21.15.181:6803/3314820936', 'heartbeat_front_addr': '172.21.15.181:6802/3314820936', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'a93d2df5-781e-49e1-a507-1718b6f954ed', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 106, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6801', 'nonce': 2558565460}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6802', 'nonce': 2558565460}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6804', 'nonce': 2558565460}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6803', 'nonce': 2558565460}]}, 'public_addr': '172.21.15.195:6801/2558565460', 'cluster_addr': '172.21.15.195:6802/2558565460', 'heartbeat_back_addr': '172.21.15.195:6804/2558565460', 'heartbeat_front_addr': '172.21.15.195:6803/2558565460', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'e69053fd-f6d9-437e-a9d0-8896de6b927f', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6800', 'nonce': 858330493}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6801', 'nonce': 858330493}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6803', 'nonce': 858330493}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6802', 'nonce': 858330493}]}, 'public_addr': '172.21.15.73:6800/858330493', 'cluster_addr': '172.21.15.73:6801/858330493', 'heartbeat_back_addr': '172.21.15.73:6803/858330493', 'heartbeat_front_addr': '172.21.15.73:6802/858330493', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '3667ec10-561e-4a23-8197-6357db4af315', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 84, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6804', 'nonce': 984313481}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6805', 'nonce': 984313481}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6807', 'nonce': 984313481}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6806', 'nonce': 984313481}]}, 'public_addr': '172.21.15.136:6804/984313481', 'cluster_addr': '172.21.15.136:6805/984313481', 'heartbeat_back_addr': '172.21.15.136:6807/984313481', 'heartbeat_front_addr': '172.21.15.136:6806/984313481', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '30d4d63c-f04f-4423-91a6-816a7cbc0287', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 79, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6812', 'nonce': 1505939978}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6813', 'nonce': 1505939978}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6815', 'nonce': 1505939978}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6814', 'nonce': 1505939978}]}, 'public_addr': '172.21.15.181:6812/1505939978', 'cluster_addr': '172.21.15.181:6813/1505939978', 'heartbeat_back_addr': '172.21.15.181:6815/1505939978', 'heartbeat_front_addr': '172.21.15.181:6814/1505939978', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '48812f05-82f1-4263-8ec5-4861a6e91b61', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6809', 'nonce': 3679292757}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6810', 'nonce': 3679292757}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6812', 'nonce': 3679292757}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6811', 'nonce': 3679292757}]}, 'public_addr': '172.21.15.195:6809/3679292757', 'cluster_addr': '172.21.15.195:6810/3679292757', 'heartbeat_back_addr': '172.21.15.195:6812/3679292757', 'heartbeat_front_addr': '172.21.15.195:6811/3679292757', 'state': ['exists', 'up']}, {'osd': 8, 'uuid': '72056e61-d04b-4fd3-94bf-57ee3df4b466', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.396240234375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6804', 'nonce': 4052106885}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6805', 'nonce': 4052106885}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6807', 'nonce': 4052106885}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6806', 'nonce': 4052106885}]}, 'public_addr': '172.21.15.73:6804/4052106885', 'cluster_addr': '172.21.15.73:6805/4052106885', 'heartbeat_back_addr': '172.21.15.73:6807/4052106885', 'heartbeat_front_addr': '172.21.15.73:6806/4052106885', 'state': ['exists', 'up']}, {'osd': 9, 'uuid': '20e7f118-7148-4c27-9ec4-a33954709515', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6800', 'nonce': 4266571820}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6801', 'nonce': 4266571820}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6803', 'nonce': 4266571820}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6802', 'nonce': 4266571820}]}, 'public_addr': '172.21.15.136:6800/4266571820', 'cluster_addr': '172.21.15.136:6801/4266571820', 'heartbeat_back_addr': '172.21.15.136:6803/4266571820', 'heartbeat_front_addr': '172.21.15.136:6802/4266571820', 'state': ['exists', 'up']}, {'osd': 10, 'uuid': '012409e8-51c2-4584-aba7-95996f0d72ad', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6808', 'nonce': 958255259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6809', 'nonce': 958255259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6811', 'nonce': 958255259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6810', 'nonce': 958255259}]}, 'public_addr': '172.21.15.181:6808/958255259', 'cluster_addr': '172.21.15.181:6809/958255259', 'heartbeat_back_addr': '172.21.15.181:6811/958255259', 'heartbeat_front_addr': '172.21.15.181:6810/958255259', 'state': ['exists', 'up']}, {'osd': 11, 'uuid': '88b137b1-7d30-45f7-ac5c-ba026704ef24', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 104, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6813', 'nonce': 281884267}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6814', 'nonce': 281884267}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6816', 'nonce': 281884267}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6815', 'nonce': 281884267}]}, 'public_addr': '172.21.15.195:6813/281884267', 'cluster_addr': '172.21.15.195:6814/281884267', 'heartbeat_back_addr': '172.21.15.195:6816/281884267', 'heartbeat_front_addr': '172.21.15.195:6815/281884267', 'state': ['exists', 'up']}, {'osd': 12, 'uuid': 'b54732a2-9dff-42e1-b4ef-281cbca3ad7c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 31, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6808', 'nonce': 3532043806}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6809', 'nonce': 3532043806}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6811', 'nonce': 3532043806}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6810', 'nonce': 3532043806}]}, 'public_addr': '172.21.15.73:6808/3532043806', 'cluster_addr': '172.21.15.73:6809/3532043806', 'heartbeat_back_addr': '172.21.15.73:6811/3532043806', 'heartbeat_front_addr': '172.21.15.73:6810/3532043806', 'state': ['exists', 'up']}, {'osd': 13, 'uuid': '20b48864-44dc-4732-8dff-d0d21cc08032', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 21, 'up_thru': 100, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6812', 'nonce': 4083982131}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6813', 'nonce': 4083982131}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6815', 'nonce': 4083982131}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6814', 'nonce': 4083982131}]}, 'public_addr': '172.21.15.136:6812/4083982131', 'cluster_addr': '172.21.15.136:6813/4083982131', 'heartbeat_back_addr': '172.21.15.136:6815/4083982131', 'heartbeat_front_addr': '172.21.15.136:6814/4083982131', 'state': ['exists', 'up']}, {'osd': 14, 'uuid': '9fac3ef4-e9be-4e8b-9a21-0de35b543d65', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 23, 'up_thru': 98, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6804', 'nonce': 1173379175}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6805', 'nonce': 1173379175}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6807', 'nonce': 1173379175}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.181:6806', 'nonce': 1173379175}]}, 'public_addr': '172.21.15.181:6804/1173379175', 'cluster_addr': '172.21.15.181:6805/1173379175', 'heartbeat_back_addr': '172.21.15.181:6807/1173379175', 'heartbeat_front_addr': '172.21.15.181:6806/1173379175', 'state': ['exists', 'up']}, {'osd': 15, 'uuid': '16b34f90-3148-4d41-a403-ba0f900991e1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 22, 'up_thru': 76, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6805', 'nonce': 848686366}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6806', 'nonce': 848686366}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6808', 'nonce': 848686366}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6807', 'nonce': 848686366}]}, 'public_addr': '172.21.15.195:6805/848686366', 'cluster_addr': '172.21.15.195:6806/848686366', 'heartbeat_back_addr': '172.21.15.195:6808/848686366', 'heartbeat_front_addr': '172.21.15.195:6807/848686366', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:52.724148+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:45.986851+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:47.777103+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:47.179172+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:54.012724+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:46.147756+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:47.809723+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:38.606440+0000', 'dead_epoch': 0}, {'osd': 8, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:37.606540+0000', 'dead_epoch': 0}, {'osd': 9, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:25.726157+0000', 'dead_epoch': 0}, {'osd': 10, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:27.329800+0000', 'dead_epoch': 0}, {'osd': 11, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:26.722524+0000', 'dead_epoch': 0}, {'osd': 12, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:25.954484+0000', 'dead_epoch': 0}, {'osd': 13, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:25.900950+0000', 'dead_epoch': 0}, {'osd': 14, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:40.864138+0000', 'dead_epoch': 0}, {'osd': 15, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T10:11:48.003986+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '4.e', 'osds': [8, 3, 15]}, {'pgid': '4.12', 'osds': [1, 4, 13]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '4.6', 'osds': [1, 7, 13]}, {'pgid': '4.e', 'osds': [2, 8, 4]}, {'pgid': '4.16', 'osds': [14, 5, 13]}, {'pgid': '4.18', 'osds': [8, 10, 13]}, {'pgid': '4.19', 'osds': [2, 5, 3]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T10:12:15.301 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-13T10:12:15.301 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T10:12:15.314 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2055400 2024-09-13T10:12:15.325 INFO:tasks.rados.rados.0.smithi195.stdout:1881: seq_num 415 ranges {574905=711397,1885086=525140,2930189=44716} 2024-09-13T10:12:15.346 INFO:tasks.rados.rados.0.smithi195.stdout:1881: writing smithi19530714-32 from 574905 to 1286302 tid 1 2024-09-13T10:12:15.364 INFO:tasks.rados.rados.0.smithi195.stdout:1881: writing smithi19530714-32 from 1885086 to 2410226 tid 2 2024-09-13T10:12:15.374 INFO:tasks.rados.rados.0.smithi195.stdout:1881: writing smithi19530714-32 from 2930189 to 2974905 tid 3 2024-09-13T10:12:15.374 INFO:tasks.rados.rados.0.smithi195.stdout:1871: done (11 left) 2024-09-13T10:12:15.374 INFO:tasks.rados.rados.0.smithi195.stdout:1872: done (10 left) 2024-09-13T10:12:15.374 INFO:tasks.rados.rados.0.smithi195.stdout:1873: done (9 left) 2024-09-13T10:12:15.374 INFO:tasks.rados.rados.0.smithi195.stdout:1874: done (8 left) 2024-09-13T10:12:15.374 INFO:tasks.rados.rados.0.smithi195.stdout:1875: done (7 left) 2024-09-13T10:12:15.374 INFO:tasks.rados.rados.0.smithi195.stdout:1876: done (6 left) 2024-09-13T10:12:15.375 INFO:tasks.rados.rados.0.smithi195.stdout:1877: done (5 left) 2024-09-13T10:12:15.375 INFO:tasks.rados.rados.0.smithi195.stdout:1878: done (4 left) 2024-09-13T10:12:15.375 INFO:tasks.rados.rados.0.smithi195.stdout:1879: done (3 left) 2024-09-13T10:12:15.375 INFO:tasks.rados.rados.0.smithi195.stdout:1880: done (2 left) 2024-09-13T10:12:15.375 INFO:tasks.rados.rados.0.smithi195.stdout:1882: read oid 7 snap -1 2024-09-13T10:12:15.375 INFO:tasks.rados.rados.0.smithi195.stdout:1882: expect (ObjNum 407 snap 0 seq_num 407) 2024-09-13T10:12:15.375 INFO:tasks.rados.rados.0.smithi195.stdout:1883: tier_evict oid 23 2024-09-13T10:12:15.375 INFO:tasks.rados.rados.0.smithi195.stdout:1883: tier_evict oid 23 snap -1 2024-09-13T10:12:15.375 INFO:tasks.rados.rados.0.smithi195.stdout:1881: finishing write tid 1 to smithi19530714-32 2024-09-13T10:12:15.375 INFO:tasks.rados.rados.0.smithi195.stdout:1881: finishing write tid 2 to smithi19530714-32 2024-09-13T10:12:15.375 INFO:tasks.rados.rados.0.smithi195.stdout:1884: read oid 40 snap -1 2024-09-13T10:12:15.375 INFO:tasks.rados.rados.0.smithi195.stdout:1884: expect (ObjNum 414 snap 0 seq_num 414) 2024-09-13T10:12:15.375 INFO:tasks.rados.rados.0.smithi195.stdout:1885: tier_promote oid 50 2024-09-13T10:12:15.375 INFO:tasks.rados.rados.0.smithi195.stdout:1886: read oid 46 snap -1 2024-09-13T10:12:15.376 INFO:tasks.rados.rados.0.smithi195.stdout:1886: expect (ObjNum 329 snap 0 seq_num 329) 2024-09-13T10:12:15.376 INFO:tasks.rados.rados.0.smithi195.stdout:1887: tier_promote oid 11 2024-09-13T10:12:15.376 INFO:tasks.rados.rados.0.smithi195.stdout:1888: write (excl) oid 30 current snap is 0 2024-09-13T10:12:15.376 INFO:tasks.rados.rados.0.smithi195.stdout:1888: seq_num 416 ranges {591712=786646,1943568=668798} 2024-09-13T10:12:15.394 INFO:tasks.rados.rados.0.smithi195.stdout:1888: writing smithi19530714-30 from 591712 to 1378358 tid 1 2024-09-13T10:12:15.398 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:15.393+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:15.410 INFO:tasks.rados.rados.0.smithi195.stdout:1888: writing smithi19530714-30 from 1943568 to 2612366 tid 2 2024-09-13T10:12:15.410 INFO:tasks.rados.rados.0.smithi195.stdout:1889: write oid 21 current snap is 0 2024-09-13T10:12:15.410 INFO:tasks.rados.rados.0.smithi195.stdout:1889: seq_num 417 ranges {608519=461895,1602050=528809,2585035=423484} 2024-09-13T10:12:15.421 INFO:tasks.rados.rados.0.smithi195.stdout:1889: writing smithi19530714-21 from 608519 to 1070414 tid 1 2024-09-13T10:12:15.430 INFO:tasks.rados.rados.0.smithi195.stdout:1889: writing smithi19530714-21 from 1602050 to 2130859 tid 2 2024-09-13T10:12:15.436 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:15.436 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:12:15.436 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:12:15.436 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:12:15.437 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:15.437 INFO:tasks.rados.rados.0.smithi195.stdout:1889: writing smithi19530714-21 from 2585035 to 3008519 tid 3 2024-09-13T10:12:15.437 INFO:tasks.rados.rados.0.smithi195.stdout:1890: set_chunk oid 27 target oid 2024-09-13T10:12:15.451 INFO:tasks.rados.rados.0.smithi195.stdout:1890: set_chunk oid 27 offset: 2881536 length: 497152 target oid 33d164e9d86da499ae045562e30e811751a1befeee0ef13d3b9fa919092f6c6b offset: 0 snap -1 2024-09-13T10:12:15.451 INFO:tasks.rados.rados.0.smithi195.stdout:1891: tier_evict oid 10 2024-09-13T10:12:15.451 INFO:tasks.rados.rados.0.smithi195.stdout:1883: got (22) Invalid argument 2024-09-13T10:12:15.451 INFO:tasks.rados.rados.0.smithi195.stdout:1885: got (0) Success 2024-09-13T10:12:15.451 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 50 v 47 (ObjNum 303 snap 0 seq_num 303) dirty exists 2024-09-13T10:12:15.451 INFO:tasks.rados.rados.0.smithi195.stdout:1887: got (0) Success 2024-09-13T10:12:15.451 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 96 (ObjNum 359 snap 0 seq_num 359) dirty exists 2024-09-13T10:12:15.452 INFO:tasks.rados.rados.0.smithi195.stdout:1881: finishing write tid 3 to smithi19530714-32 2024-09-13T10:12:15.452 INFO:tasks.rados.rados.0.smithi195.stdout:1881: finishing write tid 4 to smithi19530714-32 2024-09-13T10:12:15.452 INFO:tasks.rados.rados.0.smithi195.stdout:1881: finishing write tid 5 to smithi19530714-32 2024-09-13T10:12:15.452 INFO:tasks.rados.rados.0.smithi195.stdout:1881: oid 32 updating version 0 to 100 2024-09-13T10:12:15.452 INFO:tasks.rados.rados.0.smithi195.stdout:1881: oid 32 updating version 100 to 103 2024-09-13T10:12:15.452 INFO:tasks.rados.rados.0.smithi195.stdout:1881: oid 32 version 103 is already newer than 101 2024-09-13T10:12:15.452 INFO:tasks.rados.rados.0.smithi195.stdout:1881: oid 32 version 103 is already newer than 102 2024-09-13T10:12:15.452 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 103 (ObjNum 415 snap 0 seq_num 415) dirty exists 2024-09-13T10:12:15.452 INFO:tasks.rados.rados.0.smithi195.stdout:1881: left oid 32 (ObjNum 415 snap 0 seq_num 415) 2024-09-13T10:12:15.452 INFO:tasks.rados.rados.0.smithi195.stdout:1886: expect (ObjNum 329 snap 0 seq_num 329) 2024-09-13T10:12:15.453 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T10:12:15.499 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:15.493+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:15.551 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 382736 2024-09-13T10:12:15.579 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:15.579 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:12:15.579 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:12:15.580 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:12:15.580 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.580 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:259 4.1 4:85f1e656:::smithi19530714-10:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:15.580 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:12:15.580 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 84.360475949000005, 2024-09-13T10:12:15.580 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014693513, 2024-09-13T10:12:15.580 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.580 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.580 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.580 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.580 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.580 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.581 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 259 2024-09-13T10:12:15.581 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.581 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.581 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.581 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:12:15.581 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.581 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.581 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.581 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215167+0000", 2024-09-13T10:12:15.581 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.582 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.582 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215171+0000", 2024-09-13T10:12:15.582 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.3329999999999998e-06 2024-09-13T10:12:15.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.582 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.582 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215190+0000", 2024-09-13T10:12:15.582 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8672999999999999e-05 2024-09-13T10:12:15.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.582 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.583 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215229+0000", 2024-09-13T10:12:15.583 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9076000000000003e-05 2024-09-13T10:12:15.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.583 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.583 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215287+0000", 2024-09-13T10:12:15.583 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8142e-05 2024-09-13T10:12:15.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.583 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.583 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215435+0000", 2024-09-13T10:12:15.583 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014800600000000001 2024-09-13T10:12:15.584 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.584 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.584 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.584 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215593+0000", 2024-09-13T10:12:15.584 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015864300000000001 2024-09-13T10:12:15.584 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.584 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.584 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:12:15.584 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226037+0000", 2024-09-13T10:12:15.584 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010443948 2024-09-13T10:12:15.584 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.584 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.584 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:15.585 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228312+0000", 2024-09-13T10:12:15.585 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022743009999999998 2024-09-13T10:12:15.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.585 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.585 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.585 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.228708+0000", 2024-09-13T10:12:15.585 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039651999999999998 2024-09-13T10:12:15.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.585 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.585 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.585 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229625+0000", 2024-09-13T10:12:15.585 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00091692300000000004 2024-09-13T10:12:15.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.586 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:15.586 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229750+0000", 2024-09-13T10:12:15.586 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012532099999999999 2024-09-13T10:12:15.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.586 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.586 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229860+0000", 2024-09-13T10:12:15.586 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000109627 2024-09-13T10:12:15.586 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.586 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.586 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.587 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.587 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.587 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:260 4.3 4:c426503f:::smithi19530714-11:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:15.587 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:12:15.587 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 84.360307989999995, 2024-09-13T10:12:15.587 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041961871999999997, 2024-09-13T10:12:15.587 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.587 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.587 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.587 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.587 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.587 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.587 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 260 2024-09-13T10:12:15.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.588 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.588 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.588 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.588 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:12:15.588 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.588 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.588 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.588 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215335+0000", 2024-09-13T10:12:15.588 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.589 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.589 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215336+0000", 2024-09-13T10:12:15.589 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9069999999999999e-06 2024-09-13T10:12:15.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.589 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.589 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215345+0000", 2024-09-13T10:12:15.589 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.6659999999999997e-06 2024-09-13T10:12:15.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.589 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.589 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215363+0000", 2024-09-13T10:12:15.590 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8153000000000001e-05 2024-09-13T10:12:15.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.590 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.590 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.590 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215388+0000", 2024-09-13T10:12:15.590 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4556e-05 2024-09-13T10:12:15.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.590 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.590 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.590 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215542+0000", 2024-09-13T10:12:15.590 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000154637 2024-09-13T10:12:15.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.590 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.591 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.591 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215825+0000", 2024-09-13T10:12:15.591 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028208600000000002 2024-09-13T10:12:15.591 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.591 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.591 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:12:15.591 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254572+0000", 2024-09-13T10:12:15.591 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.038747464000000002 2024-09-13T10:12:15.591 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.591 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.591 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:15.591 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.255694+0000", 2024-09-13T10:12:15.591 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011222929999999999 2024-09-13T10:12:15.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.592 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.592 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.256931+0000", 2024-09-13T10:12:15.592 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0012370980000000001 2024-09-13T10:12:15.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.592 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.592 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257080+0000", 2024-09-13T10:12:15.592 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014874800000000001 2024-09-13T10:12:15.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.593 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:15.593 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257211+0000", 2024-09-13T10:12:15.593 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000130674 2024-09-13T10:12:15.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.593 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.593 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.593 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.257296+0000", 2024-09-13T10:12:15.593 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5589999999999999e-05 2024-09-13T10:12:15.593 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.593 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.593 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.594 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.594 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:269 4.3 4:c5b5ff75:::smithi19530714-20:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:15.594 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:12:15.594 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 84.359803752000005, 2024-09-13T10:12:15.594 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014988243, 2024-09-13T10:12:15.594 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.594 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.594 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.594 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.594 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.594 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.594 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 269 2024-09-13T10:12:15.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.594 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.595 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.595 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.595 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:12:15.595 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.595 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.595 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.595 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215839+0000", 2024-09-13T10:12:15.595 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.595 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.595 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.595 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215841+0000", 2024-09-13T10:12:15.596 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0829999999999998e-06 2024-09-13T10:12:15.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.596 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.596 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215854+0000", 2024-09-13T10:12:15.596 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2976000000000001e-05 2024-09-13T10:12:15.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.596 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.596 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215875+0000", 2024-09-13T10:12:15.596 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1100000000000001e-05 2024-09-13T10:12:15.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.597 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.597 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.215892+0000", 2024-09-13T10:12:15.597 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7388999999999999e-05 2024-09-13T10:12:15.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.597 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.597 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.597 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216353+0000", 2024-09-13T10:12:15.597 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00046100900000000002 2024-09-13T10:12:15.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.597 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.597 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.597 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216611+0000", 2024-09-13T10:12:15.598 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025716299999999998 2024-09-13T10:12:15.598 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.598 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.598 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:12:15.598 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.227177+0000", 2024-09-13T10:12:15.598 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010566676000000001 2024-09-13T10:12:15.598 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.598 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.598 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:15.598 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229574+0000", 2024-09-13T10:12:15.598 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023964469999999999 2024-09-13T10:12:15.598 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.598 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.598 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.599 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230619+0000", 2024-09-13T10:12:15.599 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0010455950000000001 2024-09-13T10:12:15.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.599 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.599 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.599 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230697+0000", 2024-09-13T10:12:15.599 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8021000000000004e-05 2024-09-13T10:12:15.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.599 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.599 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:15.599 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230767+0000", 2024-09-13T10:12:15.599 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9544999999999999e-05 2024-09-13T10:12:15.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.600 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.600 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.600 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230827+0000", 2024-09-13T10:12:15.600 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.0238999999999997e-05 2024-09-13T10:12:15.600 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.600 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.600 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.600 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.600 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.600 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:310 4.3 4:c426503f:::smithi19530714-11:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:15.600 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:12:15.600 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 84.302507679000001, 2024-09-13T10:12:15.600 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013782279999999999, 2024-09-13T10:12:15.601 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.601 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.601 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.601 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.601 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.601 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.601 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 310 2024-09-13T10:12:15.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.601 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.601 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.601 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.601 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:12:15.601 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.602 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.602 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.602 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273135+0000", 2024-09-13T10:12:15.602 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.602 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.602 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.602 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273138+0000", 2024-09-13T10:12:15.602 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8459999999999999e-06 2024-09-13T10:12:15.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.602 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.602 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.602 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273150+0000", 2024-09-13T10:12:15.603 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2133999999999999e-05 2024-09-13T10:12:15.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.603 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.603 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.603 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273175+0000", 2024-09-13T10:12:15.603 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5321999999999999e-05 2024-09-13T10:12:15.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.603 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.603 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.603 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273192+0000", 2024-09-13T10:12:15.603 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.681e-05 2024-09-13T10:12:15.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.604 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.604 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273314+0000", 2024-09-13T10:12:15.604 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012228 2024-09-13T10:12:15.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.604 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.604 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273474+0000", 2024-09-13T10:12:15.604 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016019799999999999 2024-09-13T10:12:15.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.604 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:12:15.604 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.284136+0000", 2024-09-13T10:12:15.604 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010661761000000001 2024-09-13T10:12:15.605 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.605 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.605 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.605 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286085+0000", 2024-09-13T10:12:15.605 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019484929999999999 2024-09-13T10:12:15.605 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.605 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.605 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:15.605 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286111+0000", 2024-09-13T10:12:15.605 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5834e-05 2024-09-13T10:12:15.605 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.605 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.605 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.605 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286721+0000", 2024-09-13T10:12:15.606 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00061038599999999998 2024-09-13T10:12:15.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.606 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.606 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:15.606 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286838+0000", 2024-09-13T10:12:15.606 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117064 2024-09-13T10:12:15.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.606 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.606 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.606 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286917+0000", 2024-09-13T10:12:15.606 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9152000000000004e-05 2024-09-13T10:12:15.606 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.606 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.606 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.607 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.607 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1006 4.3 4:c5b5ff75:::smithi19530714-20:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:15.607 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:12:15.607 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 79.921594431000003, 2024-09-13T10:12:15.607 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.047764635, 2024-09-13T10:12:15.607 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.607 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.607 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:15.607 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.607 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.607 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.607 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1006 2024-09-13T10:12:15.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.608 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.608 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.608 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:12:15.608 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.608 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.608 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654048+0000", 2024-09-13T10:12:15.608 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.608 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.608 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654049+0000", 2024-09-13T10:12:15.608 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5600000000000005e-07 2024-09-13T10:12:15.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.609 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.609 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.609 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654058+0000", 2024-09-13T10:12:15.609 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8200000000000003e-06 2024-09-13T10:12:15.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.609 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.609 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.609 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654067+0000", 2024-09-13T10:12:15.609 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.4599999999999992e-06 2024-09-13T10:12:15.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.609 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.610 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.610 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654075+0000", 2024-09-13T10:12:15.610 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.1070000000000005e-06 2024-09-13T10:12:15.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.610 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.610 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.610 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654188+0000", 2024-09-13T10:12:15.610 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000112734 2024-09-13T10:12:15.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.610 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.610 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.610 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.654395+0000", 2024-09-13T10:12:15.610 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000207235 2024-09-13T10:12:15.611 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.611 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.611 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.611 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.701813+0000", 2024-09-13T10:12:15.611 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.047417523000000003 2024-09-13T10:12:15.611 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.611 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.611 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.611 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.611 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.611 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1013 4.3 4:c426503f:::smithi19530714-11:head [write 477332~766101] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:15.611 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:12:15.611 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 79.914273176999998, 2024-09-13T10:12:15.611 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027363838000000001, 2024-09-13T10:12:15.612 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.612 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.612 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.612 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.612 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.612 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.612 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1013 2024-09-13T10:12:15.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.612 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.612 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.612 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.612 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:12:15.612 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.613 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.613 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661369+0000", 2024-09-13T10:12:15.613 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.613 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.613 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.661374+0000", 2024-09-13T10:12:15.613 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.0930000000000002e-06 2024-09-13T10:12:15.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.613 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.614 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665828+0000", 2024-09-13T10:12:15.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044532119999999998 2024-09-13T10:12:15.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.614 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.614 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.614 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665864+0000", 2024-09-13T10:12:15.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5979000000000002e-05 2024-09-13T10:12:15.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.614 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.614 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.614 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.665881+0000", 2024-09-13T10:12:15.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7836e-05 2024-09-13T10:12:15.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.615 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.615 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.615 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.678976+0000", 2024-09-13T10:12:15.615 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.01309448 2024-09-13T10:12:15.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.615 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.615 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.615 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.679188+0000", 2024-09-13T10:12:15.615 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000211945 2024-09-13T10:12:15.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.615 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.615 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:12:15.616 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.679468+0000", 2024-09-13T10:12:15.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028034000000000002 2024-09-13T10:12:15.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.616 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:15.616 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688233+0000", 2024-09-13T10:12:15.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0087645369999999993 2024-09-13T10:12:15.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.616 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.616 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688388+0000", 2024-09-13T10:12:15.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015488000000000001 2024-09-13T10:12:15.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688505+0000", 2024-09-13T10:12:15.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117114 2024-09-13T10:12:15.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.617 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:15.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688612+0000", 2024-09-13T10:12:15.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010741100000000001 2024-09-13T10:12:15.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.617 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688733+0000", 2024-09-13T10:12:15.618 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000121011 2024-09-13T10:12:15.618 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.618 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.618 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.618 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.618 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.618 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1014 4.3 4:c426503f:::smithi19530714-11:head [write 1681102~474010] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:15.618 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:12:15.618 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 79.908999790999999, 2024-09-13T10:12:15.618 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022694374, 2024-09-13T10:12:15.618 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.618 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.618 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.618 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.619 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.619 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.619 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1014 2024-09-13T10:12:15.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.619 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.619 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.619 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.619 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:12:15.619 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.619 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.619 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.619 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666643+0000", 2024-09-13T10:12:15.620 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.620 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.620 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.620 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.666645+0000", 2024-09-13T10:12:15.620 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2970000000000002e-06 2024-09-13T10:12:15.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.620 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.620 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.620 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667326+0000", 2024-09-13T10:12:15.620 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00068073199999999997 2024-09-13T10:12:15.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.620 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.621 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.621 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667346+0000", 2024-09-13T10:12:15.621 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0188999999999999e-05 2024-09-13T10:12:15.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.621 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.621 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.621 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.667361+0000", 2024-09-13T10:12:15.621 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5299999999999999e-05 2024-09-13T10:12:15.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.621 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.621 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.621 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.683468+0000", 2024-09-13T10:12:15.621 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016106944000000002 2024-09-13T10:12:15.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.622 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.622 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.622 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.683714+0000", 2024-09-13T10:12:15.622 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024595699999999999 2024-09-13T10:12:15.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.622 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.622 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:12:15.622 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.684132+0000", 2024-09-13T10:12:15.622 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00041775800000000001 2024-09-13T10:12:15.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.622 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.622 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:15.622 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688771+0000", 2024-09-13T10:12:15.623 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0046394540000000003 2024-09-13T10:12:15.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.623 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.623 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.623 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.688920+0000", 2024-09-13T10:12:15.623 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014886000000000001 2024-09-13T10:12:15.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.623 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.623 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.623 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689085+0000", 2024-09-13T10:12:15.623 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000164539 2024-09-13T10:12:15.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.623 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.623 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:15.624 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689230+0000", 2024-09-13T10:12:15.624 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000145394 2024-09-13T10:12:15.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.624 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.624 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689337+0000", 2024-09-13T10:12:15.624 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010695 2024-09-13T10:12:15.624 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.624 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.624 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.625 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1015 4.3 4:c426503f:::smithi19530714-11:head [write 2687912~589420] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:15.625 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:12:15.625 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 79.902683113999998, 2024-09-13T10:12:15.625 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.018489873, 2024-09-13T10:12:15.625 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.625 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.625 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.625 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.625 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.625 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.625 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1015 2024-09-13T10:12:15.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.625 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.626 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.626 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:12:15.626 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.626 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.626 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672959+0000", 2024-09-13T10:12:15.626 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.626 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.626 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.672962+0000", 2024-09-13T10:12:15.626 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.277e-06 2024-09-13T10:12:15.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.627 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.627 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.627 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673758+0000", 2024-09-13T10:12:15.627 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00079599100000000002 2024-09-13T10:12:15.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.627 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.627 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.627 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673787+0000", 2024-09-13T10:12:15.627 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9280000000000001e-05 2024-09-13T10:12:15.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.627 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.628 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.628 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673801+0000", 2024-09-13T10:12:15.628 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4493e-05 2024-09-13T10:12:15.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.628 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.628 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.628 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.685885+0000", 2024-09-13T10:12:15.628 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012083692 2024-09-13T10:12:15.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.628 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.628 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.628 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.686057+0000", 2024-09-13T10:12:15.628 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017133900000000001 2024-09-13T10:12:15.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.629 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:12:15.629 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.686396+0000", 2024-09-13T10:12:15.629 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000339809 2024-09-13T10:12:15.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.629 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.629 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691040+0000", 2024-09-13T10:12:15.629 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.004643215 2024-09-13T10:12:15.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.629 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:15.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691082+0000", 2024-09-13T10:12:15.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.2499000000000001e-05 2024-09-13T10:12:15.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.630 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691179+0000", 2024-09-13T10:12:15.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7044000000000001e-05 2024-09-13T10:12:15.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.630 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:15.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691324+0000", 2024-09-13T10:12:15.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014532299999999999 2024-09-13T10:12:15.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.631 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.631 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691449+0000", 2024-09-13T10:12:15.631 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012491100000000001 2024-09-13T10:12:15.631 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.631 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.631 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.631 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1016 4.3 4:c426503f:::smithi19530714-11:head [setxattr _header (56) in=7b,truncate 3277332] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:15.631 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:12:15.631 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 79.901814157000004, 2024-09-13T10:12:15.632 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.019820562999999999, 2024-09-13T10:12:15.632 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.632 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.632 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.632 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.632 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.632 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.632 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1016 2024-09-13T10:12:15.632 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.632 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.632 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.632 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.632 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:12:15.633 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.633 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.633 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673828+0000", 2024-09-13T10:12:15.633 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.633 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.633 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673830+0000", 2024-09-13T10:12:15.633 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1170000000000001e-06 2024-09-13T10:12:15.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.634 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.634 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.634 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673835+0000", 2024-09-13T10:12:15.634 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5559999999999998e-06 2024-09-13T10:12:15.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.634 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.634 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.634 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673845+0000", 2024-09-13T10:12:15.634 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6330000000000008e-06 2024-09-13T10:12:15.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.634 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.634 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.634 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673856+0000", 2024-09-13T10:12:15.635 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0963e-05 2024-09-13T10:12:15.635 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.635 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.635 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689480+0000", 2024-09-13T10:12:15.635 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015624605 2024-09-13T10:12:15.635 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.635 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.635 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.689739+0000", 2024-09-13T10:12:15.635 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025878700000000001 2024-09-13T10:12:15.635 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.636 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:12:15.636 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.690154+0000", 2024-09-13T10:12:15.636 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00041533800000000001 2024-09-13T10:12:15.636 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.636 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.636 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.636 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691858+0000", 2024-09-13T10:12:15.636 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001703811 2024-09-13T10:12:15.637 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.637 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.637 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:15.637 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691880+0000", 2024-09-13T10:12:15.637 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1997e-05 2024-09-13T10:12:15.637 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.637 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.637 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.637 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693384+0000", 2024-09-13T10:12:15.637 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0015040190000000001 2024-09-13T10:12:15.637 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.637 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.638 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:15.638 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693541+0000", 2024-09-13T10:12:15.638 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015723899999999999 2024-09-13T10:12:15.638 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.638 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.638 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.638 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693649+0000", 2024-09-13T10:12:15.638 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010749800000000001 2024-09-13T10:12:15.638 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.638 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.638 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.638 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.638 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.638 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1017 4.3 4:c426503f:::smithi19530714-11:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:15.639 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:12:15.639 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 79.901774145000005, 2024-09-13T10:12:15.639 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020166488999999999, 2024-09-13T10:12:15.639 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.639 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.639 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:15.639 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.639 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.639 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.639 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1017 2024-09-13T10:12:15.639 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.639 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.640 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.640 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.640 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:12:15.640 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.640 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.640 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.640 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.640 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673868+0000", 2024-09-13T10:12:15.640 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.640 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.640 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.640 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.640 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673869+0000", 2024-09-13T10:12:15.641 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9699999999999995e-07 2024-09-13T10:12:15.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.641 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.641 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.641 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673873+0000", 2024-09-13T10:12:15.641 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.4970000000000001e-06 2024-09-13T10:12:15.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.641 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.641 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.641 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673885+0000", 2024-09-13T10:12:15.641 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2398999999999999e-05 2024-09-13T10:12:15.641 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.641 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.642 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.642 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.673893+0000", 2024-09-13T10:12:15.642 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7530000000000001e-06 2024-09-13T10:12:15.642 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.642 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.642 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.642 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691638+0000", 2024-09-13T10:12:15.642 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017745007 2024-09-13T10:12:15.642 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.642 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.642 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:15.642 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.691789+0000", 2024-09-13T10:12:15.642 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015133400000000001 2024-09-13T10:12:15.642 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.643 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.643 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.643 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693731+0000", 2024-09-13T10:12:15.643 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019422459999999999 2024-09-13T10:12:15.643 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.643 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.643 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.643 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.693870+0000", 2024-09-13T10:12:15.643 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013820299999999999 2024-09-13T10:12:15.643 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.643 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.643 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.643 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:55.694035+0000", 2024-09-13T10:12:15.644 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000165253 2024-09-13T10:12:15.644 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.644 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.644 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.644 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.644 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.644 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1405 4.3 4:c5b5ff75:::smithi19530714-20:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-13T10:12:15.644 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:01.492787+0000", 2024-09-13T10:12:15.644 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 74.082855500999997, 2024-09-13T10:12:15.644 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021225698000000001, 2024-09-13T10:12:15.644 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.644 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.644 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:15.645 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.645 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.645 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.645 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1405 2024-09-13T10:12:15.645 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.645 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.645 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.645 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.645 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492787+0000", 2024-09-13T10:12:15.645 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.645 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.645 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.645 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.646 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492787+0000", 2024-09-13T10:12:15.646 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.646 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.646 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.646 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.646 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492791+0000", 2024-09-13T10:12:15.646 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.89e-06 2024-09-13T10:12:15.646 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.646 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.646 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.646 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492805+0000", 2024-09-13T10:12:15.646 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4083000000000001e-05 2024-09-13T10:12:15.646 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.647 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.647 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.647 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492846+0000", 2024-09-13T10:12:15.647 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0849000000000001e-05 2024-09-13T10:12:15.647 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.647 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.647 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.647 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492867+0000", 2024-09-13T10:12:15.647 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1058999999999999e-05 2024-09-13T10:12:15.647 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.647 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.647 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.647 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.492973+0000", 2024-09-13T10:12:15.648 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000106084 2024-09-13T10:12:15.648 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.648 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.648 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.648 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.493105+0000", 2024-09-13T10:12:15.648 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013215899999999999 2024-09-13T10:12:15.648 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.648 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.648 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.648 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:01.514013+0000", 2024-09-13T10:12:15.648 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020907574000000002 2024-09-13T10:12:15.648 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.648 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.649 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.649 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.649 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.649 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2816 4.3 4:c5b5ff75:::smithi19530714-20:head [assert-version v76,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:12:15.649 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.891848+0000", 2024-09-13T10:12:15.649 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 42.683794476000003, 2024-09-13T10:12:15.649 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020571445000000001, 2024-09-13T10:12:15.649 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.649 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.649 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.649 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.649 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.649 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.650 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2816 2024-09-13T10:12:15.650 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.650 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.650 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.650 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.650 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891848+0000", 2024-09-13T10:12:15.650 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.650 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.650 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.650 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.650 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891848+0000", 2024-09-13T10:12:15.650 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.650 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.651 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.651 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.651 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891852+0000", 2024-09-13T10:12:15.651 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1729999999999997e-06 2024-09-13T10:12:15.651 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.651 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.651 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.651 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891869+0000", 2024-09-13T10:12:15.651 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6639000000000001e-05 2024-09-13T10:12:15.651 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.651 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.651 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.652 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891910+0000", 2024-09-13T10:12:15.652 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1125999999999999e-05 2024-09-13T10:12:15.652 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.652 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.652 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.652 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.891934+0000", 2024-09-13T10:12:15.652 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4039000000000001e-05 2024-09-13T10:12:15.652 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.652 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.652 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.652 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892099+0000", 2024-09-13T10:12:15.652 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016462000000000001 2024-09-13T10:12:15.652 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.653 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.653 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.653 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892368+0000", 2024-09-13T10:12:15.653 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026922699999999998 2024-09-13T10:12:15.653 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.653 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.653 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:12:15.653 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.907670+0000", 2024-09-13T10:12:15.653 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015302299 2024-09-13T10:12:15.653 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.653 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.653 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.654 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.910850+0000", 2024-09-13T10:12:15.654 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003179383 2024-09-13T10:12:15.654 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.654 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.654 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:15.654 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.910901+0000", 2024-09-13T10:12:15.654 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.1548000000000002e-05 2024-09-13T10:12:15.654 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.654 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.654 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.654 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.912089+0000", 2024-09-13T10:12:15.654 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011878170000000001 2024-09-13T10:12:15.654 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.654 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.655 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:15.655 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.912267+0000", 2024-09-13T10:12:15.655 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000178393 2024-09-13T10:12:15.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.655 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.655 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.655 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.912420+0000", 2024-09-13T10:12:15.655 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015218100000000001 2024-09-13T10:12:15.655 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.655 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.655 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.655 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.656 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.656 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2818 4.3 4:c426503f:::smithi19530714-11:head [sparse-read 0~2823261] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:12:15.656 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.892118+0000", 2024-09-13T10:12:15.656 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 42.683524216000002, 2024-09-13T10:12:15.656 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014982577, 2024-09-13T10:12:15.656 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.656 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.656 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:15.656 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.656 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.656 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.656 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2818 2024-09-13T10:12:15.656 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.657 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.657 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.657 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.657 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892118+0000", 2024-09-13T10:12:15.657 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.657 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.657 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.657 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.657 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892118+0000", 2024-09-13T10:12:15.657 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.657 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.657 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.657 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.658 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892120+0000", 2024-09-13T10:12:15.658 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5260000000000001e-06 2024-09-13T10:12:15.658 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.658 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.658 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.658 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892128+0000", 2024-09-13T10:12:15.658 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7640000000000005e-06 2024-09-13T10:12:15.658 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.658 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.658 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.658 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892172+0000", 2024-09-13T10:12:15.658 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4561000000000002e-05 2024-09-13T10:12:15.658 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.658 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892231+0000", 2024-09-13T10:12:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9194999999999998e-05 2024-09-13T10:12:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.903452+0000", 2024-09-13T10:12:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011221024 2024-09-13T10:12:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.903711+0000", 2024-09-13T10:12:15.659 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025891100000000002 2024-09-13T10:12:15.660 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.660 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.660 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.660 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.907101+0000", 2024-09-13T10:12:15.660 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0033895959999999999 2024-09-13T10:12:15.660 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.660 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.660 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.660 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.660 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.660 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2819 4.3 4:c426503f:::smithi19530714-11:head [sparse-read 1467121~1802604] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:12:15.660 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.892259+0000", 2024-09-13T10:12:15.660 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 42.683383208000002, 2024-09-13T10:12:15.661 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.018443480000000002, 2024-09-13T10:12:15.661 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.661 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.661 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:15.661 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.661 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.661 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.661 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2819 2024-09-13T10:12:15.661 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.661 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.661 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.662 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.662 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892259+0000", 2024-09-13T10:12:15.662 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.662 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.662 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.662 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.662 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892259+0000", 2024-09-13T10:12:15.662 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.662 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.662 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.662 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.662 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892261+0000", 2024-09-13T10:12:15.662 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5200000000000001e-06 2024-09-13T10:12:15.663 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.663 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.663 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.663 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892287+0000", 2024-09-13T10:12:15.663 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5976e-05 2024-09-13T10:12:15.663 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.663 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.663 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.663 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892306+0000", 2024-09-13T10:12:15.663 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9208999999999999e-05 2024-09-13T10:12:15.663 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.663 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.663 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.664 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892322+0000", 2024-09-13T10:12:15.664 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6426000000000002e-05 2024-09-13T10:12:15.664 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.664 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.664 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.664 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.908963+0000", 2024-09-13T10:12:15.664 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016640922999999998 2024-09-13T10:12:15.664 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.664 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.664 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.664 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.909202+0000", 2024-09-13T10:12:15.664 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023879500000000001 2024-09-13T10:12:15.665 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.665 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.665 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.665 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.910703+0000", 2024-09-13T10:12:15.665 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0015006310000000001 2024-09-13T10:12:15.665 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.665 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.665 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.665 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.665 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.665 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3078 4.3 4:c426503f:::smithi19530714-11:head [read 0~2487469,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-13T10:12:15.665 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:38.708725+0000", 2024-09-13T10:12:15.665 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 36.866917608999998, 2024-09-13T10:12:15.666 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013854750000000001, 2024-09-13T10:12:15.666 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.666 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.666 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:15.666 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.666 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.666 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.666 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3078 2024-09-13T10:12:15.666 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.666 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.666 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.666 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.666 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708725+0000", 2024-09-13T10:12:15.667 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.667 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.667 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.667 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.667 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708725+0000", 2024-09-13T10:12:15.667 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.667 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.667 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.667 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.667 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708726+0000", 2024-09-13T10:12:15.667 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.08e-06 2024-09-13T10:12:15.667 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.667 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.668 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.668 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708731+0000", 2024-09-13T10:12:15.668 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.3029999999999999e-06 2024-09-13T10:12:15.668 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.668 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.668 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.668 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708741+0000", 2024-09-13T10:12:15.668 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0077e-05 2024-09-13T10:12:15.668 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.668 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.668 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.668 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.708750+0000", 2024-09-13T10:12:15.668 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5560000000000008e-06 2024-09-13T10:12:15.669 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.669 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.669 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.669 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.720444+0000", 2024-09-13T10:12:15.669 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011694334000000001 2024-09-13T10:12:15.669 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.669 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.669 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.669 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.720692+0000", 2024-09-13T10:12:15.669 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024811599999999999 2024-09-13T10:12:15.669 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.669 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.669 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.670 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:38.722580+0000", 2024-09-13T10:12:15.670 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0018872839999999999 2024-09-13T10:12:15.670 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.670 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.670 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.670 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.670 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.670 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3282 4.1 4:85f1e656:::smithi19530714-10:head [write 574872~695118,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:15.670 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:43.118594+0000", 2024-09-13T10:12:15.670 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 32.457048823000001, 2024-09-13T10:12:15.670 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015113895, 2024-09-13T10:12:15.670 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.670 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3282 2024-09-13T10:12:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.118594+0000", 2024-09-13T10:12:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.671 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.672 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.672 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.118594+0000", 2024-09-13T10:12:15.672 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.672 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.672 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.672 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.672 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.118597+0000", 2024-09-13T10:12:15.672 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7860000000000001e-06 2024-09-13T10:12:15.672 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.672 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.672 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.672 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.119871+0000", 2024-09-13T10:12:15.672 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0012741530000000001 2024-09-13T10:12:15.673 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.673 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.673 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.673 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.119904+0000", 2024-09-13T10:12:15.673 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3325999999999999e-05 2024-09-13T10:12:15.673 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.673 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.673 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.673 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.119927+0000", 2024-09-13T10:12:15.673 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2580000000000001e-05 2024-09-13T10:12:15.673 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.673 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.674 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.674 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.120067+0000", 2024-09-13T10:12:15.674 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014075200000000001 2024-09-13T10:12:15.674 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.674 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.674 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op must wait for map", 2024-09-13T10:12:15.674 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.120078+0000", 2024-09-13T10:12:15.674 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0776e-05 2024-09-13T10:12:15.674 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.674 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.674 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.674 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.124523+0000", 2024-09-13T10:12:15.674 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044453089999999997 2024-09-13T10:12:15.674 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.675 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.675 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.675 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.124748+0000", 2024-09-13T10:12:15.675 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022474100000000001 2024-09-13T10:12:15.675 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.675 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.675 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:12:15.675 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.125194+0000", 2024-09-13T10:12:15.675 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00044562699999999998 2024-09-13T10:12:15.675 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.675 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.675 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:15.675 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.132524+0000", 2024-09-13T10:12:15.676 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0073297850000000001 2024-09-13T10:12:15.676 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.676 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.676 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.676 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.132626+0000", 2024-09-13T10:12:15.676 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010255399999999999 2024-09-13T10:12:15.676 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.676 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.676 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.676 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.132778+0000", 2024-09-13T10:12:15.676 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000151459 2024-09-13T10:12:15.676 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.677 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.677 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:15.677 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.133577+0000", 2024-09-13T10:12:15.677 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00079910099999999998 2024-09-13T10:12:15.677 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.677 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.677 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.677 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:43.133708+0000", 2024-09-13T10:12:15.677 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000130946 2024-09-13T10:12:15.677 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.677 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.677 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.677 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.678 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.678 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3439 4.1 4:85f1e656:::smithi19530714-10:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e85)", 2024-09-13T10:12:15.678 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:47.513941+0000", 2024-09-13T10:12:15.678 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 28.061701256999999, 2024-09-13T10:12:15.678 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051734025000000003, 2024-09-13T10:12:15.678 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.678 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.678 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:15.678 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.678 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.678 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.678 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3439 2024-09-13T10:12:15.678 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.679 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.679 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.679 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.679 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.513941+0000", 2024-09-13T10:12:15.679 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.679 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.679 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.679 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.679 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.513941+0000", 2024-09-13T10:12:15.679 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.679 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.679 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.679 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.680 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.513945+0000", 2024-09-13T10:12:15.680 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.207e-06 2024-09-13T10:12:15.680 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.680 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.680 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.680 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.513956+0000", 2024-09-13T10:12:15.680 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1899000000000001e-05 2024-09-13T10:12:15.680 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.680 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.680 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.680 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.513996+0000", 2024-09-13T10:12:15.680 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9456000000000003e-05 2024-09-13T10:12:15.681 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.681 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.681 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.681 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.514014+0000", 2024-09-13T10:12:15.681 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8050000000000002e-05 2024-09-13T10:12:15.681 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.681 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.681 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.681 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.514184+0000", 2024-09-13T10:12:15.681 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017006500000000001 2024-09-13T10:12:15.681 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.681 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.681 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op must wait for map", 2024-09-13T10:12:15.681 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.514196+0000", 2024-09-13T10:12:15.682 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2223e-05 2024-09-13T10:12:15.682 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.682 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.682 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.682 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.517357+0000", 2024-09-13T10:12:15.682 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0031610919999999999 2024-09-13T10:12:15.682 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.682 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.682 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.682 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.517577+0000", 2024-09-13T10:12:15.682 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021954400000000001 2024-09-13T10:12:15.682 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.683 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.683 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.683 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:47.565675+0000", 2024-09-13T10:12:15.683 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048098489000000001 2024-09-13T10:12:15.683 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.683 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.683 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.683 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.683 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.683 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3529 4.3 4:c5b5ff75:::smithi19530714-20:head [write 494205~473908] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e87)", 2024-09-13T10:12:15.683 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:48.974598+0000", 2024-09-13T10:12:15.683 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 26.601044432999998, 2024-09-13T10:12:15.683 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017452275, 2024-09-13T10:12:15.683 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.684 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.684 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.684 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.684 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.684 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.684 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3529 2024-09-13T10:12:15.684 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.684 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.684 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.684 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.684 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.974598+0000", 2024-09-13T10:12:15.684 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.684 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.685 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.685 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.685 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.974598+0000", 2024-09-13T10:12:15.685 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.685 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.685 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.685 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.685 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.974602+0000", 2024-09-13T10:12:15.685 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.393e-06 2024-09-13T10:12:15.685 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.685 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.685 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.685 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.975525+0000", 2024-09-13T10:12:15.686 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00092394999999999999 2024-09-13T10:12:15.686 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.686 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.686 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.686 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.975562+0000", 2024-09-13T10:12:15.686 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6949000000000001e-05 2024-09-13T10:12:15.686 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.686 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.686 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.686 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.975587+0000", 2024-09-13T10:12:15.686 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4805999999999999e-05 2024-09-13T10:12:15.686 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.686 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.686 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.975799+0000", 2024-09-13T10:12:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021148200000000001 2024-09-13T10:12:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op must wait for map", 2024-09-13T10:12:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.975813+0000", 2024-09-13T10:12:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4090000000000001e-05 2024-09-13T10:12:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.980728+0000", 2024-09-13T10:12:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0049153310000000002 2024-09-13T10:12:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.687 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.980922+0000", 2024-09-13T10:12:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000194372 2024-09-13T10:12:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,11", 2024-09-13T10:12:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.981273+0000", 2024-09-13T10:12:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00035047799999999999 2024-09-13T10:12:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.991703+0000", 2024-09-13T10:12:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010430406 2024-09-13T10:12:15.688 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.991802+0000", 2024-09-13T10:12:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.8664000000000003e-05 2024-09-13T10:12:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.991903+0000", 2024-09-13T10:12:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010054499999999999 2024-09-13T10:12:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:15.689 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.991977+0000", 2024-09-13T10:12:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.4017999999999995e-05 2024-09-13T10:12:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:48.992050+0000", 2024-09-13T10:12:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.3790999999999999e-05 2024-09-13T10:12:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:4170 4.b 4:dd0af53e:::smithi19530714-13:head [tier-promote] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:12:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:05.795169+0000", 2024-09-13T10:12:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.7804739699999992, 2024-09-13T10:12:15.690 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041634358000000003, 2024-09-13T10:12:15.691 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.691 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.691 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:15.691 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.691 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.691 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.691 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 4170 2024-09-13T10:12:15.691 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.691 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.691 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.691 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.691 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.795169+0000", 2024-09-13T10:12:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.795169+0000", 2024-09-13T10:12:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.795171+0000", 2024-09-13T10:12:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6900000000000001e-06 2024-09-13T10:12:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.692 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.795183+0000", 2024-09-13T10:12:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.153e-05 2024-09-13T10:12:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.795213+0000", 2024-09-13T10:12:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0335e-05 2024-09-13T10:12:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.795238+0000", 2024-09-13T10:12:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5273999999999999e-05 2024-09-13T10:12:15.693 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.795921+0000", 2024-09-13T10:12:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00068254399999999999 2024-09-13T10:12:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:12:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.795949+0000", 2024-09-13T10:12:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7559000000000002e-05 2024-09-13T10:12:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.825792+0000", 2024-09-13T10:12:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029843064999999998 2024-09-13T10:12:15.694 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.826065+0000", 2024-09-13T10:12:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00027375899999999999 2024-09-13T10:12:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.836803+0000", 2024-09-13T10:12:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010737602000000001 2024-09-13T10:12:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.695 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.696 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.696 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:4343 4.1 4:85f1e656:::smithi19530714-10:head [write 655572~648960,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-13T10:12:15.696 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:09.849415+0000", 2024-09-13T10:12:15.696 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 5.7262273009999998, 2024-09-13T10:12:15.696 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017310056000000001, 2024-09-13T10:12:15.696 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:15.696 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:15.696 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:15.696 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:15.696 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:15.696 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:15.696 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 4343 2024-09-13T10:12:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.849415+0000", 2024-09-13T10:12:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.849415+0000", 2024-09-13T10:12:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.697 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.849420+0000", 2024-09-13T10:12:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.7400000000000004e-06 2024-09-13T10:12:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.850543+0000", 2024-09-13T10:12:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011233630000000001 2024-09-13T10:12:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.850578+0000", 2024-09-13T10:12:15.698 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.4749000000000002e-05 2024-09-13T10:12:15.699 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.699 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.699 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:15.699 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.850596+0000", 2024-09-13T10:12:15.699 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7439999999999999e-05 2024-09-13T10:12:15.699 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.699 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.699 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.699 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.850743+0000", 2024-09-13T10:12:15.699 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000147787 2024-09-13T10:12:15.699 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.699 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op must wait for map", 2024-09-13T10:12:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.850755+0000", 2024-09-13T10:12:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.169e-05 2024-09-13T10:12:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.854284+0000", 2024-09-13T10:12:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003528991 2024-09-13T10:12:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.854413+0000", 2024-09-13T10:12:15.700 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012876399999999999 2024-09-13T10:12:15.701 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.701 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.701 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,12", 2024-09-13T10:12:15.701 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.854653+0000", 2024-09-13T10:12:15.701 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024048299999999999 2024-09-13T10:12:15.701 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.701 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.701 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.701 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.865931+0000", 2024-09-13T10:12:15.701 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011277886000000001 2024-09-13T10:12:15.701 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.701 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.701 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.866082+0000", 2024-09-13T10:12:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015101399999999999 2024-09-13T10:12:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.866444+0000", 2024-09-13T10:12:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036221700000000002 2024-09-13T10:12:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.866609+0000", 2024-09-13T10:12:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016461399999999999 2024-09-13T10:12:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:15.702 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:15.703 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.866725+0000", 2024-09-13T10:12:15.703 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011631799999999999 2024-09-13T10:12:15.703 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.703 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.703 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.703 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:15.703 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:15.703 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:15.705 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T10:12:15.705 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:15.703+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:15.716 INFO:tasks.rados.rados.0.smithi195.stdout:1884: expect (ObjNum 414 snap 0 seq_num 414) 2024-09-13T10:12:15.716 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:15.806 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:15.802+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:15.833 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:15.833 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:12:15.833 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:12:15.833 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:15.841 INFO:tasks.rados.rados.0.smithi195.stdout:1882: expect (ObjNum 407 snap 0 seq_num 407) 2024-09-13T10:12:15.841 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:15.847 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T10:12:15.904 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 771986 2024-09-13T10:12:15.932 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:15.930+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:16.007 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:16.001+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:16.009 INFO:tasks.rados.rados.0.smithi195.stdout:1888: finishing write tid 1 to smithi19530714-30 2024-09-13T10:12:16.009 INFO:tasks.rados.rados.0.smithi195.stdout:1888: finishing write tid 2 to smithi19530714-30 2024-09-13T10:12:16.009 INFO:tasks.rados.rados.0.smithi195.stdout:1888: finishing write tid 3 to smithi19530714-30 2024-09-13T10:12:16.009 INFO:tasks.rados.rados.0.smithi195.stdout:1888: finishing write tid 4 to smithi19530714-30 2024-09-13T10:12:16.010 INFO:tasks.rados.rados.0.smithi195.stdout:1888: oid 30 updating version 0 to 48 2024-09-13T10:12:16.010 INFO:tasks.rados.rados.0.smithi195.stdout:1888: oid 30 version 48 is already newer than 47 2024-09-13T10:12:16.010 INFO:tasks.rados.rados.0.smithi195.stdout:1888: oid 30 version 48 is already newer than 46 2024-09-13T10:12:16.010 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 48 (ObjNum 416 snap 0 seq_num 416) dirty exists 2024-09-13T10:12:16.010 INFO:tasks.rados.rados.0.smithi195.stdout:1888: left oid 30 (ObjNum 416 snap 0 seq_num 416) 2024-09-13T10:12:16.010 INFO:tasks.rados.rados.0.smithi195.stdout:1889: finishing write tid 1 to smithi19530714-21 2024-09-13T10:12:16.010 INFO:tasks.rados.rados.0.smithi195.stdout:1889: finishing write tid 2 to smithi19530714-21 2024-09-13T10:12:16.010 INFO:tasks.rados.rados.0.smithi195.stdout:1889: finishing write tid 3 to smithi19530714-21 2024-09-13T10:12:16.010 INFO:tasks.rados.rados.0.smithi195.stdout:1889: finishing write tid 4 to smithi19530714-21 2024-09-13T10:12:16.010 INFO:tasks.rados.rados.0.smithi195.stdout:1889: finishing write tid 5 to smithi19530714-21 2024-09-13T10:12:16.010 INFO:tasks.rados.rados.0.smithi195.stdout:1889: oid 21 updating version 0 to 163 2024-09-13T10:12:16.010 INFO:tasks.rados.rados.0.smithi195.stdout:1889: oid 21 version 163 is already newer than 162 2024-09-13T10:12:16.010 INFO:tasks.rados.rados.0.smithi195.stdout:1889: oid 21 updating version 163 to 164 2024-09-13T10:12:16.011 INFO:tasks.rados.rados.0.smithi195.stdout:1889: oid 21 updating version 164 to 165 2024-09-13T10:12:16.011 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 165 (ObjNum 417 snap 0 seq_num 417) dirty exists 2024-09-13T10:12:16.011 INFO:tasks.rados.rados.0.smithi195.stdout:1889: left oid 21 (ObjNum 417 snap 0 seq_num 417) 2024-09-13T10:12:16.011 INFO:tasks.rados.rados.0.smithi195.stdout:1891: tier_evict oid 10 snap -1 2024-09-13T10:12:16.011 INFO:tasks.rados.rados.0.smithi195.stdout:1890: finishing set_chunk to oid 27 2024-09-13T10:12:16.011 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 27 set_chunk returned error code -95 offset: 2881536 length: 497152 2024-09-13T10:12:16.011 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 0 (ObjNum 221 snap 0 seq_num 221) dirty exists 2024-09-13T10:12:16.011 INFO:tasks.rados.rados.0.smithi195.stdout:1863: expect (ObjNum 360 snap 0 seq_num 360) 2024-09-13T10:12:16.011 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:16.011 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:16.012 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:12:16.012 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:12:16.012 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:12:16.012 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:16.025 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T10:12:16.108 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:16.105+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:16.117 INFO:tasks.rados.rados.0.smithi195.stdout:1891: got (22) Invalid argument 2024-09-13T10:12:16.117 INFO:tasks.rados.rados.0.smithi195.stdout:1863: done (11 left) 2024-09-13T10:12:16.117 INFO:tasks.rados.rados.0.smithi195.stdout:1881: done (10 left) 2024-09-13T10:12:16.117 INFO:tasks.rados.rados.0.smithi195.stdout:1882: done (9 left) 2024-09-13T10:12:16.117 INFO:tasks.rados.rados.0.smithi195.stdout:1883: done (8 left) 2024-09-13T10:12:16.117 INFO:tasks.rados.rados.0.smithi195.stdout:1884: done (7 left) 2024-09-13T10:12:16.117 INFO:tasks.rados.rados.0.smithi195.stdout:1885: done (6 left) 2024-09-13T10:12:16.118 INFO:tasks.rados.rados.0.smithi195.stdout:1886: done (5 left) 2024-09-13T10:12:16.118 INFO:tasks.rados.rados.0.smithi195.stdout:1887: done (4 left) 2024-09-13T10:12:16.118 INFO:tasks.rados.rados.0.smithi195.stdout:1888: done (3 left) 2024-09-13T10:12:16.118 INFO:tasks.rados.rados.0.smithi195.stdout:1889: done (2 left) 2024-09-13T10:12:16.118 INFO:tasks.rados.rados.0.smithi195.stdout:1890: done (1 left) 2024-09-13T10:12:16.118 INFO:tasks.rados.rados.0.smithi195.stdout:1891: done (0 left) 2024-09-13T10:12:16.118 INFO:tasks.rados.rados.0.smithi195.stdout:1892: set_chunk oid 32 target oid 2024-09-13T10:12:16.144 INFO:tasks.rados.rados.0.smithi195.stdout:1892: set_chunk oid 32 offset: 1108992 length: 1782272 target oid 6645470a7e645aca786c8ce12398ca47e644173f2ae5072526b45004bb195a7a offset: 0 snap -1 2024-09-13T10:12:16.144 INFO:tasks.rados.rados.0.smithi195.stdout:1893: write oid 18 current snap is 0 2024-09-13T10:12:16.144 INFO:tasks.rados.rados.0.smithi195.stdout:1893: seq_num 418 ranges {625326=537144,1944179=672467} 2024-09-13T10:12:16.163 INFO:tasks.rados.rados.0.smithi195.stdout:1893: writing smithi19530714-18 from 625326 to 1162470 tid 1 2024-09-13T10:12:16.187 INFO:tasks.rados.rados.0.smithi195.stdout:1893: writing smithi19530714-18 from 1944179 to 2616646 tid 2 2024-09-13T10:12:16.187 INFO:tasks.rados.rados.0.smithi195.stdout:1894: tier_promote oid 41 2024-09-13T10:12:16.187 INFO:tasks.rados.rados.0.smithi195.stdout:1892: finishing set_chunk to oid 32 2024-09-13T10:12:16.187 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 103 (ObjNum 415 snap 0 seq_num 415) dirty exists 2024-09-13T10:12:16.187 INFO:tasks.rados.rados.0.smithi195.stdout:1892: done (2 left) 2024-09-13T10:12:16.187 INFO:tasks.rados.rados.0.smithi195.stdout:1895: read oid 39 snap -1 2024-09-13T10:12:16.187 INFO:tasks.rados.rados.0.smithi195.stdout:1895: expect (ObjNum 377 snap 0 seq_num 377) 2024-09-13T10:12:16.188 INFO:tasks.rados.rados.0.smithi195.stdout:1896: read oid 43 snap -1 2024-09-13T10:12:16.188 INFO:tasks.rados.rados.0.smithi195.stdout:1896: expect (ObjNum 344 snap 0 seq_num 344) 2024-09-13T10:12:16.188 INFO:tasks.rados.rados.0.smithi195.stdout:1897: write (excl) oid 46 current snap is 0 2024-09-13T10:12:16.188 INFO:tasks.rados.rados.0.smithi195.stdout:1897: seq_num 419 ranges {642133=728746,2119014=532478} 2024-09-13T10:12:16.195 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:16.195 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:12:16.195 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:12:16.195 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:12:16.195 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.195 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:251 4.5 4:a71ca386:::smithi19530714-2:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.196 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:12:16.196 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 85.075141555000002, 2024-09-13T10:12:16.196 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041813326999999997, 2024-09-13T10:12:16.196 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.196 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.196 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.196 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.196 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.196 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.196 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 251 2024-09-13T10:12:16.197 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.197 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.197 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.197 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.197 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:12:16.197 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.197 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.197 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.197 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.197 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117989+0000", 2024-09-13T10:12:16.197 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.197 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.198 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.198 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.198 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117990+0000", 2024-09-13T10:12:16.198 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.203e-06 2024-09-13T10:12:16.198 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.198 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.198 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.198 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117997+0000", 2024-09-13T10:12:16.198 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9750000000000001e-06 2024-09-13T10:12:16.198 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.198 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.199 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.199 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118009+0000", 2024-09-13T10:12:16.199 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2478e-05 2024-09-13T10:12:16.199 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.199 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.199 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.199 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118017+0000", 2024-09-13T10:12:16.199 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6909999999999997e-06 2024-09-13T10:12:16.199 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.199 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.199 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.199 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118106+0000", 2024-09-13T10:12:16.199 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.9331000000000002e-05 2024-09-13T10:12:16.199 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.200 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.200 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.200 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118184+0000", 2024-09-13T10:12:16.200 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7448999999999997e-05 2024-09-13T10:12:16.200 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.200 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.200 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,10", 2024-09-13T10:12:16.200 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158604+0000", 2024-09-13T10:12:16.200 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.040419954000000001 2024-09-13T10:12:16.200 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.200 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.200 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.200 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159306+0000", 2024-09-13T10:12:16.201 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00070229299999999999 2024-09-13T10:12:16.201 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.201 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.201 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.201 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159539+0000", 2024-09-13T10:12:16.201 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00023353600000000001 2024-09-13T10:12:16.201 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.201 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.201 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.201 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159696+0000", 2024-09-13T10:12:16.201 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015616 2024-09-13T10:12:16.201 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.202 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.202 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.202 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159754+0000", 2024-09-13T10:12:16.202 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8513e-05 2024-09-13T10:12:16.202 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.202 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.202 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.202 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.159802+0000", 2024-09-13T10:12:16.202 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.7744000000000002e-05 2024-09-13T10:12:16.202 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.202 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.202 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.202 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.202 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.203 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:270 4.5 4:a59e88f7:::smithi19530714-21:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.203 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:12:16.203 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 84.967095428999997, 2024-09-13T10:12:16.203 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041472594000000002, 2024-09-13T10:12:16.203 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.203 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.203 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.203 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.203 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.203 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.203 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 270 2024-09-13T10:12:16.203 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.203 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.204 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.204 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.204 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:12:16.204 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.204 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.204 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.204 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.204 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226035+0000", 2024-09-13T10:12:16.204 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.204 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.204 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.205 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.205 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226037+0000", 2024-09-13T10:12:16.205 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6560000000000001e-06 2024-09-13T10:12:16.205 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.205 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.205 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.205 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226048+0000", 2024-09-13T10:12:16.205 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0893000000000001e-05 2024-09-13T10:12:16.205 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.205 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.205 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.205 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226079+0000", 2024-09-13T10:12:16.205 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0412999999999998e-05 2024-09-13T10:12:16.206 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.206 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.206 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.206 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226106+0000", 2024-09-13T10:12:16.206 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7160000000000001e-05 2024-09-13T10:12:16.206 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.206 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.206 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.206 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226234+0000", 2024-09-13T10:12:16.206 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000128393 2024-09-13T10:12:16.206 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.207 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.207 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.207 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.226388+0000", 2024-09-13T10:12:16.207 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015424300000000001 2024-09-13T10:12:16.207 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.207 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.207 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,10", 2024-09-13T10:12:16.207 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.262647+0000", 2024-09-13T10:12:16.207 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036258460999999999 2024-09-13T10:12:16.207 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.207 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.207 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.208 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.264690+0000", 2024-09-13T10:12:16.208 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020429889999999998 2024-09-13T10:12:16.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.208 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.208 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.208 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.266720+0000", 2024-09-13T10:12:16.208 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002029669 2024-09-13T10:12:16.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.208 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.208 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.208 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267350+0000", 2024-09-13T10:12:16.208 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00063076699999999996 2024-09-13T10:12:16.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.209 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.209 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.209 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267409+0000", 2024-09-13T10:12:16.209 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8366000000000003e-05 2024-09-13T10:12:16.209 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.209 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.209 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.209 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267507+0000", 2024-09-13T10:12:16.209 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.8584000000000003e-05 2024-09-13T10:12:16.209 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.209 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.209 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.209 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.210 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.210 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:275 4.2 4:4ac406d9:::smithi19530714-26:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.210 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:12:16.210 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 84.963348366000005, 2024-09-13T10:12:16.210 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024603505000000001, 2024-09-13T10:12:16.210 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.210 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.210 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.210 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.210 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.210 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.210 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 275 2024-09-13T10:12:16.210 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.211 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.211 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.211 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.211 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:12:16.211 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.211 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.211 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.211 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.211 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229782+0000", 2024-09-13T10:12:16.211 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.211 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.211 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.211 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.212 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229783+0000", 2024-09-13T10:12:16.212 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.44e-06 2024-09-13T10:12:16.212 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.212 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.212 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.212 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229793+0000", 2024-09-13T10:12:16.212 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0163000000000001e-05 2024-09-13T10:12:16.212 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.212 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.212 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.212 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229814+0000", 2024-09-13T10:12:16.212 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0564000000000002e-05 2024-09-13T10:12:16.212 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.212 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.213 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.213 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229831+0000", 2024-09-13T10:12:16.213 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7142e-05 2024-09-13T10:12:16.213 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.213 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.213 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.213 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.229949+0000", 2024-09-13T10:12:16.213 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000118014 2024-09-13T10:12:16.213 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.213 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.213 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.213 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.230081+0000", 2024-09-13T10:12:16.214 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013181299999999999 2024-09-13T10:12:16.214 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.214 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.214 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,14", 2024-09-13T10:12:16.214 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.236541+0000", 2024-09-13T10:12:16.214 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0064597680000000003 2024-09-13T10:12:16.214 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.214 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.214 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.214 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.238773+0000", 2024-09-13T10:12:16.214 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022326680000000002 2024-09-13T10:12:16.214 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.214 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.215 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.215 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.241978+0000", 2024-09-13T10:12:16.215 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00320455 2024-09-13T10:12:16.215 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.215 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.215 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.216 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254159+0000", 2024-09-13T10:12:16.216 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012181149 2024-09-13T10:12:16.216 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.216 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.216 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.216 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254259+0000", 2024-09-13T10:12:16.216 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000100184 2024-09-13T10:12:16.216 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.216 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.216 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.216 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254385+0000", 2024-09-13T10:12:16.216 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012605 2024-09-13T10:12:16.216 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.217 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.217 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.217 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.217 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.217 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:270 4.5 4:a59e88f7:::smithi19530714-21:head [assert-version v9,set-chunk] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.217 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:12:16.217 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 84.959693935999994, 2024-09-13T10:12:16.217 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033985986000000003, 2024-09-13T10:12:16.217 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.217 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.217 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "waiting for ondisk", 2024-09-13T10:12:16.217 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.217 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.218 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.218 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 270 2024-09-13T10:12:16.218 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.218 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.218 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.218 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.218 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:12:16.218 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.218 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.218 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.218 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.218 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233436+0000", 2024-09-13T10:12:16.218 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.219 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.219 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.219 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.219 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233438+0000", 2024-09-13T10:12:16.219 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.187e-06 2024-09-13T10:12:16.219 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.219 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.219 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.219 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233449+0000", 2024-09-13T10:12:16.219 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0926e-05 2024-09-13T10:12:16.219 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.219 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.220 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.220 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233477+0000", 2024-09-13T10:12:16.220 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7515999999999999e-05 2024-09-13T10:12:16.220 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.220 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.220 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.220 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233499+0000", 2024-09-13T10:12:16.220 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1719999999999999e-05 2024-09-13T10:12:16.220 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.220 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.220 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.220 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233670+0000", 2024-09-13T10:12:16.220 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000171146 2024-09-13T10:12:16.221 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.221 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.221 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for blocked object", 2024-09-13T10:12:16.221 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.233823+0000", 2024-09-13T10:12:16.221 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015309300000000001 2024-09-13T10:12:16.221 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.221 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.221 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.222 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.263191+0000", 2024-09-13T10:12:16.222 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029368157999999998 2024-09-13T10:12:16.222 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.222 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.222 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for ondisk", 2024-09-13T10:12:16.222 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.263301+0000", 2024-09-13T10:12:16.222 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110214 2024-09-13T10:12:16.222 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.222 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.222 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.223 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.267422+0000", 2024-09-13T10:12:16.223 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0041210259999999999 2024-09-13T10:12:16.223 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.223 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.223 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.223 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.223 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.223 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:277 4.4 4:20c7201f:::smithi19530714-28:head [assert-version v9,set-chunk] snapc 0=[] RETRY=1 ondisk+retry+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.223 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:12:16.223 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 84.959087851000007, 2024-09-13T10:12:16.223 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.042430906999999997, 2024-09-13T10:12:16.223 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.223 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.224 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.224 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.224 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.224 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.224 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 277 2024-09-13T10:12:16.224 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.224 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.224 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.224 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.224 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:12:16.225 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.225 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.225 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.225 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.225 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234042+0000", 2024-09-13T10:12:16.225 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.225 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.225 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.225 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.225 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234043+0000", 2024-09-13T10:12:16.225 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.13e-06 2024-09-13T10:12:16.225 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.226 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.226 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.226 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234050+0000", 2024-09-13T10:12:16.226 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.263e-06 2024-09-13T10:12:16.226 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.226 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.226 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.226 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234065+0000", 2024-09-13T10:12:16.226 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5207e-05 2024-09-13T10:12:16.226 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.226 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.226 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.226 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.234078+0000", 2024-09-13T10:12:16.226 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3643e-05 2024-09-13T10:12:16.227 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.227 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.227 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.227 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.235383+0000", 2024-09-13T10:12:16.227 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0013048269999999999 2024-09-13T10:12:16.227 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.227 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.227 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.227 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.235604+0000", 2024-09-13T10:12:16.227 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022116900000000001 2024-09-13T10:12:16.227 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.227 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.227 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:12:16.228 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.271455+0000", 2024-09-13T10:12:16.228 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035850173999999999 2024-09-13T10:12:16.228 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.228 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.228 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.228 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.273083+0000", 2024-09-13T10:12:16.228 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0016287649999999999 2024-09-13T10:12:16.228 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.228 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.228 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.228 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.275756+0000", 2024-09-13T10:12:16.228 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0026721140000000002 2024-09-13T10:12:16.228 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.229 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.229 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.229 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276255+0000", 2024-09-13T10:12:16.229 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00049961600000000004 2024-09-13T10:12:16.229 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.229 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.229 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.229 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276401+0000", 2024-09-13T10:12:16.229 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014584099999999999 2024-09-13T10:12:16.229 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.229 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.229 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.229 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.276473+0000", 2024-09-13T10:12:16.230 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2157999999999996e-05 2024-09-13T10:12:16.230 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.230 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.230 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.230 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.230 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.230 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:324 4.4 4:26b855f3:::smithi19530714-25:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.230 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:12:16.230 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 84.912425604999996, 2024-09-13T10:12:16.230 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033888489000000001, 2024-09-13T10:12:16.230 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.230 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.230 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.231 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.231 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.231 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.231 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 324 2024-09-13T10:12:16.231 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.231 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.231 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.231 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.231 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:12:16.231 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.231 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.231 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.231 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.231 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280704+0000", 2024-09-13T10:12:16.232 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.232 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.232 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.232 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.232 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280706+0000", 2024-09-13T10:12:16.232 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6479999999999999e-06 2024-09-13T10:12:16.232 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.232 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.232 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.233 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280715+0000", 2024-09-13T10:12:16.233 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1500000000000005e-06 2024-09-13T10:12:16.233 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.233 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.233 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.233 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280733+0000", 2024-09-13T10:12:16.233 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8168e-05 2024-09-13T10:12:16.233 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.233 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.234 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.234 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.280747+0000", 2024-09-13T10:12:16.234 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3661e-05 2024-09-13T10:12:16.234 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.234 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.234 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.234 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281013+0000", 2024-09-13T10:12:16.234 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026565099999999999 2024-09-13T10:12:16.234 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.234 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.235 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.235 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281124+0000", 2024-09-13T10:12:16.235 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000111715 2024-09-13T10:12:16.235 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.235 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.235 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:12:16.235 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.286502+0000", 2024-09-13T10:12:16.235 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0053775259999999997 2024-09-13T10:12:16.235 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.235 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.235 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.235 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.289334+0000", 2024-09-13T10:12:16.235 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0028316779999999998 2024-09-13T10:12:16.236 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.236 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.236 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.236 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.290331+0000", 2024-09-13T10:12:16.236 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0009974929999999999 2024-09-13T10:12:16.236 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.236 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.236 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.236 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314453+0000", 2024-09-13T10:12:16.236 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024121417999999999 2024-09-13T10:12:16.236 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.237 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.237 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.237 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314531+0000", 2024-09-13T10:12:16.237 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8841e-05 2024-09-13T10:12:16.237 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.237 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.237 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.237 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314593+0000", 2024-09-13T10:12:16.237 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.1539999999999997e-05 2024-09-13T10:12:16.237 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.237 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.237 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.237 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.238 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.238 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:327 4.4 4:20c7201f:::smithi19530714-28:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.238 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:12:16.238 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 84.911957775999994, 2024-09-13T10:12:16.238 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035855512999999999, 2024-09-13T10:12:16.238 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.238 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.238 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.238 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.238 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.238 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.238 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 327 2024-09-13T10:12:16.238 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.239 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.239 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.239 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.239 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:12:16.239 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.239 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.239 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.239 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.239 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281172+0000", 2024-09-13T10:12:16.239 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.239 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.239 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.239 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.240 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281173+0000", 2024-09-13T10:12:16.240 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0020000000000001e-06 2024-09-13T10:12:16.240 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.240 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.240 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.240 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281180+0000", 2024-09-13T10:12:16.240 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.8260000000000004e-06 2024-09-13T10:12:16.240 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.240 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.240 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.240 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281192+0000", 2024-09-13T10:12:16.240 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1725e-05 2024-09-13T10:12:16.240 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.241 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.241 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.241 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281202+0000", 2024-09-13T10:12:16.241 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0492000000000001e-05 2024-09-13T10:12:16.241 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.241 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.241 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.241 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281396+0000", 2024-09-13T10:12:16.241 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019404200000000001 2024-09-13T10:12:16.241 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.241 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.241 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.241 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.281533+0000", 2024-09-13T10:12:16.242 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000136267 2024-09-13T10:12:16.242 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.242 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.242 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:12:16.242 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.313834+0000", 2024-09-13T10:12:16.242 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032301491000000002 2024-09-13T10:12:16.242 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.242 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.242 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.242 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314993+0000", 2024-09-13T10:12:16.242 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001158585 2024-09-13T10:12:16.242 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.242 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.242 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.243 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.315097+0000", 2024-09-13T10:12:16.243 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000104685 2024-09-13T10:12:16.243 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.243 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.243 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.243 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.316808+0000", 2024-09-13T10:12:16.243 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0017110529999999999 2024-09-13T10:12:16.243 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.243 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.243 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.243 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.316946+0000", 2024-09-13T10:12:16.243 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000137737 2024-09-13T10:12:16.243 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.244 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.244 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.244 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.317028+0000", 2024-09-13T10:12:16.244 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.1607999999999995e-05 2024-09-13T10:12:16.244 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.244 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.244 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.244 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.244 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.244 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:335 4.2 4:4a24ecac:::smithi19530714-36:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.244 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:12:16.244 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 84.910639029999999, 2024-09-13T10:12:16.244 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032260373000000002, 2024-09-13T10:12:16.245 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.245 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.245 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.245 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.245 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.245 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.245 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 335 2024-09-13T10:12:16.245 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.245 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.245 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.245 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.245 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:12:16.245 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.246 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.246 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.246 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.246 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282491+0000", 2024-09-13T10:12:16.246 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.246 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.246 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.246 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.246 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282494+0000", 2024-09-13T10:12:16.246 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5390000000000002e-06 2024-09-13T10:12:16.246 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.246 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.247 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.247 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282506+0000", 2024-09-13T10:12:16.247 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2646000000000001e-05 2024-09-13T10:12:16.247 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.247 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.247 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.247 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282539+0000", 2024-09-13T10:12:16.247 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3176000000000002e-05 2024-09-13T10:12:16.247 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.247 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.247 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.247 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282558+0000", 2024-09-13T10:12:16.247 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9065000000000001e-05 2024-09-13T10:12:16.248 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.248 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.248 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.248 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282697+0000", 2024-09-13T10:12:16.248 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013857999999999999 2024-09-13T10:12:16.248 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.248 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.248 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.248 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.282874+0000", 2024-09-13T10:12:16.248 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017656900000000001 2024-09-13T10:12:16.248 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.248 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.249 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 8,14", 2024-09-13T10:12:16.249 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.287911+0000", 2024-09-13T10:12:16.249 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0050369220000000001 2024-09-13T10:12:16.249 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.249 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.249 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.249 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.290491+0000", 2024-09-13T10:12:16.249 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025802500000000001 2024-09-13T10:12:16.249 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.249 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.249 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.249 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.291374+0000", 2024-09-13T10:12:16.249 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00088367800000000005 2024-09-13T10:12:16.250 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.250 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.250 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.250 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314557+0000", 2024-09-13T10:12:16.250 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023182094 2024-09-13T10:12:16.250 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.250 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.250 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.250 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314664+0000", 2024-09-13T10:12:16.250 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010751099999999999 2024-09-13T10:12:16.250 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.250 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.250 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.251 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314751+0000", 2024-09-13T10:12:16.251 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7342999999999994e-05 2024-09-13T10:12:16.251 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.251 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.251 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.251 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.251 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.251 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:337 4.4 4:24275569:::smithi19530714-38:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.251 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:12:16.251 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 84.909982479000007, 2024-09-13T10:12:16.251 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.031660147, 2024-09-13T10:12:16.251 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.252 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.252 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.252 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.252 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.252 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.252 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 337 2024-09-13T10:12:16.252 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.252 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.252 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.252 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.252 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:12:16.252 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.252 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.253 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.253 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.253 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283148+0000", 2024-09-13T10:12:16.253 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.253 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.253 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.253 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.253 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283149+0000", 2024-09-13T10:12:16.253 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.655e-06 2024-09-13T10:12:16.253 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.253 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.253 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.253 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283159+0000", 2024-09-13T10:12:16.253 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.9529999999999994e-06 2024-09-13T10:12:16.254 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.254 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.254 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.254 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283177+0000", 2024-09-13T10:12:16.254 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8168e-05 2024-09-13T10:12:16.254 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.254 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.254 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.254 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283193+0000", 2024-09-13T10:12:16.254 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5563e-05 2024-09-13T10:12:16.254 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.254 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.255 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.255 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283317+0000", 2024-09-13T10:12:16.255 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012421700000000001 2024-09-13T10:12:16.255 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.255 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.255 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.255 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.283478+0000", 2024-09-13T10:12:16.255 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000160378 2024-09-13T10:12:16.255 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.255 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.255 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:12:16.255 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.289593+0000", 2024-09-13T10:12:16.255 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0061152200000000002 2024-09-13T10:12:16.255 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.256 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.256 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.256 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.291636+0000", 2024-09-13T10:12:16.256 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020435430000000001 2024-09-13T10:12:16.256 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.256 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.256 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.256 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.294186+0000", 2024-09-13T10:12:16.256 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025494760000000002 2024-09-13T10:12:16.256 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.256 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.256 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.257 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314672+0000", 2024-09-13T10:12:16.257 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020486392999999999 2024-09-13T10:12:16.257 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.257 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.257 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.257 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314741+0000", 2024-09-13T10:12:16.257 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9003000000000003e-05 2024-09-13T10:12:16.257 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.257 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.257 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.257 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.314808+0000", 2024-09-13T10:12:16.257 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.6577999999999999e-05 2024-09-13T10:12:16.257 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.258 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.258 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.258 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.258 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.258 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1865 4.12 4:4ac406d9:::smithi19530714-26:head [read 0~2050454,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:16.258 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.728406+0000", 2024-09-13T10:12:16.258 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 64.464723903999996, 2024-09-13T10:12:16.258 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029384652000000001, 2024-09-13T10:12:16.258 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.258 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.258 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:16.258 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.258 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.259 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.259 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1865 2024-09-13T10:12:16.259 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.259 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.259 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.259 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.259 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728406+0000", 2024-09-13T10:12:16.259 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.259 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.259 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.259 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.259 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728406+0000", 2024-09-13T10:12:16.259 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.260 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.260 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.260 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.260 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728410+0000", 2024-09-13T10:12:16.260 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9129999999999996e-06 2024-09-13T10:12:16.260 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.260 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.260 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.260 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728423+0000", 2024-09-13T10:12:16.260 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2425999999999999e-05 2024-09-13T10:12:16.260 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.260 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.261 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.261 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728453+0000", 2024-09-13T10:12:16.261 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0673999999999998e-05 2024-09-13T10:12:16.261 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.261 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.261 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.261 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728478+0000", 2024-09-13T10:12:16.261 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5099000000000001e-05 2024-09-13T10:12:16.261 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.261 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.261 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.261 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730729+0000", 2024-09-13T10:12:16.261 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002251009 2024-09-13T10:12:16.262 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.262 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.262 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:12:16.262 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730758+0000", 2024-09-13T10:12:16.262 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8623e-05 2024-09-13T10:12:16.262 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.262 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.262 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.262 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.754517+0000", 2024-09-13T10:12:16.262 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023759199000000002 2024-09-13T10:12:16.262 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.262 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.262 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.263 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.754761+0000", 2024-09-13T10:12:16.263 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000243923 2024-09-13T10:12:16.263 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.263 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.263 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.263 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.757791+0000", 2024-09-13T10:12:16.263 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003029786 2024-09-13T10:12:16.263 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.263 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.263 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.263 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.263 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.263 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1866 4.12 4:4ac406d9:::smithi19530714-26:head [sparse-read 0~1216897] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:16.264 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.728515+0000", 2024-09-13T10:12:16.264 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 64.464614849, 2024-09-13T10:12:16.264 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.031206358999999999, 2024-09-13T10:12:16.264 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.264 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.264 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:16.264 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.264 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.264 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.264 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1866 2024-09-13T10:12:16.264 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.264 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.264 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.265 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.265 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728515+0000", 2024-09-13T10:12:16.265 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.265 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.265 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.265 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.265 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728515+0000", 2024-09-13T10:12:16.265 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.265 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.265 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.265 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.265 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728516+0000", 2024-09-13T10:12:16.266 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1429999999999999e-06 2024-09-13T10:12:16.266 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.266 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.266 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.266 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728522+0000", 2024-09-13T10:12:16.266 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5740000000000003e-06 2024-09-13T10:12:16.266 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.266 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.266 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.266 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728534+0000", 2024-09-13T10:12:16.266 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1539000000000001e-05 2024-09-13T10:12:16.266 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.267 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.267 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.267 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728543+0000", 2024-09-13T10:12:16.267 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6059999999999992e-06 2024-09-13T10:12:16.267 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.267 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.267 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.267 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730840+0000", 2024-09-13T10:12:16.267 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0022966789999999998 2024-09-13T10:12:16.267 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.267 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.267 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:12:16.267 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730859+0000", 2024-09-13T10:12:16.268 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9613e-05 2024-09-13T10:12:16.268 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.268 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.268 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.268 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.758405+0000", 2024-09-13T10:12:16.268 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027545656000000002 2024-09-13T10:12:16.268 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.268 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.268 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.268 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.758574+0000", 2024-09-13T10:12:16.268 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000168808 2024-09-13T10:12:16.268 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.268 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.269 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.269 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759722+0000", 2024-09-13T10:12:16.269 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011477410000000001 2024-09-13T10:12:16.269 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.269 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.269 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.269 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.269 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.269 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1867 4.12 4:4ac406d9:::smithi19530714-26:head [read 116628~1933826,checksum] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:16.269 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.728557+0000", 2024-09-13T10:12:16.269 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 64.464573367, 2024-09-13T10:12:16.269 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.033656772000000001, 2024-09-13T10:12:16.270 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.270 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.270 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:16.270 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.270 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.270 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.270 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1867 2024-09-13T10:12:16.270 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.270 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.270 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.270 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.270 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728557+0000", 2024-09-13T10:12:16.270 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.271 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.271 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.271 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.271 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728557+0000", 2024-09-13T10:12:16.271 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.271 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.271 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.271 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.271 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728558+0000", 2024-09-13T10:12:16.271 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8400000000000003e-07 2024-09-13T10:12:16.271 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.271 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.271 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.272 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728562+0000", 2024-09-13T10:12:16.272 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.8029999999999996e-06 2024-09-13T10:12:16.272 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.272 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.272 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.272 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728589+0000", 2024-09-13T10:12:16.272 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6536000000000001e-05 2024-09-13T10:12:16.272 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.272 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.272 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.272 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.728603+0000", 2024-09-13T10:12:16.272 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4483e-05 2024-09-13T10:12:16.272 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.273 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.273 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.273 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730933+0000", 2024-09-13T10:12:16.273 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023295939999999999 2024-09-13T10:12:16.273 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.273 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.273 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for peered", 2024-09-13T10:12:16.273 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.730952+0000", 2024-09-13T10:12:16.273 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8614000000000002e-05 2024-09-13T10:12:16.273 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.273 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.273 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.273 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759881+0000", 2024-09-13T10:12:16.274 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.02892962 2024-09-13T10:12:16.274 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.274 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.274 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.274 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.760061+0000", 2024-09-13T10:12:16.274 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00018033 2024-09-13T10:12:16.274 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.274 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.274 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.274 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.762214+0000", 2024-09-13T10:12:16.274 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002152008 2024-09-13T10:12:16.274 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.274 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.274 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.275 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.275 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.275 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1932 4.5 4:a3098556:::smithi19530714-22:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:12:16.275 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:11.759351+0000", 2024-09-13T10:12:16.275 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 64.433778954000005, 2024-09-13T10:12:16.275 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.043979880999999998, 2024-09-13T10:12:16.275 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.275 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.275 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:16.275 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.275 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.275 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.275 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1932 2024-09-13T10:12:16.275 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.276 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.276 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.276 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.276 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759351+0000", 2024-09-13T10:12:16.276 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.276 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.276 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.276 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.276 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759351+0000", 2024-09-13T10:12:16.276 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.276 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.276 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.277 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.277 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759352+0000", 2024-09-13T10:12:16.277 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.28e-06 2024-09-13T10:12:16.277 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.277 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.277 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.277 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759358+0000", 2024-09-13T10:12:16.277 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8699999999999997e-06 2024-09-13T10:12:16.277 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.277 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.277 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.277 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759376+0000", 2024-09-13T10:12:16.277 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7532000000000001e-05 2024-09-13T10:12:16.278 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.278 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.278 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.278 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759391+0000", 2024-09-13T10:12:16.278 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5590000000000002e-05 2024-09-13T10:12:16.278 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.278 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.278 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.278 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759607+0000", 2024-09-13T10:12:16.278 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021536999999999999 2024-09-13T10:12:16.278 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.278 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.278 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.279 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.759831+0000", 2024-09-13T10:12:16.279 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000224243 2024-09-13T10:12:16.279 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.279 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.279 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.279 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:11.803331+0000", 2024-09-13T10:12:16.279 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.043499995999999999 2024-09-13T10:12:16.279 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.279 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.279 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.279 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.279 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.279 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2189 4.4 4:26b855f3:::smithi19530714-25:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e59)", 2024-09-13T10:12:16.280 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:17.462885+0000", 2024-09-13T10:12:16.280 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 58.730245248000003, 2024-09-13T10:12:16.280 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041082475, 2024-09-13T10:12:16.280 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.280 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.280 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:16.280 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.280 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.280 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.280 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2189 2024-09-13T10:12:16.280 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.280 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.280 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.280 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.281 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462885+0000", 2024-09-13T10:12:16.281 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.281 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.281 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.281 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.281 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462885+0000", 2024-09-13T10:12:16.281 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.281 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.281 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.281 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.281 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462888+0000", 2024-09-13T10:12:16.281 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.74e-06 2024-09-13T10:12:16.282 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.282 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.282 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.282 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462898+0000", 2024-09-13T10:12:16.282 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0618999999999999e-05 2024-09-13T10:12:16.282 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.282 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.282 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.282 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462928+0000", 2024-09-13T10:12:16.282 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9685000000000001e-05 2024-09-13T10:12:16.282 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.282 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.282 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.283 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.462944+0000", 2024-09-13T10:12:16.283 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6175999999999999e-05 2024-09-13T10:12:16.283 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.283 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.283 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.283 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.463063+0000", 2024-09-13T10:12:16.283 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000119393 2024-09-13T10:12:16.283 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.283 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.283 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.283 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.463204+0000", 2024-09-13T10:12:16.283 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014020600000000001 2024-09-13T10:12:16.284 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.284 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.284 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.284 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:17.503967+0000", 2024-09-13T10:12:16.284 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.040763656000000002 2024-09-13T10:12:16.284 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.284 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.284 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.284 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.284 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.284 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2832 4.4 4:26b855f3:::smithi19530714-25:head [read 0~3613505,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:12:16.284 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.892960+0000", 2024-09-13T10:12:16.285 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 43.300169982, 2024-09-13T10:12:16.285 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.026786087, 2024-09-13T10:12:16.285 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.285 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.285 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:16.285 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.285 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.285 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.285 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2832 2024-09-13T10:12:16.285 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.285 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.285 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.285 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.286 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892960+0000", 2024-09-13T10:12:16.286 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.287 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.288 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.288 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.288 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892960+0000", 2024-09-13T10:12:16.288 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.288 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.288 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.288 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.288 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892962+0000", 2024-09-13T10:12:16.288 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.573e-06 2024-09-13T10:12:16.288 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.288 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.288 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.289 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892969+0000", 2024-09-13T10:12:16.289 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.277e-06 2024-09-13T10:12:16.289 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.289 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.289 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.289 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892984+0000", 2024-09-13T10:12:16.289 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5013e-05 2024-09-13T10:12:16.289 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.289 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.289 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.289 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.892997+0000", 2024-09-13T10:12:16.289 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2692e-05 2024-09-13T10:12:16.290 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.290 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.290 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.290 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.917535+0000", 2024-09-13T10:12:16.290 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024538806 2024-09-13T10:12:16.290 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.290 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.290 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.290 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.917788+0000", 2024-09-13T10:12:16.290 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025253100000000001 2024-09-13T10:12:16.290 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.290 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.291 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.291 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.919746+0000", 2024-09-13T10:12:16.291 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0019581949999999998 2024-09-13T10:12:16.291 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.291 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.291 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.291 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.291 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.291 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2833 4.4 4:26b855f3:::smithi19530714-25:head [read 2395497~1218008,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:12:16.291 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.893015+0000", 2024-09-13T10:12:16.291 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 43.300115177000002, 2024-09-13T10:12:16.291 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027607530000000002, 2024-09-13T10:12:16.291 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.292 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.292 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:16.292 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.292 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.292 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.292 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2833 2024-09-13T10:12:16.292 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.292 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.292 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.292 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.292 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893015+0000", 2024-09-13T10:12:16.292 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.292 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.293 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.293 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.293 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893015+0000", 2024-09-13T10:12:16.293 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.293 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.293 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.293 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.293 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893016+0000", 2024-09-13T10:12:16.293 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.156e-06 2024-09-13T10:12:16.293 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.293 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.293 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.293 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893024+0000", 2024-09-13T10:12:16.294 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5900000000000002e-06 2024-09-13T10:12:16.294 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.294 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.294 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.294 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893039+0000", 2024-09-13T10:12:16.294 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4949999999999999e-05 2024-09-13T10:12:16.294 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.294 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.294 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.294 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893052+0000", 2024-09-13T10:12:16.294 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3362e-05 2024-09-13T10:12:16.294 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.295 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.295 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.295 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.919791+0000", 2024-09-13T10:12:16.295 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.026739237999999999 2024-09-13T10:12:16.295 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.295 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.295 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.295 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.919930+0000", 2024-09-13T10:12:16.295 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000139202 2024-09-13T10:12:16.295 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.295 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.295 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.295 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.920622+0000", 2024-09-13T10:12:16.296 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00069203199999999998 2024-09-13T10:12:16.296 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.296 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.296 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.296 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.296 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.296 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2840 4.4 4:24da8222:::smithi19530714-49:head [sparse-read 2104672~451177,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:12:16.296 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.893098+0000", 2024-09-13T10:12:16.296 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 43.300031609999998, 2024-09-13T10:12:16.296 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029549518, 2024-09-13T10:12:16.296 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.296 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.297 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:16.297 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.297 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.297 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.297 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2840 2024-09-13T10:12:16.297 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.297 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.297 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.297 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.297 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893098+0000", 2024-09-13T10:12:16.297 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.297 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.298 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.298 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.298 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893098+0000", 2024-09-13T10:12:16.298 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.298 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.298 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.298 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.298 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893107+0000", 2024-09-13T10:12:16.298 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7430000000000008e-06 2024-09-13T10:12:16.298 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.298 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.298 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.298 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893117+0000", 2024-09-13T10:12:16.299 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7000000000000003e-06 2024-09-13T10:12:16.299 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.299 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.299 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.299 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893136+0000", 2024-09-13T10:12:16.299 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8705000000000001e-05 2024-09-13T10:12:16.299 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.299 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.299 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.299 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893151+0000", 2024-09-13T10:12:16.299 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5183e-05 2024-09-13T10:12:16.299 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.300 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.300 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.300 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.920721+0000", 2024-09-13T10:12:16.300 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027569917999999999 2024-09-13T10:12:16.300 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.300 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.300 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.300 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.920886+0000", 2024-09-13T10:12:16.300 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016508699999999999 2024-09-13T10:12:16.300 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.300 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.300 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.300 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.922648+0000", 2024-09-13T10:12:16.301 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0017621819999999999 2024-09-13T10:12:16.301 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.301 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.301 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.301 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.301 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.301 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2841 4.4 4:24da8222:::smithi19530714-49:head [read 0~3042067,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:12:16.301 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.893173+0000", 2024-09-13T10:12:16.301 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 43.299957188999997, 2024-09-13T10:12:16.301 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.037712765000000002, 2024-09-13T10:12:16.301 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.301 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.302 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:16.302 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.302 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.302 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.302 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2841 2024-09-13T10:12:16.302 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.302 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.302 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.302 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.302 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893173+0000", 2024-09-13T10:12:16.302 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.302 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.302 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.303 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.303 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893173+0000", 2024-09-13T10:12:16.303 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.303 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.303 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.303 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.303 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893174+0000", 2024-09-13T10:12:16.303 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4959999999999999e-06 2024-09-13T10:12:16.303 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.303 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.303 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.303 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893182+0000", 2024-09-13T10:12:16.303 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7060000000000004e-06 2024-09-13T10:12:16.304 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.304 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.304 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.304 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893201+0000", 2024-09-13T10:12:16.304 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8655999999999999e-05 2024-09-13T10:12:16.304 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.304 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.304 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.304 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893215+0000", 2024-09-13T10:12:16.304 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4513e-05 2024-09-13T10:12:16.304 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.304 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.304 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.305 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.922869+0000", 2024-09-13T10:12:16.305 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029653845000000002 2024-09-13T10:12:16.305 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.305 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.305 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.305 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.923431+0000", 2024-09-13T10:12:16.305 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00056212899999999997 2024-09-13T10:12:16.305 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.305 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.305 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.305 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.930886+0000", 2024-09-13T10:12:16.305 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0074544199999999998 2024-09-13T10:12:16.306 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.306 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.306 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.306 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.306 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.306 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:2842 4.4 4:24da8222:::smithi19530714-49:head [sparse-read 0~1861544] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e72)", 2024-09-13T10:12:16.306 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:32.893233+0000", 2024-09-13T10:12:16.306 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 43.299896709000002, 2024-09-13T10:12:16.306 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.038772649999999999, 2024-09-13T10:12:16.306 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.306 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.306 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:16.306 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.306 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.307 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.307 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 2842 2024-09-13T10:12:16.307 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.307 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.307 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.307 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.307 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893233+0000", 2024-09-13T10:12:16.307 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.307 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.307 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.307 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.307 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893233+0000", 2024-09-13T10:12:16.307 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.307 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.308 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.308 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.308 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893235+0000", 2024-09-13T10:12:16.308 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.217e-06 2024-09-13T10:12:16.308 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.308 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.308 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.308 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893240+0000", 2024-09-13T10:12:16.308 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.6999999999999996e-06 2024-09-13T10:12:16.308 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.308 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.308 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.308 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893254+0000", 2024-09-13T10:12:16.308 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.401e-05 2024-09-13T10:12:16.309 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.309 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.309 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.309 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.893267+0000", 2024-09-13T10:12:16.309 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3135e-05 2024-09-13T10:12:16.309 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.309 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.309 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.309 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.930989+0000", 2024-09-13T10:12:16.309 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.037721452000000003 2024-09-13T10:12:16.309 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.309 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.309 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.310 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.931214+0000", 2024-09-13T10:12:16.310 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000225529 2024-09-13T10:12:16.310 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.310 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.310 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.310 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:32.932006+0000", 2024-09-13T10:12:16.310 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00079160700000000005 2024-09-13T10:12:16.310 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.310 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.310 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.310 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.310 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.310 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:4362 4.4 4:25c904b0:::smithi19530714-45:head [write 672379~724209] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-13T10:12:16.310 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:09.869982+0000", 2024-09-13T10:12:16.311 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 6.3231482870000004, 2024-09-13T10:12:16.311 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024621703000000002, 2024-09-13T10:12:16.311 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.311 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.311 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.311 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.311 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.311 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.311 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 4362 2024-09-13T10:12:16.311 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.311 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.311 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.311 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.312 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.869982+0000", 2024-09-13T10:12:16.312 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.312 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.312 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.312 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.312 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.869982+0000", 2024-09-13T10:12:16.312 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.312 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.312 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.312 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.312 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.869984+0000", 2024-09-13T10:12:16.313 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3630000000000001e-06 2024-09-13T10:12:16.313 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.313 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.313 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.313 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.871451+0000", 2024-09-13T10:12:16.313 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0014671840000000001 2024-09-13T10:12:16.313 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.313 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.313 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.313 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.871477+0000", 2024-09-13T10:12:16.313 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5556000000000001e-05 2024-09-13T10:12:16.313 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.313 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.314 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.314 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.871498+0000", 2024-09-13T10:12:16.314 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1212999999999998e-05 2024-09-13T10:12:16.314 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.314 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.314 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.314 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.883074+0000", 2024-09-13T10:12:16.314 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011576041 2024-09-13T10:12:16.314 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.314 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.314 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.314 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.883304+0000", 2024-09-13T10:12:16.314 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000229559 2024-09-13T10:12:16.315 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.315 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.315 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 5,10", 2024-09-13T10:12:16.315 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.883580+0000", 2024-09-13T10:12:16.315 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00027630200000000003 2024-09-13T10:12:16.315 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.315 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.315 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.315 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.894212+0000", 2024-09-13T10:12:16.315 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010632058999999999 2024-09-13T10:12:16.315 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.315 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.316 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.316 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.894258+0000", 2024-09-13T10:12:16.316 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.5908999999999997e-05 2024-09-13T10:12:16.316 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.316 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.316 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.316 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.894360+0000", 2024-09-13T10:12:16.316 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010241799999999999 2024-09-13T10:12:16.316 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.316 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.316 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.316 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.894483+0000", 2024-09-13T10:12:16.317 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012225100000000001 2024-09-13T10:12:16.317 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.317 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.317 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.317 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.894604+0000", 2024-09-13T10:12:16.317 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012084800000000001 2024-09-13T10:12:16.317 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.317 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.317 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.317 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.317 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.317 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:16.320 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-13T10:12:16.320 INFO:tasks.rados.rados.0.smithi195.stdout:1897: writing smithi19530714-46 from 642133 to 1370879 tid 1 2024-09-13T10:12:16.320 INFO:tasks.rados.rados.0.smithi195.stdout:1897: writing smithi19530714-46 from 2119014 to 2651492 tid 2 2024-09-13T10:12:16.321 INFO:tasks.rados.rados.0.smithi195.stdout:1898: read oid 50 snap -1 2024-09-13T10:12:16.321 INFO:tasks.rados.rados.0.smithi195.stdout:1898: expect (ObjNum 303 snap 0 seq_num 303) 2024-09-13T10:12:16.321 INFO:tasks.rados.rados.0.smithi195.stdout:1899: tier_promote oid 23 2024-09-13T10:12:16.321 INFO:tasks.rados.rados.0.smithi195.stdout:1894: got (2) No such file or directory 2024-09-13T10:12:16.321 INFO:tasks.rados.rados.0.smithi195.stdout:1894: got expected ENOENT (src dne) 2024-09-13T10:12:16.321 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 68 (ObjNum 49 snap 0 seq_num 1953066355) clean dne 2024-09-13T10:12:16.321 INFO:tasks.rados.rados.0.smithi195.stdout:1896: expect (ObjNum 344 snap 0 seq_num 344) 2024-09-13T10:12:16.321 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:16.313+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:16.337 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3556901 2024-09-13T10:12:16.395 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3731266 2024-09-13T10:12:16.398 INFO:tasks.rados.rados.0.smithi195.stdout:1895: expect (ObjNum 377 snap 0 seq_num 377) 2024-09-13T10:12:16.399 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:16.420 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:12:16.419+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:12:16.450 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:16.450 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:12:16.450 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:12:16.450 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:16.460 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:16.464 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:12:16.465 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-13T10:12:16.494 INFO:tasks.rados.rados.0.smithi195.stdout:1897: finishing write tid 1 to smithi19530714-46 2024-09-13T10:12:16.494 INFO:tasks.rados.rados.0.smithi195.stdout:1897: finishing write tid 2 to smithi19530714-46 2024-09-13T10:12:16.494 INFO:tasks.rados.rados.0.smithi195.stdout:1897: finishing write tid 3 to smithi19530714-46 2024-09-13T10:12:16.495 INFO:tasks.rados.rados.0.smithi195.stdout:1897: finishing write tid 4 to smithi19530714-46 2024-09-13T10:12:16.495 INFO:tasks.rados.rados.0.smithi195.stdout:1897: oid 46 updating version 0 to 71 2024-09-13T10:12:16.495 INFO:tasks.rados.rados.0.smithi195.stdout:1897: oid 46 updating version 71 to 73 2024-09-13T10:12:16.495 INFO:tasks.rados.rados.0.smithi195.stdout:1897: oid 46 version 73 is already newer than 72 2024-09-13T10:12:16.495 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 73 (ObjNum 419 snap 0 seq_num 419) dirty exists 2024-09-13T10:12:16.495 INFO:tasks.rados.rados.0.smithi195.stdout:1897: left oid 46 (ObjNum 419 snap 0 seq_num 419) 2024-09-13T10:12:16.495 INFO:tasks.rados.rados.0.smithi195.stdout:1899: got (0) Success 2024-09-13T10:12:16.495 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 42 (ObjNum 286 snap 0 seq_num 286) dirty exists 2024-09-13T10:12:16.495 INFO:tasks.rados.rados.0.smithi195.stdout:1898: expect (ObjNum 303 snap 0 seq_num 303) 2024-09-13T10:12:16.507 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1068271 2024-09-13T10:12:16.521 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:12:16.519+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:12:16.555 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:16.581 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:16.581 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:12:16.581 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:12:16.582 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:12:16.582 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:16.596 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-13T10:12:16.619 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:12:16.619 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:12:16.622 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:16.617+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:16.640 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:12:16.640 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:12:16.657 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:12:16.658 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:12:16.678 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:12:16.678 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:12:16.684 INFO:tasks.rados.rados.0.smithi195.stdout:1894: done (6 left) 2024-09-13T10:12:16.684 INFO:tasks.rados.rados.0.smithi195.stdout:1895: done (5 left) 2024-09-13T10:12:16.684 INFO:tasks.rados.rados.0.smithi195.stdout:1896: done (4 left) 2024-09-13T10:12:16.684 INFO:tasks.rados.rados.0.smithi195.stdout:1897: done (3 left) 2024-09-13T10:12:16.684 INFO:tasks.rados.rados.0.smithi195.stdout:1898: done (2 left) 2024-09-13T10:12:16.684 INFO:tasks.rados.rados.0.smithi195.stdout:1899: done (1 left) 2024-09-13T10:12:16.684 INFO:tasks.rados.rados.0.smithi195.stdout:1900: write oid 40 current snap is 0 2024-09-13T10:12:16.684 INFO:tasks.rados.rados.0.smithi195.stdout:1900: seq_num 420 ranges {658940=403995,1777496=676136,2950951=107989} 2024-09-13T10:12:16.697 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:12:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:12:16.701 INFO:tasks.rados.rados.0.smithi195.stdout:1900: writing smithi19530714-40 from 658940 to 1062935 tid 1 2024-09-13T10:12:16.718 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:12:16.718 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:12:16.719 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:16.720 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:12:16.720 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:12:16.720 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:12:16.720 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.720 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:201 3.3 3:d4fb61be:::smithi19530714-2 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.720 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:12:16.720 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 85.722092629000002, 2024-09-13T10:12:16.720 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.052937043000000003, 2024-09-13T10:12:16.720 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.720 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.720 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.721 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.721 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.721 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.721 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 201 2024-09-13T10:12:16.721 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.721 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.721 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.721 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.721 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:12:16.721 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.721 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.721 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.722 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.722 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:12:16.722 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.722 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.722 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.722 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.722 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993192+0000", 2024-09-13T10:12:16.722 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8559999999999996e-06 2024-09-13T10:12:16.722 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.722 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.722 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.722 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993203+0000", 2024-09-13T10:12:16.723 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1113e-05 2024-09-13T10:12:16.723 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.723 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.723 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.723 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993225+0000", 2024-09-13T10:12:16.723 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2793e-05 2024-09-13T10:12:16.723 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.723 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.723 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.723 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993241+0000", 2024-09-13T10:12:16.723 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5806000000000001e-05 2024-09-13T10:12:16.723 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.724 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.724 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.724 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993401+0000", 2024-09-13T10:12:16.724 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000160149 2024-09-13T10:12:16.724 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.724 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.724 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.724 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994047+0000", 2024-09-13T10:12:16.724 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00064594100000000003 2024-09-13T10:12:16.724 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.724 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.011243+0000", 2024-09-13T10:12:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017196026999999999 2024-09-13T10:12:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029096+0000", 2024-09-13T10:12:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017852115000000002 2024-09-13T10:12:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029271+0000", 2024-09-13T10:12:16.725 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017589299999999999 2024-09-13T10:12:16.726 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.726 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.726 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.726 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.045867+0000", 2024-09-13T10:12:16.726 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016595625999999999 2024-09-13T10:12:16.726 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.726 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.726 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.726 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.046017+0000", 2024-09-13T10:12:16.726 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014974199999999999 2024-09-13T10:12:16.726 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.726 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.727 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.727 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.046120+0000", 2024-09-13T10:12:16.727 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000102982 2024-09-13T10:12:16.727 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.727 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.727 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.727 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.727 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.727 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:208 3.3 3:dbb0d8aa:::smithi19530714-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.727 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:12:16.727 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 85.721996818999997, 2024-09-13T10:12:16.727 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.071216475000000001, 2024-09-13T10:12:16.727 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.728 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.728 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.728 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.728 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.728 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.728 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 208 2024-09-13T10:12:16.728 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.728 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.728 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.728 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.728 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:12:16.728 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.728 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.729 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.729 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.729 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:12:16.729 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.729 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.729 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.729 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.729 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993280+0000", 2024-09-13T10:12:16.729 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1999999999999999e-06 2024-09-13T10:12:16.729 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.729 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.729 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.729 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993289+0000", 2024-09-13T10:12:16.730 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8200000000000003e-06 2024-09-13T10:12:16.730 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.730 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.730 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.730 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994000+0000", 2024-09-13T10:12:16.730 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00071192499999999997 2024-09-13T10:12:16.730 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.730 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.730 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.730 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994024+0000", 2024-09-13T10:12:16.730 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3578999999999999e-05 2024-09-13T10:12:16.730 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.730 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.730 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.731 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994468+0000", 2024-09-13T10:12:16.731 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00044410599999999999 2024-09-13T10:12:16.731 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.731 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.731 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.731 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994842+0000", 2024-09-13T10:12:16.731 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037380300000000003 2024-09-13T10:12:16.731 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.731 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.731 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.731 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031572+0000", 2024-09-13T10:12:16.731 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036730479000000003 2024-09-13T10:12:16.731 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.731 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.731 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.732 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.039007+0000", 2024-09-13T10:12:16.732 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.007434871 2024-09-13T10:12:16.732 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.732 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.732 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.732 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.041001+0000", 2024-09-13T10:12:16.732 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001994104 2024-09-13T10:12:16.732 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.732 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.732 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.732 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064330+0000", 2024-09-13T10:12:16.732 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023328912 2024-09-13T10:12:16.732 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.733 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.733 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.733 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064401+0000", 2024-09-13T10:12:16.733 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.1123000000000003e-05 2024-09-13T10:12:16.733 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.733 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.733 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.733 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064495+0000", 2024-09-13T10:12:16.733 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.3553000000000004e-05 2024-09-13T10:12:16.733 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.733 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.733 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.733 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.734 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.734 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:215 3.3 3:ed6c0a17:::smithi19530714-16 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.734 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:12:16.734 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 85.721216565000006, 2024-09-13T10:12:16.734 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.055806517999999999, 2024-09-13T10:12:16.734 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.734 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.734 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.734 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.734 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.734 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.734 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 215 2024-09-13T10:12:16.734 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.735 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.735 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.735 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.735 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:12:16.735 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.735 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.735 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.735 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.735 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:12:16.735 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.735 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.735 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.735 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.735 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994060+0000", 2024-09-13T10:12:16.736 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5030000000000001e-06 2024-09-13T10:12:16.736 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.736 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.736 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.736 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994070+0000", 2024-09-13T10:12:16.736 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0046e-05 2024-09-13T10:12:16.736 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.736 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.736 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.736 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994087+0000", 2024-09-13T10:12:16.736 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6283999999999999e-05 2024-09-13T10:12:16.736 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.737 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.737 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.737 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994096+0000", 2024-09-13T10:12:16.737 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.969e-06 2024-09-13T10:12:16.737 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.737 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.737 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.737 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.995226+0000", 2024-09-13T10:12:16.737 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011299540000000001 2024-09-13T10:12:16.737 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.738 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.738 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.738 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.995548+0000", 2024-09-13T10:12:16.738 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00032266199999999998 2024-09-13T10:12:16.738 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.738 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.738 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.738 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.018922+0000", 2024-09-13T10:12:16.738 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023373485999999999 2024-09-13T10:12:16.739 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.739 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.739 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.739 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029400+0000", 2024-09-13T10:12:16.739 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010478721 2024-09-13T10:12:16.739 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.739 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.739 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.739 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038932+0000", 2024-09-13T10:12:16.739 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0095316859999999993 2024-09-13T10:12:16.739 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.739 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.739 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.739 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049620+0000", 2024-09-13T10:12:16.739 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010687688000000001 2024-09-13T10:12:16.740 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.740 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.740 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.740 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049754+0000", 2024-09-13T10:12:16.740 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013457700000000001 2024-09-13T10:12:16.740 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.740 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.740 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.740 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049865+0000", 2024-09-13T10:12:16.740 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110942 2024-09-13T10:12:16.740 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.740 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.741 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.741 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.741 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.741 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:224 3.3 3:f941658c:::smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.741 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:12:16.741 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 85.664708528000006, 2024-09-13T10:12:16.741 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.025014027000000001, 2024-09-13T10:12:16.741 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.741 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.741 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.741 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.741 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.741 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.742 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 224 2024-09-13T10:12:16.742 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.742 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.742 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.742 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.742 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:12:16.742 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.742 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.742 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.742 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.742 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:12:16.742 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.742 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.743 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.743 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.743 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050570+0000", 2024-09-13T10:12:16.743 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9799999999999998e-06 2024-09-13T10:12:16.743 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.743 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.743 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.743 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050582+0000", 2024-09-13T10:12:16.743 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2170999999999999e-05 2024-09-13T10:12:16.743 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.743 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.743 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.744 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050605+0000", 2024-09-13T10:12:16.744 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3543000000000001e-05 2024-09-13T10:12:16.744 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.744 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.744 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.744 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050619+0000", 2024-09-13T10:12:16.744 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3478999999999999e-05 2024-09-13T10:12:16.744 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.744 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.744 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.744 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050725+0000", 2024-09-13T10:12:16.744 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000105585 2024-09-13T10:12:16.744 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.745 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.745 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.745 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.051086+0000", 2024-09-13T10:12:16.745 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036162100000000001 2024-09-13T10:12:16.745 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.745 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.745 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.745 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.058628+0000", 2024-09-13T10:12:16.745 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075415569999999999 2024-09-13T10:12:16.745 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.745 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.745 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.745 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065344+0000", 2024-09-13T10:12:16.746 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0067161740000000001 2024-09-13T10:12:16.746 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.746 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.746 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.746 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.066222+0000", 2024-09-13T10:12:16.746 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00087765599999999999 2024-09-13T10:12:16.746 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.746 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.746 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.746 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075372+0000", 2024-09-13T10:12:16.746 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0091499609999999999 2024-09-13T10:12:16.746 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.746 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.747 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.747 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075489+0000", 2024-09-13T10:12:16.747 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011718300000000001 2024-09-13T10:12:16.747 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.747 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.747 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.747 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075581+0000", 2024-09-13T10:12:16.747 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.2116999999999999e-05 2024-09-13T10:12:16.747 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.747 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.747 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.747 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.747 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.747 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:226 3.3 3:fe5bfa3a:::smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.747 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:12:16.748 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 85.655175518999997, 2024-09-13T10:12:16.748 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051515848000000003, 2024-09-13T10:12:16.748 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.748 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.748 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.748 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.748 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.748 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.748 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 226 2024-09-13T10:12:16.748 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.748 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.748 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.748 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.749 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:12:16.749 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.749 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.749 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.749 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.749 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:12:16.749 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.749 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.749 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.749 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.749 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060102+0000", 2024-09-13T10:12:16.749 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0480000000000001e-06 2024-09-13T10:12:16.750 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.750 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.750 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.750 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060112+0000", 2024-09-13T10:12:16.750 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6649999999999998e-06 2024-09-13T10:12:16.750 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.750 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.750 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.750 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060132+0000", 2024-09-13T10:12:16.750 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0262e-05 2024-09-13T10:12:16.750 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.750 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.750 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.751 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060143+0000", 2024-09-13T10:12:16.751 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1420000000000001e-05 2024-09-13T10:12:16.751 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.751 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.751 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.751 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064563+0000", 2024-09-13T10:12:16.751 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044199119999999998 2024-09-13T10:12:16.751 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.751 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.751 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.751 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064882+0000", 2024-09-13T10:12:16.751 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000318735 2024-09-13T10:12:16.751 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.751 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.751 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.752 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.078858+0000", 2024-09-13T10:12:16.752 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013976455 2024-09-13T10:12:16.752 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.752 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.752 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.752 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088977+0000", 2024-09-13T10:12:16.752 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010118544 2024-09-13T10:12:16.752 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.752 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.752 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.752 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.090451+0000", 2024-09-13T10:12:16.752 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001474026 2024-09-13T10:12:16.752 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.752 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.753 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.753 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111539+0000", 2024-09-13T10:12:16.753 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.02108786 2024-09-13T10:12:16.753 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.753 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.753 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.753 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111580+0000", 2024-09-13T10:12:16.753 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1245000000000002e-05 2024-09-13T10:12:16.753 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.753 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.753 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.753 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111616+0000", 2024-09-13T10:12:16.754 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5676000000000002e-05 2024-09-13T10:12:16.754 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.754 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.754 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.754 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.754 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.754 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:235 3.3 3:e217e96e:::smithi19530714-36 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.754 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:12:16.754 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 85.644397889999993, 2024-09-13T10:12:16.754 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051184682000000002, 2024-09-13T10:12:16.754 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.754 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.755 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.755 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.755 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.755 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.755 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 235 2024-09-13T10:12:16.755 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.755 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.755 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.755 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.755 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:12:16.755 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.755 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.756 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.756 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.756 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:12:16.756 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.756 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.756 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.756 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.756 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070881+0000", 2024-09-13T10:12:16.756 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0579999999999999e-06 2024-09-13T10:12:16.756 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.756 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.756 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.756 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070897+0000", 2024-09-13T10:12:16.757 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5967000000000001e-05 2024-09-13T10:12:16.757 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.757 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.757 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.757 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070942+0000", 2024-09-13T10:12:16.757 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4947000000000002e-05 2024-09-13T10:12:16.757 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.757 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.757 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.757 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070963+0000", 2024-09-13T10:12:16.757 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0098e-05 2024-09-13T10:12:16.757 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.757 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.758 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.758 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.071099+0000", 2024-09-13T10:12:16.758 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013697199999999999 2024-09-13T10:12:16.758 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.758 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.758 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.758 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.071330+0000", 2024-09-13T10:12:16.758 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000230881 2024-09-13T10:12:16.758 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.758 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.758 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.758 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.084668+0000", 2024-09-13T10:12:16.758 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013337663 2024-09-13T10:12:16.759 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.759 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.759 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.759 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.092656+0000", 2024-09-13T10:12:16.759 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0079878880000000003 2024-09-13T10:12:16.759 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.759 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.759 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.759 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.100238+0000", 2024-09-13T10:12:16.759 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075819449999999997 2024-09-13T10:12:16.759 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.759 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.760 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.760 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121877+0000", 2024-09-13T10:12:16.760 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021639104999999999 2024-09-13T10:12:16.760 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.760 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.760 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.760 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121978+0000", 2024-09-13T10:12:16.760 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000101363 2024-09-13T10:12:16.760 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.760 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.760 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.760 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.122062+0000", 2024-09-13T10:12:16.760 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.3795000000000007e-05 2024-09-13T10:12:16.761 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.761 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.761 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.761 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.761 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.761 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:237 3.3 3:de40747e:::smithi19530714-38 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.761 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:12:16.761 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 85.636208686000003, 2024-09-13T10:12:16.761 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.073341331999999995, 2024-09-13T10:12:16.761 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.761 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.761 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.762 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.762 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.762 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.762 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 237 2024-09-13T10:12:16.762 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.762 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.762 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.762 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.762 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:12:16.762 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.762 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.762 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.762 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.763 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:12:16.763 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.763 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.763 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.763 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.763 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079069+0000", 2024-09-13T10:12:16.763 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6979999999999998e-06 2024-09-13T10:12:16.763 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.763 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.763 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.763 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079083+0000", 2024-09-13T10:12:16.763 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.384e-05 2024-09-13T10:12:16.763 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.763 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.764 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.764 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079105+0000", 2024-09-13T10:12:16.764 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1736e-05 2024-09-13T10:12:16.764 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.764 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.764 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.764 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079123+0000", 2024-09-13T10:12:16.764 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7889000000000001e-05 2024-09-13T10:12:16.764 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.764 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.764 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.764 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088459+0000", 2024-09-13T10:12:16.764 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0093359700000000007 2024-09-13T10:12:16.764 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.765 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.765 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.765 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088726+0000", 2024-09-13T10:12:16.765 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026732299999999997 2024-09-13T10:12:16.765 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.765 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.765 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.765 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.097522+0000", 2024-09-13T10:12:16.765 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0087962630000000003 2024-09-13T10:12:16.765 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.765 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.765 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.765 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.103972+0000", 2024-09-13T10:12:16.766 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0064494840000000001 2024-09-13T10:12:16.766 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.766 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.766 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.766 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.104160+0000", 2024-09-13T10:12:16.766 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00018782200000000001 2024-09-13T10:12:16.766 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.767 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.767 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.768 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152294+0000", 2024-09-13T10:12:16.768 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048134458999999997 2024-09-13T10:12:16.768 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.768 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.768 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.768 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152373+0000", 2024-09-13T10:12:16.768 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8574999999999999e-05 2024-09-13T10:12:16.768 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.768 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.768 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.768 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152408+0000", 2024-09-13T10:12:16.768 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5272999999999999e-05 2024-09-13T10:12:16.769 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.769 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.769 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.769 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.769 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.769 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:238 3.3 3:c43dd60b:::smithi19530714-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.769 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:12:16.769 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 85.626623038999995, 2024-09-13T10:12:16.769 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.069426360000000006, 2024-09-13T10:12:16.769 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.769 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.769 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.770 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.770 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.770 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.770 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 238 2024-09-13T10:12:16.770 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.770 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.770 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.770 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.770 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:12:16.770 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.770 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.770 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.770 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.771 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:12:16.771 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.771 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.771 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.771 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.771 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088655+0000", 2024-09-13T10:12:16.771 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2359999999999999e-06 2024-09-13T10:12:16.771 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.771 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.771 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.771 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088663+0000", 2024-09-13T10:12:16.771 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7800000000000006e-06 2024-09-13T10:12:16.771 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.772 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.772 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.772 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088688+0000", 2024-09-13T10:12:16.772 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4507000000000001e-05 2024-09-13T10:12:16.772 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.772 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.772 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.772 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088717+0000", 2024-09-13T10:12:16.772 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9204e-05 2024-09-13T10:12:16.772 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.772 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.772 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.772 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.089038+0000", 2024-09-13T10:12:16.772 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00032087199999999999 2024-09-13T10:12:16.773 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.773 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.773 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.773 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.089435+0000", 2024-09-13T10:12:16.773 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039756600000000002 2024-09-13T10:12:16.773 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.773 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.773 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.773 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.101672+0000", 2024-09-13T10:12:16.773 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012236528 2024-09-13T10:12:16.773 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.773 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.773 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.773 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.105256+0000", 2024-09-13T10:12:16.774 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0035837159999999998 2024-09-13T10:12:16.774 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.774 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.774 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.774 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111504+0000", 2024-09-13T10:12:16.774 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.006248577 2024-09-13T10:12:16.774 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.774 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.774 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.774 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.157950+0000", 2024-09-13T10:12:16.774 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.046446031999999998 2024-09-13T10:12:16.774 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.774 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.774 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.775 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158027+0000", 2024-09-13T10:12:16.775 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6402999999999993e-05 2024-09-13T10:12:16.775 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.775 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.775 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.775 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158079+0000", 2024-09-13T10:12:16.775 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.1938999999999999e-05 2024-09-13T10:12:16.775 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.775 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.775 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.775 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.775 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.775 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:246 3.3 3:f1be28e7:::smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.775 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:12:16.775 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 85.601912291999994, 2024-09-13T10:12:16.776 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051630810999999999, 2024-09-13T10:12:16.776 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.776 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.776 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.776 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.776 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.776 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.776 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 246 2024-09-13T10:12:16.776 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.776 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.776 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.776 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.777 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:12:16.777 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.777 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.777 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.777 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.777 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:12:16.777 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.777 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.777 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.777 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.777 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113365+0000", 2024-09-13T10:12:16.777 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.762e-06 2024-09-13T10:12:16.778 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.778 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.778 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.778 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113372+0000", 2024-09-13T10:12:16.778 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5020000000000003e-06 2024-09-13T10:12:16.778 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.778 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.778 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.778 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113390+0000", 2024-09-13T10:12:16.778 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8056000000000001e-05 2024-09-13T10:12:16.778 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.778 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.778 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.778 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113400+0000", 2024-09-13T10:12:16.779 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5119999999999997e-06 2024-09-13T10:12:16.779 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.779 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.779 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.779 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113456+0000", 2024-09-13T10:12:16.779 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.6221999999999998e-05 2024-09-13T10:12:16.779 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.779 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.779 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.779 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113576+0000", 2024-09-13T10:12:16.779 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000119466 2024-09-13T10:12:16.779 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.779 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.779 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.779 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.129127+0000", 2024-09-13T10:12:16.780 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015551003000000001 2024-09-13T10:12:16.780 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.780 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.780 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.780 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142482+0000", 2024-09-13T10:12:16.780 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013355104 2024-09-13T10:12:16.780 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.780 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.780 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.780 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142518+0000", 2024-09-13T10:12:16.780 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6532000000000002e-05 2024-09-13T10:12:16.780 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.781 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.781 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.781 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164919+0000", 2024-09-13T10:12:16.781 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022401038000000002 2024-09-13T10:12:16.781 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.781 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.781 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.781 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164962+0000", 2024-09-13T10:12:16.781 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.2329000000000001e-05 2024-09-13T10:12:16.781 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.781 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.781 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.781 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164994+0000", 2024-09-13T10:12:16.781 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2285e-05 2024-09-13T10:12:16.782 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.782 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.782 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.782 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.782 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.782 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:249 3.3 3:f565152a:::smithi19530714-50 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.782 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:12:16.782 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 85.600309815000003, 2024-09-13T10:12:16.782 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051233251, 2024-09-13T10:12:16.782 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.782 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.782 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.782 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.783 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.783 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.783 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 249 2024-09-13T10:12:16.783 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.783 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.783 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.783 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.783 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:12:16.783 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.783 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.783 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.783 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.783 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:12:16.784 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.784 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.784 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.784 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.784 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114967+0000", 2024-09-13T10:12:16.784 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.313e-06 2024-09-13T10:12:16.784 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.784 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.784 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.784 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114972+0000", 2024-09-13T10:12:16.784 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5620000000000003e-06 2024-09-13T10:12:16.784 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.784 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.785 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.785 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114997+0000", 2024-09-13T10:12:16.785 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4845999999999999e-05 2024-09-13T10:12:16.785 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.785 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.785 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.785 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115007+0000", 2024-09-13T10:12:16.785 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7200000000000001e-06 2024-09-13T10:12:16.785 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.785 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.785 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.785 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115076+0000", 2024-09-13T10:12:16.785 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9473000000000004e-05 2024-09-13T10:12:16.786 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.786 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.786 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.786 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115221+0000", 2024-09-13T10:12:16.786 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000144604 2024-09-13T10:12:16.786 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.786 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.786 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.786 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.136862+0000", 2024-09-13T10:12:16.786 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021641365999999999 2024-09-13T10:12:16.786 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.786 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.786 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.787 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.145775+0000", 2024-09-13T10:12:16.787 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0089130649999999995 2024-09-13T10:12:16.787 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.787 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.787 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.787 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.150578+0000", 2024-09-13T10:12:16.787 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0048025999999999998 2024-09-13T10:12:16.787 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.787 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.787 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.787 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166133+0000", 2024-09-13T10:12:16.787 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015554853 2024-09-13T10:12:16.787 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.788 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.788 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.788 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166167+0000", 2024-09-13T10:12:16.788 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3973999999999997e-05 2024-09-13T10:12:16.788 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.788 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.788 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.788 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166199+0000", 2024-09-13T10:12:16.788 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1875000000000002e-05 2024-09-13T10:12:16.788 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.788 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.788 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.788 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.789 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.789 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.6.23:1 3.3 3:d4fb61be:::smithi19530714-2 low_tier:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:16.789 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:12:16.789 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 85.595146765999999, 2024-09-13T10:12:16.789 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.038168746000000003, 2024-09-13T10:12:16.789 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.789 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.789 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.789 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.789 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.789 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.789 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:12:16.789 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.790 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.790 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.790 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.790 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:12:16.790 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.790 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.790 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.790 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.790 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120131+0000", 2024-09-13T10:12:16.790 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5620000000000002e-06 2024-09-13T10:12:16.790 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.790 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.790 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.791 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120140+0000", 2024-09-13T10:12:16.791 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1889999999999996e-06 2024-09-13T10:12:16.791 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.791 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.791 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.791 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120161+0000", 2024-09-13T10:12:16.791 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0485000000000001e-05 2024-09-13T10:12:16.791 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.791 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.791 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.791 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120200+0000", 2024-09-13T10:12:16.791 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9648999999999999e-05 2024-09-13T10:12:16.791 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.792 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.792 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.792 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120285+0000", 2024-09-13T10:12:16.792 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.4488999999999997e-05 2024-09-13T10:12:16.792 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.792 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.792 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.792 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120399+0000", 2024-09-13T10:12:16.792 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000113857 2024-09-13T10:12:16.792 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.792 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.792 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.792 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120640+0000", 2024-09-13T10:12:16.793 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024112600000000001 2024-09-13T10:12:16.793 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.793 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.793 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.793 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121715+0000", 2024-09-13T10:12:16.793 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001074885 2024-09-13T10:12:16.793 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.793 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.793 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.793 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.122870+0000", 2024-09-13T10:12:16.793 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00115504 2024-09-13T10:12:16.793 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.793 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.794 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.794 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158160+0000", 2024-09-13T10:12:16.794 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035289760000000003 2024-09-13T10:12:16.794 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.794 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.794 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.794 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158211+0000", 2024-09-13T10:12:16.794 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.1005999999999999e-05 2024-09-13T10:12:16.794 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.794 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.794 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.794 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158297+0000", 2024-09-13T10:12:16.794 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.6698000000000002e-05 2024-09-13T10:12:16.794 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.795 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.795 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.795 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.795 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.795 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.14.23:47 3.3 3:ffec9690:::b0d2dba9f365561c17cd4cce8c8a8f4d8f931d05ca998f81ae8f5caa986fcf1e:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:16.795 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:10.781559+0000", 2024-09-13T10:12:16.795 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 65.933715860000007, 2024-09-13T10:12:16.795 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015576100000000001, 2024-09-13T10:12:16.795 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.795 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.795 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.795 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.795 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.796 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.796 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781559+0000", 2024-09-13T10:12:16.796 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.796 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.796 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.796 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.796 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781559+0000", 2024-09-13T10:12:16.796 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.796 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.796 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.796 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.796 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781561+0000", 2024-09-13T10:12:16.797 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5460000000000001e-06 2024-09-13T10:12:16.797 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.797 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.797 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.797 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781708+0000", 2024-09-13T10:12:16.797 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000147291 2024-09-13T10:12:16.797 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.797 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.797 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.797 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781735+0000", 2024-09-13T10:12:16.797 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7175999999999999e-05 2024-09-13T10:12:16.797 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.797 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.798 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.798 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781755+0000", 2024-09-13T10:12:16.798 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9653e-05 2024-09-13T10:12:16.798 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.798 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.798 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.798 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.790861+0000", 2024-09-13T10:12:16.798 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0091056920000000003 2024-09-13T10:12:16.798 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.798 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.798 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.798 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.791072+0000", 2024-09-13T10:12:16.798 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000211103 2024-09-13T10:12:16.799 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.799 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.799 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.799 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.791511+0000", 2024-09-13T10:12:16.799 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00043908899999999998 2024-09-13T10:12:16.799 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.799 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.799 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.799 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794857+0000", 2024-09-13T10:12:16.799 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0033462940000000001 2024-09-13T10:12:16.799 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.799 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.799 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.799 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794945+0000", 2024-09-13T10:12:16.799 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7626000000000004e-05 2024-09-13T10:12:16.800 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.800 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.800 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.800 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.797001+0000", 2024-09-13T10:12:16.800 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020557280000000002 2024-09-13T10:12:16.800 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.800 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.800 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.800 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.797080+0000", 2024-09-13T10:12:16.800 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9169e-05 2024-09-13T10:12:16.800 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.800 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.800 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.801 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.797136+0000", 2024-09-13T10:12:16.801 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5733000000000003e-05 2024-09-13T10:12:16.801 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.801 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.801 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.801 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.801 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.801 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3970 4.18 4:1de9da87:::smithi19530714-5:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e94)", 2024-09-13T10:12:16.801 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:00.220914+0000", 2024-09-13T10:12:16.801 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 16.494361824999999, 2024-09-13T10:12:16.801 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048456570999999997, 2024-09-13T10:12:16.801 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.801 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.802 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:16.802 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:16.802 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:16.802 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:16.802 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3970 2024-09-13T10:12:16.802 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.802 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.802 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.802 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.802 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.220914+0000", 2024-09-13T10:12:16.802 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.802 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.802 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.802 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.802 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.220914+0000", 2024-09-13T10:12:16.803 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.803 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.803 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.803 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.803 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.220917+0000", 2024-09-13T10:12:16.803 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.36e-06 2024-09-13T10:12:16.803 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.803 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.803 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.803 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.220934+0000", 2024-09-13T10:12:16.803 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6748999999999998e-05 2024-09-13T10:12:16.803 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.803 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.804 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.804 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.220973+0000", 2024-09-13T10:12:16.804 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9693000000000001e-05 2024-09-13T10:12:16.804 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.804 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.804 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.804 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.221001+0000", 2024-09-13T10:12:16.804 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7209e-05 2024-09-13T10:12:16.804 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.804 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.804 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.804 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.221170+0000", 2024-09-13T10:12:16.804 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016995700000000001 2024-09-13T10:12:16.804 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.804 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.805 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.805 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.221316+0000", 2024-09-13T10:12:16.805 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000145981 2024-09-13T10:12:16.805 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.805 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.805 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.805 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.269370+0000", 2024-09-13T10:12:16.805 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048053621999999997 2024-09-13T10:12:16.805 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.805 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.805 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.805 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.805 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.805 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.15.22:53 3.3 3:c0dd3fe6:::ec0aa0931246a621c18aa6befa600bcc59df35053a1247b4e1863cf02828cf81:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:16.806 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:05.193032+0000", 2024-09-13T10:12:16.806 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 11.522243076000001, 2024-09-13T10:12:16.806 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015830879999999999, 2024-09-13T10:12:16.806 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.806 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.806 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.806 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.806 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.806 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.806 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193032+0000", 2024-09-13T10:12:16.806 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.806 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.806 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.806 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.807 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193032+0000", 2024-09-13T10:12:16.807 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.807 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.807 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.807 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.807 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193034+0000", 2024-09-13T10:12:16.807 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6330000000000001e-06 2024-09-13T10:12:16.807 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.807 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.807 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.807 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193471+0000", 2024-09-13T10:12:16.808 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000437503 2024-09-13T10:12:16.808 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.808 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.808 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.808 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193493+0000", 2024-09-13T10:12:16.808 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1786000000000001e-05 2024-09-13T10:12:16.808 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.808 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.808 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.808 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193504+0000", 2024-09-13T10:12:16.808 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1043000000000001e-05 2024-09-13T10:12:16.808 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.808 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.808 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.809 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.202746+0000", 2024-09-13T10:12:16.809 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.009241773 2024-09-13T10:12:16.809 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.809 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.809 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.809 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.202973+0000", 2024-09-13T10:12:16.809 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000226956 2024-09-13T10:12:16.809 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.809 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.809 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.809 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.203492+0000", 2024-09-13T10:12:16.809 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00051952400000000005 2024-09-13T10:12:16.810 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.810 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.810 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.810 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.206115+0000", 2024-09-13T10:12:16.810 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0026221899999999999 2024-09-13T10:12:16.810 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.810 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.810 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.810 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.208604+0000", 2024-09-13T10:12:16.810 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0024892870000000002 2024-09-13T10:12:16.810 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.810 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.810 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.810 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.208715+0000", 2024-09-13T10:12:16.811 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110568 2024-09-13T10:12:16.811 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.811 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.811 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.811 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.208800+0000", 2024-09-13T10:12:16.811 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5414999999999995e-05 2024-09-13T10:12:16.811 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.811 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.811 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.811 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.208863+0000", 2024-09-13T10:12:16.811 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.3201999999999996e-05 2024-09-13T10:12:16.811 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.811 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.811 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.812 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.812 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.812 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.15.22:55 3.3 3:cdcec4b2:::f314d852c33ac9b893465b328249830227013d5ec45b1156c69d7b9a27a71710:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:16.812 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:05.193532+0000", 2024-09-13T10:12:16.812 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 11.521743225, 2024-09-13T10:12:16.812 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017555576, 2024-09-13T10:12:16.812 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.812 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.812 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.812 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.812 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.812 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.813 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193532+0000", 2024-09-13T10:12:16.813 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.813 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.813 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.813 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.813 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193532+0000", 2024-09-13T10:12:16.813 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.813 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.813 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.813 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.813 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193534+0000", 2024-09-13T10:12:16.813 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5030000000000001e-06 2024-09-13T10:12:16.813 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.814 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.814 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.814 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193691+0000", 2024-09-13T10:12:16.814 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015784100000000001 2024-09-13T10:12:16.814 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.814 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.814 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.814 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193711+0000", 2024-09-13T10:12:16.814 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.98e-05 2024-09-13T10:12:16.814 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.814 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.814 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.814 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193721+0000", 2024-09-13T10:12:16.815 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5799999999999998e-06 2024-09-13T10:12:16.815 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.815 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.815 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.815 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.206570+0000", 2024-09-13T10:12:16.815 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012849572 2024-09-13T10:12:16.815 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.815 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.815 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.815 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.206946+0000", 2024-09-13T10:12:16.815 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037542700000000001 2024-09-13T10:12:16.815 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.815 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.815 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.816 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.207419+0000", 2024-09-13T10:12:16.816 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00047319199999999999 2024-09-13T10:12:16.816 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.816 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.816 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.816 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.210769+0000", 2024-09-13T10:12:16.816 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0033497599999999998 2024-09-13T10:12:16.816 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.816 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.816 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.816 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.210878+0000", 2024-09-13T10:12:16.816 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000109028 2024-09-13T10:12:16.816 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.817 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.817 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.817 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.210911+0000", 2024-09-13T10:12:16.817 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2750000000000003e-05 2024-09-13T10:12:16.817 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.817 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.817 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.817 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.211024+0000", 2024-09-13T10:12:16.817 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000113714 2024-09-13T10:12:16.817 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.817 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.817 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.817 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.211088+0000", 2024-09-13T10:12:16.818 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.3409000000000002e-05 2024-09-13T10:12:16.818 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.818 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.818 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.818 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.818 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.818 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.15.22:62 3.3 3:f4bd62ee:::c19f5965474b2636dbb114434ccba3e174847a0d497c0002fe234287445262fc:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:16.818 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:05.193742+0000", 2024-09-13T10:12:16.818 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 11.521533475, 2024-09-13T10:12:16.818 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.018130356, 2024-09-13T10:12:16.818 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.818 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.818 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.819 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.819 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.819 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.819 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193742+0000", 2024-09-13T10:12:16.819 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.819 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.819 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.819 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.819 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193742+0000", 2024-09-13T10:12:16.819 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.819 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.819 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.819 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.819 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193743+0000", 2024-09-13T10:12:16.819 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.2699999999999998e-07 2024-09-13T10:12:16.819 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.820 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.820 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.820 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193849+0000", 2024-09-13T10:12:16.820 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010634800000000001 2024-09-13T10:12:16.820 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.820 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.820 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.820 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193861+0000", 2024-09-13T10:12:16.820 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2016e-05 2024-09-13T10:12:16.820 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.820 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.820 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.821 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193872+0000", 2024-09-13T10:12:16.821 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1243000000000001e-05 2024-09-13T10:12:16.821 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.821 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.821 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.821 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.209010+0000", 2024-09-13T10:12:16.821 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015137711999999999 2024-09-13T10:12:16.821 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.821 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.821 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.821 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.209292+0000", 2024-09-13T10:12:16.821 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028191499999999999 2024-09-13T10:12:16.822 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.822 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.822 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.822 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.209651+0000", 2024-09-13T10:12:16.822 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00035888700000000001 2024-09-13T10:12:16.822 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.822 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.822 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.822 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.211151+0000", 2024-09-13T10:12:16.822 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0015004969999999999 2024-09-13T10:12:16.822 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.822 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.822 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.822 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.211678+0000", 2024-09-13T10:12:16.823 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00052626699999999997 2024-09-13T10:12:16.823 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.823 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.823 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.823 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.211746+0000", 2024-09-13T10:12:16.823 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.8639000000000005e-05 2024-09-13T10:12:16.823 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.823 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.823 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.823 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.211816+0000", 2024-09-13T10:12:16.823 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9785999999999999e-05 2024-09-13T10:12:16.823 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.823 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.824 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.824 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.211872+0000", 2024-09-13T10:12:16.824 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.6119000000000003e-05 2024-09-13T10:12:16.824 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.824 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.824 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.824 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.824 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.824 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.2.23:176 3.3 3:fac4d691:::814864e4c5245cfaf79ad18a0e61662368667270ceb2e98bad1fed4e25945c9c:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:12:16.824 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:05.835347+0000", 2024-09-13T10:12:16.824 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 10.879928542, 2024-09-13T10:12:16.824 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015415807, 2024-09-13T10:12:16.824 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.825 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.825 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.825 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.825 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.825 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.825 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.835347+0000", 2024-09-13T10:12:16.825 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.825 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.825 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.825 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.825 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.835347+0000", 2024-09-13T10:12:16.825 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.825 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.826 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.826 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.826 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.835348+0000", 2024-09-13T10:12:16.826 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0869999999999999e-06 2024-09-13T10:12:16.826 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.826 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.826 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.826 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.835353+0000", 2024-09-13T10:12:16.826 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.4070000000000001e-06 2024-09-13T10:12:16.826 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.826 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.826 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.827 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.835363+0000", 2024-09-13T10:12:16.827 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.9830000000000008e-06 2024-09-13T10:12:16.827 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.827 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.827 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.827 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.835378+0000", 2024-09-13T10:12:16.827 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5106000000000001e-05 2024-09-13T10:12:16.827 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.827 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.827 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.827 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.835621+0000", 2024-09-13T10:12:16.827 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024247599999999999 2024-09-13T10:12:16.827 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.828 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.828 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting_for_map not empty", 2024-09-13T10:12:16.828 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.835627+0000", 2024-09-13T10:12:16.828 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4829999999999996e-06 2024-09-13T10:12:16.828 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.828 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.828 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.828 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.847651+0000", 2024-09-13T10:12:16.828 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012023574 2024-09-13T10:12:16.828 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.828 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.828 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.828 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.847898+0000", 2024-09-13T10:12:16.828 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000246798 2024-09-13T10:12:16.829 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.829 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.829 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.829 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.848357+0000", 2024-09-13T10:12:16.829 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00045895899999999999 2024-09-13T10:12:16.829 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.829 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.829 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.829 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.850450+0000", 2024-09-13T10:12:16.829 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002093003 2024-09-13T10:12:16.829 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.829 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.830 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.830 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.850574+0000", 2024-09-13T10:12:16.830 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012480400000000001 2024-09-13T10:12:16.830 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.830 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.830 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.830 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.850594+0000", 2024-09-13T10:12:16.830 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9233999999999999e-05 2024-09-13T10:12:16.830 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.830 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.830 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.830 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.850696+0000", 2024-09-13T10:12:16.830 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000102704 2024-09-13T10:12:16.830 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.830 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.831 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.831 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.850763+0000", 2024-09-13T10:12:16.831 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.6188999999999999e-05 2024-09-13T10:12:16.831 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.831 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.831 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.831 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.831 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.831 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.13.21:111 3.3 3:f8b6ff76:::0df7b452b15ded2caa5361cab4a0527aa26bbdbd1c62c3d43532df9f8363c465:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e106)", 2024-09-13T10:12:16.831 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:14.701910+0000", 2024-09-13T10:12:16.831 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 2.013365716, 2024-09-13T10:12:16.831 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016339989999999999, 2024-09-13T10:12:16.832 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.832 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.832 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.832 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.832 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.832 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.832 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.701910+0000", 2024-09-13T10:12:16.832 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.832 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.832 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.832 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.832 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.701910+0000", 2024-09-13T10:12:16.832 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.833 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.833 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.833 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.833 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.701911+0000", 2024-09-13T10:12:16.833 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0529999999999999e-06 2024-09-13T10:12:16.833 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.833 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.833 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.833 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702056+0000", 2024-09-13T10:12:16.833 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014561199999999999 2024-09-13T10:12:16.833 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.833 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.833 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.834 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702078+0000", 2024-09-13T10:12:16.834 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1403000000000002e-05 2024-09-13T10:12:16.834 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.834 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.834 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.834 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702093+0000", 2024-09-13T10:12:16.834 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5092999999999999e-05 2024-09-13T10:12:16.834 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.834 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.834 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.834 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.712785+0000", 2024-09-13T10:12:16.834 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010691740999999999 2024-09-13T10:12:16.834 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.834 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.835 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.835 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.713115+0000", 2024-09-13T10:12:16.835 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000330895 2024-09-13T10:12:16.835 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.835 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.835 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.835 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.713706+0000", 2024-09-13T10:12:16.835 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00059055999999999996 2024-09-13T10:12:16.835 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.835 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.835 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.835 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.716028+0000", 2024-09-13T10:12:16.836 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023215940000000002 2024-09-13T10:12:16.836 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.836 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.836 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.836 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.717873+0000", 2024-09-13T10:12:16.836 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0018456080000000001 2024-09-13T10:12:16.836 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.836 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.836 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.836 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.718003+0000", 2024-09-13T10:12:16.836 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012976099999999999 2024-09-13T10:12:16.837 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.837 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.837 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.837 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.718142+0000", 2024-09-13T10:12:16.837 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013915100000000001 2024-09-13T10:12:16.837 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.837 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.837 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.837 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.718250+0000", 2024-09-13T10:12:16.837 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000107519 2024-09-13T10:12:16.837 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.837 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.838 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.838 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.838 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.838 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.13.21:112 3.3 3:de19817f:::f36543c9e1229abc2db2e0c089492cde9aa332b42fc2e36672be1ba4d1ccbfa5:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e106)", 2024-09-13T10:12:16.838 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:14.702115+0000", 2024-09-13T10:12:16.838 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 2.0131602819999999, 2024-09-13T10:12:16.838 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.018916507999999999, 2024-09-13T10:12:16.838 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.838 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.838 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.838 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.839 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.839 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.839 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702115+0000", 2024-09-13T10:12:16.839 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.839 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.839 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.839 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.839 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702115+0000", 2024-09-13T10:12:16.839 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.839 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.839 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.839 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.839 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702116+0000", 2024-09-13T10:12:16.839 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3200000000000001e-06 2024-09-13T10:12:16.840 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.840 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.840 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.840 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702924+0000", 2024-09-13T10:12:16.840 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00080727999999999996 2024-09-13T10:12:16.840 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.840 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.840 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.840 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702942+0000", 2024-09-13T10:12:16.840 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7903000000000001e-05 2024-09-13T10:12:16.840 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.840 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.840 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.841 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702952+0000", 2024-09-13T10:12:16.841 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.076e-05 2024-09-13T10:12:16.841 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.841 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.841 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.841 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.716127+0000", 2024-09-13T10:12:16.841 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013174455999999999 2024-09-13T10:12:16.841 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.841 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.841 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.841 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.716430+0000", 2024-09-13T10:12:16.841 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00030353200000000002 2024-09-13T10:12:16.842 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.842 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.842 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.842 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.716892+0000", 2024-09-13T10:12:16.842 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00046132900000000001 2024-09-13T10:12:16.842 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.842 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.842 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.842 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.720049+0000", 2024-09-13T10:12:16.842 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0031571139999999999 2024-09-13T10:12:16.842 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.842 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.842 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.843 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.720083+0000", 2024-09-13T10:12:16.843 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.4508999999999998e-05 2024-09-13T10:12:16.843 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.843 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.843 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.843 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.720895+0000", 2024-09-13T10:12:16.843 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00081174700000000003 2024-09-13T10:12:16.843 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.843 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.843 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.843 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.720972+0000", 2024-09-13T10:12:16.843 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7113000000000005e-05 2024-09-13T10:12:16.843 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.843 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.844 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.844 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.721032+0000", 2024-09-13T10:12:16.844 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9444999999999998e-05 2024-09-13T10:12:16.844 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.844 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.844 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.844 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.844 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.844 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.13.21:114 3.3 3:c560edf3:::05229772e6b32f13629a0acc34dce638ea214bd4aa8c7e9fffd3aed735eaea96:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e106)", 2024-09-13T10:12:16.844 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:14.702971+0000", 2024-09-13T10:12:16.844 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 2.0123048190000001, 2024-09-13T10:12:16.844 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.018655682999999999, 2024-09-13T10:12:16.844 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:16.844 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:16.845 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:16.845 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:16.845 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.845 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:16.845 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702971+0000", 2024-09-13T10:12:16.845 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.845 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.845 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.845 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:16.845 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702971+0000", 2024-09-13T10:12:16.845 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:16.845 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.845 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.845 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:16.845 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702972+0000", 2024-09-13T10:12:16.846 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.04e-06 2024-09-13T10:12:16.846 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.846 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.846 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:16.846 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702994+0000", 2024-09-13T10:12:16.846 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2246e-05 2024-09-13T10:12:16.846 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.846 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.846 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:16.846 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.703004+0000", 2024-09-13T10:12:16.846 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0223e-05 2024-09-13T10:12:16.846 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.846 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.846 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:16.847 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.703013+0000", 2024-09-13T10:12:16.847 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5469999999999997e-06 2024-09-13T10:12:16.847 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.847 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.847 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:16.847 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.718316+0000", 2024-09-13T10:12:16.847 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015303186 2024-09-13T10:12:16.847 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.847 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.847 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:16.847 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.718538+0000", 2024-09-13T10:12:16.847 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022254600000000001 2024-09-13T10:12:16.847 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.847 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.848 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:16.848 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.718956+0000", 2024-09-13T10:12:16.848 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00041724999999999998 2024-09-13T10:12:16.848 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.848 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.848 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:16.848 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.721059+0000", 2024-09-13T10:12:16.848 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0021034040000000001 2024-09-13T10:12:16.848 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.848 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.848 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.848 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.721124+0000", 2024-09-13T10:12:16.848 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4635999999999996e-05 2024-09-13T10:12:16.848 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.849 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.849 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:16.849 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.721481+0000", 2024-09-13T10:12:16.849 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000357647 2024-09-13T10:12:16.849 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.849 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.849 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:16.849 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.721564+0000", 2024-09-13T10:12:16.849 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.2738999999999995e-05 2024-09-13T10:12:16.849 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:16.849 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:16.849 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:16.849 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.721626+0000", 2024-09-13T10:12:16.849 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.2218999999999995e-05 2024-09-13T10:12:16.850 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.850 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.850 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.850 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:16.850 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:16.850 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:16.850 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T10:12:16.852 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:12:16.852 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:12:16.853 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:12:16.854 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:12:16.854 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-13T10:12:16.854 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:12:16.846+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:12:16.855 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:12:16.855 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:12:16.856 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:12:16.856 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:12:16.857 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:12:16.857 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:12:16.858 INFO:tasks.rados.rados.0.smithi195.stdout:1900: writing smithi19530714-40 from 1777496 to 2453632 tid 2 2024-09-13T10:12:16.858 INFO:tasks.rados.rados.0.smithi195.stdout:1900: writing smithi19530714-40 from 2950951 to 3058940 tid 3 2024-09-13T10:12:16.858 INFO:tasks.rados.rados.0.smithi195.stdout:1901: tier_evict oid 6 2024-09-13T10:12:16.858 INFO:tasks.rados.rados.0.smithi195.stdout:1901: tier_evict oid 6 snap -1 2024-09-13T10:12:16.858 INFO:tasks.rados.rados.0.smithi195.stdout:1900: finishing write tid 1 to smithi19530714-40 2024-09-13T10:12:16.858 INFO:tasks.rados.rados.0.smithi195.stdout:1902: tier_evict oid 31 2024-09-13T10:12:16.858 INFO:tasks.rados.rados.0.smithi195.stdout:1902: tier_evict oid 31 snap -1 2024-09-13T10:12:16.858 INFO:tasks.rados.rados.0.smithi195.stdout:1903: read oid 48 snap -1 2024-09-13T10:12:16.858 INFO:tasks.rados.rados.0.smithi195.stdout:1903: expect (ObjNum 410 snap 0 seq_num 410) 2024-09-13T10:12:16.858 INFO:tasks.rados.rados.0.smithi195.stdout:1904: read oid 41 snap -1 2024-09-13T10:12:16.859 INFO:tasks.rados.rados.0.smithi195.stdout:1904: expect deleted 2024-09-13T10:12:16.859 INFO:tasks.rados.rados.0.smithi195.stdout:1905: tier_promote oid 44 2024-09-13T10:12:16.859 INFO:tasks.rados.rados.0.smithi195.stdout:1906: read oid 33 snap -1 2024-09-13T10:12:16.859 INFO:tasks.rados.rados.0.smithi195.stdout:1906: expect (ObjNum 408 snap 0 seq_num 408) 2024-09-13T10:12:16.859 INFO:tasks.rados.rados.0.smithi195.stdout:1907: read oid 30 snap -1 2024-09-13T10:12:16.859 INFO:tasks.rados.rados.0.smithi195.stdout:1907: expect (ObjNum 416 snap 0 seq_num 416) 2024-09-13T10:12:16.859 INFO:tasks.rados.rados.0.smithi195.stdout:1908: write oid 21 current snap is 0 2024-09-13T10:12:16.859 INFO:tasks.rados.rados.0.smithi195.stdout:1908: seq_num 421 ranges {675747=479244,1835978=536147} 2024-09-13T10:12:16.859 INFO:tasks.rados.rados.0.smithi195.stdout:1908: writing smithi19530714-21 from 675747 to 1154991 tid 1 2024-09-13T10:12:16.859 INFO:tasks.rados.rados.0.smithi195.stdout:1908: writing smithi19530714-21 from 1835978 to 2372125 tid 2 2024-09-13T10:12:16.859 INFO:tasks.rados.rados.0.smithi195.stdout:1909: read oid 26 snap -1 2024-09-13T10:12:16.860 INFO:tasks.rados.rados.0.smithi195.stdout:1909: expect (ObjNum 379 snap 0 seq_num 379) 2024-09-13T10:12:16.860 INFO:tasks.rados.rados.0.smithi195.stdout:1910: write oid 27 current snap is 0 2024-09-13T10:12:16.860 INFO:tasks.rados.rados.0.smithi195.stdout:1900: finishing write tid 2 to smithi19530714-40 2024-09-13T10:12:16.860 INFO:tasks.rados.rados.0.smithi195.stdout:1901: got (22) Invalid argument 2024-09-13T10:12:16.860 INFO:tasks.rados.rados.0.smithi195.stdout:1902: got (22) Invalid argument 2024-09-13T10:12:16.860 INFO:tasks.rados.rados.0.smithi195.stdout:1900: finishing write tid 3 to smithi19530714-40 2024-09-13T10:12:16.860 INFO:tasks.rados.rados.0.smithi195.stdout:1907: expect (ObjNum 416 snap 0 seq_num 416) 2024-09-13T10:12:16.860 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2910952 2024-09-13T10:12:16.860 INFO:tasks.rados.rados.0.smithi195.stdout:1900: finishing write tid 4 to smithi19530714-40 2024-09-13T10:12:16.860 INFO:tasks.rados.rados.0.smithi195.stdout:1900: finishing write tid 5 to smithi19530714-40 2024-09-13T10:12:16.860 INFO:tasks.rados.rados.0.smithi195.stdout:1900: oid 40 updating version 0 to 39 2024-09-13T10:12:16.860 INFO:tasks.rados.rados.0.smithi195.stdout:1900: oid 40 version 39 is already newer than 37 2024-09-13T10:12:16.860 INFO:tasks.rados.rados.0.smithi195.stdout:1900: oid 40 version 39 is already newer than 36 2024-09-13T10:12:16.861 INFO:tasks.rados.rados.0.smithi195.stdout:1900: oid 40 version 39 is already newer than 38 2024-09-13T10:12:16.861 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 39 (ObjNum 420 snap 0 seq_num 420) dirty exists 2024-09-13T10:12:16.861 INFO:tasks.rados.rados.0.smithi195.stdout:1900: left oid 40 (ObjNum 420 snap 0 seq_num 420) 2024-09-13T10:12:16.861 INFO:tasks.rados.rados.0.smithi195.stdout:1906: expect (ObjNum 408 snap 0 seq_num 408) 2024-09-13T10:12:16.861 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1358328 2024-09-13T10:12:16.861 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:12:16.861 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:12:16.862 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:12:16.862 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:12:16.882 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:12:16.882 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:12:16.902 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:12:16.903 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:12:16.911 INFO:tasks.rados.rados.0.smithi195.stdout:1905: got (2) No such file or directory 2024-09-13T10:12:16.911 INFO:tasks.rados.rados.0.smithi195.stdout:1905: got expected ENOENT (src dne) 2024-09-13T10:12:16.911 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 44 v 87 (ObjNum 49 snap 0 seq_num 629532464) clean dne 2024-09-13T10:12:16.911 INFO:tasks.rados.rados.0.smithi195.stdout:1903: expect (ObjNum 410 snap 0 seq_num 410) 2024-09-13T10:12:16.923 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:12:16.923 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:12:16.952 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:16.949+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:17.011 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1281361 2024-09-13T10:12:17.028 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:17.028 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:17.042 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-13T10:12:17.093 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:17.090+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:17.154 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:12:17.151+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:12:17.162 INFO:tasks.rados.rados.0.smithi195.stdout:1908: finishing write tid 1 to smithi19530714-21 2024-09-13T10:12:17.162 INFO:tasks.rados.rados.0.smithi195.stdout:1910: seq_num 422 ranges {692554=554493,1778107=679805,2889444=203110} 2024-09-13T10:12:17.165 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:17.165 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:17.168 INFO:tasks.rados.rados.0.smithi195.stdout:1910: writing smithi19530714-27 from 692554 to 1247047 tid 1 2024-09-13T10:12:17.175 INFO:tasks.rados.rados.0.smithi195.stdout:1910: writing smithi19530714-27 from 1778107 to 2457912 tid 2 2024-09-13T10:12:17.178 INFO:tasks.rados.rados.0.smithi195.stdout:1910: writing smithi19530714-27 from 2889444 to 3092554 tid 3 2024-09-13T10:12:17.178 INFO:tasks.rados.rados.0.smithi195.stdout:1900: done (11 left) 2024-09-13T10:12:17.178 INFO:tasks.rados.rados.0.smithi195.stdout:1901: done (10 left) 2024-09-13T10:12:17.178 INFO:tasks.rados.rados.0.smithi195.stdout:1902: done (9 left) 2024-09-13T10:12:17.178 INFO:tasks.rados.rados.0.smithi195.stdout:1903: done (8 left) 2024-09-13T10:12:17.178 INFO:tasks.rados.rados.0.smithi195.stdout:1904: done (7 left) 2024-09-13T10:12:17.179 INFO:tasks.rados.rados.0.smithi195.stdout:1905: done (6 left) 2024-09-13T10:12:17.179 INFO:tasks.rados.rados.0.smithi195.stdout:1906: done (5 left) 2024-09-13T10:12:17.179 INFO:tasks.rados.rados.0.smithi195.stdout:1907: done (4 left) 2024-09-13T10:12:17.179 INFO:tasks.rados.rados.0.smithi195.stdout:1911: tier_promote oid 29 2024-09-13T10:12:17.179 INFO:tasks.rados.rados.0.smithi195.stdout:1908: finishing write tid 2 to smithi19530714-21 2024-09-13T10:12:17.179 INFO:tasks.rados.rados.0.smithi195.stdout:1912: read oid 28 snap -1 2024-09-13T10:12:17.179 INFO:tasks.rados.rados.0.smithi195.stdout:1912: expect (ObjNum 369 snap 0 seq_num 369) 2024-09-13T10:12:17.179 INFO:tasks.rados.rados.0.smithi195.stdout:1913: set_chunk oid 45 target oid 2024-09-13T10:12:17.181 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-13T10:12:17.214 INFO:tasks.rados.rados.0.smithi195.stdout:1913: set_chunk oid 45 offset: 651264 length: 2015232 target oid 44cf2cddab3d109134b81684ee96e046f6e9b17cabe819b90c6afa514995aa0f offset: 0 snap -1 2024-09-13T10:12:17.214 INFO:tasks.rados.rados.0.smithi195.stdout:1914: read oid 6 snap -1 2024-09-13T10:12:17.214 INFO:tasks.rados.rados.0.smithi195.stdout:1914: expect (ObjNum 337 snap 0 seq_num 337) 2024-09-13T10:12:17.214 INFO:tasks.rados.rados.0.smithi195.stdout:1915: read oid 5 snap -1 2024-09-13T10:12:17.214 INFO:tasks.rados.rados.0.smithi195.stdout:1915: expect (ObjNum 360 snap 0 seq_num 360) 2024-09-13T10:12:17.214 INFO:tasks.rados.rados.0.smithi195.stdout:1916: read oid 23 snap -1 2024-09-13T10:12:17.214 INFO:tasks.rados.rados.0.smithi195.stdout:1916: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:12:17.215 INFO:tasks.rados.rados.0.smithi195.stdout:1917: write (excl) oid 10 current snap is 0 2024-09-13T10:12:17.215 INFO:tasks.rados.rados.0.smithi195.stdout:1917: seq_num 423 ranges {709361=629742,1836589=539816,3033220=76141} 2024-09-13T10:12:17.228 INFO:tasks.rados.rados.0.smithi195.stdout:1917: writing smithi19530714-10 from 709361 to 1339103 tid 1 2024-09-13T10:12:17.240 INFO:tasks.rados.rados.0.smithi195.stdout:1917: writing smithi19530714-10 from 1836589 to 2376405 tid 2 2024-09-13T10:12:17.247 INFO:tasks.rados.rados.0.smithi195.stdout:1917: writing smithi19530714-10 from 3033220 to 3109361 tid 3 2024-09-13T10:12:17.247 INFO:tasks.rados.rados.0.smithi195.stdout:1918: write (excl) oid 15 current snap is 0 2024-09-13T10:12:17.247 INFO:tasks.rados.rados.0.smithi195.stdout:1908: finishing write tid 3 to smithi19530714-21 2024-09-13T10:12:17.247 INFO:tasks.rados.rados.0.smithi195.stdout:1908: finishing write tid 4 to smithi19530714-21 2024-09-13T10:12:17.247 INFO:tasks.rados.rados.0.smithi195.stdout:1908: oid 21 updating version 0 to 167 2024-09-13T10:12:17.248 INFO:tasks.rados.rados.0.smithi195.stdout:1908: oid 21 version 167 is already newer than 166 2024-09-13T10:12:17.248 INFO:tasks.rados.rados.0.smithi195.stdout:1908: oid 21 updating version 167 to 168 2024-09-13T10:12:17.248 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 168 (ObjNum 421 snap 0 seq_num 421) dirty exists 2024-09-13T10:12:17.248 INFO:tasks.rados.rados.0.smithi195.stdout:1908: left oid 21 (ObjNum 421 snap 0 seq_num 421) 2024-09-13T10:12:17.248 INFO:tasks.rados.rados.0.smithi195.stdout:1909: expect (ObjNum 379 snap 0 seq_num 379) 2024-09-13T10:12:17.276 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:17.274+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:17.345 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1107275 2024-09-13T10:12:17.352 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:17.352 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:17.355 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:17.350+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:17.366 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T10:12:17.371 INFO:tasks.rados.rados.0.smithi195.stdout:1910: finishing write tid 1 to smithi19530714-27 2024-09-13T10:12:17.371 INFO:tasks.rados.rados.0.smithi195.stdout:1911: got (0) Success 2024-09-13T10:12:17.372 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 130 (ObjNum 401 snap 0 seq_num 401) dirty exists 2024-09-13T10:12:17.372 INFO:tasks.rados.rados.0.smithi195.stdout:1912: expect (ObjNum 369 snap 0 seq_num 369) 2024-09-13T10:12:17.372 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:17.455 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:17.456 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:17.450+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:17.483 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:17.484 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:17.496 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T10:12:17.499 INFO:tasks.rados.rados.0.smithi195.stdout:1910: finishing write tid 2 to smithi19530714-27 2024-09-13T10:12:17.499 INFO:tasks.rados.rados.0.smithi195.stdout:1910: finishing write tid 3 to smithi19530714-27 2024-09-13T10:12:17.499 INFO:tasks.rados.rados.0.smithi195.stdout:1918: seq_num 424 ranges {726168=704991,1895071=683474} 2024-09-13T10:12:17.507 INFO:tasks.rados.rados.0.smithi195.stdout:1918: writing smithi19530714-15 from 726168 to 1431159 tid 1 2024-09-13T10:12:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1918: writing smithi19530714-15 from 1895071 to 2578545 tid 2 2024-09-13T10:12:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1908: done (11 left) 2024-09-13T10:12:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1909: done (10 left) 2024-09-13T10:12:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1911: done (9 left) 2024-09-13T10:12:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1912: done (8 left) 2024-09-13T10:12:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1919: read oid 31 snap -1 2024-09-13T10:12:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1919: expect (ObjNum 402 snap 0 seq_num 402) 2024-09-13T10:12:17.516 INFO:tasks.rados.rados.0.smithi195.stdout:1920: read oid 21 snap -1 2024-09-13T10:12:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1920: expect (ObjNum 421 snap 0 seq_num 421) 2024-09-13T10:12:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1921: read oid 35 snap -1 2024-09-13T10:12:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1921: expect (ObjNum 409 snap 0 seq_num 409) 2024-09-13T10:12:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1922: tier_promote oid 49 2024-09-13T10:12:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1910: finishing write tid 4 to smithi19530714-27 2024-09-13T10:12:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1910: finishing write tid 5 to smithi19530714-27 2024-09-13T10:12:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1910: oid 27 updating version 0 to 170 2024-09-13T10:12:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1910: oid 27 updating version 170 to 172 2024-09-13T10:12:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1910: oid 27 version 172 is already newer than 171 2024-09-13T10:12:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1910: oid 27 version 172 is already newer than 169 2024-09-13T10:12:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 172 (ObjNum 422 snap 0 seq_num 422) dirty exists 2024-09-13T10:12:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1910: left oid 27 (ObjNum 422 snap 0 seq_num 422) 2024-09-13T10:12:17.517 INFO:tasks.rados.rados.0.smithi195.stdout:1916: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:12:17.554 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2574820 2024-09-13T10:12:17.556 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:12:17.555+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:12:17.621 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:17.622 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:17.635 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T10:12:17.671 INFO:tasks.rados.rados.0.smithi195.stdout:1913: finishing set_chunk to oid 45 2024-09-13T10:12:17.671 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 180 (ObjNum 397 snap 0 seq_num 397) dirty exists 2024-09-13T10:12:17.671 INFO:tasks.rados.rados.0.smithi195.stdout:1914: expect (ObjNum 337 snap 0 seq_num 337) 2024-09-13T10:12:17.671 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:17.665+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:17.713 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-13T10:12:17.751 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:17.783 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:12:17.778+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:12:17.798 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:17.798 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:17.812 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T10:12:17.839 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:17.858 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:12:17.850+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:12:17.919 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:17.919 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:17.928 INFO:tasks.rados.rados.0.smithi195.stdout:1917: finishing write tid 1 to smithi19530714-10 2024-09-13T10:12:17.928 INFO:tasks.rados.rados.0.smithi195.stdout:1917: finishing write tid 2 to smithi19530714-10 2024-09-13T10:12:17.928 INFO:tasks.rados.rados.0.smithi195.stdout:1910: done (11 left) 2024-09-13T10:12:17.928 INFO:tasks.rados.rados.0.smithi195.stdout:1913: done (10 left) 2024-09-13T10:12:17.928 INFO:tasks.rados.rados.0.smithi195.stdout:1914: done (9 left) 2024-09-13T10:12:17.929 INFO:tasks.rados.rados.0.smithi195.stdout:1916: done (8 left) 2024-09-13T10:12:17.929 INFO:tasks.rados.rados.0.smithi195.stdout:1923: write oid 45 current snap is 0 2024-09-13T10:12:17.929 INFO:tasks.rados.rados.0.smithi195.stdout:1923: seq_num 425 ranges {742975=780240,1953553=427132,2971713=171262} 2024-09-13T10:12:17.933 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T10:12:17.936 INFO:tasks.rados.rados.0.smithi195.stdout:1923: writing smithi19530714-45 from 742975 to 1523215 tid 1 2024-09-13T10:12:17.940 INFO:tasks.rados.rados.0.smithi195.stdout:1923: writing smithi19530714-45 from 1953553 to 2380685 tid 2 2024-09-13T10:12:17.945 INFO:tasks.rados.rados.0.smithi195.stdout:1923: writing smithi19530714-45 from 2971713 to 3142975 tid 3 2024-09-13T10:12:17.945 INFO:tasks.rados.rados.0.smithi195.stdout:1924: read oid 19 snap -1 2024-09-13T10:12:17.945 INFO:tasks.rados.rados.0.smithi195.stdout:1924: expect (ObjNum 346 snap 0 seq_num 346) 2024-09-13T10:12:17.945 INFO:tasks.rados.rados.0.smithi195.stdout:1925: read oid 43 snap -1 2024-09-13T10:12:17.945 INFO:tasks.rados.rados.0.smithi195.stdout:1925: expect (ObjNum 344 snap 0 seq_num 344) 2024-09-13T10:12:17.945 INFO:tasks.rados.rados.0.smithi195.stdout:1926: delete oid 4 current snap is 0 2024-09-13T10:12:17.945 INFO:tasks.rados.rados.0.smithi195.stdout:1917: finishing write tid 3 to smithi19530714-10 2024-09-13T10:12:17.945 INFO:tasks.rados.rados.0.smithi195.stdout:1917: finishing write tid 4 to smithi19530714-10 2024-09-13T10:12:17.946 INFO:tasks.rados.rados.0.smithi195.stdout:1917: finishing write tid 5 to smithi19530714-10 2024-09-13T10:12:17.946 INFO:tasks.rados.rados.0.smithi195.stdout:1917: oid 10 updating version 0 to 41 2024-09-13T10:12:17.946 INFO:tasks.rados.rados.0.smithi195.stdout:1917: oid 10 updating version 41 to 43 2024-09-13T10:12:17.946 INFO:tasks.rados.rados.0.smithi195.stdout:1917: oid 10 updating version 43 to 44 2024-09-13T10:12:17.946 INFO:tasks.rados.rados.0.smithi195.stdout:1917: oid 10 version 44 is already newer than 42 2024-09-13T10:12:17.946 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 10 v 44 (ObjNum 423 snap 0 seq_num 423) dirty exists 2024-09-13T10:12:17.946 INFO:tasks.rados.rados.0.smithi195.stdout:1917: left oid 10 (ObjNum 423 snap 0 seq_num 423) 2024-09-13T10:12:17.946 INFO:tasks.rados.rados.0.smithi195.stdout:1918: finishing write tid 1 to smithi19530714-15 2024-09-13T10:12:17.946 INFO:tasks.rados.rados.0.smithi195.stdout:1918: finishing write tid 2 to smithi19530714-15 2024-09-13T10:12:17.946 INFO:tasks.rados.rados.0.smithi195.stdout:1918: finishing write tid 3 to smithi19530714-15 2024-09-13T10:12:17.946 INFO:tasks.rados.rados.0.smithi195.stdout:1918: finishing write tid 4 to smithi19530714-15 2024-09-13T10:12:17.946 INFO:tasks.rados.rados.0.smithi195.stdout:1918: oid 15 updating version 0 to 143 2024-09-13T10:12:17.946 INFO:tasks.rados.rados.0.smithi195.stdout:1918: oid 15 version 143 is already newer than 141 2024-09-13T10:12:17.947 INFO:tasks.rados.rados.0.smithi195.stdout:1918: oid 15 version 143 is already newer than 142 2024-09-13T10:12:17.947 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 15 v 143 (ObjNum 424 snap 0 seq_num 424) dirty exists 2024-09-13T10:12:17.947 INFO:tasks.rados.rados.0.smithi195.stdout:1918: left oid 15 (ObjNum 424 snap 0 seq_num 424) 2024-09-13T10:12:17.947 INFO:tasks.rados.rados.0.smithi195.stdout:1922: got (0) Success 2024-09-13T10:12:17.947 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 190 (ObjNum 413 snap 0 seq_num 413) dirty exists 2024-09-13T10:12:17.947 INFO:tasks.rados.rados.0.smithi195.stdout:1921: expect (ObjNum 409 snap 0 seq_num 409) 2024-09-13T10:12:17.954 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2573751 2024-09-13T10:12:17.959 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:17.954+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:17.982 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:18.012 INFO:tasks.rados.rados.0.smithi195.stdout:1920: expect (ObjNum 421 snap 0 seq_num 421) 2024-09-13T10:12:18.018 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 661485 2024-09-13T10:12:18.046 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:18.047 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:18.059 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T10:12:18.060 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:18.054+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:18.160 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:18.157+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:18.161 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:18.227 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:18.227 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:18.237 INFO:tasks.rados.rados.0.smithi195.stdout:1919: expect (ObjNum 402 snap 0 seq_num 402) 2024-09-13T10:12:18.237 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:18.240 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-13T10:12:18.261 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:18.254+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:18.311 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2292911 2024-09-13T10:12:18.359 INFO:tasks.rados.rados.0.smithi195.stdout:1915: expect (ObjNum 360 snap 0 seq_num 360) 2024-09-13T10:12:18.360 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:18.362 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:18.362+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:18.396 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:18.396 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:18.397 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:18.410 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-13T10:12:18.463 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:12:18.458+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:12:18.468 INFO:tasks.rados.rados.0.smithi195.stdout:1923: finishing write tid 1 to smithi19530714-45 2024-09-13T10:12:18.469 INFO:tasks.rados.rados.0.smithi195.stdout:1923: finishing write tid 2 to smithi19530714-45 2024-09-13T10:12:18.469 INFO:tasks.rados.rados.0.smithi195.stdout:1915: done (11 left) 2024-09-13T10:12:18.469 INFO:tasks.rados.rados.0.smithi195.stdout:1917: done (10 left) 2024-09-13T10:12:18.469 INFO:tasks.rados.rados.0.smithi195.stdout:1918: done (9 left) 2024-09-13T10:12:18.469 INFO:tasks.rados.rados.0.smithi195.stdout:1919: done (8 left) 2024-09-13T10:12:18.469 INFO:tasks.rados.rados.0.smithi195.stdout:1920: done (7 left) 2024-09-13T10:12:18.469 INFO:tasks.rados.rados.0.smithi195.stdout:1921: done (6 left) 2024-09-13T10:12:18.469 INFO:tasks.rados.rados.0.smithi195.stdout:1922: done (5 left) 2024-09-13T10:12:18.469 INFO:tasks.rados.rados.0.smithi195.stdout:1926: done (4 left) 2024-09-13T10:12:18.469 INFO:tasks.rados.rados.0.smithi195.stdout:1927: write (excl) oid 42 current snap is 0 2024-09-13T10:12:18.469 INFO:tasks.rados.rados.0.smithi195.stdout:1923: finishing write tid 3 to smithi19530714-45 2024-09-13T10:12:18.469 INFO:tasks.rados.rados.0.smithi195.stdout:1923: finishing write tid 4 to smithi19530714-45 2024-09-13T10:12:18.469 INFO:tasks.rados.rados.0.smithi195.stdout:1923: finishing write tid 5 to smithi19530714-45 2024-09-13T10:12:18.469 INFO:tasks.rados.rados.0.smithi195.stdout:1923: oid 45 updating version 0 to 194 2024-09-13T10:12:18.470 INFO:tasks.rados.rados.0.smithi195.stdout:1923: oid 45 version 194 is already newer than 193 2024-09-13T10:12:18.470 INFO:tasks.rados.rados.0.smithi195.stdout:1923: oid 45 updating version 194 to 196 2024-09-13T10:12:18.470 INFO:tasks.rados.rados.0.smithi195.stdout:1923: oid 45 version 196 is already newer than 195 2024-09-13T10:12:18.470 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 45 v 196 (ObjNum 425 snap 0 seq_num 425) dirty exists 2024-09-13T10:12:18.470 INFO:tasks.rados.rados.0.smithi195.stdout:1923: left oid 45 (ObjNum 425 snap 0 seq_num 425) 2024-09-13T10:12:18.470 INFO:tasks.rados.rados.0.smithi195.stdout:1925: expect (ObjNum 344 snap 0 seq_num 344) 2024-09-13T10:12:18.563 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:18.557+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:18.569 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:18.569 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:18.583 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-13T10:12:18.599 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:18.664 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:12:18.662+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:12:18.666 INFO:tasks.rados.rados.0.smithi195.stdout:1893: finishing write tid 1 to smithi19530714-18 2024-09-13T10:12:18.666 INFO:tasks.rados.rados.0.smithi195.stdout:1893: finishing write tid 2 to smithi19530714-18 2024-09-13T10:12:18.666 INFO:tasks.rados.rados.0.smithi195.stdout:1893: finishing write tid 3 to smithi19530714-18 2024-09-13T10:12:18.666 INFO:tasks.rados.rados.0.smithi195.stdout:1893: finishing write tid 4 to smithi19530714-18 2024-09-13T10:12:18.667 INFO:tasks.rados.rados.0.smithi195.stdout:1893: oid 18 updating version 0 to 132 2024-09-13T10:12:18.667 INFO:tasks.rados.rados.0.smithi195.stdout:1893: oid 18 updating version 132 to 133 2024-09-13T10:12:18.667 INFO:tasks.rados.rados.0.smithi195.stdout:1893: oid 18 updating version 133 to 134 2024-09-13T10:12:18.667 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 134 (ObjNum 418 snap 0 seq_num 418) dirty exists 2024-09-13T10:12:18.667 INFO:tasks.rados.rados.0.smithi195.stdout:1893: left oid 18 (ObjNum 418 snap 0 seq_num 418) 2024-09-13T10:12:18.668 INFO:tasks.rados.rados.0.smithi195.stdout:1927: seq_num 427 ranges {776589=647091,2070517=430801,3026559=150030} 2024-09-13T10:12:18.689 INFO:tasks.rados.rados.0.smithi195.stdout:1927: writing smithi19530714-42 from 776589 to 1423680 tid 1 2024-09-13T10:12:18.704 INFO:tasks.rados.rados.0.smithi195.stdout:1927: writing smithi19530714-42 from 2070517 to 2501318 tid 2 2024-09-13T10:12:18.711 INFO:tasks.rados.rados.0.smithi195.stdout:1927: writing smithi19530714-42 from 3026559 to 3176589 tid 3 2024-09-13T10:12:18.711 INFO:tasks.rados.rados.0.smithi195.stdout:1893: done (4 left) 2024-09-13T10:12:18.711 INFO:tasks.rados.rados.0.smithi195.stdout:1923: done (3 left) 2024-09-13T10:12:18.711 INFO:tasks.rados.rados.0.smithi195.stdout:1925: done (2 left) 2024-09-13T10:12:18.711 INFO:tasks.rados.rados.0.smithi195.stdout:1928: write (excl) oid 8 current snap is 0 2024-09-13T10:12:18.712 INFO:tasks.rados.rados.0.smithi195.stdout:1928: seq_num 428 ranges {793396=722340,2128999=690812} 2024-09-13T10:12:18.718 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:18.718 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:18.725 INFO:tasks.rados.rados.0.smithi195.stdout:1928: writing smithi19530714-8 from 793396 to 1515736 tid 1 2024-09-13T10:12:18.731 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-13T10:12:18.737 INFO:tasks.rados.rados.0.smithi195.stdout:1928: writing smithi19530714-8 from 2128999 to 2819811 tid 2 2024-09-13T10:12:18.737 INFO:tasks.rados.rados.0.smithi195.stdout:1927: finishing write tid 1 to smithi19530714-42 2024-09-13T10:12:18.737 INFO:tasks.rados.rados.0.smithi195.stdout:1927: finishing write tid 2 to smithi19530714-42 2024-09-13T10:12:18.737 INFO:tasks.rados.rados.0.smithi195.stdout:1927: finishing write tid 3 to smithi19530714-42 2024-09-13T10:12:18.737 INFO:tasks.rados.rados.0.smithi195.stdout:1927: finishing write tid 4 to smithi19530714-42 2024-09-13T10:12:18.737 INFO:tasks.rados.rados.0.smithi195.stdout:1927: finishing write tid 5 to smithi19530714-42 2024-09-13T10:12:18.737 INFO:tasks.rados.rados.0.smithi195.stdout:1927: oid 42 updating version 0 to 32 2024-09-13T10:12:18.737 INFO:tasks.rados.rados.0.smithi195.stdout:1927: oid 42 updating version 32 to 34 2024-09-13T10:12:18.737 INFO:tasks.rados.rados.0.smithi195.stdout:1927: oid 42 version 34 is already newer than 33 2024-09-13T10:12:18.737 INFO:tasks.rados.rados.0.smithi195.stdout:1927: oid 42 version 34 is already newer than 31 2024-09-13T10:12:18.737 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 34 (ObjNum 427 snap 0 seq_num 427) dirty exists 2024-09-13T10:12:18.738 INFO:tasks.rados.rados.0.smithi195.stdout:1927: left oid 42 (ObjNum 427 snap 0 seq_num 427) 2024-09-13T10:12:18.738 INFO:tasks.rados.rados.0.smithi195.stdout:1927: done (2 left) 2024-09-13T10:12:18.738 INFO:tasks.rados.rados.0.smithi195.stdout:1929: read oid 24 snap -1 2024-09-13T10:12:18.738 INFO:tasks.rados.rados.0.smithi195.stdout:1929: expect (ObjNum 411 snap 0 seq_num 411) 2024-09-13T10:12:18.738 INFO:tasks.rados.rados.0.smithi195.stdout:1930: read oid 38 snap -1 2024-09-13T10:12:18.738 INFO:tasks.rados.rados.0.smithi195.stdout:1930: expect (ObjNum 298 snap 0 seq_num 298) 2024-09-13T10:12:18.738 INFO:tasks.rados.rados.0.smithi195.stdout:1931: read oid 11 snap -1 2024-09-13T10:12:18.738 INFO:tasks.rados.rados.0.smithi195.stdout:1931: expect (ObjNum 359 snap 0 seq_num 359) 2024-09-13T10:12:18.738 INFO:tasks.rados.rados.0.smithi195.stdout:1932: write (excl) oid 43 current snap is 0 2024-09-13T10:12:18.738 INFO:tasks.rados.rados.0.smithi195.stdout:1932: seq_num 429 ranges {410203=797589,1787481=434470,2681405=528798} 2024-09-13T10:12:18.747 INFO:tasks.rados.rados.0.smithi195.stdout:1932: writing smithi19530714-43 from 410203 to 1207792 tid 1 2024-09-13T10:12:18.756 INFO:tasks.rados.rados.0.smithi195.stdout:1932: writing smithi19530714-43 from 1787481 to 2221951 tid 2 2024-09-13T10:12:18.764 INFO:tasks.rados.rados.0.smithi195.stdout:1932: writing smithi19530714-43 from 2681405 to 3210203 tid 3 2024-09-13T10:12:18.764 INFO:tasks.rados.rados.0.smithi195.stdout:1933: write oid 29 current snap is 0 2024-09-13T10:12:18.764 INFO:tasks.rados.rados.0.smithi195.stdout:1933: seq_num 430 ranges {427010=472838,1329610=694481,2708828=518182} 2024-09-13T10:12:18.765 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:18.763+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:18.769 INFO:tasks.rados.rados.0.smithi195.stdout:1933: writing smithi19530714-29 from 427010 to 899848 tid 1 2024-09-13T10:12:18.777 INFO:tasks.rados.rados.0.smithi195.stdout:1933: writing smithi19530714-29 from 1329610 to 2024091 tid 2 2024-09-13T10:12:18.784 INFO:tasks.rados.rados.0.smithi195.stdout:1933: writing smithi19530714-29 from 2708828 to 3227010 tid 3 2024-09-13T10:12:18.784 INFO:tasks.rados.rados.0.smithi195.stdout:1934: write oid 40 current snap is 0 2024-09-13T10:12:18.784 INFO:tasks.rados.rados.0.smithi195.stdout:1934: seq_num 431 ranges {443817=548087,1788092=438139,3019898=223919} 2024-09-13T10:12:18.789 INFO:tasks.rados.rados.0.smithi195.stdout:1934: writing smithi19530714-40 from 443817 to 991904 tid 1 2024-09-13T10:12:18.796 INFO:tasks.rados.rados.0.smithi195.stdout:1934: writing smithi19530714-40 from 1788092 to 2226231 tid 2 2024-09-13T10:12:18.802 INFO:tasks.rados.rados.0.smithi195.stdout:1934: writing smithi19530714-40 from 3019898 to 3243817 tid 3 2024-09-13T10:12:18.802 INFO:tasks.rados.rados.0.smithi195.stdout:1935: delete oid 10 current snap is 0 2024-09-13T10:12:18.802 INFO:tasks.rados.rados.0.smithi195.stdout:1931: expect (ObjNum 359 snap 0 seq_num 359) 2024-09-13T10:12:18.802 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:18.860 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 595993 2024-09-13T10:12:18.864 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:18.864 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:18.866 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:18.861+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:18.873 INFO:tasks.rados.rados.0.smithi195.stdout:1928: finishing write tid 1 to smithi19530714-8 2024-09-13T10:12:18.874 INFO:tasks.rados.rados.0.smithi195.stdout:1928: finishing write tid 2 to smithi19530714-8 2024-09-13T10:12:18.874 INFO:tasks.rados.rados.0.smithi195.stdout:1928: finishing write tid 3 to smithi19530714-8 2024-09-13T10:12:18.874 INFO:tasks.rados.rados.0.smithi195.stdout:1928: finishing write tid 4 to smithi19530714-8 2024-09-13T10:12:18.874 INFO:tasks.rados.rados.0.smithi195.stdout:1928: oid 8 updating version 0 to 32 2024-09-13T10:12:18.874 INFO:tasks.rados.rados.0.smithi195.stdout:1928: oid 8 version 32 is already newer than 31 2024-09-13T10:12:18.874 INFO:tasks.rados.rados.0.smithi195.stdout:1928: oid 8 version 32 is already newer than 30 2024-09-13T10:12:18.874 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 8 v 32 (ObjNum 428 snap 0 seq_num 428) dirty exists 2024-09-13T10:12:18.874 INFO:tasks.rados.rados.0.smithi195.stdout:1928: left oid 8 (ObjNum 428 snap 0 seq_num 428) 2024-09-13T10:12:18.874 INFO:tasks.rados.rados.0.smithi195.stdout:1932: finishing write tid 1 to smithi19530714-43 2024-09-13T10:12:18.874 INFO:tasks.rados.rados.0.smithi195.stdout:1929: expect (ObjNum 411 snap 0 seq_num 411) 2024-09-13T10:12:18.874 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:18.877 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-13T10:12:18.911 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:18.944 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:12:18.949 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:18.966 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:18.961+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:18.988 INFO:tasks.rados.rados.0.smithi195.stdout:1932: finishing write tid 2 to smithi19530714-43 2024-09-13T10:12:18.988 INFO:tasks.rados.rados.0.smithi195.stdout:1932: finishing write tid 3 to smithi19530714-43 2024-09-13T10:12:18.988 INFO:tasks.rados.rados.0.smithi195.stdout:1932: finishing write tid 4 to smithi19530714-43 2024-09-13T10:12:18.988 INFO:tasks.rados.rados.0.smithi195.stdout:1932: finishing write tid 5 to smithi19530714-43 2024-09-13T10:12:18.988 INFO:tasks.rados.rados.0.smithi195.stdout:1932: oid 43 updating version 0 to 119 2024-09-13T10:12:18.988 INFO:tasks.rados.rados.0.smithi195.stdout:1932: oid 43 version 119 is already newer than 118 2024-09-13T10:12:18.988 INFO:tasks.rados.rados.0.smithi195.stdout:1932: oid 43 updating version 119 to 121 2024-09-13T10:12:18.988 INFO:tasks.rados.rados.0.smithi195.stdout:1932: oid 43 version 121 is already newer than 120 2024-09-13T10:12:18.988 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 121 (ObjNum 429 snap 0 seq_num 429) dirty exists 2024-09-13T10:12:18.988 INFO:tasks.rados.rados.0.smithi195.stdout:1932: left oid 43 (ObjNum 429 snap 0 seq_num 429) 2024-09-13T10:12:18.988 INFO:tasks.rados.rados.0.smithi195.stdout:1930: expect (ObjNum 298 snap 0 seq_num 298) 2024-09-13T10:12:19.035 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:19.035 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:19.048 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-13T10:12:19.067 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:19.062+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:19.092 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1475678 2024-09-13T10:12:19.100 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:12:19.100 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:12:19.121 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:12:19.121 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:12:19.142 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:12:19.143 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:12:19.152 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 305433 2024-09-13T10:12:19.161 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:12:19.161 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:12:19.169 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:19.165+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:19.178 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:12:19.179 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:12:19.199 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:12:19.199 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:12:19.210 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:19.210 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:19.221 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:12:19.221 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:12:19.223 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T10:12:19.231 INFO:tasks.rados.rados.0.smithi195.stdout:1924: expect (ObjNum 346 snap 0 seq_num 346) 2024-09-13T10:12:19.231 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:19.240 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:12:19.240 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:12:19.258 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:12:19.258 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:12:19.269 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:19.262+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:19.280 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:12:19.280 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:12:19.300 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:12:19.300 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:12:19.320 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:12:19.321 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:12:19.341 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:12:19.342 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:12:19.346 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:19.346 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:12:19.346 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:12:19.347 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:19.361 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T10:12:19.363 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:12:19.364 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:12:19.386 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:12:19.386 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:12:19.387 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:19.382+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:19.406 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:12:19.406 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:12:19.464 INFO:tasks.rados.rados.0.smithi195.stdout:1933: finishing write tid 1 to smithi19530714-29 2024-09-13T10:12:19.464 INFO:tasks.rados.rados.0.smithi195.stdout:1933: finishing write tid 2 to smithi19530714-29 2024-09-13T10:12:19.465 INFO:tasks.rados.rados.0.smithi195.stdout:1933: finishing write tid 3 to smithi19530714-29 2024-09-13T10:12:19.465 INFO:tasks.rados.rados.0.smithi195.stdout:1933: finishing write tid 4 to smithi19530714-29 2024-09-13T10:12:19.465 INFO:tasks.rados.rados.0.smithi195.stdout:1933: finishing write tid 5 to smithi19530714-29 2024-09-13T10:12:19.465 INFO:tasks.rados.rados.0.smithi195.stdout:1933: oid 29 updating version 0 to 146 2024-09-13T10:12:19.465 INFO:tasks.rados.rados.0.smithi195.stdout:1933: oid 29 version 146 is already newer than 144 2024-09-13T10:12:19.465 INFO:tasks.rados.rados.0.smithi195.stdout:1933: oid 29 updating version 146 to 147 2024-09-13T10:12:19.465 INFO:tasks.rados.rados.0.smithi195.stdout:1933: oid 29 version 147 is already newer than 145 2024-09-13T10:12:19.465 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 147 (ObjNum 430 snap 0 seq_num 430) dirty exists 2024-09-13T10:12:19.465 INFO:tasks.rados.rados.0.smithi195.stdout:1933: left oid 29 (ObjNum 430 snap 0 seq_num 430) 2024-09-13T10:12:19.465 INFO:tasks.rados.rados.0.smithi195.stdout:1934: finishing write tid 1 to smithi19530714-40 2024-09-13T10:12:19.465 INFO:tasks.rados.rados.0.smithi195.stdout:1934: finishing write tid 2 to smithi19530714-40 2024-09-13T10:12:19.465 INFO:tasks.rados.rados.0.smithi195.stdout:1934: finishing write tid 3 to smithi19530714-40 2024-09-13T10:12:19.465 INFO:tasks.rados.rados.0.smithi195.stdout:1934: finishing write tid 4 to smithi19530714-40 2024-09-13T10:12:19.466 INFO:tasks.rados.rados.0.smithi195.stdout:1934: finishing write tid 5 to smithi19530714-40 2024-09-13T10:12:19.466 INFO:tasks.rados.rados.0.smithi195.stdout:1934: oid 40 updating version 0 to 42 2024-09-13T10:12:19.466 INFO:tasks.rados.rados.0.smithi195.stdout:1934: oid 40 version 42 is already newer than 40 2024-09-13T10:12:19.466 INFO:tasks.rados.rados.0.smithi195.stdout:1934: oid 40 updating version 42 to 43 2024-09-13T10:12:19.466 INFO:tasks.rados.rados.0.smithi195.stdout:1934: oid 40 version 43 is already newer than 41 2024-09-13T10:12:19.466 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 40 v 43 (ObjNum 431 snap 0 seq_num 431) dirty exists 2024-09-13T10:12:19.466 INFO:tasks.rados.rados.0.smithi195.stdout:1934: left oid 40 (ObjNum 431 snap 0 seq_num 431) 2024-09-13T10:12:19.466 INFO:tasks.rados.rados.0.smithi195.stdout:1924: done (8 left) 2024-09-13T10:12:19.466 INFO:tasks.rados.rados.0.smithi195.stdout:1928: done (7 left) 2024-09-13T10:12:19.466 INFO:tasks.rados.rados.0.smithi195.stdout:1929: done (6 left) 2024-09-13T10:12:19.466 INFO:tasks.rados.rados.0.smithi195.stdout:1930: done (5 left) 2024-09-13T10:12:19.466 INFO:tasks.rados.rados.0.smithi195.stdout:1931: done (4 left) 2024-09-13T10:12:19.467 INFO:tasks.rados.rados.0.smithi195.stdout:1932: done (3 left) 2024-09-13T10:12:19.467 INFO:tasks.rados.rados.0.smithi195.stdout:1933: done (2 left) 2024-09-13T10:12:19.467 INFO:tasks.rados.rados.0.smithi195.stdout:1934: done (1 left) 2024-09-13T10:12:19.467 INFO:tasks.rados.rados.0.smithi195.stdout:1935: done (0 left) 2024-09-13T10:12:19.467 INFO:tasks.rados.rados.0.smithi195.stdout:1936: read oid 12 snap -1 2024-09-13T10:12:19.467 INFO:tasks.rados.rados.0.smithi195.stdout:1936: expect (ObjNum 395 snap 0 seq_num 395) 2024-09-13T10:12:19.467 INFO:tasks.rados.rados.0.smithi195.stdout:1937: set_chunk oid 44 target oid 2024-09-13T10:12:19.467 INFO:tasks.rados.rados.0.smithi195.stdout:1937: set_chunk oid 44 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:12:19.467 INFO:tasks.rados.rados.0.smithi195.stdout:1938: read oid 13 snap -1 2024-09-13T10:12:19.467 INFO:tasks.rados.rados.0.smithi195.stdout:1938: expect (ObjNum 342 snap 0 seq_num 342) 2024-09-13T10:12:19.467 INFO:tasks.rados.rados.0.smithi195.stdout:1939: write oid 19 current snap is 0 2024-09-13T10:12:19.467 INFO:tasks.rados.rados.0.smithi195.stdout:1939: seq_num 433 ranges {477431=698585,1905056=441808,3074744=202687} 2024-09-13T10:12:19.471 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:19.465+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:19.479 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:19.480 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:12:19.480 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:12:19.480 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:12:19.480 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:19.480 INFO:tasks.rados.rados.0.smithi195.stdout:1939: writing smithi19530714-19 from 477431 to 1176016 tid 1 2024-09-13T10:12:19.494 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T10:12:19.496 INFO:tasks.rados.rados.0.smithi195.stdout:1939: writing smithi19530714-19 from 1905056 to 2346864 tid 2 2024-09-13T10:12:19.508 INFO:tasks.rados.rados.0.smithi195.stdout:1939: writing smithi19530714-19 from 3074744 to 3277431 tid 3 2024-09-13T10:12:19.509 INFO:tasks.rados.rados.0.smithi195.stdout:1940: read oid 36 snap -1 2024-09-13T10:12:19.509 INFO:tasks.rados.rados.0.smithi195.stdout:1940: expect deleted 2024-09-13T10:12:19.509 INFO:tasks.rados.rados.0.smithi195.stdout:1941: write (excl) oid 5 current snap is 0 2024-09-13T10:12:19.509 INFO:tasks.rados.rados.0.smithi195.stdout:1937: finishing set_chunk to oid 44 2024-09-13T10:12:19.509 INFO:tasks.rados.rados.0.smithi195.stdout:1937: got expected ENOENT (src dne) 2024-09-13T10:12:19.509 INFO:tasks.rados.rados.0.smithi195.stdout:1936: expect (ObjNum 395 snap 0 seq_num 395) 2024-09-13T10:12:19.572 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:12:19.571+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:12:19.575 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1687176 2024-09-13T10:12:19.594 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:19.618 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:19.619 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:12:19.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:12:19.619 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:12:19.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.619 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:202 3.2 e38/30 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'1, mlcod=0'0)", 2024-09-13T10:12:19.619 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:12:19.619 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 88.613024938999999, 2024-09-13T10:12:19.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016741669000000001, 2024-09-13T10:12:19.619 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.619 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.619 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.619 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:12:19.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002367+0000", 2024-09-13T10:12:19.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002368+0000", 2024-09-13T10:12:19.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1370000000000001e-06 2024-09-13T10:12:19.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010047+0000", 2024-09-13T10:12:19.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007678748 2024-09-13T10:12:19.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010081+0000", 2024-09-13T10:12:19.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4011999999999999e-05 2024-09-13T10:12:19.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010098+0000", 2024-09-13T10:12:19.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7125999999999998e-05 2024-09-13T10:12:19.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010253+0000", 2024-09-13T10:12:19.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000154881 2024-09-13T10:12:19.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010294+0000", 2024-09-13T10:12:19.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1746000000000003e-05 2024-09-13T10:12:19.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.018922+0000", 2024-09-13T10:12:19.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0086279619999999994 2024-09-13T10:12:19.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.623 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.623 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.019109+0000", 2024-09-13T10:12:19.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000186057 2024-09-13T10:12:19.623 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.623 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.624 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.624 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:206 3.2 e38/30 3:4cbdbe9e:::smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:12:19.624 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:12:19.624 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 88.604640208000006, 2024-09-13T10:12:19.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011224615, 2024-09-13T10:12:19.624 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.624 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.624 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.624 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:12:19.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010752+0000", 2024-09-13T10:12:19.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.010754+0000", 2024-09-13T10:12:19.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9999999999999999e-06 2024-09-13T10:12:19.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016750+0000", 2024-09-13T10:12:19.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059966480000000003 2024-09-13T10:12:19.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016782+0000", 2024-09-13T10:12:19.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2218999999999998e-05 2024-09-13T10:12:19.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016799+0000", 2024-09-13T10:12:19.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6685999999999999e-05 2024-09-13T10:12:19.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016923+0000", 2024-09-13T10:12:19.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000124122 2024-09-13T10:12:19.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.016962+0000", 2024-09-13T10:12:19.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8748999999999998e-05 2024-09-13T10:12:19.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021869+0000", 2024-09-13T10:12:19.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049073249999999997 2024-09-13T10:12:19.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021976+0000", 2024-09-13T10:12:19.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000106866 2024-09-13T10:12:19.628 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.628 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.628 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.629 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:214 3.2 e38/30 3:68ba95ba:::smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'3, mlcod=0'0)", 2024-09-13T10:12:19.629 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:12:19.629 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 88.594937552000005, 2024-09-13T10:12:19.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011518149, 2024-09-13T10:12:19.629 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.629 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.629 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.629 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.629 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.629 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:12:19.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020454+0000", 2024-09-13T10:12:19.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020456+0000", 2024-09-13T10:12:19.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0430000000000002e-06 2024-09-13T10:12:19.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026998+0000", 2024-09-13T10:12:19.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065420169999999998 2024-09-13T10:12:19.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027030+0000", 2024-09-13T10:12:19.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1976e-05 2024-09-13T10:12:19.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027048+0000", 2024-09-13T10:12:19.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7615e-05 2024-09-13T10:12:19.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027206+0000", 2024-09-13T10:12:19.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015788900000000001 2024-09-13T10:12:19.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027283+0000", 2024-09-13T10:12:19.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6942999999999998e-05 2024-09-13T10:12:19.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.031808+0000", 2024-09-13T10:12:19.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045256940000000002 2024-09-13T10:12:19.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.031972+0000", 2024-09-13T10:12:19.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016397199999999999 2024-09-13T10:12:19.633 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.633 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.633 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.633 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:210 3.2 e38/30 3:46ab5247:::smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:12:19.633 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:12:19.633 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 88.587071834, 2024-09-13T10:12:19.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015681245, 2024-09-13T10:12:19.634 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.634 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.634 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.634 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:12:19.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028320+0000", 2024-09-13T10:12:19.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.028322+0000", 2024-09-13T10:12:19.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0200000000000001e-06 2024-09-13T10:12:19.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037903+0000", 2024-09-13T10:12:19.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095811639999999997 2024-09-13T10:12:19.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037945+0000", 2024-09-13T10:12:19.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1891000000000003e-05 2024-09-13T10:12:19.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.037971+0000", 2024-09-13T10:12:19.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5829e-05 2024-09-13T10:12:19.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038125+0000", 2024-09-13T10:12:19.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000153852 2024-09-13T10:12:19.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038193+0000", 2024-09-13T10:12:19.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7995000000000002e-05 2024-09-13T10:12:19.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.043904+0000", 2024-09-13T10:12:19.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0057110340000000002 2024-09-13T10:12:19.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.044001+0000", 2024-09-13T10:12:19.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7460000000000005e-05 2024-09-13T10:12:19.638 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.638 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.638 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.638 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:218 3.2 e38/30 3:5915c927:::smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'5, mlcod=38'2)", 2024-09-13T10:12:19.638 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:12:19.639 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 88.573709248, 2024-09-13T10:12:19.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012988942, 2024-09-13T10:12:19.639 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.639 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.639 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.639 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:12:19.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041683+0000", 2024-09-13T10:12:19.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.041685+0000", 2024-09-13T10:12:19.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7769999999999998e-06 2024-09-13T10:12:19.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046575+0000", 2024-09-13T10:12:19.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048899520000000004 2024-09-13T10:12:19.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046633+0000", 2024-09-13T10:12:19.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7561e-05 2024-09-13T10:12:19.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046657+0000", 2024-09-13T10:12:19.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4223000000000002e-05 2024-09-13T10:12:19.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046819+0000", 2024-09-13T10:12:19.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016167699999999999 2024-09-13T10:12:19.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046882+0000", 2024-09-13T10:12:19.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3467999999999996e-05 2024-09-13T10:12:19.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054529+0000", 2024-09-13T10:12:19.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0076467289999999997 2024-09-13T10:12:19.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054671+0000", 2024-09-13T10:12:19.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014255499999999999 2024-09-13T10:12:19.643 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.643 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.643 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.643 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:217 3.2 e38/30 3:554c406d:::smithi19530714-18 low_tier:head v 38'6, mlcod=38'2)", 2024-09-13T10:12:19.643 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:12:19.643 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 88.568662458000006, 2024-09-13T10:12:19.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013473855999999999, 2024-09-13T10:12:19.643 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.643 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.643 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.644 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:12:19.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046729+0000", 2024-09-13T10:12:19.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046731+0000", 2024-09-13T10:12:19.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.813e-06 2024-09-13T10:12:19.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054994+0000", 2024-09-13T10:12:19.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0082631279999999998 2024-09-13T10:12:19.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055024+0000", 2024-09-13T10:12:19.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9839e-05 2024-09-13T10:12:19.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055044+0000", 2024-09-13T10:12:19.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9507000000000001e-05 2024-09-13T10:12:19.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055186+0000", 2024-09-13T10:12:19.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014243099999999999 2024-09-13T10:12:19.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.646 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055240+0000", 2024-09-13T10:12:19.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4444999999999998e-05 2024-09-13T10:12:19.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060135+0000", 2024-09-13T10:12:19.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048942949999999999 2024-09-13T10:12:19.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060203+0000", 2024-09-13T10:12:19.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8398000000000005e-05 2024-09-13T10:12:19.648 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.648 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.648 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.648 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:220 3.2 e38/30 3:753d5785:::smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'7, mlcod=38'3)", 2024-09-13T10:12:19.648 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:12:19.648 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 88.560281189999998, 2024-09-13T10:12:19.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.009287455, 2024-09-13T10:12:19.648 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.648 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.648 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.648 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:12:19.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055111+0000", 2024-09-13T10:12:19.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055112+0000", 2024-09-13T10:12:19.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6589999999999999e-06 2024-09-13T10:12:19.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.650 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057327+0000", 2024-09-13T10:12:19.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0022145229999999999 2024-09-13T10:12:19.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.650 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057354+0000", 2024-09-13T10:12:19.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7642000000000001e-05 2024-09-13T10:12:19.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057373+0000", 2024-09-13T10:12:19.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8768999999999999e-05 2024-09-13T10:12:19.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057511+0000", 2024-09-13T10:12:19.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013793900000000001 2024-09-13T10:12:19.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.057550+0000", 2024-09-13T10:12:19.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9301999999999997e-05 2024-09-13T10:12:19.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.064324+0000", 2024-09-13T10:12:19.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0067737429999999996 2024-09-13T10:12:19.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.064398+0000", 2024-09-13T10:12:19.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3878e-05 2024-09-13T10:12:19.652 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.652 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.652 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.653 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:227 3.2 e38/30 3:6ac7800c:::smithi19530714-28 low_tier:head v 38'8, mlcod=38'6)", 2024-09-13T10:12:19.653 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:12:19.653 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 88.54069054, 2024-09-13T10:12:19.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018812816, 2024-09-13T10:12:19.653 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.653 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.653 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.653 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:12:19.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074701+0000", 2024-09-13T10:12:19.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.074708+0000", 2024-09-13T10:12:19.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.968e-06 2024-09-13T10:12:19.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085262+0000", 2024-09-13T10:12:19.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010553842000000001 2024-09-13T10:12:19.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085289+0000", 2024-09-13T10:12:19.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7288999999999999e-05 2024-09-13T10:12:19.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085305+0000", 2024-09-13T10:12:19.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5925000000000001e-05 2024-09-13T10:12:19.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085418+0000", 2024-09-13T10:12:19.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112584 2024-09-13T10:12:19.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.085451+0000", 2024-09-13T10:12:19.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3605000000000001e-05 2024-09-13T10:12:19.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093417+0000", 2024-09-13T10:12:19.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0079654249999999999 2024-09-13T10:12:19.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093514+0000", 2024-09-13T10:12:19.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7177999999999997e-05 2024-09-13T10:12:19.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.657 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.658 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:230 3.2 e38/30 3:5d108080:::smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'9, mlcod=38'6)", 2024-09-13T10:12:19.658 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:12:19.658 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 88.526407509999999, 2024-09-13T10:12:19.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01953365, 2024-09-13T10:12:19.658 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.658 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.658 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.658 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:12:19.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088984+0000", 2024-09-13T10:12:19.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.088987+0000", 2024-09-13T10:12:19.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4600000000000002e-06 2024-09-13T10:12:19.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095842+0000", 2024-09-13T10:12:19.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0068548760000000002 2024-09-13T10:12:19.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095872+0000", 2024-09-13T10:12:19.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0476000000000001e-05 2024-09-13T10:12:19.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095890+0000", 2024-09-13T10:12:19.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7554000000000002e-05 2024-09-13T10:12:19.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.096006+0000", 2024-09-13T10:12:19.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011597199999999999 2024-09-13T10:12:19.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.096058+0000", 2024-09-13T10:12:19.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2274000000000001e-05 2024-09-13T10:12:19.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.108451+0000", 2024-09-13T10:12:19.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012392815 2024-09-13T10:12:19.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.108518+0000", 2024-09-13T10:12:19.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7223000000000003e-05 2024-09-13T10:12:19.662 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.662 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.662 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.662 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:232 3.2 e38/30 3:6538713c:::smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'10, mlcod=38'6)", 2024-09-13T10:12:19.662 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:12:19.663 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 88.519456629000004, 2024-09-13T10:12:19.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015307497, 2024-09-13T10:12:19.663 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.663 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.663 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.663 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:12:19.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095935+0000", 2024-09-13T10:12:19.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.095937+0000", 2024-09-13T10:12:19.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.367e-06 2024-09-13T10:12:19.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099841+0000", 2024-09-13T10:12:19.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0039043960000000001 2024-09-13T10:12:19.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099865+0000", 2024-09-13T10:12:19.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4538000000000001e-05 2024-09-13T10:12:19.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099881+0000", 2024-09-13T10:12:19.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5809000000000001e-05 2024-09-13T10:12:19.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.099995+0000", 2024-09-13T10:12:19.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113843 2024-09-13T10:12:19.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.100043+0000", 2024-09-13T10:12:19.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8291999999999998e-05 2024-09-13T10:12:19.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111200+0000", 2024-09-13T10:12:19.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011156291 2024-09-13T10:12:19.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111243+0000", 2024-09-13T10:12:19.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2960999999999997e-05 2024-09-13T10:12:19.667 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.667 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.667 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.667 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:236 3.2 e38/30 3:73c2893b:::smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'11, mlcod=38'7)", 2024-09-13T10:12:19.667 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:12:19.667 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 88.509956510999999, 2024-09-13T10:12:19.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0098707670000000008, 2024-09-13T10:12:19.667 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.668 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.668 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.668 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:12:19.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105435+0000", 2024-09-13T10:12:19.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.105438+0000", 2024-09-13T10:12:19.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3010000000000001e-06 2024-09-13T10:12:19.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112262+0000", 2024-09-13T10:12:19.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.006824793 2024-09-13T10:12:19.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112279+0000", 2024-09-13T10:12:19.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6829000000000001e-05 2024-09-13T10:12:19.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112293+0000", 2024-09-13T10:12:19.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3801000000000001e-05 2024-09-13T10:12:19.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112381+0000", 2024-09-13T10:12:19.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7825999999999996e-05 2024-09-13T10:12:19.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112411+0000", 2024-09-13T10:12:19.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0434e-05 2024-09-13T10:12:19.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.115246+0000", 2024-09-13T10:12:19.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028352360000000001 2024-09-13T10:12:19.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.115306+0000", 2024-09-13T10:12:19.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9546999999999998e-05 2024-09-13T10:12:19.671 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.671 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.672 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.672 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:240 3.2 e38/30 3:7eee2f4a:::smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'12, mlcod=38'7)", 2024-09-13T10:12:19.672 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:12:19.672 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 88.503046724000001, 2024-09-13T10:12:19.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011743134000000001, 2024-09-13T10:12:19.672 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.672 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.672 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.672 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:12:19.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112345+0000", 2024-09-13T10:12:19.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112346+0000", 2024-09-13T10:12:19.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.037e-06 2024-09-13T10:12:19.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120152+0000", 2024-09-13T10:12:19.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0078054700000000001 2024-09-13T10:12:19.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120173+0000", 2024-09-13T10:12:19.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1498999999999999e-05 2024-09-13T10:12:19.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120188+0000", 2024-09-13T10:12:19.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5159e-05 2024-09-13T10:12:19.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120313+0000", 2024-09-13T10:12:19.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012440199999999999 2024-09-13T10:12:19.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120358+0000", 2024-09-13T10:12:19.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5828000000000002e-05 2024-09-13T10:12:19.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.123945+0000", 2024-09-13T10:12:19.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035865490000000001 2024-09-13T10:12:19.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.124088+0000", 2024-09-13T10:12:19.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014318999999999999 2024-09-13T10:12:19.676 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.676 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.676 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.677 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:242 3.2 e38/30 3:501d22a1:::smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'13, mlcod=38'7)", 2024-09-13T10:12:19.677 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:12:19.677 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 88.495153716000004, 2024-09-13T10:12:19.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0088394330000000007, 2024-09-13T10:12:19.677 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.677 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.677 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.677 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:12:19.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120238+0000", 2024-09-13T10:12:19.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.120240+0000", 2024-09-13T10:12:19.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.731e-06 2024-09-13T10:12:19.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125395+0000", 2024-09-13T10:12:19.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0051547490000000001 2024-09-13T10:12:19.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125412+0000", 2024-09-13T10:12:19.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7833e-05 2024-09-13T10:12:19.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125426+0000", 2024-09-13T10:12:19.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3237999999999999e-05 2024-09-13T10:12:19.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125541+0000", 2024-09-13T10:12:19.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011566299999999999 2024-09-13T10:12:19.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.125578+0000", 2024-09-13T10:12:19.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7197000000000003e-05 2024-09-13T10:12:19.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.128885+0000", 2024-09-13T10:12:19.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0033067119999999998 2024-09-13T10:12:19.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.129077+0000", 2024-09-13T10:12:19.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019231000000000001 2024-09-13T10:12:19.681 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.681 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.682 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.682 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:244 3.2 e38/30 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'14, mlcod=38'10)", 2024-09-13T10:12:19.682 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:12:19.682 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 88.489134347999993, 2024-09-13T10:12:19.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01536599, 2024-09-13T10:12:19.682 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.682 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.682 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.682 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:12:19.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126257+0000", 2024-09-13T10:12:19.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.126259+0000", 2024-09-13T10:12:19.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.528e-06 2024-09-13T10:12:19.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134336+0000", 2024-09-13T10:12:19.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080767410000000001 2024-09-13T10:12:19.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134356+0000", 2024-09-13T10:12:19.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9936999999999999e-05 2024-09-13T10:12:19.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134367+0000", 2024-09-13T10:12:19.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1352000000000001e-05 2024-09-13T10:12:19.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134532+0000", 2024-09-13T10:12:19.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001651 2024-09-13T10:12:19.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134578+0000", 2024-09-13T10:12:19.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5442000000000003e-05 2024-09-13T10:12:19.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.141521+0000", 2024-09-13T10:12:19.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069432720000000003 2024-09-13T10:12:19.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.141623+0000", 2024-09-13T10:12:19.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102618 2024-09-13T10:12:19.687 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.687 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.687 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.687 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:250 3.2 e38/30 3:78d40eb9:::smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'15, mlcod=38'10)", 2024-09-13T10:12:19.687 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:12:19.687 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 88.480937877000002, 2024-09-13T10:12:19.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0095447210000000008, 2024-09-13T10:12:19.687 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.687 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.687 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.687 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:12:19.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134454+0000", 2024-09-13T10:12:19.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.134455+0000", 2024-09-13T10:12:19.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2699999999999999e-07 2024-09-13T10:12:19.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139352+0000", 2024-09-13T10:12:19.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048977810000000004 2024-09-13T10:12:19.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139374+0000", 2024-09-13T10:12:19.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1339e-05 2024-09-13T10:12:19.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139384+0000", 2024-09-13T10:12:19.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0183999999999999e-05 2024-09-13T10:12:19.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139477+0000", 2024-09-13T10:12:19.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2590999999999995e-05 2024-09-13T10:12:19.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139515+0000", 2024-09-13T10:12:19.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8624999999999997e-05 2024-09-13T10:12:19.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.143939+0000", 2024-09-13T10:12:19.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0044241189999999998 2024-09-13T10:12:19.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.143999+0000", 2024-09-13T10:12:19.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9354999999999997e-05 2024-09-13T10:12:19.691 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.691 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.691 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.692 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:245 3.2 e38/30 3:7837e849:::smithi19530714-46 low_tier:head v 38'16, mlcod=38'10)", 2024-09-13T10:12:19.692 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:12:19.692 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 88.475981523000002, 2024-09-13T10:12:19.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012687423999999999, 2024-09-13T10:12:19.692 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.692 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.692 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.692 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:12:19.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139410+0000", 2024-09-13T10:12:19.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.139411+0000", 2024-09-13T10:12:19.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6300000000000005e-07 2024-09-13T10:12:19.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144635+0000", 2024-09-13T10:12:19.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0052242169999999997 2024-09-13T10:12:19.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144651+0000", 2024-09-13T10:12:19.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6371e-05 2024-09-13T10:12:19.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144662+0000", 2024-09-13T10:12:19.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0003000000000001e-05 2024-09-13T10:12:19.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144741+0000", 2024-09-13T10:12:19.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9614e-05 2024-09-13T10:12:19.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.144768+0000", 2024-09-13T10:12:19.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6826999999999999e-05 2024-09-13T10:12:19.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152015+0000", 2024-09-13T10:12:19.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0072470879999999996 2024-09-13T10:12:19.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152098+0000", 2024-09-13T10:12:19.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2640999999999997e-05 2024-09-13T10:12:19.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.696 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.696 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:2811 3.2 e71/30 3:5a2483ca:::890eeb33aa7613d6935d7cb9ed4860e522183188e5dc23a51d7045443a8d23dd:head v 71'264, mlcod=70'262)", 2024-09-13T10:12:19.696 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:32.583079+0000", 2024-09-13T10:12:19.696 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.032312326000003, 2024-09-13T10:12:19.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0094005459999999992, 2024-09-13T10:12:19.697 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.697 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.697 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.697 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.583079+0000", 2024-09-13T10:12:19.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.583079+0000", 2024-09-13T10:12:19.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.583082+0000", 2024-09-13T10:12:19.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3070000000000001e-06 2024-09-13T10:12:19.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.587892+0000", 2024-09-13T10:12:19.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048105889999999997 2024-09-13T10:12:19.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.587924+0000", 2024-09-13T10:12:19.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1217999999999998e-05 2024-09-13T10:12:19.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.587938+0000", 2024-09-13T10:12:19.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4341999999999999e-05 2024-09-13T10:12:19.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.588076+0000", 2024-09-13T10:12:19.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013836600000000001 2024-09-13T10:12:19.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.588116+0000", 2024-09-13T10:12:19.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9480999999999997e-05 2024-09-13T10:12:19.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.592358+0000", 2024-09-13T10:12:19.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042423740000000001 2024-09-13T10:12:19.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:32.592480+0000", 2024-09-13T10:12:19.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012186899999999999 2024-09-13T10:12:19.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.701 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.701 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(osd.6.0:365 4.5 e82/41 4:a3098556:::smithi19530714-22:head v 82'129, mlcod=79'127)", 2024-09-13T10:12:19.701 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:44.595715+0000", 2024-09-13T10:12:19.701 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 35.019676683, 2024-09-13T10:12:19.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010604906000000001, 2024-09-13T10:12:19.701 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.701 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.701 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.701 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.595715+0000", 2024-09-13T10:12:19.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.595715+0000", 2024-09-13T10:12:19.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.595716+0000", 2024-09-13T10:12:19.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8400000000000004e-07 2024-09-13T10:12:19.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.601483+0000", 2024-09-13T10:12:19.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0057671789999999999 2024-09-13T10:12:19.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.601511+0000", 2024-09-13T10:12:19.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8218999999999999e-05 2024-09-13T10:12:19.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.601526+0000", 2024-09-13T10:12:19.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4956e-05 2024-09-13T10:12:19.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.601667+0000", 2024-09-13T10:12:19.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000140774 2024-09-13T10:12:19.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.601709+0000", 2024-09-13T10:12:19.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2688999999999997e-05 2024-09-13T10:12:19.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.606248+0000", 2024-09-13T10:12:19.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004538065 2024-09-13T10:12:19.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:44.606320+0000", 2024-09-13T10:12:19.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2440000000000004e-05 2024-09-13T10:12:19.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.705 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.706 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:3445 3.2 e86/30 3:7ba83ace:::4c246a1df6a0d5acaf0a3a85249ff707b21f90a5f3db763dc6d544e5b7cbec73:head v 86'330, mlcod=85'328)", 2024-09-13T10:12:19.706 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:48.129770+0000", 2024-09-13T10:12:19.706 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.485621986999998, 2024-09-13T10:12:19.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010414537999999999, 2024-09-13T10:12:19.706 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.706 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.706 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.706 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.129770+0000", 2024-09-13T10:12:19.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.129770+0000", 2024-09-13T10:12:19.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.129771+0000", 2024-09-13T10:12:19.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.59e-06 2024-09-13T10:12:19.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.136002+0000", 2024-09-13T10:12:19.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0062306380000000001 2024-09-13T10:12:19.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.136024+0000", 2024-09-13T10:12:19.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2106000000000001e-05 2024-09-13T10:12:19.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.136035+0000", 2024-09-13T10:12:19.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1293e-05 2024-09-13T10:12:19.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.136115+0000", 2024-09-13T10:12:19.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9294000000000003e-05 2024-09-13T10:12:19.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.136136+0000", 2024-09-13T10:12:19.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1495999999999999e-05 2024-09-13T10:12:19.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.140076+0000", 2024-09-13T10:12:19.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0039398849999999997 2024-09-13T10:12:19.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:48.140184+0000", 2024-09-13T10:12:19.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108236 2024-09-13T10:12:19.710 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.710 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.710 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.710 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:3917 3.2 e94/30 3:55300012:::a6332e51e016c24df8505825880ac4e703da29058ee795e9033d5e12bb724144:head v 94'352, mlcod=93'350)", 2024-09-13T10:12:19.710 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:58.912836+0000", 2024-09-13T10:12:19.711 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 20.702555725, 2024-09-13T10:12:19.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0083088199999999997, 2024-09-13T10:12:19.711 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:19.711 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:19.711 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:19.711 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:19.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:19.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.912836+0000", 2024-09-13T10:12:19.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:19.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.912836+0000", 2024-09-13T10:12:19.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:19.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:19.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.912836+0000", 2024-09-13T10:12:19.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7499999999999999e-07 2024-09-13T10:12:19.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:19.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.917813+0000", 2024-09-13T10:12:19.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049764029999999999 2024-09-13T10:12:19.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:19.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.917847+0000", 2024-09-13T10:12:19.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4144999999999999e-05 2024-09-13T10:12:19.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:19.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.917869+0000", 2024-09-13T10:12:19.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.196e-05 2024-09-13T10:12:19.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.917959+0000", 2024-09-13T10:12:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0421999999999996e-05 2024-09-13T10:12:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.917978+0000", 2024-09-13T10:12:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8368000000000001e-05 2024-09-13T10:12:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.921076+0000", 2024-09-13T10:12:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030987689999999999 2024-09-13T10:12:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:19.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:19.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:19.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:58.921145+0000", 2024-09-13T10:12:19.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8578000000000006e-05 2024-09-13T10:12:19.715 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.715 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.715 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.715 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:19.715 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:19.715 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:19.716 INFO:tasks.rados.rados.0.smithi195.stdout:1938: expect (ObjNum 342 snap 0 seq_num 342) 2024-09-13T10:12:19.716 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:19.717 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T10:12:19.718 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:12:19.714+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:12:19.738 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T10:12:19.801 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:19.817 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:19.814+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:19.885 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:19.885 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:12:19.885 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:12:19.885 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:19.887 INFO:tasks.rados.rados.0.smithi195.stdout:1939: finishing write tid 1 to smithi19530714-19 2024-09-13T10:12:19.887 INFO:tasks.rados.rados.0.smithi195.stdout:1939: finishing write tid 2 to smithi19530714-19 2024-09-13T10:12:19.887 INFO:tasks.rados.rados.0.smithi195.stdout:1939: finishing write tid 3 to smithi19530714-19 2024-09-13T10:12:19.887 INFO:tasks.rados.rados.0.smithi195.stdout:1939: finishing write tid 4 to smithi19530714-19 2024-09-13T10:12:19.887 INFO:tasks.rados.rados.0.smithi195.stdout:1939: finishing write tid 5 to smithi19530714-19 2024-09-13T10:12:19.887 INFO:tasks.rados.rados.0.smithi195.stdout:1939: oid 19 updating version 0 to 81 2024-09-13T10:12:19.887 INFO:tasks.rados.rados.0.smithi195.stdout:1939: oid 19 updating version 81 to 82 2024-09-13T10:12:19.887 INFO:tasks.rados.rados.0.smithi195.stdout:1939: oid 19 version 82 is already newer than 79 2024-09-13T10:12:19.887 INFO:tasks.rados.rados.0.smithi195.stdout:1939: oid 19 version 82 is already newer than 80 2024-09-13T10:12:19.887 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 82 (ObjNum 433 snap 0 seq_num 433) dirty exists 2024-09-13T10:12:19.887 INFO:tasks.rados.rados.0.smithi195.stdout:1939: left oid 19 (ObjNum 433 snap 0 seq_num 433) 2024-09-13T10:12:19.887 INFO:tasks.rados.rados.0.smithi195.stdout:1941: seq_num 434 ranges {494238=490187,1563538=701819,2818520=475718} 2024-09-13T10:12:19.899 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T10:12:19.902 INFO:tasks.rados.rados.0.smithi195.stdout:1941: writing smithi19530714-5 from 494238 to 984425 tid 1 2024-09-13T10:12:19.918 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:19.914+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:19.923 INFO:tasks.rados.rados.0.smithi195.stdout:1941: writing smithi19530714-5 from 1563538 to 2265357 tid 2 2024-09-13T10:12:19.940 INFO:tasks.rados.rados.0.smithi195.stdout:1941: writing smithi19530714-5 from 2818520 to 3294238 tid 3 2024-09-13T10:12:19.940 INFO:tasks.rados.rados.0.smithi195.stdout:1936: done (5 left) 2024-09-13T10:12:19.940 INFO:tasks.rados.rados.0.smithi195.stdout:1937: done (4 left) 2024-09-13T10:12:19.940 INFO:tasks.rados.rados.0.smithi195.stdout:1938: done (3 left) 2024-09-13T10:12:19.940 INFO:tasks.rados.rados.0.smithi195.stdout:1939: done (2 left) 2024-09-13T10:12:19.941 INFO:tasks.rados.rados.0.smithi195.stdout:1940: done (1 left) 2024-09-13T10:12:19.941 INFO:tasks.rados.rados.0.smithi195.stdout:1942: write oid 4 current snap is 0 2024-09-13T10:12:19.941 INFO:tasks.rados.rados.0.smithi195.stdout:1941: finishing write tid 1 to smithi19530714-5 2024-09-13T10:12:19.941 INFO:tasks.rados.rados.0.smithi195.stdout:1941: finishing write tid 2 to smithi19530714-5 2024-09-13T10:12:19.941 INFO:tasks.rados.rados.0.smithi195.stdout:1942: seq_num 435 ranges {511045=565436,1622020=445477,2729590=556855} 2024-09-13T10:12:19.954 INFO:tasks.rados.rados.0.smithi195.stdout:1942: writing smithi19530714-4 from 511045 to 1076481 tid 1 2024-09-13T10:12:19.967 INFO:tasks.rados.rados.0.smithi195.stdout:1942: writing smithi19530714-4 from 1622020 to 2067497 tid 2 2024-09-13T10:12:19.981 INFO:tasks.rados.rados.0.smithi195.stdout:1942: writing smithi19530714-4 from 2729590 to 3286445 tid 3 2024-09-13T10:12:19.982 INFO:tasks.rados.rados.0.smithi195.stdout:1943: delete oid 27 current snap is 0 2024-09-13T10:12:19.982 INFO:tasks.rados.rados.0.smithi195.stdout:1941: finishing write tid 3 to smithi19530714-5 2024-09-13T10:12:19.982 INFO:tasks.rados.rados.0.smithi195.stdout:1941: finishing write tid 4 to smithi19530714-5 2024-09-13T10:12:19.982 INFO:tasks.rados.rados.0.smithi195.stdout:1941: finishing write tid 5 to smithi19530714-5 2024-09-13T10:12:19.982 INFO:tasks.rados.rados.0.smithi195.stdout:1941: oid 5 updating version 0 to 32 2024-09-13T10:12:19.982 INFO:tasks.rados.rados.0.smithi195.stdout:1941: oid 5 updating version 32 to 34 2024-09-13T10:12:19.982 INFO:tasks.rados.rados.0.smithi195.stdout:1941: oid 5 version 34 is already newer than 31 2024-09-13T10:12:19.982 INFO:tasks.rados.rados.0.smithi195.stdout:1941: oid 5 version 34 is already newer than 33 2024-09-13T10:12:19.982 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 34 (ObjNum 434 snap 0 seq_num 434) dirty exists 2024-09-13T10:12:19.982 INFO:tasks.rados.rados.0.smithi195.stdout:1941: left oid 5 (ObjNum 434 snap 0 seq_num 434) 2024-09-13T10:12:19.982 INFO:tasks.rados.rados.0.smithi195.stdout:1942: finishing write tid 1 to smithi19530714-4 2024-09-13T10:12:19.982 INFO:tasks.rados.rados.0.smithi195.stdout:1942: finishing write tid 2 to smithi19530714-4 2024-09-13T10:12:19.986 INFO:tasks.rados.rados.0.smithi195.stdout:1941: done (2 left) 2024-09-13T10:12:19.987 INFO:tasks.rados.rados.0.smithi195.stdout:1943: done (1 left) 2024-09-13T10:12:19.987 INFO:tasks.rados.rados.0.smithi195.stdout:1944: read oid 47 snap -1 2024-09-13T10:12:19.987 INFO:tasks.rados.rados.0.smithi195.stdout:1944: expect (ObjNum 383 snap 0 seq_num 383) 2024-09-13T10:12:19.987 INFO:tasks.rados.rados.0.smithi195.stdout:1945: set_chunk oid 46 target oid 2024-09-13T10:12:20.003 INFO:tasks.rados.rados.0.smithi195.stdout:1945: set_chunk oid 46 offset: 796160 length: 1150464 target oid 8055bfabe692ee4d7644327344a433d0507424339716d7052e40b3a988c5bc8d offset: 0 snap -1 2024-09-13T10:12:20.004 INFO:tasks.rados.rados.0.smithi195.stdout:1946: write oid 42 current snap is 0 2024-09-13T10:12:20.004 INFO:tasks.rados.rados.0.smithi195.stdout:1942: finishing write tid 3 to smithi19530714-4 2024-09-13T10:12:20.004 INFO:tasks.rados.rados.0.smithi195.stdout:1942: finishing write tid 4 to smithi19530714-4 2024-09-13T10:12:20.004 INFO:tasks.rados.rados.0.smithi195.stdout:1942: finishing write tid 5 to smithi19530714-4 2024-09-13T10:12:20.004 INFO:tasks.rados.rados.0.smithi195.stdout:1942: oid 4 updating version 0 to 139 2024-09-13T10:12:20.004 INFO:tasks.rados.rados.0.smithi195.stdout:1942: oid 4 version 139 is already newer than 138 2024-09-13T10:12:20.004 INFO:tasks.rados.rados.0.smithi195.stdout:1942: oid 4 version 139 is already newer than 137 2024-09-13T10:12:20.004 INFO:tasks.rados.rados.0.smithi195.stdout:1942: oid 4 version 139 is already newer than 136 2024-09-13T10:12:20.004 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 139 (ObjNum 435 snap 0 seq_num 435) dirty exists 2024-09-13T10:12:20.004 INFO:tasks.rados.rados.0.smithi195.stdout:1942: left oid 4 (ObjNum 435 snap 0 seq_num 435) 2024-09-13T10:12:20.004 INFO:tasks.rados.rados.0.smithi195.stdout:1944: expect (ObjNum 383 snap 0 seq_num 383) 2024-09-13T10:12:20.018 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:20.018 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:12:20.018 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:12:20.018 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:12:20.018 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:20.019 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:20.013+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:20.033 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T10:12:20.075 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2093081 2024-09-13T10:12:20.119 INFO:tasks.rados.rados.0.smithi195.stdout:1945: finishing set_chunk to oid 46 2024-09-13T10:12:20.119 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 46 set_chunk returned error code -95 offset: 796160 length: 1150464 2024-09-13T10:12:20.119 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 46 v 0 (ObjNum 419 snap 0 seq_num 419) dirty exists 2024-09-13T10:12:20.119 INFO:tasks.rados.rados.0.smithi195.stdout:1946: seq_num 437 ranges {544659=715934,1738984=449146,2784436=560223} 2024-09-13T10:12:20.120 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:20.119+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:20.136 INFO:tasks.rados.rados.0.smithi195.stdout:1946: writing smithi19530714-42 from 544659 to 1260593 tid 1 2024-09-13T10:12:20.149 INFO:tasks.rados.rados.0.smithi195.stdout:1946: writing smithi19530714-42 from 1738984 to 2188130 tid 2 2024-09-13T10:12:20.153 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:20.153 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:12:20.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:12:20.154 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:12:20.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.154 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:203 3.1 e38/30 3:b033e801:::smithi19530714-4 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:12:20.154 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.002504+0000", 2024-09-13T10:12:20.154 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.148145064999994, 2024-09-13T10:12:20.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016371101999999998, 2024-09-13T10:12:20.154 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.154 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.154 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.154 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002504+0000", 2024-09-13T10:12:20.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002504+0000", 2024-09-13T10:12:20.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002507+0000", 2024-09-13T10:12:20.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2699999999999999e-06 2024-09-13T10:12:20.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007079+0000", 2024-09-13T10:12:20.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045722200000000001 2024-09-13T10:12:20.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007114+0000", 2024-09-13T10:12:20.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5389000000000003e-05 2024-09-13T10:12:20.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007132+0000", 2024-09-13T10:12:20.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7489000000000001e-05 2024-09-13T10:12:20.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007266+0000", 2024-09-13T10:12:20.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013401900000000001 2024-09-13T10:12:20.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007299+0000", 2024-09-13T10:12:20.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3136000000000003e-05 2024-09-13T10:12:20.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.018741+0000", 2024-09-13T10:12:20.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011441957000000001 2024-09-13T10:12:20.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.018875+0000", 2024-09-13T10:12:20.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013462200000000001 2024-09-13T10:12:20.158 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.158 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.158 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.159 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:211 3.0 e38/30 3:0596651d:::smithi19530714-12 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:12:20.159 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.007453+0000", 2024-09-13T10:12:20.159 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.143196240999998, 2024-09-13T10:12:20.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018632359000000001, 2024-09-13T10:12:20.159 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.159 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.159 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.159 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007453+0000", 2024-09-13T10:12:20.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007453+0000", 2024-09-13T10:12:20.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.007455+0000", 2024-09-13T10:12:20.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4030000000000001e-06 2024-09-13T10:12:20.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020746+0000", 2024-09-13T10:12:20.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013290219000000001 2024-09-13T10:12:20.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020788+0000", 2024-09-13T10:12:20.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2342000000000002e-05 2024-09-13T10:12:20.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.020868+0000", 2024-09-13T10:12:20.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9519999999999998e-05 2024-09-13T10:12:20.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021080+0000", 2024-09-13T10:12:20.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021249600000000001 2024-09-13T10:12:20.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021113+0000", 2024-09-13T10:12:20.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2786000000000001e-05 2024-09-13T10:12:20.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.025974+0000", 2024-09-13T10:12:20.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048614929999999997 2024-09-13T10:12:20.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026085+0000", 2024-09-13T10:12:20.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001111 2024-09-13T10:12:20.163 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.163 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.163 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.163 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:205 3.1 e38/30 3:aec11a92:::smithi19530714-6 low_tier:head v 38'2, mlcod=0'0)", 2024-09-13T10:12:20.163 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.014358+0000", 2024-09-13T10:12:20.163 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.136291616999998, 2024-09-13T10:12:20.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033491923999999999, 2024-09-13T10:12:20.164 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.164 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.164 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.164 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.014358+0000", 2024-09-13T10:12:20.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.014358+0000", 2024-09-13T10:12:20.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.014360+0000", 2024-09-13T10:12:20.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.057e-06 2024-09-13T10:12:20.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026358+0000", 2024-09-13T10:12:20.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011998688 2024-09-13T10:12:20.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.026392+0000", 2024-09-13T10:12:20.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3575999999999998e-05 2024-09-13T10:12:20.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027340+0000", 2024-09-13T10:12:20.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00094826100000000005 2024-09-13T10:12:20.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027449+0000", 2024-09-13T10:12:20.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108797 2024-09-13T10:12:20.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027500+0000", 2024-09-13T10:12:20.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0757999999999998e-05 2024-09-13T10:12:20.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047748+0000", 2024-09-13T10:12:20.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020247884000000001 2024-09-13T10:12:20.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047850+0000", 2024-09-13T10:12:20.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101903 2024-09-13T10:12:20.168 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.168 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.168 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.168 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:212 3.0 e38/30 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:12:20.168 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.021018+0000", 2024-09-13T10:12:20.168 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.129630809000005, 2024-09-13T10:12:20.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044503290000000001, 2024-09-13T10:12:20.169 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.169 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.169 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.169 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021018+0000", 2024-09-13T10:12:20.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021018+0000", 2024-09-13T10:12:20.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.021019+0000", 2024-09-13T10:12:20.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7400000000000002e-07 2024-09-13T10:12:20.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038659+0000", 2024-09-13T10:12:20.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017639555000000001 2024-09-13T10:12:20.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038678+0000", 2024-09-13T10:12:20.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8912000000000001e-05 2024-09-13T10:12:20.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038693+0000", 2024-09-13T10:12:20.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5393e-05 2024-09-13T10:12:20.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038829+0000", 2024-09-13T10:12:20.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000135912 2024-09-13T10:12:20.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038887+0000", 2024-09-13T10:12:20.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7858000000000001e-05 2024-09-13T10:12:20.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065395+0000", 2024-09-13T10:12:20.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.02650773 2024-09-13T10:12:20.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065522+0000", 2024-09-13T10:12:20.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012705600000000001 2024-09-13T10:12:20.173 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.173 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.173 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.173 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:213 3.1 e38/30 3:accf4e56:::smithi19530714-14 low_tier:head v 38'3, mlcod=0'0)", 2024-09-13T10:12:20.173 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.027392+0000", 2024-09-13T10:12:20.173 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.123257300000006, 2024-09-13T10:12:20.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025423449000000001, 2024-09-13T10:12:20.173 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.174 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.174 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.174 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027392+0000", 2024-09-13T10:12:20.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027392+0000", 2024-09-13T10:12:20.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.027394+0000", 2024-09-13T10:12:20.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7370000000000001e-06 2024-09-13T10:12:20.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032354+0000", 2024-09-13T10:12:20.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049601589999999996 2024-09-13T10:12:20.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032413+0000", 2024-09-13T10:12:20.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9095000000000003e-05 2024-09-13T10:12:20.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032426+0000", 2024-09-13T10:12:20.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3465999999999999e-05 2024-09-13T10:12:20.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032805+0000", 2024-09-13T10:12:20.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000378704 2024-09-13T10:12:20.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032862+0000", 2024-09-13T10:12:20.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6878e-05 2024-09-13T10:12:20.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.052688+0000", 2024-09-13T10:12:20.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019826308000000001 2024-09-13T10:12:20.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.052815+0000", 2024-09-13T10:12:20.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127102 2024-09-13T10:12:20.178 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.178 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.178 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.178 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:209 3.1 e38/30 3:9798d58d:::smithi19530714-10 low_tier:head v 38'4, mlcod=0'0)", 2024-09-13T10:12:20.178 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.032594+0000", 2024-09-13T10:12:20.178 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.118054865000005, 2024-09-13T10:12:20.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028120108000000001, 2024-09-13T10:12:20.178 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.178 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.178 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.179 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032594+0000", 2024-09-13T10:12:20.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032594+0000", 2024-09-13T10:12:20.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.032596+0000", 2024-09-13T10:12:20.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2330000000000001e-06 2024-09-13T10:12:20.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.035217+0000", 2024-09-13T10:12:20.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026212679999999999 2024-09-13T10:12:20.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.035250+0000", 2024-09-13T10:12:20.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3065000000000003e-05 2024-09-13T10:12:20.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.035272+0000", 2024-09-13T10:12:20.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2163000000000001e-05 2024-09-13T10:12:20.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038562+0000", 2024-09-13T10:12:20.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003290322 2024-09-13T10:12:20.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038642+0000", 2024-09-13T10:12:20.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9457999999999997e-05 2024-09-13T10:12:20.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060605+0000", 2024-09-13T10:12:20.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021962751999999999 2024-09-13T10:12:20.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.060715+0000", 2024-09-13T10:12:20.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010984699999999999 2024-09-13T10:12:20.182 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.183 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.183 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.183 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:207 3.0 e38/30 3:332cd44b:::smithi19530714-8 low_tier:head v 38'3, mlcod=0'0)", 2024-09-13T10:12:20.183 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.038735+0000", 2024-09-13T10:12:20.183 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.111914286000001, 2024-09-13T10:12:20.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.026872200999999998, 2024-09-13T10:12:20.183 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.183 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.183 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.183 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038735+0000", 2024-09-13T10:12:20.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038735+0000", 2024-09-13T10:12:20.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.038737+0000", 2024-09-13T10:12:20.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5430000000000001e-06 2024-09-13T10:12:20.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046860+0000", 2024-09-13T10:12:20.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081236629999999997 2024-09-13T10:12:20.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046895+0000", 2024-09-13T10:12:20.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4989e-05 2024-09-13T10:12:20.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046915+0000", 2024-09-13T10:12:20.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0026000000000001e-05 2024-09-13T10:12:20.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047057+0000", 2024-09-13T10:12:20.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014151800000000001 2024-09-13T10:12:20.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.047111+0000", 2024-09-13T10:12:20.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3885000000000003e-05 2024-09-13T10:12:20.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065539+0000", 2024-09-13T10:12:20.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018428225999999999 2024-09-13T10:12:20.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065607+0000", 2024-09-13T10:12:20.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8350999999999997e-05 2024-09-13T10:12:20.187 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.187 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.187 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:216 3.0 e38/30 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:12:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.046965+0000", 2024-09-13T10:12:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.103683822999997, 2024-09-13T10:12:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.022018617000000001, 2024-09-13T10:12:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046965+0000", 2024-09-13T10:12:20.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046965+0000", 2024-09-13T10:12:20.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.046967+0000", 2024-09-13T10:12:20.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3689999999999999e-06 2024-09-13T10:12:20.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054950+0000", 2024-09-13T10:12:20.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0079834910000000005 2024-09-13T10:12:20.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054967+0000", 2024-09-13T10:12:20.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6382e-05 2024-09-13T10:12:20.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.054981+0000", 2024-09-13T10:12:20.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3974999999999999e-05 2024-09-13T10:12:20.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055057+0000", 2024-09-13T10:12:20.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5964000000000007e-05 2024-09-13T10:12:20.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.055100+0000", 2024-09-13T10:12:20.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3522000000000001e-05 2024-09-13T10:12:20.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.068892+0000", 2024-09-13T10:12:20.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01379175 2024-09-13T10:12:20.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.068984+0000", 2024-09-13T10:12:20.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2163999999999994e-05 2024-09-13T10:12:20.192 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.192 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.192 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.192 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:219 3.1 e38/30 3:87bfc7b1:::smithi19530714-20 low_tier:head v 38'5, mlcod=0'0)", 2024-09-13T10:12:20.193 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.048853+0000", 2024-09-13T10:12:20.193 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.101796351000004, 2024-09-13T10:12:20.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017128925999999999, 2024-09-13T10:12:20.193 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.193 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.193 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.193 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.048853+0000", 2024-09-13T10:12:20.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.048853+0000", 2024-09-13T10:12:20.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.048855+0000", 2024-09-13T10:12:20.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.877e-06 2024-09-13T10:12:20.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053357+0000", 2024-09-13T10:12:20.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0045020149999999998 2024-09-13T10:12:20.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053378+0000", 2024-09-13T10:12:20.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1639000000000001e-05 2024-09-13T10:12:20.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053396+0000", 2024-09-13T10:12:20.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7135000000000001e-05 2024-09-13T10:12:20.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053526+0000", 2024-09-13T10:12:20.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013083999999999999 2024-09-13T10:12:20.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.053590+0000", 2024-09-13T10:12:20.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3355000000000006e-05 2024-09-13T10:12:20.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065909+0000", 2024-09-13T10:12:20.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012319583 2024-09-13T10:12:20.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065982+0000", 2024-09-13T10:12:20.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2482000000000002e-05 2024-09-13T10:12:20.197 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.197 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.197 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.197 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:221 3.0 e38/30 3:18ae852b:::smithi19530714-22 low_tier:head v 38'5, mlcod=0'0)", 2024-09-13T10:12:20.197 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.058895+0000", 2024-09-13T10:12:20.197 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.091754136000006, 2024-09-13T10:12:20.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019544733000000002, 2024-09-13T10:12:20.197 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.197 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.198 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.198 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.058895+0000", 2024-09-13T10:12:20.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.058895+0000", 2024-09-13T10:12:20.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.058897+0000", 2024-09-13T10:12:20.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0420000000000001e-06 2024-09-13T10:12:20.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.061589+0000", 2024-09-13T10:12:20.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026919869999999999 2024-09-13T10:12:20.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.061614+0000", 2024-09-13T10:12:20.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4984999999999999e-05 2024-09-13T10:12:20.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.061629+0000", 2024-09-13T10:12:20.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4501999999999999e-05 2024-09-13T10:12:20.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065668+0000", 2024-09-13T10:12:20.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040393260000000002 2024-09-13T10:12:20.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.065711+0000", 2024-09-13T10:12:20.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2673999999999998e-05 2024-09-13T10:12:20.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.078336+0000", 2024-09-13T10:12:20.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012625113 2024-09-13T10:12:20.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.078440+0000", 2024-09-13T10:12:20.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000104104 2024-09-13T10:12:20.201 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.201 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.202 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.202 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:231 3.1 e38/30 3:9d247594:::smithi19530714-32 low_tier:head v 38'7, mlcod=38'4)", 2024-09-13T10:12:20.202 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.076614+0000", 2024-09-13T10:12:20.202 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.074035172999999, 2024-09-13T10:12:20.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017222986999999999, 2024-09-13T10:12:20.202 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.202 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.202 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.202 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.076614+0000", 2024-09-13T10:12:20.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.076614+0000", 2024-09-13T10:12:20.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.076616+0000", 2024-09-13T10:12:20.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5069999999999999e-06 2024-09-13T10:12:20.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082649+0000", 2024-09-13T10:12:20.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0060329499999999996 2024-09-13T10:12:20.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082674+0000", 2024-09-13T10:12:20.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5624000000000001e-05 2024-09-13T10:12:20.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082690+0000", 2024-09-13T10:12:20.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6152999999999999e-05 2024-09-13T10:12:20.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082812+0000", 2024-09-13T10:12:20.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012209499999999999 2024-09-13T10:12:20.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.082863+0000", 2024-09-13T10:12:20.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0781000000000001e-05 2024-09-13T10:12:20.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093763+0000", 2024-09-13T10:12:20.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010899382000000001 2024-09-13T10:12:20.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.093837+0000", 2024-09-13T10:12:20.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4494999999999997e-05 2024-09-13T10:12:20.206 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.206 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.206 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.207 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:234 3.1 e38/30 3:bdce9caa:::smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'8, mlcod=38'4)", 2024-09-13T10:12:20.207 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.086591+0000", 2024-09-13T10:12:20.207 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.064058059999994, 2024-09-13T10:12:20.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025699538000000001, 2024-09-13T10:12:20.207 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.207 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.207 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.207 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.086591+0000", 2024-09-13T10:12:20.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.086591+0000", 2024-09-13T10:12:20.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.086623+0000", 2024-09-13T10:12:20.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1335999999999999e-05 2024-09-13T10:12:20.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094695+0000", 2024-09-13T10:12:20.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080726919999999994 2024-09-13T10:12:20.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094716+0000", 2024-09-13T10:12:20.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1015000000000001e-05 2024-09-13T10:12:20.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094732+0000", 2024-09-13T10:12:20.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5532999999999998e-05 2024-09-13T10:12:20.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094807+0000", 2024-09-13T10:12:20.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4837000000000002e-05 2024-09-13T10:12:20.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094836+0000", 2024-09-13T10:12:20.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8912999999999999e-05 2024-09-13T10:12:20.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112239+0000", 2024-09-13T10:12:20.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017402995000000001 2024-09-13T10:12:20.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.112291+0000", 2024-09-13T10:12:20.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2216999999999998e-05 2024-09-13T10:12:20.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.211 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.211 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.211 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:233 3.0 e38/30 3:24ead787:::smithi19530714-34 low_tier:head v 38'9, mlcod=38'4)", 2024-09-13T10:12:20.211 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.090071+0000", 2024-09-13T10:12:20.211 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.060578054000004, 2024-09-13T10:12:20.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.023396641999999999, 2024-09-13T10:12:20.212 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.212 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.212 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.212 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090071+0000", 2024-09-13T10:12:20.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090071+0000", 2024-09-13T10:12:20.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.090073+0000", 2024-09-13T10:12:20.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0470000000000001e-06 2024-09-13T10:12:20.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097736+0000", 2024-09-13T10:12:20.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007662416 2024-09-13T10:12:20.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097761+0000", 2024-09-13T10:12:20.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4919e-05 2024-09-13T10:12:20.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097773+0000", 2024-09-13T10:12:20.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2182e-05 2024-09-13T10:12:20.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097877+0000", 2024-09-13T10:12:20.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010438900000000001 2024-09-13T10:12:20.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097910+0000", 2024-09-13T10:12:20.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2808999999999997e-05 2024-09-13T10:12:20.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113429+0000", 2024-09-13T10:12:20.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015518557000000001 2024-09-13T10:12:20.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113468+0000", 2024-09-13T10:12:20.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9323000000000003e-05 2024-09-13T10:12:20.216 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.216 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.216 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.216 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:223 3.1 e38/30 3:8759f4e0:::smithi19530714-24 low_tier:head v 38'9, mlcod=38'4)", 2024-09-13T10:12:20.216 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.094775+0000", 2024-09-13T10:12:20.216 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.055874536999994, 2024-09-13T10:12:20.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018476023000000001, 2024-09-13T10:12:20.216 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.216 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.216 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.217 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094775+0000", 2024-09-13T10:12:20.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094775+0000", 2024-09-13T10:12:20.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.094776+0000", 2024-09-13T10:12:20.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.466e-06 2024-09-13T10:12:20.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097100+0000", 2024-09-13T10:12:20.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002323435 2024-09-13T10:12:20.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097123+0000", 2024-09-13T10:12:20.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3756000000000001e-05 2024-09-13T10:12:20.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097139+0000", 2024-09-13T10:12:20.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5424e-05 2024-09-13T10:12:20.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.098642+0000", 2024-09-13T10:12:20.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015034359999999999 2024-09-13T10:12:20.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.098683+0000", 2024-09-13T10:12:20.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0756000000000001e-05 2024-09-13T10:12:20.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113193+0000", 2024-09-13T10:12:20.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014509831000000001 2024-09-13T10:12:20.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.113251+0000", 2024-09-13T10:12:20.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7918999999999999e-05 2024-09-13T10:12:20.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.221 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.221 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.221 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1959 4.13 4.e5457733 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:12:20.221 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.866379+0000", 2024-09-13T10:12:20.221 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.284270172000006, 2024-09-13T10:12:20.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.83266194900000001, 2024-09-13T10:12:20.221 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.221 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.221 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:12:20.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:20.222 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:20.222 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1959 2024-09-13T10:12:20.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.222 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.866379+0000", 2024-09-13T10:12:20.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.866379+0000", 2024-09-13T10:12:20.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.866381+0000", 2024-09-13T10:12:20.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.125e-06 2024-09-13T10:12:20.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867454+0000", 2024-09-13T10:12:20.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001072649 2024-09-13T10:12:20.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867475+0000", 2024-09-13T10:12:20.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1299999999999999e-05 2024-09-13T10:12:20.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867486+0000", 2024-09-13T10:12:20.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1167e-05 2024-09-13T10:12:20.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867581+0000", 2024-09-13T10:12:20.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4561999999999993e-05 2024-09-13T10:12:20.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:12:20.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.867592+0000", 2024-09-13T10:12:20.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1031e-05 2024-09-13T10:12:20.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699018+0000", 2024-09-13T10:12:20.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.83142604099999995 2024-09-13T10:12:20.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699041+0000", 2024-09-13T10:12:20.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3074e-05 2024-09-13T10:12:20.226 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.226 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.226 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.226 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1960 4.13 4.e5457733 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:12:20.226 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.874165+0000", 2024-09-13T10:12:20.226 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.276484531999998, 2024-09-13T10:12:20.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82492870799999996, 2024-09-13T10:12:20.226 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.226 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.226 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:12:20.227 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.227 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:20.227 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:20.227 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1960 2024-09-13T10:12:20.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.227 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874165+0000", 2024-09-13T10:12:20.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874165+0000", 2024-09-13T10:12:20.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874166+0000", 2024-09-13T10:12:20.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.325e-06 2024-09-13T10:12:20.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874706+0000", 2024-09-13T10:12:20.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00053996899999999995 2024-09-13T10:12:20.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874719+0000", 2024-09-13T10:12:20.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2866e-05 2024-09-13T10:12:20.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874726+0000", 2024-09-13T10:12:20.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7560000000000002e-06 2024-09-13T10:12:20.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874801+0000", 2024-09-13T10:12:20.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5612999999999996e-05 2024-09-13T10:12:20.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:12:20.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.874810+0000", 2024-09-13T10:12:20.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2619999999999998e-06 2024-09-13T10:12:20.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699082+0000", 2024-09-13T10:12:20.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82427217699999999 2024-09-13T10:12:20.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699093+0000", 2024-09-13T10:12:20.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1739999999999999e-05 2024-09-13T10:12:20.231 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.231 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.231 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.231 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1961 4.13 4.e5457733 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:12:20.231 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.877465+0000", 2024-09-13T10:12:20.231 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.273184280999999, 2024-09-13T10:12:20.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82170111099999998, 2024-09-13T10:12:20.232 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.232 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.232 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:12:20.232 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.232 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:20.232 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:20.232 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1961 2024-09-13T10:12:20.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.232 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877465+0000", 2024-09-13T10:12:20.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877465+0000", 2024-09-13T10:12:20.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877466+0000", 2024-09-13T10:12:20.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0100000000000004e-07 2024-09-13T10:12:20.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877640+0000", 2024-09-13T10:12:20.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000173795 2024-09-13T10:12:20.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877650+0000", 2024-09-13T10:12:20.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0465000000000001e-05 2024-09-13T10:12:20.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877656+0000", 2024-09-13T10:12:20.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6849999999999998e-06 2024-09-13T10:12:20.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877714+0000", 2024-09-13T10:12:20.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8097999999999998e-05 2024-09-13T10:12:20.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:12:20.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877726+0000", 2024-09-13T10:12:20.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1857e-05 2024-09-13T10:12:20.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699146+0000", 2024-09-13T10:12:20.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82142019700000002 2024-09-13T10:12:20.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699166+0000", 2024-09-13T10:12:20.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0213000000000001e-05 2024-09-13T10:12:20.236 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.236 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.236 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.237 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1962 4.13 4.e5457733 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:12:20.237 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.877669+0000", 2024-09-13T10:12:20.237 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.272980145999995, 2024-09-13T10:12:20.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82156359899999998, 2024-09-13T10:12:20.237 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.237 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.237 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:12:20.237 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.237 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:20.237 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:20.237 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1962 2024-09-13T10:12:20.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.237 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877669+0000", 2024-09-13T10:12:20.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877669+0000", 2024-09-13T10:12:20.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877670+0000", 2024-09-13T10:12:20.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9899999999999999e-07 2024-09-13T10:12:20.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877675+0000", 2024-09-13T10:12:20.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5510000000000001e-06 2024-09-13T10:12:20.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877684+0000", 2024-09-13T10:12:20.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5829999999999996e-06 2024-09-13T10:12:20.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877694+0000", 2024-09-13T10:12:20.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0207000000000001e-05 2024-09-13T10:12:20.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877784+0000", 2024-09-13T10:12:20.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9990999999999999e-05 2024-09-13T10:12:20.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:12:20.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877798+0000", 2024-09-13T10:12:20.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3444e-05 2024-09-13T10:12:20.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699222+0000", 2024-09-13T10:12:20.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82142437999999995 2024-09-13T10:12:20.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699233+0000", 2024-09-13T10:12:20.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0644e-05 2024-09-13T10:12:20.241 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.241 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.241 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.242 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1963 4.13 4.e5457733 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e55)", 2024-09-13T10:12:20.242 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:11.877721+0000", 2024-09-13T10:12:20.242 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 68.272927887999998, 2024-09-13T10:12:20.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82158741499999999, 2024-09-13T10:12:20.242 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.242 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.242 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-09-13T10:12:20.242 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.242 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:20.242 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:20.242 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1963 2024-09-13T10:12:20.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.243 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877721+0000", 2024-09-13T10:12:20.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877721+0000", 2024-09-13T10:12:20.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877722+0000", 2024-09-13T10:12:20.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4099999999999998e-07 2024-09-13T10:12:20.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877728+0000", 2024-09-13T10:12:20.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1129999999999997e-06 2024-09-13T10:12:20.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877737+0000", 2024-09-13T10:12:20.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3969999999999994e-06 2024-09-13T10:12:20.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877743+0000", 2024-09-13T10:12:20.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7950000000000002e-06 2024-09-13T10:12:20.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877826+0000", 2024-09-13T10:12:20.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2213999999999996e-05 2024-09-13T10:12:20.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:12:20.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:11.877831+0000", 2024-09-13T10:12:20.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.604e-06 2024-09-13T10:12:20.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699295+0000", 2024-09-13T10:12:20.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82146349299999999 2024-09-13T10:12:20.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:12.699309+0000", 2024-09-13T10:12:20.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4058e-05 2024-09-13T10:12:20.246 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.246 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.247 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.247 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:3745 3.0 e90/30 3:161dd296:::f6d96d7a8dac6a7d56d3d9df9ef9c133dce9c515f90d85faf1e037f770fc9485:head v 90'300, mlcod=89'298)", 2024-09-13T10:12:20.247 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:54.612179+0000", 2024-09-13T10:12:20.247 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 25.538469913, 2024-09-13T10:12:20.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017220320000000001, 2024-09-13T10:12:20.247 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.247 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.247 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.247 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.612179+0000", 2024-09-13T10:12:20.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.612179+0000", 2024-09-13T10:12:20.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.612181+0000", 2024-09-13T10:12:20.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.897e-06 2024-09-13T10:12:20.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.618107+0000", 2024-09-13T10:12:20.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059256550000000002 2024-09-13T10:12:20.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.618138+0000", 2024-09-13T10:12:20.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0715e-05 2024-09-13T10:12:20.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.618153+0000", 2024-09-13T10:12:20.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5187e-05 2024-09-13T10:12:20.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.618319+0000", 2024-09-13T10:12:20.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016641099999999999 2024-09-13T10:12:20.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.618384+0000", 2024-09-13T10:12:20.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5121000000000002e-05 2024-09-13T10:12:20.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.629302+0000", 2024-09-13T10:12:20.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010917237999999999 2024-09-13T10:12:20.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:54.629400+0000", 2024-09-13T10:12:20.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8096000000000004e-05 2024-09-13T10:12:20.251 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.251 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.251 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.251 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.252 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.252 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:20.254 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-13T10:12:20.256 INFO:tasks.rados.rados.0.smithi195.stdout:1946: writing smithi19530714-42 from 2784436 to 3344659 tid 3 2024-09-13T10:12:20.256 INFO:tasks.rados.rados.0.smithi195.stdout:1942: done (3 left) 2024-09-13T10:12:20.256 INFO:tasks.rados.rados.0.smithi195.stdout:1944: done (2 left) 2024-09-13T10:12:20.256 INFO:tasks.rados.rados.0.smithi195.stdout:1945: done (1 left) 2024-09-13T10:12:20.256 INFO:tasks.rados.rados.0.smithi195.stdout:1947: tier_promote oid 19 2024-09-13T10:12:20.256 INFO:tasks.rados.rados.0.smithi195.stdout:1946: finishing write tid 1 to smithi19530714-42 2024-09-13T10:12:20.256 INFO:tasks.rados.rados.0.smithi195.stdout:1948: read oid 7 snap -1 2024-09-13T10:12:20.256 INFO:tasks.rados.rados.0.smithi195.stdout:1948: expect (ObjNum 407 snap 0 seq_num 407) 2024-09-13T10:12:20.256 INFO:tasks.rados.rados.0.smithi195.stdout:1949: set_chunk oid 11 target oid 2024-09-13T10:12:20.256 INFO:tasks.rados.rados.0.smithi195.stdout:1949: set_chunk oid 11 offset: 400384 length: 701952 target oid 0a295b6f40e223f567ea861aaee52f447ffa1e46b2508629c15e67b171d97add offset: 0 snap -1 2024-09-13T10:12:20.256 INFO:tasks.rados.rados.0.smithi195.stdout:1950: read oid 18 snap -1 2024-09-13T10:12:20.257 INFO:tasks.rados.rados.0.smithi195.stdout:1950: expect (ObjNum 418 snap 0 seq_num 418) 2024-09-13T10:12:20.257 INFO:tasks.rados.rados.0.smithi195.stdout:1951: read oid 13 snap -1 2024-09-13T10:12:20.257 INFO:tasks.rados.rados.0.smithi195.stdout:1951: expect (ObjNum 342 snap 0 seq_num 342) 2024-09-13T10:12:20.257 INFO:tasks.rados.rados.0.smithi195.stdout:1952: read oid 44 snap -1 2024-09-13T10:12:20.257 INFO:tasks.rados.rados.0.smithi195.stdout:1952: expect deleted 2024-09-13T10:12:20.257 INFO:tasks.rados.rados.0.smithi195.stdout:1953: set_chunk oid 31 target oid 2024-09-13T10:12:20.257 INFO:tasks.rados.rados.0.smithi195.stdout:1953: set_chunk oid 31 offset: 140800 length: 488448 target oid 312d1ff7d077900cf17fe4f94c503ec40a86bfa4b6c3330f7966316ba8f9f97d offset: 0 snap -1 2024-09-13T10:12:20.257 INFO:tasks.rados.rados.0.smithi195.stdout:1954: write (excl) oid 37 current snap is 0 2024-09-13T10:12:20.257 INFO:tasks.rados.rados.0.smithi195.stdout:1954: seq_num 438 ranges {561466=791183,2081113=592804,3095506=265960} 2024-09-13T10:12:20.257 INFO:tasks.rados.rados.0.smithi195.stdout:1954: writing smithi19530714-37 from 561466 to 1352649 tid 1 2024-09-13T10:12:20.257 INFO:tasks.rados.rados.0.smithi195.stdout:1954: writing smithi19530714-37 from 2081113 to 2673917 tid 2 2024-09-13T10:12:20.257 INFO:tasks.rados.rados.0.smithi195.stdout:1954: writing smithi19530714-37 from 3095506 to 3361466 tid 3 2024-09-13T10:12:20.257 INFO:tasks.rados.rados.0.smithi195.stdout:1955: read oid 21 snap -1 2024-09-13T10:12:20.258 INFO:tasks.rados.rados.0.smithi195.stdout:1955: expect (ObjNum 421 snap 0 seq_num 421) 2024-09-13T10:12:20.258 INFO:tasks.rados.rados.0.smithi195.stdout:1956: read oid 45 snap -1 2024-09-13T10:12:20.258 INFO:tasks.rados.rados.0.smithi195.stdout:1956: expect (ObjNum 425 snap 0 seq_num 425) 2024-09-13T10:12:20.258 INFO:tasks.rados.rados.0.smithi195.stdout:1957: read oid 6 snap -1 2024-09-13T10:12:20.258 INFO:tasks.rados.rados.0.smithi195.stdout:1957: expect (ObjNum 337 snap 0 seq_num 337) 2024-09-13T10:12:20.258 INFO:tasks.rados.rados.0.smithi195.stdout:1958: read oid 36 snap -1 2024-09-13T10:12:20.258 INFO:tasks.rados.rados.0.smithi195.stdout:1958: expect deleted 2024-09-13T10:12:20.258 INFO:tasks.rados.rados.0.smithi195.stdout:1959: write oid 30 current snap is 0 2024-09-13T10:12:20.258 INFO:tasks.rados.rados.0.smithi195.stdout:1959: seq_num 439 ranges {578273=466432,1739595=452815,2722929=655344} 2024-09-13T10:12:20.258 INFO:tasks.rados.rados.0.smithi195.stdout:1959: writing smithi19530714-30 from 578273 to 1044705 tid 1 2024-09-13T10:12:20.259 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:12:20.251+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:12:20.260 INFO:tasks.rados.rados.0.smithi195.stdout:1959: writing smithi19530714-30 from 1739595 to 2192410 tid 2 2024-09-13T10:12:20.269 INFO:tasks.rados.rados.0.smithi195.stdout:1959: writing smithi19530714-30 from 2722929 to 3378273 tid 3 2024-09-13T10:12:20.269 INFO:tasks.rados.rados.0.smithi195.stdout:1960: read oid 40 snap -1 2024-09-13T10:12:20.269 INFO:tasks.rados.rados.0.smithi195.stdout:1960: expect (ObjNum 431 snap 0 seq_num 431) 2024-09-13T10:12:20.269 INFO:tasks.rados.rados.0.smithi195.stdout:1961: read oid 10 snap -1 2024-09-13T10:12:20.269 INFO:tasks.rados.rados.0.smithi195.stdout:1961: expect deleted 2024-09-13T10:12:20.270 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 16 2024-09-13T10:12:20.270 INFO:tasks.rados.rados.0.smithi195.stdout:1946: finishing write tid 2 to smithi19530714-42 2024-09-13T10:12:20.270 INFO:tasks.rados.rados.0.smithi195.stdout:1946: finishing write tid 3 to smithi19530714-42 2024-09-13T10:12:20.270 INFO:tasks.rados.rados.0.smithi195.stdout:1946: finishing write tid 4 to smithi19530714-42 2024-09-13T10:12:20.270 INFO:tasks.rados.rados.0.smithi195.stdout:1946: finishing write tid 5 to smithi19530714-42 2024-09-13T10:12:20.270 INFO:tasks.rados.rados.0.smithi195.stdout:1946: oid 42 updating version 0 to 37 2024-09-13T10:12:20.270 INFO:tasks.rados.rados.0.smithi195.stdout:1946: oid 42 version 37 is already newer than 35 2024-09-13T10:12:20.270 INFO:tasks.rados.rados.0.smithi195.stdout:1946: oid 42 version 37 is already newer than 36 2024-09-13T10:12:20.270 INFO:tasks.rados.rados.0.smithi195.stdout:1946: oid 42 updating version 37 to 38 2024-09-13T10:12:20.270 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 42 v 38 (ObjNum 437 snap 0 seq_num 437) dirty exists 2024-09-13T10:12:20.270 INFO:tasks.rados.rados.0.smithi195.stdout:1946: left oid 42 (ObjNum 437 snap 0 seq_num 437) 2024-09-13T10:12:20.270 INFO:tasks.rados.rados.0.smithi195.stdout:1947: got (0) Success 2024-09-13T10:12:20.270 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 19 v 82 (ObjNum 433 snap 0 seq_num 433) dirty exists 2024-09-13T10:12:20.270 INFO:tasks.rados.rados.0.smithi195.stdout:1948: expect (ObjNum 407 snap 0 seq_num 407) 2024-09-13T10:12:20.302 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:12:20.302 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:12:20.302 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:12:20.302 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:12:20.354 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:20.350+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:20.381 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:20.386 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:20.386 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:12:20.386 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:12:20.386 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:20.401 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-13T10:12:20.441 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1477338 2024-09-13T10:12:20.455 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:20.455+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:20.490 INFO:tasks.rados.rados.0.smithi195.stdout:1950: expect (ObjNum 418 snap 0 seq_num 418) 2024-09-13T10:12:20.521 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:12:20.521 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:12:20.522 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:12:20.522 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:20.536 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-13T10:12:20.555 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:12:20.554+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:12:20.610 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:12:20.611 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:12:20.630 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-13T10:12:19.720831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":230165069,"num_objects":179,"num_object_clones":0,"num_object_copies":537,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":5802,"num_read_kb":4076449,"num_write":3398,"num_write_kb":769067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":60274071,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":18,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":117,"acting":117,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":125,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":1047948,"kb_used_data":549584,"kb_used_omap":24,"kb_used_meta":486183,"kb_avail":1508901492,"statfs":{"total":1546188226560,"available":1545115127808,"internally_reserved":0,"allocated":562774016,"data_stored":553561036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":497851471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11777732,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":11,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":277,"num_read_kb":193381,"num_write":144,"num_write_kb":35501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1489368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000933"},"pg_stats":[{"pgid":"4.19","version":"106'70","reported_seq":369,"reported_epoch":111,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:12:18.723982+0000","last_change":"2024-09-13T10:12:13.894581+0000","last_active":"2024-09-13T10:12:18.723982+0000","last_peered":"2024-09-13T10:12:18.723982+0000","last_clean":"2024-09-13T10:12:09.846417+0000","last_became_active":"2024-09-13T10:12:13.882915+0000","last_became_peered":"2024-09-13T10:12:13.882915+0000","last_unstale":"2024-09-13T10:12:18.723982+0000","last_undegraded":"2024-09-13T10:12:18.723982+0000","last_fullsized":"2024-09-13T10:12:18.723982+0000","mapping_epoch":105,"log_start":"106'67","ondisk_log_start":"106'67","created":41,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'28","last_scrub_stamp":"2024-09-13T10:11:00.388132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.388132+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":67,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5294337,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":168,"num_read_kb":129261,"num_write":46,"num_write_kb":11392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2376704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,9],"acting":[2,5,3],"avail_no_missing":["2","3","5"],"object_location_counts":[{"shards":"2,3,5","objects":2}],"blocked_by":[],"up_primary":11,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"95'30","reported_seq":270,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.726663+0000","last_change":"2024-09-13T10:12:17.724625+0000","last_active":"2024-09-13T10:12:17.726663+0000","last_peered":"2024-09-13T10:12:17.726663+0000","last_clean":"2024-09-13T10:12:17.726663+0000","last_became_active":"2024-09-13T10:12:17.721145+0000","last_became_peered":"2024-09-13T10:12:17.721145+0000","last_unstale":"2024-09-13T10:12:17.726663+0000","last_undegraded":"2024-09-13T10:12:17.726663+0000","last_fullsized":"2024-09-13T10:12:17.726663+0000","mapping_epoch":109,"log_start":"95'28","ondisk_log_start":"95'28","created":41,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'5","last_scrub_stamp":"2024-09-13T10:11:01.783496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.783496+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":28,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:25.388761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2050520,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":101,"num_read_kb":89614,"num_write":28,"num_write_kb":7995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3925645,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,0],"acting":[3,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"105'131","reported_seq":481,"reported_epoch":110,"state":"activating+remapped","last_fresh":"2024-09-13T10:12:17.722100+0000","last_change":"2024-09-13T10:12:17.720355+0000","last_active":"2024-09-13T10:12:15.728711+0000","last_peered":"2024-09-13T10:12:12.968285+0000","last_clean":"2024-09-13T10:12:12.968285+0000","last_became_active":"2024-09-13T10:11:34.720812+0000","last_became_peered":"2024-09-13T10:11:34.720812+0000","last_unstale":"2024-09-13T10:12:17.722100+0000","last_undegraded":"2024-09-13T10:12:17.722100+0000","last_fullsized":"2024-09-13T10:12:17.722100+0000","mapping_epoch":109,"log_start":"103'128","ondisk_log_start":"103'128","created":41,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":5,"last_scrub":"105'131","last_scrub_stamp":"2024-09-13T10:12:14.226050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:14.226050+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":128,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8823840,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":244,"num_read_kb":218117,"num_write":122,"num_write_kb":35810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3911454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,10],"acting":[1,0,5],"avail_no_missing":["1","0","5"],"object_location_counts":[{"shards":"0,1,5","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"93'78","reported_seq":405,"reported_epoch":111,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:12:18.770392+0000","last_change":"2024-09-13T10:12:18.770392+0000","last_active":"2024-09-13T10:12:18.770392+0000","last_peered":"2024-09-13T10:12:18.770392+0000","last_clean":"2024-09-13T10:12:03.112281+0000","last_became_active":"2024-09-13T10:12:18.754536+0000","last_became_peered":"2024-09-13T10:12:18.754536+0000","last_unstale":"2024-09-13T10:12:18.770392+0000","last_undegraded":"2024-09-13T10:12:18.770392+0000","last_fullsized":"2024-09-13T10:12:18.770392+0000","mapping_epoch":110,"log_start":"93'75","ondisk_log_start":"93'75","created":41,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":5,"last_scrub":"46'44","last_scrub_stamp":"2024-09-13T10:11:00.456457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.456457+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":72,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7327918,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":166,"num_read_kb":193390,"num_write":39,"num_write_kb":10677,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3338234,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,4],"acting":[14,5,13],"avail_no_missing":["14","5","13"],"object_location_counts":[{"shards":"5,13,14","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.15","version":"107'48","reported_seq":322,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.941818+0000","last_change":"2024-09-13T10:12:16.084525+0000","last_active":"2024-09-13T10:12:17.941818+0000","last_peered":"2024-09-13T10:12:17.941818+0000","last_clean":"2024-09-13T10:12:17.941818+0000","last_became_active":"2024-09-13T10:11:23.705370+0000","last_became_peered":"2024-09-13T10:11:23.705370+0000","last_unstale":"2024-09-13T10:12:17.941818+0000","last_undegraded":"2024-09-13T10:12:17.941818+0000","last_fullsized":"2024-09-13T10:12:17.941818+0000","mapping_epoch":64,"log_start":"101'45","ondisk_log_start":"101'45","created":41,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":5,"last_scrub":"107'48","last_scrub_stamp":"2024-09-13T10:12:16.084431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:16.084431+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":43,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:25.274164+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2991712,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":44,"num_read_kb":39289,"num_write":10,"num_write_kb":2891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1702851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,13],"acting":[6,7,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"39'40","reported_seq":228,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.715797+0000","last_change":"2024-09-13T10:12:00.310899+0000","last_active":"2024-09-13T10:12:17.715797+0000","last_peered":"2024-09-13T10:12:17.715797+0000","last_clean":"2024-09-13T10:12:17.715797+0000","last_became_active":"2024-09-13T10:11:47.715516+0000","last_became_peered":"2024-09-13T10:11:47.715516+0000","last_unstale":"2024-09-13T10:12:17.715797+0000","last_undegraded":"2024-09-13T10:12:17.715797+0000","last_fullsized":"2024-09-13T10:12:17.715797+0000","mapping_epoch":85,"log_start":"39'37","ondisk_log_start":"39'37","created":41,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'40","last_scrub_stamp":"2024-09-13T10:12:00.310748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.310748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":37,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:05.649596+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.13","version":"98'44","reported_seq":324,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.729176+0000","last_change":"2024-09-13T10:12:12.572121+0000","last_active":"2024-09-13T10:12:17.729176+0000","last_peered":"2024-09-13T10:12:17.729176+0000","last_clean":"2024-09-13T10:12:17.729176+0000","last_became_active":"2024-09-13T10:11:15.757364+0000","last_became_peered":"2024-09-13T10:11:15.757364+0000","last_unstale":"2024-09-13T10:12:17.729176+0000","last_undegraded":"2024-09-13T10:12:17.729176+0000","last_fullsized":"2024-09-13T10:12:17.729176+0000","mapping_epoch":58,"log_start":"97'41","ondisk_log_start":"97'41","created":41,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":5,"last_scrub":"98'44","last_scrub_stamp":"2024-09-13T10:12:12.572014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:12.572014+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":39,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:21.886670+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2336239,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":75,"num_read_kb":61934,"num_write":31,"num_write_kb":7734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1493592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"100'53","reported_seq":369,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.718324+0000","last_change":"2024-09-13T10:11:55.737652+0000","last_active":"2024-09-13T10:12:17.718324+0000","last_peered":"2024-09-13T10:12:17.718324+0000","last_clean":"2024-09-13T10:12:17.718324+0000","last_became_active":"2024-09-13T10:11:55.736402+0000","last_became_peered":"2024-09-13T10:11:55.736402+0000","last_unstale":"2024-09-13T10:12:17.718324+0000","last_undegraded":"2024-09-13T10:12:17.718324+0000","last_fullsized":"2024-09-13T10:12:17.718324+0000","mapping_epoch":91,"log_start":"96'50","ondisk_log_start":"96'50","created":41,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'17","last_scrub_stamp":"2024-09-13T10:11:02.582829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:02.582829+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":50,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:25.395942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2369853,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":168,"num_read_kb":136653,"num_write":36,"num_write_kb":10155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":7321937,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,13],"acting":[1,4,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"105'30","reported_seq":285,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-13T10:12:15.728292+0000","last_change":"2024-09-13T10:11:40.065958+0000","last_active":"2024-09-13T10:12:15.728292+0000","last_peered":"2024-09-13T10:12:15.728292+0000","last_clean":"2024-09-13T10:12:15.728292+0000","last_became_active":"2024-09-13T10:11:40.064608+0000","last_became_peered":"2024-09-13T10:11:40.064608+0000","last_unstale":"2024-09-13T10:12:15.728292+0000","last_undegraded":"2024-09-13T10:12:15.728292+0000","last_fullsized":"2024-09-13T10:12:15.728292+0000","mapping_epoch":77,"log_start":"100'27","ondisk_log_start":"100'27","created":41,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":5,"last_scrub":"40'13","last_scrub_stamp":"2024-09-13T10:11:04.242723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.242723+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":24,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:21.197902+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2823642,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":61836,"num_write":20,"num_write_kb":7252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1805029,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,12],"acting":[13,4,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.10","version":"107'61","reported_seq":385,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-13T10:12:16.228044+0000","last_change":"2024-09-13T10:12:13.384690+0000","last_active":"2024-09-13T10:12:16.228044+0000","last_peered":"2024-09-13T10:12:16.228044+0000","last_clean":"2024-09-13T10:12:16.228044+0000","last_became_active":"2024-09-13T10:11:40.063380+0000","last_became_peered":"2024-09-13T10:11:40.063380+0000","last_unstale":"2024-09-13T10:12:16.228044+0000","last_undegraded":"2024-09-13T10:12:16.228044+0000","last_fullsized":"2024-09-13T10:12:16.228044+0000","mapping_epoch":77,"log_start":"97'58","ondisk_log_start":"97'58","created":41,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":5,"last_scrub":"98'60","last_scrub_stamp":"2024-09-13T10:12:13.384591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:13.384591+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":56,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:20.904412+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5344725,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":197,"num_read_kb":161975,"num_write":51,"num_write_kb":14184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2889991,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,14,3],"acting":[9,14,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":129,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.717404+0000","last_change":"2024-09-13T10:12:03.961952+0000","last_active":"2024-09-13T10:12:17.717404+0000","last_peered":"2024-09-13T10:12:17.717404+0000","last_clean":"2024-09-13T10:12:17.717404+0000","last_became_active":"2024-09-13T10:11:12.744090+0000","last_became_peered":"2024-09-13T10:11:12.744090+0000","last_unstale":"2024-09-13T10:12:17.717404+0000","last_undegraded":"2024-09-13T10:12:17.717404+0000","last_fullsized":"2024-09-13T10:12:17.717404+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:03.961779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:03.961779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:12.481787+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,12],"acting":[3,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"39'12","reported_seq":229,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.941363+0000","last_change":"2024-09-13T10:11:01.537080+0000","last_active":"2024-09-13T10:12:17.941363+0000","last_peered":"2024-09-13T10:12:17.941363+0000","last_clean":"2024-09-13T10:12:17.941363+0000","last_became_active":"2024-09-13T10:10:54.318519+0000","last_became_peered":"2024-09-13T10:10:54.318519+0000","last_unstale":"2024-09-13T10:12:17.941363+0000","last_undegraded":"2024-09-13T10:12:17.941363+0000","last_fullsized":"2024-09-13T10:12:17.941363+0000","mapping_epoch":31,"log_start":"38'9","ondisk_log_start":"38'9","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'12","last_scrub_stamp":"2024-09-13T10:11:01.536963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.536963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":9,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:20.620036+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,14],"acting":[6,8,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":193,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.717239+0000","last_change":"2024-09-13T10:12:12.892985+0000","last_active":"2024-09-13T10:12:17.717239+0000","last_peered":"2024-09-13T10:12:17.717239+0000","last_clean":"2024-09-13T10:12:17.717239+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:12:17.717239+0000","last_undegraded":"2024-09-13T10:12:17.717239+0000","last_fullsized":"2024-09-13T10:12:17.717239+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:12.892836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:12.892836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:20.216623+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"107'101","reported_seq":422,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-13T10:12:18.742222+0000","last_change":"2024-09-13T10:11:01.219554+0000","last_active":"2024-09-13T10:12:18.742222+0000","last_peered":"2024-09-13T10:12:18.742222+0000","last_clean":"2024-09-13T10:12:18.742222+0000","last_became_active":"2024-09-13T10:10:54.314545+0000","last_became_peered":"2024-09-13T10:10:54.314545+0000","last_unstale":"2024-09-13T10:12:18.742222+0000","last_undegraded":"2024-09-13T10:12:18.742222+0000","last_fullsized":"2024-09-13T10:12:18.742222+0000","mapping_epoch":31,"log_start":"101'98","ondisk_log_start":"101'98","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'27","last_scrub_stamp":"2024-09-13T10:11:01.219486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.219486+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":97,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:23.384023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4571636,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":222,"num_read_kb":179289,"num_write":91,"num_write_kb":24881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,8],"acting":[2,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":191,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-13T10:12:16.737222+0000","last_change":"2024-09-13T10:12:01.407368+0000","last_active":"2024-09-13T10:12:16.737222+0000","last_peered":"2024-09-13T10:12:16.737222+0000","last_clean":"2024-09-13T10:12:16.737222+0000","last_became_active":"2024-09-13T10:10:30.128658+0000","last_became_peered":"2024-09-13T10:10:30.128658+0000","last_unstale":"2024-09-13T10:12:16.737222+0000","last_undegraded":"2024-09-13T10:12:16.737222+0000","last_fullsized":"2024-09-13T10:12:16.737222+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:01.407274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:01.407274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:11.291191+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.0","version":"109'39","reported_seq":294,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.723459+0000","last_change":"2024-09-13T10:11:05.681812+0000","last_active":"2024-09-13T10:12:17.723459+0000","last_peered":"2024-09-13T10:12:17.723459+0000","last_clean":"2024-09-13T10:12:17.723459+0000","last_became_active":"2024-09-13T10:10:54.320014+0000","last_became_peered":"2024-09-13T10:10:54.320014+0000","last_unstale":"2024-09-13T10:12:17.723459+0000","last_undegraded":"2024-09-13T10:12:17.723459+0000","last_fullsized":"2024-09-13T10:12:17.723459+0000","mapping_epoch":31,"log_start":"108'36","ondisk_log_start":"108'36","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'14","last_scrub_stamp":"2024-09-13T10:11:05.681653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.681653+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":35,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:28.623762+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3058940,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":61,"num_read_kb":51669,"num_write":28,"num_write_kb":7675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.717254+0000","last_change":"2024-09-13T10:12:05.204517+0000","last_active":"2024-09-13T10:12:17.717254+0000","last_peered":"2024-09-13T10:12:17.717254+0000","last_clean":"2024-09-13T10:12:17.717254+0000","last_became_active":"2024-09-13T10:12:05.203738+0000","last_became_peered":"2024-09-13T10:12:05.203738+0000","last_unstale":"2024-09-13T10:12:17.717254+0000","last_undegraded":"2024-09-13T10:12:17.717254+0000","last_fullsized":"2024-09-13T10:12:17.717254+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:11:41.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:41.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:49.027751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12,1],"acting":[3,12,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"111'45","reported_seq":295,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-13T10:12:18.724159+0000","last_change":"2024-09-13T10:11:02.190704+0000","last_active":"2024-09-13T10:12:18.724159+0000","last_peered":"2024-09-13T10:12:18.724159+0000","last_clean":"2024-09-13T10:12:18.724159+0000","last_became_active":"2024-09-13T10:10:54.317600+0000","last_became_peered":"2024-09-13T10:10:54.317600+0000","last_unstale":"2024-09-13T10:12:18.724159+0000","last_undegraded":"2024-09-13T10:12:18.724159+0000","last_fullsized":"2024-09-13T10:12:18.724159+0000","mapping_epoch":31,"log_start":"109'42","ondisk_log_start":"109'42","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'18","last_scrub_stamp":"2024-09-13T10:11:02.190506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:02.190506+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":42,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:23.049999+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":73519,"num_write":32,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,12],"acting":[2,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"105'310","reported_seq":571,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-13T10:12:15.728758+0000","last_change":"2024-09-13T10:12:07.816766+0000","last_active":"2024-09-13T10:12:15.728758+0000","last_peered":"2024-09-13T10:12:15.728758+0000","last_clean":"2024-09-13T10:12:15.728758+0000","last_became_active":"2024-09-13T10:12:07.813457+0000","last_became_peered":"2024-09-13T10:12:07.813457+0000","last_unstale":"2024-09-13T10:12:15.728758+0000","last_undegraded":"2024-09-13T10:12:15.728758+0000","last_fullsized":"2024-09-13T10:12:15.728758+0000","mapping_epoch":100,"log_start":"103'307","ondisk_log_start":"103'307","created":30,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":322,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26546652,"num_objects":33,"num_object_clones":0,"num_object_copies":99,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":0,"num_read":287,"num_read_kb":6904,"num_write":402,"num_write_kb":76781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":26621319,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,10],"acting":[13,14,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":276,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.717547+0000","last_change":"2024-09-13T10:11:48.946651+0000","last_active":"2024-09-13T10:12:17.717547+0000","last_peered":"2024-09-13T10:12:17.717547+0000","last_clean":"2024-09-13T10:12:17.717547+0000","last_became_active":"2024-09-13T10:11:12.699043+0000","last_became_peered":"2024-09-13T10:11:12.699043+0000","last_unstale":"2024-09-13T10:12:17.717547+0000","last_undegraded":"2024-09-13T10:12:17.717547+0000","last_fullsized":"2024-09-13T10:12:17.717547+0000","mapping_epoch":55,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'112","last_scrub_stamp":"2024-09-13T10:11:48.946478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:48.946478+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:12.399266+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,7],"acting":[3,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":97,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-13T10:12:15.728154+0000","last_change":"2024-09-13T10:11:38.808286+0000","last_active":"2024-09-13T10:12:15.728154+0000","last_peered":"2024-09-13T10:12:15.728154+0000","last_clean":"2024-09-13T10:12:15.728154+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:12:15.728154+0000","last_undegraded":"2024-09-13T10:12:15.728154+0000","last_fullsized":"2024-09-13T10:12:15.728154+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T10:11:38.808119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:38.808119+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:43.158000+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.6","version":"105'93","reported_seq":393,"reported_epoch":110,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:12:17.718493+0000","last_change":"2024-09-13T10:12:05.805129+0000","last_active":"2024-09-13T10:12:17.718493+0000","last_peered":"2024-09-13T10:12:17.718493+0000","last_clean":"2024-09-13T10:12:00.406498+0000","last_became_active":"2024-09-13T10:12:05.800127+0000","last_became_peered":"2024-09-13T10:12:05.800127+0000","last_unstale":"2024-09-13T10:12:17.718493+0000","last_undegraded":"2024-09-13T10:12:17.718493+0000","last_fullsized":"2024-09-13T10:12:17.718493+0000","mapping_epoch":98,"log_start":"105'90","ondisk_log_start":"105'90","created":31,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'44","last_scrub_stamp":"2024-09-13T10:11:05.396838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.396838+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":90,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6538022,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":145,"num_read_kb":139161,"num_write":57,"num_write_kb":16492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,4],"acting":[1,7,13],"avail_no_missing":["1","7","13"],"object_location_counts":[{"shards":"1,7,13","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"109'351","reported_seq":550,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.717392+0000","last_change":"2024-09-13T10:12:00.967692+0000","last_active":"2024-09-13T10:12:17.717392+0000","last_peered":"2024-09-13T10:12:17.717392+0000","last_clean":"2024-09-13T10:12:17.717392+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:12:17.717392+0000","last_undegraded":"2024-09-13T10:12:17.717392+0000","last_fullsized":"2024-09-13T10:12:17.717392+0000","mapping_epoch":30,"log_start":"108'348","ondisk_log_start":"108'348","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'314","last_scrub_stamp":"2024-09-13T10:12:00.967585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.967585+0000","objects_scrubbed":33,"log_size":3,"log_dups_size":360,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:19.050419+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30523851,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":323,"num_read_kb":12992,"num_write":462,"num_write_kb":96482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":199,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-13T10:12:18.723965+0000","last_change":"2024-09-13T10:11:30.291380+0000","last_active":"2024-09-13T10:12:18.723965+0000","last_peered":"2024-09-13T10:12:18.723965+0000","last_clean":"2024-09-13T10:12:18.723965+0000","last_became_active":"2024-09-13T10:10:30.127232+0000","last_became_peered":"2024-09-13T10:10:30.127232+0000","last_unstale":"2024-09-13T10:12:18.723965+0000","last_undegraded":"2024-09-13T10:12:18.723965+0000","last_fullsized":"2024-09-13T10:12:18.723965+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-13T10:11:30.291157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:30.291157+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:31.417661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0034622780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"108'104","reported_seq":404,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.719672+0000","last_change":"2024-09-13T10:11:05.436449+0000","last_active":"2024-09-13T10:12:17.719672+0000","last_peered":"2024-09-13T10:12:17.719672+0000","last_clean":"2024-09-13T10:12:17.719672+0000","last_became_active":"2024-09-13T10:10:54.314465+0000","last_became_peered":"2024-09-13T10:10:54.314465+0000","last_unstale":"2024-09-13T10:12:17.719672+0000","last_undegraded":"2024-09-13T10:12:17.719672+0000","last_fullsized":"2024-09-13T10:12:17.719672+0000","mapping_epoch":31,"log_start":"107'101","ondisk_log_start":"107'101","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'40","last_scrub_stamp":"2024-09-13T10:11:05.436272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.436272+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":98,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:23.127724+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5496021,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":118154,"num_write":75,"num_write_kb":22251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,10],"acting":[1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"109'330","reported_seq":515,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.946831+0000","last_change":"2024-09-13T10:12:02.384819+0000","last_active":"2024-09-13T10:12:17.946831+0000","last_peered":"2024-09-13T10:12:17.946831+0000","last_clean":"2024-09-13T10:12:17.946831+0000","last_became_active":"2024-09-13T10:10:43.220074+0000","last_became_peered":"2024-09-13T10:10:43.220074+0000","last_unstale":"2024-09-13T10:12:17.946831+0000","last_undegraded":"2024-09-13T10:12:17.946831+0000","last_fullsized":"2024-09-13T10:12:17.946831+0000","mapping_epoch":30,"log_start":"108'327","ondisk_log_start":"108'327","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'292","last_scrub_stamp":"2024-09-13T10:12:02.384617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:02.384617+0000","objects_scrubbed":28,"log_size":3,"log_dups_size":339,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:26.698118+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25885028,"num_objects":37,"num_object_clones":0,"num_object_copies":111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":0,"num_read":296,"num_read_kb":4421,"num_write":436,"num_write_kb":79964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-13T10:12:16.737018+0000","last_change":"2024-09-13T10:11:53.335859+0000","last_active":"2024-09-13T10:12:16.737018+0000","last_peered":"2024-09-13T10:12:16.737018+0000","last_clean":"2024-09-13T10:12:16.737018+0000","last_became_active":"2024-09-13T10:11:39.046090+0000","last_became_peered":"2024-09-13T10:11:39.046090+0000","last_unstale":"2024-09-13T10:12:16.737018+0000","last_undegraded":"2024-09-13T10:12:16.737018+0000","last_fullsized":"2024-09-13T10:12:16.737018+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:11:53.335756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:53.335756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:17.649356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,7,9],"acting":[15,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.4","version":"110'196","reported_seq":704,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.959386+0000","last_change":"2024-09-13T10:11:03.633738+0000","last_active":"2024-09-13T10:12:17.959386+0000","last_peered":"2024-09-13T10:12:17.959386+0000","last_clean":"2024-09-13T10:12:17.959386+0000","last_became_active":"2024-09-13T10:10:54.316518+0000","last_became_peered":"2024-09-13T10:10:54.316518+0000","last_unstale":"2024-09-13T10:12:17.959386+0000","last_undegraded":"2024-09-13T10:12:17.959386+0000","last_fullsized":"2024-09-13T10:12:17.959386+0000","mapping_epoch":31,"log_start":"109'192","ondisk_log_start":"109'192","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'66","last_scrub_stamp":"2024-09-13T10:11:03.633499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:03.633499+0000","objects_scrubbed":5,"log_size":4,"log_dups_size":192,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17647680,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":592,"num_read_kb":559342,"num_write":146,"num_write_kb":41010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10],"acting":[6,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"111'391","reported_seq":586,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-13T10:12:18.728528+0000","last_change":"2024-09-13T10:12:00.627361+0000","last_active":"2024-09-13T10:12:18.728528+0000","last_peered":"2024-09-13T10:12:18.728528+0000","last_clean":"2024-09-13T10:12:18.728528+0000","last_became_active":"2024-09-13T10:10:43.229961+0000","last_became_peered":"2024-09-13T10:10:43.229961+0000","last_unstale":"2024-09-13T10:12:18.728528+0000","last_undegraded":"2024-09-13T10:12:18.728528+0000","last_fullsized":"2024-09-13T10:12:18.728528+0000","mapping_epoch":30,"log_start":"109'388","ondisk_log_start":"109'388","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'359","last_scrub_stamp":"2024-09-13T10:12:00.627252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.627252+0000","objects_scrubbed":25,"log_size":3,"log_dups_size":397,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:24.042469+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16362327,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":357,"num_read_kb":6439,"num_write":486,"num_write_kb":94253,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":194,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.941093+0000","last_change":"2024-09-13T10:11:51.041068+0000","last_active":"2024-09-13T10:12:17.941093+0000","last_peered":"2024-09-13T10:12:17.941093+0000","last_clean":"2024-09-13T10:12:17.941093+0000","last_became_active":"2024-09-13T10:10:30.272007+0000","last_became_peered":"2024-09-13T10:10:30.272007+0000","last_unstale":"2024-09-13T10:12:17.941093+0000","last_undegraded":"2024-09-13T10:12:17.941093+0000","last_fullsized":"2024-09-13T10:12:17.941093+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:11:51.040975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:51.040975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:13.227058+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.000987604,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"109'172","reported_seq":611,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.941525+0000","last_change":"2024-09-13T10:11:00.541038+0000","last_active":"2024-09-13T10:12:17.941525+0000","last_peered":"2024-09-13T10:12:17.941525+0000","last_clean":"2024-09-13T10:12:17.941525+0000","last_became_active":"2024-09-13T10:10:54.315196+0000","last_became_peered":"2024-09-13T10:10:54.315196+0000","last_unstale":"2024-09-13T10:12:17.941525+0000","last_undegraded":"2024-09-13T10:12:17.941525+0000","last_fullsized":"2024-09-13T10:12:17.941525+0000","mapping_epoch":31,"log_start":"109'169","ondisk_log_start":"109'169","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'48","last_scrub_stamp":"2024-09-13T10:11:00.540905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.540905+0000","objects_scrubbed":5,"log_size":3,"log_dups_size":168,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15176985,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":416,"num_read_kb":376040,"num_write":146,"num_write_kb":45597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,0],"acting":[6,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"39'5","reported_seq":220,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.717138+0000","last_change":"2024-09-13T10:12:13.871233+0000","last_active":"2024-09-13T10:12:17.717138+0000","last_peered":"2024-09-13T10:12:17.717138+0000","last_clean":"2024-09-13T10:12:17.717138+0000","last_became_active":"2024-09-13T10:10:54.314935+0000","last_became_peered":"2024-09-13T10:10:54.314935+0000","last_unstale":"2024-09-13T10:12:17.717138+0000","last_undegraded":"2024-09-13T10:12:17.717138+0000","last_fullsized":"2024-09-13T10:12:17.717138+0000","mapping_epoch":31,"log_start":"38'2","ondisk_log_start":"38'2","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'5","last_scrub_stamp":"2024-09-13T10:12:13.871148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:13.871148+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:42.986641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,0],"acting":[3,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"108'45","reported_seq":314,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.715515+0000","last_change":"2024-09-13T10:11:04.446852+0000","last_active":"2024-09-13T10:12:17.715515+0000","last_peered":"2024-09-13T10:12:17.715515+0000","last_clean":"2024-09-13T10:12:17.715515+0000","last_became_active":"2024-09-13T10:10:54.316057+0000","last_became_peered":"2024-09-13T10:10:54.316057+0000","last_unstale":"2024-09-13T10:12:17.715515+0000","last_undegraded":"2024-09-13T10:12:17.715515+0000","last_fullsized":"2024-09-13T10:12:17.715515+0000","mapping_epoch":31,"log_start":"79'42","ondisk_log_start":"79'42","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'29","last_scrub_stamp":"2024-09-13T10:11:04.446717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.446717+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":39,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:29.249908+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2806802,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":127,"num_read_kb":108933,"num_write":14,"num_write_kb":3748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,9],"acting":[11,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.a","version":"96'25","reported_seq":251,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.941310+0000","last_change":"2024-09-13T10:11:12.708135+0000","last_active":"2024-09-13T10:12:17.941310+0000","last_peered":"2024-09-13T10:12:17.941310+0000","last_clean":"2024-09-13T10:12:17.941310+0000","last_became_active":"2024-09-13T10:11:12.707306+0000","last_became_peered":"2024-09-13T10:11:12.707306+0000","last_unstale":"2024-09-13T10:12:17.941310+0000","last_undegraded":"2024-09-13T10:12:17.941310+0000","last_fullsized":"2024-09-13T10:12:17.941310+0000","mapping_epoch":55,"log_start":"69'22","ondisk_log_start":"69'22","created":31,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":22,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2184976,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":68,"num_read_kb":62891,"num_write":25,"num_write_kb":8053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,13],"acting":[6,8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"109'89","reported_seq":436,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-09-13T10:12:18.723984+0000","last_change":"2024-09-13T10:12:17.219674+0000","last_active":"2024-09-13T10:12:18.723984+0000","last_peered":"2024-09-13T10:12:18.723984+0000","last_clean":"2024-09-13T10:12:18.723984+0000","last_became_active":"2024-09-13T10:12:07.819670+0000","last_became_peered":"2024-09-13T10:12:07.819670+0000","last_unstale":"2024-09-13T10:12:18.723984+0000","last_undegraded":"2024-09-13T10:12:18.723984+0000","last_fullsized":"2024-09-13T10:12:18.723984+0000","mapping_epoch":100,"log_start":"99'86","ondisk_log_start":"99'86","created":31,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'89","last_scrub_stamp":"2024-09-13T10:12:17.219609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:17.219609+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":86,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:43.994854+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3747994,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":335,"num_read_kb":307626,"num_write":85,"num_write_kb":23348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4887315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"109'143","reported_seq":564,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-13T10:12:17.941545+0000","last_change":"2024-09-13T10:10:59.373229+0000","last_active":"2024-09-13T10:12:17.941545+0000","last_peered":"2024-09-13T10:12:17.941545+0000","last_clean":"2024-09-13T10:12:17.941545+0000","last_became_active":"2024-09-13T10:10:59.366341+0000","last_became_peered":"2024-09-13T10:10:59.366341+0000","last_unstale":"2024-09-13T10:12:17.941545+0000","last_undegraded":"2024-09-13T10:12:17.941545+0000","last_fullsized":"2024-09-13T10:12:17.941545+0000","mapping_epoch":44,"log_start":"109'140","ondisk_log_start":"109'140","created":31,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":140,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14084596,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":564,"num_read_kb":512726,"num_write":150,"num_write_kb":41970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,12,3],"acting":[6,12,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"108'73","reported_seq":338,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-13T10:12:16.736980+0000","last_change":"2024-09-13T10:12:03.448785+0000","last_active":"2024-09-13T10:12:16.736980+0000","last_peered":"2024-09-13T10:12:16.736980+0000","last_clean":"2024-09-13T10:12:16.736980+0000","last_became_active":"2024-09-13T10:10:59.364443+0000","last_became_peered":"2024-09-13T10:10:59.364443+0000","last_unstale":"2024-09-13T10:12:16.736980+0000","last_undegraded":"2024-09-13T10:12:16.736980+0000","last_fullsized":"2024-09-13T10:12:16.736980+0000","mapping_epoch":44,"log_start":"108'70","ondisk_log_start":"108'70","created":31,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'67","last_scrub_stamp":"2024-09-13T10:12:03.448609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:03.448609+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":70,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:11.055324+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3042133,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":150,"num_read_kb":144843,"num_write":72,"num_write_kb":20360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,12,7],"acting":[15,12,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.e","version":"111'32","reported_seq":224,"reported_epoch":111,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:12:18.747141+0000","last_change":"2024-09-13T10:11:50.878103+0000","last_active":"2024-09-13T10:12:18.747141+0000","last_peered":"2024-09-13T10:12:18.747141+0000","last_clean":"2024-09-13T10:11:03.155929+0000","last_became_active":"2024-09-13T10:11:20.874865+0000","last_became_peered":"2024-09-13T10:11:20.874865+0000","last_unstale":"2024-09-13T10:12:18.747141+0000","last_undegraded":"2024-09-13T10:12:18.747141+0000","last_fullsized":"2024-09-13T10:12:18.747141+0000","mapping_epoch":61,"log_start":"93'29","ondisk_log_start":"93'29","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":29,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3193396,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":67,"num_read_kb":67323,"num_write":34,"num_write_kb":10536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,15],"acting":[2,8,4],"avail_no_missing":["2","4","8"],"object_location_counts":[{"shards":"2,4,8","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"106'117","reported_seq":424,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-13T10:12:15.728595+0000","last_change":"2024-09-13T10:11:53.668346+0000","last_active":"2024-09-13T10:12:15.728595+0000","last_peered":"2024-09-13T10:12:15.728595+0000","last_clean":"2024-09-13T10:12:15.728595+0000","last_became_active":"2024-09-13T10:10:44.223700+0000","last_became_peered":"2024-09-13T10:10:44.223700+0000","last_unstale":"2024-09-13T10:12:15.728595+0000","last_undegraded":"2024-09-13T10:12:15.728595+0000","last_fullsized":"2024-09-13T10:12:15.728595+0000","mapping_epoch":31,"log_start":"103'113","ondisk_log_start":"103'113","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'87","last_scrub_stamp":"2024-09-13T10:11:53.668179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:53.668179+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":113,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:57.139846+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9344857,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":261,"num_read_kb":252035,"num_write":127,"num_write_kb":35498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,7],"acting":[13,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":26,"stat_sum":{"num_bytes":130256824,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":4453,"num_read_kb":4045620,"num_write":1465,"num_write_kb":418913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":33652752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":18,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":263122944,"data_stored":262350865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":78,"acting":78,"num_store_stats":16},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":99317858,"num_objects":130,"num_object_clones":0,"num_object_copies":390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":0,"num_read":1263,"num_read_kb":30756,"num_write":1786,"num_write_kb":347480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":26621319,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":267763712,"data_stored":267155764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":13}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247833,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82540,"kb_used_data":51356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94289300,"statfs":{"total":96636764160,"available":96552243200,"internally_reserved":0,"allocated":52588544,"data_stored":52005250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247835,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48244,"kb_used_data":17060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323596,"statfs":{"total":96636764160,"available":96587362304,"internally_reserved":0,"allocated":17469440,"data_stored":16955975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247835,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63552,"kb_used_data":32368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308288,"statfs":{"total":96636764160,"available":96571686912,"internally_reserved":0,"allocated":33144832,"data_stored":32562184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313242,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33492,"kb_used_data":6468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338348,"statfs":{"total":96636764160,"available":96602468352,"internally_reserved":0,"allocated":6623232,"data_stored":6131360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,5,6,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313241,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81832,"kb_used_data":50648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94290008,"statfs":{"total":96636764160,"available":96552968192,"internally_reserved":0,"allocated":51863552,"data_stored":51297033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,6,7,8,9,10,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247833,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62020,"kb_used_data":27964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94309820,"statfs":{"total":96636764160,"available":96573255680,"internally_reserved":0,"allocated":28635136,"data_stored":28040556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,7,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280537,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64908,"kb_used_data":32100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306932,"statfs":{"total":96636764160,"available":96570298368,"internally_reserved":0,"allocated":32870400,"data_stored":32237504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280537,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102604,"kb_used_data":71420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94269236,"statfs":{"total":96636764160,"available":96531697664,"internally_reserved":0,"allocated":73134080,"data_stored":72505501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,10,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280538,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75360,"kb_used_data":44176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94296480,"statfs":{"total":96636764160,"available":96559595520,"internally_reserved":0,"allocated":45236224,"data_stored":44673363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280536,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58016,"kb_used_data":30992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94313824,"statfs":{"total":96636764160,"available":96577355776,"internally_reserved":0,"allocated":31735808,"data_stored":31206218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280537,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77936,"kb_used_data":46752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94293904,"statfs":{"total":96636764160,"available":96556957696,"internally_reserved":0,"allocated":47874048,"data_stored":47299219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280537,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99848,"kb_used_data":63252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94271992,"statfs":{"total":96636764160,"available":96534519808,"internally_reserved":0,"allocated":64770048,"data_stored":64069061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280537,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37628,"kb_used_data":6444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334212,"statfs":{"total":96636764160,"available":96598233088,"internally_reserved":0,"allocated":6598656,"data_stored":6102045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280537,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30564,"kb_used_data":3540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341276,"statfs":{"total":96636764160,"available":96605466624,"internally_reserved":0,"allocated":3624960,"data_stored":3137964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247833,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51428,"kb_used_data":20244,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94320412,"statfs":{"total":96636764160,"available":96584101888,"internally_reserved":0,"allocated":20729856,"data_stored":20198896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247832,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77976,"kb_used_data":44800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94293864,"statfs":{"total":96636764160,"available":96556916736,"internally_reserved":0,"allocated":45875200,"data_stored":45138907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12549463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16232448,"data_stored":16181515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":46993408,"data_stored":46886178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":30584832,"data_stored":30523851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":30584832,"data_stored":30523851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":25956352,"data_stored":25885028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":35934208,"data_stored":35847751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":26595328,"data_stored":26546652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":42299392,"data_stored":42211475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":14778368,"data_stored":14706609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15364096,"data_stored":15271505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16949248,"data_stored":16916803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22257664,"data_stored":22229239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13389824,"data_stored":13373745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28418048,"data_stored":28375548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":44597248,"data_stored":44354404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14249984,"data_stored":14226513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":5308416,"data_stored":5294008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":5369856,"data_stored":5363936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":26308608,"data_stored":26181256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":5337088,"data_stored":5326278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":15396864,"data_stored":15376557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":24018944,"data_stored":23982976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":9015296,"data_stored":9009291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":2363392,"data_stored":2362197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:12:20.655 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:20.655 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:12:20.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:12:20.655 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:12:20.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.655 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:70 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:20.655 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:12:20.655 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 110.270791961, 2024-09-13T10:12:20.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034624456999999997, 2024-09-13T10:12:20.656 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.656 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.656 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.656 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.656 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:20.656 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:20.656 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 70 2024-09-13T10:12:20.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.656 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:12:20.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381722+0000", 2024-09-13T10:12:20.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381723+0000", 2024-09-13T10:12:20.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.540000000000001e-07 2024-09-13T10:12:20.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381731+0000", 2024-09-13T10:12:20.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.486e-06 2024-09-13T10:12:20.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381742+0000", 2024-09-13T10:12:20.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1269e-05 2024-09-13T10:12:20.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381750+0000", 2024-09-13T10:12:20.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8830000000000006e-06 2024-09-13T10:12:20.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401255+0000", 2024-09-13T10:12:20.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019505110999999999 2024-09-13T10:12:20.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401388+0000", 2024-09-13T10:12:20.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013253200000000001 2024-09-13T10:12:20.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:20.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.401609+0000", 2024-09-13T10:12:20.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000220981 2024-09-13T10:12:20.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407538+0000", 2024-09-13T10:12:20.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059291220000000002 2024-09-13T10:12:20.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407626+0000", 2024-09-13T10:12:20.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8361000000000003e-05 2024-09-13T10:12:20.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:20.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416154+0000", 2024-09-13T10:12:20.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0085281000000000003 2024-09-13T10:12:20.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416234+0000", 2024-09-13T10:12:20.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9594999999999999e-05 2024-09-13T10:12:20.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.662 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.662 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416347+0000", 2024-09-13T10:12:20.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113063 2024-09-13T10:12:20.662 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.662 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.662 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.662 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:71 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:20.662 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:12:20.662 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 110.270751379, 2024-09-13T10:12:20.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035004, 2024-09-13T10:12:20.663 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.663 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.663 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.663 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.663 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:20.663 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:20.663 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 71 2024-09-13T10:12:20.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.663 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:12:20.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381763+0000", 2024-09-13T10:12:20.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381764+0000", 2024-09-13T10:12:20.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4600000000000004e-07 2024-09-13T10:12:20.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381769+0000", 2024-09-13T10:12:20.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.48e-06 2024-09-13T10:12:20.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381780+0000", 2024-09-13T10:12:20.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0893000000000001e-05 2024-09-13T10:12:20.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381788+0000", 2024-09-13T10:12:20.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6389999999999992e-06 2024-09-13T10:12:20.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.407896+0000", 2024-09-13T10:12:20.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.026107807 2024-09-13T10:12:20.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.408029+0000", 2024-09-13T10:12:20.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013383899999999999 2024-09-13T10:12:20.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:20.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.408250+0000", 2024-09-13T10:12:20.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022105399999999999 2024-09-13T10:12:20.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416130+0000", 2024-09-13T10:12:20.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0078798500000000007 2024-09-13T10:12:20.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.667 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.667 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416488+0000", 2024-09-13T10:12:20.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00035819299999999998 2024-09-13T10:12:20.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:20.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416595+0000", 2024-09-13T10:12:20.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0001064 2024-09-13T10:12:20.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416662+0000", 2024-09-13T10:12:20.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6775999999999999e-05 2024-09-13T10:12:20.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416767+0000", 2024-09-13T10:12:20.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000105423 2024-09-13T10:12:20.669 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.669 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.669 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.669 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:72 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:20.669 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:12:20.669 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 110.270714345, 2024-09-13T10:12:20.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039399762999999997, 2024-09-13T10:12:20.669 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.670 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.670 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.670 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.670 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:20.670 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:20.670 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 72 2024-09-13T10:12:20.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.670 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:12:20.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381800+0000", 2024-09-13T10:12:20.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381801+0000", 2024-09-13T10:12:20.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7700000000000004e-07 2024-09-13T10:12:20.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381898+0000", 2024-09-13T10:12:20.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7066999999999997e-05 2024-09-13T10:12:20.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381908+0000", 2024-09-13T10:12:20.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0336000000000001e-05 2024-09-13T10:12:20.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381917+0000", 2024-09-13T10:12:20.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5169999999999999e-06 2024-09-13T10:12:20.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.411900+0000", 2024-09-13T10:12:20.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029983381999999999 2024-09-13T10:12:20.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.412028+0000", 2024-09-13T10:12:20.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127916 2024-09-13T10:12:20.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:20.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.412248+0000", 2024-09-13T10:12:20.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022014699999999999 2024-09-13T10:12:20.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416576+0000", 2024-09-13T10:12:20.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004327549 2024-09-13T10:12:20.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416860+0000", 2024-09-13T10:12:20.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028391000000000001 2024-09-13T10:12:20.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:20.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421014+0000", 2024-09-13T10:12:20.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041539330000000003 2024-09-13T10:12:20.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421093+0000", 2024-09-13T10:12:20.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9097999999999994e-05 2024-09-13T10:12:20.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421200+0000", 2024-09-13T10:12:20.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000107231 2024-09-13T10:12:20.675 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.675 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.676 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.676 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:73 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:20.676 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:12:20.676 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 110.27058063299999, 2024-09-13T10:12:20.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044034528000000003, 2024-09-13T10:12:20.676 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.676 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.676 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.676 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.676 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:20.676 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:20.677 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 73 2024-09-13T10:12:20.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.677 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:12:20.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381934+0000", 2024-09-13T10:12:20.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381935+0000", 2024-09-13T10:12:20.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0999999999999997e-07 2024-09-13T10:12:20.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381940+0000", 2024-09-13T10:12:20.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7030000000000003e-06 2024-09-13T10:12:20.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381949+0000", 2024-09-13T10:12:20.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5359999999999992e-06 2024-09-13T10:12:20.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381961+0000", 2024-09-13T10:12:20.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2539e-05 2024-09-13T10:12:20.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.416942+0000", 2024-09-13T10:12:20.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.034980270000000001 2024-09-13T10:12:20.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.417074+0000", 2024-09-13T10:12:20.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013246199999999999 2024-09-13T10:12:20.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:20.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.417298+0000", 2024-09-13T10:12:20.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000223823 2024-09-13T10:12:20.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425459+0000", 2024-09-13T10:12:20.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081613750000000002 2024-09-13T10:12:20.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425575+0000", 2024-09-13T10:12:20.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115513 2024-09-13T10:12:20.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:20.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425774+0000", 2024-09-13T10:12:20.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019959399999999999 2024-09-13T10:12:20.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425854+0000", 2024-09-13T10:12:20.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9532999999999999e-05 2024-09-13T10:12:20.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425968+0000", 2024-09-13T10:12:20.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011437 2024-09-13T10:12:20.682 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.682 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.682 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.682 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:74 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:20.682 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:12:20.683 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 110.27053775900001, 2024-09-13T10:12:20.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052701496, 2024-09-13T10:12:20.683 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.683 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.683 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.683 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.683 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:20.683 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:20.683 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 74 2024-09-13T10:12:20.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.683 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:12:20.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:12:20.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381977+0000", 2024-09-13T10:12:20.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7999999999999995e-07 2024-09-13T10:12:20.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381983+0000", 2024-09-13T10:12:20.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4430000000000002e-06 2024-09-13T10:12:20.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381991+0000", 2024-09-13T10:12:20.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8960000000000003e-06 2024-09-13T10:12:20.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.381998+0000", 2024-09-13T10:12:20.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7729999999999999e-06 2024-09-13T10:12:20.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421291+0000", 2024-09-13T10:12:20.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039292753 2024-09-13T10:12:20.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421424+0000", 2024-09-13T10:12:20.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013291900000000001 2024-09-13T10:12:20.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:20.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.421646+0000", 2024-09-13T10:12:20.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000221843 2024-09-13T10:12:20.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425667+0000", 2024-09-13T10:12:20.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040206779999999998 2024-09-13T10:12:20.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.425755+0000", 2024-09-13T10:12:20.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8848e-05 2024-09-13T10:12:20.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:20.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434472+0000", 2024-09-13T10:12:20.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0087164879999999997 2024-09-13T10:12:20.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434544+0000", 2024-09-13T10:12:20.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2273000000000004e-05 2024-09-13T10:12:20.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434678+0000", 2024-09-13T10:12:20.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013390200000000001 2024-09-13T10:12:20.689 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.689 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.689 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.689 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:75 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:20.689 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:12:20.689 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 110.27050089399999, 2024-09-13T10:12:20.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.052864585999999998, 2024-09-13T10:12:20.690 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.690 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.690 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.690 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.690 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:20.690 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:20.690 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 75 2024-09-13T10:12:20.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.690 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:12:20.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:12:20.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382014+0000", 2024-09-13T10:12:20.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8299999999999996e-07 2024-09-13T10:12:20.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382120+0000", 2024-09-13T10:12:20.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010627400000000001 2024-09-13T10:12:20.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382131+0000", 2024-09-13T10:12:20.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0796e-05 2024-09-13T10:12:20.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382140+0000", 2024-09-13T10:12:20.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5560000000000008e-06 2024-09-13T10:12:20.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426014+0000", 2024-09-13T10:12:20.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.043873843000000003 2024-09-13T10:12:20.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426205+0000", 2024-09-13T10:12:20.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019154600000000001 2024-09-13T10:12:20.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:20.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.426429+0000", 2024-09-13T10:12:20.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022344699999999999 2024-09-13T10:12:20.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434352+0000", 2024-09-13T10:12:20.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.007923355 2024-09-13T10:12:20.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434452+0000", 2024-09-13T10:12:20.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9961000000000001e-05 2024-09-13T10:12:20.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:20.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434700+0000", 2024-09-13T10:12:20.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024790899999999999 2024-09-13T10:12:20.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434774+0000", 2024-09-13T10:12:20.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4457999999999998e-05 2024-09-13T10:12:20.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.434878+0000", 2024-09-13T10:12:20.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010375800000000001 2024-09-13T10:12:20.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.696 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.696 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:76 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:20.696 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:12:20.696 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 110.27033391400001, 2024-09-13T10:12:20.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053104138000000002, 2024-09-13T10:12:20.697 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.697 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.697 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:20.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:20.697 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 76 2024-09-13T10:12:20.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.697 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:12:20.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:12:20.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382181+0000", 2024-09-13T10:12:20.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4600000000000003e-07 2024-09-13T10:12:20.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382188+0000", 2024-09-13T10:12:20.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4869999999999999e-06 2024-09-13T10:12:20.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382197+0000", 2024-09-13T10:12:20.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5519999999999993e-06 2024-09-13T10:12:20.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.382207+0000", 2024-09-13T10:12:20.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7240000000000004e-06 2024-09-13T10:12:20.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430285+0000", 2024-09-13T10:12:20.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.048078093000000002 2024-09-13T10:12:20.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430429+0000", 2024-09-13T10:12:20.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000143981 2024-09-13T10:12:20.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:20.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.430717+0000", 2024-09-13T10:12:20.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028802 2024-09-13T10:12:20.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435015+0000", 2024-09-13T10:12:20.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042973150000000003 2024-09-13T10:12:20.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435083+0000", 2024-09-13T10:12:20.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8448999999999995e-05 2024-09-13T10:12:20.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:20.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435109+0000", 2024-09-13T10:12:20.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6064999999999999e-05 2024-09-13T10:12:20.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435176+0000", 2024-09-13T10:12:20.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7362999999999999e-05 2024-09-13T10:12:20.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.435285+0000", 2024-09-13T10:12:20.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108243 2024-09-13T10:12:20.703 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.703 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.703 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.703 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:96 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:20.703 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:12:20.703 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 110.128702484, 2024-09-13T10:12:20.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032701419000000002, 2024-09-13T10:12:20.703 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.703 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.703 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.704 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.704 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:20.704 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:20.704 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 96 2024-09-13T10:12:20.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.704 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:12:20.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523812+0000", 2024-09-13T10:12:20.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523813+0000", 2024-09-13T10:12:20.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.364e-06 2024-09-13T10:12:20.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523913+0000", 2024-09-13T10:12:20.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9827000000000005e-05 2024-09-13T10:12:20.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523927+0000", 2024-09-13T10:12:20.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4029000000000001e-05 2024-09-13T10:12:20.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523936+0000", 2024-09-13T10:12:20.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1030000000000008e-06 2024-09-13T10:12:20.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.544707+0000", 2024-09-13T10:12:20.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0207706 2024-09-13T10:12:20.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.544834+0000", 2024-09-13T10:12:20.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012683200000000001 2024-09-13T10:12:20.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:20.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.545057+0000", 2024-09-13T10:12:20.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022338100000000001 2024-09-13T10:12:20.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549279+0000", 2024-09-13T10:12:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042223080000000001 2024-09-13T10:12:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549556+0000", 2024-09-13T10:12:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000276307 2024-09-13T10:12:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556364+0000", 2024-09-13T10:12:20.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0068082510000000004 2024-09-13T10:12:20.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556445+0000", 2024-09-13T10:12:20.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1081000000000005e-05 2024-09-13T10:12:20.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556513+0000", 2024-09-13T10:12:20.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8336000000000004e-05 2024-09-13T10:12:20.709 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.709 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.710 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.710 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:97 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:20.710 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:12:20.710 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 110.128559599, 2024-09-13T10:12:20.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035410807000000002, 2024-09-13T10:12:20.710 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.710 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.710 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.710 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.710 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:20.711 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:20.711 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 97 2024-09-13T10:12:20.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.711 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:12:20.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523955+0000", 2024-09-13T10:12:20.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523956+0000", 2024-09-13T10:12:20.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8599999999999997e-07 2024-09-13T10:12:20.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523962+0000", 2024-09-13T10:12:20.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1240000000000002e-06 2024-09-13T10:12:20.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523971+0000", 2024-09-13T10:12:20.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7320000000000004e-06 2024-09-13T10:12:20.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523978+0000", 2024-09-13T10:12:20.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0899999999999999e-06 2024-09-13T10:12:20.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549637+0000", 2024-09-13T10:12:20.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025658876000000001 2024-09-13T10:12:20.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549757+0000", 2024-09-13T10:12:20.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000120837 2024-09-13T10:12:20.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:20.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.549971+0000", 2024-09-13T10:12:20.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021356000000000001 2024-09-13T10:12:20.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556335+0000", 2024-09-13T10:12:20.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0063639329999999996 2024-09-13T10:12:20.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556725+0000", 2024-09-13T10:12:20.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000390221 2024-09-13T10:12:20.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:20.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559219+0000", 2024-09-13T10:12:20.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0024943500000000002 2024-09-13T10:12:20.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559296+0000", 2024-09-13T10:12:20.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7065999999999997e-05 2024-09-13T10:12:20.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559366+0000", 2024-09-13T10:12:20.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9132000000000002e-05 2024-09-13T10:12:20.716 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.716 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.716 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.716 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:98 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:20.717 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:12:20.717 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 110.12852499100001, 2024-09-13T10:12:20.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035839604999999997, 2024-09-13T10:12:20.717 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.717 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.717 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.717 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.717 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:20.717 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:20.717 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 98 2024-09-13T10:12:20.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.717 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:12:20.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523989+0000", 2024-09-13T10:12:20.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523990+0000", 2024-09-13T10:12:20.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6300000000000005e-07 2024-09-13T10:12:20.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.523995+0000", 2024-09-13T10:12:20.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2669999999999997e-06 2024-09-13T10:12:20.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524007+0000", 2024-09-13T10:12:20.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2002e-05 2024-09-13T10:12:20.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524015+0000", 2024-09-13T10:12:20.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4869999999999996e-06 2024-09-13T10:12:20.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.553854+0000", 2024-09-13T10:12:20.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.029839633000000001 2024-09-13T10:12:20.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.553985+0000", 2024-09-13T10:12:20.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013032899999999999 2024-09-13T10:12:20.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:20.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.554212+0000", 2024-09-13T10:12:20.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022723400000000001 2024-09-13T10:12:20.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559523+0000", 2024-09-13T10:12:20.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053113509999999997 2024-09-13T10:12:20.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559662+0000", 2024-09-13T10:12:20.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013833500000000001 2024-09-13T10:12:20.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:20.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559698+0000", 2024-09-13T10:12:20.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5982000000000002e-05 2024-09-13T10:12:20.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559764+0000", 2024-09-13T10:12:20.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6019000000000006e-05 2024-09-13T10:12:20.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559829+0000", 2024-09-13T10:12:20.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5302999999999995e-05 2024-09-13T10:12:20.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.723 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.723 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:99 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:20.723 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:12:20.723 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 110.12848729700001, 2024-09-13T10:12:20.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.037637084000000001, 2024-09-13T10:12:20.724 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.724 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.724 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:20.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:20.724 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 99 2024-09-13T10:12:20.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.724 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:12:20.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524027+0000", 2024-09-13T10:12:20.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524028+0000", 2024-09-13T10:12:20.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7999999999999995e-07 2024-09-13T10:12:20.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524109+0000", 2024-09-13T10:12:20.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0834999999999994e-05 2024-09-13T10:12:20.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524119+0000", 2024-09-13T10:12:20.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0149999999999999e-05 2024-09-13T10:12:20.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524127+0000", 2024-09-13T10:12:20.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8660000000000006e-06 2024-09-13T10:12:20.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.556868+0000", 2024-09-13T10:12:20.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.03274146 2024-09-13T10:12:20.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.557001+0000", 2024-09-13T10:12:20.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000133249 2024-09-13T10:12:20.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:20.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.557224+0000", 2024-09-13T10:12:20.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000222914 2024-09-13T10:12:20.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561387+0000", 2024-09-13T10:12:20.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041627019999999999 2024-09-13T10:12:20.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:20.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561419+0000", 2024-09-13T10:12:20.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1924999999999997e-05 2024-09-13T10:12:20.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561476+0000", 2024-09-13T10:12:20.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7346999999999998e-05 2024-09-13T10:12:20.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561552+0000", 2024-09-13T10:12:20.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5847999999999996e-05 2024-09-13T10:12:20.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.561664+0000", 2024-09-13T10:12:20.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112108 2024-09-13T10:12:20.730 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.730 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.730 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.730 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4281.0:100 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T10:12:20.730 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:12:20.730 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 110.12831114700001, 2024-09-13T10:12:20.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.038825510000000001, 2024-09-13T10:12:20.730 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.730 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.731 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.731 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.731 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4281", 2024-09-13T10:12:20.731 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/2444672215", 2024-09-13T10:12:20.731 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 100 2024-09-13T10:12:20.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.731 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:12:20.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524203+0000", 2024-09-13T10:12:20.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524204+0000", 2024-09-13T10:12:20.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.54e-07 2024-09-13T10:12:20.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524211+0000", 2024-09-13T10:12:20.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6919999999999996e-06 2024-09-13T10:12:20.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524221+0000", 2024-09-13T10:12:20.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0223e-05 2024-09-13T10:12:20.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.524229+0000", 2024-09-13T10:12:20.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9859999999999994e-06 2024-09-13T10:12:20.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.559920+0000", 2024-09-13T10:12:20.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.035690854000000001 2024-09-13T10:12:20.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.560052+0000", 2024-09-13T10:12:20.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013199899999999999 2024-09-13T10:12:20.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,7", 2024-09-13T10:12:20.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.560356+0000", 2024-09-13T10:12:20.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030450200000000001 2024-09-13T10:12:20.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:20.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562115+0000", 2024-09-13T10:12:20.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0017586430000000001 2024-09-13T10:12:20.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562512+0000", 2024-09-13T10:12:20.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00039728099999999998 2024-09-13T10:12:20.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562828+0000", 2024-09-13T10:12:20.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00031528799999999999 2024-09-13T10:12:20.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.562913+0000", 2024-09-13T10:12:20.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5628000000000001e-05 2024-09-13T10:12:20.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:30.563029+0000", 2024-09-13T10:12:20.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011556 2024-09-13T10:12:20.737 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.737 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.737 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.737 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:201 3.3 e38/30 3:d4fb61be:::smithi19530714-2 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:12:20.737 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:12:20.737 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.638916855000005, 2024-09-13T10:12:20.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032127035999999998, 2024-09-13T10:12:20.737 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.737 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.737 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.738 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:12:20.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:12:20.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.013598+0000", 2024-09-13T10:12:20.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2600000000000001e-07 2024-09-13T10:12:20.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030449+0000", 2024-09-13T10:12:20.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016850477999999999 2024-09-13T10:12:20.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030499+0000", 2024-09-13T10:12:20.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0212e-05 2024-09-13T10:12:20.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030524+0000", 2024-09-13T10:12:20.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5222e-05 2024-09-13T10:12:20.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030739+0000", 2024-09-13T10:12:20.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000214396 2024-09-13T10:12:20.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.030818+0000", 2024-09-13T10:12:20.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8941000000000002e-05 2024-09-13T10:12:20.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.045512+0000", 2024-09-13T10:12:20.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014694248 2024-09-13T10:12:20.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.045725+0000", 2024-09-13T10:12:20.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021261300000000001 2024-09-13T10:12:20.741 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.742 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.742 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.742 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:226 3.3 e38/30 3:fe5bfa3a:::smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'5, mlcod=38'3)", 2024-09-13T10:12:20.742 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:12:20.742 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.572056611999997, 2024-09-13T10:12:20.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030943633000000002, 2024-09-13T10:12:20.742 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.742 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.742 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.742 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:12:20.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080458+0000", 2024-09-13T10:12:20.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.080461+0000", 2024-09-13T10:12:20.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7049999999999999e-06 2024-09-13T10:12:20.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097571+0000", 2024-09-13T10:12:20.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017110658000000001 2024-09-13T10:12:20.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097615+0000", 2024-09-13T10:12:20.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3404e-05 2024-09-13T10:12:20.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097635+0000", 2024-09-13T10:12:20.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0279e-05 2024-09-13T10:12:20.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.097995+0000", 2024-09-13T10:12:20.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00036038699999999999 2024-09-13T10:12:20.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.098029+0000", 2024-09-13T10:12:20.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4162000000000003e-05 2024-09-13T10:12:20.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111330+0000", 2024-09-13T10:12:20.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013300268000000001 2024-09-13T10:12:20.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.111401+0000", 2024-09-13T10:12:20.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1769999999999999e-05 2024-09-13T10:12:20.746 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.747 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.747 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.747 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.4607.0:237 3.3 e38/30 3:de40747e:::smithi19530714-38 low_tier:head v 38'7, mlcod=38'3)", 2024-09-13T10:12:20.747 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:12:20.747 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.53782382, 2024-09-13T10:12:20.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.037747504000000001, 2024-09-13T10:12:20.747 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.747 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.747 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.747 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:12:20.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:12:20.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.114691+0000", 2024-09-13T10:12:20.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7299999999999995e-07 2024-09-13T10:12:20.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145311+0000", 2024-09-13T10:12:20.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030619621 2024-09-13T10:12:20.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145336+0000", 2024-09-13T10:12:20.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5009999999999999e-05 2024-09-13T10:12:20.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145349+0000", 2024-09-13T10:12:20.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2724999999999999e-05 2024-09-13T10:12:20.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145445+0000", 2024-09-13T10:12:20.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6108999999999997e-05 2024-09-13T10:12:20.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.145470+0000", 2024-09-13T10:12:20.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5514e-05 2024-09-13T10:12:20.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152389+0000", 2024-09-13T10:12:20.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069190739999999999 2024-09-13T10:12:20.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.152438+0000", 2024-09-13T10:12:20.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8778e-05 2024-09-13T10:12:20.751 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.751 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.751 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.752 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:286 4.a 4:5ad8883b:::smithi19530714-37:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:20.752 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:12:20.752 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.404227192999997, 2024-09-13T10:12:20.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032980435000000002, 2024-09-13T10:12:20.752 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.752 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.752 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:20.752 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.752 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:20.752 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:20.752 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 286 2024-09-13T10:12:20.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.753 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:12:20.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248287+0000", 2024-09-13T10:12:20.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248290+0000", 2024-09-13T10:12:20.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0170000000000001e-06 2024-09-13T10:12:20.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248302+0000", 2024-09-13T10:12:20.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1602e-05 2024-09-13T10:12:20.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248328+0000", 2024-09-13T10:12:20.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.622e-05 2024-09-13T10:12:20.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248344+0000", 2024-09-13T10:12:20.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6192999999999999e-05 2024-09-13T10:12:20.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248466+0000", 2024-09-13T10:12:20.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012186899999999999 2024-09-13T10:12:20.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.248602+0000", 2024-09-13T10:12:20.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013623500000000001 2024-09-13T10:12:20.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 6,13", 2024-09-13T10:12:20.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274091+0000", 2024-09-13T10:12:20.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.025489050999999999 2024-09-13T10:12:20.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275976+0000", 2024-09-13T10:12:20.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018847289999999999 2024-09-13T10:12:20.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:20.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276801+0000", 2024-09-13T10:12:20.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00082477899999999999 2024-09-13T10:12:20.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:20.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281064+0000", 2024-09-13T10:12:20.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0042625950000000001 2024-09-13T10:12:20.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:20.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281173+0000", 2024-09-13T10:12:20.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000109391 2024-09-13T10:12:20.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281268+0000", 2024-09-13T10:12:20.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4753999999999994e-05 2024-09-13T10:12:20.758 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.758 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.758 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.758 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1631 4.a 4:5ad8883b:::smithi19530714-37:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:12:20.759 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:11:06.099753+0000", 2024-09-13T10:12:20.759 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 74.552761619999998, 2024-09-13T10:12:20.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.033125612999999998, 2024-09-13T10:12:20.759 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.759 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.759 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:12:20.759 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.759 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:20.759 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:20.759 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1631 2024-09-13T10:12:20.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.759 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099753+0000", 2024-09-13T10:12:20.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099753+0000", 2024-09-13T10:12:20.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099754+0000", 2024-09-13T10:12:20.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2699999999999999e-06 2024-09-13T10:12:20.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099758+0000", 2024-09-13T10:12:20.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6540000000000001e-06 2024-09-13T10:12:20.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099768+0000", 2024-09-13T10:12:20.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0412e-05 2024-09-13T10:12:20.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099777+0000", 2024-09-13T10:12:20.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4600000000000003e-06 2024-09-13T10:12:20.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099830+0000", 2024-09-13T10:12:20.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3192000000000002e-05 2024-09-13T10:12:20.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.099889+0000", 2024-09-13T10:12:20.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9576e-05 2024-09-13T10:12:20.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:11:06.132878+0000", 2024-09-13T10:12:20.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.032989048999999999 2024-09-13T10:12:20.763 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.763 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.763 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.763 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:4521 4.18 4:1de9da87:::smithi19530714-5:head [read 0~2050520,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T10:12:20.763 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:12:15.731244+0000", 2024-09-13T10:12:20.764 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9212704590000005, 2024-09-13T10:12:20.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.17622544800000001, 2024-09-13T10:12:20.764 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.764 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.764 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:12:20.764 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.764 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:20.764 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:20.764 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 4521 2024-09-13T10:12:20.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.764 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731244+0000", 2024-09-13T10:12:20.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731244+0000", 2024-09-13T10:12:20.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731247+0000", 2024-09-13T10:12:20.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7930000000000002e-06 2024-09-13T10:12:20.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731259+0000", 2024-09-13T10:12:20.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2583e-05 2024-09-13T10:12:20.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731281+0000", 2024-09-13T10:12:20.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2010000000000001e-05 2024-09-13T10:12:20.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731305+0000", 2024-09-13T10:12:20.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3203000000000001e-05 2024-09-13T10:12:20.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731449+0000", 2024-09-13T10:12:20.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000144077 2024-09-13T10:12:20.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:12:20.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731470+0000", 2024-09-13T10:12:20.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1393e-05 2024-09-13T10:12:20.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.902023+0000", 2024-09-13T10:12:20.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.17055287 2024-09-13T10:12:20.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.902327+0000", 2024-09-13T10:12:20.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030380400000000002 2024-09-13T10:12:20.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.769 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.907469+0000", 2024-09-13T10:12:20.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.005142715 2024-09-13T10:12:20.769 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.769 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.769 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.769 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:4522 4.18 4:1de9da87:::smithi19530714-5:head [sparse-read 0~1489368] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T10:12:20.769 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:12:15.731356+0000", 2024-09-13T10:12:20.769 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9211585250000001, 2024-09-13T10:12:20.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.177464128, 2024-09-13T10:12:20.769 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.769 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.770 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:12:20.770 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.770 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:20.770 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:20.770 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 4522 2024-09-13T10:12:20.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.770 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731356+0000", 2024-09-13T10:12:20.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731356+0000", 2024-09-13T10:12:20.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731357+0000", 2024-09-13T10:12:20.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.497e-06 2024-09-13T10:12:20.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731365+0000", 2024-09-13T10:12:20.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.34e-06 2024-09-13T10:12:20.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731377+0000", 2024-09-13T10:12:20.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2346e-05 2024-09-13T10:12:20.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731387+0000", 2024-09-13T10:12:20.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6730000000000004e-06 2024-09-13T10:12:20.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731586+0000", 2024-09-13T10:12:20.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000199426 2024-09-13T10:12:20.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:12:20.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731618+0000", 2024-09-13T10:12:20.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1922999999999999e-05 2024-09-13T10:12:20.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.908111+0000", 2024-09-13T10:12:20.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.17649244 2024-09-13T10:12:20.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.908267+0000", 2024-09-13T10:12:20.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015627999999999999 2024-09-13T10:12:20.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.908820+0000", 2024-09-13T10:12:20.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00055320300000000005 2024-09-13T10:12:20.774 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.774 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.774 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.775 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:4523 4.18 4:1de9da87:::smithi19530714-5:head [sparse-read 0~1489368] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T10:12:20.775 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:12:15.731403+0000", 2024-09-13T10:12:20.775 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9211117719999997, 2024-09-13T10:12:20.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.17873582199999999, 2024-09-13T10:12:20.775 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:20.775 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:20.775 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:12:20.775 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:20.775 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:20.775 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:20.776 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 4523 2024-09-13T10:12:20.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.776 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:20.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:20.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731403+0000", 2024-09-13T10:12:20.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:20.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731403+0000", 2024-09-13T10:12:20.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:20.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:20.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731403+0000", 2024-09-13T10:12:20.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7300000000000005e-07 2024-09-13T10:12:20.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:20.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731407+0000", 2024-09-13T10:12:20.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4699999999999998e-06 2024-09-13T10:12:20.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:20.778 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731418+0000", 2024-09-13T10:12:20.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.078e-05 2024-09-13T10:12:20.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.778 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:20.778 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731431+0000", 2024-09-13T10:12:20.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3753e-05 2024-09-13T10:12:20.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.778 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.778 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731700+0000", 2024-09-13T10:12:20.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00026818099999999999 2024-09-13T10:12:20.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for peered", 2024-09-13T10:12:20.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.731716+0000", 2024-09-13T10:12:20.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6637e-05 2024-09-13T10:12:20.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:20.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.908969+0000", 2024-09-13T10:12:20.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.177252835 2024-09-13T10:12:20.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:20.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.909132+0000", 2024-09-13T10:12:20.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000162759 2024-09-13T10:12:20.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:20.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:20.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:20.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:12:15.910138+0000", 2024-09-13T10:12:20.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001006634 2024-09-13T10:12:20.780 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.780 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.780 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.780 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:20.780 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:20.781 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:20.783 INFO:tasks.rados.rados.0.smithi195.stdout:1951: expect (ObjNum 342 snap 0 seq_num 342) 2024-09-13T10:12:20.784 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-13T10:12:20.785 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:20.782+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:20.883 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:12:20.883+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:12:20.886 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-13T10:12:20.955 INFO:tasks.rados.rados.0.smithi195.stdout:1949: finishing set_chunk to oid 11 2024-09-13T10:12:20.956 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 11 v 96 (ObjNum 359 snap 0 seq_num 359) dirty exists 2024-09-13T10:12:20.956 INFO:tasks.rados.rados.0.smithi195.stdout:1953: finishing set_chunk to oid 31 2024-09-13T10:12:20.956 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 31 v 110 (ObjNum 402 snap 0 seq_num 402) dirty exists 2024-09-13T10:12:20.956 INFO:tasks.rados.rados.0.smithi195.stdout:1954: finishing write tid 1 to smithi19530714-37 2024-09-13T10:12:20.956 INFO:tasks.rados.rados.0.smithi195.stdout:1954: finishing write tid 2 to smithi19530714-37 2024-09-13T10:12:20.956 INFO:tasks.rados.rados.0.smithi195.stdout:1954: finishing write tid 3 to smithi19530714-37 2024-09-13T10:12:20.956 INFO:tasks.rados.rados.0.smithi195.stdout:1954: finishing write tid 4 to smithi19530714-37 2024-09-13T10:12:20.956 INFO:tasks.rados.rados.0.smithi195.stdout:1954: finishing write tid 5 to smithi19530714-37 2024-09-13T10:12:20.956 INFO:tasks.rados.rados.0.smithi195.stdout:1954: oid 37 updating version 0 to 27 2024-09-13T10:12:20.956 INFO:tasks.rados.rados.0.smithi195.stdout:1954: oid 37 updating version 27 to 28 2024-09-13T10:12:20.956 INFO:tasks.rados.rados.0.smithi195.stdout:1954: oid 37 updating version 28 to 29 2024-09-13T10:12:20.957 INFO:tasks.rados.rados.0.smithi195.stdout:1954: oid 37 version 29 is already newer than 26 2024-09-13T10:12:20.957 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 29 (ObjNum 438 snap 0 seq_num 438) dirty exists 2024-09-13T10:12:20.957 INFO:tasks.rados.rados.0.smithi195.stdout:1954: left oid 37 (ObjNum 438 snap 0 seq_num 438) 2024-09-13T10:12:20.957 INFO:tasks.rados.rados.0.smithi195.stdout:1946: done (15 left) 2024-09-13T10:12:20.957 INFO:tasks.rados.rados.0.smithi195.stdout:1947: done (14 left) 2024-09-13T10:12:20.957 INFO:tasks.rados.rados.0.smithi195.stdout:1948: done (13 left) 2024-09-13T10:12:20.957 INFO:tasks.rados.rados.0.smithi195.stdout:1949: done (12 left) 2024-09-13T10:12:20.957 INFO:tasks.rados.rados.0.smithi195.stdout:1950: done (11 left) 2024-09-13T10:12:20.957 INFO:tasks.rados.rados.0.smithi195.stdout:1951: done (10 left) 2024-09-13T10:12:20.957 INFO:tasks.rados.rados.0.smithi195.stdout:1952: done (9 left) 2024-09-13T10:12:20.957 INFO:tasks.rados.rados.0.smithi195.stdout:1953: done (8 left) 2024-09-13T10:12:20.957 INFO:tasks.rados.rados.0.smithi195.stdout:1954: done (7 left) 2024-09-13T10:12:20.957 INFO:tasks.rados.rados.0.smithi195.stdout:1958: done (6 left) 2024-09-13T10:12:20.958 INFO:tasks.rados.rados.0.smithi195.stdout:1962: read oid 2 snap -1 2024-09-13T10:12:20.958 INFO:tasks.rados.rados.0.smithi195.stdout:1962: expect (ObjNum 412 snap 0 seq_num 412) 2024-09-13T10:12:20.958 INFO:tasks.rados.rados.0.smithi195.stdout:1963: tier_flush oid 5 2024-09-13T10:12:20.958 INFO:tasks.rados.rados.0.smithi195.stdout:1963: tier_flush oid 5 snap -1 2024-09-13T10:12:20.958 INFO:tasks.rados.rados.0.smithi195.stdout:1955: expect (ObjNum 421 snap 0 seq_num 421) 2024-09-13T10:12:20.958 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:20.958 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:12:20.958 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-13T10:12:20.959 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:20.971 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-13T10:12:20.984 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:20.978+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:21.036 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:21.085 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:21.081+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:21.097 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:21.097 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-13T10:12:21.097 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-09-13T10:12:21.097 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-09-13T10:12:21.097 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:21.112 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-13T10:12:21.112 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:21.186 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:12:21.183+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:12:21.187 INFO:tasks.rados.rados.0.smithi195.stdout:1957: expect (ObjNum 337 snap 0 seq_num 337) 2024-09-13T10:12:21.187 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:21.229 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:12:21.230 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-09-13T10:12:21.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-09-13T10:12:21.230 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-13T10:12:21.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.230 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:53 4.c 4:3e6706e5:::smithi19530714-14:head [write 618491~694590,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:21.230 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:50.494212+0000", 2024-09-13T10:12:21.230 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 90.733516651000002, 2024-09-13T10:12:21.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0086614210000000007, 2024-09-13T10:12:21.230 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.230 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.231 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.231 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.231 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.231 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.231 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 53 2024-09-13T10:12:21.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.231 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494212+0000", 2024-09-13T10:12:21.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494212+0000", 2024-09-13T10:12:21.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494213+0000", 2024-09-13T10:12:21.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.84e-06 2024-09-13T10:12:21.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494889+0000", 2024-09-13T10:12:21.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00067531799999999995 2024-09-13T10:12:21.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494907+0000", 2024-09-13T10:12:21.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7884999999999999e-05 2024-09-13T10:12:21.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494917+0000", 2024-09-13T10:12:21.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0043e-05 2024-09-13T10:12:21.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.494971+0000", 2024-09-13T10:12:21.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4775000000000003e-05 2024-09-13T10:12:21.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.495081+0000", 2024-09-13T10:12:21.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110055 2024-09-13T10:12:21.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:21.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.495184+0000", 2024-09-13T10:12:21.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010221 2024-09-13T10:12:21.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.496618+0000", 2024-09-13T10:12:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014343590000000001 2024-09-13T10:12:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.498806+0000", 2024-09-13T10:12:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021880269999999999 2024-09-13T10:12:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.502814+0000", 2024-09-13T10:12:21.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040077890000000003 2024-09-13T10:12:21.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.502844+0000", 2024-09-13T10:12:21.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0462000000000001e-05 2024-09-13T10:12:21.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.502873+0000", 2024-09-13T10:12:21.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8657999999999999e-05 2024-09-13T10:12:21.236 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.236 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.236 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.237 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:161 4.d 4:b19b932a:::smithi19530714-41:head [write 672280~791725,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:21.237 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:50.852563+0000", 2024-09-13T10:12:21.237 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 90.375164683999998, 2024-09-13T10:12:21.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011425286999999999, 2024-09-13T10:12:21.237 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.237 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.237 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.237 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.237 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.237 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.238 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 161 2024-09-13T10:12:21.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.238 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.852563+0000", 2024-09-13T10:12:21.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.852563+0000", 2024-09-13T10:12:21.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.852567+0000", 2024-09-13T10:12:21.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1269999999999999e-06 2024-09-13T10:12:21.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853363+0000", 2024-09-13T10:12:21.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00079620899999999998 2024-09-13T10:12:21.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853394+0000", 2024-09-13T10:12:21.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1192000000000002e-05 2024-09-13T10:12:21.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853410+0000", 2024-09-13T10:12:21.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5896000000000002e-05 2024-09-13T10:12:21.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853574+0000", 2024-09-13T10:12:21.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000164358 2024-09-13T10:12:21.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.853920+0000", 2024-09-13T10:12:21.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034612500000000002 2024-09-13T10:12:21.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:12:21.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.854305+0000", 2024-09-13T10:12:21.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038458900000000001 2024-09-13T10:12:21.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.858018+0000", 2024-09-13T10:12:21.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0037128619999999999 2024-09-13T10:12:21.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.858905+0000", 2024-09-13T10:12:21.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00088737899999999999 2024-09-13T10:12:21.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.863681+0000", 2024-09-13T10:12:21.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004775311 2024-09-13T10:12:21.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.863813+0000", 2024-09-13T10:12:21.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013254900000000001 2024-09-13T10:12:21.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.863989+0000", 2024-09-13T10:12:21.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017568999999999999 2024-09-13T10:12:21.243 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.243 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.243 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.244 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(osd.14.23:4 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:21.244 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:50.996997+0000", 2024-09-13T10:12:21.244 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 90.230731277999993, 2024-09-13T10:12:21.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0083262170000000003, 2024-09-13T10:12:21.244 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.244 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.244 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:12:21.244 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.996997+0000", 2024-09-13T10:12:21.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.996997+0000", 2024-09-13T10:12:21.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997000+0000", 2024-09-13T10:12:21.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3560000000000001e-06 2024-09-13T10:12:21.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997015+0000", 2024-09-13T10:12:21.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.523e-05 2024-09-13T10:12:21.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997048+0000", 2024-09-13T10:12:21.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2162000000000001e-05 2024-09-13T10:12:21.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:50.997071+0000", 2024-09-13T10:12:21.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2918999999999999e-05 2024-09-13T10:12:21.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.001996+0000", 2024-09-13T10:12:21.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049256559999999996 2024-09-13T10:12:21.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.002198+0000", 2024-09-13T10:12:21.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020196600000000001 2024-09-13T10:12:21.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.005323+0000", 2024-09-13T10:12:21.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0031249279999999999 2024-09-13T10:12:21.248 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.248 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.248 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.248 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:261 4.c 4:3d56d753:::smithi19530714-12:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:21.248 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.215641+0000", 2024-09-13T10:12:21.248 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 90.012086960000005, 2024-09-13T10:12:21.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010509116000000001, 2024-09-13T10:12:21.248 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.248 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.249 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.249 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.249 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.249 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.249 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 261 2024-09-13T10:12:21.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.249 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215641+0000", 2024-09-13T10:12:21.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215641+0000", 2024-09-13T10:12:21.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215646+0000", 2024-09-13T10:12:21.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9300000000000002e-06 2024-09-13T10:12:21.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215666+0000", 2024-09-13T10:12:21.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0095999999999999e-05 2024-09-13T10:12:21.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215713+0000", 2024-09-13T10:12:21.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7125e-05 2024-09-13T10:12:21.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215739+0000", 2024-09-13T10:12:21.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5488999999999999e-05 2024-09-13T10:12:21.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215918+0000", 2024-09-13T10:12:21.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017948399999999999 2024-09-13T10:12:21.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216165+0000", 2024-09-13T10:12:21.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024625200000000003 2024-09-13T10:12:21.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:21.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.223467+0000", 2024-09-13T10:12:21.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0073023840000000003 2024-09-13T10:12:21.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.225020+0000", 2024-09-13T10:12:21.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015529809999999999 2024-09-13T10:12:21.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.225406+0000", 2024-09-13T10:12:21.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038614699999999999 2024-09-13T10:12:21.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.225903+0000", 2024-09-13T10:12:21.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00049721900000000002 2024-09-13T10:12:21.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.226026+0000", 2024-09-13T10:12:21.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000123163 2024-09-13T10:12:21.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.226150+0000", 2024-09-13T10:12:21.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012384599999999999 2024-09-13T10:12:21.254 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.254 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.254 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.255 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:263 4.c 4:3e6706e5:::smithi19530714-14:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:21.255 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.215802+0000", 2024-09-13T10:12:21.255 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 90.011925751999996, 2024-09-13T10:12:21.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015708573999999999, 2024-09-13T10:12:21.255 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.255 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.255 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.255 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.255 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.255 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.255 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 263 2024-09-13T10:12:21.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.256 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215802+0000", 2024-09-13T10:12:21.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215802+0000", 2024-09-13T10:12:21.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215804+0000", 2024-09-13T10:12:21.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.624e-06 2024-09-13T10:12:21.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215813+0000", 2024-09-13T10:12:21.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4159999999999993e-06 2024-09-13T10:12:21.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215832+0000", 2024-09-13T10:12:21.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8768999999999999e-05 2024-09-13T10:12:21.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215843+0000", 2024-09-13T10:12:21.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0336999999999999e-05 2024-09-13T10:12:21.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216612+0000", 2024-09-13T10:12:21.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00076899399999999995 2024-09-13T10:12:21.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216767+0000", 2024-09-13T10:12:21.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015566399999999999 2024-09-13T10:12:21.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:21.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.227138+0000", 2024-09-13T10:12:21.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010370701 2024-09-13T10:12:21.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.229299+0000", 2024-09-13T10:12:21.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021611870000000002 2024-09-13T10:12:21.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.229896+0000", 2024-09-13T10:12:21.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00059659700000000003 2024-09-13T10:12:21.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231283+0000", 2024-09-13T10:12:21.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001387133 2024-09-13T10:12:21.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231392+0000", 2024-09-13T10:12:21.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010878999999999999 2024-09-13T10:12:21.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231511+0000", 2024-09-13T10:12:21.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119362 2024-09-13T10:12:21.261 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.261 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.261 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.262 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:264 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:21.262 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.215936+0000", 2024-09-13T10:12:21.262 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 90.011791772999999, 2024-09-13T10:12:21.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.054954390999999998, 2024-09-13T10:12:21.262 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.262 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.262 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.262 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.262 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.262 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.262 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 264 2024-09-13T10:12:21.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.262 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215936+0000", 2024-09-13T10:12:21.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215936+0000", 2024-09-13T10:12:21.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215938+0000", 2024-09-13T10:12:21.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9860000000000001e-06 2024-09-13T10:12:21.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215951+0000", 2024-09-13T10:12:21.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2449999999999999e-05 2024-09-13T10:12:21.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.215982+0000", 2024-09-13T10:12:21.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1133000000000002e-05 2024-09-13T10:12:21.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.216009+0000", 2024-09-13T10:12:21.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6652000000000002e-05 2024-09-13T10:12:21.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.217072+0000", 2024-09-13T10:12:21.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010634240000000001 2024-09-13T10:12:21.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.217268+0000", 2024-09-13T10:12:21.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019641 2024-09-13T10:12:21.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:21.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.265415+0000", 2024-09-13T10:12:21.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.048146807999999999 2024-09-13T10:12:21.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.267067+0000", 2024-09-13T10:12:21.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00165211 2024-09-13T10:12:21.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.267319+0000", 2024-09-13T10:12:21.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025160500000000001 2024-09-13T10:12:21.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270591+0000", 2024-09-13T10:12:21.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0032720959999999999 2024-09-13T10:12:21.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270753+0000", 2024-09-13T10:12:21.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016215199999999999 2024-09-13T10:12:21.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270891+0000", 2024-09-13T10:12:21.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000137565 2024-09-13T10:12:21.268 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.268 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.268 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.268 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:278 4.c 4:39f17ee9:::smithi19530714-29:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:21.268 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.231988+0000", 2024-09-13T10:12:21.268 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.995739907000001, 2024-09-13T10:12:21.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011030108, 2024-09-13T10:12:21.268 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.268 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.269 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.269 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.269 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.269 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.269 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 278 2024-09-13T10:12:21.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.269 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231988+0000", 2024-09-13T10:12:21.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231988+0000", 2024-09-13T10:12:21.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.231990+0000", 2024-09-13T10:12:21.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0499999999999999e-06 2024-09-13T10:12:21.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232002+0000", 2024-09-13T10:12:21.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2189e-05 2024-09-13T10:12:21.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232029+0000", 2024-09-13T10:12:21.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6673e-05 2024-09-13T10:12:21.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232043+0000", 2024-09-13T10:12:21.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4188999999999999e-05 2024-09-13T10:12:21.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232239+0000", 2024-09-13T10:12:21.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019557 2024-09-13T10:12:21.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.232472+0000", 2024-09-13T10:12:21.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00023310900000000001 2024-09-13T10:12:21.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:21.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.237292+0000", 2024-09-13T10:12:21.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048195069999999998 2024-09-13T10:12:21.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.238595+0000", 2024-09-13T10:12:21.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013033260000000001 2024-09-13T10:12:21.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.239056+0000", 2024-09-13T10:12:21.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00046106800000000002 2024-09-13T10:12:21.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.242820+0000", 2024-09-13T10:12:21.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003763897 2024-09-13T10:12:21.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.242905+0000", 2024-09-13T10:12:21.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5580000000000004e-05 2024-09-13T10:12:21.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.243018+0000", 2024-09-13T10:12:21.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011294999999999999 2024-09-13T10:12:21.275 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.275 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.275 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.275 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:290 4.d 4:b19b932a:::smithi19530714-41:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:21.275 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.255335+0000", 2024-09-13T10:12:21.275 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.972393385000004, 2024-09-13T10:12:21.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.021335703000000001, 2024-09-13T10:12:21.276 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.276 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.276 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.276 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.276 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.276 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.276 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 290 2024-09-13T10:12:21.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.276 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255335+0000", 2024-09-13T10:12:21.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255335+0000", 2024-09-13T10:12:21.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255338+0000", 2024-09-13T10:12:21.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6940000000000001e-06 2024-09-13T10:12:21.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255353+0000", 2024-09-13T10:12:21.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4136e-05 2024-09-13T10:12:21.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255390+0000", 2024-09-13T10:12:21.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7364999999999999e-05 2024-09-13T10:12:21.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255413+0000", 2024-09-13T10:12:21.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2690000000000001e-05 2024-09-13T10:12:21.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255615+0000", 2024-09-13T10:12:21.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000202306 2024-09-13T10:12:21.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.255813+0000", 2024-09-13T10:12:21.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019769300000000001 2024-09-13T10:12:21.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:12:21.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274190+0000", 2024-09-13T10:12:21.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01837714 2024-09-13T10:12:21.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275400+0000", 2024-09-13T10:12:21.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001210275 2024-09-13T10:12:21.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276346+0000", 2024-09-13T10:12:21.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00094586699999999998 2024-09-13T10:12:21.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276497+0000", 2024-09-13T10:12:21.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015150299999999999 2024-09-13T10:12:21.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.281 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276597+0000", 2024-09-13T10:12:21.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9092999999999995e-05 2024-09-13T10:12:21.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.281 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276670+0000", 2024-09-13T10:12:21.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3941000000000002e-05 2024-09-13T10:12:21.281 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.281 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.281 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.282 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:295 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:21.282 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.261467+0000", 2024-09-13T10:12:21.282 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.966261244999998, 2024-09-13T10:12:21.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.020013564000000001, 2024-09-13T10:12:21.282 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.282 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.282 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.282 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.282 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.282 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.282 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 295 2024-09-13T10:12:21.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.282 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261467+0000", 2024-09-13T10:12:21.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261467+0000", 2024-09-13T10:12:21.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261470+0000", 2024-09-13T10:12:21.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8200000000000001e-06 2024-09-13T10:12:21.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261487+0000", 2024-09-13T10:12:21.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7646e-05 2024-09-13T10:12:21.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261523+0000", 2024-09-13T10:12:21.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5299000000000002e-05 2024-09-13T10:12:21.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261545+0000", 2024-09-13T10:12:21.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2436e-05 2024-09-13T10:12:21.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261717+0000", 2024-09-13T10:12:21.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017231000000000001 2024-09-13T10:12:21.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.261980+0000", 2024-09-13T10:12:21.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000262959 2024-09-13T10:12:21.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:12:21.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277104+0000", 2024-09-13T10:12:21.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.015123983000000001 2024-09-13T10:12:21.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.279042+0000", 2024-09-13T10:12:21.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001937338 2024-09-13T10:12:21.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.279396+0000", 2024-09-13T10:12:21.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00035408199999999998 2024-09-13T10:12:21.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281302+0000", 2024-09-13T10:12:21.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001906346 2024-09-13T10:12:21.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281400+0000", 2024-09-13T10:12:21.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7528000000000005e-05 2024-09-13T10:12:21.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.281480+0000", 2024-09-13T10:12:21.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0816999999999996e-05 2024-09-13T10:12:21.288 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.288 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.288 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.288 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:297 4.c 4:381e677b:::smithi19530714-48:head [assert-version v15,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:21.288 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.262493+0000", 2024-09-13T10:12:21.288 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.965235075999999, 2024-09-13T10:12:21.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.012458446, 2024-09-13T10:12:21.289 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.289 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.289 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.289 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.289 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.289 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.289 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 297 2024-09-13T10:12:21.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.289 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262493+0000", 2024-09-13T10:12:21.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.290 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.290 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262493+0000", 2024-09-13T10:12:21.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.290 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.290 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262496+0000", 2024-09-13T10:12:21.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4729999999999999e-06 2024-09-13T10:12:21.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.290 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.290 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262510+0000", 2024-09-13T10:12:21.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4205999999999999e-05 2024-09-13T10:12:21.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262545+0000", 2024-09-13T10:12:21.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4789000000000002e-05 2024-09-13T10:12:21.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262566+0000", 2024-09-13T10:12:21.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1789000000000001e-05 2024-09-13T10:12:21.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262748+0000", 2024-09-13T10:12:21.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000181848 2024-09-13T10:12:21.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.262986+0000", 2024-09-13T10:12:21.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00023747500000000001 2024-09-13T10:12:21.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:21.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.267794+0000", 2024-09-13T10:12:21.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0048086479999999996 2024-09-13T10:12:21.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.269878+0000", 2024-09-13T10:12:21.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0020833900000000001 2024-09-13T10:12:21.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.270928+0000", 2024-09-13T10:12:21.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001049901 2024-09-13T10:12:21.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274724+0000", 2024-09-13T10:12:21.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0037964880000000002 2024-09-13T10:12:21.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274862+0000", 2024-09-13T10:12:21.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000138286 2024-09-13T10:12:21.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.274952+0000", 2024-09-13T10:12:21.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9153000000000005e-05 2024-09-13T10:12:21.294 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.295 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.295 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.295 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.295 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:313 4.c 4:3e6706e5:::smithi19530714-14:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:21.295 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.275533+0000", 2024-09-13T10:12:21.295 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.952194859000002, 2024-09-13T10:12:21.295 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.019582932000000001, 2024-09-13T10:12:21.295 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.295 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.295 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.295 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.296 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.296 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.296 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 313 2024-09-13T10:12:21.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.296 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275533+0000", 2024-09-13T10:12:21.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275533+0000", 2024-09-13T10:12:21.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275536+0000", 2024-09-13T10:12:21.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4899999999999999e-06 2024-09-13T10:12:21.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275549+0000", 2024-09-13T10:12:21.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3699e-05 2024-09-13T10:12:21.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275574+0000", 2024-09-13T10:12:21.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4586000000000001e-05 2024-09-13T10:12:21.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275588+0000", 2024-09-13T10:12:21.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3746e-05 2024-09-13T10:12:21.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275705+0000", 2024-09-13T10:12:21.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000117352 2024-09-13T10:12:21.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.299 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.299 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.275898+0000", 2024-09-13T10:12:21.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019293000000000001 2024-09-13T10:12:21.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.299 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:21.299 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286176+0000", 2024-09-13T10:12:21.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010277578000000001 2024-09-13T10:12:21.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.299 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.299 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.287526+0000", 2024-09-13T10:12:21.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013505349999999999 2024-09-13T10:12:21.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.300 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.288025+0000", 2024-09-13T10:12:21.300 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00049897500000000003 2024-09-13T10:12:21.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.300 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294958+0000", 2024-09-13T10:12:21.300 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069327980000000004 2024-09-13T10:12:21.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.300 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.295047+0000", 2024-09-13T10:12:21.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9154999999999996e-05 2024-09-13T10:12:21.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.295116+0000", 2024-09-13T10:12:21.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9088e-05 2024-09-13T10:12:21.301 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.301 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.301 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.301 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:314 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:21.301 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.276956+0000", 2024-09-13T10:12:21.301 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.950772180000001, 2024-09-13T10:12:21.302 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017940078000000002, 2024-09-13T10:12:21.302 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.302 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.302 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.302 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.302 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.302 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.302 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 314 2024-09-13T10:12:21.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.302 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.302 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.302 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.302 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276956+0000", 2024-09-13T10:12:21.302 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.303 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.303 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.303 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276956+0000", 2024-09-13T10:12:21.303 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.303 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.303 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.303 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276959+0000", 2024-09-13T10:12:21.303 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5730000000000002e-06 2024-09-13T10:12:21.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.303 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.303 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.304 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276971+0000", 2024-09-13T10:12:21.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2680000000000001e-05 2024-09-13T10:12:21.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.304 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.276997+0000", 2024-09-13T10:12:21.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5579e-05 2024-09-13T10:12:21.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.304 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277018+0000", 2024-09-13T10:12:21.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1409000000000001e-05 2024-09-13T10:12:21.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277136+0000", 2024-09-13T10:12:21.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000117353 2024-09-13T10:12:21.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.277237+0000", 2024-09-13T10:12:21.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101412 2024-09-13T10:12:21.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:21.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.285332+0000", 2024-09-13T10:12:21.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080951770000000003 2024-09-13T10:12:21.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.287120+0000", 2024-09-13T10:12:21.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001787828 2024-09-13T10:12:21.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.287200+0000", 2024-09-13T10:12:21.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0279999999999997e-05 2024-09-13T10:12:21.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294808+0000", 2024-09-13T10:12:21.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0076072400000000004 2024-09-13T10:12:21.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.307 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.307 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.307 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294853+0000", 2024-09-13T10:12:21.307 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5438000000000001e-05 2024-09-13T10:12:21.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.307 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.307 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.307 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294896+0000", 2024-09-13T10:12:21.307 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3108999999999997e-05 2024-09-13T10:12:21.307 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.307 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.307 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.308 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.308 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:340 4.d 4:b19b932a:::smithi19530714-41:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:21.308 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.284107+0000", 2024-09-13T10:12:21.308 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.943620949999996, 2024-09-13T10:12:21.308 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010293416, 2024-09-13T10:12:21.308 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.308 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.308 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.308 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.308 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.309 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.309 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 340 2024-09-13T10:12:21.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.309 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.309 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.309 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.309 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284107+0000", 2024-09-13T10:12:21.309 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.309 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.309 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.309 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284107+0000", 2024-09-13T10:12:21.309 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.310 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.310 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.310 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284109+0000", 2024-09-13T10:12:21.310 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.007e-06 2024-09-13T10:12:21.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.310 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.310 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.310 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284120+0000", 2024-09-13T10:12:21.310 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0346999999999999e-05 2024-09-13T10:12:21.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.310 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.310 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.310 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284144+0000", 2024-09-13T10:12:21.311 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3998999999999998e-05 2024-09-13T10:12:21.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.311 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284157+0000", 2024-09-13T10:12:21.311 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3045e-05 2024-09-13T10:12:21.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.311 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284244+0000", 2024-09-13T10:12:21.311 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7882999999999999e-05 2024-09-13T10:12:21.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.284360+0000", 2024-09-13T10:12:21.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115467 2024-09-13T10:12:21.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.312 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.312 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:12:21.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.292368+0000", 2024-09-13T10:12:21.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080080380000000003 2024-09-13T10:12:21.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.312 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.312 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.293486+0000", 2024-09-13T10:12:21.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001117776 2024-09-13T10:12:21.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.313 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.313 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.313 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294261+0000", 2024-09-13T10:12:21.313 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00077481100000000003 2024-09-13T10:12:21.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.313 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.313 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.313 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294309+0000", 2024-09-13T10:12:21.313 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8086e-05 2024-09-13T10:12:21.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.313 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.314 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.314 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294358+0000", 2024-09-13T10:12:21.314 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8853999999999997e-05 2024-09-13T10:12:21.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.314 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.314 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.314 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294401+0000", 2024-09-13T10:12:21.314 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3102999999999997e-05 2024-09-13T10:12:21.314 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.314 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.314 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.314 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.315 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:345 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:21.315 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.286309+0000", 2024-09-13T10:12:21.315 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.941418772000006, 2024-09-13T10:12:21.315 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010478568000000001, 2024-09-13T10:12:21.315 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.315 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.315 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.315 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.315 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.315 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.315 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 345 2024-09-13T10:12:21.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.315 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.315 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.316 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.316 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286309+0000", 2024-09-13T10:12:21.316 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.316 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.316 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.316 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286309+0000", 2024-09-13T10:12:21.316 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.316 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.316 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.316 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286311+0000", 2024-09-13T10:12:21.316 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9230000000000001e-06 2024-09-13T10:12:21.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.317 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.317 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.317 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286320+0000", 2024-09-13T10:12:21.317 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8370000000000003e-06 2024-09-13T10:12:21.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.317 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.317 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.317 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286342+0000", 2024-09-13T10:12:21.317 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1446000000000001e-05 2024-09-13T10:12:21.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.318 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.318 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.318 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286385+0000", 2024-09-13T10:12:21.318 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3083000000000001e-05 2024-09-13T10:12:21.318 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.318 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.318 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.318 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286549+0000", 2024-09-13T10:12:21.318 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016463 2024-09-13T10:12:21.318 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.318 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.318 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.318 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.286739+0000", 2024-09-13T10:12:21.319 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018965400000000001 2024-09-13T10:12:21.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.319 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:12:21.319 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.294808+0000", 2024-09-13T10:12:21.319 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0080692609999999994 2024-09-13T10:12:21.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.319 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.319 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.295388+0000", 2024-09-13T10:12:21.319 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00057994000000000003 2024-09-13T10:12:21.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296321+0000", 2024-09-13T10:12:21.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00093279099999999998 2024-09-13T10:12:21.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296622+0000", 2024-09-13T10:12:21.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030084300000000002 2024-09-13T10:12:21.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296702+0000", 2024-09-13T10:12:21.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0547e-05 2024-09-13T10:12:21.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.321 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.321 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.296788+0000", 2024-09-13T10:12:21.321 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5612999999999995e-05 2024-09-13T10:12:21.321 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.321 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.321 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.321 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:364 4.c 4:36d575b1:::smithi19530714-15:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:12:21.321 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.297175+0000", 2024-09-13T10:12:21.321 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.930553595999996, 2024-09-13T10:12:21.322 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0085499329999999991, 2024-09-13T10:12:21.322 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.322 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.322 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.322 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.322 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.322 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.322 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 364 2024-09-13T10:12:21.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.322 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.322 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.322 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.322 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297175+0000", 2024-09-13T10:12:21.323 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.323 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.323 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.323 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297175+0000", 2024-09-13T10:12:21.323 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.323 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.323 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.323 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297176+0000", 2024-09-13T10:12:21.323 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9199999999999999e-07 2024-09-13T10:12:21.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.323 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.324 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297181+0000", 2024-09-13T10:12:21.324 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5149999999999997e-06 2024-09-13T10:12:21.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.324 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.324 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297191+0000", 2024-09-13T10:12:21.324 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8260000000000005e-06 2024-09-13T10:12:21.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.324 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.324 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297198+0000", 2024-09-13T10:12:21.325 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6950000000000003e-06 2024-09-13T10:12:21.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.325 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.325 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.325 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297244+0000", 2024-09-13T10:12:21.325 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6397999999999999e-05 2024-09-13T10:12:21.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.325 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.325 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.325 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.297291+0000", 2024-09-13T10:12:21.325 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7342999999999997e-05 2024-09-13T10:12:21.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.326 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.326 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:21.326 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.303658+0000", 2024-09-13T10:12:21.326 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0063669 2024-09-13T10:12:21.326 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.326 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.326 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.326 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.304437+0000", 2024-09-13T10:12:21.326 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00077861399999999996 2024-09-13T10:12:21.326 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.326 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.326 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.326 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.304902+0000", 2024-09-13T10:12:21.327 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00046505899999999998 2024-09-13T10:12:21.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.327 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.327 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.305534+0000", 2024-09-13T10:12:21.327 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00063224199999999998 2024-09-13T10:12:21.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.327 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.327 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.305643+0000", 2024-09-13T10:12:21.327 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108384 2024-09-13T10:12:21.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.328 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.328 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.305725+0000", 2024-09-13T10:12:21.328 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1965000000000006e-05 2024-09-13T10:12:21.328 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.328 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.328 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.328 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.328 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:378 4.c 4:39f17ee9:::smithi19530714-29:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:12:21.328 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.320710+0000", 2024-09-13T10:12:21.328 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.907018225000002, 2024-09-13T10:12:21.328 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0079049020000000001, 2024-09-13T10:12:21.328 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.329 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.329 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.329 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.329 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.329 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.329 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 378 2024-09-13T10:12:21.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.329 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.329 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.329 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.329 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320710+0000", 2024-09-13T10:12:21.330 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.330 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.330 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.330 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320710+0000", 2024-09-13T10:12:21.330 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.330 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.330 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.330 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320712+0000", 2024-09-13T10:12:21.330 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4099999999999998e-06 2024-09-13T10:12:21.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.330 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.331 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.331 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320724+0000", 2024-09-13T10:12:21.331 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.134e-05 2024-09-13T10:12:21.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.331 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.331 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.331 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320746+0000", 2024-09-13T10:12:21.331 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2342999999999999e-05 2024-09-13T10:12:21.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.331 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.331 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.331 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320758+0000", 2024-09-13T10:12:21.331 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1557e-05 2024-09-13T10:12:21.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.332 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.332 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.332 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320840+0000", 2024-09-13T10:12:21.332 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2274000000000005e-05 2024-09-13T10:12:21.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.332 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.332 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.332 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.320933+0000", 2024-09-13T10:12:21.332 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3375999999999995e-05 2024-09-13T10:12:21.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.333 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.333 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:21.333 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.326291+0000", 2024-09-13T10:12:21.333 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053575089999999999 2024-09-13T10:12:21.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.333 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.333 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.333 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.327254+0000", 2024-09-13T10:12:21.333 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00096337099999999998 2024-09-13T10:12:21.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.333 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.333 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.333 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.327902+0000", 2024-09-13T10:12:21.333 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00064762100000000003 2024-09-13T10:12:21.334 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.334 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.334 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.334 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328501+0000", 2024-09-13T10:12:21.334 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000599232 2024-09-13T10:12:21.334 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.334 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.334 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.334 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328547+0000", 2024-09-13T10:12:21.334 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5973000000000002e-05 2024-09-13T10:12:21.334 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.334 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.334 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.334 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328615+0000", 2024-09-13T10:12:21.334 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7896000000000002e-05 2024-09-13T10:12:21.335 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.335 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.335 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.335 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.335 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:395 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:12:21.335 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.322105+0000", 2024-09-13T10:12:21.335 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.905623152999993, 2024-09-13T10:12:21.335 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0084827089999999997, 2024-09-13T10:12:21.335 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.335 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.335 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.335 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.336 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.336 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.336 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 395 2024-09-13T10:12:21.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.336 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.336 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.336 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.336 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322105+0000", 2024-09-13T10:12:21.336 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.336 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.336 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322105+0000", 2024-09-13T10:12:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322106+0000", 2024-09-13T10:12:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2099999999999995e-07 2024-09-13T10:12:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322110+0000", 2024-09-13T10:12:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.827e-06 2024-09-13T10:12:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322120+0000", 2024-09-13T10:12:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0227999999999999e-05 2024-09-13T10:12:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322160+0000", 2024-09-13T10:12:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9941999999999998e-05 2024-09-13T10:12:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322232+0000", 2024-09-13T10:12:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2451000000000001e-05 2024-09-13T10:12:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.322314+0000", 2024-09-13T10:12:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2089000000000006e-05 2024-09-13T10:12:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:12:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.328374+0000", 2024-09-13T10:12:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0060591350000000002 2024-09-13T10:12:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.329439+0000", 2024-09-13T10:12:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010654830000000001 2024-09-13T10:12:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330219+0000", 2024-09-13T10:12:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00078003700000000003 2024-09-13T10:12:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330488+0000", 2024-09-13T10:12:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000269411 2024-09-13T10:12:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.340 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330544+0000", 2024-09-13T10:12:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5704000000000001e-05 2024-09-13T10:12:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.330588+0000", 2024-09-13T10:12:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3581000000000002e-05 2024-09-13T10:12:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.341 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:595 4.d 4:ba7d2bc5:::smithi19530714-46:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:12:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.370445+0000", 2024-09-13T10:12:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.857283119000002, 2024-09-13T10:12:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0084586149999999992, 2024-09-13T10:12:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.342 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 595 2024-09-13T10:12:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370445+0000", 2024-09-13T10:12:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370445+0000", 2024-09-13T10:12:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370447+0000", 2024-09-13T10:12:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.66e-06 2024-09-13T10:12:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370455+0000", 2024-09-13T10:12:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3059999999999998e-06 2024-09-13T10:12:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.344 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370472+0000", 2024-09-13T10:12:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7479999999999999e-05 2024-09-13T10:12:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370484+0000", 2024-09-13T10:12:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1882000000000001e-05 2024-09-13T10:12:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370562+0000", 2024-09-13T10:12:21.345 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7531e-05 2024-09-13T10:12:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.370738+0000", 2024-09-13T10:12:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017587200000000001 2024-09-13T10:12:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 7,15", 2024-09-13T10:12:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.377463+0000", 2024-09-13T10:12:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0067247829999999998 2024-09-13T10:12:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.346 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378047+0000", 2024-09-13T10:12:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00058493100000000001 2024-09-13T10:12:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378730+0000", 2024-09-13T10:12:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00068235799999999999 2024-09-13T10:12:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378840+0000", 2024-09-13T10:12:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110296 2024-09-13T10:12:21.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378875+0000", 2024-09-13T10:12:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4552999999999999e-05 2024-09-13T10:12:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.378904+0000", 2024-09-13T10:12:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8963e-05 2024-09-13T10:12:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:782 4.c 4:39f17ee9:::smithi19530714-29:head [write 1543557~550004] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:12:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:51.601833+0000", 2024-09-13T10:12:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 89.625895244999995, 2024-09-13T10:12:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081663250000000003, 2024-09-13T10:12:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 782 2024-09-13T10:12:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.349 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.601833+0000", 2024-09-13T10:12:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.601833+0000", 2024-09-13T10:12:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.601835+0000", 2024-09-13T10:12:21.350 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.576e-06 2024-09-13T10:12:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602350+0000", 2024-09-13T10:12:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00051449700000000004 2024-09-13T10:12:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602379+0000", 2024-09-13T10:12:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8598999999999998e-05 2024-09-13T10:12:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.351 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602394+0000", 2024-09-13T10:12:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4952e-05 2024-09-13T10:12:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602534+0000", 2024-09-13T10:12:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014037899999999999 2024-09-13T10:12:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.602796+0000", 2024-09-13T10:12:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000262415 2024-09-13T10:12:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.352 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for subops from 3,6", 2024-09-13T10:12:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.603214+0000", 2024-09-13T10:12:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00041785599999999998 2024-09-13T10:12:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.606389+0000", 2024-09-13T10:12:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0031751549999999998 2024-09-13T10:12:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op_commit", 2024-09-13T10:12:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.608306+0000", 2024-09-13T10:12:21.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0019163540000000001 2024-09-13T10:12:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.609577+0000", 2024-09-13T10:12:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001271466 2024-09-13T10:12:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-09-13T10:12:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.609872+0000", 2024-09-13T10:12:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00029501400000000002 2024-09-13T10:12:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.354 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:51.609999+0000", 2024-09-13T10:12:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000127062 2024-09-13T10:12:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4607.0:1085 4.c 4:3e6706e5:::smithi19530714-14:head [sparse-read 0~906036] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-13T10:10:56.648267+0000", 2024-09-13T10:12:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 84.579460721999993, 2024-09-13T10:12:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0083262089999999993, 2024-09-13T10:12:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-13T10:12:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-13T10:12:21.355 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-13T10:12:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-09-13T10:12:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4607", 2024-09-13T10:12:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1085 2024-09-13T10:12:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-13T10:12:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-13T10:12:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648267+0000", 2024-09-13T10:12:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-13T10:12:21.356 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648267+0000", 2024-09-13T10:12:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-13T10:12:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-13T10:12:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648269+0000", 2024-09-13T10:12:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1769999999999999e-06 2024-09-13T10:12:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-13T10:12:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648276+0000", 2024-09-13T10:12:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0530000000000001e-06 2024-09-13T10:12:21.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-13T10:12:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648292+0000", 2024-09-13T10:12:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6209e-05 2024-09-13T10:12:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-13T10:12:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.648307+0000", 2024-09-13T10:12:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4647000000000001e-05 2024-09-13T10:12:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-13T10:12:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.655806+0000", 2024-09-13T10:12:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0074996500000000001 2024-09-13T10:12:21.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-13T10:12:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.656060+0000", 2024-09-13T10:12:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000253639 2024-09-13T10:12:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-13T10:12:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-13T10:12:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-09-13T10:12:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-13T10:10:56.656594+0000", 2024-09-13T10:12:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00053383399999999996 2024-09-13T10:12:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-13T10:12:21.359 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-13T10:12:21.360 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:12:21.361 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T10:12:21.362 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:21.357+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:21.425 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:12:21.436 INFO:tasks.rados.rados.0.smithi195.stdout:1956: expect (ObjNum 425 snap 0 seq_num 425) 2024-09-13T10:12:21.436 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:21.462 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:21.457+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:21.493 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:21.493 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:12:21.493 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:12:21.493 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:21.507 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T10:12:21.544 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2841293 2024-09-13T10:12:21.563 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:21.557+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:21.575 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:12:21.575 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:12:21.593 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:12:21.593 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:12:21.614 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:12:21.614 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:12:21.615 INFO:tasks.rados.rados.0.smithi195.stdout:1959: finishing write tid 1 to smithi19530714-30 2024-09-13T10:12:21.615 INFO:tasks.rados.rados.0.smithi195.stdout:1959: finishing write tid 2 to smithi19530714-30 2024-09-13T10:12:21.615 INFO:tasks.rados.rados.0.smithi195.stdout:1959: finishing write tid 3 to smithi19530714-30 2024-09-13T10:12:21.615 INFO:tasks.rados.rados.0.smithi195.stdout:1959: finishing write tid 4 to smithi19530714-30 2024-09-13T10:12:21.615 INFO:tasks.rados.rados.0.smithi195.stdout:1959: finishing write tid 5 to smithi19530714-30 2024-09-13T10:12:21.615 INFO:tasks.rados.rados.0.smithi195.stdout:1959: oid 30 updating version 0 to 51 2024-09-13T10:12:21.615 INFO:tasks.rados.rados.0.smithi195.stdout:1959: oid 30 updating version 51 to 52 2024-09-13T10:12:21.616 INFO:tasks.rados.rados.0.smithi195.stdout:1959: oid 30 version 52 is already newer than 49 2024-09-13T10:12:21.616 INFO:tasks.rados.rados.0.smithi195.stdout:1959: oid 30 version 52 is already newer than 50 2024-09-13T10:12:21.616 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 30 v 52 (ObjNum 439 snap 0 seq_num 439) dirty exists 2024-09-13T10:12:21.616 INFO:tasks.rados.rados.0.smithi195.stdout:1959: left oid 30 (ObjNum 439 snap 0 seq_num 439) 2024-09-13T10:12:21.616 INFO:tasks.rados.rados.0.smithi195.stdout:1960: expect (ObjNum 431 snap 0 seq_num 431) 2024-09-13T10:12:21.633 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:12:21.633 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:12:21.635 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:21.636 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:21.650 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T10:12:21.650 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:12:21.651 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:12:21.663 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:12:21.657+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:12:21.669 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:12:21.669 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:12:21.691 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:12:21.691 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:12:21.711 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:12:21.712 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:12:21.732 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:12:21.732 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:12:21.753 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:12:21.753 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:12:21.757 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:21.763 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:21.757+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:21.775 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:12:21.776 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:12:21.781 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T10:12:21.797 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:12:21.797 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:12:21.816 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:12:21.817 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:12:21.829 INFO:tasks.rados.rados.0.smithi195.stdout:1962: expect (ObjNum 412 snap 0 seq_num 412) 2024-09-13T10:12:21.829 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:21.837 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:12:21.837 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:12:21.859 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:12:21.859 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:12:21.864 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:21.858+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:21.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:12:21.870+0000 7f84dec53640 0 -- 172.21.15.73:0/534198228 >> v2:172.21.15.195:6805/848686366 conn(0x7f84d8227d70 msgr2=0x7f84d822a160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2024-09-13T10:12:21.882 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:12:21.882 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:12:21.905 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:21.905 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:21.920 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T10:12:21.965 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:21.961+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:22.005 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:22.040 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:22.040 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:22.054 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T10:12:22.066 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:22.061+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:22.093 INFO:tasks.rados.rados.0.smithi195.stdout:1963: got (0) Success 2024-09-13T10:12:22.093 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 5 v 0 (ObjNum 434 snap 0 seq_num 434) dirty exists 2024-09-13T10:12:22.093 INFO:tasks.rados.rados.0.smithi195.stdout:1955: done (7 left) 2024-09-13T10:12:22.093 INFO:tasks.rados.rados.0.smithi195.stdout:1956: done (6 left) 2024-09-13T10:12:22.093 INFO:tasks.rados.rados.0.smithi195.stdout:1957: done (5 left) 2024-09-13T10:12:22.093 INFO:tasks.rados.rados.0.smithi195.stdout:1959: done (4 left) 2024-09-13T10:12:22.093 INFO:tasks.rados.rados.0.smithi195.stdout:1960: done (3 left) 2024-09-13T10:12:22.093 INFO:tasks.rados.rados.0.smithi195.stdout:1961: done (2 left) 2024-09-13T10:12:22.093 INFO:tasks.rados.rados.0.smithi195.stdout:1962: done (1 left) 2024-09-13T10:12:22.094 INFO:tasks.rados.rados.0.smithi195.stdout:1963: done (0 left) 2024-09-13T10:12:22.094 INFO:tasks.rados.rados.0.smithi195.stdout:1964: read oid 40 snap -1 2024-09-13T10:12:22.094 INFO:tasks.rados.rados.0.smithi195.stdout:1964: expect (ObjNum 431 snap 0 seq_num 431) 2024-09-13T10:12:22.094 INFO:tasks.rados.rados.0.smithi195.stdout:1965: set_chunk oid 26 target oid 2024-09-13T10:12:22.100 INFO:tasks.rados.rados.0.smithi195.stdout:1965: set_chunk oid 26 offset: 8704 length: 178176 target oid a6da8bfb471cea14092bdab0e7edbc8706e3c4b8b1cd47a4b4f1a1dfbb814a77 offset: 0 snap -1 2024-09-13T10:12:22.100 INFO:tasks.rados.rados.0.smithi195.stdout:1966: delete oid 36 current snap is 0 2024-09-13T10:12:22.103 INFO:tasks.rados.rados.0.smithi195.stdout:1964: expect (ObjNum 431 snap 0 seq_num 431) 2024-09-13T10:12:22.167 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:22.165+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:22.179 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:22.179 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:22.193 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1571788 2024-09-13T10:12:22.195 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-13T10:12:22.242 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:22.268 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:22.265+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:22.314 INFO:tasks.rados.rados.0.smithi195.stdout:1965: finishing set_chunk to oid 26 2024-09-13T10:12:22.314 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 26 v 53 (ObjNum 379 snap 0 seq_num 379) dirty exists 2024-09-13T10:12:22.314 INFO:tasks.rados.rados.0.smithi195.stdout:1964: done (2 left) 2024-09-13T10:12:22.314 INFO:tasks.rados.rados.0.smithi195.stdout:1965: done (1 left) 2024-09-13T10:12:22.314 INFO:tasks.rados.rados.0.smithi195.stdout:1966: done (0 left) 2024-09-13T10:12:22.315 INFO:tasks.rados.rados.0.smithi195.stdout:1967: read oid 19 snap -1 2024-09-13T10:12:22.315 INFO:tasks.rados.rados.0.smithi195.stdout:1967: expect (ObjNum 433 snap 0 seq_num 433) 2024-09-13T10:12:22.315 INFO:tasks.rados.rados.0.smithi195.stdout:1968: read oid 37 snap -1 2024-09-13T10:12:22.315 INFO:tasks.rados.rados.0.smithi195.stdout:1968: expect (ObjNum 438 snap 0 seq_num 438) 2024-09-13T10:12:22.315 INFO:tasks.rados.rados.0.smithi195.stdout:1969: read oid 3 snap -1 2024-09-13T10:12:22.315 INFO:tasks.rados.rados.0.smithi195.stdout:1969: expect (ObjNum 328 snap 0 seq_num 328) 2024-09-13T10:12:22.315 INFO:tasks.rados.rados.0.smithi195.stdout:1970: write (excl) oid 38 current snap is 0 2024-09-13T10:12:22.315 INFO:tasks.rados.rados.0.smithi195.stdout:1970: seq_num 441 ranges {611887=733283,1972912=456484,2894128=517759} 2024-09-13T10:12:22.317 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:22.317 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:22.332 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-13T10:12:22.335 INFO:tasks.rados.rados.0.smithi195.stdout:1970: writing smithi19530714-38 from 611887 to 1345170 tid 1 2024-09-13T10:12:22.353 INFO:tasks.rados.rados.0.smithi195.stdout:1970: writing smithi19530714-38 from 1972912 to 2429396 tid 2 2024-09-13T10:12:22.367 INFO:tasks.rados.rados.0.smithi195.stdout:1970: writing smithi19530714-38 from 2894128 to 3411887 tid 3 2024-09-13T10:12:22.368 INFO:tasks.rados.rados.0.smithi195.stdout:1971: delete oid 15 current snap is 0 2024-09-13T10:12:22.368 INFO:tasks.rados.rados.0.smithi195.stdout:1968: expect (ObjNum 438 snap 0 seq_num 438) 2024-09-13T10:12:22.369 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-09-13T10:12:22.366+0000 7f320de95640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 26448) UID: 0 2024-09-13T10:12:22.406 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3223069 2024-09-13T10:12:22.456 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:22.456 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:22.470 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:22.465+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:22.471 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-13T10:12:22.473 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2103130 2024-09-13T10:12:22.489 INFO:tasks.rados.rados.0.smithi195.stdout:1967: expect (ObjNum 433 snap 0 seq_num 433) 2024-09-13T10:12:22.570 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:22.565+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:22.587 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:22.587 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:22.601 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-13T10:12:22.613 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1776896 2024-09-13T10:12:22.671 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:22.670+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:22.694 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1606504 2024-09-13T10:12:22.729 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:22.729 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:22.745 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-13T10:12:22.746 INFO:tasks.rados.rados.0.smithi195.stdout:1969: expect (ObjNum 328 snap 0 seq_num 328) 2024-09-13T10:12:22.795 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:12:22.794+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:12:22.869 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:22.869 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:22.873 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:22.869+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:22.883 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-13T10:12:22.894 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:22.910 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T10:12:22.964 INFO:tasks.rados.rados.0.smithi195.stdout:1970: finishing write tid 1 to smithi19530714-38 2024-09-13T10:12:22.965 INFO:tasks.rados.rados.0.smithi195.stdout:1970: finishing write tid 2 to smithi19530714-38 2024-09-13T10:12:22.965 INFO:tasks.rados.rados.0.smithi195.stdout:1970: finishing write tid 3 to smithi19530714-38 2024-09-13T10:12:22.965 INFO:tasks.rados.rados.0.smithi195.stdout:1967: done (4 left) 2024-09-13T10:12:22.965 INFO:tasks.rados.rados.0.smithi195.stdout:1968: done (3 left) 2024-09-13T10:12:22.965 INFO:tasks.rados.rados.0.smithi195.stdout:1969: done (2 left) 2024-09-13T10:12:22.965 INFO:tasks.rados.rados.0.smithi195.stdout:1971: done (1 left) 2024-09-13T10:12:22.965 INFO:tasks.rados.rados.0.smithi195.stdout:1972: read oid 43 snap -1 2024-09-13T10:12:22.965 INFO:tasks.rados.rados.0.smithi195.stdout:1972: expect (ObjNum 429 snap 0 seq_num 429) 2024-09-13T10:12:22.965 INFO:tasks.rados.rados.0.smithi195.stdout:1973: read oid 49 snap -1 2024-09-13T10:12:22.965 INFO:tasks.rados.rados.0.smithi195.stdout:1973: expect (ObjNum 413 snap 0 seq_num 413) 2024-09-13T10:12:22.965 INFO:tasks.rados.rados.0.smithi195.stdout:1974: set_chunk oid 1 target oid 2024-09-13T10:12:22.975 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:22.974+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:23.005 INFO:tasks.rados.rados.0.smithi195.stdout:1974: set_chunk oid 1 offset: 1019904 length: 1762304 target oid 563c2b3ed33cda263b98c76c80dd1315106291eddeff2d4114d11777e3d50d69 offset: 0 snap -1 2024-09-13T10:12:23.005 INFO:tasks.rados.rados.0.smithi195.stdout:1975: read oid 25 snap -1 2024-09-13T10:12:23.005 INFO:tasks.rados.rados.0.smithi195.stdout:1975: expect (ObjNum 324 snap 0 seq_num 324) 2024-09-13T10:12:23.006 INFO:tasks.rados.rados.0.smithi195.stdout:1976: tier_evict oid 30 2024-09-13T10:12:23.006 INFO:tasks.rados.rados.0.smithi195.stdout:1976: tier_evict oid 30 snap -1 2024-09-13T10:12:23.006 INFO:tasks.rados.rados.0.smithi195.stdout:1970: finishing write tid 4 to smithi19530714-38 2024-09-13T10:12:23.006 INFO:tasks.rados.rados.0.smithi195.stdout:1970: finishing write tid 5 to smithi19530714-38 2024-09-13T10:12:23.006 INFO:tasks.rados.rados.0.smithi195.stdout:1970: oid 38 updating version 0 to 199 2024-09-13T10:12:23.006 INFO:tasks.rados.rados.0.smithi195.stdout:1970: oid 38 version 199 is already newer than 198 2024-09-13T10:12:23.006 INFO:tasks.rados.rados.0.smithi195.stdout:1970: oid 38 updating version 199 to 200 2024-09-13T10:12:23.006 INFO:tasks.rados.rados.0.smithi195.stdout:1970: oid 38 version 200 is already newer than 197 2024-09-13T10:12:23.006 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 38 v 200 (ObjNum 441 snap 0 seq_num 441) dirty exists 2024-09-13T10:12:23.006 INFO:tasks.rados.rados.0.smithi195.stdout:1970: left oid 38 (ObjNum 441 snap 0 seq_num 441) 2024-09-13T10:12:23.006 INFO:tasks.rados.rados.0.smithi195.stdout:1970: done (5 left) 2024-09-13T10:12:23.006 INFO:tasks.rados.rados.0.smithi195.stdout:1977: tier_evict oid 13 2024-09-13T10:12:23.006 INFO:tasks.rados.rados.0.smithi195.stdout:1977: tier_evict oid 13 snap -1 2024-09-13T10:12:23.006 INFO:tasks.rados.rados.0.smithi195.stdout:1972: expect (ObjNum 429 snap 0 seq_num 429) 2024-09-13T10:12:23.043 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2315707 2024-09-13T10:12:23.055 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:23.056 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:23.070 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T10:12:23.117 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:12:23.113+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:12:23.138 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1340374 2024-09-13T10:12:23.175 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:12:23.174+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:12:23.183 INFO:tasks.rados.rados.0.smithi195.stdout:1973: expect (ObjNum 413 snap 0 seq_num 413) 2024-09-13T10:12:23.202 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:23.202 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:23.216 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T10:12:23.259 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1587614 2024-09-13T10:12:23.276 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:23.274+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:23.351 INFO:tasks.rados.rados.0.smithi195.stdout:1974: finishing set_chunk to oid 1 2024-09-13T10:12:23.351 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 1 set_chunk returned error code -95 offset: 1019904 length: 1762304 2024-09-13T10:12:23.351 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 1 v 0 (ObjNum 405 snap 0 seq_num 405) dirty exists 2024-09-13T10:12:23.351 INFO:tasks.rados.rados.0.smithi195.stdout:1972: done (5 left) 2024-09-13T10:12:23.351 INFO:tasks.rados.rados.0.smithi195.stdout:1973: done (4 left) 2024-09-13T10:12:23.351 INFO:tasks.rados.rados.0.smithi195.stdout:1974: done (3 left) 2024-09-13T10:12:23.351 INFO:tasks.rados.rados.0.smithi195.stdout:1978: read oid 21 snap -1 2024-09-13T10:12:23.351 INFO:tasks.rados.rados.0.smithi195.stdout:1978: expect (ObjNum 421 snap 0 seq_num 421) 2024-09-13T10:12:23.351 INFO:tasks.rados.rados.0.smithi195.stdout:1979: write (excl) oid 49 current snap is 0 2024-09-13T10:12:23.351 INFO:tasks.rados.rados.0.smithi195.stdout:1976: got (22) Invalid argument 2024-09-13T10:12:23.352 INFO:tasks.rados.rados.0.smithi195.stdout:1977: got (22) Invalid argument 2024-09-13T10:12:23.352 INFO:tasks.rados.rados.0.smithi195.stdout:1975: expect (ObjNum 324 snap 0 seq_num 324) 2024-09-13T10:12:23.352 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:23.377 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:23.373+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:23.378 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:23.378 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:23.393 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T10:12:23.419 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 344716 2024-09-13T10:12:23.477 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:23.473+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:23.518 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:23.518 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:23.533 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T10:12:23.552 INFO:tasks.rados.rados.0.smithi195.stdout:1978: expect (ObjNum 421 snap 0 seq_num 421) 2024-09-13T10:12:23.552 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:23.579 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:23.578+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:23.679 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:23.673+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:23.698 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:23.698 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:23.712 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T10:12:23.735 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1613250 2024-09-13T10:12:23.771 INFO:tasks.rados.rados.0.smithi195.stdout:1979: seq_num 443 ranges {645501=483781,1573523=460153,2548974=763384} 2024-09-13T10:12:23.780 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:23.778+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:23.790 INFO:tasks.rados.rados.0.smithi195.stdout:1979: writing smithi19530714-49 from 645501 to 1129282 tid 1 2024-09-13T10:12:23.804 INFO:tasks.rados.rados.0.smithi195.stdout:1979: writing smithi19530714-49 from 1573523 to 2033676 tid 2 2024-09-13T10:12:23.825 INFO:tasks.rados.rados.0.smithi195.stdout:1979: writing smithi19530714-49 from 2548974 to 3312358 tid 3 2024-09-13T10:12:23.825 INFO:tasks.rados.rados.0.smithi195.stdout:1975: done (4 left) 2024-09-13T10:12:23.825 INFO:tasks.rados.rados.0.smithi195.stdout:1976: done (3 left) 2024-09-13T10:12:23.825 INFO:tasks.rados.rados.0.smithi195.stdout:1977: done (2 left) 2024-09-13T10:12:23.825 INFO:tasks.rados.rados.0.smithi195.stdout:1978: done (1 left) 2024-09-13T10:12:23.825 INFO:tasks.rados.rados.0.smithi195.stdout:1980: read oid 40 snap -1 2024-09-13T10:12:23.825 INFO:tasks.rados.rados.0.smithi195.stdout:1980: expect (ObjNum 431 snap 0 seq_num 431) 2024-09-13T10:12:23.826 INFO:tasks.rados.rados.0.smithi195.stdout:1981: read oid 17 snap -1 2024-09-13T10:12:23.826 INFO:tasks.rados.rados.0.smithi195.stdout:1981: expect (ObjNum 388 snap 0 seq_num 388) 2024-09-13T10:12:23.826 INFO:tasks.rados.rados.0.smithi195.stdout:1982: read oid 10 snap -1 2024-09-13T10:12:23.826 INFO:tasks.rados.rados.0.smithi195.stdout:1982: expect deleted 2024-09-13T10:12:23.826 INFO:tasks.rados.rados.0.smithi195.stdout:1983: read oid 23 snap -1 2024-09-13T10:12:23.826 INFO:tasks.rados.rados.0.smithi195.stdout:1983: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:12:23.826 INFO:tasks.rados.rados.0.smithi195.stdout:1984: write (excl) oid 27 current snap is 0 2024-09-13T10:12:23.826 INFO:tasks.rados.rados.0.smithi195.stdout:1979: finishing write tid 1 to smithi19530714-49 2024-09-13T10:12:23.826 INFO:tasks.rados.rados.0.smithi195.stdout:1979: finishing write tid 2 to smithi19530714-49 2024-09-13T10:12:23.826 INFO:tasks.rados.rados.0.smithi195.stdout:1984: seq_num 444 ranges {662308=559030,1632005=603811,2860044=602264} 2024-09-13T10:12:23.835 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:23.836 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:23.847 INFO:tasks.rados.rados.0.smithi195.stdout:1984: writing smithi19530714-27 from 662308 to 1221338 tid 1 2024-09-13T10:12:23.851 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T10:12:23.860 INFO:tasks.rados.rados.0.smithi195.stdout:1984: writing smithi19530714-27 from 1632005 to 2235816 tid 2 2024-09-13T10:12:23.872 INFO:tasks.rados.rados.0.smithi195.stdout:1984: writing smithi19530714-27 from 2860044 to 3462308 tid 3 2024-09-13T10:12:23.872 INFO:tasks.rados.rados.0.smithi195.stdout:1985: read oid 37 snap -1 2024-09-13T10:12:23.872 INFO:tasks.rados.rados.0.smithi195.stdout:1985: expect (ObjNum 438 snap 0 seq_num 438) 2024-09-13T10:12:23.873 INFO:tasks.rados.rados.0.smithi195.stdout:1986: write (excl) oid 41 current snap is 0 2024-09-13T10:12:23.873 INFO:tasks.rados.rados.0.smithi195.stdout:1986: seq_num 445 ranges {679115=634279,2090487=463822,3003820=475295} 2024-09-13T10:12:23.881 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:23.878+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:23.883 INFO:tasks.rados.rados.0.smithi195.stdout:1986: writing smithi19530714-41 from 679115 to 1313394 tid 1 2024-09-13T10:12:23.893 INFO:tasks.rados.rados.0.smithi195.stdout:1986: writing smithi19530714-41 from 2090487 to 2554309 tid 2 2024-09-13T10:12:23.899 INFO:tasks.rados.rados.0.smithi195.stdout:1986: writing smithi19530714-41 from 3003820 to 3479115 tid 3 2024-09-13T10:12:23.900 INFO:tasks.rados.rados.0.smithi195.stdout:1987: delete oid 39 current snap is 0 2024-09-13T10:12:23.900 INFO:tasks.rados.rados.0.smithi195.stdout:1983: expect (ObjNum 286 snap 0 seq_num 286) 2024-09-13T10:12:23.903 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:12:23.907 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-13T10:12:23.977 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:23.977 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:23.981 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:23.978+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:23.992 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-13T10:12:24.002 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:24.047 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:12:24.047 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:12:24.056 INFO:tasks.rados.rados.0.smithi195.stdout:1979: finishing write tid 3 to smithi19530714-49 2024-09-13T10:12:24.056 INFO:tasks.rados.rados.0.smithi195.stdout:1979: finishing write tid 4 to smithi19530714-49 2024-09-13T10:12:24.056 INFO:tasks.rados.rados.0.smithi195.stdout:1979: finishing write tid 5 to smithi19530714-49 2024-09-13T10:12:24.056 INFO:tasks.rados.rados.0.smithi195.stdout:1979: oid 49 updating version 0 to 203 2024-09-13T10:12:24.056 INFO:tasks.rados.rados.0.smithi195.stdout:1979: oid 49 version 203 is already newer than 201 2024-09-13T10:12:24.056 INFO:tasks.rados.rados.0.smithi195.stdout:1979: oid 49 version 203 is already newer than 202 2024-09-13T10:12:24.056 INFO:tasks.rados.rados.0.smithi195.stdout:1979: oid 49 updating version 203 to 204 2024-09-13T10:12:24.056 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 204 (ObjNum 443 snap 0 seq_num 443) dirty exists 2024-09-13T10:12:24.057 INFO:tasks.rados.rados.0.smithi195.stdout:1979: left oid 49 (ObjNum 443 snap 0 seq_num 443) 2024-09-13T10:12:24.057 INFO:tasks.rados.rados.0.smithi195.stdout:1981: expect (ObjNum 388 snap 0 seq_num 388) 2024-09-13T10:12:24.057 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:24.068 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:12:24.068 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:12:24.083 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:24.077+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:24.087 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:12:24.088 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:12:24.108 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:12:24.108 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:12:24.115 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:24.116 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:24.126 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:12:24.126 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:12:24.130 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-13T10:12:24.148 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:12:24.148 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:12:24.169 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:12:24.170 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:12:24.177 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1664066 2024-09-13T10:12:24.183 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:24.177+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:24.191 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:12:24.191 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:12:24.192 INFO:tasks.rados.rados.0.smithi195.stdout:1980: expect (ObjNum 431 snap 0 seq_num 431) 2024-09-13T10:12:24.192 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:24.210 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:12:24.211 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:12:24.233 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:12:24.234 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:12:24.255 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:12:24.255 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:12:24.264 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:24.278 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:12:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:12:24.285 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:12:24.281+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:12:24.297 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:12:24.298 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:12:24.298 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:24.298 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:12:24.298 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:12:24.298 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:12:24.299 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:24.312 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-13T10:12:24.321 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:12:24.321 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:12:24.343 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:12:24.343 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:12:24.366 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:12:24.366 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:12:24.386 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:24.381+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:24.411 INFO:tasks.rados.rados.0.smithi195.stdout:1984: finishing write tid 1 to smithi19530714-27 2024-09-13T10:12:24.412 INFO:tasks.rados.rados.0.smithi195.stdout:1984: finishing write tid 2 to smithi19530714-27 2024-09-13T10:12:24.412 INFO:tasks.rados.rados.0.smithi195.stdout:1985: expect (ObjNum 438 snap 0 seq_num 438) 2024-09-13T10:12:24.484 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:24.484 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:12:24.485 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:12:24.485 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:12:24.485 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.485 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:201 3.3 3:d4fb61be:::smithi19530714-2 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:24.485 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:12:24.485 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.486678698999995, 2024-09-13T10:12:24.485 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.052937043000000003, 2024-09-13T10:12:24.485 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.485 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.485 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.485 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:24.486 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:24.486 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:24.486 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 201 2024-09-13T10:12:24.486 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.486 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.486 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.486 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.486 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:12:24.486 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.486 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.486 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.486 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.487 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993183+0000", 2024-09-13T10:12:24.487 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.487 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.487 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.487 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.487 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993192+0000", 2024-09-13T10:12:24.487 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8559999999999996e-06 2024-09-13T10:12:24.487 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.487 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.487 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.487 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993203+0000", 2024-09-13T10:12:24.488 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1113e-05 2024-09-13T10:12:24.488 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.488 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.488 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.488 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993225+0000", 2024-09-13T10:12:24.488 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2793e-05 2024-09-13T10:12:24.488 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.488 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.488 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.488 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993241+0000", 2024-09-13T10:12:24.488 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5806000000000001e-05 2024-09-13T10:12:24.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.489 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.489 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.489 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993401+0000", 2024-09-13T10:12:24.489 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000160149 2024-09-13T10:12:24.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.489 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.489 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.489 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994047+0000", 2024-09-13T10:12:24.489 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00064594100000000003 2024-09-13T10:12:24.489 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.490 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.490 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.490 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.011243+0000", 2024-09-13T10:12:24.490 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017196026999999999 2024-09-13T10:12:24.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.490 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.490 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.490 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029096+0000", 2024-09-13T10:12:24.490 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017852115000000002 2024-09-13T10:12:24.490 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.491 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.491 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.491 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029271+0000", 2024-09-13T10:12:24.491 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00017589299999999999 2024-09-13T10:12:24.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.491 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.491 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.491 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.045867+0000", 2024-09-13T10:12:24.491 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016595625999999999 2024-09-13T10:12:24.491 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.491 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.491 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.492 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.046017+0000", 2024-09-13T10:12:24.492 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014974199999999999 2024-09-13T10:12:24.492 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.492 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.492 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.492 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.046120+0000", 2024-09-13T10:12:24.492 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000102982 2024-09-13T10:12:24.492 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.492 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.493 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.493 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.493 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.493 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:208 3.3 3:dbb0d8aa:::smithi19530714-9 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:24.493 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:12:24.493 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.486582889000005, 2024-09-13T10:12:24.493 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.071216475000000001, 2024-09-13T10:12:24.494 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.494 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.494 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.494 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:24.494 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:24.494 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:24.494 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 208 2024-09-13T10:12:24.494 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.495 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.495 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.495 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.495 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:12:24.495 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.495 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.495 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.495 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.495 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993279+0000", 2024-09-13T10:12:24.495 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.495 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.496 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.496 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.496 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993280+0000", 2024-09-13T10:12:24.496 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1999999999999999e-06 2024-09-13T10:12:24.496 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.496 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.496 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.496 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993289+0000", 2024-09-13T10:12:24.496 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.8200000000000003e-06 2024-09-13T10:12:24.496 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.496 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.497 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.497 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994000+0000", 2024-09-13T10:12:24.497 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00071192499999999997 2024-09-13T10:12:24.497 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.497 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.497 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.497 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994024+0000", 2024-09-13T10:12:24.497 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3578999999999999e-05 2024-09-13T10:12:24.497 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.498 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.498 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.498 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994468+0000", 2024-09-13T10:12:24.498 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00044410599999999999 2024-09-13T10:12:24.498 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.498 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.498 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.498 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994842+0000", 2024-09-13T10:12:24.498 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037380300000000003 2024-09-13T10:12:24.498 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.498 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.499 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.499 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031572+0000", 2024-09-13T10:12:24.499 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036730479000000003 2024-09-13T10:12:24.499 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.499 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.499 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.499 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.039007+0000", 2024-09-13T10:12:24.499 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.007434871 2024-09-13T10:12:24.499 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.499 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.499 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.499 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.041001+0000", 2024-09-13T10:12:24.500 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001994104 2024-09-13T10:12:24.500 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.500 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.500 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.500 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064330+0000", 2024-09-13T10:12:24.500 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023328912 2024-09-13T10:12:24.500 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.500 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.500 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.500 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064401+0000", 2024-09-13T10:12:24.500 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.1123000000000003e-05 2024-09-13T10:12:24.500 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.501 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.501 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.501 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064495+0000", 2024-09-13T10:12:24.501 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.3553000000000004e-05 2024-09-13T10:12:24.501 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.501 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.501 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.502 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.502 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.502 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:215 3.3 3:ed6c0a17:::smithi19530714-16 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:24.502 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:12:24.502 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.485802634999999, 2024-09-13T10:12:24.502 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.055806517999999999, 2024-09-13T10:12:24.502 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.503 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.503 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.503 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:24.503 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:24.503 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:24.503 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 215 2024-09-13T10:12:24.503 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.503 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.503 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.503 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.503 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:12:24.504 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.504 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.504 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.504 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.504 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994059+0000", 2024-09-13T10:12:24.504 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.504 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.504 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.504 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.504 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994060+0000", 2024-09-13T10:12:24.504 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5030000000000001e-06 2024-09-13T10:12:24.505 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.505 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.505 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.505 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994070+0000", 2024-09-13T10:12:24.505 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0046e-05 2024-09-13T10:12:24.505 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.505 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.505 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.505 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994087+0000", 2024-09-13T10:12:24.505 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6283999999999999e-05 2024-09-13T10:12:24.506 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.506 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.506 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.506 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.994096+0000", 2024-09-13T10:12:24.506 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.969e-06 2024-09-13T10:12:24.506 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.506 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.506 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.506 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.995226+0000", 2024-09-13T10:12:24.506 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0011299540000000001 2024-09-13T10:12:24.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.507 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.507 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.507 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.995548+0000", 2024-09-13T10:12:24.507 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00032266199999999998 2024-09-13T10:12:24.507 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.507 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.507 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.508 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.018922+0000", 2024-09-13T10:12:24.508 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023373485999999999 2024-09-13T10:12:24.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.508 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.508 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.508 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.029400+0000", 2024-09-13T10:12:24.508 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010478721 2024-09-13T10:12:24.508 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.508 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.509 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.509 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038932+0000", 2024-09-13T10:12:24.509 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0095316859999999993 2024-09-13T10:12:24.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.509 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.509 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.509 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049620+0000", 2024-09-13T10:12:24.509 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010687688000000001 2024-09-13T10:12:24.509 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.509 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.509 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.510 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049754+0000", 2024-09-13T10:12:24.510 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013457700000000001 2024-09-13T10:12:24.510 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.510 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.510 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.510 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.049865+0000", 2024-09-13T10:12:24.510 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110942 2024-09-13T10:12:24.510 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.510 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.510 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.511 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.511 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.511 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:224 3.3 3:f941658c:::smithi19530714-25 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:24.511 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:12:24.511 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.429294597999998, 2024-09-13T10:12:24.511 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.025014027000000001, 2024-09-13T10:12:24.511 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.511 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.511 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.511 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:24.512 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:24.512 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:24.512 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 224 2024-09-13T10:12:24.512 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.512 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.512 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.512 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.512 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:12:24.512 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.513 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.513 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.513 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.513 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050567+0000", 2024-09-13T10:12:24.513 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.513 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.513 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.513 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.514 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050570+0000", 2024-09-13T10:12:24.514 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9799999999999998e-06 2024-09-13T10:12:24.514 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.514 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.514 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.514 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050582+0000", 2024-09-13T10:12:24.514 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2170999999999999e-05 2024-09-13T10:12:24.514 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.514 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.515 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.515 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050605+0000", 2024-09-13T10:12:24.515 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3543000000000001e-05 2024-09-13T10:12:24.515 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.515 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.515 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.515 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050619+0000", 2024-09-13T10:12:24.515 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3478999999999999e-05 2024-09-13T10:12:24.516 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.516 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.516 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.516 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050725+0000", 2024-09-13T10:12:24.516 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000105585 2024-09-13T10:12:24.516 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.516 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.516 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.516 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.051086+0000", 2024-09-13T10:12:24.516 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036162100000000001 2024-09-13T10:12:24.517 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.517 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.517 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.517 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.058628+0000", 2024-09-13T10:12:24.517 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075415569999999999 2024-09-13T10:12:24.517 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.517 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.517 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.517 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065344+0000", 2024-09-13T10:12:24.518 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0067161740000000001 2024-09-13T10:12:24.518 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.518 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.518 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.518 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.066222+0000", 2024-09-13T10:12:24.518 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00087765599999999999 2024-09-13T10:12:24.518 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.518 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.518 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.518 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075372+0000", 2024-09-13T10:12:24.518 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0091499609999999999 2024-09-13T10:12:24.519 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.519 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.519 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.519 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075489+0000", 2024-09-13T10:12:24.519 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011718300000000001 2024-09-13T10:12:24.519 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.519 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.519 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.520 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.075581+0000", 2024-09-13T10:12:24.520 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.2116999999999999e-05 2024-09-13T10:12:24.520 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.520 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.520 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.520 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.520 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.520 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:226 3.3 3:fe5bfa3a:::smithi19530714-27 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:24.520 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:12:24.521 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.419761589000004, 2024-09-13T10:12:24.521 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051515848000000003, 2024-09-13T10:12:24.521 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.521 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.521 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.521 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:24.521 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:24.522 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:24.522 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 226 2024-09-13T10:12:24.522 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.522 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.522 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.522 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.522 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:12:24.522 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.523 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.523 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.523 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.523 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060100+0000", 2024-09-13T10:12:24.523 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.523 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.523 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.523 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.524 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060102+0000", 2024-09-13T10:12:24.524 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0480000000000001e-06 2024-09-13T10:12:24.524 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.524 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.524 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.524 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060112+0000", 2024-09-13T10:12:24.524 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6649999999999998e-06 2024-09-13T10:12:24.524 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.524 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.524 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.524 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060132+0000", 2024-09-13T10:12:24.525 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0262e-05 2024-09-13T10:12:24.525 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.525 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.525 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.525 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060143+0000", 2024-09-13T10:12:24.525 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1420000000000001e-05 2024-09-13T10:12:24.525 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.525 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.525 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.525 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064563+0000", 2024-09-13T10:12:24.525 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0044199119999999998 2024-09-13T10:12:24.525 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.526 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.526 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.526 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064882+0000", 2024-09-13T10:12:24.526 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000318735 2024-09-13T10:12:24.526 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.526 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.526 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.526 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.078858+0000", 2024-09-13T10:12:24.526 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013976455 2024-09-13T10:12:24.526 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.527 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.527 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.527 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088977+0000", 2024-09-13T10:12:24.527 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010118544 2024-09-13T10:12:24.527 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.527 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.527 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.527 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.090451+0000", 2024-09-13T10:12:24.528 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001474026 2024-09-13T10:12:24.528 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.528 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.528 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.528 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111539+0000", 2024-09-13T10:12:24.528 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.02108786 2024-09-13T10:12:24.528 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.528 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.528 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.529 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111580+0000", 2024-09-13T10:12:24.529 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.1245000000000002e-05 2024-09-13T10:12:24.529 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.529 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.529 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.529 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111616+0000", 2024-09-13T10:12:24.529 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5676000000000002e-05 2024-09-13T10:12:24.529 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.530 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.530 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.530 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.530 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.530 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:235 3.3 3:e217e96e:::smithi19530714-36 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:24.530 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:12:24.530 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.40898396, 2024-09-13T10:12:24.530 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051184682000000002, 2024-09-13T10:12:24.531 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.531 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.531 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.531 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:24.531 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:24.531 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:24.531 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 235 2024-09-13T10:12:24.531 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.531 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.532 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.532 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.532 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:12:24.532 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.532 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.532 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.532 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.533 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070877+0000", 2024-09-13T10:12:24.533 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.533 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.533 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.533 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.533 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070881+0000", 2024-09-13T10:12:24.533 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0579999999999999e-06 2024-09-13T10:12:24.533 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.533 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.533 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.533 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070897+0000", 2024-09-13T10:12:24.534 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5967000000000001e-05 2024-09-13T10:12:24.534 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.534 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.534 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.534 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070942+0000", 2024-09-13T10:12:24.534 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4947000000000002e-05 2024-09-13T10:12:24.534 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.534 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.534 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.534 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.070963+0000", 2024-09-13T10:12:24.534 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0098e-05 2024-09-13T10:12:24.535 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.535 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.535 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.535 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.071099+0000", 2024-09-13T10:12:24.535 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013697199999999999 2024-09-13T10:12:24.535 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.535 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.535 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.535 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.071330+0000", 2024-09-13T10:12:24.536 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000230881 2024-09-13T10:12:24.536 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.536 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.536 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.536 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.084668+0000", 2024-09-13T10:12:24.536 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013337663 2024-09-13T10:12:24.536 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.536 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.537 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.537 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.092656+0000", 2024-09-13T10:12:24.537 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0079878880000000003 2024-09-13T10:12:24.537 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.537 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.537 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.537 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.100238+0000", 2024-09-13T10:12:24.537 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0075819449999999997 2024-09-13T10:12:24.537 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.538 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.538 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.538 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121877+0000", 2024-09-13T10:12:24.538 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021639104999999999 2024-09-13T10:12:24.538 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.538 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.538 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.538 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121978+0000", 2024-09-13T10:12:24.538 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000101363 2024-09-13T10:12:24.539 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.539 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.539 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.539 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.122062+0000", 2024-09-13T10:12:24.539 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.3795000000000007e-05 2024-09-13T10:12:24.539 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.539 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.539 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.539 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.540 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.540 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:237 3.3 3:de40747e:::smithi19530714-38 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:24.540 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:12:24.540 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.400794755999996, 2024-09-13T10:12:24.540 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.073341331999999995, 2024-09-13T10:12:24.540 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.540 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.540 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.540 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:24.540 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:24.541 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:24.541 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 237 2024-09-13T10:12:24.541 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.541 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.541 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.541 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.541 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:12:24.541 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.541 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.541 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.541 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.542 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079067+0000", 2024-09-13T10:12:24.542 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.542 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.542 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.542 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.542 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079069+0000", 2024-09-13T10:12:24.542 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6979999999999998e-06 2024-09-13T10:12:24.542 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.542 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.543 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.543 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079083+0000", 2024-09-13T10:12:24.543 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.384e-05 2024-09-13T10:12:24.543 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.543 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.543 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.543 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079105+0000", 2024-09-13T10:12:24.543 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1736e-05 2024-09-13T10:12:24.544 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.544 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.544 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.544 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079123+0000", 2024-09-13T10:12:24.544 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7889000000000001e-05 2024-09-13T10:12:24.544 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.544 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.545 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.545 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088459+0000", 2024-09-13T10:12:24.545 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0093359700000000007 2024-09-13T10:12:24.545 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.545 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.545 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.545 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088726+0000", 2024-09-13T10:12:24.545 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026732299999999997 2024-09-13T10:12:24.545 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.545 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.545 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.546 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.097522+0000", 2024-09-13T10:12:24.546 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0087962630000000003 2024-09-13T10:12:24.546 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.546 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.546 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.546 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.103972+0000", 2024-09-13T10:12:24.546 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0064494840000000001 2024-09-13T10:12:24.546 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.546 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.546 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.546 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.104160+0000", 2024-09-13T10:12:24.547 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00018782200000000001 2024-09-13T10:12:24.547 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.547 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.547 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.547 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152294+0000", 2024-09-13T10:12:24.547 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048134458999999997 2024-09-13T10:12:24.547 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.547 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.547 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.547 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152373+0000", 2024-09-13T10:12:24.547 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.8574999999999999e-05 2024-09-13T10:12:24.548 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.548 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.548 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.548 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.152408+0000", 2024-09-13T10:12:24.548 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5272999999999999e-05 2024-09-13T10:12:24.548 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.548 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.548 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.548 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.548 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.549 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:238 3.3 3:c43dd60b:::smithi19530714-39 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:24.549 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:12:24.549 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.391209109000002, 2024-09-13T10:12:24.549 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.069426360000000006, 2024-09-13T10:12:24.549 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.549 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.549 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.549 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:24.549 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:24.550 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:24.550 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 238 2024-09-13T10:12:24.550 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.550 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.550 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.550 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.550 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:12:24.550 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.550 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.550 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.550 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.550 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088652+0000", 2024-09-13T10:12:24.551 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.551 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.551 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.551 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.551 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088655+0000", 2024-09-13T10:12:24.551 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2359999999999999e-06 2024-09-13T10:12:24.551 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.551 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.551 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.551 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088663+0000", 2024-09-13T10:12:24.552 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7800000000000006e-06 2024-09-13T10:12:24.552 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.552 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.552 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.552 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088688+0000", 2024-09-13T10:12:24.552 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4507000000000001e-05 2024-09-13T10:12:24.552 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.552 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.552 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.552 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.088717+0000", 2024-09-13T10:12:24.552 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9204e-05 2024-09-13T10:12:24.552 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.553 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.553 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.553 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.089038+0000", 2024-09-13T10:12:24.553 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00032087199999999999 2024-09-13T10:12:24.553 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.553 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.553 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.553 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.089435+0000", 2024-09-13T10:12:24.553 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039756600000000002 2024-09-13T10:12:24.553 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.553 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.553 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.554 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.101672+0000", 2024-09-13T10:12:24.554 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012236528 2024-09-13T10:12:24.554 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.554 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.554 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.554 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.105256+0000", 2024-09-13T10:12:24.554 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0035837159999999998 2024-09-13T10:12:24.554 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.554 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.554 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.554 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111504+0000", 2024-09-13T10:12:24.554 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.006248577 2024-09-13T10:12:24.555 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.555 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.555 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.555 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.157950+0000", 2024-09-13T10:12:24.555 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.046446031999999998 2024-09-13T10:12:24.555 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.555 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.555 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.555 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158027+0000", 2024-09-13T10:12:24.555 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6402999999999993e-05 2024-09-13T10:12:24.555 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.555 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.556 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.556 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158079+0000", 2024-09-13T10:12:24.556 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.1938999999999999e-05 2024-09-13T10:12:24.556 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.556 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.556 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.556 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.556 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.556 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:246 3.3 3:f1be28e7:::smithi19530714-47 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:24.556 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:12:24.556 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.366498362000002, 2024-09-13T10:12:24.556 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051630810999999999, 2024-09-13T10:12:24.557 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.557 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.557 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.557 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:24.557 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:24.557 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:24.557 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 246 2024-09-13T10:12:24.557 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.557 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.557 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.558 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.558 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:12:24.558 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.558 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.558 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.558 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.558 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113363+0000", 2024-09-13T10:12:24.558 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.558 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.558 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.558 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.558 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113365+0000", 2024-09-13T10:12:24.559 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.762e-06 2024-09-13T10:12:24.559 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.559 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.559 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.559 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113372+0000", 2024-09-13T10:12:24.559 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.5020000000000003e-06 2024-09-13T10:12:24.559 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.559 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.559 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.559 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113390+0000", 2024-09-13T10:12:24.559 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8056000000000001e-05 2024-09-13T10:12:24.559 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.560 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.560 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.560 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113400+0000", 2024-09-13T10:12:24.560 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5119999999999997e-06 2024-09-13T10:12:24.560 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.560 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.560 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.560 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113456+0000", 2024-09-13T10:12:24.560 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.6221999999999998e-05 2024-09-13T10:12:24.560 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.560 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.561 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.561 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.113576+0000", 2024-09-13T10:12:24.561 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000119466 2024-09-13T10:12:24.561 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.561 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.561 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.561 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.129127+0000", 2024-09-13T10:12:24.561 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015551003000000001 2024-09-13T10:12:24.561 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.561 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.561 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.562 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142482+0000", 2024-09-13T10:12:24.562 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013355104 2024-09-13T10:12:24.562 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.562 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.562 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.562 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142518+0000", 2024-09-13T10:12:24.562 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6532000000000002e-05 2024-09-13T10:12:24.562 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.562 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.562 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.562 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164919+0000", 2024-09-13T10:12:24.562 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022401038000000002 2024-09-13T10:12:24.563 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.563 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.563 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.563 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164962+0000", 2024-09-13T10:12:24.563 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.2329000000000001e-05 2024-09-13T10:12:24.563 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.563 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.563 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.563 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.164994+0000", 2024-09-13T10:12:24.563 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2285e-05 2024-09-13T10:12:24.563 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.563 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.563 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.564 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.564 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.564 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:249 3.3 3:f565152a:::smithi19530714-50 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:24.564 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:12:24.564 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.364895884999996, 2024-09-13T10:12:24.564 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.051233251, 2024-09-13T10:12:24.564 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.564 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.564 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.564 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:24.564 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:24.565 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:24.565 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 249 2024-09-13T10:12:24.565 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.565 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.565 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.565 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.565 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:12:24.565 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.566 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.566 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.566 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.566 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114966+0000", 2024-09-13T10:12:24.566 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.566 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.566 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.566 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.566 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114967+0000", 2024-09-13T10:12:24.566 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.313e-06 2024-09-13T10:12:24.566 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.566 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.567 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.567 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114972+0000", 2024-09-13T10:12:24.567 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5620000000000003e-06 2024-09-13T10:12:24.567 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.567 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.567 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.567 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114997+0000", 2024-09-13T10:12:24.567 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4845999999999999e-05 2024-09-13T10:12:24.567 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.567 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.567 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.567 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115007+0000", 2024-09-13T10:12:24.568 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.7200000000000001e-06 2024-09-13T10:12:24.568 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.568 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.568 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.568 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115076+0000", 2024-09-13T10:12:24.568 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9473000000000004e-05 2024-09-13T10:12:24.568 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.568 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.568 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.569 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115221+0000", 2024-09-13T10:12:24.569 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000144604 2024-09-13T10:12:24.569 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.569 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.569 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.569 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.136862+0000", 2024-09-13T10:12:24.569 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021641365999999999 2024-09-13T10:12:24.569 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.569 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.570 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.570 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.145775+0000", 2024-09-13T10:12:24.570 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0089130649999999995 2024-09-13T10:12:24.570 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.570 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.570 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.570 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.150578+0000", 2024-09-13T10:12:24.570 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0048025999999999998 2024-09-13T10:12:24.571 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.571 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.571 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.571 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166133+0000", 2024-09-13T10:12:24.571 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015554853 2024-09-13T10:12:24.571 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.571 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.571 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.571 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166167+0000", 2024-09-13T10:12:24.571 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3973999999999997e-05 2024-09-13T10:12:24.571 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.572 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.572 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.572 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.166199+0000", 2024-09-13T10:12:24.572 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1875000000000002e-05 2024-09-13T10:12:24.572 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.572 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.572 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.572 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.572 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.572 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.6.23:1 3.3 3:d4fb61be:::smithi19530714-2 low_tier:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:24.572 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:12:24.573 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.359732836000006, 2024-09-13T10:12:24.573 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.038168746000000003, 2024-09-13T10:12:24.573 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.573 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.573 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.573 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.573 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.573 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.573 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:12:24.573 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.573 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.574 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.574 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.574 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120129+0000", 2024-09-13T10:12:24.574 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.574 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.574 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.574 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120131+0000", 2024-09-13T10:12:24.574 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5620000000000002e-06 2024-09-13T10:12:24.574 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.574 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.574 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.575 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120140+0000", 2024-09-13T10:12:24.575 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1889999999999996e-06 2024-09-13T10:12:24.575 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.575 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.575 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120161+0000", 2024-09-13T10:12:24.575 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0485000000000001e-05 2024-09-13T10:12:24.575 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.575 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.575 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.575 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120200+0000", 2024-09-13T10:12:24.575 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9648999999999999e-05 2024-09-13T10:12:24.575 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.576 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.576 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.576 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120285+0000", 2024-09-13T10:12:24.576 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.4488999999999997e-05 2024-09-13T10:12:24.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.576 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.576 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.576 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120399+0000", 2024-09-13T10:12:24.576 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000113857 2024-09-13T10:12:24.576 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.576 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.576 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.577 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.120640+0000", 2024-09-13T10:12:24.577 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024112600000000001 2024-09-13T10:12:24.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.577 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.577 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.577 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.121715+0000", 2024-09-13T10:12:24.577 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001074885 2024-09-13T10:12:24.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.577 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.577 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.577 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.122870+0000", 2024-09-13T10:12:24.577 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00115504 2024-09-13T10:12:24.577 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.577 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.578 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.578 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158160+0000", 2024-09-13T10:12:24.578 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.035289760000000003 2024-09-13T10:12:24.578 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.578 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.578 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.578 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158211+0000", 2024-09-13T10:12:24.578 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.1005999999999999e-05 2024-09-13T10:12:24.578 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.578 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.578 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.578 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.158297+0000", 2024-09-13T10:12:24.579 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.6698000000000002e-05 2024-09-13T10:12:24.579 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.579 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.579 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.579 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.579 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.579 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.14.23:47 3.3 3:ffec9690:::b0d2dba9f365561c17cd4cce8c8a8f4d8f931d05ca998f81ae8f5caa986fcf1e:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:24.579 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:10.781559+0000", 2024-09-13T10:12:24.579 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 73.69830193, 2024-09-13T10:12:24.579 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015576100000000001, 2024-09-13T10:12:24.579 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.579 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.580 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.580 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.580 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.580 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.580 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781559+0000", 2024-09-13T10:12:24.580 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.580 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.580 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.580 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.580 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781559+0000", 2024-09-13T10:12:24.580 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.580 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.581 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.581 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781561+0000", 2024-09-13T10:12:24.581 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5460000000000001e-06 2024-09-13T10:12:24.581 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.581 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.581 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781708+0000", 2024-09-13T10:12:24.581 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000147291 2024-09-13T10:12:24.581 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.581 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.581 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.582 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781735+0000", 2024-09-13T10:12:24.582 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7175999999999999e-05 2024-09-13T10:12:24.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.582 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.582 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.781755+0000", 2024-09-13T10:12:24.582 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9653e-05 2024-09-13T10:12:24.582 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.582 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.582 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.582 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.790861+0000", 2024-09-13T10:12:24.582 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0091056920000000003 2024-09-13T10:12:24.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.583 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.583 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.791072+0000", 2024-09-13T10:12:24.583 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000211103 2024-09-13T10:12:24.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.583 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.583 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.791511+0000", 2024-09-13T10:12:24.583 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00043908899999999998 2024-09-13T10:12:24.583 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.583 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.583 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.584 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794857+0000", 2024-09-13T10:12:24.584 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0033462940000000001 2024-09-13T10:12:24.584 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.584 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.584 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.584 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.794945+0000", 2024-09-13T10:12:24.584 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7626000000000004e-05 2024-09-13T10:12:24.584 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.584 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.584 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.584 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.797001+0000", 2024-09-13T10:12:24.584 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020557280000000002 2024-09-13T10:12:24.584 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.585 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.585 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.585 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.797080+0000", 2024-09-13T10:12:24.585 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9169e-05 2024-09-13T10:12:24.585 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.585 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.585 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.585 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.797136+0000", 2024-09-13T10:12:24.585 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5733000000000003e-05 2024-09-13T10:12:24.585 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.585 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.586 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.586 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.586 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.586 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:3970 4.18 4:1de9da87:::smithi19530714-5:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e94)", 2024-09-13T10:12:24.586 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:00.220914+0000", 2024-09-13T10:12:24.586 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 24.258947894999999, 2024-09-13T10:12:24.586 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048456570999999997, 2024-09-13T10:12:24.586 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.586 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.586 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:24.586 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:24.586 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:24.587 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:24.587 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 3970 2024-09-13T10:12:24.587 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.587 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.587 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.587 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.587 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.220914+0000", 2024-09-13T10:12:24.587 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.587 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.587 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.587 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.587 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.220914+0000", 2024-09-13T10:12:24.588 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.588 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.588 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.588 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.220917+0000", 2024-09-13T10:12:24.588 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.36e-06 2024-09-13T10:12:24.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.588 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.588 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.588 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.220934+0000", 2024-09-13T10:12:24.588 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6748999999999998e-05 2024-09-13T10:12:24.588 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.589 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.589 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.220973+0000", 2024-09-13T10:12:24.589 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9693000000000001e-05 2024-09-13T10:12:24.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.589 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.589 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.221001+0000", 2024-09-13T10:12:24.589 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7209e-05 2024-09-13T10:12:24.589 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.589 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.589 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.589 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.221170+0000", 2024-09-13T10:12:24.589 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016995700000000001 2024-09-13T10:12:24.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.590 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.590 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.590 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.221316+0000", 2024-09-13T10:12:24.590 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000145981 2024-09-13T10:12:24.590 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.590 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.590 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.590 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:00.269370+0000", 2024-09-13T10:12:24.590 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048053621999999997 2024-09-13T10:12:24.590 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.590 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.591 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.591 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.591 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.591 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.15.22:53 3.3 3:c0dd3fe6:::ec0aa0931246a621c18aa6befa600bcc59df35053a1247b4e1863cf02828cf81:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:24.591 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:05.193032+0000", 2024-09-13T10:12:24.591 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.286829145999999, 2024-09-13T10:12:24.591 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015830879999999999, 2024-09-13T10:12:24.591 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.591 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.591 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.591 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.591 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.592 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.592 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193032+0000", 2024-09-13T10:12:24.592 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.592 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.592 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193032+0000", 2024-09-13T10:12:24.592 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.592 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.592 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.592 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193034+0000", 2024-09-13T10:12:24.592 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6330000000000001e-06 2024-09-13T10:12:24.592 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.593 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.593 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.593 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193471+0000", 2024-09-13T10:12:24.593 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000437503 2024-09-13T10:12:24.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.593 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.593 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.593 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193493+0000", 2024-09-13T10:12:24.593 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1786000000000001e-05 2024-09-13T10:12:24.593 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.593 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.593 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.594 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193504+0000", 2024-09-13T10:12:24.594 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1043000000000001e-05 2024-09-13T10:12:24.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.594 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.594 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.594 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.202746+0000", 2024-09-13T10:12:24.594 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.009241773 2024-09-13T10:12:24.594 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.594 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.594 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.594 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.202973+0000", 2024-09-13T10:12:24.594 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000226956 2024-09-13T10:12:24.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.595 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.595 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.595 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.203492+0000", 2024-09-13T10:12:24.595 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00051952400000000005 2024-09-13T10:12:24.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.595 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.595 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.595 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.206115+0000", 2024-09-13T10:12:24.595 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0026221899999999999 2024-09-13T10:12:24.595 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.595 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.595 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.208604+0000", 2024-09-13T10:12:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0024892870000000002 2024-09-13T10:12:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.208715+0000", 2024-09-13T10:12:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000110568 2024-09-13T10:12:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.208800+0000", 2024-09-13T10:12:24.596 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5414999999999995e-05 2024-09-13T10:12:24.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.597 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.597 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.597 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.208863+0000", 2024-09-13T10:12:24.597 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.3201999999999996e-05 2024-09-13T10:12:24.597 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.597 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.597 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.597 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.597 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.597 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.15.22:55 3.3 3:cdcec4b2:::f314d852c33ac9b893465b328249830227013d5ec45b1156c69d7b9a27a71710:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:24.597 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:05.193532+0000", 2024-09-13T10:12:24.597 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.286329295000002, 2024-09-13T10:12:24.598 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017555576, 2024-09-13T10:12:24.598 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.598 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.598 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.598 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.598 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.598 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.598 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193532+0000", 2024-09-13T10:12:24.598 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.598 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.598 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.598 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.598 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193532+0000", 2024-09-13T10:12:24.599 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.599 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.599 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.599 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193534+0000", 2024-09-13T10:12:24.599 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5030000000000001e-06 2024-09-13T10:12:24.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.599 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.599 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.599 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193691+0000", 2024-09-13T10:12:24.599 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015784100000000001 2024-09-13T10:12:24.599 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.599 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.600 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.600 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193711+0000", 2024-09-13T10:12:24.600 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.98e-05 2024-09-13T10:12:24.600 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.600 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.600 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.600 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193721+0000", 2024-09-13T10:12:24.600 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.5799999999999998e-06 2024-09-13T10:12:24.600 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.600 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.600 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.600 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.206570+0000", 2024-09-13T10:12:24.601 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012849572 2024-09-13T10:12:24.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.601 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.601 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.601 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.206946+0000", 2024-09-13T10:12:24.601 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037542700000000001 2024-09-13T10:12:24.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.601 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.601 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.601 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.207419+0000", 2024-09-13T10:12:24.601 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00047319199999999999 2024-09-13T10:12:24.601 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.601 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.602 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.602 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.210769+0000", 2024-09-13T10:12:24.602 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0033497599999999998 2024-09-13T10:12:24.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.602 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.602 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.602 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.210878+0000", 2024-09-13T10:12:24.602 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000109028 2024-09-13T10:12:24.602 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.602 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.602 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.602 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.210911+0000", 2024-09-13T10:12:24.602 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2750000000000003e-05 2024-09-13T10:12:24.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.603 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.603 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.603 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.211024+0000", 2024-09-13T10:12:24.603 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000113714 2024-09-13T10:12:24.603 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.603 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.603 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.603 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.211088+0000", 2024-09-13T10:12:24.603 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.3409000000000002e-05 2024-09-13T10:12:24.603 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.603 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.603 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.604 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.604 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.15.22:62 3.3 3:f4bd62ee:::c19f5965474b2636dbb114434ccba3e174847a0d497c0002fe234287445262fc:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:24.604 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:05.193742+0000", 2024-09-13T10:12:24.604 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 19.286119544999998, 2024-09-13T10:12:24.604 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.018130356, 2024-09-13T10:12:24.604 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.604 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.604 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.604 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.604 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.604 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.605 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193742+0000", 2024-09-13T10:12:24.605 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.605 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.605 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.605 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.605 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193742+0000", 2024-09-13T10:12:24.605 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.605 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.605 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.605 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.605 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193743+0000", 2024-09-13T10:12:24.605 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.2699999999999998e-07 2024-09-13T10:12:24.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.606 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.606 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.606 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193849+0000", 2024-09-13T10:12:24.606 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00010634800000000001 2024-09-13T10:12:24.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.606 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.606 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.606 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193861+0000", 2024-09-13T10:12:24.606 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2016e-05 2024-09-13T10:12:24.606 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.606 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.607 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.607 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.193872+0000", 2024-09-13T10:12:24.607 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1243000000000001e-05 2024-09-13T10:12:24.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.607 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.607 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.607 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.209010+0000", 2024-09-13T10:12:24.607 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015137711999999999 2024-09-13T10:12:24.607 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.607 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.607 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.607 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.209292+0000", 2024-09-13T10:12:24.607 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028191499999999999 2024-09-13T10:12:24.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.608 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.608 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.209651+0000", 2024-09-13T10:12:24.608 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00035888700000000001 2024-09-13T10:12:24.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.608 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.608 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.211151+0000", 2024-09-13T10:12:24.608 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0015004969999999999 2024-09-13T10:12:24.608 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.608 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.608 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.609 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.211678+0000", 2024-09-13T10:12:24.609 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00052626699999999997 2024-09-13T10:12:24.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.609 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.609 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.609 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.211746+0000", 2024-09-13T10:12:24.609 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.8639000000000005e-05 2024-09-13T10:12:24.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.609 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.609 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.609 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.211816+0000", 2024-09-13T10:12:24.609 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.9785999999999999e-05 2024-09-13T10:12:24.609 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.610 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.610 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.610 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.211872+0000", 2024-09-13T10:12:24.610 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.6119000000000003e-05 2024-09-13T10:12:24.610 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.610 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.610 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.610 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.610 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.610 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.2.23:176 3.3 3:fac4d691:::814864e4c5245cfaf79ad18a0e61662368667270ceb2e98bad1fed4e25945c9c:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:12:24.610 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:05.835347+0000", 2024-09-13T10:12:24.611 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 18.644514612000002, 2024-09-13T10:12:24.611 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015415807, 2024-09-13T10:12:24.611 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.611 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.611 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.611 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.611 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.611 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.611 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.835347+0000", 2024-09-13T10:12:24.611 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.611 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.611 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.611 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.612 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.835347+0000", 2024-09-13T10:12:24.612 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.612 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.612 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.612 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.835348+0000", 2024-09-13T10:12:24.612 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0869999999999999e-06 2024-09-13T10:12:24.612 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.612 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.612 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.612 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.835353+0000", 2024-09-13T10:12:24.612 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.4070000000000001e-06 2024-09-13T10:12:24.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.613 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.613 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.835363+0000", 2024-09-13T10:12:24.613 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.9830000000000008e-06 2024-09-13T10:12:24.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.613 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.613 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.835378+0000", 2024-09-13T10:12:24.613 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5106000000000001e-05 2024-09-13T10:12:24.613 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.613 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.614 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.614 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.835621+0000", 2024-09-13T10:12:24.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024247599999999999 2024-09-13T10:12:24.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.614 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.614 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting_for_map not empty", 2024-09-13T10:12:24.614 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.835627+0000", 2024-09-13T10:12:24.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4829999999999996e-06 2024-09-13T10:12:24.614 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.614 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.614 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.614 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.847651+0000", 2024-09-13T10:12:24.614 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012023574 2024-09-13T10:12:24.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.615 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.615 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.615 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.847898+0000", 2024-09-13T10:12:24.615 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000246798 2024-09-13T10:12:24.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.615 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.615 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.615 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.848357+0000", 2024-09-13T10:12:24.615 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00045895899999999999 2024-09-13T10:12:24.615 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.615 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.615 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.616 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.850450+0000", 2024-09-13T10:12:24.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.002093003 2024-09-13T10:12:24.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.616 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.616 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.850574+0000", 2024-09-13T10:12:24.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012480400000000001 2024-09-13T10:12:24.616 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.616 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.616 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.616 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.850594+0000", 2024-09-13T10:12:24.616 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9233999999999999e-05 2024-09-13T10:12:24.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.617 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.850696+0000", 2024-09-13T10:12:24.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000102704 2024-09-13T10:12:24.617 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.617 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.617 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.617 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:05.850763+0000", 2024-09-13T10:12:24.617 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.6188999999999999e-05 2024-09-13T10:12:24.617 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.617 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.618 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.618 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.618 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.618 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.13.21:111 3.3 3:f8b6ff76:::0df7b452b15ded2caa5361cab4a0527aa26bbdbd1c62c3d43532df9f8363c465:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e106)", 2024-09-13T10:12:24.618 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:14.701910+0000", 2024-09-13T10:12:24.618 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.7779517859999991, 2024-09-13T10:12:24.618 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.016339989999999999, 2024-09-13T10:12:24.618 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.618 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.618 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.618 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.619 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.619 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.619 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.701910+0000", 2024-09-13T10:12:24.619 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.619 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.619 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.619 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.701910+0000", 2024-09-13T10:12:24.619 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.619 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.619 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.620 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.620 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.701911+0000", 2024-09-13T10:12:24.620 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0529999999999999e-06 2024-09-13T10:12:24.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.620 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.620 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.620 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702056+0000", 2024-09-13T10:12:24.620 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00014561199999999999 2024-09-13T10:12:24.620 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.620 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.621 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.621 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702078+0000", 2024-09-13T10:12:24.621 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1403000000000002e-05 2024-09-13T10:12:24.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.621 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.621 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.621 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702093+0000", 2024-09-13T10:12:24.621 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5092999999999999e-05 2024-09-13T10:12:24.621 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.621 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.621 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.622 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.712785+0000", 2024-09-13T10:12:24.622 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010691740999999999 2024-09-13T10:12:24.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.622 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.622 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.622 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.713115+0000", 2024-09-13T10:12:24.622 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000330895 2024-09-13T10:12:24.622 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.622 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.622 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.622 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.713706+0000", 2024-09-13T10:12:24.622 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00059055999999999996 2024-09-13T10:12:24.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.623 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.623 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.623 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.716028+0000", 2024-09-13T10:12:24.623 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023215940000000002 2024-09-13T10:12:24.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.623 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.623 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.623 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.717873+0000", 2024-09-13T10:12:24.623 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0018456080000000001 2024-09-13T10:12:24.623 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.623 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.623 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.624 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.718003+0000", 2024-09-13T10:12:24.624 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012976099999999999 2024-09-13T10:12:24.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.624 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.624 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.718142+0000", 2024-09-13T10:12:24.624 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013915100000000001 2024-09-13T10:12:24.624 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.624 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.624 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.624 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.718250+0000", 2024-09-13T10:12:24.624 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000107519 2024-09-13T10:12:24.625 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.625 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.625 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.625 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.625 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.625 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.13.21:112 3.3 3:de19817f:::f36543c9e1229abc2db2e0c089492cde9aa332b42fc2e36672be1ba4d1ccbfa5:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e106)", 2024-09-13T10:12:24.625 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:14.702115+0000", 2024-09-13T10:12:24.625 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.7777463519999994, 2024-09-13T10:12:24.625 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.018916507999999999, 2024-09-13T10:12:24.625 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.625 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.625 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.626 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.626 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.626 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702115+0000", 2024-09-13T10:12:24.626 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.626 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.626 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702115+0000", 2024-09-13T10:12:24.626 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.626 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.626 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.627 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.627 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702116+0000", 2024-09-13T10:12:24.627 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3200000000000001e-06 2024-09-13T10:12:24.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.627 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.627 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.627 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702924+0000", 2024-09-13T10:12:24.627 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00080727999999999996 2024-09-13T10:12:24.627 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.627 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.628 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.628 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702942+0000", 2024-09-13T10:12:24.628 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7903000000000001e-05 2024-09-13T10:12:24.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.628 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.628 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.628 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702952+0000", 2024-09-13T10:12:24.628 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.076e-05 2024-09-13T10:12:24.628 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.628 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.628 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.628 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.716127+0000", 2024-09-13T10:12:24.629 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.013174455999999999 2024-09-13T10:12:24.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.629 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.629 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.716430+0000", 2024-09-13T10:12:24.629 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00030353200000000002 2024-09-13T10:12:24.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.629 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.629 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.629 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.716892+0000", 2024-09-13T10:12:24.629 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00046132900000000001 2024-09-13T10:12:24.629 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.630 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.720049+0000", 2024-09-13T10:12:24.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0031571139999999999 2024-09-13T10:12:24.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.630 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.720083+0000", 2024-09-13T10:12:24.630 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.4508999999999998e-05 2024-09-13T10:12:24.630 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.630 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.630 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.630 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.720895+0000", 2024-09-13T10:12:24.631 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00081174700000000003 2024-09-13T10:12:24.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.631 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.631 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.720972+0000", 2024-09-13T10:12:24.631 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.7113000000000005e-05 2024-09-13T10:12:24.631 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.631 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.631 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.631 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.721032+0000", 2024-09-13T10:12:24.631 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9444999999999998e-05 2024-09-13T10:12:24.631 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.632 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.632 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.632 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.632 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.632 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.13.21:114 3.3 3:c560edf3:::05229772e6b32f13629a0acc34dce638ea214bd4aa8c7e9fffd3aed735eaea96:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e106)", 2024-09-13T10:12:24.632 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:14.702971+0000", 2024-09-13T10:12:24.632 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 9.7768908890000006, 2024-09-13T10:12:24.632 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.018655682999999999, 2024-09-13T10:12:24.632 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:24.632 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:24.632 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:24.632 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:24.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.633 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:24.633 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702971+0000", 2024-09-13T10:12:24.633 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.633 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:24.633 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702971+0000", 2024-09-13T10:12:24.633 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:24.633 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.633 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.633 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:24.634 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702972+0000", 2024-09-13T10:12:24.634 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.04e-06 2024-09-13T10:12:24.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.634 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.634 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:24.634 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.702994+0000", 2024-09-13T10:12:24.634 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2246e-05 2024-09-13T10:12:24.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.634 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.634 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:24.634 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.703004+0000", 2024-09-13T10:12:24.634 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0223e-05 2024-09-13T10:12:24.634 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.635 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:24.635 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.703013+0000", 2024-09-13T10:12:24.635 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.5469999999999997e-06 2024-09-13T10:12:24.635 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.635 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:24.635 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.718316+0000", 2024-09-13T10:12:24.635 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015303186 2024-09-13T10:12:24.635 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.635 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.635 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:24.635 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.718538+0000", 2024-09-13T10:12:24.636 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022254600000000001 2024-09-13T10:12:24.636 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.636 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.636 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 1,8", 2024-09-13T10:12:24.636 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.718956+0000", 2024-09-13T10:12:24.636 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00041724999999999998 2024-09-13T10:12:24.636 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.636 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.636 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:24.636 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.721059+0000", 2024-09-13T10:12:24.636 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0021034040000000001 2024-09-13T10:12:24.636 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.636 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.637 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.637 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.721124+0000", 2024-09-13T10:12:24.637 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4635999999999996e-05 2024-09-13T10:12:24.637 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.637 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.637 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:24.637 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.721481+0000", 2024-09-13T10:12:24.637 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000357647 2024-09-13T10:12:24.637 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.637 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.637 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:24.637 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.721564+0000", 2024-09-13T10:12:24.638 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.2738999999999995e-05 2024-09-13T10:12:24.638 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:24.638 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:24.638 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:24.638 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:14.721626+0000", 2024-09-13T10:12:24.638 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.2218999999999995e-05 2024-09-13T10:12:24.638 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.638 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.638 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.638 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:24.638 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:24.638 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:24.641 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-13T10:12:24.643 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3256687 2024-09-13T10:12:24.643 INFO:tasks.rados.rados.0.smithi195.stdout:1984: finishing write tid 3 to smithi19530714-27 2024-09-13T10:12:24.643 INFO:tasks.rados.rados.0.smithi195.stdout:1984: finishing write tid 4 to smithi19530714-27 2024-09-13T10:12:24.643 INFO:tasks.rados.rados.0.smithi195.stdout:1984: finishing write tid 5 to smithi19530714-27 2024-09-13T10:12:24.643 INFO:tasks.rados.rados.0.smithi195.stdout:1984: oid 27 updating version 0 to 175 2024-09-13T10:12:24.643 INFO:tasks.rados.rados.0.smithi195.stdout:1984: oid 27 updating version 175 to 176 2024-09-13T10:12:24.643 INFO:tasks.rados.rados.0.smithi195.stdout:1984: oid 27 updating version 176 to 177 2024-09-13T10:12:24.643 INFO:tasks.rados.rados.0.smithi195.stdout:1984: oid 27 version 177 is already newer than 174 2024-09-13T10:12:24.643 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 27 v 177 (ObjNum 444 snap 0 seq_num 444) dirty exists 2024-09-13T10:12:24.643 INFO:tasks.rados.rados.0.smithi195.stdout:1984: left oid 27 (ObjNum 444 snap 0 seq_num 444) 2024-09-13T10:12:24.643 INFO:tasks.rados.rados.0.smithi195.stdout:1986: finishing write tid 1 to smithi19530714-41 2024-09-13T10:12:24.643 INFO:tasks.rados.rados.0.smithi195.stdout:1986: finishing write tid 2 to smithi19530714-41 2024-09-13T10:12:24.644 INFO:tasks.rados.rados.0.smithi195.stdout:1986: finishing write tid 3 to smithi19530714-41 2024-09-13T10:12:24.644 INFO:tasks.rados.rados.0.smithi195.stdout:1986: finishing write tid 4 to smithi19530714-41 2024-09-13T10:12:24.644 INFO:tasks.rados.rados.0.smithi195.stdout:1986: finishing write tid 5 to smithi19530714-41 2024-09-13T10:12:24.644 INFO:tasks.rados.rados.0.smithi195.stdout:1986: oid 41 updating version 0 to 77 2024-09-13T10:12:24.644 INFO:tasks.rados.rados.0.smithi195.stdout:1986: oid 41 version 77 is already newer than 75 2024-09-13T10:12:24.644 INFO:tasks.rados.rados.0.smithi195.stdout:1986: oid 41 version 77 is already newer than 76 2024-09-13T10:12:24.644 INFO:tasks.rados.rados.0.smithi195.stdout:1986: oid 41 version 77 is already newer than 74 2024-09-13T10:12:24.644 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 41 v 77 (ObjNum 445 snap 0 seq_num 445) dirty exists 2024-09-13T10:12:24.644 INFO:tasks.rados.rados.0.smithi195.stdout:1986: left oid 41 (ObjNum 445 snap 0 seq_num 445) 2024-09-13T10:12:24.644 INFO:tasks.rados.rados.0.smithi195.stdout:1979: done (8 left) 2024-09-13T10:12:24.644 INFO:tasks.rados.rados.0.smithi195.stdout:1980: done (7 left) 2024-09-13T10:12:24.644 INFO:tasks.rados.rados.0.smithi195.stdout:1981: done (6 left) 2024-09-13T10:12:24.644 INFO:tasks.rados.rados.0.smithi195.stdout:1982: done (5 left) 2024-09-13T10:12:24.645 INFO:tasks.rados.rados.0.smithi195.stdout:1983: done (4 left) 2024-09-13T10:12:24.645 INFO:tasks.rados.rados.0.smithi195.stdout:1984: done (3 left) 2024-09-13T10:12:24.645 INFO:tasks.rados.rados.0.smithi195.stdout:1985: done (2 left) 2024-09-13T10:12:24.645 INFO:tasks.rados.rados.0.smithi195.stdout:1986: done (1 left) 2024-09-13T10:12:24.645 INFO:tasks.rados.rados.0.smithi195.stdout:1987: done (0 left) 2024-09-13T10:12:24.645 INFO:tasks.rados.rados.0.smithi195.stdout:1988: read oid 34 snap -1 2024-09-13T10:12:24.645 INFO:tasks.rados.rados.0.smithi195.stdout:1988: expect deleted 2024-09-13T10:12:24.645 INFO:tasks.rados.rados.0.smithi195.stdout:1989: write oid 29 current snap is 0 2024-09-13T10:12:24.645 INFO:tasks.rados.rados.0.smithi195.stdout:1989: seq_num 447 ranges {712729=784777,2091098=467491,3342313=170416} 2024-09-13T10:12:24.645 INFO:tasks.rados.rados.0.smithi195.stdout:1989: writing smithi19530714-29 from 712729 to 1497506 tid 1 2024-09-13T10:12:24.645 INFO:tasks.rados.rados.0.smithi195.stdout:1989: writing smithi19530714-29 from 2091098 to 2558589 tid 2 2024-09-13T10:12:24.645 INFO:tasks.rados.rados.0.smithi195.stdout:1989: writing smithi19530714-29 from 3342313 to 3512729 tid 3 2024-09-13T10:12:24.645 INFO:tasks.rados.rados.0.smithi195.stdout:1990: write oid 49 current snap is 0 2024-09-13T10:12:24.646 INFO:tasks.rados.rados.0.smithi195.stdout:1989: finishing write tid 1 to smithi19530714-29 2024-09-13T10:12:24.646 INFO:tasks.rados.rados.0.smithi195.stdout:1989: finishing write tid 2 to smithi19530714-29 2024-09-13T10:12:24.646 INFO:tasks.rados.rados.0.smithi195.stdout:1990: seq_num 448 ranges {729536=460026,1749580=611149,2853383=452450} 2024-09-13T10:12:24.646 INFO:tasks.rados.rados.0.smithi195.stdout:1990: writing smithi19530714-49 from 729536 to 1189562 tid 1 2024-09-13T10:12:24.646 INFO:tasks.rados.rados.0.smithi195.stdout:1990: writing smithi19530714-49 from 1749580 to 2360729 tid 2 2024-09-13T10:12:24.646 INFO:tasks.rados.rados.0.smithi195.stdout:1990: writing smithi19530714-49 from 2853383 to 3305833 tid 3 2024-09-13T10:12:24.646 INFO:tasks.rados.rados.0.smithi195.stdout:1988: done (2 left) 2024-09-13T10:12:24.646 INFO:tasks.rados.rados.0.smithi195.stdout:1991: write oid 16 current snap is 0 2024-09-13T10:12:24.646 INFO:tasks.rados.rados.0.smithi195.stdout:1989: finishing write tid 3 to smithi19530714-29 2024-09-13T10:12:24.646 INFO:tasks.rados.rados.0.smithi195.stdout:1989: finishing write tid 4 to smithi19530714-29 2024-09-13T10:12:24.646 INFO:tasks.rados.rados.0.smithi195.stdout:1989: finishing write tid 5 to smithi19530714-29 2024-09-13T10:12:24.646 INFO:tasks.rados.rados.0.smithi195.stdout:1989: oid 29 updating version 0 to 150 2024-09-13T10:12:24.647 INFO:tasks.rados.rados.0.smithi195.stdout:1989: oid 29 updating version 150 to 152 2024-09-13T10:12:24.647 INFO:tasks.rados.rados.0.smithi195.stdout:1989: oid 29 version 152 is already newer than 151 2024-09-13T10:12:24.647 INFO:tasks.rados.rados.0.smithi195.stdout:1989: oid 29 version 152 is already newer than 149 2024-09-13T10:12:24.647 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 29 v 152 (ObjNum 447 snap 0 seq_num 447) dirty exists 2024-09-13T10:12:24.647 INFO:tasks.rados.rados.0.smithi195.stdout:1989: left oid 29 (ObjNum 447 snap 0 seq_num 447) 2024-09-13T10:12:24.647 INFO:tasks.rados.rados.0.smithi195.stdout:1990: finishing write tid 1 to smithi19530714-49 2024-09-13T10:12:24.647 INFO:tasks.rados.rados.0.smithi195.stdout:1990: finishing write tid 2 to smithi19530714-49 2024-09-13T10:12:24.647 INFO:tasks.rados.rados.0.smithi195.stdout:1991: seq_num 449 ranges {746343=651628,1924415=754807,3397159=149184} 2024-09-13T10:12:24.648 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:24.638+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:24.648 INFO:tasks.rados.rados.0.smithi195.stdout:1991: writing smithi19530714-16 from 746343 to 1397971 tid 1 2024-09-13T10:12:24.661 INFO:tasks.rados.rados.0.smithi195.stdout:1991: writing smithi19530714-16 from 1924415 to 2679222 tid 2 2024-09-13T10:12:24.669 INFO:tasks.rados.rados.0.smithi195.stdout:1991: writing smithi19530714-16 from 3397159 to 3546343 tid 3 2024-09-13T10:12:24.669 INFO:tasks.rados.rados.0.smithi195.stdout:1989: done (2 left) 2024-09-13T10:12:24.669 INFO:tasks.rados.rados.0.smithi195.stdout:1992: read oid 38 snap -1 2024-09-13T10:12:24.669 INFO:tasks.rados.rados.0.smithi195.stdout:1992: expect (ObjNum 441 snap 0 seq_num 441) 2024-09-13T10:12:24.669 INFO:tasks.rados.rados.0.smithi195.stdout:1993: set_chunk oid 13 target oid 2024-09-13T10:12:24.699 INFO:tasks.rados.rados.0.smithi195.stdout:1993: set_chunk oid 13 offset: 1057792 length: 2187776 target oid d23670181e0a8847f279e770592cac00fb38dbe69fde5b26d2a9f0d209364995 offset: 0 snap -1 2024-09-13T10:12:24.699 INFO:tasks.rados.rados.0.smithi195.stdout:1994: tier_flush oid 10 2024-09-13T10:12:24.700 INFO:tasks.rados.rados.0.smithi195.stdout:1994: tier_flush oid 10 snap -1 2024-09-13T10:12:24.700 INFO:tasks.rados.rados.0.smithi195.stdout:1990: finishing write tid 3 to smithi19530714-49 2024-09-13T10:12:24.700 INFO:tasks.rados.rados.0.smithi195.stdout:1995: read oid 1 snap -1 2024-09-13T10:12:24.700 INFO:tasks.rados.rados.0.smithi195.stdout:1995: expect (ObjNum 405 snap 0 seq_num 405) 2024-09-13T10:12:24.700 INFO:tasks.rados.rados.0.smithi195.stdout:1996: write oid 43 current snap is 0 2024-09-13T10:12:24.700 INFO:tasks.rados.rados.0.smithi195.stdout:1990: finishing write tid 4 to smithi19530714-49 2024-09-13T10:12:24.700 INFO:tasks.rados.rados.0.smithi195.stdout:1990: finishing write tid 5 to smithi19530714-49 2024-09-13T10:12:24.700 INFO:tasks.rados.rados.0.smithi195.stdout:1990: oid 49 updating version 0 to 208 2024-09-13T10:12:24.700 INFO:tasks.rados.rados.0.smithi195.stdout:1990: oid 49 version 208 is already newer than 206 2024-09-13T10:12:24.700 INFO:tasks.rados.rados.0.smithi195.stdout:1990: oid 49 version 208 is already newer than 207 2024-09-13T10:12:24.700 INFO:tasks.rados.rados.0.smithi195.stdout:1990: oid 49 version 208 is already newer than 205 2024-09-13T10:12:24.700 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 49 v 208 (ObjNum 448 snap 0 seq_num 448) dirty exists 2024-09-13T10:12:24.700 INFO:tasks.rados.rados.0.smithi195.stdout:1990: left oid 49 (ObjNum 448 snap 0 seq_num 448) 2024-09-13T10:12:24.701 INFO:tasks.rados.rados.0.smithi195.stdout:1991: finishing write tid 1 to smithi19530714-16 2024-09-13T10:12:24.701 INFO:tasks.rados.rados.0.smithi195.stdout:1991: finishing write tid 2 to smithi19530714-16 2024-09-13T10:12:24.701 INFO:tasks.rados.rados.0.smithi195.stdout:1991: finishing write tid 3 to smithi19530714-16 2024-09-13T10:12:24.701 INFO:tasks.rados.rados.0.smithi195.stdout:1991: finishing write tid 4 to smithi19530714-16 2024-09-13T10:12:24.701 INFO:tasks.rados.rados.0.smithi195.stdout:1991: finishing write tid 5 to smithi19530714-16 2024-09-13T10:12:24.701 INFO:tasks.rados.rados.0.smithi195.stdout:1991: oid 16 updating version 0 to 65 2024-09-13T10:12:24.701 INFO:tasks.rados.rados.0.smithi195.stdout:1991: oid 16 version 65 is already newer than 64 2024-09-13T10:12:24.701 INFO:tasks.rados.rados.0.smithi195.stdout:1991: oid 16 version 65 is already newer than 63 2024-09-13T10:12:24.701 INFO:tasks.rados.rados.0.smithi195.stdout:1991: oid 16 version 65 is already newer than 62 2024-09-13T10:12:24.701 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 16 v 65 (ObjNum 449 snap 0 seq_num 449) dirty exists 2024-09-13T10:12:24.701 INFO:tasks.rados.rados.0.smithi195.stdout:1991: left oid 16 (ObjNum 449 snap 0 seq_num 449) 2024-09-13T10:12:24.701 INFO:tasks.rados.rados.0.smithi195.stdout:1996: seq_num 450 ranges {763150=726877,1982897=614818,3024582=474994} 2024-09-13T10:12:24.713 INFO:tasks.rados.rados.0.smithi195.stdout:1996: writing smithi19530714-43 from 763150 to 1490027 tid 1 2024-09-13T10:12:24.724 INFO:tasks.rados.rados.0.smithi195.stdout:1996: writing smithi19530714-43 from 1982897 to 2597715 tid 2 2024-09-13T10:12:24.732 INFO:tasks.rados.rados.0.smithi195.stdout:1996: writing smithi19530714-43 from 3024582 to 3499576 tid 3 2024-09-13T10:12:24.732 INFO:tasks.rados.rados.0.smithi195.stdout:1990: done (6 left) 2024-09-13T10:12:24.732 INFO:tasks.rados.rados.0.smithi195.stdout:1991: done (5 left) 2024-09-13T10:12:24.732 INFO:tasks.rados.rados.0.smithi195.stdout:1997: set_chunk oid 12 target oid 2024-09-13T10:12:24.740 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:24.738+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:24.751 INFO:tasks.rados.rados.0.smithi195.stdout:1997: set_chunk oid 12 offset: 845312 length: 1506816 target oid 84dd1b89f4e5c2434f60ae90d446811cf70cab68d0b7408ea958401f533d19de offset: 0 snap -1 2024-09-13T10:12:24.751 INFO:tasks.rados.rados.0.smithi195.stdout:1998: write oid 17 current snap is 0 2024-09-13T10:12:24.751 INFO:tasks.rados.rados.0.smithi195.stdout:1998: seq_num 451 ranges {779957=402126,1925026=758476,3335652=244305} 2024-09-13T10:12:24.761 INFO:tasks.rados.rados.0.smithi195.stdout:1998: writing smithi19530714-17 from 779957 to 1182083 tid 1 2024-09-13T10:12:24.774 INFO:tasks.rados.rados.0.smithi195.stdout:1998: writing smithi19530714-17 from 1925026 to 2683502 tid 2 2024-09-13T10:12:24.781 INFO:tasks.rados.rados.0.smithi195.stdout:1998: writing smithi19530714-17 from 3335652 to 3579957 tid 3 2024-09-13T10:12:24.781 INFO:tasks.rados.rados.0.smithi195.stdout:1999: set_chunk oid 25 target oid 2024-09-13T10:12:24.787 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:12:24.787 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:12:24.787 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:24.793 INFO:tasks.rados.rados.0.smithi195.stdout:1999: set_chunk oid 25 offset: 1084416 length: 240128 target oid 839169da9a77cfa37b7e3010b68bbb1d269a136002f95ab6e5aa6688fb76fe70 offset: 0 snap -1 2024-09-13T10:12:24.793 INFO:tasks.rados.rados.0.smithi195.stdout:2000: read oid 35 snap -1 2024-09-13T10:12:24.793 INFO:tasks.rados.rados.0.smithi195.stdout:2000: expect (ObjNum 409 snap 0 seq_num 409) 2024-09-13T10:12:24.794 INFO:tasks.rados.rados.0.smithi195.stdout:2001: write (excl) oid 32 current snap is 0 2024-09-13T10:12:24.794 INFO:tasks.rados.rados.0.smithi195.stdout:1992: expect (ObjNum 441 snap 0 seq_num 441) 2024-09-13T10:12:24.803 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-13T10:12:24.841 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:24.837+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:24.876 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1777489 2024-09-13T10:12:24.927 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:24.927 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:12:24.927 INFO:teuthology.orchestra.run.smithi181.stdout: "complaint_time": 30, 2024-09-13T10:12:24.927 INFO:teuthology.orchestra.run.smithi181.stdout: "num_blocked_ops": 0 2024-09-13T10:12:24.927 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:24.942 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:24.938+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:24.943 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-13T10:12:25.042 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:25.037+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:25.045 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:25.070 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:25.070 INFO:teuthology.orchestra.run.smithi181.stdout: "size": 20, 2024-09-13T10:12:25.070 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 600, 2024-09-13T10:12:25.070 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [ 2024-09-13T10:12:25.070 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.070 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:202 3.2 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.070 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.991310+0000", 2024-09-13T10:12:25.070 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 94.073749789000004, 2024-09-13T10:12:25.070 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.040945251000000002, 2024-09-13T10:12:25.070 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.070 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.071 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.071 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.071 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.071 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.071 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 202 2024-09-13T10:12:25.071 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.071 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.071 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.071 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.071 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991310+0000", 2024-09-13T10:12:25.071 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.071 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.072 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.072 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.072 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991310+0000", 2024-09-13T10:12:25.072 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.072 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.072 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.072 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.072 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991314+0000", 2024-09-13T10:12:25.072 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.7100000000000001e-06 2024-09-13T10:12:25.072 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.072 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.072 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.072 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991328+0000", 2024-09-13T10:12:25.073 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.413e-05 2024-09-13T10:12:25.073 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.073 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.073 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.073 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991361+0000", 2024-09-13T10:12:25.073 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2512999999999999e-05 2024-09-13T10:12:25.073 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.073 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.073 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.073 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991377+0000", 2024-09-13T10:12:25.073 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6478999999999999e-05 2024-09-13T10:12:25.073 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.073 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.074 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.074 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991546+0000", 2024-09-13T10:12:25.074 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016856600000000001 2024-09-13T10:12:25.074 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.074 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.074 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.074 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991850+0000", 2024-09-13T10:12:25.074 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00030478800000000001 2024-09-13T10:12:25.074 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.074 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.074 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.074 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.001416+0000", 2024-09-13T10:12:25.075 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0095659279999999996 2024-09-13T10:12:25.075 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.075 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.075 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.075 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031502+0000", 2024-09-13T10:12:25.075 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.030085526000000001 2024-09-13T10:12:25.075 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.075 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.075 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.075 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031585+0000", 2024-09-13T10:12:25.075 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.3175000000000002e-05 2024-09-13T10:12:25.075 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.075 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.076 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.076 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031775+0000", 2024-09-13T10:12:25.076 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000190161 2024-09-13T10:12:25.076 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.076 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.076 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.076 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.031982+0000", 2024-09-13T10:12:25.076 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000207245 2024-09-13T10:12:25.076 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.076 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.076 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.076 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032255+0000", 2024-09-13T10:12:25.076 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00027303000000000001 2024-09-13T10:12:25.076 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.077 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.077 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.077 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.077 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.077 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:206 3.2 3:4cbdbe9e:::smithi19530714-7 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.077 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.991649+0000", 2024-09-13T10:12:25.077 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 94.073411461000006, 2024-09-13T10:12:25.077 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041234240999999998, 2024-09-13T10:12:25.077 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.077 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.077 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.077 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.077 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.078 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.078 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 206 2024-09-13T10:12:25.078 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.078 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.078 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.078 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.078 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991649+0000", 2024-09-13T10:12:25.078 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.078 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.078 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.078 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.078 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991649+0000", 2024-09-13T10:12:25.078 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.078 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.079 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.079 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.079 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991650+0000", 2024-09-13T10:12:25.079 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.483e-06 2024-09-13T10:12:25.079 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.079 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.079 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.079 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991657+0000", 2024-09-13T10:12:25.079 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.0899999999999999e-06 2024-09-13T10:12:25.079 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.079 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.079 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.079 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991672+0000", 2024-09-13T10:12:25.079 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4514e-05 2024-09-13T10:12:25.080 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.080 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.080 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.080 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.991681+0000", 2024-09-13T10:12:25.080 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.6360000000000006e-06 2024-09-13T10:12:25.080 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.080 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.080 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.080 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992159+0000", 2024-09-13T10:12:25.080 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00047820099999999998 2024-09-13T10:12:25.080 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.080 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.081 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.081 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992411+0000", 2024-09-13T10:12:25.081 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00025199700000000001 2024-09-13T10:12:25.081 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.081 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.081 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.081 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.009775+0000", 2024-09-13T10:12:25.081 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017363841000000001 2024-09-13T10:12:25.081 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.081 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.081 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.081 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032446+0000", 2024-09-13T10:12:25.081 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.022670490000000001 2024-09-13T10:12:25.082 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.082 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.082 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.082 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032482+0000", 2024-09-13T10:12:25.082 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5902000000000003e-05 2024-09-13T10:12:25.082 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.082 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.082 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.082 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032612+0000", 2024-09-13T10:12:25.084 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013022700000000001 2024-09-13T10:12:25.084 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.084 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.084 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.084 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032767+0000", 2024-09-13T10:12:25.084 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015477000000000001 2024-09-13T10:12:25.084 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.084 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.085 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.085 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032883+0000", 2024-09-13T10:12:25.085 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00011608999999999999 2024-09-13T10:12:25.085 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.085 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.085 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.085 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.085 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.085 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:210 3.2 3:46ab5247:::smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.085 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.992337+0000", 2024-09-13T10:12:25.085 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 94.072723318000001, 2024-09-13T10:12:25.086 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.057985696000000003, 2024-09-13T10:12:25.086 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.086 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.086 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.086 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.086 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.086 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.086 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 210 2024-09-13T10:12:25.086 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.086 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.086 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.086 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.086 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992337+0000", 2024-09-13T10:12:25.087 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.087 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.087 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.087 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.087 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992337+0000", 2024-09-13T10:12:25.087 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.087 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.087 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.087 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.087 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992339+0000", 2024-09-13T10:12:25.087 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2400000000000002e-06 2024-09-13T10:12:25.087 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.087 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.088 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.088 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992350+0000", 2024-09-13T10:12:25.088 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1259e-05 2024-09-13T10:12:25.088 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.088 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.088 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.088 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992368+0000", 2024-09-13T10:12:25.088 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.8133000000000001e-05 2024-09-13T10:12:25.088 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.088 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.088 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.088 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992382+0000", 2024-09-13T10:12:25.088 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3837e-05 2024-09-13T10:12:25.089 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.089 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.089 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.089 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992752+0000", 2024-09-13T10:12:25.089 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00037008000000000001 2024-09-13T10:12:25.089 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.089 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.089 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.089 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992980+0000", 2024-09-13T10:12:25.089 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022799100000000001 2024-09-13T10:12:25.089 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.089 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.090 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.090 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027651+0000", 2024-09-13T10:12:25.090 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.034671027 2024-09-13T10:12:25.090 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.090 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.090 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.090 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.034868+0000", 2024-09-13T10:12:25.090 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0072169920000000002 2024-09-13T10:12:25.090 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.090 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.090 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.090 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038033+0000", 2024-09-13T10:12:25.091 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.003164966 2024-09-13T10:12:25.091 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.091 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.091 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.091 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050187+0000", 2024-09-13T10:12:25.091 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012153815 2024-09-13T10:12:25.091 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.091 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.091 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.091 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050266+0000", 2024-09-13T10:12:25.091 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9359999999999999e-05 2024-09-13T10:12:25.091 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.091 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.092 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.092 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.050322+0000", 2024-09-13T10:12:25.092 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.5995999999999997e-05 2024-09-13T10:12:25.092 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.092 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.092 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.092 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.092 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.092 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:214 3.2 3:68ba95ba:::smithi19530714-15 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.092 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:50.992479+0000", 2024-09-13T10:12:25.092 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 94.072581244999995, 2024-09-13T10:12:25.092 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.045987867000000002, 2024-09-13T10:12:25.093 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.093 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.093 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.093 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.093 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.093 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.093 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 214 2024-09-13T10:12:25.093 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.093 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.093 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.093 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.093 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992479+0000", 2024-09-13T10:12:25.094 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.094 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.094 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.094 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.094 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992479+0000", 2024-09-13T10:12:25.094 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.094 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.094 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.094 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.094 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992481+0000", 2024-09-13T10:12:25.094 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.2960000000000001e-06 2024-09-13T10:12:25.094 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.094 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.095 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.095 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992493+0000", 2024-09-13T10:12:25.095 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2077e-05 2024-09-13T10:12:25.095 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.095 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.095 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.095 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992519+0000", 2024-09-13T10:12:25.095 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6282e-05 2024-09-13T10:12:25.095 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.095 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.095 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.095 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.992538+0000", 2024-09-13T10:12:25.096 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9083e-05 2024-09-13T10:12:25.096 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.096 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.096 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.096 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993393+0000", 2024-09-13T10:12:25.096 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00085445200000000001 2024-09-13T10:12:25.096 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.096 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.096 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.096 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:50.993682+0000", 2024-09-13T10:12:25.096 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000288895 2024-09-13T10:12:25.096 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.097 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.097 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.097 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.019700+0000", 2024-09-13T10:12:25.097 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.02601794 2024-09-13T10:12:25.097 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.097 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.097 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.097 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.032498+0000", 2024-09-13T10:12:25.097 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012798535999999999 2024-09-13T10:12:25.097 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.097 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.097 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.098 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033641+0000", 2024-09-13T10:12:25.098 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001142569 2024-09-13T10:12:25.098 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.098 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.098 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.098 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038259+0000", 2024-09-13T10:12:25.098 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0046180939999999997 2024-09-13T10:12:25.098 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.098 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.098 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.099 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038379+0000", 2024-09-13T10:12:25.099 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000119914 2024-09-13T10:12:25.099 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.099 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.099 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.099 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.038467+0000", 2024-09-13T10:12:25.099 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7729e-05 2024-09-13T10:12:25.099 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.100 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.100 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.100 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.100 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.100 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:217 3.2 3:554c406d:::smithi19530714-18 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.101 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.027515+0000", 2024-09-13T10:12:25.101 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 94.037545410999996, 2024-09-13T10:12:25.101 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032787180999999999, 2024-09-13T10:12:25.101 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.101 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.101 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.101 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.101 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.101 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.101 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 217 2024-09-13T10:12:25.101 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.101 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.102 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.102 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.102 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027515+0000", 2024-09-13T10:12:25.102 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.102 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.102 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.102 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.102 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027515+0000", 2024-09-13T10:12:25.102 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.102 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.102 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.103 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.103 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027518+0000", 2024-09-13T10:12:25.103 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.4429999999999999e-06 2024-09-13T10:12:25.103 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.103 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.103 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.103 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027532+0000", 2024-09-13T10:12:25.103 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3517e-05 2024-09-13T10:12:25.103 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.103 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.103 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.103 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027563+0000", 2024-09-13T10:12:25.104 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1158999999999998e-05 2024-09-13T10:12:25.104 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.104 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.104 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.104 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.027585+0000", 2024-09-13T10:12:25.104 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.228e-05 2024-09-13T10:12:25.104 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.104 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.104 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.104 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033014+0000", 2024-09-13T10:12:25.104 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0054288649999999997 2024-09-13T10:12:25.104 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.104 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.033269+0000", 2024-09-13T10:12:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000255054 2024-09-13T10:12:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.044618+0000", 2024-09-13T10:12:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.011349391 2024-09-13T10:12:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.054627+0000", 2024-09-13T10:12:25.105 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010008992 2024-09-13T10:12:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.058047+0000", 2024-09-13T10:12:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0034193000000000001 2024-09-13T10:12:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060139+0000", 2024-09-13T10:12:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020925240000000001 2024-09-13T10:12:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.106 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060222+0000", 2024-09-13T10:12:25.107 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.2917999999999995e-05 2024-09-13T10:12:25.107 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.107 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.107 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.107 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060302+0000", 2024-09-13T10:12:25.107 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.9738000000000001e-05 2024-09-13T10:12:25.107 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.107 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.107 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.107 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.107 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.107 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:220 3.2 3:753d5785:::smithi19530714-21 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.107 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.035716+0000", 2024-09-13T10:12:25.107 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 94.029343650000001, 2024-09-13T10:12:25.107 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.030171203000000001, 2024-09-13T10:12:25.108 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.108 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.108 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.108 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.108 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.108 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.108 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 220 2024-09-13T10:12:25.108 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.108 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.108 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.108 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.108 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035716+0000", 2024-09-13T10:12:25.108 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.109 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.109 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.109 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.109 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035716+0000", 2024-09-13T10:12:25.109 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.109 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.109 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.109 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.109 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035720+0000", 2024-09-13T10:12:25.109 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5310000000000002e-06 2024-09-13T10:12:25.109 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.109 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.109 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.109 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035735+0000", 2024-09-13T10:12:25.110 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5517000000000001e-05 2024-09-13T10:12:25.110 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.110 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.110 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.110 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035765+0000", 2024-09-13T10:12:25.110 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.9771e-05 2024-09-13T10:12:25.110 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.110 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.110 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.110 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035786+0000", 2024-09-13T10:12:25.110 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.0928e-05 2024-09-13T10:12:25.110 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.111 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.111 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.111 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.035945+0000", 2024-09-13T10:12:25.111 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015871900000000001 2024-09-13T10:12:25.111 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.111 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.111 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.111 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.036337+0000", 2024-09-13T10:12:25.111 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00039231299999999998 2024-09-13T10:12:25.111 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.111 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.111 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.112 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.051063+0000", 2024-09-13T10:12:25.112 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014725607 2024-09-13T10:12:25.112 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.112 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.112 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.112 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.055802+0000", 2024-09-13T10:12:25.112 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0047387929999999998 2024-09-13T10:12:25.112 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.112 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.112 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.112 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.064353+0000", 2024-09-13T10:12:25.112 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0085516629999999993 2024-09-13T10:12:25.112 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.112 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.113 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.113 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065750+0000", 2024-09-13T10:12:25.113 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001396748 2024-09-13T10:12:25.113 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.113 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.113 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.113 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065811+0000", 2024-09-13T10:12:25.113 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.0770000000000003e-05 2024-09-13T10:12:25.113 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.113 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.113 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.113 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065888+0000", 2024-09-13T10:12:25.113 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.6842999999999996e-05 2024-09-13T10:12:25.114 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.114 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.114 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.114 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.114 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.114 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:227 3.2 3:6ac7800c:::smithi19530714-28 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.114 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.060091+0000", 2024-09-13T10:12:25.114 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 94.004968585, 2024-09-13T10:12:25.114 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.034993362, 2024-09-13T10:12:25.114 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.114 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.115 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.115 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.115 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.115 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.115 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 227 2024-09-13T10:12:25.115 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.115 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.115 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.115 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.115 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060091+0000", 2024-09-13T10:12:25.115 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.116 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.116 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.116 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.116 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060091+0000", 2024-09-13T10:12:25.116 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.116 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.116 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.116 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.116 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060094+0000", 2024-09-13T10:12:25.116 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5330000000000001e-06 2024-09-13T10:12:25.116 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.116 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.117 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.117 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060103+0000", 2024-09-13T10:12:25.117 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.7020000000000007e-06 2024-09-13T10:12:25.117 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.117 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.117 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.117 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060121+0000", 2024-09-13T10:12:25.117 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.791e-05 2024-09-13T10:12:25.117 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.117 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.117 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.117 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060135+0000", 2024-09-13T10:12:25.117 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4059999999999999e-05 2024-09-13T10:12:25.118 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.118 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.118 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.118 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060346+0000", 2024-09-13T10:12:25.118 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021175600000000001 2024-09-13T10:12:25.118 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.118 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.118 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.118 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.060559+0000", 2024-09-13T10:12:25.118 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00021239499999999999 2024-09-13T10:12:25.118 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.118 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.118 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.119 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.073003+0000", 2024-09-13T10:12:25.119 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012444255 2024-09-13T10:12:25.119 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.119 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.119 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.119 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.081388+0000", 2024-09-13T10:12:25.119 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0083852119999999995 2024-09-13T10:12:25.119 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.119 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.119 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.119 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094289+0000", 2024-09-13T10:12:25.120 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012900271 2024-09-13T10:12:25.120 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.120 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.120 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.120 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094913+0000", 2024-09-13T10:12:25.120 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00062429200000000003 2024-09-13T10:12:25.120 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.120 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.120 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.120 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094972+0000", 2024-09-13T10:12:25.120 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9457999999999999e-05 2024-09-13T10:12:25.121 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.121 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.121 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.121 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095085+0000", 2024-09-13T10:12:25.121 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000112518 2024-09-13T10:12:25.121 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.121 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.121 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.121 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.121 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.121 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:230 3.2 3:5d108080:::smithi19530714-31 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.121 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.065002+0000", 2024-09-13T10:12:25.121 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 94.000058140999997, 2024-09-13T10:12:25.122 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.049477431000000002, 2024-09-13T10:12:25.122 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.122 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.122 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.122 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.122 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.122 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.122 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 230 2024-09-13T10:12:25.122 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.122 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.122 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.122 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.122 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065002+0000", 2024-09-13T10:12:25.122 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.123 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.123 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.123 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.123 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065002+0000", 2024-09-13T10:12:25.123 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.123 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.123 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.123 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.123 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065004+0000", 2024-09-13T10:12:25.123 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4269999999999998e-06 2024-09-13T10:12:25.123 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.123 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.123 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.124 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065014+0000", 2024-09-13T10:12:25.124 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.9799999999999993e-06 2024-09-13T10:12:25.124 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.124 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.124 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.124 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065038+0000", 2024-09-13T10:12:25.124 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4229000000000001e-05 2024-09-13T10:12:25.124 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.124 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.124 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.124 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065054+0000", 2024-09-13T10:12:25.124 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5764999999999999e-05 2024-09-13T10:12:25.124 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.124 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.125 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.125 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065171+0000", 2024-09-13T10:12:25.125 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000117032 2024-09-13T10:12:25.125 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.125 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.125 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.125 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065513+0000", 2024-09-13T10:12:25.125 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00034131 2024-09-13T10:12:25.125 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.125 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.125 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.125 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.087042+0000", 2024-09-13T10:12:25.126 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021529534999999999 2024-09-13T10:12:25.126 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.126 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.126 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.126 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.097903+0000", 2024-09-13T10:12:25.126 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010861282999999999 2024-09-13T10:12:25.126 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.126 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.126 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.127 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.099922+0000", 2024-09-13T10:12:25.127 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0020186539999999999 2024-09-13T10:12:25.127 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.127 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.127 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.127 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114391+0000", 2024-09-13T10:12:25.127 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014468880999999999 2024-09-13T10:12:25.127 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.127 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.127 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.127 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114434+0000", 2024-09-13T10:12:25.127 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.3511999999999999e-05 2024-09-13T10:12:25.128 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.128 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.128 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.128 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114479+0000", 2024-09-13T10:12:25.128 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4823000000000001e-05 2024-09-13T10:12:25.128 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.128 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.128 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.128 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.128 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.128 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:232 3.2 3:6538713c:::smithi19530714-33 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.128 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.065891+0000", 2024-09-13T10:12:25.128 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.999169269999996, 2024-09-13T10:12:25.128 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048804489999999999, 2024-09-13T10:12:25.129 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.129 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.129 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.129 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.129 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.129 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.129 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 232 2024-09-13T10:12:25.129 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.129 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.129 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.129 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.129 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065891+0000", 2024-09-13T10:12:25.130 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.130 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.130 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.130 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.130 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065891+0000", 2024-09-13T10:12:25.130 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.130 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.130 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.130 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.130 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065892+0000", 2024-09-13T10:12:25.130 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6249999999999999e-06 2024-09-13T10:12:25.130 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.130 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.130 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.131 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065902+0000", 2024-09-13T10:12:25.131 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.4150000000000005e-06 2024-09-13T10:12:25.131 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.131 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.131 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.131 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065920+0000", 2024-09-13T10:12:25.131 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7907e-05 2024-09-13T10:12:25.131 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.131 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.131 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.131 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.065932+0000", 2024-09-13T10:12:25.131 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2469999999999999e-05 2024-09-13T10:12:25.131 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.132 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.132 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.132 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.066072+0000", 2024-09-13T10:12:25.132 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013935999999999999 2024-09-13T10:12:25.132 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.132 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.133 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.133 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.066435+0000", 2024-09-13T10:12:25.133 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036323300000000002 2024-09-13T10:12:25.133 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.133 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.133 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.133 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091133+0000", 2024-09-13T10:12:25.133 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.024698494000000001 2024-09-13T10:12:25.133 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.133 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.134 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.134 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.101407+0000", 2024-09-13T10:12:25.134 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010273835 2024-09-13T10:12:25.134 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.134 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.134 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.134 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114342+0000", 2024-09-13T10:12:25.134 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012934826 2024-09-13T10:12:25.134 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.134 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.134 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.134 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114548+0000", 2024-09-13T10:12:25.135 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000206243 2024-09-13T10:12:25.135 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.135 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.135 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.135 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114630+0000", 2024-09-13T10:12:25.135 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 8.2127999999999997e-05 2024-09-13T10:12:25.135 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.135 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.135 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.135 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114695+0000", 2024-09-13T10:12:25.135 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.4954000000000002e-05 2024-09-13T10:12:25.135 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.135 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.135 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.136 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.136 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.136 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:236 3.2 3:73c2893b:::smithi19530714-37 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.136 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.076316+0000", 2024-09-13T10:12:25.136 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.988744148999999, 2024-09-13T10:12:25.136 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.041217193999999999, 2024-09-13T10:12:25.136 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.136 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.136 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.136 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.136 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.136 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.136 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 236 2024-09-13T10:12:25.137 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.137 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.137 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.137 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.137 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076316+0000", 2024-09-13T10:12:25.137 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.137 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.137 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.137 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.137 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076316+0000", 2024-09-13T10:12:25.137 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.137 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.138 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.138 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.138 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076319+0000", 2024-09-13T10:12:25.138 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1750000000000001e-06 2024-09-13T10:12:25.138 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.138 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.138 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.139 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076333+0000", 2024-09-13T10:12:25.139 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3862e-05 2024-09-13T10:12:25.139 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.139 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.139 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.139 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076359+0000", 2024-09-13T10:12:25.140 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.5641999999999999e-05 2024-09-13T10:12:25.140 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.140 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.140 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.140 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.076378+0000", 2024-09-13T10:12:25.140 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.9406999999999999e-05 2024-09-13T10:12:25.140 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.140 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.140 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.140 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.078831+0000", 2024-09-13T10:12:25.141 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0024532590000000002 2024-09-13T10:12:25.141 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.141 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.141 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.141 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.079137+0000", 2024-09-13T10:12:25.141 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00030614500000000002 2024-09-13T10:12:25.141 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.141 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.141 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.141 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.102393+0000", 2024-09-13T10:12:25.141 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023255175999999999 2024-09-13T10:12:25.141 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.142 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.142 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.142 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114728+0000", 2024-09-13T10:12:25.142 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012335376 2024-09-13T10:12:25.142 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.142 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.142 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.142 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114735+0000", 2024-09-13T10:12:25.142 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.2969999999999998e-06 2024-09-13T10:12:25.142 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.142 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.142 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.142 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117465+0000", 2024-09-13T10:12:25.143 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0027298019999999999 2024-09-13T10:12:25.143 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.143 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.143 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.143 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117501+0000", 2024-09-13T10:12:25.143 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.5837999999999998e-05 2024-09-13T10:12:25.143 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.143 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.143 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.143 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.117533+0000", 2024-09-13T10:12:25.143 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2215000000000002e-05 2024-09-13T10:12:25.143 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.143 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.144 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.144 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.144 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.144 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:240 3.2 3:7eee2f4a:::smithi19530714-41 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.144 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.091046+0000", 2024-09-13T10:12:25.144 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.974014307999994, 2024-09-13T10:12:25.144 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048376970999999998, 2024-09-13T10:12:25.144 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.144 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.144 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.144 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.144 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.144 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.145 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 240 2024-09-13T10:12:25.145 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.145 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.145 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.145 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.145 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091046+0000", 2024-09-13T10:12:25.145 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.145 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.145 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.145 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.145 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091046+0000", 2024-09-13T10:12:25.145 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.146 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.146 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.146 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.146 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091048+0000", 2024-09-13T10:12:25.146 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.6299999999999998e-06 2024-09-13T10:12:25.146 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.146 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.146 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.146 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091060+0000", 2024-09-13T10:12:25.146 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1229999999999999e-05 2024-09-13T10:12:25.146 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.146 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.146 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.147 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091075+0000", 2024-09-13T10:12:25.147 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5313000000000001e-05 2024-09-13T10:12:25.147 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.147 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.147 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.147 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.091085+0000", 2024-09-13T10:12:25.147 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0284e-05 2024-09-13T10:12:25.147 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.147 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.147 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.148 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094402+0000", 2024-09-13T10:12:25.148 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00331708 2024-09-13T10:12:25.148 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.148 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.148 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.148 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.094649+0000", 2024-09-13T10:12:25.148 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024695300000000001 2024-09-13T10:12:25.148 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.148 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.148 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.148 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.107510+0000", 2024-09-13T10:12:25.148 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.012860778999999999 2024-09-13T10:12:25.148 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.149 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.149 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.149 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114739+0000", 2024-09-13T10:12:25.149 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0072289700000000004 2024-09-13T10:12:25.149 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.149 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.149 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.150 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.118693+0000", 2024-09-13T10:12:25.150 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0039540560000000001 2024-09-13T10:12:25.150 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.150 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.150 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.150 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139282+0000", 2024-09-13T10:12:25.150 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.020589423999999999 2024-09-13T10:12:25.150 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.150 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.150 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.150 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139357+0000", 2024-09-13T10:12:25.150 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.4276999999999994e-05 2024-09-13T10:12:25.151 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.151 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.151 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.151 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139423+0000", 2024-09-13T10:12:25.151 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.5975000000000004e-05 2024-09-13T10:12:25.151 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.151 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.151 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.151 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.151 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.151 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:242 3.2 3:501d22a1:::smithi19530714-43 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.152 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.095846+0000", 2024-09-13T10:12:25.152 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.969214104000002, 2024-09-13T10:12:25.152 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.043813831999999997, 2024-09-13T10:12:25.152 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.152 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.152 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.153 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.153 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.153 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.153 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 242 2024-09-13T10:12:25.153 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.154 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.154 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.154 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.154 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095846+0000", 2024-09-13T10:12:25.154 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.154 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.154 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.154 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.154 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095846+0000", 2024-09-13T10:12:25.154 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.154 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.155 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.155 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.155 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095848+0000", 2024-09-13T10:12:25.155 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3760000000000002e-06 2024-09-13T10:12:25.155 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.155 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.155 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.155 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095859+0000", 2024-09-13T10:12:25.155 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0429e-05 2024-09-13T10:12:25.155 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.155 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.155 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.155 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095876+0000", 2024-09-13T10:12:25.156 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7504e-05 2024-09-13T10:12:25.156 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.156 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.156 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.156 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.095886+0000", 2024-09-13T10:12:25.156 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0182e-05 2024-09-13T10:12:25.156 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.156 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.156 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.156 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.096010+0000", 2024-09-13T10:12:25.156 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012336700000000001 2024-09-13T10:12:25.156 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.156 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.157 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.157 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.096252+0000", 2024-09-13T10:12:25.157 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00024189200000000001 2024-09-13T10:12:25.157 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.157 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.157 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.157 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.111492+0000", 2024-09-13T10:12:25.157 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.015239920000000001 2024-09-13T10:12:25.157 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.157 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.157 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.157 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115691+0000", 2024-09-13T10:12:25.158 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0041992410000000003 2024-09-13T10:12:25.158 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.158 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.158 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.158 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139492+0000", 2024-09-13T10:12:25.158 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.023801495999999998 2024-09-13T10:12:25.158 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.158 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.158 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.158 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139560+0000", 2024-09-13T10:12:25.158 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.8163000000000005e-05 2024-09-13T10:12:25.158 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.158 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.158 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.158 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139619+0000", 2024-09-13T10:12:25.159 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8678999999999998e-05 2024-09-13T10:12:25.159 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.159 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.159 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.159 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.139660+0000", 2024-09-13T10:12:25.159 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.0583000000000001e-05 2024-09-13T10:12:25.159 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.159 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.159 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.159 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.159 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.159 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:244 3.2 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.160 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.112158+0000", 2024-09-13T10:12:25.160 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.952901640999997, 2024-09-13T10:12:25.160 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.030058202999999999, 2024-09-13T10:12:25.160 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.160 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.160 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.160 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.160 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.160 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.160 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 244 2024-09-13T10:12:25.160 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.160 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.160 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.160 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.161 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112158+0000", 2024-09-13T10:12:25.161 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.161 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.161 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.161 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.161 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112158+0000", 2024-09-13T10:12:25.161 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.161 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.161 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.161 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.161 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112161+0000", 2024-09-13T10:12:25.161 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.384e-06 2024-09-13T10:12:25.161 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.162 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.162 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.162 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112168+0000", 2024-09-13T10:12:25.162 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 6.7780000000000002e-06 2024-09-13T10:12:25.162 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.162 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.162 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.162 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112183+0000", 2024-09-13T10:12:25.162 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5917999999999999e-05 2024-09-13T10:12:25.162 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.162 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.162 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.162 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112195+0000", 2024-09-13T10:12:25.163 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.1578e-05 2024-09-13T10:12:25.163 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.163 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.163 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.163 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114766+0000", 2024-09-13T10:12:25.163 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0025710659999999999 2024-09-13T10:12:25.163 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.163 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.163 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.163 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.114995+0000", 2024-09-13T10:12:25.163 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022844 2024-09-13T10:12:25.163 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.164 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.164 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.164 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.125677+0000", 2024-09-13T10:12:25.164 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.010682677999999999 2024-09-13T10:12:25.164 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.164 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.164 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.164 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.140355+0000", 2024-09-13T10:12:25.164 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.014677773999999999 2024-09-13T10:12:25.164 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.164 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.165 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.165 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.141368+0000", 2024-09-13T10:12:25.165 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0010134040000000001 2024-09-13T10:12:25.165 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.165 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.165 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.165 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142112+0000", 2024-09-13T10:12:25.165 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00074347599999999997 2024-09-13T10:12:25.165 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.165 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.165 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.165 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142172+0000", 2024-09-13T10:12:25.166 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.9749e-05 2024-09-13T10:12:25.166 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.166 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.166 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.166 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142217+0000", 2024-09-13T10:12:25.166 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 4.4957999999999999e-05 2024-09-13T10:12:25.166 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.166 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.166 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.166 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.167 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.167 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:245 3.2 3:7837e849:::smithi19530714-46 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.167 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.112398+0000", 2024-09-13T10:12:25.167 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.952661583999998, 2024-09-13T10:12:25.167 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.039535731999999997, 2024-09-13T10:12:25.167 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.167 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.167 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.167 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.167 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.168 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.168 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 245 2024-09-13T10:12:25.168 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.168 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.168 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.168 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.168 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112398+0000", 2024-09-13T10:12:25.168 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.168 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.168 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.168 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.168 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112398+0000", 2024-09-13T10:12:25.168 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.169 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.169 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.169 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.169 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112399+0000", 2024-09-13T10:12:25.169 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.0470000000000001e-06 2024-09-13T10:12:25.169 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.169 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.169 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.169 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112405+0000", 2024-09-13T10:12:25.169 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.6899999999999997e-06 2024-09-13T10:12:25.169 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.170 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.170 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.170 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112417+0000", 2024-09-13T10:12:25.170 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2334000000000001e-05 2024-09-13T10:12:25.170 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.170 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.170 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.170 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.112427+0000", 2024-09-13T10:12:25.170 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.0929999999999992e-06 2024-09-13T10:12:25.170 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.170 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.170 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.171 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115182+0000", 2024-09-13T10:12:25.171 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0027552739999999998 2024-09-13T10:12:25.171 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.171 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.171 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.171 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115305+0000", 2024-09-13T10:12:25.171 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00012364200000000001 2024-09-13T10:12:25.171 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.171 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.171 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.171 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.136435+0000", 2024-09-13T10:12:25.171 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.021129479999999999 2024-09-13T10:12:25.171 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.171 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.172 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.172 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143376+0000", 2024-09-13T10:12:25.172 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0069408600000000001 2024-09-13T10:12:25.172 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.172 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.172 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.172 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.150416+0000", 2024-09-13T10:12:25.172 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0070403669999999996 2024-09-13T10:12:25.172 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.172 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.172 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.172 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.151872+0000", 2024-09-13T10:12:25.172 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001456286 2024-09-13T10:12:25.173 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.173 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.173 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.173 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.151906+0000", 2024-09-13T10:12:25.173 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.3167999999999998e-05 2024-09-13T10:12:25.173 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.173 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.173 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.173 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.151934+0000", 2024-09-13T10:12:25.173 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.8490999999999999e-05 2024-09-13T10:12:25.173 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.173 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.174 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.174 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.174 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.174 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:250 3.2 3:78d40eb9:::smithi19530714-1 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.174 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.115099+0000", 2024-09-13T10:12:25.174 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.949961341000005, 2024-09-13T10:12:25.174 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.028750412999999999, 2024-09-13T10:12:25.174 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.174 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.174 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.174 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.174 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.174 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.174 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 250 2024-09-13T10:12:25.175 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.175 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.175 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.175 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.175 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115099+0000", 2024-09-13T10:12:25.175 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.175 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.175 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.175 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.175 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115099+0000", 2024-09-13T10:12:25.175 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.175 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.175 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.175 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.176 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115101+0000", 2024-09-13T10:12:25.176 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.1789999999999998e-06 2024-09-13T10:12:25.176 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.176 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.176 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.176 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115132+0000", 2024-09-13T10:12:25.176 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.1163e-05 2024-09-13T10:12:25.176 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.176 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.176 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.176 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115150+0000", 2024-09-13T10:12:25.176 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7864999999999999e-05 2024-09-13T10:12:25.176 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.177 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.177 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.177 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115159+0000", 2024-09-13T10:12:25.177 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.1719999999999996e-06 2024-09-13T10:12:25.177 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.177 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.177 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.177 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115440+0000", 2024-09-13T10:12:25.177 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00028068199999999998 2024-09-13T10:12:25.177 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.177 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.177 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.177 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.115579+0000", 2024-09-13T10:12:25.177 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013910800000000001 2024-09-13T10:12:25.178 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.178 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.178 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.178 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.132849+0000", 2024-09-13T10:12:25.178 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.017270127999999999 2024-09-13T10:12:25.178 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.178 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.178 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.178 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.142332+0000", 2024-09-13T10:12:25.178 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0094831859999999994 2024-09-13T10:12:25.178 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.178 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.178 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.178 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143034+0000", 2024-09-13T10:12:25.179 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00070226499999999999 2024-09-13T10:12:25.179 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.179 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.179 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.179 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143793+0000", 2024-09-13T10:12:25.179 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00075876700000000004 2024-09-13T10:12:25.179 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.179 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.179 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.179 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143825+0000", 2024-09-13T10:12:25.179 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.2017999999999997e-05 2024-09-13T10:12:25.179 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.179 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.180 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.180 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.143849+0000", 2024-09-13T10:12:25.180 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3879999999999998e-05 2024-09-13T10:12:25.180 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.180 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.180 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.180 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.180 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.180 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.7.22:3 3.2 3:5915c927:::smithi19530714-19 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.180 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.216433+0000", 2024-09-13T10:12:25.180 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.848626963000001, 2024-09-13T10:12:25.180 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.037069469000000001, 2024-09-13T10:12:25.180 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.181 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.181 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.181 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.181 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.181 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.181 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216433+0000", 2024-09-13T10:12:25.181 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.181 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.181 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.181 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.181 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216433+0000", 2024-09-13T10:12:25.181 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.181 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.182 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.182 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.182 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216436+0000", 2024-09-13T10:12:25.182 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7800000000000001e-06 2024-09-13T10:12:25.182 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.182 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.182 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.182 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216473+0000", 2024-09-13T10:12:25.182 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.6931999999999998e-05 2024-09-13T10:12:25.182 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.182 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.182 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.182 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216512+0000", 2024-09-13T10:12:25.182 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.8831999999999997e-05 2024-09-13T10:12:25.183 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.183 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.183 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.183 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216539+0000", 2024-09-13T10:12:25.183 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.7705999999999999e-05 2024-09-13T10:12:25.183 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.183 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.183 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.183 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.216732+0000", 2024-09-13T10:12:25.183 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00019224500000000001 2024-09-13T10:12:25.183 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.183 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.184 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.184 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217000+0000", 2024-09-13T10:12:25.184 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00026813100000000001 2024-09-13T10:12:25.184 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.184 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.184 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.184 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217638+0000", 2024-09-13T10:12:25.184 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00063874400000000001 2024-09-13T10:12:25.184 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.184 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.185 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.185 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.219940+0000", 2024-09-13T10:12:25.185 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023015890000000002 2024-09-13T10:12:25.185 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.185 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.185 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.185 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.220304+0000", 2024-09-13T10:12:25.185 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00036399099999999998 2024-09-13T10:12:25.185 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.185 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.185 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.186 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253232+0000", 2024-09-13T10:12:25.186 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.032927640000000001 2024-09-13T10:12:25.186 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.186 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.186 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.186 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253393+0000", 2024-09-13T10:12:25.186 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000161142 2024-09-13T10:12:25.186 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.186 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.186 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.186 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253503+0000", 2024-09-13T10:12:25.186 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000109737 2024-09-13T10:12:25.186 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.187 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.187 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.187 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.187 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.187 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(osd.2.23:2 3.2 3:46ab5247:::smithi19530714-11 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [call cas.chunk_get_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.187 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:10:51.217995+0000", 2024-09-13T10:12:25.187 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 93.847065491999999, 2024-09-13T10:12:25.187 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.036081690999999999, 2024-09-13T10:12:25.187 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.187 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.187 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.187 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.188 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.188 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.188 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217995+0000", 2024-09-13T10:12:25.188 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.188 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.188 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.188 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.188 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217995+0000", 2024-09-13T10:12:25.188 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.188 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.188 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.188 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.188 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.217997+0000", 2024-09-13T10:12:25.189 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.3429999999999998e-06 2024-09-13T10:12:25.189 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.189 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.189 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.189 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218007+0000", 2024-09-13T10:12:25.189 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 9.91e-06 2024-09-13T10:12:25.189 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.189 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.189 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.189 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218024+0000", 2024-09-13T10:12:25.189 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.7549000000000001e-05 2024-09-13T10:12:25.189 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.190 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.190 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.190 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218038+0000", 2024-09-13T10:12:25.190 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3509999999999999e-05 2024-09-13T10:12:25.190 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.190 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.190 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.190 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218384+0000", 2024-09-13T10:12:25.190 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00034618499999999999 2024-09-13T10:12:25.190 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.190 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.190 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.191 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.218605+0000", 2024-09-13T10:12:25.191 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00022059400000000001 2024-09-13T10:12:25.191 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.191 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.191 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "waiting for subops from 0,3", 2024-09-13T10:12:25.191 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.219124+0000", 2024-09-13T10:12:25.191 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00051934699999999997 2024-09-13T10:12:25.191 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.191 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.191 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.191 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.222285+0000", 2024-09-13T10:12:25.191 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0031605790000000002 2024-09-13T10:12:25.191 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.192 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.192 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "op_commit", 2024-09-13T10:12:25.192 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.224618+0000", 2024-09-13T10:12:25.192 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0023338209999999998 2024-09-13T10:12:25.192 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.192 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.192 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.192 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253766+0000", 2024-09-13T10:12:25.192 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.029147890999999999 2024-09-13T10:12:25.192 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.192 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.192 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "commit_sent", 2024-09-13T10:12:25.192 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.253920+0000", 2024-09-13T10:12:25.193 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00015327299999999999 2024-09-13T10:12:25.193 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.193 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.193 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.193 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:10:51.254076+0000", 2024-09-13T10:12:25.193 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.000156689 2024-09-13T10:12:25.193 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.193 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.193 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.193 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.193 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.193 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:1854 4.b 4:d8e9927b:::smithi19530714-44:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:25.194 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:11:10.749110+0000", 2024-09-13T10:12:25.194 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 74.315950401999999, 2024-09-13T10:12:25.194 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.049047407000000001, 2024-09-13T10:12:25.194 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.194 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.194 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:25.194 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.194 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.194 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.194 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 1854 2024-09-13T10:12:25.194 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.194 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.195 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.195 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.195 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749110+0000", 2024-09-13T10:12:25.195 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.195 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.195 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.195 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.195 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749110+0000", 2024-09-13T10:12:25.195 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.195 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.195 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.195 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.195 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749113+0000", 2024-09-13T10:12:25.196 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0970000000000002e-06 2024-09-13T10:12:25.196 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.196 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.196 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.196 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749125+0000", 2024-09-13T10:12:25.196 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.2048999999999999e-05 2024-09-13T10:12:25.197 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.197 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.197 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.197 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749164+0000", 2024-09-13T10:12:25.197 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.9579999999999997e-05 2024-09-13T10:12:25.197 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.197 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.197 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.197 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749188+0000", 2024-09-13T10:12:25.197 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 2.4026e-05 2024-09-13T10:12:25.198 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.198 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.198 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.198 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749320+0000", 2024-09-13T10:12:25.198 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00013178500000000001 2024-09-13T10:12:25.198 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.198 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.198 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.198 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.749506+0000", 2024-09-13T10:12:25.198 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00018623700000000001 2024-09-13T10:12:25.198 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.198 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.198 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.198 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:11:10.798157+0000", 2024-09-13T10:12:25.199 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.048650632999999999 2024-09-13T10:12:25.199 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.199 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.199 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.199 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.199 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.199 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:4370 4.16 4:6fb28ed4:::smithi19530714-19:head [sparse-read 0~2678613] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-13T10:12:25.199 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:09.876474+0000", 2024-09-13T10:12:25.199 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 15.188585810999999, 2024-09-13T10:12:25.199 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027247817000000001, 2024-09-13T10:12:25.199 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.199 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.200 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:25.200 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.200 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.200 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.200 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 4370 2024-09-13T10:12:25.200 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.200 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.201 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.201 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.201 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.876474+0000", 2024-09-13T10:12:25.201 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.201 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.201 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.201 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.201 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.876474+0000", 2024-09-13T10:12:25.201 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.201 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.201 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.201 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.202 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.876476+0000", 2024-09-13T10:12:25.202 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.606e-06 2024-09-13T10:12:25.202 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.202 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.202 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.202 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.876482+0000", 2024-09-13T10:12:25.202 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 5.8470000000000001e-06 2024-09-13T10:12:25.202 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.202 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.202 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.202 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.876498+0000", 2024-09-13T10:12:25.203 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.6195999999999999e-05 2024-09-13T10:12:25.203 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.203 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.203 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.203 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.876513+0000", 2024-09-13T10:12:25.203 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.4966e-05 2024-09-13T10:12:25.203 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.203 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.203 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.203 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.901803+0000", 2024-09-13T10:12:25.203 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.025289879000000001 2024-09-13T10:12:25.203 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.204 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.204 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.204 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.902098+0000", 2024-09-13T10:12:25.204 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00029503500000000002 2024-09-13T10:12:25.204 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.204 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.204 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.204 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.903722+0000", 2024-09-13T10:12:25.204 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.001624288 2024-09-13T10:12:25.204 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.204 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.204 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.205 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.205 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.205 INFO:teuthology.orchestra.run.smithi181.stdout: "description": "osd_op(client.4607.0:4371 4.16 4:6fb28ed4:::smithi19530714-19:head [read 0~3815222,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103)", 2024-09-13T10:12:25.205 INFO:teuthology.orchestra.run.smithi181.stdout: "initiated_at": "2024-09-13T10:12:09.876532+0000", 2024-09-13T10:12:25.205 INFO:teuthology.orchestra.run.smithi181.stdout: "age": 15.188528279, 2024-09-13T10:12:25.205 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.030600585, 2024-09-13T10:12:25.205 INFO:teuthology.orchestra.run.smithi181.stdout: "continuous": false, 2024-09-13T10:12:25.205 INFO:teuthology.orchestra.run.smithi181.stdout: "type_data": { 2024-09-13T10:12:25.205 INFO:teuthology.orchestra.run.smithi181.stdout: "flag_point": "started", 2024-09-13T10:12:25.205 INFO:teuthology.orchestra.run.smithi181.stdout: "client_info": { 2024-09-13T10:12:25.205 INFO:teuthology.orchestra.run.smithi181.stdout: "client": "client.4607", 2024-09-13T10:12:25.205 INFO:teuthology.orchestra.run.smithi181.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.206 INFO:teuthology.orchestra.run.smithi181.stdout: "tid": 4371 2024-09-13T10:12:25.206 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.206 INFO:teuthology.orchestra.run.smithi181.stdout: "events": [ 2024-09-13T10:12:25.206 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.206 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "initiated", 2024-09-13T10:12:25.206 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.876532+0000", 2024-09-13T10:12:25.206 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.206 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.206 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.206 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "header_read", 2024-09-13T10:12:25.206 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.876532+0000", 2024-09-13T10:12:25.206 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0 2024-09-13T10:12:25.206 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.207 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.207 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "throttled", 2024-09-13T10:12:25.207 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.876533+0000", 2024-09-13T10:12:25.207 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.3400000000000001e-06 2024-09-13T10:12:25.207 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.207 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.207 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "all_read", 2024-09-13T10:12:25.207 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.876540+0000", 2024-09-13T10:12:25.207 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 7.1400000000000002e-06 2024-09-13T10:12:25.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.208 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.208 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "dispatched", 2024-09-13T10:12:25.208 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.876555+0000", 2024-09-13T10:12:25.208 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 1.5265999999999999e-05 2024-09-13T10:12:25.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.208 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.208 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.208 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.876586+0000", 2024-09-13T10:12:25.208 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 3.0086e-05 2024-09-13T10:12:25.208 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.208 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.208 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "reached_pg", 2024-09-13T10:12:25.208 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.903801+0000", 2024-09-13T10:12:25.209 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.027215751999999999 2024-09-13T10:12:25.209 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.209 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.209 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "started", 2024-09-13T10:12:25.209 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.903969+0000", 2024-09-13T10:12:25.209 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.00016756299999999999 2024-09-13T10:12:25.209 INFO:teuthology.orchestra.run.smithi181.stdout: }, 2024-09-13T10:12:25.209 INFO:teuthology.orchestra.run.smithi181.stdout: { 2024-09-13T10:12:25.209 INFO:teuthology.orchestra.run.smithi181.stdout: "event": "done", 2024-09-13T10:12:25.209 INFO:teuthology.orchestra.run.smithi181.stdout: "time": "2024-09-13T10:12:09.907132+0000", 2024-09-13T10:12:25.209 INFO:teuthology.orchestra.run.smithi181.stdout: "duration": 0.0031634380000000002 2024-09-13T10:12:25.209 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.209 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.210 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.210 INFO:teuthology.orchestra.run.smithi181.stdout: } 2024-09-13T10:12:25.210 INFO:teuthology.orchestra.run.smithi181.stdout: ] 2024-09-13T10:12:25.210 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:25.211 INFO:tasks.rados.rados.0.smithi195.stdout:1993: finishing set_chunk to oid 13 2024-09-13T10:12:25.211 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 13 set_chunk returned error code -95 offset: 1057792 length: 2187776 2024-09-13T10:12:25.211 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 0 (ObjNum 342 snap 0 seq_num 342) dirty exists 2024-09-13T10:12:25.211 INFO:tasks.rados.rados.0.smithi195.stdout:1994: got (2) No such file or directory 2024-09-13T10:12:25.212 INFO:tasks.rados.rados.0.smithi195.stdout:1994: got expected ENOENT (src dne) 2024-09-13T10:12:25.212 INFO:tasks.rados.rados.0.smithi195.stdout:1995: expect (ObjNum 405 snap 0 seq_num 405) 2024-09-13T10:12:25.212 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 514418 2024-09-13T10:12:25.212 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:25.205+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:25.212 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T10:12:25.235 INFO:tasks.rados.rados.0.smithi195.stdout:1996: finishing write tid 1 to smithi19530714-43 2024-09-13T10:12:25.235 INFO:tasks.rados.rados.0.smithi195.stdout:1996: finishing write tid 2 to smithi19530714-43 2024-09-13T10:12:25.235 INFO:tasks.rados.rados.0.smithi195.stdout:1996: finishing write tid 3 to smithi19530714-43 2024-09-13T10:12:25.236 INFO:tasks.rados.rados.0.smithi195.stdout:1996: finishing write tid 4 to smithi19530714-43 2024-09-13T10:12:25.236 INFO:tasks.rados.rados.0.smithi195.stdout:1996: finishing write tid 5 to smithi19530714-43 2024-09-13T10:12:25.236 INFO:tasks.rados.rados.0.smithi195.stdout:1996: oid 43 updating version 0 to 126 2024-09-13T10:12:25.236 INFO:tasks.rados.rados.0.smithi195.stdout:1996: oid 43 version 126 is already newer than 123 2024-09-13T10:12:25.236 INFO:tasks.rados.rados.0.smithi195.stdout:1996: oid 43 version 126 is already newer than 124 2024-09-13T10:12:25.236 INFO:tasks.rados.rados.0.smithi195.stdout:1996: oid 43 version 126 is already newer than 125 2024-09-13T10:12:25.236 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 43 v 126 (ObjNum 450 snap 0 seq_num 450) dirty exists 2024-09-13T10:12:25.236 INFO:tasks.rados.rados.0.smithi195.stdout:1996: left oid 43 (ObjNum 450 snap 0 seq_num 450) 2024-09-13T10:12:25.236 INFO:tasks.rados.rados.0.smithi195.stdout:1997: finishing set_chunk to oid 12 2024-09-13T10:12:25.236 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 12 set_chunk returned error code -95 offset: 845312 length: 1506816 2024-09-13T10:12:25.236 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 12 v 0 (ObjNum 395 snap 0 seq_num 395) dirty exists 2024-09-13T10:12:25.236 INFO:tasks.rados.rados.0.smithi195.stdout:1998: finishing write tid 1 to smithi19530714-17 2024-09-13T10:12:25.236 INFO:tasks.rados.rados.0.smithi195.stdout:2001: seq_num 452 ranges {796764=477375,1983508=618487,3363075=233689} 2024-09-13T10:12:25.246 INFO:tasks.rados.rados.0.smithi195.stdout:2001: writing smithi19530714-32 from 796764 to 1274139 tid 1 2024-09-13T10:12:25.257 INFO:tasks.rados.rados.0.smithi195.stdout:2001: writing smithi19530714-32 from 1983508 to 2601995 tid 2 2024-09-13T10:12:25.265 INFO:tasks.rados.rados.0.smithi195.stdout:2001: writing smithi19530714-32 from 3363075 to 3596764 tid 3 2024-09-13T10:12:25.265 INFO:tasks.rados.rados.0.smithi195.stdout:1992: done (9 left) 2024-09-13T10:12:25.265 INFO:tasks.rados.rados.0.smithi195.stdout:1993: done (8 left) 2024-09-13T10:12:25.266 INFO:tasks.rados.rados.0.smithi195.stdout:1994: done (7 left) 2024-09-13T10:12:25.266 INFO:tasks.rados.rados.0.smithi195.stdout:1995: done (6 left) 2024-09-13T10:12:25.266 INFO:tasks.rados.rados.0.smithi195.stdout:1996: done (5 left) 2024-09-13T10:12:25.266 INFO:tasks.rados.rados.0.smithi195.stdout:1997: done (4 left) 2024-09-13T10:12:25.266 INFO:tasks.rados.rados.0.smithi195.stdout:2002: read oid 21 snap -1 2024-09-13T10:12:25.266 INFO:tasks.rados.rados.0.smithi195.stdout:2002: expect (ObjNum 421 snap 0 seq_num 421) 2024-09-13T10:12:25.266 INFO:tasks.rados.rados.0.smithi195.stdout:2003: set_chunk oid 18 target oid 2024-09-13T10:12:25.266 INFO:tasks.rados.rados.0.smithi195.stdout:1998: finishing write tid 2 to smithi19530714-17 2024-09-13T10:12:25.293 INFO:tasks.rados.rados.0.smithi195.stdout:2003: set_chunk oid 18 offset: 448000 length: 2037760 target oid 0fc03762d37c2de369650b3fbcdd90f51bdb3fd961a1f4644917f44e9c0cbb2e offset: 0 snap -1 2024-09-13T10:12:25.293 INFO:tasks.rados.rados.0.smithi195.stdout:2004: read oid 10 snap -1 2024-09-13T10:12:25.293 INFO:tasks.rados.rados.0.smithi195.stdout:2004: expect deleted 2024-09-13T10:12:25.293 INFO:tasks.rados.rados.0.smithi195.stdout:2005: tier_flush oid 39 2024-09-13T10:12:25.294 INFO:tasks.rados.rados.0.smithi195.stdout:2005: tier_flush oid 39 snap -1 2024-09-13T10:12:25.294 INFO:tasks.rados.rados.0.smithi195.stdout:1998: finishing write tid 3 to smithi19530714-17 2024-09-13T10:12:25.294 INFO:tasks.rados.rados.0.smithi195.stdout:2006: delete oid 7 current snap is 0 2024-09-13T10:12:25.294 INFO:tasks.rados.rados.0.smithi195.stdout:1998: finishing write tid 4 to smithi19530714-17 2024-09-13T10:12:25.294 INFO:tasks.rados.rados.0.smithi195.stdout:1998: finishing write tid 5 to smithi19530714-17 2024-09-13T10:12:25.294 INFO:tasks.rados.rados.0.smithi195.stdout:1998: oid 17 updating version 0 to 108 2024-09-13T10:12:25.294 INFO:tasks.rados.rados.0.smithi195.stdout:1998: oid 17 version 108 is already newer than 106 2024-09-13T10:12:25.294 INFO:tasks.rados.rados.0.smithi195.stdout:1998: oid 17 version 108 is already newer than 107 2024-09-13T10:12:25.294 INFO:tasks.rados.rados.0.smithi195.stdout:1998: oid 17 version 108 is already newer than 105 2024-09-13T10:12:25.294 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 17 v 108 (ObjNum 451 snap 0 seq_num 451) dirty exists 2024-09-13T10:12:25.294 INFO:tasks.rados.rados.0.smithi195.stdout:1998: left oid 17 (ObjNum 451 snap 0 seq_num 451) 2024-09-13T10:12:25.294 INFO:tasks.rados.rados.0.smithi195.stdout:1999: finishing set_chunk to oid 25 2024-09-13T10:12:25.294 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 25 set_chunk returned error code -95 offset: 1084416 length: 240128 2024-09-13T10:12:25.294 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 25 v 0 (ObjNum 324 snap 0 seq_num 324) dirty exists 2024-09-13T10:12:25.295 INFO:tasks.rados.rados.0.smithi195.stdout:2000: expect (ObjNum 409 snap 0 seq_num 409) 2024-09-13T10:12:25.297 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1257783 2024-09-13T10:12:25.313 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:25.310+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:25.331 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:25.331 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:12:25.331 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:12:25.331 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:25.345 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T10:12:25.365 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2820785 2024-09-13T10:12:25.365 INFO:tasks.rados.rados.0.smithi195.stdout:2001: finishing write tid 1 to smithi19530714-32 2024-09-13T10:12:25.365 INFO:tasks.rados.rados.0.smithi195.stdout:2001: finishing write tid 2 to smithi19530714-32 2024-09-13T10:12:25.365 INFO:tasks.rados.rados.0.smithi195.stdout:2002: expect (ObjNum 421 snap 0 seq_num 421) 2024-09-13T10:12:25.413 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:25.409+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:25.489 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:25.490 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:12:25.490 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:12:25.490 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:12:25.490 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:25.504 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T10:12:25.514 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:25.509+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:25.565 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1802385 2024-09-13T10:12:25.600 INFO:tasks.rados.rados.0.smithi195.stdout:2001: finishing write tid 3 to smithi19530714-32 2024-09-13T10:12:25.601 INFO:tasks.rados.rados.0.smithi195.stdout:2001: finishing write tid 4 to smithi19530714-32 2024-09-13T10:12:25.601 INFO:tasks.rados.rados.0.smithi195.stdout:2001: finishing write tid 5 to smithi19530714-32 2024-09-13T10:12:25.601 INFO:tasks.rados.rados.0.smithi195.stdout:2001: oid 32 updating version 0 to 110 2024-09-13T10:12:25.601 INFO:tasks.rados.rados.0.smithi195.stdout:2001: oid 32 updating version 110 to 111 2024-09-13T10:12:25.601 INFO:tasks.rados.rados.0.smithi195.stdout:2001: oid 32 version 111 is already newer than 109 2024-09-13T10:12:25.601 INFO:tasks.rados.rados.0.smithi195.stdout:2001: oid 32 updating version 111 to 112 2024-09-13T10:12:25.601 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 32 v 112 (ObjNum 452 snap 0 seq_num 452) dirty exists 2024-09-13T10:12:25.601 INFO:tasks.rados.rados.0.smithi195.stdout:2001: left oid 32 (ObjNum 452 snap 0 seq_num 452) 2024-09-13T10:12:25.601 INFO:tasks.rados.rados.0.smithi195.stdout:2005: got (2) No such file or directory 2024-09-13T10:12:25.601 INFO:tasks.rados.rados.0.smithi195.stdout:2005: got expected ENOENT (src dne) 2024-09-13T10:12:25.602 INFO:tasks.rados.rados.0.smithi195.stdout:2003: finishing set_chunk to oid 18 2024-09-13T10:12:25.602 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 18 v 134 (ObjNum 418 snap 0 seq_num 418) dirty exists 2024-09-13T10:12:25.602 INFO:tasks.rados.rados.0.smithi195.stdout:1998: done (8 left) 2024-09-13T10:12:25.602 INFO:tasks.rados.rados.0.smithi195.stdout:1999: done (7 left) 2024-09-13T10:12:25.602 INFO:tasks.rados.rados.0.smithi195.stdout:2000: done (6 left) 2024-09-13T10:12:25.602 INFO:tasks.rados.rados.0.smithi195.stdout:2001: done (5 left) 2024-09-13T10:12:25.602 INFO:tasks.rados.rados.0.smithi195.stdout:2002: done (4 left) 2024-09-13T10:12:25.602 INFO:tasks.rados.rados.0.smithi195.stdout:2003: done (3 left) 2024-09-13T10:12:25.602 INFO:tasks.rados.rados.0.smithi195.stdout:2004: done (2 left) 2024-09-13T10:12:25.602 INFO:tasks.rados.rados.0.smithi195.stdout:2005: done (1 left) 2024-09-13T10:12:25.602 INFO:tasks.rados.rados.0.smithi195.stdout:2006: done (0 left) 2024-09-13T10:12:25.603 INFO:tasks.rados.rados.0.smithi195.stdout:2007: write (excl) oid 39 current snap is 0 2024-09-13T10:12:25.603 INFO:tasks.rados.rados.0.smithi195.stdout:2007: seq_num 454 ranges {430378=627873,1700472=622156,3017921=612457} 2024-09-13T10:12:25.609 INFO:tasks.rados.rados.0.smithi195.stdout:2007: writing smithi19530714-39 from 430378 to 1058251 tid 1 2024-09-13T10:12:25.615 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:25.614+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:25.619 INFO:tasks.rados.rados.0.smithi195.stdout:2007: writing smithi19530714-39 from 1700472 to 2322628 tid 2 2024-09-13T10:12:25.630 INFO:tasks.rados.rados.0.smithi195.stdout:2007: writing smithi19530714-39 from 3017921 to 3630378 tid 3 2024-09-13T10:12:25.630 INFO:tasks.rados.rados.0.smithi195.stdout:2008: read oid 10 snap -1 2024-09-13T10:12:25.630 INFO:tasks.rados.rados.0.smithi195.stdout:2008: expect deleted 2024-09-13T10:12:25.630 INFO:tasks.rados.rados.0.smithi195.stdout:2009: read oid 26 snap -1 2024-09-13T10:12:25.630 INFO:tasks.rados.rados.0.smithi195.stdout:2009: expect (ObjNum 379 snap 0 seq_num 379) 2024-09-13T10:12:25.630 INFO:tasks.rados.rados.0.smithi195.stdout:2010: set_chunk oid 2 target oid 2024-09-13T10:12:25.633 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:12:25.633 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:12:25.633 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:12:25.633 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2024-09-13T10:12:25.633 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:12:25.646 INFO:tasks.rados.rados.0.smithi195.stdout:2010: set_chunk oid 2 offset: 1137664 length: 616960 target oid 95c983cf32b821cb874f96f43ac98145baed7d43f2ca42254f9115ab78d710fa offset: 0 snap -1 2024-09-13T10:12:25.646 INFO:tasks.rados.rados.0.smithi195.stdout:2011: write oid 14 current snap is 0 2024-09-13T10:12:25.646 INFO:tasks.rados.rados.0.smithi195.stdout:2007: finishing write tid 1 to smithi19530714-39 2024-09-13T10:12:25.646 INFO:tasks.rados.rados.0.smithi195.stdout:2007: finishing write tid 2 to smithi19530714-39 2024-09-13T10:12:25.646 INFO:tasks.rados.rados.0.smithi195.stdout:2007: finishing write tid 3 to smithi19530714-39 2024-09-13T10:12:25.646 INFO:tasks.rados.rados.0.smithi195.stdout:2007: finishing write tid 4 to smithi19530714-39 2024-09-13T10:12:25.646 INFO:tasks.rados.rados.0.smithi195.stdout:2007: finishing write tid 5 to smithi19530714-39 2024-09-13T10:12:25.646 INFO:tasks.rados.rados.0.smithi195.stdout:2007: oid 39 updating version 0 to 49 2024-09-13T10:12:25.646 INFO:tasks.rados.rados.0.smithi195.stdout:2007: oid 39 version 49 is already newer than 48 2024-09-13T10:12:25.647 INFO:tasks.rados.rados.0.smithi195.stdout:2007: oid 39 version 49 is already newer than 46 2024-09-13T10:12:25.647 INFO:tasks.rados.rados.0.smithi195.stdout:2007: oid 39 version 49 is already newer than 47 2024-09-13T10:12:25.647 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 49 (ObjNum 454 snap 0 seq_num 454) dirty exists 2024-09-13T10:12:25.647 INFO:tasks.rados.rados.0.smithi195.stdout:2007: left oid 39 (ObjNum 454 snap 0 seq_num 454) 2024-09-13T10:12:25.647 INFO:tasks.rados.rados.0.smithi195.stdout:2009: expect (ObjNum 379 snap 0 seq_num 379) 2024-09-13T10:12:25.647 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:25.678 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:25.678 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:12:25.678 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:12:25.679 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:12:25.679 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.679 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:207 3.0 3:332cd44b:::smithi19530714-8 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.679 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:12:25.679 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.682962829000004, 2024-09-13T10:12:25.679 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.074034500000000003, 2024-09-13T10:12:25.679 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.679 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.679 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.679 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.679 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.679 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.679 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 207 2024-09-13T10:12:25.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.680 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:12:25.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.680 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.680 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991268+0000", 2024-09-13T10:12:25.680 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.680 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.680 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991271+0000", 2024-09-13T10:12:25.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9289999999999998e-06 2024-09-13T10:12:25.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991284+0000", 2024-09-13T10:12:25.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2597e-05 2024-09-13T10:12:25.681 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.681 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.681 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.681 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991313+0000", 2024-09-13T10:12:25.681 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8982000000000001e-05 2024-09-13T10:12:25.682 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.682 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.682 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.682 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991334+0000", 2024-09-13T10:12:25.682 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1046000000000001e-05 2024-09-13T10:12:25.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991416+0000", 2024-09-13T10:12:25.683 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.153e-05 2024-09-13T10:12:25.683 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.683 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.683 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.683 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991504+0000", 2024-09-13T10:12:25.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.8220999999999994e-05 2024-09-13T10:12:25.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:25.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017232+0000", 2024-09-13T10:12:25.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025728328000000002 2024-09-13T10:12:25.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.684 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.684 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:25.684 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.031588+0000", 2024-09-13T10:12:25.684 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014355557 2024-09-13T10:12:25.684 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034211+0000", 2024-09-13T10:12:25.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0026232790000000001 2024-09-13T10:12:25.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.685 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065168+0000", 2024-09-13T10:12:25.685 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030956833999999999 2024-09-13T10:12:25.685 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.685 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.685 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065260+0000", 2024-09-13T10:12:25.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2044000000000002e-05 2024-09-13T10:12:25.686 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.686 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.686 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.686 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065303+0000", 2024-09-13T10:12:25.686 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3152999999999998e-05 2024-09-13T10:12:25.687 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.687 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.687 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.687 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.687 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.687 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:211 3.0 3:0596651d:::smithi19530714-12 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.687 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:12:25.687 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.682239451000001, 2024-09-13T10:12:25.687 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.042923122000000001, 2024-09-13T10:12:25.687 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.687 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.688 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.688 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.688 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.688 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.688 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 211 2024-09-13T10:12:25.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.688 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.688 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.688 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.688 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:12:25.688 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.688 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991992+0000", 2024-09-13T10:12:25.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.991995+0000", 2024-09-13T10:12:25.689 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7640000000000001e-06 2024-09-13T10:12:25.689 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.689 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.689 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.689 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992007+0000", 2024-09-13T10:12:25.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2058999999999999e-05 2024-09-13T10:12:25.690 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.690 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.690 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.690 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992040+0000", 2024-09-13T10:12:25.690 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3466000000000001e-05 2024-09-13T10:12:25.691 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.691 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.691 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.691 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992059+0000", 2024-09-13T10:12:25.691 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8485999999999999e-05 2024-09-13T10:12:25.692 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.692 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.692 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.692 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992136+0000", 2024-09-13T10:12:25.692 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6994000000000002e-05 2024-09-13T10:12:25.693 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.693 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.693 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.693 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992207+0000", 2024-09-13T10:12:25.693 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.1357000000000001e-05 2024-09-13T10:12:25.694 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.694 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.694 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:25.694 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.006377+0000", 2024-09-13T10:12:25.694 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014170455 2024-09-13T10:12:25.695 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.695 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.695 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:25.695 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.022844+0000", 2024-09-13T10:12:25.695 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016466275999999998 2024-09-13T10:12:25.696 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.696 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.696 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.696 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023031+0000", 2024-09-13T10:12:25.696 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018686000000000001 2024-09-13T10:12:25.697 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.697 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.697 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.697 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034788+0000", 2024-09-13T10:12:25.697 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011756967 2024-09-13T10:12:25.698 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.698 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.698 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.698 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034857+0000", 2024-09-13T10:12:25.698 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9169000000000001e-05 2024-09-13T10:12:25.699 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.699 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.699 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.699 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034915+0000", 2024-09-13T10:12:25.700 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8269000000000001e-05 2024-09-13T10:12:25.700 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.700 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.700 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.700 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.701 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.701 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:212 3.0 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.701 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:12:25.701 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.682133117999996, 2024-09-13T10:12:25.701 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.073011809999999996, 2024-09-13T10:12:25.702 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.702 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.702 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.703 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.703 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.703 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.703 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 212 2024-09-13T10:12:25.703 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.704 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.704 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.704 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.704 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:12:25.704 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.705 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.705 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.705 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.705 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:12:25.706 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.706 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.706 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.706 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.706 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992099+0000", 2024-09-13T10:12:25.707 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4899999999999995e-07 2024-09-13T10:12:25.707 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.707 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.707 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.707 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992103+0000", 2024-09-13T10:12:25.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.33e-06 2024-09-13T10:12:25.708 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.708 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.708 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.708 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992110+0000", 2024-09-13T10:12:25.708 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8399999999999997e-06 2024-09-13T10:12:25.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.709 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992116+0000", 2024-09-13T10:12:25.709 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6790000000000002e-06 2024-09-13T10:12:25.709 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.709 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.709 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992292+0000", 2024-09-13T10:12:25.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017659199999999999 2024-09-13T10:12:25.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:50.992385+0000", 2024-09-13T10:12:25.710 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2608000000000005e-05 2024-09-13T10:12:25.710 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.710 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.710 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:25.710 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011428+0000", 2024-09-13T10:12:25.711 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.019042631000000001 2024-09-13T10:12:25.711 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.711 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.711 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:25.711 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.022852+0000", 2024-09-13T10:12:25.712 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011424528999999999 2024-09-13T10:12:25.712 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.712 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.712 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.031660+0000", 2024-09-13T10:12:25.713 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088079899999999999 2024-09-13T10:12:25.713 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.713 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.713 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.713 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065032+0000", 2024-09-13T10:12:25.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033372247000000001 2024-09-13T10:12:25.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065073+0000", 2024-09-13T10:12:25.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.0598e-05 2024-09-13T10:12:25.714 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.714 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.714 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.714 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065110+0000", 2024-09-13T10:12:25.714 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7116999999999997e-05 2024-09-13T10:12:25.714 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.715 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.715 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.715 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.715 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.715 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:202 3.2 e38/30 3:470123eb:::smithi19530714-3 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'1, mlcod=0'0)", 2024-09-13T10:12:25.715 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:12:25.715 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.672642511999996, 2024-09-13T10:12:25.715 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017280298999999999, 2024-09-13T10:12:25.715 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.715 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.716 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.716 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:12:25.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.716 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.716 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.716 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001589+0000", 2024-09-13T10:12:25.716 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.716 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.001594+0000", 2024-09-13T10:12:25.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5550000000000002e-06 2024-09-13T10:12:25.717 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.717 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.717 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.717 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009486+0000", 2024-09-13T10:12:25.717 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078916799999999999 2024-09-13T10:12:25.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.718 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009519+0000", 2024-09-13T10:12:25.718 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2898999999999998e-05 2024-09-13T10:12:25.718 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.718 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.718 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009540+0000", 2024-09-13T10:12:25.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1559000000000001e-05 2024-09-13T10:12:25.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009655+0000", 2024-09-13T10:12:25.719 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000114271 2024-09-13T10:12:25.719 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.719 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.719 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.719 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.009683+0000", 2024-09-13T10:12:25.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8379000000000001e-05 2024-09-13T10:12:25.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.018766+0000", 2024-09-13T10:12:25.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0090829589999999998 2024-09-13T10:12:25.720 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.720 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.720 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.720 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.018869+0000", 2024-09-13T10:12:25.720 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000102997 2024-09-13T10:12:25.720 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.721 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.721 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.721 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.721 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.721 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:216 3.0 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.721 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:12:25.721 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.669629975999996, 2024-09-13T10:12:25.721 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.063936142000000001, 2024-09-13T10:12:25.721 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.721 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.721 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.721 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.721 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.722 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.722 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 216 2024-09-13T10:12:25.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.722 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:12:25.722 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.722 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.722 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.722 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.722 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004601+0000", 2024-09-13T10:12:25.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004603+0000", 2024-09-13T10:12:25.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5540000000000001e-06 2024-09-13T10:12:25.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.723 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.723 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004615+0000", 2024-09-13T10:12:25.723 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2597e-05 2024-09-13T10:12:25.723 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.723 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004637+0000", 2024-09-13T10:12:25.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1713e-05 2024-09-13T10:12:25.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004651+0000", 2024-09-13T10:12:25.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3878e-05 2024-09-13T10:12:25.724 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.724 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.724 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.724 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.004985+0000", 2024-09-13T10:12:25.724 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00033347299999999998 2024-09-13T10:12:25.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.005706+0000", 2024-09-13T10:12:25.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00072122199999999999 2024-09-13T10:12:25.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.725 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.725 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:25.725 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023801+0000", 2024-09-13T10:12:25.725 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018095466000000001 2024-09-13T10:12:25.725 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:25.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.032153+0000", 2024-09-13T10:12:25.726 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0083516010000000002 2024-09-13T10:12:25.726 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.726 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.726 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.726 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.035250+0000", 2024-09-13T10:12:25.727 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0030974409999999998 2024-09-13T10:12:25.727 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.727 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.727 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068439+0000", 2024-09-13T10:12:25.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033189042000000002 2024-09-13T10:12:25.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068499+0000", 2024-09-13T10:12:25.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.9969000000000001e-05 2024-09-13T10:12:25.728 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.728 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.728 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.728 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.068537+0000", 2024-09-13T10:12:25.728 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8186000000000003e-05 2024-09-13T10:12:25.729 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.729 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.729 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.729 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.729 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.729 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:221 3.0 3:18ae852b:::smithi19530714-22 low_tier:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.729 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:12:25.729 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.635705469000001, 2024-09-13T10:12:25.729 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039809298, 2024-09-13T10:12:25.729 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.730 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.730 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.730 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.730 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.730 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.730 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 221 2024-09-13T10:12:25.730 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.730 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.730 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.730 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.730 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:12:25.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038526+0000", 2024-09-13T10:12:25.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.731 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.731 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.731 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038528+0000", 2024-09-13T10:12:25.731 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8819999999999999e-06 2024-09-13T10:12:25.731 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038534+0000", 2024-09-13T10:12:25.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0730000000000001e-06 2024-09-13T10:12:25.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.732 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.732 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038546+0000", 2024-09-13T10:12:25.732 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.219e-05 2024-09-13T10:12:25.732 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.732 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038566+0000", 2024-09-13T10:12:25.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9867000000000001e-05 2024-09-13T10:12:25.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.733 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038608+0000", 2024-09-13T10:12:25.733 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1977999999999997e-05 2024-09-13T10:12:25.733 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.733 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.733 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.038707+0000", 2024-09-13T10:12:25.734 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8708000000000004e-05 2024-09-13T10:12:25.734 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.734 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.734 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:25.734 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.057374+0000", 2024-09-13T10:12:25.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.018667652999999999 2024-09-13T10:12:25.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:25.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061983+0000", 2024-09-13T10:12:25.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0046087089999999999 2024-09-13T10:12:25.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.735 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.735 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.735 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.063027+0000", 2024-09-13T10:12:25.735 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001044239 2024-09-13T10:12:25.735 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078233+0000", 2024-09-13T10:12:25.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015205639999999999 2024-09-13T10:12:25.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.736 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.736 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078298+0000", 2024-09-13T10:12:25.736 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5180999999999998e-05 2024-09-13T10:12:25.736 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.736 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.737 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.737 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078335+0000", 2024-09-13T10:12:25.737 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7178000000000002e-05 2024-09-13T10:12:25.737 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.737 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.737 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.737 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.737 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.737 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:225 3.0 3:26960251:::smithi19530714-26 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.737 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:12:25.737 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.615779195000002, 2024-09-13T10:12:25.738 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030704875999999999, 2024-09-13T10:12:25.738 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.738 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.738 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.738 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.738 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.738 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.738 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 225 2024-09-13T10:12:25.738 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.738 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.738 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.738 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.738 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:12:25.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058452+0000", 2024-09-13T10:12:25.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.739 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.739 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.739 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058455+0000", 2024-09-13T10:12:25.739 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3220000000000001e-06 2024-09-13T10:12:25.739 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.740 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.740 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.740 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058467+0000", 2024-09-13T10:12:25.740 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1987999999999999e-05 2024-09-13T10:12:25.740 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058502+0000", 2024-09-13T10:12:25.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4873000000000003e-05 2024-09-13T10:12:25.741 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.741 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.741 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.741 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.058520+0000", 2024-09-13T10:12:25.741 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7977000000000001e-05 2024-09-13T10:12:25.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060066+0000", 2024-09-13T10:12:25.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001545261 2024-09-13T10:12:25.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.742 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.742 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060680+0000", 2024-09-13T10:12:25.742 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00061431799999999998 2024-09-13T10:12:25.742 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.742 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:25.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070650+0000", 2024-09-13T10:12:25.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0099703099999999996 2024-09-13T10:12:25.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:25.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.078192+0000", 2024-09-13T10:12:25.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0075419709999999997 2024-09-13T10:12:25.743 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.743 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.743 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.743 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084486+0000", 2024-09-13T10:12:25.743 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0062943779999999998 2024-09-13T10:12:25.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089029+0000", 2024-09-13T10:12:25.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045422379999999997 2024-09-13T10:12:25.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.744 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.744 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089092+0000", 2024-09-13T10:12:25.744 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3435999999999994e-05 2024-09-13T10:12:25.744 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.744 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.745 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.745 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089157+0000", 2024-09-13T10:12:25.745 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4803999999999998e-05 2024-09-13T10:12:25.745 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.745 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.745 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.745 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.745 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.745 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:228 3.0 3:0168a995:::smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.745 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:12:25.745 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.613978236999998, 2024-09-13T10:12:25.746 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.030139446, 2024-09-13T10:12:25.746 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.746 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.746 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.746 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.746 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.746 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.746 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 228 2024-09-13T10:12:25.746 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.746 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.746 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.746 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:12:25.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060253+0000", 2024-09-13T10:12:25.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.747 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.747 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.747 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060255+0000", 2024-09-13T10:12:25.747 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.787e-06 2024-09-13T10:12:25.747 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.748 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.748 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060262+0000", 2024-09-13T10:12:25.748 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8630000000000002e-06 2024-09-13T10:12:25.748 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.748 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060285+0000", 2024-09-13T10:12:25.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2935e-05 2024-09-13T10:12:25.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.060294+0000", 2024-09-13T10:12:25.749 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3940000000000002e-06 2024-09-13T10:12:25.749 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.749 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.749 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.749 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061041+0000", 2024-09-13T10:12:25.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00074706700000000002 2024-09-13T10:12:25.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061148+0000", 2024-09-13T10:12:25.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000107364 2024-09-13T10:12:25.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.750 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:25.750 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074199+0000", 2024-09-13T10:12:25.750 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013050447999999999 2024-09-13T10:12:25.750 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.750 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:25.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084506+0000", 2024-09-13T10:12:25.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010306789 2024-09-13T10:12:25.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089222+0000", 2024-09-13T10:12:25.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047163480000000004 2024-09-13T10:12:25.751 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.751 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.751 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.751 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090310+0000", 2024-09-13T10:12:25.751 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010880919999999999 2024-09-13T10:12:25.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090356+0000", 2024-09-13T10:12:25.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5522000000000002e-05 2024-09-13T10:12:25.752 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.752 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.752 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.752 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090393+0000", 2024-09-13T10:12:25.752 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.6837e-05 2024-09-13T10:12:25.752 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.752 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.753 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.753 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.753 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.753 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:229 3.0 3:1e6cc58c:::smithi19530714-30 low_tier:head [copy-from ver 18] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.753 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:12:25.753 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.612671861999999, 2024-09-13T10:12:25.753 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.033410479, 2024-09-13T10:12:25.753 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.753 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.753 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.753 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.753 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.753 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.754 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 229 2024-09-13T10:12:25.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.754 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:12:25.754 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.754 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.754 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.754 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.754 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061559+0000", 2024-09-13T10:12:25.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.755 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.755 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.755 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061562+0000", 2024-09-13T10:12:25.755 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9730000000000002e-06 2024-09-13T10:12:25.755 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.756 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.756 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.756 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061577+0000", 2024-09-13T10:12:25.756 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5044e-05 2024-09-13T10:12:25.756 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.757 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.757 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.757 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061602+0000", 2024-09-13T10:12:25.757 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4494999999999998e-05 2024-09-13T10:12:25.757 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061615+0000", 2024-09-13T10:12:25.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3207000000000001e-05 2024-09-13T10:12:25.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.758 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.758 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061701+0000", 2024-09-13T10:12:25.758 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.5694999999999999e-05 2024-09-13T10:12:25.758 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.758 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.061871+0000", 2024-09-13T10:12:25.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017004100000000001 2024-09-13T10:12:25.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:25.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.079511+0000", 2024-09-13T10:12:25.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.017639708 2024-09-13T10:12:25.759 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.759 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.759 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.759 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089295+0000", 2024-09-13T10:12:25.759 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0097846359999999993 2024-09-13T10:12:25.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:25.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089316+0000", 2024-09-13T10:12:25.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0288999999999998e-05 2024-09-13T10:12:25.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.760 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.760 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.760 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094875+0000", 2024-09-13T10:12:25.760 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0055598389999999996 2024-09-13T10:12:25.760 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094931+0000", 2024-09-13T10:12:25.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5556999999999997e-05 2024-09-13T10:12:25.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.761 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.761 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.761 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.094970+0000", 2024-09-13T10:12:25.761 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8995000000000002e-05 2024-09-13T10:12:25.761 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.761 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.761 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.761 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.762 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.762 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:233 3.0 3:24ead787:::smithi19530714-34 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.762 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:12:25.762 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.608495357999999, 2024-09-13T10:12:25.762 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.047200234000000001, 2024-09-13T10:12:25.762 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.762 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.763 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.763 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.763 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.763 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.763 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 233 2024-09-13T10:12:25.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.763 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.763 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.763 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:12:25.763 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.763 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.763 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065736+0000", 2024-09-13T10:12:25.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065737+0000", 2024-09-13T10:12:25.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.283e-06 2024-09-13T10:12:25.764 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.764 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.764 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.764 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065745+0000", 2024-09-13T10:12:25.764 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3130000000000001e-06 2024-09-13T10:12:25.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065757+0000", 2024-09-13T10:12:25.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2627e-05 2024-09-13T10:12:25.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.765 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065765+0000", 2024-09-13T10:12:25.765 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.4669999999999998e-06 2024-09-13T10:12:25.765 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.765 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.765 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065816+0000", 2024-09-13T10:12:25.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1545000000000002e-05 2024-09-13T10:12:25.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.065940+0000", 2024-09-13T10:12:25.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012366699999999999 2024-09-13T10:12:25.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.766 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.766 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:25.766 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.086064+0000", 2024-09-13T10:12:25.766 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.020124487 2024-09-13T10:12:25.766 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:25.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.090747+0000", 2024-09-13T10:12:25.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.004682795 2024-09-13T10:12:25.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.767 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.767 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.091235+0000", 2024-09-13T10:12:25.767 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00048779900000000002 2024-09-13T10:12:25.767 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.767 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.768 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.768 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112809+0000", 2024-09-13T10:12:25.768 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.021573967999999999 2024-09-13T10:12:25.768 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112878+0000", 2024-09-13T10:12:25.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9413999999999996e-05 2024-09-13T10:12:25.769 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.769 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.769 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.769 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.112936+0000", 2024-09-13T10:12:25.769 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7868999999999998e-05 2024-09-13T10:12:25.769 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.769 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.769 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.770 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.770 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:227 3.2 e38/30 3:6ac7800c:::smithi19530714-28 low_tier:head v 38'8, mlcod=38'6)", 2024-09-13T10:12:25.770 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:12:25.770 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.600132294999995, 2024-09-13T10:12:25.770 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014433425999999999, 2024-09-13T10:12:25.770 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.770 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.770 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.770 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.770 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.770 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.770 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:12:25.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074099+0000", 2024-09-13T10:12:25.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.771 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.771 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074101+0000", 2024-09-13T10:12:25.771 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.511e-06 2024-09-13T10:12:25.771 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.771 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.772 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083754+0000", 2024-09-13T10:12:25.772 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0096533799999999996 2024-09-13T10:12:25.772 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.772 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.772 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083772+0000", 2024-09-13T10:12:25.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8241e-05 2024-09-13T10:12:25.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.773 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.773 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.773 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083782+0000", 2024-09-13T10:12:25.773 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.004e-05 2024-09-13T10:12:25.773 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083875+0000", 2024-09-13T10:12:25.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.268e-05 2024-09-13T10:12:25.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.083918+0000", 2024-09-13T10:12:25.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2951000000000003e-05 2024-09-13T10:12:25.774 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.774 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.774 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.774 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.088456+0000", 2024-09-13T10:12:25.774 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045385540000000002 2024-09-13T10:12:25.775 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.775 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.775 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.775 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.088532+0000", 2024-09-13T10:12:25.775 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6069000000000006e-05 2024-09-13T10:12:25.775 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.775 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.776 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.776 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.776 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.776 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:239 3.0 3:078df4f4:::smithi19530714-40 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.776 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:12:25.776 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.584928782999995, 2024-09-13T10:12:25.776 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.025192791999999999, 2024-09-13T10:12:25.776 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.776 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.776 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.776 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.776 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.777 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.777 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 239 2024-09-13T10:12:25.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.777 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:12:25.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.777 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.777 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.777 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.777 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089303+0000", 2024-09-13T10:12:25.777 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089305+0000", 2024-09-13T10:12:25.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1710000000000001e-06 2024-09-13T10:12:25.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.778 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089312+0000", 2024-09-13T10:12:25.778 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6860000000000006e-06 2024-09-13T10:12:25.778 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.778 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.778 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089323+0000", 2024-09-13T10:12:25.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0467e-05 2024-09-13T10:12:25.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089330+0000", 2024-09-13T10:12:25.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.3939999999999998e-06 2024-09-13T10:12:25.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.779 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.779 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089379+0000", 2024-09-13T10:12:25.779 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8990999999999999e-05 2024-09-13T10:12:25.779 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.779 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.089543+0000", 2024-09-13T10:12:25.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016351700000000001 2024-09-13T10:12:25.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:25.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.098886+0000", 2024-09-13T10:12:25.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0093434980000000004 2024-09-13T10:12:25.780 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.780 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.780 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.780 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.105855+0000", 2024-09-13T10:12:25.780 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.006968279 2024-09-13T10:12:25.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:25.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.105868+0000", 2024-09-13T10:12:25.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3560999999999999e-05 2024-09-13T10:12:25.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.781 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.781 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114382+0000", 2024-09-13T10:12:25.781 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0085134730000000006 2024-09-13T10:12:25.781 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.781 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.782 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.782 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114458+0000", 2024-09-13T10:12:25.782 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6143999999999995e-05 2024-09-13T10:12:25.782 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.782 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.783 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.783 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.114495+0000", 2024-09-13T10:12:25.783 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7611000000000003e-05 2024-09-13T10:12:25.783 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.783 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.783 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.784 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.784 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.784 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:241 3.0 3:388ca573:::smithi19530714-42 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.784 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:12:25.784 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.579055061000005, 2024-09-13T10:12:25.784 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.057595649999999998, 2024-09-13T10:12:25.784 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.784 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.784 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.784 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.784 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.784 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.785 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 241 2024-09-13T10:12:25.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.785 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:12:25.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.785 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.785 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.785 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095176+0000", 2024-09-13T10:12:25.785 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.785 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095178+0000", 2024-09-13T10:12:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.345e-06 2024-09-13T10:12:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095184+0000", 2024-09-13T10:12:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4239999999999998e-06 2024-09-13T10:12:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.786 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095198+0000", 2024-09-13T10:12:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.4037e-05 2024-09-13T10:12:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095206+0000", 2024-09-13T10:12:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.1629999999999996e-06 2024-09-13T10:12:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095274+0000", 2024-09-13T10:12:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.7948999999999996e-05 2024-09-13T10:12:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.787 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.095485+0000", 2024-09-13T10:12:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00021050100000000001 2024-09-13T10:12:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.134651+0000", 2024-09-13T10:12:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.039166200999999998 2024-09-13T10:12:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.140536+0000", 2024-09-13T10:12:25.788 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0058849719999999996 2024-09-13T10:12:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.140980+0000", 2024-09-13T10:12:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00044394200000000001 2024-09-13T10:12:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152700+0000", 2024-09-13T10:12:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011720431999999999 2024-09-13T10:12:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.789 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152739+0000", 2024-09-13T10:12:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8448999999999997e-05 2024-09-13T10:12:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152772+0000", 2024-09-13T10:12:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.3235000000000003e-05 2024-09-13T10:12:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.790 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:247 3.0 3:0b30fd85:::smithi19530714-48 low_tier:head [copy-from ver 15] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:12:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.561155409999998, 2024-09-13T10:12:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.032448926000000003, 2024-09-13T10:12:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 247 2024-09-13T10:12:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.791 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:12:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113076+0000", 2024-09-13T10:12:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113078+0000", 2024-09-13T10:12:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8670000000000001e-06 2024-09-13T10:12:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.792 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113086+0000", 2024-09-13T10:12:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2260000000000005e-06 2024-09-13T10:12:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113104+0000", 2024-09-13T10:12:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7887e-05 2024-09-13T10:12:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113115+0000", 2024-09-13T10:12:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0763e-05 2024-09-13T10:12:25.793 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113163+0000", 2024-09-13T10:12:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8541999999999997e-05 2024-09-13T10:12:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.113322+0000", 2024-09-13T10:12:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000159121 2024-09-13T10:12:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:25.794 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.125165+0000", 2024-09-13T10:12:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011842320999999999 2024-09-13T10:12:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.131189+0000", 2024-09-13T10:12:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.006024728 2024-09-13T10:12:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.131203+0000", 2024-09-13T10:12:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3551e-05 2024-09-13T10:12:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.795 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145435+0000", 2024-09-13T10:12:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014232273 2024-09-13T10:12:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145481+0000", 2024-09-13T10:12:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.5918000000000003e-05 2024-09-13T10:12:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.145525+0000", 2024-09-13T10:12:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3729000000000001e-05 2024-09-13T10:12:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.796 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:244 3.2 e38/30 3:5d22c2d4:::smithi19530714-45 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'14, mlcod=38'10)", 2024-09-13T10:12:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:12:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.547689606999995, 2024-09-13T10:12:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014967827, 2024-09-13T10:12:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.797 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:12:25.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126542+0000", 2024-09-13T10:12:25.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.798 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.798 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.798 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.798 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.126544+0000", 2024-09-13T10:12:25.798 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.03e-06 2024-09-13T10:12:25.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133374+0000", 2024-09-13T10:12:25.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068299119999999996 2024-09-13T10:12:25.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.799 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133398+0000", 2024-09-13T10:12:25.799 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4209000000000001e-05 2024-09-13T10:12:25.799 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.799 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.799 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133404+0000", 2024-09-13T10:12:25.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3570000000000003e-06 2024-09-13T10:12:25.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133457+0000", 2024-09-13T10:12:25.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.2320000000000001e-05 2024-09-13T10:12:25.800 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.800 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.800 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.800 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.133477+0000", 2024-09-13T10:12:25.800 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0721999999999999e-05 2024-09-13T10:12:25.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.141475+0000", 2024-09-13T10:12:25.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0079973649999999993 2024-09-13T10:12:25.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.801 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.801 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.801 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.141510+0000", 2024-09-13T10:12:25.801 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4912e-05 2024-09-13T10:12:25.801 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.801 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.801 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.801 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.802 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.802 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1836 3.0 3:0cf17fa4:::e22ed036f41f6a390924cf1e5e56fa98e0edea9e6c33080c396679f1537fbc7a:head [write 0~2655744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-09-13T10:12:25.802 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:10.675934+0000", 2024-09-13T10:12:25.802 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 74.998296972999995, 2024-09-13T10:12:25.802 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.015481818, 2024-09-13T10:12:25.802 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.802 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.802 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.802 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.803 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.803 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.803 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1836 2024-09-13T10:12:25.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.803 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.803 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.675934+0000", 2024-09-13T10:12:25.803 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.803 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.803 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.803 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.675934+0000", 2024-09-13T10:12:25.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.675939+0000", 2024-09-13T10:12:25.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.2830000000000003e-06 2024-09-13T10:12:25.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.804 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.804 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.804 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677769+0000", 2024-09-13T10:12:25.804 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0018299830000000001 2024-09-13T10:12:25.804 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677796+0000", 2024-09-13T10:12:25.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7858999999999999e-05 2024-09-13T10:12:25.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.805 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677809+0000", 2024-09-13T10:12:25.805 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2291e-05 2024-09-13T10:12:25.805 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.805 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.805 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.677895+0000", 2024-09-13T10:12:25.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6581000000000003e-05 2024-09-13T10:12:25.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.678079+0000", 2024-09-13T10:12:25.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000183802 2024-09-13T10:12:25.806 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.806 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.806 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:25.806 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.678268+0000", 2024-09-13T10:12:25.806 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000188383 2024-09-13T10:12:25.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:25.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.683389+0000", 2024-09-13T10:12:25.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0051215840000000002 2024-09-13T10:12:25.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.807 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.684549+0000", 2024-09-13T10:12:25.807 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0011600169999999999 2024-09-13T10:12:25.807 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.807 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.807 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.691333+0000", 2024-09-13T10:12:25.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0067840510000000001 2024-09-13T10:12:25.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.691377+0000", 2024-09-13T10:12:25.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.4313000000000001e-05 2024-09-13T10:12:25.808 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.808 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.808 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.808 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:10.691416+0000", 2024-09-13T10:12:25.808 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8671000000000003e-05 2024-09-13T10:12:25.809 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.809 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.809 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.809 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.809 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.809 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3640 4.12 4:4ac406d9:::smithi19530714-26:head [tier-promote] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e89)", 2024-09-13T10:12:25.809 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:53.723202+0000", 2024-09-13T10:12:25.809 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 31.951029459000001, 2024-09-13T10:12:25.809 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.084555116999999999, 2024-09-13T10:12:25.809 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.809 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.809 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:25.810 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.810 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.810 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.810 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3640 2024-09-13T10:12:25.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.810 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723202+0000", 2024-09-13T10:12:25.810 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.810 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.810 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.810 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.810 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723202+0000", 2024-09-13T10:12:25.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723204+0000", 2024-09-13T10:12:25.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0420000000000001e-06 2024-09-13T10:12:25.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.811 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.811 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723212+0000", 2024-09-13T10:12:25.811 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2169999999999994e-06 2024-09-13T10:12:25.811 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.811 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723232+0000", 2024-09-13T10:12:25.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9419000000000002e-05 2024-09-13T10:12:25.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.812 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723245+0000", 2024-09-13T10:12:25.812 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3073e-05 2024-09-13T10:12:25.812 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.812 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.812 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.813 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723889+0000", 2024-09-13T10:12:25.813 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00064398000000000005 2024-09-13T10:12:25.813 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.813 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.813 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-13T10:12:25.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723926+0000", 2024-09-13T10:12:25.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7685999999999998e-05 2024-09-13T10:12:25.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.805162+0000", 2024-09-13T10:12:25.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.081235561999999997 2024-09-13T10:12:25.814 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.814 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.814 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.814 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.805279+0000", 2024-09-13T10:12:25.814 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000116883 2024-09-13T10:12:25.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.815 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.815 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.807757+0000", 2024-09-13T10:12:25.815 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002478255 2024-09-13T10:12:25.815 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.815 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.815 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.815 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.815 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.815 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3710 4.12 4:4a24ecac:::smithi19530714-36:head [tier-promote] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-13T10:12:25.815 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:53.723265+0000", 2024-09-13T10:12:25.815 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 31.950966413, 2024-09-13T10:12:25.816 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.085310183999999997, 2024-09-13T10:12:25.816 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.816 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.816 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:25.816 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.816 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.816 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.816 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3710 2024-09-13T10:12:25.816 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.816 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.816 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.816 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.816 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723265+0000", 2024-09-13T10:12:25.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723265+0000", 2024-09-13T10:12:25.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.817 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.817 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.817 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.817 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723266+0000", 2024-09-13T10:12:25.817 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7300000000000004e-07 2024-09-13T10:12:25.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723269+0000", 2024-09-13T10:12:25.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5130000000000001e-06 2024-09-13T10:12:25.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.818 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.818 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.818 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723279+0000", 2024-09-13T10:12:25.818 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.3049999999999998e-06 2024-09-13T10:12:25.818 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.723286+0000", 2024-09-13T10:12:25.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6990000000000003e-06 2024-09-13T10:12:25.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.724004+0000", 2024-09-13T10:12:25.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00071808899999999999 2024-09-13T10:12:25.819 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.819 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.819 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for peered", 2024-09-13T10:12:25.819 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.724020+0000", 2024-09-13T10:12:25.819 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5586999999999998e-05 2024-09-13T10:12:25.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.805788+0000", 2024-09-13T10:12:25.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.081768020999999996 2024-09-13T10:12:25.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.805868+0000", 2024-09-13T10:12:25.820 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.9469999999999996e-05 2024-09-13T10:12:25.820 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.820 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.820 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.820 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:53.808575+0000", 2024-09-13T10:12:25.821 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0027075269999999999 2024-09-13T10:12:25.821 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.821 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.821 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.821 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.821 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.821 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:3745 3.0 3:161dd296:::f6d96d7a8dac6a7d56d3d9df9ef9c133dce9c515f90d85faf1e037f770fc9485:head [write 0~3512832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-13T10:12:25.821 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:54.607585+0000", 2024-09-13T10:12:25.821 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 31.066646168999998, 2024-09-13T10:12:25.822 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.021860623999999999, 2024-09-13T10:12:25.822 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.822 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.822 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:25.822 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.822 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.823 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.823 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 3745 2024-09-13T10:12:25.823 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.823 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.823 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.823 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.607585+0000", 2024-09-13T10:12:25.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.607585+0000", 2024-09-13T10:12:25.824 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.824 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.824 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.824 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.824 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.607588+0000", 2024-09-13T10:12:25.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5790000000000002e-06 2024-09-13T10:12:25.825 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.825 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.825 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.825 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.610509+0000", 2024-09-13T10:12:25.825 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0029216020000000001 2024-09-13T10:12:25.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.610542+0000", 2024-09-13T10:12:25.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2848000000000001e-05 2024-09-13T10:12:25.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.826 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.610559+0000", 2024-09-13T10:12:25.826 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7170999999999999e-05 2024-09-13T10:12:25.826 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.826 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.826 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.610699+0000", 2024-09-13T10:12:25.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000140032 2024-09-13T10:12:25.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.611058+0000", 2024-09-13T10:12:25.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000358246 2024-09-13T10:12:25.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 4,7", 2024-09-13T10:12:25.827 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.611462+0000", 2024-09-13T10:12:25.827 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000404165 2024-09-13T10:12:25.827 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.827 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.827 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:25.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.622881+0000", 2024-09-13T10:12:25.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011419558999999999 2024-09-13T10:12:25.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.624371+0000", 2024-09-13T10:12:25.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001489331 2024-09-13T10:12:25.828 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.828 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.828 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:25.828 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.629336+0000", 2024-09-13T10:12:25.828 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0049654850000000004 2024-09-13T10:12:25.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:25.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.629395+0000", 2024-09-13T10:12:25.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8405000000000001e-05 2024-09-13T10:12:25.829 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.829 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.829 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.829 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.629446+0000", 2024-09-13T10:12:25.829 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1201e-05 2024-09-13T10:12:25.829 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.829 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.830 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.830 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.830 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.830 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:4825 4.18 4:1de9da87:::smithi19530714-5:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e112)", 2024-09-13T10:12:25.830 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:12:20.955621+0000", 2024-09-13T10:12:25.830 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 4.71861028, 2024-09-13T10:12:25.830 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052702040999999998, 2024-09-13T10:12:25.830 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:25.830 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:25.830 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:25.830 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:25.830 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:25.830 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:25.830 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 4825 2024-09-13T10:12:25.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.831 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:25.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:25.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:20.955621+0000", 2024-09-13T10:12:25.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:25.831 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:20.955621+0000", 2024-09-13T10:12:25.831 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:25.831 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.831 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.831 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:25.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:20.955624+0000", 2024-09-13T10:12:25.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5170000000000002e-06 2024-09-13T10:12:25.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:25.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:20.955635+0000", 2024-09-13T10:12:25.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1005999999999999e-05 2024-09-13T10:12:25.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.832 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:25.832 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:20.955665+0000", 2024-09-13T10:12:25.832 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.057e-05 2024-09-13T10:12:25.832 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.832 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:25.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:20.955681+0000", 2024-09-13T10:12:25.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6172999999999999e-05 2024-09-13T10:12:25.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:25.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:20.955809+0000", 2024-09-13T10:12:25.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012774300000000001 2024-09-13T10:12:25.833 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.833 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.833 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:25.833 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:20.955955+0000", 2024-09-13T10:12:25.833 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014566199999999999 2024-09-13T10:12:25.834 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:25.834 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:25.834 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:25.834 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:12:21.008323+0000", 2024-09-13T10:12:25.834 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.052368369999999997 2024-09-13T10:12:25.834 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.834 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.834 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.834 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:25.834 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:25.834 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:25.836 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T10:12:25.837 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:25.837 INFO:tasks.rados.rados.0.smithi195.stdout:2010: finishing set_chunk to oid 2 2024-09-13T10:12:25.837 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 159 (ObjNum 412 snap 0 seq_num 412) dirty exists 2024-09-13T10:12:25.837 INFO:tasks.rados.rados.0.smithi195.stdout:2011: seq_num 455 ranges {447185=703122,1642601=765814,2928991=447707} 2024-09-13T10:12:25.837 INFO:tasks.rados.rados.0.smithi195.stdout:2011: writing smithi19530714-14 from 447185 to 1150307 tid 1 2024-09-13T10:12:25.837 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:25.829+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:25.840 INFO:tasks.rados.rados.0.smithi195.stdout:2011: writing smithi19530714-14 from 1642601 to 2408415 tid 2 2024-09-13T10:12:25.853 INFO:tasks.rados.rados.0.smithi195.stdout:2011: writing smithi19530714-14 from 2928991 to 3376698 tid 3 2024-09-13T10:12:25.853 INFO:tasks.rados.rados.0.smithi195.stdout:2007: done (4 left) 2024-09-13T10:12:25.853 INFO:tasks.rados.rados.0.smithi195.stdout:2008: done (3 left) 2024-09-13T10:12:25.853 INFO:tasks.rados.rados.0.smithi195.stdout:2009: done (2 left) 2024-09-13T10:12:25.853 INFO:tasks.rados.rados.0.smithi195.stdout:2010: done (1 left) 2024-09-13T10:12:25.853 INFO:tasks.rados.rados.0.smithi195.stdout:2012: read oid 4 snap -1 2024-09-13T10:12:25.853 INFO:tasks.rados.rados.0.smithi195.stdout:2012: expect (ObjNum 435 snap 0 seq_num 435) 2024-09-13T10:12:25.853 INFO:tasks.rados.rados.0.smithi195.stdout:2013: read oid 25 snap -1 2024-09-13T10:12:25.853 INFO:tasks.rados.rados.0.smithi195.stdout:2013: expect (ObjNum 324 snap 0 seq_num 324) 2024-09-13T10:12:25.853 INFO:tasks.rados.rados.0.smithi195.stdout:2014: read oid 3 snap -1 2024-09-13T10:12:25.853 INFO:tasks.rados.rados.0.smithi195.stdout:2014: expect (ObjNum 328 snap 0 seq_num 328) 2024-09-13T10:12:25.854 INFO:tasks.rados.rados.0.smithi195.stdout:2015: set_chunk oid 21 target oid 2024-09-13T10:12:25.866 INFO:tasks.rados.rados.0.smithi195.stdout:2015: set_chunk oid 21 offset: 1020416 length: 338432 target oid 163364a9d72d81aa44ec3087835cc944802d115cf7d5688e424a77e9920bda59 offset: 0 snap -1 2024-09-13T10:12:25.866 INFO:tasks.rados.rados.0.smithi195.stdout:2016: tier_evict oid 43 2024-09-13T10:12:25.866 INFO:tasks.rados.rados.0.smithi195.stdout:2016: tier_evict oid 43 snap -1 2024-09-13T10:12:25.866 INFO:tasks.rados.rados.0.smithi195.stdout:2011: finishing write tid 1 to smithi19530714-14 2024-09-13T10:12:25.867 INFO:tasks.rados.rados.0.smithi195.stdout:2017: delete oid 19 current snap is 0 2024-09-13T10:12:25.867 INFO:tasks.rados.rados.0.smithi195.stdout:2011: finishing write tid 2 to smithi19530714-14 2024-09-13T10:12:25.867 INFO:tasks.rados.rados.0.smithi195.stdout:2012: expect (ObjNum 435 snap 0 seq_num 435) 2024-09-13T10:12:25.871 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 630449 2024-09-13T10:12:25.915 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3230955 2024-09-13T10:12:25.934 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T10:12:25.936 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:12:25.934+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:12:25.949 INFO:tasks.rados.rados.0.smithi195.stdout:2016: got (22) Invalid argument 2024-09-13T10:12:25.949 INFO:tasks.rados.rados.0.smithi195.stdout:2014: expect (ObjNum 328 snap 0 seq_num 328) 2024-09-13T10:12:25.958 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:25.958 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:12:25.958 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:12:25.958 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:25.959 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:12:25.960 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:12:25.971 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T10:12:25.979 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-13T10:12:25.919811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":230588592,"num_objects":183,"num_object_clones":0,"num_object_copies":549,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":6053,"num_read_kb":4305231,"num_write":3493,"num_write_kb":791567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":63971336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":20,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":98,"ondisk_log_size":98,"up":117,"acting":117,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":1040560,"kb_used_data":551400,"kb_used_omap":24,"kb_used_meta":486183,"kb_avail":1508908880,"statfs":{"total":1546188226560,"available":1545122693120,"internally_reserved":0,"allocated":564633600,"data_stored":555134025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":497851471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":423523,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-6,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":251,"num_read_kb":228782,"num_write":95,"num_write_kb":22500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3697265,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.959146"},"pg_stats":[{"pgid":"4.19","version":"106'70","reported_seq":380,"reported_epoch":115,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:12:23.909877+0000","last_change":"2024-09-13T10:12:13.894581+0000","last_active":"2024-09-13T10:12:23.909877+0000","last_peered":"2024-09-13T10:12:23.909877+0000","last_clean":"2024-09-13T10:12:09.846417+0000","last_became_active":"2024-09-13T10:12:13.882915+0000","last_became_peered":"2024-09-13T10:12:13.882915+0000","last_unstale":"2024-09-13T10:12:23.909877+0000","last_undegraded":"2024-09-13T10:12:23.909877+0000","last_fullsized":"2024-09-13T10:12:23.909877+0000","mapping_epoch":105,"log_start":"106'67","ondisk_log_start":"106'67","created":41,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'28","last_scrub_stamp":"2024-09-13T10:11:00.388132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.388132+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":67,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5294337,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":131646,"num_write":46,"num_write_kb":11392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2376704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,9],"acting":[2,5,3],"avail_no_missing":["2","3","5"],"object_location_counts":[{"shards":"2,3,5","objects":2}],"blocked_by":[],"up_primary":11,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"112'35","reported_seq":282,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:21.390230+0000","last_change":"2024-09-13T10:12:17.724625+0000","last_active":"2024-09-13T10:12:21.390230+0000","last_peered":"2024-09-13T10:12:21.390230+0000","last_clean":"2024-09-13T10:12:21.390230+0000","last_became_active":"2024-09-13T10:12:17.721145+0000","last_became_peered":"2024-09-13T10:12:17.721145+0000","last_unstale":"2024-09-13T10:12:21.390230+0000","last_undegraded":"2024-09-13T10:12:21.390230+0000","last_fullsized":"2024-09-13T10:12:21.390230+0000","mapping_epoch":109,"log_start":"112'31","ondisk_log_start":"112'31","created":41,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'5","last_scrub_stamp":"2024-09-13T10:11:01.783496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.783496+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":31,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:25.388761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294238,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":89615,"num_write":33,"num_write_kb":9625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3925645,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,0],"acting":[3,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"112'139","reported_seq":527,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:21.010174+0000","last_change":"2024-09-13T10:12:19.734795+0000","last_active":"2024-09-13T10:12:21.010174+0000","last_peered":"2024-09-13T10:12:21.010174+0000","last_clean":"2024-09-13T10:12:21.010174+0000","last_became_active":"2024-09-13T10:12:19.734082+0000","last_became_peered":"2024-09-13T10:12:19.734082+0000","last_unstale":"2024-09-13T10:12:21.010174+0000","last_undegraded":"2024-09-13T10:12:21.010174+0000","last_fullsized":"2024-09-13T10:12:21.010174+0000","mapping_epoch":111,"log_start":"112'136","ondisk_log_start":"112'136","created":41,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":5,"last_scrub":"105'131","last_scrub_stamp":"2024-09-13T10:12:14.226050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:14.226050+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":136,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:32.254492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9176820,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":263,"num_read_kb":230127,"num_write":132,"num_write_kb":38525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":7608719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,10],"acting":[1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"112'84","reported_seq":424,"reported_epoch":114,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:12:23.035781+0000","last_change":"2024-09-13T10:12:18.770392+0000","last_active":"2024-09-13T10:12:23.035781+0000","last_peered":"2024-09-13T10:12:23.035781+0000","last_clean":"2024-09-13T10:12:03.112281+0000","last_became_active":"2024-09-13T10:12:18.754536+0000","last_became_peered":"2024-09-13T10:12:18.754536+0000","last_unstale":"2024-09-13T10:12:23.035781+0000","last_undegraded":"2024-09-13T10:12:23.035781+0000","last_fullsized":"2024-09-13T10:12:23.035781+0000","mapping_epoch":110,"log_start":"111'81","ondisk_log_start":"111'81","created":41,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":5,"last_scrub":"46'44","last_scrub_stamp":"2024-09-13T10:11:00.456457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.456457+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":78,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6790127,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":184,"num_read_kb":214332,"num_write":46,"num_write_kb":12320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3338234,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,4],"acting":[14,5,13],"avail_no_missing":["14","5","13"],"object_location_counts":[{"shards":"5,13,14","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.15","version":"112'52","reported_seq":335,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-13T10:12:22.967943+0000","last_change":"2024-09-13T10:12:16.084525+0000","last_active":"2024-09-13T10:12:22.967943+0000","last_peered":"2024-09-13T10:12:22.967943+0000","last_clean":"2024-09-13T10:12:22.967943+0000","last_became_active":"2024-09-13T10:11:23.705370+0000","last_became_peered":"2024-09-13T10:11:23.705370+0000","last_unstale":"2024-09-13T10:12:22.967943+0000","last_undegraded":"2024-09-13T10:12:22.967943+0000","last_fullsized":"2024-09-13T10:12:22.967943+0000","mapping_epoch":64,"log_start":"112'49","ondisk_log_start":"112'49","created":41,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":5,"last_scrub":"107'48","last_scrub_stamp":"2024-09-13T10:12:16.084431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:16.084431+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":47,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:25.274164+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3378273,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":45,"num_read_kb":39290,"num_write":15,"num_write_kb":4430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1702851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,13],"acting":[6,7,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"39'40","reported_seq":234,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:22.081863+0000","last_change":"2024-09-13T10:12:00.310899+0000","last_active":"2024-09-13T10:12:22.081863+0000","last_peered":"2024-09-13T10:12:22.081863+0000","last_clean":"2024-09-13T10:12:22.081863+0000","last_became_active":"2024-09-13T10:11:47.715516+0000","last_became_peered":"2024-09-13T10:11:47.715516+0000","last_unstale":"2024-09-13T10:12:22.081863+0000","last_undegraded":"2024-09-13T10:12:22.081863+0000","last_fullsized":"2024-09-13T10:12:22.081863+0000","mapping_epoch":85,"log_start":"39'37","ondisk_log_start":"39'37","created":41,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'40","last_scrub_stamp":"2024-09-13T10:12:00.310748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.310748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":37,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:05.649596+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.13","version":"98'44","reported_seq":330,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:21.622176+0000","last_change":"2024-09-13T10:12:12.572121+0000","last_active":"2024-09-13T10:12:21.622176+0000","last_peered":"2024-09-13T10:12:21.622176+0000","last_clean":"2024-09-13T10:12:21.622176+0000","last_became_active":"2024-09-13T10:11:15.757364+0000","last_became_peered":"2024-09-13T10:11:15.757364+0000","last_unstale":"2024-09-13T10:12:21.622176+0000","last_undegraded":"2024-09-13T10:12:21.622176+0000","last_fullsized":"2024-09-13T10:12:21.622176+0000","mapping_epoch":58,"log_start":"97'41","ondisk_log_start":"97'41","created":41,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":5,"last_scrub":"98'44","last_scrub_stamp":"2024-09-13T10:12:12.572014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:12.572014+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":39,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:21.886670+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2336239,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":75,"num_read_kb":61934,"num_write":31,"num_write_kb":7734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1493592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"113'55","reported_seq":378,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:22.103145+0000","last_change":"2024-09-13T10:11:55.737652+0000","last_active":"2024-09-13T10:12:22.103145+0000","last_peered":"2024-09-13T10:12:22.103145+0000","last_clean":"2024-09-13T10:12:22.103145+0000","last_became_active":"2024-09-13T10:11:55.736402+0000","last_became_peered":"2024-09-13T10:11:55.736402+0000","last_unstale":"2024-09-13T10:12:22.103145+0000","last_undegraded":"2024-09-13T10:12:22.103145+0000","last_fullsized":"2024-09-13T10:12:22.103145+0000","mapping_epoch":91,"log_start":"100'52","ondisk_log_start":"100'52","created":41,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'17","last_scrub_stamp":"2024-09-13T10:11:02.582829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:02.582829+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":52,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:25.395942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2369853,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":169,"num_read_kb":136827,"num_write":36,"num_write_kb":10155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":7321937,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,13],"acting":[1,4,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"112'38","reported_seq":304,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-13T10:12:20.172233+0000","last_change":"2024-09-13T10:11:40.065958+0000","last_active":"2024-09-13T10:12:20.172233+0000","last_peered":"2024-09-13T10:12:20.172233+0000","last_clean":"2024-09-13T10:12:20.172233+0000","last_became_active":"2024-09-13T10:11:40.064608+0000","last_became_peered":"2024-09-13T10:11:40.064608+0000","last_unstale":"2024-09-13T10:12:20.172233+0000","last_undegraded":"2024-09-13T10:12:20.172233+0000","last_fullsized":"2024-09-13T10:12:20.172233+0000","mapping_epoch":77,"log_start":"112'35","ondisk_log_start":"112'35","created":41,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":5,"last_scrub":"40'13","last_scrub_stamp":"2024-09-13T10:11:04.242723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.242723+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":32,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:29.857627+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3344659,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":79,"num_read_kb":61838,"num_write":30,"num_write_kb":10139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1805029,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,12],"acting":[13,4,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.10","version":"107'61","reported_seq":396,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:20.890080+0000","last_change":"2024-09-13T10:12:13.384690+0000","last_active":"2024-09-13T10:12:20.890080+0000","last_peered":"2024-09-13T10:12:20.890080+0000","last_clean":"2024-09-13T10:12:20.890080+0000","last_became_active":"2024-09-13T10:11:40.063380+0000","last_became_peered":"2024-09-13T10:11:40.063380+0000","last_unstale":"2024-09-13T10:12:20.890080+0000","last_undegraded":"2024-09-13T10:12:20.890080+0000","last_fullsized":"2024-09-13T10:12:20.890080+0000","mapping_epoch":77,"log_start":"97'58","ondisk_log_start":"97'58","created":41,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":5,"last_scrub":"98'60","last_scrub_stamp":"2024-09-13T10:12:13.384591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:13.384591+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":56,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:20.055922+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5344725,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":197,"num_read_kb":161975,"num_write":51,"num_write_kb":14184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2889991,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,14,3],"acting":[9,14,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":135,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:21.390055+0000","last_change":"2024-09-13T10:12:03.961952+0000","last_active":"2024-09-13T10:12:21.390055+0000","last_peered":"2024-09-13T10:12:21.390055+0000","last_clean":"2024-09-13T10:12:21.390055+0000","last_became_active":"2024-09-13T10:11:12.744090+0000","last_became_peered":"2024-09-13T10:11:12.744090+0000","last_unstale":"2024-09-13T10:12:21.390055+0000","last_undegraded":"2024-09-13T10:12:21.390055+0000","last_fullsized":"2024-09-13T10:12:21.390055+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:03.961779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:03.961779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:12.481787+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,12],"acting":[3,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"39'12","reported_seq":237,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-13T10:12:22.967678+0000","last_change":"2024-09-13T10:11:01.537080+0000","last_active":"2024-09-13T10:12:22.967678+0000","last_peered":"2024-09-13T10:12:22.967678+0000","last_clean":"2024-09-13T10:12:22.967678+0000","last_became_active":"2024-09-13T10:10:54.318519+0000","last_became_peered":"2024-09-13T10:10:54.318519+0000","last_unstale":"2024-09-13T10:12:22.967678+0000","last_undegraded":"2024-09-13T10:12:22.967678+0000","last_fullsized":"2024-09-13T10:12:22.967678+0000","mapping_epoch":31,"log_start":"38'9","ondisk_log_start":"38'9","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'12","last_scrub_stamp":"2024-09-13T10:11:01.536963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.536963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":9,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,14],"acting":[6,8,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":199,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:21.390209+0000","last_change":"2024-09-13T10:12:12.892985+0000","last_active":"2024-09-13T10:12:21.390209+0000","last_peered":"2024-09-13T10:12:21.390209+0000","last_clean":"2024-09-13T10:12:21.390209+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:12:21.390209+0000","last_undegraded":"2024-09-13T10:12:21.390209+0000","last_fullsized":"2024-09-13T10:12:21.390209+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:12.892836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:12.892836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:20.216623+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"112'102","reported_seq":433,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T10:12:23.909750+0000","last_change":"2024-09-13T10:11:01.219554+0000","last_active":"2024-09-13T10:12:23.909750+0000","last_peered":"2024-09-13T10:12:23.909750+0000","last_clean":"2024-09-13T10:12:23.909750+0000","last_became_active":"2024-09-13T10:10:54.314545+0000","last_became_peered":"2024-09-13T10:10:54.314545+0000","last_unstale":"2024-09-13T10:12:23.909750+0000","last_undegraded":"2024-09-13T10:12:23.909750+0000","last_fullsized":"2024-09-13T10:12:23.909750+0000","mapping_epoch":31,"log_start":"102'99","ondisk_log_start":"102'99","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'27","last_scrub_stamp":"2024-09-13T10:11:01.219486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.219486+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":98,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4571636,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":223,"num_read_kb":179975,"num_write":91,"num_write_kb":24881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,8],"acting":[2,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":199,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:21.026732+0000","last_change":"2024-09-13T10:12:01.407368+0000","last_active":"2024-09-13T10:12:21.026732+0000","last_peered":"2024-09-13T10:12:21.026732+0000","last_clean":"2024-09-13T10:12:21.026732+0000","last_became_active":"2024-09-13T10:10:30.128658+0000","last_became_peered":"2024-09-13T10:10:30.128658+0000","last_unstale":"2024-09-13T10:12:21.026732+0000","last_undegraded":"2024-09-13T10:12:21.026732+0000","last_fullsized":"2024-09-13T10:12:21.026732+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:01.407274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:01.407274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:11.291191+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.0","version":"111'43","reported_seq":315,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-13T10:12:22.718181+0000","last_change":"2024-09-13T10:11:05.681812+0000","last_active":"2024-09-13T10:12:22.718181+0000","last_peered":"2024-09-13T10:12:22.718181+0000","last_clean":"2024-09-13T10:12:22.718181+0000","last_became_active":"2024-09-13T10:10:54.320014+0000","last_became_peered":"2024-09-13T10:10:54.320014+0000","last_unstale":"2024-09-13T10:12:22.718181+0000","last_undegraded":"2024-09-13T10:12:22.718181+0000","last_fullsized":"2024-09-13T10:12:22.718181+0000","mapping_epoch":31,"log_start":"111'40","ondisk_log_start":"111'40","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'14","last_scrub_stamp":"2024-09-13T10:11:05.681653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.681653+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":39,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:28.623762+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3243817,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":86,"num_read_kb":77734,"num_write":33,"num_write_kb":8858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":43,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:21.389691+0000","last_change":"2024-09-13T10:12:05.204517+0000","last_active":"2024-09-13T10:12:21.389691+0000","last_peered":"2024-09-13T10:12:21.389691+0000","last_clean":"2024-09-13T10:12:21.389691+0000","last_became_active":"2024-09-13T10:12:05.203738+0000","last_became_peered":"2024-09-13T10:12:05.203738+0000","last_unstale":"2024-09-13T10:12:21.389691+0000","last_undegraded":"2024-09-13T10:12:21.389691+0000","last_fullsized":"2024-09-13T10:12:21.389691+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:11:41.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:41.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:49.027751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12,1],"acting":[3,12,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"111'45","reported_seq":304,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T10:12:23.909637+0000","last_change":"2024-09-13T10:11:02.190704+0000","last_active":"2024-09-13T10:12:23.909637+0000","last_peered":"2024-09-13T10:12:23.909637+0000","last_clean":"2024-09-13T10:12:23.909637+0000","last_became_active":"2024-09-13T10:10:54.317600+0000","last_became_peered":"2024-09-13T10:10:54.317600+0000","last_unstale":"2024-09-13T10:12:23.909637+0000","last_undegraded":"2024-09-13T10:12:23.909637+0000","last_fullsized":"2024-09-13T10:12:23.909637+0000","mapping_epoch":31,"log_start":"109'42","ondisk_log_start":"109'42","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'18","last_scrub_stamp":"2024-09-13T10:11:02.190506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:02.190506+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":42,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":73519,"num_write":32,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,12],"acting":[2,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"112'316","reported_seq":586,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-13T10:12:20.192511+0000","last_change":"2024-09-13T10:12:07.816766+0000","last_active":"2024-09-13T10:12:20.192511+0000","last_peered":"2024-09-13T10:12:20.192511+0000","last_clean":"2024-09-13T10:12:20.192511+0000","last_became_active":"2024-09-13T10:12:07.813457+0000","last_became_peered":"2024-09-13T10:12:07.813457+0000","last_unstale":"2024-09-13T10:12:20.192511+0000","last_undegraded":"2024-09-13T10:12:20.192511+0000","last_fullsized":"2024-09-13T10:12:20.192511+0000","mapping_epoch":100,"log_start":"108'313","ondisk_log_start":"108'313","created":30,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":328,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:22.752162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26337596,"num_objects":30,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":293,"num_read_kb":6999,"num_write":408,"num_write_kb":77258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":26621319,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,10],"acting":[13,14,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":282,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:21.390347+0000","last_change":"2024-09-13T10:11:48.946651+0000","last_active":"2024-09-13T10:12:21.390347+0000","last_peered":"2024-09-13T10:12:21.390347+0000","last_clean":"2024-09-13T10:12:21.390347+0000","last_became_active":"2024-09-13T10:11:12.699043+0000","last_became_peered":"2024-09-13T10:11:12.699043+0000","last_unstale":"2024-09-13T10:12:21.390347+0000","last_undegraded":"2024-09-13T10:12:21.390347+0000","last_fullsized":"2024-09-13T10:12:21.390347+0000","mapping_epoch":55,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'112","last_scrub_stamp":"2024-09-13T10:11:48.946478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:48.946478+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:12.399266+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,7],"acting":[3,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":105,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-13T10:12:19.723896+0000","last_change":"2024-09-13T10:11:38.808286+0000","last_active":"2024-09-13T10:12:19.723896+0000","last_peered":"2024-09-13T10:12:19.723896+0000","last_clean":"2024-09-13T10:12:19.723896+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:12:19.723896+0000","last_undegraded":"2024-09-13T10:12:19.723896+0000","last_fullsized":"2024-09-13T10:12:19.723896+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T10:11:38.808119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:38.808119+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:43.158000+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.6","version":"105'93","reported_seq":402,"reported_epoch":113,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:12:21.010057+0000","last_change":"2024-09-13T10:12:05.805129+0000","last_active":"2024-09-13T10:12:21.010057+0000","last_peered":"2024-09-13T10:12:21.010057+0000","last_clean":"2024-09-13T10:12:00.406498+0000","last_became_active":"2024-09-13T10:12:05.800127+0000","last_became_peered":"2024-09-13T10:12:05.800127+0000","last_unstale":"2024-09-13T10:12:21.010057+0000","last_undegraded":"2024-09-13T10:12:21.010057+0000","last_fullsized":"2024-09-13T10:12:21.010057+0000","mapping_epoch":98,"log_start":"105'90","ondisk_log_start":"105'90","created":31,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'44","last_scrub_stamp":"2024-09-13T10:11:05.396838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.396838+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":90,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6538022,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":153,"num_read_kb":149407,"num_write":57,"num_write_kb":16492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,4],"acting":[1,7,13],"avail_no_missing":["1","7","13"],"object_location_counts":[{"shards":"1,7,13","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"112'356","reported_seq":561,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:21.389973+0000","last_change":"2024-09-13T10:12:00.967692+0000","last_active":"2024-09-13T10:12:21.389973+0000","last_peered":"2024-09-13T10:12:21.389973+0000","last_clean":"2024-09-13T10:12:21.389973+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:12:21.389973+0000","last_undegraded":"2024-09-13T10:12:21.389973+0000","last_fullsized":"2024-09-13T10:12:21.389973+0000","mapping_epoch":30,"log_start":"111'352","ondisk_log_start":"111'352","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'314","last_scrub_stamp":"2024-09-13T10:12:00.967585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.967585+0000","objects_scrubbed":33,"log_size":4,"log_dups_size":364,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:19.050419+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30645781,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":328,"num_read_kb":12993,"num_write":471,"num_write_kb":97114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":207,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T10:12:23.909541+0000","last_change":"2024-09-13T10:11:30.291380+0000","last_active":"2024-09-13T10:12:23.909541+0000","last_peered":"2024-09-13T10:12:23.909541+0000","last_clean":"2024-09-13T10:12:23.909541+0000","last_became_active":"2024-09-13T10:10:30.127232+0000","last_became_peered":"2024-09-13T10:10:30.127232+0000","last_unstale":"2024-09-13T10:12:23.909541+0000","last_undegraded":"2024-09-13T10:12:23.909541+0000","last_fullsized":"2024-09-13T10:12:23.909541+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-13T10:11:30.291157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:30.291157+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:31.417661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0034622780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"108'104","reported_seq":410,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:21.010337+0000","last_change":"2024-09-13T10:11:05.436449+0000","last_active":"2024-09-13T10:12:21.010337+0000","last_peered":"2024-09-13T10:12:21.010337+0000","last_clean":"2024-09-13T10:12:21.010337+0000","last_became_active":"2024-09-13T10:10:54.314465+0000","last_became_peered":"2024-09-13T10:10:54.314465+0000","last_unstale":"2024-09-13T10:12:21.010337+0000","last_undegraded":"2024-09-13T10:12:21.010337+0000","last_fullsized":"2024-09-13T10:12:21.010337+0000","mapping_epoch":31,"log_start":"107'101","ondisk_log_start":"107'101","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'40","last_scrub_stamp":"2024-09-13T10:11:05.436272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.436272+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":98,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:23.127724+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5496021,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":132,"num_read_kb":118154,"num_write":75,"num_write_kb":22251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,10],"acting":[1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"113'338","reported_seq":531,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:22.100515+0000","last_change":"2024-09-13T10:12:02.384819+0000","last_active":"2024-09-13T10:12:22.100515+0000","last_peered":"2024-09-13T10:12:22.100515+0000","last_clean":"2024-09-13T10:12:22.100515+0000","last_became_active":"2024-09-13T10:10:43.220074+0000","last_became_peered":"2024-09-13T10:10:43.220074+0000","last_unstale":"2024-09-13T10:12:22.100515+0000","last_undegraded":"2024-09-13T10:12:22.100515+0000","last_fullsized":"2024-09-13T10:12:22.100515+0000","mapping_epoch":30,"log_start":"112'335","ondisk_log_start":"112'335","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'292","last_scrub_stamp":"2024-09-13T10:12:02.384617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:02.384617+0000","objects_scrubbed":28,"log_size":3,"log_dups_size":347,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:26.698118+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27295627,"num_objects":41,"num_object_clones":0,"num_object_copies":123,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":304,"num_read_kb":4753,"num_write":446,"num_write_kb":81343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:21.026673+0000","last_change":"2024-09-13T10:11:53.335859+0000","last_active":"2024-09-13T10:12:21.026673+0000","last_peered":"2024-09-13T10:12:21.026673+0000","last_clean":"2024-09-13T10:12:21.026673+0000","last_became_active":"2024-09-13T10:11:39.046090+0000","last_became_peered":"2024-09-13T10:11:39.046090+0000","last_unstale":"2024-09-13T10:12:21.026673+0000","last_undegraded":"2024-09-13T10:12:21.026673+0000","last_fullsized":"2024-09-13T10:12:21.026673+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:11:53.335756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:53.335756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:17.649356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,7,9],"acting":[15,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.4","version":"113'200","reported_seq":732,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-13T10:12:23.018552+0000","last_change":"2024-09-13T10:11:03.633738+0000","last_active":"2024-09-13T10:12:23.018552+0000","last_peered":"2024-09-13T10:12:23.018552+0000","last_clean":"2024-09-13T10:12:23.018552+0000","last_became_active":"2024-09-13T10:10:54.316518+0000","last_became_peered":"2024-09-13T10:10:54.316518+0000","last_unstale":"2024-09-13T10:12:23.018552+0000","last_undegraded":"2024-09-13T10:12:23.018552+0000","last_fullsized":"2024-09-13T10:12:23.018552+0000","mapping_epoch":31,"log_start":"113'197","ondisk_log_start":"113'197","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'66","last_scrub_stamp":"2024-09-13T10:11:03.633499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:03.633499+0000","objects_scrubbed":5,"log_size":3,"log_dups_size":197,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18051081,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":639,"num_read_kb":617401,"num_write":151,"num_write_kb":42679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10],"acting":[6,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"112'395","reported_seq":596,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-13T10:12:23.035762+0000","last_change":"2024-09-13T10:12:00.627361+0000","last_active":"2024-09-13T10:12:23.035762+0000","last_peered":"2024-09-13T10:12:23.035762+0000","last_clean":"2024-09-13T10:12:23.035762+0000","last_became_active":"2024-09-13T10:10:43.229961+0000","last_became_peered":"2024-09-13T10:10:43.229961+0000","last_unstale":"2024-09-13T10:12:23.035762+0000","last_undegraded":"2024-09-13T10:12:23.035762+0000","last_fullsized":"2024-09-13T10:12:23.035762+0000","mapping_epoch":30,"log_start":"111'392","ondisk_log_start":"111'392","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'359","last_scrub_stamp":"2024-09-13T10:12:00.627252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.627252+0000","objects_scrubbed":25,"log_size":3,"log_dups_size":401,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:24.042469+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18033848,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":360,"num_read_kb":6440,"num_write":492,"num_write_kb":96018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":202,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-13T10:12:22.967614+0000","last_change":"2024-09-13T10:11:51.041068+0000","last_active":"2024-09-13T10:12:22.967614+0000","last_peered":"2024-09-13T10:12:22.967614+0000","last_clean":"2024-09-13T10:12:22.967614+0000","last_became_active":"2024-09-13T10:10:30.272007+0000","last_became_peered":"2024-09-13T10:10:30.272007+0000","last_unstale":"2024-09-13T10:12:22.967614+0000","last_undegraded":"2024-09-13T10:12:22.967614+0000","last_fullsized":"2024-09-13T10:12:22.967614+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:11:51.040975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:51.040975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:13.227058+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.000987604,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"112'173","reported_seq":629,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-13T10:12:23.357112+0000","last_change":"2024-09-13T10:11:00.541038+0000","last_active":"2024-09-13T10:12:23.357112+0000","last_peered":"2024-09-13T10:12:23.357112+0000","last_clean":"2024-09-13T10:12:23.357112+0000","last_became_active":"2024-09-13T10:10:54.315196+0000","last_became_peered":"2024-09-13T10:10:54.315196+0000","last_unstale":"2024-09-13T10:12:23.357112+0000","last_undegraded":"2024-09-13T10:12:23.357112+0000","last_fullsized":"2024-09-13T10:12:23.357112+0000","mapping_epoch":31,"log_start":"109'170","ondisk_log_start":"109'170","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'48","last_scrub_stamp":"2024-09-13T10:11:00.540905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.540905+0000","objects_scrubbed":5,"log_size":3,"log_dups_size":169,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:27.963105+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12084431,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":444,"num_read_kb":414871,"num_write":147,"num_write_kb":45597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,0],"acting":[6,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"39'5","reported_seq":226,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:21.389842+0000","last_change":"2024-09-13T10:12:13.871233+0000","last_active":"2024-09-13T10:12:21.389842+0000","last_peered":"2024-09-13T10:12:21.389842+0000","last_clean":"2024-09-13T10:12:21.389842+0000","last_became_active":"2024-09-13T10:10:54.314935+0000","last_became_peered":"2024-09-13T10:10:54.314935+0000","last_unstale":"2024-09-13T10:12:21.389842+0000","last_undegraded":"2024-09-13T10:12:21.389842+0000","last_fullsized":"2024-09-13T10:12:21.389842+0000","mapping_epoch":31,"log_start":"38'2","ondisk_log_start":"38'2","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'5","last_scrub_stamp":"2024-09-13T10:12:13.871148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:13.871148+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:42.986641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,0],"acting":[3,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"108'45","reported_seq":320,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:22.081848+0000","last_change":"2024-09-13T10:11:04.446852+0000","last_active":"2024-09-13T10:12:22.081848+0000","last_peered":"2024-09-13T10:12:22.081848+0000","last_clean":"2024-09-13T10:12:22.081848+0000","last_became_active":"2024-09-13T10:10:54.316057+0000","last_became_peered":"2024-09-13T10:10:54.316057+0000","last_unstale":"2024-09-13T10:12:22.081848+0000","last_undegraded":"2024-09-13T10:12:22.081848+0000","last_fullsized":"2024-09-13T10:12:22.081848+0000","mapping_epoch":31,"log_start":"79'42","ondisk_log_start":"79'42","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'29","last_scrub_stamp":"2024-09-13T10:11:04.446717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:04.446717+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":39,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:29.249908+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2806802,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":127,"num_read_kb":108933,"num_write":14,"num_write_kb":3748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,9],"acting":[11,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.a","version":"112'29","reported_seq":267,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-13T10:12:22.967619+0000","last_change":"2024-09-13T10:11:12.708135+0000","last_active":"2024-09-13T10:12:22.967619+0000","last_peered":"2024-09-13T10:12:22.967619+0000","last_clean":"2024-09-13T10:12:22.967619+0000","last_became_active":"2024-09-13T10:11:12.707306+0000","last_became_peered":"2024-09-13T10:11:12.707306+0000","last_unstale":"2024-09-13T10:12:22.967619+0000","last_undegraded":"2024-09-13T10:12:22.967619+0000","last_fullsized":"2024-09-13T10:12:22.967619+0000","mapping_epoch":55,"log_start":"112'26","ondisk_log_start":"112'26","created":31,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":26,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:23.046824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3361466,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":79,"num_read_kb":66614,"num_write":30,"num_write_kb":9665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,13],"acting":[6,8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"109'89","reported_seq":451,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-09-13T10:12:23.909632+0000","last_change":"2024-09-13T10:12:17.219674+0000","last_active":"2024-09-13T10:12:23.909632+0000","last_peered":"2024-09-13T10:12:23.909632+0000","last_clean":"2024-09-13T10:12:23.909632+0000","last_became_active":"2024-09-13T10:12:07.819670+0000","last_became_peered":"2024-09-13T10:12:07.819670+0000","last_unstale":"2024-09-13T10:12:23.909632+0000","last_undegraded":"2024-09-13T10:12:23.909632+0000","last_fullsized":"2024-09-13T10:12:23.909632+0000","mapping_epoch":100,"log_start":"99'86","ondisk_log_start":"99'86","created":31,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'89","last_scrub_stamp":"2024-09-13T10:12:17.219609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:17.219609+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":86,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:44.278286+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3747994,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":351,"num_read_kb":331385,"num_write":85,"num_write_kb":23348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4887315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"113'148","reported_seq":581,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-13T10:12:22.967808+0000","last_change":"2024-09-13T10:10:59.373229+0000","last_active":"2024-09-13T10:12:22.967808+0000","last_peered":"2024-09-13T10:12:22.967808+0000","last_clean":"2024-09-13T10:12:22.967808+0000","last_became_active":"2024-09-13T10:10:59.366341+0000","last_became_peered":"2024-09-13T10:10:59.366341+0000","last_unstale":"2024-09-13T10:12:22.967808+0000","last_undegraded":"2024-09-13T10:12:22.967808+0000","last_fullsized":"2024-09-13T10:12:22.967808+0000","mapping_epoch":44,"log_start":"111'145","ondisk_log_start":"111'145","created":31,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":145,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:26.996071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11445831,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":575,"num_read_kb":519826,"num_write":156,"num_write_kb":43618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,12,3],"acting":[6,12,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"108'73","reported_seq":348,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:21.026725+0000","last_change":"2024-09-13T10:12:03.448785+0000","last_active":"2024-09-13T10:12:21.026725+0000","last_peered":"2024-09-13T10:12:21.026725+0000","last_clean":"2024-09-13T10:12:21.026725+0000","last_became_active":"2024-09-13T10:10:59.364443+0000","last_became_peered":"2024-09-13T10:10:59.364443+0000","last_unstale":"2024-09-13T10:12:21.026725+0000","last_undegraded":"2024-09-13T10:12:21.026725+0000","last_fullsized":"2024-09-13T10:12:21.026725+0000","mapping_epoch":44,"log_start":"108'70","ondisk_log_start":"108'70","created":31,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'67","last_scrub_stamp":"2024-09-13T10:12:03.448609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:03.448609+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":70,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:20.556488+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3042133,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":151,"num_read_kb":145967,"num_write":72,"num_write_kb":20360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,12,7],"acting":[15,12,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.e","version":"111'32","reported_seq":236,"reported_epoch":115,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:12:23.909605+0000","last_change":"2024-09-13T10:12:20.880028+0000","last_active":"2024-09-13T10:12:23.909605+0000","last_peered":"2024-09-13T10:12:23.909605+0000","last_clean":"2024-09-13T10:11:03.155929+0000","last_became_active":"2024-09-13T10:11:20.874865+0000","last_became_peered":"2024-09-13T10:11:20.874865+0000","last_unstale":"2024-09-13T10:12:23.909605+0000","last_undegraded":"2024-09-13T10:12:23.909605+0000","last_fullsized":"2024-09-13T10:12:23.909605+0000","mapping_epoch":61,"log_start":"93'29","ondisk_log_start":"93'29","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":29,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3193396,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":67,"num_read_kb":67323,"num_write":34,"num_write_kb":10536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,15],"acting":[2,8,4],"avail_no_missing":["2","4","8"],"object_location_counts":[{"shards":"2,4,8","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"112'122","reported_seq":449,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-13T10:12:20.181628+0000","last_change":"2024-09-13T10:11:53.668346+0000","last_active":"2024-09-13T10:12:20.181628+0000","last_peered":"2024-09-13T10:12:20.181628+0000","last_clean":"2024-09-13T10:12:20.181628+0000","last_became_active":"2024-09-13T10:10:44.223700+0000","last_became_peered":"2024-09-13T10:10:44.223700+0000","last_unstale":"2024-09-13T10:12:20.181628+0000","last_undegraded":"2024-09-13T10:12:20.181628+0000","last_fullsized":"2024-09-13T10:12:20.181628+0000","mapping_epoch":31,"log_start":"111'119","ondisk_log_start":"111'119","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'87","last_scrub_stamp":"2024-09-13T10:11:53.668179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:53.668179+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":119,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:54.818975+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8773452,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":290,"num_read_kb":275280,"num_write":132,"num_write_kb":37219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,7],"acting":[13,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":26,"stat_sum":{"num_bytes":127685353,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":4682,"num_read_kb":4273973,"num_write":1529,"num_write_kb":437160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":37350017,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":20,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":265248768,"data_stored":264491088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":78,"acting":78,"num_store_stats":16},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":102312852,"num_objects":136,"num_object_clones":0,"num_object_copies":408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136,"num_whiteouts":0,"num_read":1285,"num_read_kb":31185,"num_write":1817,"num_write_kb":351733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":26621319,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":281448448,"data_stored":280741347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":13}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247834,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85720,"kb_used_data":54536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94286120,"statfs":{"total":96636764160,"available":96548986880,"internally_reserved":0,"allocated":55844864,"data_stored":55224625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247836,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46184,"kb_used_data":15000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325656,"statfs":{"total":96636764160,"available":96589471744,"internally_reserved":0,"allocated":15360000,"data_stored":14838917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247836,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67744,"kb_used_data":36560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94304096,"statfs":{"total":96636764160,"available":96567394304,"internally_reserved":0,"allocated":37437440,"data_stored":36832503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313243,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33552,"kb_used_data":6528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94338288,"statfs":{"total":96636764160,"available":96602406912,"internally_reserved":0,"allocated":6684672,"data_stored":6173261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,5,6,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313242,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82752,"kb_used_data":51568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94289088,"statfs":{"total":96636764160,"available":96552026112,"internally_reserved":0,"allocated":52805632,"data_stored":52229378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,6,7,8,9,10,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247834,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54916,"kb_used_data":23724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94316924,"statfs":{"total":96636764160,"available":96580530176,"internally_reserved":0,"allocated":24293376,"data_stored":23714058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,7,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280538,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64204,"kb_used_data":33016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94307636,"statfs":{"total":96636764160,"available":96571019264,"internally_reserved":0,"allocated":33808384,"data_stored":33159800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280538,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":102432,"kb_used_data":71248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94269408,"statfs":{"total":96636764160,"available":96531873792,"internally_reserved":0,"allocated":72957952,"data_stored":72300450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280539,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76192,"kb_used_data":45008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94295648,"statfs":{"total":96636764160,"available":96558743552,"internally_reserved":0,"allocated":46088192,"data_stored":45500609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280537,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52584,"kb_used_data":25560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94319256,"statfs":{"total":96636764160,"available":96582918144,"internally_reserved":0,"allocated":26173440,"data_stored":25632204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280538,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77748,"kb_used_data":46564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94294092,"statfs":{"total":96636764160,"available":96557150208,"internally_reserved":0,"allocated":47681536,"data_stored":47082133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280538,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99992,"kb_used_data":67588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94271848,"statfs":{"total":96636764160,"available":96534372352,"internally_reserved":0,"allocated":69210112,"data_stored":68455759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280538,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38356,"kb_used_data":7172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333484,"statfs":{"total":96636764160,"available":96597487616,"internally_reserved":0,"allocated":7344128,"data_stored":6826611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280538,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30612,"kb_used_data":3588,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341228,"statfs":{"total":96636764160,"available":96605417472,"internally_reserved":0,"allocated":3674112,"data_stored":3171503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247834,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49032,"kb_used_data":17848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94322808,"statfs":{"total":96636764160,"available":96586555392,"internally_reserved":0,"allocated":18276352,"data_stored":17734387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247833,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78540,"kb_used_data":45892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94293300,"statfs":{"total":96636764160,"available":96556339200,"internally_reserved":0,"allocated":46993408,"data_stored":46257827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,8,9,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":13602816,"data_stored":13565624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":17293312,"data_stored":17236274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":48799744,"data_stored":48679629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":30715904,"data_stored":30645781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":30715904,"data_stored":30645781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":29143040,"data_stored":29057931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":38215680,"data_stored":38109281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":27439104,"data_stored":27383599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":45522944,"data_stored":45417447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11145216,"data_stored":11095974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15204352,"data_stored":15114220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16949248,"data_stored":16916803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":22253568,"data_stored":22221147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":14061568,"data_stored":14045522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25321472,"data_stored":25283643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":45678592,"data_stored":45448502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15060992,"data_stored":15036678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":6377472,"data_stored":6366639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":5369856,"data_stored":5363936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":29683712,"data_stored":29537172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":6033408,"data_stored":6017305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":15622144,"data_stored":15601002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":25124864,"data_stored":25090735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":8998912,"data_stored":8989613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":2363392,"data_stored":2362197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:12:25.981 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set unique_pool_0 pgp_num 16 2024-09-13T10:12:26.037 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:26.033+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:26.083 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:26.128 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:26.128 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:12:26.128 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:12:26.128 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:12:26.128 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:26.138 INFO:tasks.ceph.osd.9.smithi136.stderr:2024-09-13T10:12:26.134+0000 7f3e1a715640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 26305) UID: 0 2024-09-13T10:12:26.141 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T10:12:26.154 INFO:tasks.rados.rados.0.smithi195.stdout:2011: finishing write tid 3 to smithi19530714-14 2024-09-13T10:12:26.154 INFO:tasks.rados.rados.0.smithi195.stdout:2011: finishing write tid 4 to smithi19530714-14 2024-09-13T10:12:26.154 INFO:tasks.rados.rados.0.smithi195.stdout:2011: finishing write tid 5 to smithi19530714-14 2024-09-13T10:12:26.155 INFO:tasks.rados.rados.0.smithi195.stdout:2011: oid 14 updating version 0 to 156 2024-09-13T10:12:26.155 INFO:tasks.rados.rados.0.smithi195.stdout:2011: oid 14 version 156 is already newer than 153 2024-09-13T10:12:26.155 INFO:tasks.rados.rados.0.smithi195.stdout:2011: oid 14 version 156 is already newer than 155 2024-09-13T10:12:26.155 INFO:tasks.rados.rados.0.smithi195.stdout:2011: oid 14 version 156 is already newer than 154 2024-09-13T10:12:26.155 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 156 (ObjNum 455 snap 0 seq_num 455) dirty exists 2024-09-13T10:12:26.155 INFO:tasks.rados.rados.0.smithi195.stdout:2011: left oid 14 (ObjNum 455 snap 0 seq_num 455) 2024-09-13T10:12:26.155 INFO:tasks.rados.rados.0.smithi195.stdout:2013: expect (ObjNum 324 snap 0 seq_num 324) 2024-09-13T10:12:26.156 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 464319 2024-09-13T10:12:26.238 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:26.233+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:26.267 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:26.267 INFO:teuthology.orchestra.run.smithi195.stdout: "size": 20, 2024-09-13T10:12:26.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 600, 2024-09-13T10:12:26.267 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [ 2024-09-13T10:12:26.267 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.267 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:211 3.0 e38/30 3:0596651d:::smithi19530714-12 low_tier:head v 38'1, mlcod=0'0)", 2024-09-13T10:12:26.267 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:12:26.267 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 95.255737867999997, 2024-09-13T10:12:26.267 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013316957000000001, 2024-09-13T10:12:26.267 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.267 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.267 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.268 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:12:26.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.268 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.268 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.268 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008319+0000", 2024-09-13T10:12:26.268 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.268 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.008321+0000", 2024-09-13T10:12:26.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.843e-06 2024-09-13T10:12:26.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011069+0000", 2024-09-13T10:12:26.269 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002747893 2024-09-13T10:12:26.269 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.269 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.269 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.269 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011113+0000", 2024-09-13T10:12:26.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.3921000000000002e-05 2024-09-13T10:12:26.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011140+0000", 2024-09-13T10:12:26.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7662999999999999e-05 2024-09-13T10:12:26.270 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.270 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.270 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.270 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011313+0000", 2024-09-13T10:12:26.270 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017232200000000001 2024-09-13T10:12:26.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011366+0000", 2024-09-13T10:12:26.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3888000000000003e-05 2024-09-13T10:12:26.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.271 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.271 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.271 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021577+0000", 2024-09-13T10:12:26.271 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010210472999999999 2024-09-13T10:12:26.271 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.272 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.272 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021636+0000", 2024-09-13T10:12:26.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.8953999999999998e-05 2024-09-13T10:12:26.272 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.272 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.272 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.272 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.272 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.272 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:212 3.0 e38/30 3:01490a5b:::smithi19530714-13 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'2, mlcod=0'0)", 2024-09-13T10:12:26.272 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:12:26.272 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 95.252420762, 2024-09-13T10:12:26.272 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.014217904, 2024-09-13T10:12:26.272 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.273 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.273 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.273 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:12:26.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.273 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.273 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.273 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011636+0000", 2024-09-13T10:12:26.273 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.273 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.011638+0000", 2024-09-13T10:12:26.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.672e-06 2024-09-13T10:12:26.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013289+0000", 2024-09-13T10:12:26.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001651511 2024-09-13T10:12:26.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.274 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.274 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.274 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013310+0000", 2024-09-13T10:12:26.274 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.0917999999999999e-05 2024-09-13T10:12:26.274 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.275 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.013322+0000", 2024-09-13T10:12:26.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2067e-05 2024-09-13T10:12:26.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.275 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.016882+0000", 2024-09-13T10:12:26.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0035599130000000001 2024-09-13T10:12:26.275 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.275 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.275 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.275 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.016934+0000", 2024-09-13T10:12:26.275 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1715999999999997e-05 2024-09-13T10:12:26.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025787+0000", 2024-09-13T10:12:26.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0088531809999999999 2024-09-13T10:12:26.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.276 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.276 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.276 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025854+0000", 2024-09-13T10:12:26.276 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6926000000000003e-05 2024-09-13T10:12:26.276 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.276 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.276 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.276 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.277 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:207 3.0 e38/30 3:332cd44b:::smithi19530714-8 low_tier:head v 38'3, mlcod=0'0)", 2024-09-13T10:12:26.277 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:12:26.277 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 95.246627321000005, 2024-09-13T10:12:26.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.016660892, 2024-09-13T10:12:26.277 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.277 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.277 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.277 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.277 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.277 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.277 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:12:26.277 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.277 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017429+0000", 2024-09-13T10:12:26.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.017432+0000", 2024-09-13T10:12:26.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1490000000000001e-06 2024-09-13T10:12:26.278 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.278 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.278 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.278 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021310+0000", 2024-09-13T10:12:26.278 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.003877985 2024-09-13T10:12:26.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021351+0000", 2024-09-13T10:12:26.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1105e-05 2024-09-13T10:12:26.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.279 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.279 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021376+0000", 2024-09-13T10:12:26.279 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5185999999999999e-05 2024-09-13T10:12:26.279 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.279 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021727+0000", 2024-09-13T10:12:26.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00035131300000000002 2024-09-13T10:12:26.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.021792+0000", 2024-09-13T10:12:26.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.4368000000000004e-05 2024-09-13T10:12:26.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.280 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.280 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.280 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034027+0000", 2024-09-13T10:12:26.280 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012235582 2024-09-13T10:12:26.280 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.281 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.281 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034090+0000", 2024-09-13T10:12:26.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3204e-05 2024-09-13T10:12:26.281 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.281 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.281 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.281 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.281 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.281 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:216 3.0 e38/30 3:0ac93756:::smithi19530714-17 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'4, mlcod=0'0)", 2024-09-13T10:12:26.281 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:12:26.281 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 95.240061750999999, 2024-09-13T10:12:26.281 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011044523000000001, 2024-09-13T10:12:26.281 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.282 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.282 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.282 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:12:26.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.282 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.282 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023995+0000", 2024-09-13T10:12:26.282 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.282 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.282 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.023997+0000", 2024-09-13T10:12:26.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.533e-06 2024-09-13T10:12:26.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024872+0000", 2024-09-13T10:12:26.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00087517699999999999 2024-09-13T10:12:26.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.283 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.283 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.283 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024894+0000", 2024-09-13T10:12:26.283 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.2571000000000001e-05 2024-09-13T10:12:26.283 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.024909+0000", 2024-09-13T10:12:26.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5158e-05 2024-09-13T10:12:26.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.025979+0000", 2024-09-13T10:12:26.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0010692329999999999 2024-09-13T10:12:26.284 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.284 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.284 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.284 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.026027+0000", 2024-09-13T10:12:26.284 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.8158000000000002e-05 2024-09-13T10:12:26.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.285 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.034992+0000", 2024-09-13T10:12:26.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0089650960000000005 2024-09-13T10:12:26.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.285 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.285 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.285 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.035040+0000", 2024-09-13T10:12:26.285 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.7596999999999998e-05 2024-09-13T10:12:26.285 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.285 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.285 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.285 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.286 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:225 3.0 e38/30 3:26960251:::smithi19530714-26 low_tier:head v 38'6, mlcod=38'3)", 2024-09-13T10:12:26.286 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:12:26.286 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 95.193164674000002, 2024-09-13T10:12:26.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.013205889, 2024-09-13T10:12:26.286 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.286 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.286 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.286 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.286 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.286 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.286 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:12:26.286 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.286 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070892+0000", 2024-09-13T10:12:26.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.070894+0000", 2024-09-13T10:12:26.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5e-06 2024-09-13T10:12:26.287 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.287 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.287 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.287 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072562+0000", 2024-09-13T10:12:26.287 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016683469999999999 2024-09-13T10:12:26.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072632+0000", 2024-09-13T10:12:26.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9665000000000005e-05 2024-09-13T10:12:26.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.288 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.288 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072661+0000", 2024-09-13T10:12:26.288 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.9714999999999999e-05 2024-09-13T10:12:26.288 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.288 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072800+0000", 2024-09-13T10:12:26.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013886000000000001 2024-09-13T10:12:26.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.289 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.072828+0000", 2024-09-13T10:12:26.289 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.7886e-05 2024-09-13T10:12:26.289 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.289 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.289 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084041+0000", 2024-09-13T10:12:26.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011213064999999999 2024-09-13T10:12:26.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.290 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.290 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084098+0000", 2024-09-13T10:12:26.290 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6851000000000002e-05 2024-09-13T10:12:26.290 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.290 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.290 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.290 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.290 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.290 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:228 3.0 e38/30 3:0168a995:::smithi19530714-29 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head v 38'7, mlcod=38'3)", 2024-09-13T10:12:26.290 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:12:26.290 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 95.189381093999998, 2024-09-13T10:12:26.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0094647819999999997, 2024-09-13T10:12:26.291 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.291 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.291 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.291 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:12:26.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.291 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.291 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.291 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.291 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074676+0000", 2024-09-13T10:12:26.291 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.074677+0000", 2024-09-13T10:12:26.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7099999999999999e-06 2024-09-13T10:12:26.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.292 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.292 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.292 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076599+0000", 2024-09-13T10:12:26.292 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0019212459999999999 2024-09-13T10:12:26.292 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.293 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076616+0000", 2024-09-13T10:12:26.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7008999999999999e-05 2024-09-13T10:12:26.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.293 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076641+0000", 2024-09-13T10:12:26.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5380000000000001e-05 2024-09-13T10:12:26.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.293 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.293 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.293 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076772+0000", 2024-09-13T10:12:26.293 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013101599999999999 2024-09-13T10:12:26.293 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.076867+0000", 2024-09-13T10:12:26.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.5262999999999999e-05 2024-09-13T10:12:26.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084106+0000", 2024-09-13T10:12:26.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0072390859999999996 2024-09-13T10:12:26.294 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.294 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.294 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.294 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.084140+0000", 2024-09-13T10:12:26.294 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4072000000000002e-05 2024-09-13T10:12:26.295 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.295 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.295 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.295 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.295 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.295 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:252 4.6 4:6ddb2b23:::smithi19530714-3:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:26.295 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:12:26.295 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 95.141993196000001, 2024-09-13T10:12:26.295 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.03110711, 2024-09-13T10:12:26.295 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.295 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.295 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.295 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:26.296 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:26.296 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:26.296 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 252 2024-09-13T10:12:26.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.296 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.296 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.296 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:12:26.296 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.296 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.296 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122064+0000", 2024-09-13T10:12:26.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122066+0000", 2024-09-13T10:12:26.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.4619999999999999e-06 2024-09-13T10:12:26.297 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.297 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.297 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.297 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122097+0000", 2024-09-13T10:12:26.297 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0488999999999999e-05 2024-09-13T10:12:26.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122120+0000", 2024-09-13T10:12:26.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.3649999999999999e-05 2024-09-13T10:12:26.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122130+0000", 2024-09-13T10:12:26.298 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7950000000000002e-06 2024-09-13T10:12:26.298 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.298 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.298 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.298 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122212+0000", 2024-09-13T10:12:26.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.2090999999999997e-05 2024-09-13T10:12:26.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.122317+0000", 2024-09-13T10:12:26.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104954 2024-09-13T10:12:26.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.299 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.299 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:26.299 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.151952+0000", 2024-09-13T10:12:26.299 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.029634908000000001 2024-09-13T10:12:26.299 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.300 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:26.300 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152580+0000", 2024-09-13T10:12:26.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00062796499999999997 2024-09-13T10:12:26.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.300 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.300 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.300 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.152891+0000", 2024-09-13T10:12:26.300 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000310747 2024-09-13T10:12:26.300 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153077+0000", 2024-09-13T10:12:26.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018588200000000001 2024-09-13T10:12:26.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153130+0000", 2024-09-13T10:12:26.301 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.3016000000000002e-05 2024-09-13T10:12:26.301 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.301 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.301 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.301 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.153171+0000", 2024-09-13T10:12:26.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.1151e-05 2024-09-13T10:12:26.302 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.302 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.302 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.302 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.302 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.302 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:255 4.6 4:6150ba38:::smithi19530714-6:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:26.302 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:12:26.302 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 95.049578991999994, 2024-09-13T10:12:26.302 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011708387000000001, 2024-09-13T10:12:26.302 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.302 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.302 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.302 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:26.302 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:26.303 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:26.303 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 255 2024-09-13T10:12:26.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.303 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:12:26.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.303 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.303 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.303 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214478+0000", 2024-09-13T10:12:26.303 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.303 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.304 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.304 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.304 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214482+0000", 2024-09-13T10:12:26.304 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8139999999999998e-06 2024-09-13T10:12:26.304 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214498+0000", 2024-09-13T10:12:26.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6636000000000001e-05 2024-09-13T10:12:26.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214537+0000", 2024-09-13T10:12:26.305 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8668999999999999e-05 2024-09-13T10:12:26.305 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.305 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.305 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.305 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214562+0000", 2024-09-13T10:12:26.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.5242999999999999e-05 2024-09-13T10:12:26.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.306 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214770+0000", 2024-09-13T10:12:26.306 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00020771299999999999 2024-09-13T10:12:26.306 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.306 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.306 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.214996+0000", 2024-09-13T10:12:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022615999999999999 2024-09-13T10:12:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.223455+0000", 2024-09-13T10:12:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0084591960000000004 2024-09-13T10:12:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225042+0000", 2024-09-13T10:12:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001586896 2024-09-13T10:12:26.308 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225719+0000", 2024-09-13T10:12:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00067701700000000003 2024-09-13T10:12:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.225845+0000", 2024-09-13T10:12:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012626299999999999 2024-09-13T10:12:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.309 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226070+0000", 2024-09-13T10:12:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00022411600000000001 2024-09-13T10:12:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.226186+0000", 2024-09-13T10:12:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011666400000000001 2024-09-13T10:12:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:268 4.6 4:6fb28ed4:::smithi19530714-19:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:26.310 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:12:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 95.048660432000005, 2024-09-13T10:12:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.041608707000000002, 2024-09-13T10:12:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 268 2024-09-13T10:12:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.311 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:12:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215396+0000", 2024-09-13T10:12:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.312 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215399+0000", 2024-09-13T10:12:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8669999999999998e-06 2024-09-13T10:12:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215416+0000", 2024-09-13T10:12:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.6792000000000001e-05 2024-09-13T10:12:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215480+0000", 2024-09-13T10:12:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.3805000000000003e-05 2024-09-13T10:12:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.313 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215499+0000", 2024-09-13T10:12:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9349999999999999e-05 2024-09-13T10:12:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215600+0000", 2024-09-13T10:12:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010119 2024-09-13T10:12:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.215715+0000", 2024-09-13T10:12:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00011491 2024-09-13T10:12:26.314 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.253967+0000", 2024-09-13T10:12:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.038251470000000003 2024-09-13T10:12:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.255748+0000", 2024-09-13T10:12:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0017815089999999999 2024-09-13T10:12:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.315 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256188+0000", 2024-09-13T10:12:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00043959799999999997 2024-09-13T10:12:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256840+0000", 2024-09-13T10:12:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000652591 2024-09-13T10:12:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.256932+0000", 2024-09-13T10:12:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1691e-05 2024-09-13T10:12:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.316 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.257005+0000", 2024-09-13T10:12:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.2934000000000003e-05 2024-09-13T10:12:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:305 4.6 4:6150ba38:::smithi19530714-6:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:12:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 94.995429739000002, 2024-09-13T10:12:26.317 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010323297, 2024-09-13T10:12:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:26.318 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 305 2024-09-13T10:12:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:12:26.319 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268627+0000", 2024-09-13T10:12:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268630+0000", 2024-09-13T10:12:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.548e-06 2024-09-13T10:12:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.320 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268644+0000", 2024-09-13T10:12:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.394e-05 2024-09-13T10:12:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268672+0000", 2024-09-13T10:12:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8242000000000002e-05 2024-09-13T10:12:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268689+0000", 2024-09-13T10:12:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7062e-05 2024-09-13T10:12:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.321 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268789+0000", 2024-09-13T10:12:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00010026600000000001 2024-09-13T10:12:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.268905+0000", 2024-09-13T10:12:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000115908 2024-09-13T10:12:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.276776+0000", 2024-09-13T10:12:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0078712000000000001 2024-09-13T10:12:26.322 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278412+0000", 2024-09-13T10:12:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0016358670000000001 2024-09-13T10:12:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278664+0000", 2024-09-13T10:12:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025162600000000002 2024-09-13T10:12:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.323 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278785+0000", 2024-09-13T10:12:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012162100000000001 2024-09-13T10:12:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278895+0000", 2024-09-13T10:12:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109774 2024-09-13T10:12:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:51.278950+0000", 2024-09-13T10:12:26.324 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.5242999999999999e-05 2024-09-13T10:12:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1023 4.16 4:6fb28ed4:::smithi19530714-19:head [write 1798066~477679] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.078200+0000", 2024-09-13T10:12:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 90.185856379000001, 2024-09-13T10:12:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0095557869999999996, 2024-09-13T10:12:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.325 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1023 2024-09-13T10:12:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078200+0000", 2024-09-13T10:12:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078200+0000", 2024-09-13T10:12:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.326 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078202+0000", 2024-09-13T10:12:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.2529999999999999e-06 2024-09-13T10:12:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078505+0000", 2024-09-13T10:12:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000303228 2024-09-13T10:12:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078519+0000", 2024-09-13T10:12:26.327 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3918e-05 2024-09-13T10:12:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078526+0000", 2024-09-13T10:12:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.311e-06 2024-09-13T10:12:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078577+0000", 2024-09-13T10:12:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.1125999999999998e-05 2024-09-13T10:12:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.328 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078682+0000", 2024-09-13T10:12:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000104671 2024-09-13T10:12:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.078864+0000", 2024-09-13T10:12:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00018166999999999999 2024-09-13T10:12:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.080583+0000", 2024-09-13T10:12:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001719138 2024-09-13T10:12:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.329 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.085104+0000", 2024-09-13T10:12:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0045216950000000001 2024-09-13T10:12:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.087686+0000", 2024-09-13T10:12:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0025817499999999998 2024-09-13T10:12:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.330 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.087721+0000", 2024-09-13T10:12:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5009999999999999e-05 2024-09-13T10:12:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.087756+0000", 2024-09-13T10:12:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.5017e-05 2024-09-13T10:12:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.331 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1040 4.6 4:61ec6a5a:::smithi19530714-35:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.095747+0000", 2024-09-13T10:12:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 90.168309402000006, 2024-09-13T10:12:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036815318999999999, 2024-09-13T10:12:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1040 2024-09-13T10:12:26.332 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095747+0000", 2024-09-13T10:12:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095747+0000", 2024-09-13T10:12:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.333 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095749+0000", 2024-09-13T10:12:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.9039999999999999e-06 2024-09-13T10:12:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095757+0000", 2024-09-13T10:12:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.6660000000000008e-06 2024-09-13T10:12:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095784+0000", 2024-09-13T10:12:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.6860999999999999e-05 2024-09-13T10:12:26.334 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095794+0000", 2024-09-13T10:12:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7149999999999993e-06 2024-09-13T10:12:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095860+0000", 2024-09-13T10:12:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.6699999999999995e-05 2024-09-13T10:12:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.335 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.095984+0000", 2024-09-13T10:12:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000123676 2024-09-13T10:12:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.132563+0000", 2024-09-13T10:12:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.036578797000000003 2024-09-13T10:12:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.336 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:1045 4.6 4:6150ba38:::smithi19530714-6:head [write 527753~708201,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:10:56.313424+0000", 2024-09-13T10:12:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 89.950632467999995, 2024-09-13T10:12:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011098433, 2024-09-13T10:12:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:26.337 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 1045 2024-09-13T10:12:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313424+0000", 2024-09-13T10:12:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313424+0000", 2024-09-13T10:12:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.338 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313427+0000", 2024-09-13T10:12:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0230000000000001e-06 2024-09-13T10:12:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313756+0000", 2024-09-13T10:12:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032905500000000003 2024-09-13T10:12:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.339 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313789+0000", 2024-09-13T10:12:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.2359e-05 2024-09-13T10:12:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313807+0000", 2024-09-13T10:12:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8130000000000001e-05 2024-09-13T10:12:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.313953+0000", 2024-09-13T10:12:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00014562200000000001 2024-09-13T10:12:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.340 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.314185+0000", 2024-09-13T10:12:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00023246399999999999 2024-09-13T10:12:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.314442+0000", 2024-09-13T10:12:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000257023 2024-09-13T10:12:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.317258+0000", 2024-09-13T10:12:26.341 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0028160149999999998 2024-09-13T10:12:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.319440+0000", 2024-09-13T10:12:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0021817149999999999 2024-09-13T10:12:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.324214+0000", 2024-09-13T10:12:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0047739189999999997 2024-09-13T10:12:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.342 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.324440+0000", 2024-09-13T10:12:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000225944 2024-09-13T10:12:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:10:56.324523+0000", 2024-09-13T10:12:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3164000000000005e-05 2024-09-13T10:12:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.343 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2603 4.6 4:6150ba38:::smithi19530714-6:head [setxattr _header (55) in=7b,truncate 3848803] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67)", 2024-09-13T10:12:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:26.522327+0000", 2024-09-13T10:12:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.741729661000001, 2024-09-13T10:12:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0099854200000000001, 2024-09-13T10:12:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2603 2024-09-13T10:12:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.344 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522327+0000", 2024-09-13T10:12:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522327+0000", 2024-09-13T10:12:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522329+0000", 2024-09-13T10:12:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.813e-06 2024-09-13T10:12:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.345 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522338+0000", 2024-09-13T10:12:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1269999999999992e-06 2024-09-13T10:12:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522359+0000", 2024-09-13T10:12:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.1242000000000001e-05 2024-09-13T10:12:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522374+0000", 2024-09-13T10:12:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5085999999999999e-05 2024-09-13T10:12:26.346 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.523648+0000", 2024-09-13T10:12:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0012740049999999999 2024-09-13T10:12:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.523917+0000", 2024-09-13T10:12:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00026876100000000001 2024-09-13T10:12:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.524319+0000", 2024-09-13T10:12:26.347 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040155200000000001 2024-09-13T10:12:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.525257+0000", 2024-09-13T10:12:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00093831600000000002 2024-09-13T10:12:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532117+0000", 2024-09-13T10:12:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0068596530000000003 2024-09-13T10:12:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.348 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532185+0000", 2024-09-13T10:12:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.8785e-05 2024-09-13T10:12:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532243+0000", 2024-09-13T10:12:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7627999999999998e-05 2024-09-13T10:12:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532313+0000", 2024-09-13T10:12:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.9451999999999998e-05 2024-09-13T10:12:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.349 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2604 4.6 4:6150ba38:::smithi19530714-6:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e67)", 2024-09-13T10:12:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:26.522408+0000", 2024-09-13T10:12:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 59.741649193999997, 2024-09-13T10:12:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010226071999999999, 2024-09-13T10:12:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:26.350 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2604 2024-09-13T10:12:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522408+0000", 2024-09-13T10:12:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.351 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522408+0000", 2024-09-13T10:12:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522409+0000", 2024-09-13T10:12:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3999999999999999e-06 2024-09-13T10:12:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522415+0000", 2024-09-13T10:12:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.7790000000000001e-06 2024-09-13T10:12:26.352 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522433+0000", 2024-09-13T10:12:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7972999999999999e-05 2024-09-13T10:12:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.522449+0000", 2024-09-13T10:12:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.5899000000000001e-05 2024-09-13T10:12:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.353 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.524993+0000", 2024-09-13T10:12:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002544499 2024-09-13T10:12:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.525196+0000", 2024-09-13T10:12:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000202721 2024-09-13T10:12:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532374+0000", 2024-09-13T10:12:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0071786259999999996 2024-09-13T10:12:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.354 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532498+0000", 2024-09-13T10:12:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012336899999999999 2024-09-13T10:12:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:26.532634+0000", 2024-09-13T10:12:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013580600000000001 2024-09-13T10:12:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2991 4.6 4:61ec6a5a:::smithi19530714-35:head [write 453855~696987,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T10:12:26.355 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:37.300931+0000", 2024-09-13T10:12:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 48.963126174000003, 2024-09-13T10:12:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.012633428, 2024-09-13T10:12:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2991 2024-09-13T10:12:26.356 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.300931+0000", 2024-09-13T10:12:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.300931+0000", 2024-09-13T10:12:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.357 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.300934+0000", 2024-09-13T10:12:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.856e-06 2024-09-13T10:12:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.301789+0000", 2024-09-13T10:12:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00085478099999999999 2024-09-13T10:12:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.301821+0000", 2024-09-13T10:12:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.1301999999999999e-05 2024-09-13T10:12:26.358 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.301838+0000", 2024-09-13T10:12:26.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.7229999999999999e-05 2024-09-13T10:12:26.359 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.359 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.359 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.359 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.302009+0000", 2024-09-13T10:12:26.359 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00017093699999999999 2024-09-13T10:12:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op must wait for map", 2024-09-13T10:12:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.302027+0000", 2024-09-13T10:12:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.8487000000000001e-05 2024-09-13T10:12:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.304197+0000", 2024-09-13T10:12:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002169706 2024-09-13T10:12:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.360 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.304321+0000", 2024-09-13T10:12:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00012425899999999999 2024-09-13T10:12:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.304430+0000", 2024-09-13T10:12:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000109323 2024-09-13T10:12:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.307797+0000", 2024-09-13T10:12:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0033668000000000001 2024-09-13T10:12:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.361 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313063+0000", 2024-09-13T10:12:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.005265947 2024-09-13T10:12:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313113+0000", 2024-09-13T10:12:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 4.9592000000000002e-05 2024-09-13T10:12:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313513+0000", 2024-09-13T10:12:26.362 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00040035 2024-09-13T10:12:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313564+0000", 2024-09-13T10:12:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.0858e-05 2024-09-13T10:12:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2992 4.6 4:61ec6a5a:::smithi19530714-35:head [write 1890787~548083] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T10:12:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:37.308827+0000", 2024-09-13T10:12:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 48.955229727999999, 2024-09-13T10:12:26.363 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011134115, 2024-09-13T10:12:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2992 2024-09-13T10:12:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.308827+0000", 2024-09-13T10:12:26.364 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.308827+0000", 2024-09-13T10:12:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.308830+0000", 2024-09-13T10:12:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.4000000000000001e-06 2024-09-13T10:12:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.365 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309309+0000", 2024-09-13T10:12:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000478901 2024-09-13T10:12:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309348+0000", 2024-09-13T10:12:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.8182000000000001e-05 2024-09-13T10:12:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309376+0000", 2024-09-13T10:12:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 2.8082e-05 2024-09-13T10:12:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.366 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309433+0000", 2024-09-13T10:12:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 5.6981000000000002e-05 2024-09-13T10:12:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309508+0000", 2024-09-13T10:12:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 7.5378999999999998e-05 2024-09-13T10:12:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309603+0000", 2024-09-13T10:12:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.4871999999999996e-05 2024-09-13T10:12:26.367 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313952+0000", 2024-09-13T10:12:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0043492139999999997 2024-09-13T10:12:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.317432+0000", 2024-09-13T10:12:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0034796969999999999 2024-09-13T10:12:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.318860+0000", 2024-09-13T10:12:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001428627 2024-09-13T10:12:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.319830+0000", 2024-09-13T10:12:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00096928100000000001 2024-09-13T10:12:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.319961+0000", 2024-09-13T10:12:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00013149900000000001 2024-09-13T10:12:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2993 4.6 4:61ec6a5a:::smithi19530714-35:head [setxattr _header (56) in=7b,truncate 2453855] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T10:12:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:37.309438+0000", 2024-09-13T10:12:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 48.954618887999999, 2024-09-13T10:12:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.010978862000000001, 2024-09-13T10:12:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2993 2024-09-13T10:12:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309438+0000", 2024-09-13T10:12:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309438+0000", 2024-09-13T10:12:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309439+0000", 2024-09-13T10:12:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3370000000000001e-06 2024-09-13T10:12:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309449+0000", 2024-09-13T10:12:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.0159e-05 2024-09-13T10:12:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309463+0000", 2024-09-13T10:12:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3305999999999999e-05 2024-09-13T10:12:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309473+0000", 2024-09-13T10:12:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.8169999999999994e-06 2024-09-13T10:12:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.310756+0000", 2024-09-13T10:12:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0012835100000000001 2024-09-13T10:12:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.311065+0000", 2024-09-13T10:12:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00030920900000000001 2024-09-13T10:12:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for subops from 1,13", 2024-09-13T10:12:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.311478+0000", 2024-09-13T10:12:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.000413106 2024-09-13T10:12:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "op_commit", 2024-09-13T10:12:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313969+0000", 2024-09-13T10:12:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.002490216 2024-09-13T10:12:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.318098+0000", 2024-09-13T10:12:26.375 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0041298020000000001 2024-09-13T10:12:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320032+0000", 2024-09-13T10:12:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001933349 2024-09-13T10:12:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320356+0000", 2024-09-13T10:12:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00032416699999999999 2024-09-13T10:12:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.376 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320417+0000", 2024-09-13T10:12:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.0884000000000002e-05 2024-09-13T10:12:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_op(client.4607.0:2994 4.6 4:61ec6a5a:::smithi19530714-35:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e75)", 2024-09-13T10:12:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:37.309490+0000", 2024-09-13T10:12:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 48.954566569999997, 2024-09-13T10:12:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011140761000000001, 2024-09-13T10:12:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.377 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "started", 2024-09-13T10:12:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: "client_info": { 2024-09-13T10:12:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: "client": "client.4607", 2024-09-13T10:12:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: "tid": 2994 2024-09-13T10:12:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309490+0000", 2024-09-13T10:12:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.378 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309490+0000", 2024-09-13T10:12:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309491+0000", 2024-09-13T10:12:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.2699999999999998e-07 2024-09-13T10:12:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309495+0000", 2024-09-13T10:12:26.379 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.7459999999999998e-06 2024-09-13T10:12:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309505+0000", 2024-09-13T10:12:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.7969999999999995e-06 2024-09-13T10:12:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.309516+0000", 2024-09-13T10:12:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.1222e-05 2024-09-13T10:12:26.380 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313661+0000", 2024-09-13T10:12:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0041448309999999999 2024-09-13T10:12:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.313920+0000", 2024-09-13T10:12:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00025941099999999997 2024-09-13T10:12:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320456+0000", 2024-09-13T10:12:26.381 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0065354030000000004 2024-09-13T10:12:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320539+0000", 2024-09-13T10:12:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.3935000000000002e-05 2024-09-13T10:12:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:37.320631+0000", 2024-09-13T10:12:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 9.1489000000000004e-05 2024-09-13T10:12:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.382 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "description": "osd_repop(client.4607.0:3745 3.0 e90/30 3:161dd296:::f6d96d7a8dac6a7d56d3d9df9ef9c133dce9c515f90d85faf1e037f770fc9485:head v 90'300, mlcod=89'298)", 2024-09-13T10:12:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "initiated_at": "2024-09-13T10:11:54.612303+0000", 2024-09-13T10:12:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "age": 31.651753639999999, 2024-09-13T10:12:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.011846107999999999, 2024-09-13T10:12:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "continuous": false, 2024-09-13T10:12:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "type_data": { 2024-09-13T10:12:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "events": [ 2024-09-13T10:12:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "initiated", 2024-09-13T10:12:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.612303+0000", 2024-09-13T10:12:26.383 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "header_read", 2024-09-13T10:12:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.612303+0000", 2024-09-13T10:12:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0 2024-09-13T10:12:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "throttled", 2024-09-13T10:12:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.612305+0000", 2024-09-13T10:12:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.866e-06 2024-09-13T10:12:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "all_read", 2024-09-13T10:12:26.384 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.614283+0000", 2024-09-13T10:12:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.001977707 2024-09-13T10:12:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "dispatched", 2024-09-13T10:12:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.614313+0000", 2024-09-13T10:12:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 3.0443999999999999e-05 2024-09-13T10:12:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "queued_for_pg", 2024-09-13T10:12:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.614327+0000", 2024-09-13T10:12:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 1.3506000000000001e-05 2024-09-13T10:12:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.385 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "reached_pg", 2024-09-13T10:12:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.614496+0000", 2024-09-13T10:12:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.00016885500000000001 2024-09-13T10:12:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "started", 2024-09-13T10:12:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.614561+0000", 2024-09-13T10:12:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 6.5612999999999997e-05 2024-09-13T10:12:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "commit_sent", 2024-09-13T10:12:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.624063+0000", 2024-09-13T10:12:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 0.0095017829999999998 2024-09-13T10:12:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: }, 2024-09-13T10:12:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: { 2024-09-13T10:12:26.386 INFO:teuthology.orchestra.run.smithi195.stdout: "event": "done", 2024-09-13T10:12:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "time": "2024-09-13T10:11:54.624149+0000", 2024-09-13T10:12:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: "duration": 8.6334000000000004e-05 2024-09-13T10:12:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: } 2024-09-13T10:12:26.387 INFO:teuthology.orchestra.run.smithi195.stdout: ] 2024-09-13T10:12:26.387 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:26.388 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:12:26.389 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-13T10:12:26.391 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 842504 2024-09-13T10:12:26.391 INFO:tasks.rados.rados.0.smithi195.stdout:2015: finishing set_chunk to oid 21 2024-09-13T10:12:26.391 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 21 v 168 (ObjNum 421 snap 0 seq_num 421) dirty exists 2024-09-13T10:12:26.391 INFO:tasks.rados.rados.0.smithi195.stdout:2011: done (6 left) 2024-09-13T10:12:26.391 INFO:tasks.rados.rados.0.smithi195.stdout:2012: done (5 left) 2024-09-13T10:12:26.391 INFO:tasks.rados.rados.0.smithi195.stdout:2013: done (4 left) 2024-09-13T10:12:26.391 INFO:tasks.rados.rados.0.smithi195.stdout:2014: done (3 left) 2024-09-13T10:12:26.391 INFO:tasks.rados.rados.0.smithi195.stdout:2015: done (2 left) 2024-09-13T10:12:26.391 INFO:tasks.rados.rados.0.smithi195.stdout:2016: done (1 left) 2024-09-13T10:12:26.391 INFO:tasks.rados.rados.0.smithi195.stdout:2017: done (0 left) 2024-09-13T10:12:26.391 INFO:tasks.rados.rados.0.smithi195.stdout:2018: read oid 13 snap -1 2024-09-13T10:12:26.392 INFO:tasks.rados.rados.0.smithi195.stdout:2018: expect (ObjNum 342 snap 0 seq_num 342) 2024-09-13T10:12:26.392 INFO:tasks.rados.rados.0.smithi195.stdout:2019: read oid 48 snap -1 2024-09-13T10:12:26.392 INFO:tasks.rados.rados.0.smithi195.stdout:2019: expect (ObjNum 410 snap 0 seq_num 410) 2024-09-13T10:12:26.392 INFO:tasks.rados.rados.0.smithi195.stdout:2020: write oid 37 current snap is 0 2024-09-13T10:12:26.392 INFO:tasks.rados.rados.0.smithi195.stdout:2020: seq_num 457 ranges {480799=569973,1475918=769483,2700190=586604} 2024-09-13T10:12:26.392 INFO:tasks.rados.rados.0.smithi195.stdout:2020: writing smithi19530714-37 from 480799 to 1050772 tid 1 2024-09-13T10:12:26.392 INFO:tasks.rados.rados.0.smithi195.stdout:2020: writing smithi19530714-37 from 1475918 to 2245401 tid 2 2024-09-13T10:12:26.392 INFO:tasks.rados.rados.0.smithi195.stdout:2020: writing smithi19530714-37 from 2700190 to 3286794 tid 3 2024-09-13T10:12:26.392 INFO:tasks.rados.rados.0.smithi195.stdout:2021: set_chunk oid 19 target oid 2024-09-13T10:12:26.392 INFO:tasks.rados.rados.0.smithi195.stdout:2021: set_chunk oid 19 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:12:26.392 INFO:tasks.rados.rados.0.smithi195.stdout:2022: read oid 15 snap -1 2024-09-13T10:12:26.392 INFO:tasks.rados.rados.0.smithi195.stdout:2022: expect deleted 2024-09-13T10:12:26.392 INFO:tasks.rados.rados.0.smithi195.stdout:2023: tier_evict oid 17 2024-09-13T10:12:26.393 INFO:tasks.rados.rados.0.smithi195.stdout:2023: tier_evict oid 17 snap -1 2024-09-13T10:12:26.393 INFO:tasks.rados.rados.0.smithi195.stdout:2019: expect (ObjNum 410 snap 0 seq_num 410) 2024-09-13T10:12:26.393 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:12:26.386+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:12:26.489 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:26.485+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:26.513 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 616952 2024-09-13T10:12:26.535 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:12:26.535 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:12:26.551 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:26.552 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:12:26.552 INFO:teuthology.orchestra.run.smithi195.stdout: "num_ops": 0 2024-09-13T10:12:26.552 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:26.555 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:12:26.555 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:12:26.566 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-13T10:12:26.576 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:12:26.576 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:12:26.595 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:12:26.596 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:12:26.611 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:12:26.611 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:12:26.612 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:26.609+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:26.631 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:12:26.632 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:12:26.651 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:12:26.652 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:12:26.670 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:12:26.671 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:12:26.680 INFO:tasks.rados.rados.0.smithi195.stdout:2018: expect (ObjNum 342 snap 0 seq_num 342) 2024-09-13T10:12:26.687 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:12:26.687 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:12:26.688 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 832691 2024-09-13T10:12:26.689 INFO:teuthology.orchestra.run.smithi195.stdout:{ 2024-09-13T10:12:26.689 INFO:teuthology.orchestra.run.smithi195.stdout: "ops": [], 2024-09-13T10:12:26.689 INFO:teuthology.orchestra.run.smithi195.stdout: "complaint_time": 30, 2024-09-13T10:12:26.689 INFO:teuthology.orchestra.run.smithi195.stdout: "num_blocked_ops": 0 2024-09-13T10:12:26.689 INFO:teuthology.orchestra.run.smithi195.stdout:} 2024-09-13T10:12:26.691 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:12:26.686+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:12:26.704 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-13T10:12:26.706 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:12:26.707 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:12:26.728 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:12:26.728 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:12:26.746 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:12:26.746 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:12:26.765 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:12:26.765 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:12:26.786 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:12:26.787 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:12:26.791 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:12:26.790+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:12:26.808 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:12:26.808 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:12:26.829 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:12:26.829 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:12:26.871 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:26.871 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:26.885 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-13T10:12:26.892 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:12:26.890+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:12:26.930 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-13T10:12:26.930 INFO:teuthology.orchestra.run.smithi073.stderr:set pool 4 pgp_num to 16 2024-09-13T10:12:26.932 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3291275 2024-09-13T10:12:26.942 INFO:tasks.rados.rados.0.smithi195.stdout:2020: finishing write tid 1 to smithi19530714-37 2024-09-13T10:12:26.942 INFO:tasks.rados.rados.0.smithi195.stdout:2020: finishing write tid 2 to smithi19530714-37 2024-09-13T10:12:26.942 INFO:tasks.rados.rados.0.smithi195.stdout:2018: done (5 left) 2024-09-13T10:12:26.942 INFO:tasks.rados.rados.0.smithi195.stdout:2019: done (4 left) 2024-09-13T10:12:26.942 INFO:tasks.rados.rados.0.smithi195.stdout:2024: read oid 2 snap -1 2024-09-13T10:12:26.942 INFO:tasks.rados.rados.0.smithi195.stdout:2024: expect (ObjNum 412 snap 0 seq_num 412) 2024-09-13T10:12:26.942 INFO:tasks.rados.rados.0.smithi195.stdout:2025: read oid 38 snap -1 2024-09-13T10:12:26.942 INFO:tasks.rados.rados.0.smithi195.stdout:2025: expect (ObjNum 441 snap 0 seq_num 441) 2024-09-13T10:12:26.943 INFO:tasks.rados.rados.0.smithi195.stdout:2026: tier_flush oid 28 2024-09-13T10:12:26.943 INFO:tasks.rados.rados.0.smithi195.stdout:2026: tier_flush oid 28 snap -1 2024-09-13T10:12:26.943 INFO:tasks.rados.rados.0.smithi195.stdout:2023: got (22) Invalid argument 2024-09-13T10:12:26.943 INFO:tasks.rados.rados.0.smithi195.stdout:2021: finishing set_chunk to oid 19 2024-09-13T10:12:26.943 INFO:tasks.rados.rados.0.smithi195.stdout:2021: got expected ENOENT (src dne) 2024-09-13T10:12:26.943 INFO:tasks.rados.rados.0.smithi195.stdout:2020: finishing write tid 3 to smithi19530714-37 2024-09-13T10:12:26.943 INFO:tasks.rados.rados.0.smithi195.stdout:2020: finishing write tid 4 to smithi19530714-37 2024-09-13T10:12:26.943 INFO:tasks.rados.rados.0.smithi195.stdout:2020: finishing write tid 5 to smithi19530714-37 2024-09-13T10:12:26.943 INFO:tasks.rados.rados.0.smithi195.stdout:2020: oid 37 updating version 0 to 32 2024-09-13T10:12:26.943 INFO:tasks.rados.rados.0.smithi195.stdout:2020: oid 37 version 32 is already newer than 31 2024-09-13T10:12:26.943 INFO:tasks.rados.rados.0.smithi195.stdout:2020: oid 37 version 32 is already newer than 30 2024-09-13T10:12:26.943 INFO:tasks.rados.rados.0.smithi195.stdout:2020: oid 37 updating version 32 to 33 2024-09-13T10:12:26.943 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 33 (ObjNum 457 snap 0 seq_num 457) dirty exists 2024-09-13T10:12:26.943 INFO:tasks.rados.rados.0.smithi195.stdout:2020: left oid 37 (ObjNum 457 snap 0 seq_num 457) 2024-09-13T10:12:26.944 INFO:tasks.rados.rados.0.smithi195.stdout:2020: done (6 left) 2024-09-13T10:12:26.944 INFO:tasks.rados.rados.0.smithi195.stdout:2021: done (5 left) 2024-09-13T10:12:26.944 INFO:tasks.rados.rados.0.smithi195.stdout:2022: done (4 left) 2024-09-13T10:12:26.944 INFO:tasks.rados.rados.0.smithi195.stdout:2023: done (3 left) 2024-09-13T10:12:26.944 INFO:tasks.rados.rados.0.smithi195.stdout:2027: read oid 37 snap -1 2024-09-13T10:12:26.944 INFO:tasks.rados.rados.0.smithi195.stdout:2027: expect (ObjNum 457 snap 0 seq_num 457) 2024-09-13T10:12:26.944 INFO:tasks.rados.rados.0.smithi195.stdout:2028: read oid 35 snap -1 2024-09-13T10:12:26.944 INFO:tasks.rados.rados.0.smithi195.stdout:2028: expect (ObjNum 409 snap 0 seq_num 409) 2024-09-13T10:12:26.944 INFO:tasks.rados.rados.0.smithi195.stdout:2029: read oid 20 snap -1 2024-09-13T10:12:26.944 INFO:tasks.rados.rados.0.smithi195.stdout:2029: expect (ObjNum 389 snap 0 seq_num 389) 2024-09-13T10:12:26.944 INFO:tasks.rados.rados.0.smithi195.stdout:2030: set_chunk oid 23 target oid 2024-09-13T10:12:26.954 INFO:tasks.rados.rados.0.smithi195.stdout:2030: set_chunk oid 23 offset: 2163712 length: 582144 target oid fdf6d80576d520f6c2b32c3cf91b094cb9bbf7e3d5babc1e46d884ae37da9976 offset: 0 snap -1 2024-09-13T10:12:26.955 INFO:tasks.rados.rados.0.smithi195.stdout:2031: read oid 21 snap -1 2024-09-13T10:12:26.955 INFO:tasks.rados.rados.0.smithi195.stdout:2031: expect (ObjNum 421 snap 0 seq_num 421) 2024-09-13T10:12:26.955 INFO:tasks.rados.rados.0.smithi195.stdout:2032: delete oid 49 current snap is 0 2024-09-13T10:12:26.955 INFO:tasks.rados.rados.0.smithi195.stdout:2028: expect (ObjNum 409 snap 0 seq_num 409) 2024-09-13T10:12:26.958 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 697926 2024-09-13T10:12:26.993 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:12:26.990+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:12:27.010 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2826849 2024-09-13T10:12:27.026 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:27.026 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:27.039 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-13T10:12:27.042 INFO:tasks.rados.rados.0.smithi195.stdout:2028: done (8 left) 2024-09-13T10:12:27.042 INFO:tasks.rados.rados.0.smithi195.stdout:2032: done (7 left) 2024-09-13T10:12:27.042 INFO:tasks.rados.rados.0.smithi195.stdout:2033: read oid 27 snap -1 2024-09-13T10:12:27.042 INFO:tasks.rados.rados.0.smithi195.stdout:2033: expect (ObjNum 444 snap 0 seq_num 444) 2024-09-13T10:12:27.042 INFO:tasks.rados.rados.0.smithi195.stdout:2034: delete oid 31 current snap is 0 2024-09-13T10:12:27.042 INFO:tasks.rados.rados.0.smithi195.stdout:2030: finishing set_chunk to oid 23 2024-09-13T10:12:27.042 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 23 v 42 (ObjNum 286 snap 0 seq_num 286) dirty exists 2024-09-13T10:12:27.042 INFO:tasks.rados.rados.0.smithi195.stdout:2024: expect (ObjNum 412 snap 0 seq_num 412) 2024-09-13T10:12:27.094 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:27.089+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:27.145 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:27.145 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:27.159 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-13T10:12:27.194 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:27.189+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:27.217 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 218884 2024-09-13T10:12:27.296 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:27.294+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:27.309 INFO:tasks.rados.rados.0.smithi195.stdout:2029: expect (ObjNum 389 snap 0 seq_num 389) 2024-09-13T10:12:27.309 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:27.330 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:27.330 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:27.335 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:27.344 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T10:12:27.369 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2499338 2024-09-13T10:12:27.375 INFO:tasks.rados.rados.0.smithi195.stdout:2027: expect (ObjNum 457 snap 0 seq_num 457) 2024-09-13T10:12:27.396 INFO:tasks.ceph.osd.3.smithi195.stderr:2024-09-13T10:12:27.393+0000 7fbb06d82640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 26579) UID: 0 2024-09-13T10:12:27.417 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 3250260 2024-09-13T10:12:27.428 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 166401 2024-09-13T10:12:27.476 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:27.476 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:27.490 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T10:12:27.491 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:27.497 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:27.494+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:27.553 INFO:tasks.rados.rados.0.smithi195.stdout:2025: expect (ObjNum 441 snap 0 seq_num 441) 2024-09-13T10:12:27.553 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 113403 2024-09-13T10:12:27.598 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:27.594+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:27.606 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:27.606 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:27.620 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T10:12:27.698 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:27.694+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:27.737 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:27.737 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:27.742 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:27.751 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T10:12:27.799 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:12:27.798+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:12:27.839 INFO:tasks.rados.rados.0.smithi195.stdout:2031: expect (ObjNum 421 snap 0 seq_num 421) 2024-09-13T10:12:27.896 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2781572 2024-09-13T10:12:27.901 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:12:27.898+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:12:27.915 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:27.915 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:27.928 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T10:12:28.015 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:28.014+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:28.056 INFO:tasks.rados.rados.0.smithi195.stdout:2026: got (0) Success 2024-09-13T10:12:28.056 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 28 v 0 (ObjNum 369 snap 0 seq_num 369) dirty exists 2024-09-13T10:12:28.056 INFO:tasks.rados.rados.0.smithi195.stdout:2024: done (8 left) 2024-09-13T10:12:28.057 INFO:tasks.rados.rados.0.smithi195.stdout:2025: done (7 left) 2024-09-13T10:12:28.057 INFO:tasks.rados.rados.0.smithi195.stdout:2026: done (6 left) 2024-09-13T10:12:28.057 INFO:tasks.rados.rados.0.smithi195.stdout:2027: done (5 left) 2024-09-13T10:12:28.057 INFO:tasks.rados.rados.0.smithi195.stdout:2029: done (4 left) 2024-09-13T10:12:28.057 INFO:tasks.rados.rados.0.smithi195.stdout:2030: done (3 left) 2024-09-13T10:12:28.057 INFO:tasks.rados.rados.0.smithi195.stdout:2031: done (2 left) 2024-09-13T10:12:28.057 INFO:tasks.rados.rados.0.smithi195.stdout:2034: done (1 left) 2024-09-13T10:12:28.057 INFO:tasks.rados.rados.0.smithi195.stdout:2035: write (excl) oid 37 current snap is 0 2024-09-13T10:12:28.057 INFO:tasks.rados.rados.0.smithi195.stdout:2035: seq_num 460 ranges {531220=795720,1935011=516810,3066106=420420} 2024-09-13T10:12:28.062 INFO:tasks.rados.rados.0.smithi195.stdout:2035: writing smithi19530714-37 from 531220 to 1326940 tid 1 2024-09-13T10:12:28.068 INFO:tasks.rados.rados.0.smithi195.stdout:2035: writing smithi19530714-37 from 1935011 to 2451821 tid 2 2024-09-13T10:12:28.074 INFO:tasks.rados.rados.0.smithi195.stdout:2035: writing smithi19530714-37 from 3066106 to 3486526 tid 3 2024-09-13T10:12:28.074 INFO:tasks.rados.rados.0.smithi195.stdout:2036: write oid 4 current snap is 0 2024-09-13T10:12:28.074 INFO:tasks.rados.rados.0.smithi195.stdout:2036: seq_num 461 ranges {548027=470969,1593493=776821,3093529=631692} 2024-09-13T10:12:28.079 INFO:tasks.rados.rados.0.smithi195.stdout:2036: writing smithi19530714-4 from 548027 to 1018996 tid 1 2024-09-13T10:12:28.086 INFO:tasks.rados.rados.0.smithi195.stdout:2036: writing smithi19530714-4 from 1593493 to 2370314 tid 2 2024-09-13T10:12:28.092 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:28.092 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:28.095 INFO:tasks.rados.rados.0.smithi195.stdout:2036: writing smithi19530714-4 from 3093529 to 3725221 tid 3 2024-09-13T10:12:28.095 INFO:tasks.rados.rados.0.smithi195.stdout:2037: write (excl) oid 3 current snap is 0 2024-09-13T10:12:28.095 INFO:tasks.rados.rados.0.smithi195.stdout:2037: seq_num 462 ranges {564834=546218,1651975=520479,2720952=559317} 2024-09-13T10:12:28.101 INFO:tasks.rados.rados.0.smithi195.stdout:2037: writing smithi19530714-3 from 564834 to 1111052 tid 1 2024-09-13T10:12:28.102 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:12:28.098+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:12:28.106 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T10:12:28.106 INFO:tasks.rados.rados.0.smithi195.stdout:2037: writing smithi19530714-3 from 1651975 to 2172454 tid 2 2024-09-13T10:12:28.113 INFO:tasks.rados.rados.0.smithi195.stdout:2037: writing smithi19530714-3 from 2720952 to 3280269 tid 3 2024-09-13T10:12:28.113 INFO:tasks.rados.rados.0.smithi195.stdout:2038: tier_evict oid 49 2024-09-13T10:12:28.113 INFO:tasks.rados.rados.0.smithi195.stdout:2038: tier_evict oid 49 snap -1 2024-09-13T10:12:28.113 INFO:tasks.rados.rados.0.smithi195.stdout:2033: expect (ObjNum 444 snap 0 seq_num 444) 2024-09-13T10:12:28.113 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:28.195 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 2246903 2024-09-13T10:12:28.203 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:28.202+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:28.209 INFO:tasks.rados.rados.0.smithi195.stdout:2035: finishing write tid 1 to smithi19530714-37 2024-09-13T10:12:28.209 INFO:tasks.rados.rados.0.smithi195.stdout:2035: finishing write tid 2 to smithi19530714-37 2024-09-13T10:12:28.209 INFO:tasks.rados.rados.0.smithi195.stdout:2035: finishing write tid 3 to smithi19530714-37 2024-09-13T10:12:28.210 INFO:tasks.rados.rados.0.smithi195.stdout:2035: finishing write tid 4 to smithi19530714-37 2024-09-13T10:12:28.210 INFO:tasks.rados.rados.0.smithi195.stdout:2033: done (4 left) 2024-09-13T10:12:28.210 INFO:tasks.rados.rados.0.smithi195.stdout:2039: set_chunk oid 2 target oid 2024-09-13T10:12:28.232 INFO:tasks.rados.rados.0.smithi195.stdout:2039: set_chunk oid 2 offset: 1446400 length: 1341952 target oid 8e0352ce560f83d5aa88b4c90abf4a6442db0cecfa6bc5b6b7b6d46f490b5046 offset: 0 snap -1 2024-09-13T10:12:28.233 INFO:tasks.rados.rados.0.smithi195.stdout:2040: delete oid 13 current snap is 0 2024-09-13T10:12:28.233 INFO:tasks.rados.rados.0.smithi195.stdout:2035: finishing write tid 5 to smithi19530714-37 2024-09-13T10:12:28.233 INFO:tasks.rados.rados.0.smithi195.stdout:2035: oid 37 updating version 0 to 37 2024-09-13T10:12:28.233 INFO:tasks.rados.rados.0.smithi195.stdout:2035: oid 37 version 37 is already newer than 35 2024-09-13T10:12:28.233 INFO:tasks.rados.rados.0.smithi195.stdout:2035: oid 37 version 37 is already newer than 34 2024-09-13T10:12:28.233 INFO:tasks.rados.rados.0.smithi195.stdout:2035: oid 37 version 37 is already newer than 36 2024-09-13T10:12:28.233 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 37 v 37 (ObjNum 460 snap 0 seq_num 460) dirty exists 2024-09-13T10:12:28.233 INFO:tasks.rados.rados.0.smithi195.stdout:2035: left oid 37 (ObjNum 460 snap 0 seq_num 460) 2024-09-13T10:12:28.233 INFO:tasks.rados.rados.0.smithi195.stdout:2036: finishing write tid 1 to smithi19530714-4 2024-09-13T10:12:28.233 INFO:tasks.rados.rados.0.smithi195.stdout:2036: finishing write tid 2 to smithi19530714-4 2024-09-13T10:12:28.233 INFO:tasks.rados.rados.0.smithi195.stdout:2036: finishing write tid 3 to smithi19530714-4 2024-09-13T10:12:28.233 INFO:tasks.rados.rados.0.smithi195.stdout:2036: finishing write tid 4 to smithi19530714-4 2024-09-13T10:12:28.234 INFO:tasks.rados.rados.0.smithi195.stdout:2036: finishing write tid 5 to smithi19530714-4 2024-09-13T10:12:28.234 INFO:tasks.rados.rados.0.smithi195.stdout:2036: oid 4 updating version 0 to 144 2024-09-13T10:12:28.234 INFO:tasks.rados.rados.0.smithi195.stdout:2036: oid 4 version 144 is already newer than 143 2024-09-13T10:12:28.234 INFO:tasks.rados.rados.0.smithi195.stdout:2036: oid 4 version 144 is already newer than 142 2024-09-13T10:12:28.234 INFO:tasks.rados.rados.0.smithi195.stdout:2036: oid 4 updating version 144 to 145 2024-09-13T10:12:28.234 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 4 v 145 (ObjNum 461 snap 0 seq_num 461) dirty exists 2024-09-13T10:12:28.234 INFO:tasks.rados.rados.0.smithi195.stdout:2036: left oid 4 (ObjNum 461 snap 0 seq_num 461) 2024-09-13T10:12:28.234 INFO:tasks.rados.rados.0.smithi195.stdout:2038: got (22) Invalid argument 2024-09-13T10:12:28.234 INFO:tasks.rados.rados.0.smithi195.stdout:2037: finishing write tid 1 to smithi19530714-3 2024-09-13T10:12:28.234 INFO:tasks.rados.rados.0.smithi195.stdout:2037: finishing write tid 2 to smithi19530714-3 2024-09-13T10:12:28.234 INFO:tasks.rados.rados.0.smithi195.stdout:2037: finishing write tid 3 to smithi19530714-3 2024-09-13T10:12:28.234 INFO:tasks.rados.rados.0.smithi195.stdout:2037: finishing write tid 4 to smithi19530714-3 2024-09-13T10:12:28.234 INFO:tasks.rados.rados.0.smithi195.stdout:2037: finishing write tid 5 to smithi19530714-3 2024-09-13T10:12:28.234 INFO:tasks.rados.rados.0.smithi195.stdout:2037: oid 3 updating version 0 to 87 2024-09-13T10:12:28.235 INFO:tasks.rados.rados.0.smithi195.stdout:2037: oid 3 version 87 is already newer than 86 2024-09-13T10:12:28.235 INFO:tasks.rados.rados.0.smithi195.stdout:2037: oid 3 updating version 87 to 88 2024-09-13T10:12:28.235 INFO:tasks.rados.rados.0.smithi195.stdout:2037: oid 3 updating version 88 to 89 2024-09-13T10:12:28.235 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 3 v 89 (ObjNum 462 snap 0 seq_num 462) dirty exists 2024-09-13T10:12:28.235 INFO:tasks.rados.rados.0.smithi195.stdout:2037: left oid 3 (ObjNum 462 snap 0 seq_num 462) 2024-09-13T10:12:28.235 INFO:tasks.rados.rados.0.smithi195.stdout:2039: finishing set_chunk to oid 2 2024-09-13T10:12:28.235 INFO:tasks.rados.rados.0.smithi195.stdout:Range is overlapped: oid 2 set_chunk returned error code -95 offset: 1446400 length: 1341952 2024-09-13T10:12:28.235 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 2 v 0 (ObjNum 412 snap 0 seq_num 412) dirty exists 2024-09-13T10:12:28.237 INFO:tasks.rados.rados.0.smithi195.stdout:2035: done (5 left) 2024-09-13T10:12:28.237 INFO:tasks.rados.rados.0.smithi195.stdout:2036: done (4 left) 2024-09-13T10:12:28.237 INFO:tasks.rados.rados.0.smithi195.stdout:2037: done (3 left) 2024-09-13T10:12:28.237 INFO:tasks.rados.rados.0.smithi195.stdout:2038: done (2 left) 2024-09-13T10:12:28.237 INFO:tasks.rados.rados.0.smithi195.stdout:2039: done (1 left) 2024-09-13T10:12:28.237 INFO:tasks.rados.rados.0.smithi195.stdout:2040: done (0 left) 2024-09-13T10:12:28.237 INFO:tasks.rados.rados.0.smithi195.stdout:2041: read oid 25 snap -1 2024-09-13T10:12:28.237 INFO:tasks.rados.rados.0.smithi195.stdout:2041: expect (ObjNum 324 snap 0 seq_num 324) 2024-09-13T10:12:28.237 INFO:tasks.rados.rados.0.smithi195.stdout:2042: set_chunk oid 35 target oid 2024-09-13T10:12:28.246 INFO:tasks.rados.rados.0.smithi195.stdout:2042: set_chunk oid 35 offset: 946688 length: 359424 target oid 9481b46542a151d6dd3d5c0ee12552413cf160a7b0479c543ed796c1bb25462d offset: 0 snap -1 2024-09-13T10:12:28.246 INFO:tasks.rados.rados.0.smithi195.stdout:2043: read oid 49 snap -1 2024-09-13T10:12:28.246 INFO:tasks.rados.rados.0.smithi195.stdout:2043: expect deleted 2024-09-13T10:12:28.246 INFO:tasks.rados.rados.0.smithi195.stdout:2044: write (excl) oid 39 current snap is 0 2024-09-13T10:12:28.246 INFO:tasks.rados.rados.0.smithi195.stdout:2044: seq_num 464 ranges {598448=413069,1768939=524148,2775798=581861,3781488=16960} 2024-09-13T10:12:28.258 INFO:tasks.rados.rados.0.smithi195.stdout:2044: writing smithi19530714-39 from 598448 to 1011517 tid 1 2024-09-13T10:12:28.271 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:28.271 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:28.273 INFO:tasks.rados.rados.0.smithi195.stdout:2044: writing smithi19530714-39 from 1768939 to 2293087 tid 2 2024-09-13T10:12:28.285 INFO:tasks.rados.rados.0.smithi195.stdout:2044: writing smithi19530714-39 from 2775798 to 3357659 tid 3 2024-09-13T10:12:28.286 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-09-13T10:12:28.290 INFO:tasks.rados.rados.0.smithi195.stdout:2044: writing smithi19530714-39 from 3781488 to 3798448 tid 4 2024-09-13T10:12:28.290 INFO:tasks.rados.rados.0.smithi195.stdout:2045: set_chunk oid 44 target oid 2024-09-13T10:12:28.290 INFO:tasks.rados.rados.0.smithi195.stdout:2045: set_chunk oid 44 offset: 0 length: 4000000 target oid smithi19530714- offset: 0 snap -1 2024-09-13T10:12:28.290 INFO:tasks.rados.rados.0.smithi195.stdout:2046: read oid 17 snap -1 2024-09-13T10:12:28.290 INFO:tasks.rados.rados.0.smithi195.stdout:2046: expect (ObjNum 451 snap 0 seq_num 451) 2024-09-13T10:12:28.290 INFO:tasks.rados.rados.0.smithi195.stdout:2047: delete oid 32 current snap is 0 2024-09-13T10:12:28.290 INFO:tasks.rados.rados.0.smithi195.stdout:2042: finishing set_chunk to oid 35 2024-09-13T10:12:28.290 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 35 v 93 (ObjNum 409 snap 0 seq_num 409) dirty exists 2024-09-13T10:12:28.290 INFO:tasks.rados.rados.0.smithi195.stdout:2041: expect (ObjNum 324 snap 0 seq_num 324) 2024-09-13T10:12:28.304 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:12:28.302+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:12:28.356 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 875413 2024-09-13T10:12:28.405 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:12:28.401+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:12:28.419 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:28.454 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:28.454 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:28.468 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-09-13T10:12:28.487 INFO:tasks.rados.rados.0.smithi195.stdout:2045: finishing set_chunk to oid 44 2024-09-13T10:12:28.487 INFO:tasks.rados.rados.0.smithi195.stdout:2045: got expected ENOENT (src dne) 2024-09-13T10:12:28.487 INFO:tasks.rados.rados.0.smithi195.stdout:2046: expect (ObjNum 451 snap 0 seq_num 451) 2024-09-13T10:12:28.498 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1126307 2024-09-13T10:12:28.506 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:12:28.502+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:12:28.554 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 0 2024-09-13T10:12:28.586 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:28.586 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:28.601 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-09-13T10:12:28.607 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:12:28.601+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:12:28.637 INFO:tasks.rados.rados.0.smithi195.stdout:seeking to 1095486 2024-09-13T10:12:28.693 INFO:tasks.rados.rados.0.smithi195.stdout:2041: done (6 left) 2024-09-13T10:12:28.693 INFO:tasks.rados.rados.0.smithi195.stdout:2042: done (5 left) 2024-09-13T10:12:28.693 INFO:tasks.rados.rados.0.smithi195.stdout:2043: done (4 left) 2024-09-13T10:12:28.694 INFO:tasks.rados.rados.0.smithi195.stdout:2045: done (3 left) 2024-09-13T10:12:28.694 INFO:tasks.rados.rados.0.smithi195.stdout:2046: done (2 left) 2024-09-13T10:12:28.694 INFO:tasks.rados.rados.0.smithi195.stdout:2047: done (1 left) 2024-09-13T10:12:28.694 INFO:tasks.rados.rados.0.smithi195.stdout:2048: write (excl) oid 13 current snap is 0 2024-09-13T10:12:28.694 INFO:tasks.rados.rados.0.smithi195.stdout:2048: seq_num 466 ranges {632062=563567,1885903=527817,2830644=720758} 2024-09-13T10:12:28.708 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:28.705+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:28.711 INFO:tasks.rados.rados.0.smithi195.stdout:2048: writing smithi19530714-13 from 632062 to 1195629 tid 1 2024-09-13T10:12:28.724 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:28.724 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:28.729 INFO:tasks.rados.rados.0.smithi195.stdout:2048: writing smithi19530714-13 from 1885903 to 2413720 tid 2 2024-09-13T10:12:28.740 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-09-13T10:12:28.746 INFO:tasks.rados.rados.0.smithi195.stdout:2048: writing smithi19530714-13 from 2830644 to 3551402 tid 3 2024-09-13T10:12:28.746 INFO:tasks.rados.rados.0.smithi195.stdout:2049: write oid 14 current snap is 0 2024-09-13T10:12:28.746 INFO:tasks.rados.rados.0.smithi195.stdout:2049: seq_num 467 ranges {648869=638816,1828032=787828,3141714=532030} 2024-09-13T10:12:28.760 INFO:tasks.rados.rados.0.smithi195.stdout:2049: writing smithi19530714-14 from 648869 to 1287685 tid 1 2024-09-13T10:12:28.772 INFO:tasks.rados.rados.0.smithi195.stdout:2049: writing smithi19530714-14 from 1828032 to 2615860 tid 2 2024-09-13T10:12:28.781 INFO:tasks.rados.rados.0.smithi195.stdout:2049: writing smithi19530714-14 from 3141714 to 3673744 tid 3 2024-09-13T10:12:28.781 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 3 2024-09-13T10:12:28.782 INFO:tasks.rados.rados.0.smithi195.stdout:2048: finishing write tid 1 to smithi19530714-13 2024-09-13T10:12:28.782 INFO:tasks.rados.rados.0.smithi195.stdout:2048: finishing write tid 2 to smithi19530714-13 2024-09-13T10:12:28.782 INFO:tasks.rados.rados.0.smithi195.stdout:2048: finishing write tid 3 to smithi19530714-13 2024-09-13T10:12:28.782 INFO:tasks.rados.rados.0.smithi195.stdout:2048: finishing write tid 4 to smithi19530714-13 2024-09-13T10:12:28.782 INFO:tasks.rados.rados.0.smithi195.stdout:2048: finishing write tid 5 to smithi19530714-13 2024-09-13T10:12:28.782 INFO:tasks.rados.rados.0.smithi195.stdout:2048: oid 13 updating version 0 to 94 2024-09-13T10:12:28.782 INFO:tasks.rados.rados.0.smithi195.stdout:2048: oid 13 version 94 is already newer than 93 2024-09-13T10:12:28.783 INFO:tasks.rados.rados.0.smithi195.stdout:2048: oid 13 version 94 is already newer than 92 2024-09-13T10:12:28.783 INFO:tasks.rados.rados.0.smithi195.stdout:2048: oid 13 version 94 is already newer than 91 2024-09-13T10:12:28.783 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 13 v 94 (ObjNum 466 snap 0 seq_num 466) dirty exists 2024-09-13T10:12:28.783 INFO:tasks.rados.rados.0.smithi195.stdout:2048: left oid 13 (ObjNum 466 snap 0 seq_num 466) 2024-09-13T10:12:28.783 INFO:tasks.rados.rados.0.smithi195.stdout:2049: finishing write tid 1 to smithi19530714-14 2024-09-13T10:12:28.783 INFO:tasks.rados.rados.0.smithi195.stdout:2049: finishing write tid 2 to smithi19530714-14 2024-09-13T10:12:28.783 INFO:tasks.rados.rados.0.smithi195.stdout:2048: done (2 left) 2024-09-13T10:12:28.783 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 2 2024-09-13T10:12:28.787 INFO:tasks.rados.rados.0.smithi195.stdout:2049: finishing write tid 3 to smithi19530714-14 2024-09-13T10:12:28.787 INFO:tasks.rados.rados.0.smithi195.stdout:2049: finishing write tid 4 to smithi19530714-14 2024-09-13T10:12:28.788 INFO:tasks.rados.rados.0.smithi195.stdout:2049: finishing write tid 5 to smithi19530714-14 2024-09-13T10:12:28.788 INFO:tasks.rados.rados.0.smithi195.stdout:2049: oid 14 updating version 0 to 157 2024-09-13T10:12:28.788 INFO:tasks.rados.rados.0.smithi195.stdout:2049: oid 14 updating version 157 to 160 2024-09-13T10:12:28.788 INFO:tasks.rados.rados.0.smithi195.stdout:2049: oid 14 version 160 is already newer than 158 2024-09-13T10:12:28.788 INFO:tasks.rados.rados.0.smithi195.stdout:2049: oid 14 version 160 is already newer than 159 2024-09-13T10:12:28.789 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 14 v 160 (ObjNum 467 snap 0 seq_num 467) dirty exists 2024-09-13T10:12:28.789 INFO:tasks.rados.rados.0.smithi195.stdout:2049: left oid 14 (ObjNum 467 snap 0 seq_num 467) 2024-09-13T10:12:28.789 INFO:tasks.rados.rados.0.smithi195.stdout:2049: done (1 left) 2024-09-13T10:12:28.789 INFO:tasks.rados.rados.0.smithi195.stdout: waiting on 1 2024-09-13T10:12:28.809 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:28.806+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:28.848 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T10:12:28.864 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:28.864 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:28.878 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-09-13T10:12:28.911 INFO:tasks.ceph.osd.14.smithi181.stderr:2024-09-13T10:12:28.905+0000 7fafb5779640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 26310) UID: 0 2024-09-13T10:12:28.958 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T10:12:29.001 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:12:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:12:29.011 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-09-13T10:12:29.010+0000 7face83e6640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 26450) UID: 0 2024-09-13T10:12:29.022 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:12:29.022 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:12:29.042 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:29.042 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:29.043 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:12:29.043 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:12:29.058 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-09-13T10:12:29.064 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:12:29.064 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:12:29.082 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:12:29.082 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:12:29.111 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:12:29.111 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:12:29.112 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:29.109+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:29.146 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:12:29.146 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:12:29.180 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:29.180 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:29.181 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:12:29.181 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:12:29.194 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T10:12:29.212 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:12:29.212 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:12:29.213 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:29.209+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:29.245 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:12:29.245 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:12:29.280 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:12:29.280 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:12:29.313 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:29.309+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:29.314 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:12:29.314 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:12:29.326 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:29.327 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:12:29.327 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:12:29.327 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:29.341 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T10:12:29.342 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:12:29.342 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:12:29.358 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:12:29.359 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:12:29.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:12:29.358+0000 7faff4de7640 0 -- 172.21.15.73:0/3292223148 >> v2:172.21.15.181:6804/1173379175 conn(0x7faff0220b60 msgr2=0x7faff0222fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:12:29.381 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:12:29.437 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:12:29.437 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:12:29.438 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:12:29.458 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:29.459 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:12:29.459 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:12:29.459 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:12:29.459 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:29.473 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T10:12:29.483 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:12:29.482+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:12:29.540 INFO:tasks.ceph.osd.5.smithi136.stderr:2024-09-13T10:12:29.538+0000 7f79a105f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 26303) UID: 0 2024-09-13T10:12:29.593 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:12:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:12:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:12:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:253 4.7 4:e81d9ad0:::smithi19530714-4:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.134475+0000", 2024-09-13T10:12:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 98.455383015999999, 2024-09-13T10:12:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013417535, 2024-09-13T10:12:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:29.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 253 2024-09-13T10:12:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134475+0000", 2024-09-13T10:12:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134475+0000", 2024-09-13T10:12:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134477+0000", 2024-09-13T10:12:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1859999999999999e-06 2024-09-13T10:12:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134486+0000", 2024-09-13T10:12:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3139999999999993e-06 2024-09-13T10:12:29.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134506+0000", 2024-09-13T10:12:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9165999999999999e-05 2024-09-13T10:12:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134515+0000", 2024-09-13T10:12:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3670000000000003e-06 2024-09-13T10:12:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134579+0000", 2024-09-13T10:12:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4200000000000002e-05 2024-09-13T10:12:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134661+0000", 2024-09-13T10:12:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1632999999999996e-05 2024-09-13T10:12:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.144025+0000", 2024-09-13T10:12:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093644190000000006 2024-09-13T10:12:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.144574+0000", 2024-09-13T10:12:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054873699999999999 2024-09-13T10:12:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.146021+0000", 2024-09-13T10:12:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001446826 2024-09-13T10:12:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.147761+0000", 2024-09-13T10:12:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017406399999999999 2024-09-13T10:12:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.147835+0000", 2024-09-13T10:12:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3708999999999995e-05 2024-09-13T10:12:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.147893+0000", 2024-09-13T10:12:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7337999999999999e-05 2024-09-13T10:12:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.600 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:256 4.7 4:e8c2cbec:::smithi19530714-7:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.214859+0000", 2024-09-13T10:12:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 98.374998657999996, 2024-09-13T10:12:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049512370999999999, 2024-09-13T10:12:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 256 2024-09-13T10:12:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214859+0000", 2024-09-13T10:12:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214859+0000", 2024-09-13T10:12:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214863+0000", 2024-09-13T10:12:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.343e-06 2024-09-13T10:12:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214876+0000", 2024-09-13T10:12:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.308e-05 2024-09-13T10:12:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214909+0000", 2024-09-13T10:12:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2882999999999997e-05 2024-09-13T10:12:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214925+0000", 2024-09-13T10:12:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6002e-05 2024-09-13T10:12:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215088+0000", 2024-09-13T10:12:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016304199999999999 2024-09-13T10:12:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215261+0000", 2024-09-13T10:12:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000173804 2024-09-13T10:12:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258838+0000", 2024-09-13T10:12:29.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043576675000000002 2024-09-13T10:12:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.260096+0000", 2024-09-13T10:12:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012580410000000001 2024-09-13T10:12:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.263959+0000", 2024-09-13T10:12:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038630840000000001 2024-09-13T10:12:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264262+0000", 2024-09-13T10:12:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030279699999999999 2024-09-13T10:12:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264321+0000", 2024-09-13T10:12:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9107000000000002e-05 2024-09-13T10:12:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264372+0000", 2024-09-13T10:12:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0513000000000002e-05 2024-09-13T10:12:29.606 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:266 4.7 4:e55883d8:::smithi19530714-17:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.215454+0000", 2024-09-13T10:12:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 98.374403997000002, 2024-09-13T10:12:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020880112999999999, 2024-09-13T10:12:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:29.607 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 266 2024-09-13T10:12:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215454+0000", 2024-09-13T10:12:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215454+0000", 2024-09-13T10:12:29.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215457+0000", 2024-09-13T10:12:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6029999999999999e-06 2024-09-13T10:12:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215471+0000", 2024-09-13T10:12:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4629e-05 2024-09-13T10:12:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215503+0000", 2024-09-13T10:12:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1449000000000003e-05 2024-09-13T10:12:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215528+0000", 2024-09-13T10:12:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5273e-05 2024-09-13T10:12:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215632+0000", 2024-09-13T10:12:29.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010387699999999999 2024-09-13T10:12:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215898+0000", 2024-09-13T10:12:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026582100000000002 2024-09-13T10:12:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228364+0000", 2024-09-13T10:12:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012466672 2024-09-13T10:12:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.229463+0000", 2024-09-13T10:12:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001098344 2024-09-13T10:12:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.230814+0000", 2024-09-13T10:12:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013513959999999999 2024-09-13T10:12:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236141+0000", 2024-09-13T10:12:29.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053270160000000004 2024-09-13T10:12:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236223+0000", 2024-09-13T10:12:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1799999999999996e-05 2024-09-13T10:12:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.236334+0000", 2024-09-13T10:12:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111233 2024-09-13T10:12:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:267 4.7 4:ea79a1ec:::smithi19530714-18:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.215583+0000", 2024-09-13T10:12:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 98.374275025000003, 2024-09-13T10:12:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.051600035000000002, 2024-09-13T10:12:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 267 2024-09-13T10:12:29.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215583+0000", 2024-09-13T10:12:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215583+0000", 2024-09-13T10:12:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215585+0000", 2024-09-13T10:12:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9099999999999999e-06 2024-09-13T10:12:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215594+0000", 2024-09-13T10:12:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8799999999999997e-06 2024-09-13T10:12:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215613+0000", 2024-09-13T10:12:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9473000000000001e-05 2024-09-13T10:12:29.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215630+0000", 2024-09-13T10:12:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6466000000000001e-05 2024-09-13T10:12:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.216360+0000", 2024-09-13T10:12:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073001200000000004 2024-09-13T10:12:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.216526+0000", 2024-09-13T10:12:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000166775 2024-09-13T10:12:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.260535+0000", 2024-09-13T10:12:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.044008310000000002 2024-09-13T10:12:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261671+0000", 2024-09-13T10:12:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011362169999999999 2024-09-13T10:12:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.266819+0000", 2024-09-13T10:12:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0051479940000000004 2024-09-13T10:12:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.267087+0000", 2024-09-13T10:12:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026847299999999998 2024-09-13T10:12:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.267140+0000", 2024-09-13T10:12:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2054e-05 2024-09-13T10:12:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.267183+0000", 2024-09-13T10:12:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3470999999999998e-05 2024-09-13T10:12:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:303 4.7 4:e81d9ad0:::smithi19530714-4:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.268636+0000", 2024-09-13T10:12:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 98.321222410000004, 2024-09-13T10:12:29.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018691609000000001, 2024-09-13T10:12:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 303 2024-09-13T10:12:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268636+0000", 2024-09-13T10:12:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268636+0000", 2024-09-13T10:12:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268638+0000", 2024-09-13T10:12:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8990000000000001e-06 2024-09-13T10:12:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268651+0000", 2024-09-13T10:12:29.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2556e-05 2024-09-13T10:12:29.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268682+0000", 2024-09-13T10:12:29.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0979000000000003e-05 2024-09-13T10:12:29.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268707+0000", 2024-09-13T10:12:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4680000000000001e-05 2024-09-13T10:12:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268787+0000", 2024-09-13T10:12:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9878999999999999e-05 2024-09-13T10:12:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.268881+0000", 2024-09-13T10:12:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4242999999999999e-05 2024-09-13T10:12:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:29.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.283577+0000", 2024-09-13T10:12:29.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014695952999999999 2024-09-13T10:12:29.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:29.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.284599+0000", 2024-09-13T10:12:29.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001022329 2024-09-13T10:12:29.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285985+0000", 2024-09-13T10:12:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013861430000000001 2024-09-13T10:12:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.287150+0000", 2024-09-13T10:12:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001164382 2024-09-13T10:12:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.287245+0000", 2024-09-13T10:12:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5643999999999994e-05 2024-09-13T10:12:29.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.287327+0000", 2024-09-13T10:12:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1922000000000006e-05 2024-09-13T10:12:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:567 4.7 4:ea79a1ec:::smithi19530714-18:head [assert-version v12,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:12:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:12:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 98.222557257999995, 2024-09-13T10:12:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012585694999999999, 2024-09-13T10:12:29.627 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 567 2024-09-13T10:12:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:12:29.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:12:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367301+0000", 2024-09-13T10:12:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1200000000000003e-07 2024-09-13T10:12:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367306+0000", 2024-09-13T10:12:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0309999999999998e-06 2024-09-13T10:12:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367314+0000", 2024-09-13T10:12:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8960000000000003e-06 2024-09-13T10:12:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367321+0000", 2024-09-13T10:12:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4880000000000004e-06 2024-09-13T10:12:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367612+0000", 2024-09-13T10:12:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029163099999999999 2024-09-13T10:12:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.367715+0000", 2024-09-13T10:12:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102837 2024-09-13T10:12:29.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.374040+0000", 2024-09-13T10:12:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0063245539999999996 2024-09-13T10:12:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.374609+0000", 2024-09-13T10:12:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056937299999999997 2024-09-13T10:12:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379774+0000", 2024-09-13T10:12:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005164737 2024-09-13T10:12:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379810+0000", 2024-09-13T10:12:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6587000000000001e-05 2024-09-13T10:12:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379845+0000", 2024-09-13T10:12:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.468e-05 2024-09-13T10:12:29.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.379886+0000", 2024-09-13T10:12:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1269000000000001e-05 2024-09-13T10:12:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1039 4.17 4:ea79a1ec:::smithi19530714-18:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e42)", 2024-09-13T10:12:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:56.096166+0000", 2024-09-13T10:12:29.634 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 93.493692000999999, 2024-09-13T10:12:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.051040927, 2024-09-13T10:12:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1039 2024-09-13T10:12:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096166+0000", 2024-09-13T10:12:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096166+0000", 2024-09-13T10:12:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096169+0000", 2024-09-13T10:12:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8100000000000002e-06 2024-09-13T10:12:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096180+0000", 2024-09-13T10:12:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1537e-05 2024-09-13T10:12:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096212+0000", 2024-09-13T10:12:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2104999999999998e-05 2024-09-13T10:12:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096268+0000", 2024-09-13T10:12:29.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5127999999999998e-05 2024-09-13T10:12:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096480+0000", 2024-09-13T10:12:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021249100000000001 2024-09-13T10:12:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.096692+0000", 2024-09-13T10:12:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021200400000000001 2024-09-13T10:12:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:56.147207+0000", 2024-09-13T10:12:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.050514851999999999 2024-09-13T10:12:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1574 4.7 4:e79f1505:::smithi19530714-32:head [write 1799494~402223] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:12:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.716930+0000", 2024-09-13T10:12:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 84.872927735000005, 2024-09-13T10:12:29.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017286329999999999, 2024-09-13T10:12:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1574 2024-09-13T10:12:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.716930+0000", 2024-09-13T10:12:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.716930+0000", 2024-09-13T10:12:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.716962+0000", 2024-09-13T10:12:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1279999999999999e-05 2024-09-13T10:12:29.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717453+0000", 2024-09-13T10:12:29.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049160399999999998 2024-09-13T10:12:29.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717480+0000", 2024-09-13T10:12:29.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7313000000000001e-05 2024-09-13T10:12:29.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717496+0000", 2024-09-13T10:12:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5435999999999999e-05 2024-09-13T10:12:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717592+0000", 2024-09-13T10:12:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5851000000000001e-05 2024-09-13T10:12:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717732+0000", 2024-09-13T10:12:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013985000000000001 2024-09-13T10:12:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717962+0000", 2024-09-13T10:12:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000230518 2024-09-13T10:12:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722934+0000", 2024-09-13T10:12:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049717099999999998 2024-09-13T10:12:29.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724179+0000", 2024-09-13T10:12:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012455389999999999 2024-09-13T10:12:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.733963+0000", 2024-09-13T10:12:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097835349999999995 2024-09-13T10:12:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.734101+0000", 2024-09-13T10:12:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013852 2024-09-13T10:12:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.734217+0000", 2024-09-13T10:12:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011517400000000001 2024-09-13T10:12:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1575 4.7 4:e79f1505:::smithi19530714-32:head [setxattr _header (56) in=7b,truncate 2201717] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:12:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.717530+0000", 2024-09-13T10:12:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 84.872327579, 2024-09-13T10:12:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018674682000000001, 2024-09-13T10:12:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1575 2024-09-13T10:12:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.647 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717530+0000", 2024-09-13T10:12:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717530+0000", 2024-09-13T10:12:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717532+0000", 2024-09-13T10:12:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.813e-06 2024-09-13T10:12:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717542+0000", 2024-09-13T10:12:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4930000000000004e-06 2024-09-13T10:12:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717559+0000", 2024-09-13T10:12:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7337e-05 2024-09-13T10:12:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717575+0000", 2024-09-13T10:12:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6385999999999999e-05 2024-09-13T10:12:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.719954+0000", 2024-09-13T10:12:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023782679999999998 2024-09-13T10:12:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.720308+0000", 2024-09-13T10:12:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035397100000000002 2024-09-13T10:12:29.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.720827+0000", 2024-09-13T10:12:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00051909999999999999 2024-09-13T10:12:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722944+0000", 2024-09-13T10:12:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021173210000000001 2024-09-13T10:12:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724274+0000", 2024-09-13T10:12:29.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013303099999999999 2024-09-13T10:12:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.735926+0000", 2024-09-13T10:12:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011651846 2024-09-13T10:12:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736061+0000", 2024-09-13T10:12:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013452000000000001 2024-09-13T10:12:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736205+0000", 2024-09-13T10:12:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014431700000000001 2024-09-13T10:12:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1576 4.7 4:e79f1505:::smithi19530714-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:12:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.717602+0000", 2024-09-13T10:12:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 84.872256394000004, 2024-09-13T10:12:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019134776999999999, 2024-09-13T10:12:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.653 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1576 2024-09-13T10:12:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717602+0000", 2024-09-13T10:12:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717602+0000", 2024-09-13T10:12:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717603+0000", 2024-09-13T10:12:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8899999999999999e-06 2024-09-13T10:12:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717610+0000", 2024-09-13T10:12:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8900000000000001e-06 2024-09-13T10:12:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717631+0000", 2024-09-13T10:12:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0741999999999999e-05 2024-09-13T10:12:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.717644+0000", 2024-09-13T10:12:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2680000000000001e-05 2024-09-13T10:12:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722783+0000", 2024-09-13T10:12:29.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005139429 2024-09-13T10:12:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.722912+0000", 2024-09-13T10:12:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012886399999999999 2024-09-13T10:12:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736301+0000", 2024-09-13T10:12:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013389082 2024-09-13T10:12:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736498+0000", 2024-09-13T10:12:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000197059 2024-09-13T10:12:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.736736+0000", 2024-09-13T10:12:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023814100000000001 2024-09-13T10:12:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1580 4.17 4:e8c2cbec:::smithi19530714-7:head [write 618524~710869,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-09-13T10:12:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:04.724425+0000", 2024-09-13T10:12:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 84.865432652999999, 2024-09-13T10:12:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013470263999999999, 2024-09-13T10:12:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1580 2024-09-13T10:12:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.659 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724425+0000", 2024-09-13T10:12:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724425+0000", 2024-09-13T10:12:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.724427+0000", 2024-09-13T10:12:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9530000000000002e-06 2024-09-13T10:12:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725799+0000", 2024-09-13T10:12:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001371466 2024-09-13T10:12:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725823+0000", 2024-09-13T10:12:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4332999999999999e-05 2024-09-13T10:12:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725844+0000", 2024-09-13T10:12:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0539000000000001e-05 2024-09-13T10:12:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.725963+0000", 2024-09-13T10:12:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119524 2024-09-13T10:12:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.726194+0000", 2024-09-13T10:12:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023045800000000001 2024-09-13T10:12:29.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.726619+0000", 2024-09-13T10:12:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042517199999999997 2024-09-13T10:12:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.735852+0000", 2024-09-13T10:12:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0092335560000000004 2024-09-13T10:12:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.735959+0000", 2024-09-13T10:12:29.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106351 2024-09-13T10:12:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.737757+0000", 2024-09-13T10:12:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017985119999999999 2024-09-13T10:12:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.737838+0000", 2024-09-13T10:12:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0457999999999994e-05 2024-09-13T10:12:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:04.737896+0000", 2024-09-13T10:12:29.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7942000000000002e-05 2024-09-13T10:12:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1855 4.17 4:e81d9ad0:::smithi19530714-4:head [write 739541~709000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.758272+0000", 2024-09-13T10:12:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.831585949000001, 2024-09-13T10:12:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01875696, 2024-09-13T10:12:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:29.665 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1855 2024-09-13T10:12:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.758272+0000", 2024-09-13T10:12:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.758272+0000", 2024-09-13T10:12:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.758275+0000", 2024-09-13T10:12:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4429999999999999e-06 2024-09-13T10:12:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.759964+0000", 2024-09-13T10:12:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001688557 2024-09-13T10:12:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.759996+0000", 2024-09-13T10:12:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1705999999999998e-05 2024-09-13T10:12:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.760013+0000", 2024-09-13T10:12:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6846000000000001e-05 2024-09-13T10:12:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.760156+0000", 2024-09-13T10:12:29.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014331700000000001 2024-09-13T10:12:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-13T10:12:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.760167+0000", 2024-09-13T10:12:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.062e-05 2024-09-13T10:12:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.763490+0000", 2024-09-13T10:12:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033235309999999998 2024-09-13T10:12:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.763611+0000", 2024-09-13T10:12:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012099699999999999 2024-09-13T10:12:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.763830+0000", 2024-09-13T10:12:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000218519 2024-09-13T10:12:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773195+0000", 2024-09-13T10:12:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093654340000000006 2024-09-13T10:12:29.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.774701+0000", 2024-09-13T10:12:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00150588 2024-09-13T10:12:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.776817+0000", 2024-09-13T10:12:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021156180000000001 2024-09-13T10:12:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.776910+0000", 2024-09-13T10:12:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3194000000000003e-05 2024-09-13T10:12:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.777029+0000", 2024-09-13T10:12:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000119298 2024-09-13T10:12:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1856 4.17 4:e81d9ad0:::smithi19530714-4:head [write 2121859~426376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.765194+0000", 2024-09-13T10:12:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.824664155999997, 2024-09-13T10:12:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013074073, 2024-09-13T10:12:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1856 2024-09-13T10:12:29.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765194+0000", 2024-09-13T10:12:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765194+0000", 2024-09-13T10:12:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765196+0000", 2024-09-13T10:12:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.317e-06 2024-09-13T10:12:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765559+0000", 2024-09-13T10:12:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036326199999999999 2024-09-13T10:12:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765581+0000", 2024-09-13T10:12:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1676e-05 2024-09-13T10:12:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765595+0000", 2024-09-13T10:12:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4025999999999999e-05 2024-09-13T10:12:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.767613+0000", 2024-09-13T10:12:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00201768 2024-09-13T10:12:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.767751+0000", 2024-09-13T10:12:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013809 2024-09-13T10:12:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.767983+0000", 2024-09-13T10:12:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023206499999999999 2024-09-13T10:12:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773203+0000", 2024-09-13T10:12:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052195599999999998 2024-09-13T10:12:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.777481+0000", 2024-09-13T10:12:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042782840000000003 2024-09-13T10:12:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.778120+0000", 2024-09-13T10:12:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063947299999999999 2024-09-13T10:12:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.778203+0000", 2024-09-13T10:12:29.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2857999999999998e-05 2024-09-13T10:12:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.778268+0000", 2024-09-13T10:12:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4782000000000004e-05 2024-09-13T10:12:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1857 4.17 4:e81d9ad0:::smithi19530714-4:head [setxattr _header (55) in=7b,truncate 2739541] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.765630+0000", 2024-09-13T10:12:29.679 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.824228449000003, 2024-09-13T10:12:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013989379999999999, 2024-09-13T10:12:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1857 2024-09-13T10:12:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765630+0000", 2024-09-13T10:12:29.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765630+0000", 2024-09-13T10:12:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765631+0000", 2024-09-13T10:12:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.64e-06 2024-09-13T10:12:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765641+0000", 2024-09-13T10:12:29.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0047000000000001e-05 2024-09-13T10:12:29.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765658+0000", 2024-09-13T10:12:29.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6909e-05 2024-09-13T10:12:29.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765673+0000", 2024-09-13T10:12:29.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4980000000000001e-05 2024-09-13T10:12:29.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.771667+0000", 2024-09-13T10:12:29.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0059938209999999999 2024-09-13T10:12:29.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.771795+0000", 2024-09-13T10:12:29.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012816999999999999 2024-09-13T10:12:29.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:29.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.772028+0000", 2024-09-13T10:12:29.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023327800000000001 2024-09-13T10:12:29.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:29.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773657+0000", 2024-09-13T10:12:29.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016287039999999999 2024-09-13T10:12:29.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.777579+0000", 2024-09-13T10:12:29.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039218589999999998 2024-09-13T10:12:29.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779159+0000", 2024-09-13T10:12:29.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001580176 2024-09-13T10:12:29.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:29.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779343+0000", 2024-09-13T10:12:29.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000183769 2024-09-13T10:12:29.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779619+0000", 2024-09-13T10:12:29.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027602699999999998 2024-09-13T10:12:29.686 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.686 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.686 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.687 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1858 4.17 4:e81d9ad0:::smithi19530714-4:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e54)", 2024-09-13T10:12:29.687 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:10.765692+0000", 2024-09-13T10:12:29.687 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 78.824166363000003, 2024-09-13T10:12:29.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014311129000000001, 2024-09-13T10:12:29.687 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.687 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.687 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:29.687 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.687 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.687 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.687 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1858 2024-09-13T10:12:29.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.687 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765692+0000", 2024-09-13T10:12:29.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765692+0000", 2024-09-13T10:12:29.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765693+0000", 2024-09-13T10:12:29.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-09-13T10:12:29.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765697+0000", 2024-09-13T10:12:29.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6770000000000003e-06 2024-09-13T10:12:29.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765707+0000", 2024-09-13T10:12:29.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0129000000000001e-05 2024-09-13T10:12:29.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.765715+0000", 2024-09-13T10:12:29.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0469999999999994e-06 2024-09-13T10:12:29.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773032+0000", 2024-09-13T10:12:29.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0073166680000000001 2024-09-13T10:12:29.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:29.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.773169+0000", 2024-09-13T10:12:29.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136964 2024-09-13T10:12:29.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779726+0000", 2024-09-13T10:12:29.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065572499999999997 2024-09-13T10:12:29.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.779851+0000", 2024-09-13T10:12:29.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012477500000000001 2024-09-13T10:12:29.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:10.780003+0000", 2024-09-13T10:12:29.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015165299999999999 2024-09-13T10:12:29.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.692 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.692 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.692 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2113 4.17 4:e81d9ad0:::smithi19530714-4:head [write 1410296~761389] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T10:12:29.692 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:15.709580+0000", 2024-09-13T10:12:29.692 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 73.880277981000006, 2024-09-13T10:12:29.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012946568, 2024-09-13T10:12:29.692 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.692 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.692 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:29.693 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.693 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.693 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.693 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2113 2024-09-13T10:12:29.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.693 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.709580+0000", 2024-09-13T10:12:29.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.709580+0000", 2024-09-13T10:12:29.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.709582+0000", 2024-09-13T10:12:29.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4530000000000001e-06 2024-09-13T10:12:29.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710399+0000", 2024-09-13T10:12:29.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081639799999999997 2024-09-13T10:12:29.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710428+0000", 2024-09-13T10:12:29.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8736000000000001e-05 2024-09-13T10:12:29.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710443+0000", 2024-09-13T10:12:29.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5216e-05 2024-09-13T10:12:29.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710572+0000", 2024-09-13T10:12:29.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012882799999999999 2024-09-13T10:12:29.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.710808+0000", 2024-09-13T10:12:29.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023585500000000001 2024-09-13T10:12:29.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 6,10", 2024-09-13T10:12:29.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.711245+0000", 2024-09-13T10:12:29.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043798099999999999 2024-09-13T10:12:29.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:29.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.716561+0000", 2024-09-13T10:12:29.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053156760000000001 2024-09-13T10:12:29.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.718008+0000", 2024-09-13T10:12:29.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014466380000000001 2024-09-13T10:12:29.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:29.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.722191+0000", 2024-09-13T10:12:29.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041829229999999998 2024-09-13T10:12:29.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:29.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.722439+0000", 2024-09-13T10:12:29.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024823899999999998 2024-09-13T10:12:29.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:15.722527+0000", 2024-09-13T10:12:29.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7625000000000002e-05 2024-09-13T10:12:29.698 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.698 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.698 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.699 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2412 4.7 4:e55883d8:::smithi19530714-17:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e63)", 2024-09-13T10:12:29.699 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:22.233147+0000", 2024-09-13T10:12:29.699 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 67.356711015000002, 2024-09-13T10:12:29.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.042005107999999999, 2024-09-13T10:12:29.699 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.699 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.699 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:29.699 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.699 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.699 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.699 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2412 2024-09-13T10:12:29.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.699 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233147+0000", 2024-09-13T10:12:29.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233147+0000", 2024-09-13T10:12:29.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233150+0000", 2024-09-13T10:12:29.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0029999999999999e-06 2024-09-13T10:12:29.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233162+0000", 2024-09-13T10:12:29.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2449000000000001e-05 2024-09-13T10:12:29.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233195+0000", 2024-09-13T10:12:29.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2455e-05 2024-09-13T10:12:29.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233213+0000", 2024-09-13T10:12:29.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8185999999999998e-05 2024-09-13T10:12:29.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233351+0000", 2024-09-13T10:12:29.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000137942 2024-09-13T10:12:29.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.233497+0000", 2024-09-13T10:12:29.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014627100000000001 2024-09-13T10:12:29.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:22.275152+0000", 2024-09-13T10:12:29.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041654801999999998 2024-09-13T10:12:29.703 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.703 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.703 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.703 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2766 4.17 4:e81d9ad0:::smithi19530714-4:head [sparse-read 1791986~224887,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-13T10:12:29.703 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:32.706469+0000", 2024-09-13T10:12:29.703 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 56.883388615999998, 2024-09-13T10:12:29.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.186036496, 2024-09-13T10:12:29.704 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.704 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.704 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:29.704 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.704 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.704 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.704 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2766 2024-09-13T10:12:29.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.704 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706469+0000", 2024-09-13T10:12:29.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706469+0000", 2024-09-13T10:12:29.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706472+0000", 2024-09-13T10:12:29.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0800000000000002e-06 2024-09-13T10:12:29.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706483+0000", 2024-09-13T10:12:29.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0386e-05 2024-09-13T10:12:29.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706522+0000", 2024-09-13T10:12:29.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8927999999999997e-05 2024-09-13T10:12:29.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706544+0000", 2024-09-13T10:12:29.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2436e-05 2024-09-13T10:12:29.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706732+0000", 2024-09-13T10:12:29.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000187715 2024-09-13T10:12:29.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:29.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706767+0000", 2024-09-13T10:12:29.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5462e-05 2024-09-13T10:12:29.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.891841+0000", 2024-09-13T10:12:29.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.185074035 2024-09-13T10:12:29.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.892071+0000", 2024-09-13T10:12:29.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000229643 2024-09-13T10:12:29.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.892506+0000", 2024-09-13T10:12:29.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043481100000000001 2024-09-13T10:12:29.709 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.709 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.709 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.709 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2767 4.17 4:e81d9ad0:::smithi19530714-4:head [sparse-read 0~1606736] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-13T10:12:29.709 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:32.706596+0000", 2024-09-13T10:12:29.709 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 56.883261748000002, 2024-09-13T10:12:29.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18875979000000001, 2024-09-13T10:12:29.709 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.710 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.710 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:29.710 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.710 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.710 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.710 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2767 2024-09-13T10:12:29.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.710 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706596+0000", 2024-09-13T10:12:29.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706596+0000", 2024-09-13T10:12:29.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706598+0000", 2024-09-13T10:12:29.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5659999999999999e-06 2024-09-13T10:12:29.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706606+0000", 2024-09-13T10:12:29.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1899999999999995e-06 2024-09-13T10:12:29.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706624+0000", 2024-09-13T10:12:29.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7546000000000001e-05 2024-09-13T10:12:29.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706639+0000", 2024-09-13T10:12:29.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5126000000000001e-05 2024-09-13T10:12:29.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706840+0000", 2024-09-13T10:12:29.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020117099999999999 2024-09-13T10:12:29.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:29.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706860+0000", 2024-09-13T10:12:29.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0205999999999999e-05 2024-09-13T10:12:29.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.893234+0000", 2024-09-13T10:12:29.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18637414799999999 2024-09-13T10:12:29.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.893485+0000", 2024-09-13T10:12:29.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025044200000000003 2024-09-13T10:12:29.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.895356+0000", 2024-09-13T10:12:29.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018713950000000001 2024-09-13T10:12:29.714 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.714 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.715 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.715 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2768 4.17 4:e81d9ad0:::smithi19530714-4:head [sparse-read 0~1606736] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e71)", 2024-09-13T10:12:29.715 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:32.706665+0000", 2024-09-13T10:12:29.715 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 56.883193015000003, 2024-09-13T10:12:29.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18904678599999999, 2024-09-13T10:12:29.715 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:29.715 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:29.715 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:29.715 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:29.715 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:29.716 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:29.716 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2768 2024-09-13T10:12:29.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.716 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:29.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:29.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706665+0000", 2024-09-13T10:12:29.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:29.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706665+0000", 2024-09-13T10:12:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706666+0000", 2024-09-13T10:12:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.297e-06 2024-09-13T10:12:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706676+0000", 2024-09-13T10:12:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6090000000000007e-06 2024-09-13T10:12:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706692+0000", 2024-09-13T10:12:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5758999999999999e-05 2024-09-13T10:12:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706706+0000", 2024-09-13T10:12:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.482e-05 2024-09-13T10:12:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706925+0000", 2024-09-13T10:12:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000218607 2024-09-13T10:12:29.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.706945+0000", 2024-09-13T10:12:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9466e-05 2024-09-13T10:12:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.895401+0000", 2024-09-13T10:12:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18845690700000001 2024-09-13T10:12:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:29.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.895464+0000", 2024-09-13T10:12:29.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2434999999999994e-05 2024-09-13T10:12:29.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:29.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:29.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:29.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:32.895712+0000", 2024-09-13T10:12:29.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000247886 2024-09-13T10:12:29.720 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.720 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.720 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.720 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:29.720 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:29.720 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:29.722 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T10:12:29.723 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:29.717+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:29.823 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:12:29.822+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:12:29.892 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:12:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:12:29.892 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:29.906 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T10:12:29.924 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:29.917+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:29.965 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-09-13T10:12:30.025 INFO:tasks.ceph.osd.1.smithi136.stderr:2024-09-13T10:12:30.022+0000 7f8deb675640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 26301) UID: 0 2024-09-13T10:12:30.025 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:30.025 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:12:30.025 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:12:30.025 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:12:30.025 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:30.039 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T10:12:30.125 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:30.122+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:30.212 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:30.212 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:12:30.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:12:30.212 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:12:30.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.212 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:199 4.0 4:0bd0428c:::smithi19530714-50:head [setxattr _header (56) in=7b,truncate 2823543] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:30.213 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.992081+0000", 2024-09-13T10:12:30.213 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 99.216358292999999, 2024-09-13T10:12:30.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011866829000000001, 2024-09-13T10:12:30.213 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.213 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.213 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:30.213 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.213 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.213 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.213 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 199 2024-09-13T10:12:30.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.213 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992081+0000", 2024-09-13T10:12:30.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992081+0000", 2024-09-13T10:12:30.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992082+0000", 2024-09-13T10:12:30.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.004e-06 2024-09-13T10:12:30.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992090+0000", 2024-09-13T10:12:30.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7440000000000007e-06 2024-09-13T10:12:30.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992098+0000", 2024-09-13T10:12:30.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3650000000000004e-06 2024-09-13T10:12:30.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992107+0000", 2024-09-13T10:12:30.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9230000000000008e-06 2024-09-13T10:12:30.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.995938+0000", 2024-09-13T10:12:30.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003830773 2024-09-13T10:12:30.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.996229+0000", 2024-09-13T10:12:30.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029133300000000001 2024-09-13T10:12:30.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:12:30.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.996692+0000", 2024-09-13T10:12:30.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046326700000000001 2024-09-13T10:12:30.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:30.217 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997974+0000", 2024-09-13T10:12:30.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001282016 2024-09-13T10:12:30.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.217 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.999243+0000", 2024-09-13T10:12:30.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012682360000000001 2024-09-13T10:12:30.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.003767+0000", 2024-09-13T10:12:30.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045243970000000003 2024-09-13T10:12:30.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:30.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.003851+0000", 2024-09-13T10:12:30.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4195000000000003e-05 2024-09-13T10:12:30.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.003948+0000", 2024-09-13T10:12:30.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6576000000000005e-05 2024-09-13T10:12:30.219 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.219 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.219 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.219 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:200 4.0 4:0bd0428c:::smithi19530714-50:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:30.220 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.992119+0000", 2024-09-13T10:12:30.220 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 99.216320132999996, 2024-09-13T10:12:30.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012149748, 2024-09-13T10:12:30.220 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.220 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.220 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.220 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.220 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.220 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.220 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 200 2024-09-13T10:12:30.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.220 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992119+0000", 2024-09-13T10:12:30.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992119+0000", 2024-09-13T10:12:30.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992120+0000", 2024-09-13T10:12:30.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9999999999999997e-07 2024-09-13T10:12:30.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992123+0000", 2024-09-13T10:12:30.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9569999999999998e-06 2024-09-13T10:12:30.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992131+0000", 2024-09-13T10:12:30.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0020000000000006e-06 2024-09-13T10:12:30.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.222 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.222 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992140+0000", 2024-09-13T10:12:30.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7900000000000005e-06 2024-09-13T10:12:30.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997418+0000", 2024-09-13T10:12:30.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052784349999999997 2024-09-13T10:12:30.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:30.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.997546+0000", 2024-09-13T10:12:30.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012788900000000001 2024-09-13T10:12:30.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.004004+0000", 2024-09-13T10:12:30.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0064580549999999999 2024-09-13T10:12:30.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.004128+0000", 2024-09-13T10:12:30.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000123628 2024-09-13T10:12:30.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.004269+0000", 2024-09-13T10:12:30.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000141292 2024-09-13T10:12:30.224 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.225 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.225 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.225 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:265 4.0 4:0b4c105e:::smithi19530714-16:head [assert-version v3,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:30.225 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.215372+0000", 2024-09-13T10:12:30.225 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 98.993067100999994, 2024-09-13T10:12:30.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013030243, 2024-09-13T10:12:30.225 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.225 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.225 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:30.225 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.225 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.225 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.226 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 265 2024-09-13T10:12:30.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.226 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215372+0000", 2024-09-13T10:12:30.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215372+0000", 2024-09-13T10:12:30.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215376+0000", 2024-09-13T10:12:30.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.393e-06 2024-09-13T10:12:30.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215387+0000", 2024-09-13T10:12:30.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1986e-05 2024-09-13T10:12:30.227 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.227 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.227 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.227 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215421+0000", 2024-09-13T10:12:30.227 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3355999999999997e-05 2024-09-13T10:12:30.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215438+0000", 2024-09-13T10:12:30.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7059e-05 2024-09-13T10:12:30.228 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.228 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.228 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.228 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215529+0000", 2024-09-13T10:12:30.228 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1571999999999997e-05 2024-09-13T10:12:30.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.215621+0000", 2024-09-13T10:12:30.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1094999999999994e-05 2024-09-13T10:12:30.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:12:30.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.223688+0000", 2024-09-13T10:12:30.229 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0080669859999999999 2024-09-13T10:12:30.229 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.229 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.229 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:30.229 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.225291+0000", 2024-09-13T10:12:30.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001603548 2024-09-13T10:12:30.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.225833+0000", 2024-09-13T10:12:30.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054185500000000005 2024-09-13T10:12:30.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.230 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228151+0000", 2024-09-13T10:12:30.230 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023176640000000001 2024-09-13T10:12:30.230 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.230 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.230 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:30.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228300+0000", 2024-09-13T10:12:30.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000149152 2024-09-13T10:12:30.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.231 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.231 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.228402+0000", 2024-09-13T10:12:30.231 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102577 2024-09-13T10:12:30.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.231 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.231 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.231 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.231 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.231 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:289 4.0 4:03fab808:::smithi19530714-40:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:30.231 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.250617+0000", 2024-09-13T10:12:30.231 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 98.957821815000003, 2024-09-13T10:12:30.232 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010740764999999999, 2024-09-13T10:12:30.232 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.232 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.232 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:30.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.232 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.232 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 289 2024-09-13T10:12:30.232 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.232 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.232 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.232 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250617+0000", 2024-09-13T10:12:30.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.233 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250617+0000", 2024-09-13T10:12:30.233 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.233 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.233 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.233 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250621+0000", 2024-09-13T10:12:30.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4029999999999999e-06 2024-09-13T10:12:30.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250637+0000", 2024-09-13T10:12:30.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5950000000000001e-05 2024-09-13T10:12:30.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.234 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.234 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250673+0000", 2024-09-13T10:12:30.234 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6559e-05 2024-09-13T10:12:30.234 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.234 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250699+0000", 2024-09-13T10:12:30.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5479000000000001e-05 2024-09-13T10:12:30.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250825+0000", 2024-09-13T10:12:30.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012585600000000001 2024-09-13T10:12:30.235 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.235 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.235 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.235 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.250971+0000", 2024-09-13T10:12:30.235 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000146002 2024-09-13T10:12:30.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:12:30.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.257715+0000", 2024-09-13T10:12:30.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0067440360000000001 2024-09-13T10:12:30.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:30.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.258695+0000", 2024-09-13T10:12:30.236 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0009798509999999999 2024-09-13T10:12:30.236 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.236 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.236 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.236 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.260555+0000", 2024-09-13T10:12:30.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018600439999999999 2024-09-13T10:12:30.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261249+0000", 2024-09-13T10:12:30.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00069435899999999997 2024-09-13T10:12:30.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.237 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.237 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:30.237 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261307+0000", 2024-09-13T10:12:30.237 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7630000000000002e-05 2024-09-13T10:12:30.237 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.238 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.238 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.261358+0000", 2024-09-13T10:12:30.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1595999999999999e-05 2024-09-13T10:12:30.238 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.238 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.238 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.238 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.238 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.238 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:299 4.0 4:0bd0428c:::smithi19530714-50:head [assert-version v9,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:30.238 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.264107+0000", 2024-09-13T10:12:30.238 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 98.944332426000003, 2024-09-13T10:12:30.238 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.020961186999999999, 2024-09-13T10:12:30.239 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.239 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.239 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:30.239 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.239 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.239 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.239 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 299 2024-09-13T10:12:30.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.239 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.239 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.239 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264107+0000", 2024-09-13T10:12:30.239 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.239 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.239 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264107+0000", 2024-09-13T10:12:30.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264111+0000", 2024-09-13T10:12:30.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0330000000000002e-06 2024-09-13T10:12:30.240 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.240 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.240 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.240 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264121+0000", 2024-09-13T10:12:30.240 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0543999999999999e-05 2024-09-13T10:12:30.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264150+0000", 2024-09-13T10:12:30.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8581999999999998e-05 2024-09-13T10:12:30.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.241 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264171+0000", 2024-09-13T10:12:30.241 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1095999999999999e-05 2024-09-13T10:12:30.241 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.241 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.241 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264369+0000", 2024-09-13T10:12:30.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019837000000000001 2024-09-13T10:12:30.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.264535+0000", 2024-09-13T10:12:30.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000165324 2024-09-13T10:12:30.242 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.242 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.242 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:12:30.242 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.276997+0000", 2024-09-13T10:12:30.242 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012461842000000001 2024-09-13T10:12:30.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:30.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.277946+0000", 2024-09-13T10:12:30.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00094919299999999998 2024-09-13T10:12:30.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.243 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.281540+0000", 2024-09-13T10:12:30.243 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035946910000000001 2024-09-13T10:12:30.243 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.243 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.243 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.284932+0000", 2024-09-13T10:12:30.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033913799999999998 2024-09-13T10:12:30.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:30.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285010+0000", 2024-09-13T10:12:30.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8059000000000006e-05 2024-09-13T10:12:30.244 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.244 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.244 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.244 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.285068+0000", 2024-09-13T10:12:30.244 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8072999999999998e-05 2024-09-13T10:12:30.244 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.245 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.245 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.245 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.245 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.245 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:489 4.0 4:03fab808:::smithi19530714-40:head [assert-version v6,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e39)", 2024-09-13T10:12:30.245 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.350324+0000", 2024-09-13T10:12:30.245 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 98.858115409000007, 2024-09-13T10:12:30.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014394344, 2024-09-13T10:12:30.245 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.245 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.245 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:30.245 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.246 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.246 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 489 2024-09-13T10:12:30.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.246 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.246 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350324+0000", 2024-09-13T10:12:30.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.246 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.246 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350324+0000", 2024-09-13T10:12:30.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350325+0000", 2024-09-13T10:12:30.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6270000000000001e-06 2024-09-13T10:12:30.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350333+0000", 2024-09-13T10:12:30.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3719999999999999e-06 2024-09-13T10:12:30.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350349+0000", 2024-09-13T10:12:30.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6004000000000001e-05 2024-09-13T10:12:30.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350359+0000", 2024-09-13T10:12:30.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.73e-06 2024-09-13T10:12:30.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350455+0000", 2024-09-13T10:12:30.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6260000000000003e-05 2024-09-13T10:12:30.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.350526+0000", 2024-09-13T10:12:30.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0946999999999997e-05 2024-09-13T10:12:30.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,6", 2024-09-13T10:12:30.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.357046+0000", 2024-09-13T10:12:30.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065198510000000001 2024-09-13T10:12:30.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:30.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.357616+0000", 2024-09-13T10:12:30.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00056994299999999995 2024-09-13T10:12:30.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.358085+0000", 2024-09-13T10:12:30.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046959399999999999 2024-09-13T10:12:30.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.364636+0000", 2024-09-13T10:12:30.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065509560000000001 2024-09-13T10:12:30.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:30.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.364680+0000", 2024-09-13T10:12:30.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3859999999999997e-05 2024-09-13T10:12:30.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.364718+0000", 2024-09-13T10:12:30.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.82e-05 2024-09-13T10:12:30.251 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.252 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.252 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.252 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3014 4.17 4:ea79a1ec:::smithi19530714-18:head [write 470662~772236,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T10:12:30.252 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:37.357256+0000", 2024-09-13T10:12:30.252 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.851183607000003, 2024-09-13T10:12:30.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010928939, 2024-09-13T10:12:30.252 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.252 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.252 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:30.252 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.253 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.253 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.253 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3014 2024-09-13T10:12:30.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.253 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.357256+0000", 2024-09-13T10:12:30.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.357256+0000", 2024-09-13T10:12:30.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.357259+0000", 2024-09-13T10:12:30.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7969999999999998e-06 2024-09-13T10:12:30.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.358967+0000", 2024-09-13T10:12:30.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001707568 2024-09-13T10:12:30.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.359002+0000", 2024-09-13T10:12:30.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5006000000000003e-05 2024-09-13T10:12:30.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.359019+0000", 2024-09-13T10:12:30.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7391999999999999e-05 2024-09-13T10:12:30.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.359145+0000", 2024-09-13T10:12:30.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012542200000000001 2024-09-13T10:12:30.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.359315+0000", 2024-09-13T10:12:30.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000170576 2024-09-13T10:12:30.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0,1", 2024-09-13T10:12:30.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.359578+0000", 2024-09-13T10:12:30.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026278 2024-09-13T10:12:30.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.363296+0000", 2024-09-13T10:12:30.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0037180590000000001 2024-09-13T10:12:30.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:30.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.363322+0000", 2024-09-13T10:12:30.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6268999999999999e-05 2024-09-13T10:12:30.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.367999+0000", 2024-09-13T10:12:30.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0046769910000000001 2024-09-13T10:12:30.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:30.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.368114+0000", 2024-09-13T10:12:30.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000114269 2024-09-13T10:12:30.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:37.368185+0000", 2024-09-13T10:12:30.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0809999999999995e-05 2024-09-13T10:12:30.258 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.258 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.259 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.259 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3179 4.19 4.8fd9b199 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:30.259 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.076610+0000", 2024-09-13T10:12:30.259 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.131828726000002, 2024-09-13T10:12:30.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.051258910999999997, 2024-09-13T10:12:30.259 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.259 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.259 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:12:30.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.259 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.260 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3179 2024-09-13T10:12:30.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.260 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076610+0000", 2024-09-13T10:12:30.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076610+0000", 2024-09-13T10:12:30.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076614+0000", 2024-09-13T10:12:30.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4570000000000001e-06 2024-09-13T10:12:30.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076628+0000", 2024-09-13T10:12:30.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4409e-05 2024-09-13T10:12:30.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076658+0000", 2024-09-13T10:12:30.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0049e-05 2024-09-13T10:12:30.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076681+0000", 2024-09-13T10:12:30.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2246e-05 2024-09-13T10:12:30.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077042+0000", 2024-09-13T10:12:30.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036140400000000001 2024-09-13T10:12:30.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:30.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077069+0000", 2024-09-13T10:12:30.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6846e-05 2024-09-13T10:12:30.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.127848+0000", 2024-09-13T10:12:30.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.050779040999999997 2024-09-13T10:12:30.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.127869+0000", 2024-09-13T10:12:30.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1458999999999999e-05 2024-09-13T10:12:30.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.264 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.264 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.264 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3180 4.19 4.8fd9b199 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:30.264 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.076729+0000", 2024-09-13T10:12:30.264 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.131710306999999, 2024-09-13T10:12:30.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.052035665000000002, 2024-09-13T10:12:30.264 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.264 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.264 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:12:30.264 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.264 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.265 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.265 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3180 2024-09-13T10:12:30.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.265 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076729+0000", 2024-09-13T10:12:30.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076729+0000", 2024-09-13T10:12:30.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076731+0000", 2024-09-13T10:12:30.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5969999999999999e-06 2024-09-13T10:12:30.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076739+0000", 2024-09-13T10:12:30.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7700000000000007e-06 2024-09-13T10:12:30.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076753+0000", 2024-09-13T10:12:30.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3896e-05 2024-09-13T10:12:30.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076764+0000", 2024-09-13T10:12:30.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0509999999999999e-05 2024-09-13T10:12:30.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077140+0000", 2024-09-13T10:12:30.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037611300000000002 2024-09-13T10:12:30.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:30.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077159+0000", 2024-09-13T10:12:30.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9453000000000002e-05 2024-09-13T10:12:30.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.128726+0000", 2024-09-13T10:12:30.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.051567108 2024-09-13T10:12:30.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.128765+0000", 2024-09-13T10:12:30.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8217999999999999e-05 2024-09-13T10:12:30.269 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.269 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.269 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.269 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3181 4.19 4.8fd9b199 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:30.269 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.076780+0000", 2024-09-13T10:12:30.269 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.131659069000001, 2024-09-13T10:12:30.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.052105797000000002, 2024-09-13T10:12:30.270 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.270 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.270 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:12:30.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.270 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.270 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3181 2024-09-13T10:12:30.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.270 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076780+0000", 2024-09-13T10:12:30.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076780+0000", 2024-09-13T10:12:30.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076781+0000", 2024-09-13T10:12:30.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1699999999999997e-07 2024-09-13T10:12:30.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076785+0000", 2024-09-13T10:12:30.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1899999999999997e-06 2024-09-13T10:12:30.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076797+0000", 2024-09-13T10:12:30.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1579999999999999e-05 2024-09-13T10:12:30.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.076805+0000", 2024-09-13T10:12:30.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8890000000000002e-06 2024-09-13T10:12:30.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077227+0000", 2024-09-13T10:12:30.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042249300000000002 2024-09-13T10:12:30.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:30.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.077247+0000", 2024-09-13T10:12:30.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9644999999999999e-05 2024-09-13T10:12:30.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.128862+0000", 2024-09-13T10:12:30.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.051615227 2024-09-13T10:12:30.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.128886+0000", 2024-09-13T10:12:30.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3955999999999999e-05 2024-09-13T10:12:30.274 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.274 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.274 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.274 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3179 4.19 4:998d9bf1:::smithi19530714-33:head [read 1373852~878319,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:30.274 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.100709+0000", 2024-09-13T10:12:30.274 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.107730097000001, 2024-09-13T10:12:30.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.033184096000000003, 2024-09-13T10:12:30.274 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.275 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.275 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.275 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.275 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.275 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.275 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3179 2024-09-13T10:12:30.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.275 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100709+0000", 2024-09-13T10:12:30.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100709+0000", 2024-09-13T10:12:30.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100720+0000", 2024-09-13T10:12:30.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0871e-05 2024-09-13T10:12:30.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.276 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100736+0000", 2024-09-13T10:12:30.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6504999999999999e-05 2024-09-13T10:12:30.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100827+0000", 2024-09-13T10:12:30.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0713999999999999e-05 2024-09-13T10:12:30.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100852+0000", 2024-09-13T10:12:30.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5168e-05 2024-09-13T10:12:30.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101036+0000", 2024-09-13T10:12:30.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018362900000000001 2024-09-13T10:12:30.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:30.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101065+0000", 2024-09-13T10:12:30.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9352e-05 2024-09-13T10:12:30.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129506+0000", 2024-09-13T10:12:30.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.028440811 2024-09-13T10:12:30.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129801+0000", 2024-09-13T10:12:30.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000295257 2024-09-13T10:12:30.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.133893+0000", 2024-09-13T10:12:30.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040917890000000002 2024-09-13T10:12:30.279 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.279 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.279 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.280 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3180 4.19 4:998d9bf1:::smithi19530714-33:head [read 0~2252171,checksum] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:30.280 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.100905+0000", 2024-09-13T10:12:30.280 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.107534340999997, 2024-09-13T10:12:30.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.038171497999999998, 2024-09-13T10:12:30.280 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.280 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.280 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.280 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.280 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.280 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.280 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3180 2024-09-13T10:12:30.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.280 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100905+0000", 2024-09-13T10:12:30.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100905+0000", 2024-09-13T10:12:30.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100906+0000", 2024-09-13T10:12:30.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.494e-06 2024-09-13T10:12:30.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100915+0000", 2024-09-13T10:12:30.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7630000000000006e-06 2024-09-13T10:12:30.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100934+0000", 2024-09-13T10:12:30.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8722000000000001e-05 2024-09-13T10:12:30.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100952+0000", 2024-09-13T10:12:30.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7819e-05 2024-09-13T10:12:30.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101140+0000", 2024-09-13T10:12:30.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018818900000000001 2024-09-13T10:12:30.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:30.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101160+0000", 2024-09-13T10:12:30.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0429999999999999e-05 2024-09-13T10:12:30.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.133977+0000", 2024-09-13T10:12:30.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.032816970000000001 2024-09-13T10:12:30.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.134128+0000", 2024-09-13T10:12:30.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015088100000000001 2024-09-13T10:12:30.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.139076+0000", 2024-09-13T10:12:30.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049482299999999996 2024-09-13T10:12:30.285 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.285 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.285 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.285 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3181 4.19 4:998d9bf1:::smithi19530714-33:head [read 1151249~1100922,checksum] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:30.285 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.100999+0000", 2024-09-13T10:12:30.285 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.107440601999997, 2024-09-13T10:12:30.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040765649000000001, 2024-09-13T10:12:30.285 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.286 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.286 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.286 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.286 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.286 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.286 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3181 2024-09-13T10:12:30.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.286 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100999+0000", 2024-09-13T10:12:30.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.100999+0000", 2024-09-13T10:12:30.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101000+0000", 2024-09-13T10:12:30.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3090000000000001e-06 2024-09-13T10:12:30.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101007+0000", 2024-09-13T10:12:30.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0430000000000002e-06 2024-09-13T10:12:30.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101018+0000", 2024-09-13T10:12:30.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1239e-05 2024-09-13T10:12:30.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101030+0000", 2024-09-13T10:12:30.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1986e-05 2024-09-13T10:12:30.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101229+0000", 2024-09-13T10:12:30.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000198681 2024-09-13T10:12:30.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:30.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101248+0000", 2024-09-13T10:12:30.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8766999999999998e-05 2024-09-13T10:12:30.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.139403+0000", 2024-09-13T10:12:30.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.038155715999999999 2024-09-13T10:12:30.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.139715+0000", 2024-09-13T10:12:30.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000311787 2024-09-13T10:12:30.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.141764+0000", 2024-09-13T10:12:30.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020491210000000001 2024-09-13T10:12:30.290 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.290 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.290 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.291 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3286 4.19 4:9bec18ef:::smithi19530714-47:head [sparse-read 0~1464604,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:30.291 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.129001+0000", 2024-09-13T10:12:30.291 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.079438146000001, 2024-09-13T10:12:30.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018508512000000001, 2024-09-13T10:12:30.291 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.291 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.291 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.291 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.291 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.291 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.292 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3286 2024-09-13T10:12:30.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.292 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129001+0000", 2024-09-13T10:12:30.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129001+0000", 2024-09-13T10:12:30.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129003+0000", 2024-09-13T10:12:30.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3199999999999998e-06 2024-09-13T10:12:30.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129036+0000", 2024-09-13T10:12:30.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3015000000000002e-05 2024-09-13T10:12:30.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129057+0000", 2024-09-13T10:12:30.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1078999999999999e-05 2024-09-13T10:12:30.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129071+0000", 2024-09-13T10:12:30.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3743999999999999e-05 2024-09-13T10:12:30.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.141927+0000", 2024-09-13T10:12:30.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012855729 2024-09-13T10:12:30.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.142359+0000", 2024-09-13T10:12:30.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043179600000000002 2024-09-13T10:12:30.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.147510+0000", 2024-09-13T10:12:30.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005150829 2024-09-13T10:12:30.295 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.295 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.295 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.296 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3287 4.19 4:9bec18ef:::smithi19530714-47:head [sparse-read 2111652~258168] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:30.296 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.129097+0000", 2024-09-13T10:12:30.296 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.079341909, 2024-09-13T10:12:30.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018987459000000002, 2024-09-13T10:12:30.296 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.296 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.296 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.296 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.297 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.297 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3287 2024-09-13T10:12:30.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.297 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129097+0000", 2024-09-13T10:12:30.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129097+0000", 2024-09-13T10:12:30.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129098+0000", 2024-09-13T10:12:30.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6299999999999993e-07 2024-09-13T10:12:30.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129103+0000", 2024-09-13T10:12:30.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.763e-06 2024-09-13T10:12:30.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129113+0000", 2024-09-13T10:12:30.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8230000000000006e-06 2024-09-13T10:12:30.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129121+0000", 2024-09-13T10:12:30.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1459999999999996e-06 2024-09-13T10:12:30.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.147587+0000", 2024-09-13T10:12:30.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018466104000000001 2024-09-13T10:12:30.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.147732+0000", 2024-09-13T10:12:30.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014455499999999999 2024-09-13T10:12:30.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.148085+0000", 2024-09-13T10:12:30.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035310499999999999 2024-09-13T10:12:30.300 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.300 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.300 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.301 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3288 4.19 4:9bec18ef:::smithi19530714-47:head [sparse-read 2190457~179363] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:30.301 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.129133+0000", 2024-09-13T10:12:30.301 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.079305947000002, 2024-09-13T10:12:30.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019922197999999999, 2024-09-13T10:12:30.301 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.301 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.301 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.301 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.301 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3288 2024-09-13T10:12:30.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.301 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129133+0000", 2024-09-13T10:12:30.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129133+0000", 2024-09-13T10:12:30.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129134+0000", 2024-09-13T10:12:30.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3e-07 2024-09-13T10:12:30.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129137+0000", 2024-09-13T10:12:30.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1930000000000002e-06 2024-09-13T10:12:30.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129146+0000", 2024-09-13T10:12:30.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7790000000000001e-06 2024-09-13T10:12:30.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.129154+0000", 2024-09-13T10:12:30.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6539999999999999e-06 2024-09-13T10:12:30.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.148377+0000", 2024-09-13T10:12:30.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019222965000000002 2024-09-13T10:12:30.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.148497+0000", 2024-09-13T10:12:30.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012054500000000001 2024-09-13T10:12:30.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.149055+0000", 2024-09-13T10:12:30.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00055833200000000003 2024-09-13T10:12:30.305 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.305 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.305 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.305 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4149 4.17 4:e81d9ad0:::smithi19530714-4:head [write 702625~719672,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T10:12:30.305 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.147244+0000", 2024-09-13T10:12:30.305 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.061194794999999, 2024-09-13T10:12:30.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011913730000000001, 2024-09-13T10:12:30.305 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.305 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.306 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:30.306 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.306 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.306 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.306 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4149 2024-09-13T10:12:30.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.306 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.147244+0000", 2024-09-13T10:12:30.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.147244+0000", 2024-09-13T10:12:30.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.147248+0000", 2024-09-13T10:12:30.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3100000000000001e-06 2024-09-13T10:12:30.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.148398+0000", 2024-09-13T10:12:30.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001150353 2024-09-13T10:12:30.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.148430+0000", 2024-09-13T10:12:30.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2122999999999997e-05 2024-09-13T10:12:30.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.148447+0000", 2024-09-13T10:12:30.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7283e-05 2024-09-13T10:12:30.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.148684+0000", 2024-09-13T10:12:30.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000236741 2024-09-13T10:12:30.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.148954+0000", 2024-09-13T10:12:30.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027012800000000001 2024-09-13T10:12:30.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 0,1", 2024-09-13T10:12:30.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.149429+0000", 2024-09-13T10:12:30.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047418699999999999 2024-09-13T10:12:30.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.158382+0000", 2024-09-13T10:12:30.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0089531109999999997 2024-09-13T10:12:30.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.158472+0000", 2024-09-13T10:12:30.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9847999999999998e-05 2024-09-13T10:12:30.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:30.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.158671+0000", 2024-09-13T10:12:30.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019931899999999999 2024-09-13T10:12:30.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:30.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.158967+0000", 2024-09-13T10:12:30.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000296421 2024-09-13T10:12:30.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.159158+0000", 2024-09-13T10:12:30.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019090599999999999 2024-09-13T10:12:30.312 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.312 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.312 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.312 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4196 4.17 4:e81d9ad0:::smithi19530714-4:head [read 0~2302625,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:30.312 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.706619+0000", 2024-09-13T10:12:30.312 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 24.501819841, 2024-09-13T10:12:30.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014113064, 2024-09-13T10:12:30.313 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.313 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.313 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.313 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.313 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.313 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.313 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4196 2024-09-13T10:12:30.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.313 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.706619+0000", 2024-09-13T10:12:30.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.706619+0000", 2024-09-13T10:12:30.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.706621+0000", 2024-09-13T10:12:30.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9199999999999998e-06 2024-09-13T10:12:30.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.706631+0000", 2024-09-13T10:12:30.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2329999999999995e-06 2024-09-13T10:12:30.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.706648+0000", 2024-09-13T10:12:30.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7269999999999999e-05 2024-09-13T10:12:30.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.706660+0000", 2024-09-13T10:12:30.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1996e-05 2024-09-13T10:12:30.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.717127+0000", 2024-09-13T10:12:30.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01046741 2024-09-13T10:12:30.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.717405+0000", 2024-09-13T10:12:30.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027790399999999999 2024-09-13T10:12:30.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.720732+0000", 2024-09-13T10:12:30.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033273310000000002 2024-09-13T10:12:30.317 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.317 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.317 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.317 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4563 4.0 4:03fab808:::smithi19530714-40:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e106)", 2024-09-13T10:12:30.318 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:14.723964+0000", 2024-09-13T10:12:30.318 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.484475269000001, 2024-09-13T10:12:30.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010965396000000001, 2024-09-13T10:12:30.318 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.318 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.318 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.318 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.318 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4563 2024-09-13T10:12:30.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.318 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.723964+0000", 2024-09-13T10:12:30.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.723964+0000", 2024-09-13T10:12:30.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.723966+0000", 2024-09-13T10:12:30.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9130000000000001e-06 2024-09-13T10:12:30.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.723972+0000", 2024-09-13T10:12:30.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4230000000000002e-06 2024-09-13T10:12:30.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.723990+0000", 2024-09-13T10:12:30.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7594000000000001e-05 2024-09-13T10:12:30.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.724005+0000", 2024-09-13T10:12:30.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5206e-05 2024-09-13T10:12:30.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.730549+0000", 2024-09-13T10:12:30.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0065439039999999997 2024-09-13T10:12:30.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:30.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.730768+0000", 2024-09-13T10:12:30.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000218896 2024-09-13T10:12:30.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.734442+0000", 2024-09-13T10:12:30.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036736249999999998 2024-09-13T10:12:30.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.734652+0000", 2024-09-13T10:12:30.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002108 2024-09-13T10:12:30.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.734929+0000", 2024-09-13T10:12:30.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027703500000000002 2024-09-13T10:12:30.322 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.322 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.323 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.323 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4580 4.17 4:e8c2cbec:::smithi19530714-7:head [sparse-read 0~1630208] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107)", 2024-09-13T10:12:30.323 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:15.373998+0000", 2024-09-13T10:12:30.323 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.834440895, 2024-09-13T10:12:30.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011854452999999999, 2024-09-13T10:12:30.323 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.323 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.323 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.323 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.323 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.323 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.323 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4580 2024-09-13T10:12:30.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.324 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:15.373998+0000", 2024-09-13T10:12:30.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:15.373998+0000", 2024-09-13T10:12:30.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:15.374001+0000", 2024-09-13T10:12:30.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2740000000000002e-06 2024-09-13T10:12:30.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:15.374013+0000", 2024-09-13T10:12:30.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2653000000000001e-05 2024-09-13T10:12:30.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:15.374052+0000", 2024-09-13T10:12:30.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8646000000000002e-05 2024-09-13T10:12:30.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:15.374076+0000", 2024-09-13T10:12:30.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3618999999999999e-05 2024-09-13T10:12:30.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:15.384301+0000", 2024-09-13T10:12:30.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010225303 2024-09-13T10:12:30.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:15.384551+0000", 2024-09-13T10:12:30.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025006799999999999 2024-09-13T10:12:30.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:15.385853+0000", 2024-09-13T10:12:30.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00130189 2024-09-13T10:12:30.327 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.327 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.327 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.327 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.327 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.327 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:30.329 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-09-13T10:12:30.329 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:30.325+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:30.430 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-09-13T10:12:30.426+0000 7fbdf5798640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 26451) UID: 0 2024-09-13T10:12:30.456 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:30.456 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:12:30.456 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:12:30.456 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:30.470 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-09-13T10:12:30.497 INFO:tasks.rados.rados.0.smithi195.stdout:2044: finishing write tid 1 to smithi19530714-39 2024-09-13T10:12:30.498 INFO:tasks.rados.rados.0.smithi195.stdout:2044: finishing write tid 2 to smithi19530714-39 2024-09-13T10:12:30.499 INFO:tasks.rados.rados.0.smithi195.stdout:2044: finishing write tid 3 to smithi19530714-39 2024-09-13T10:12:30.504 INFO:tasks.rados.rados.0.smithi195.stdout:2044: finishing write tid 4 to smithi19530714-39 2024-09-13T10:12:30.505 INFO:tasks.rados.rados.0.smithi195.stdout:2044: finishing write tid 5 to smithi19530714-39 2024-09-13T10:12:30.506 INFO:tasks.rados.rados.0.smithi195.stdout:2044: finishing write tid 6 to smithi19530714-39 2024-09-13T10:12:30.506 INFO:tasks.rados.rados.0.smithi195.stdout:2044: oid 39 updating version 0 to 53 2024-09-13T10:12:30.506 INFO:tasks.rados.rados.0.smithi195.stdout:2044: oid 39 version 53 is already newer than 52 2024-09-13T10:12:30.506 INFO:tasks.rados.rados.0.smithi195.stdout:2044: oid 39 version 53 is already newer than 51 2024-09-13T10:12:30.507 INFO:tasks.rados.rados.0.smithi195.stdout:2044: oid 39 updating version 53 to 54 2024-09-13T10:12:30.507 INFO:tasks.rados.rados.0.smithi195.stdout:2044: oid 39 version 54 is already newer than 50 2024-09-13T10:12:30.507 INFO:tasks.rados.rados.0.smithi195.stdout:update_object_version oid 39 v 54 (ObjNum 464 snap 0 seq_num 464) dirty exists 2024-09-13T10:12:30.507 INFO:tasks.rados.rados.0.smithi195.stdout:2044: left oid 39 (ObjNum 464 snap 0 seq_num 464) 2024-09-13T10:12:30.507 INFO:tasks.rados.rados.0.smithi195.stdout:2044: done (0 left) 2024-09-13T10:12:30.530 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:30.525+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:30.594 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:30.594 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:12:30.594 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:12:30.595 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:12:30.595 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:30.609 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-09-13T10:12:30.631 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:30.625+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:30.726 INFO:tasks.rados.rados.0.smithi195.stderr:0 errors. 2024-09-13T10:12:30.726 INFO:tasks.rados.rados.0.smithi195.stderr: 2024-09-13T10:12:30.732 INFO:tasks.ceph.osd.6.smithi181.stderr:2024-09-13T10:12:30.725+0000 7ff212ba5640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 26306) UID: 0 2024-09-13T10:12:30.735 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T10:12:30.736 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:30.736 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:12:30.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:12:30.736 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:12:30.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.736 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1600 4.10 4.31420bd0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:12:30.737 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:05.990716+0000", 2024-09-13T10:12:30.737 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 84.742202762999995, 2024-09-13T10:12:30.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.412517299, 2024-09-13T10:12:30.737 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.737 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.737 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:12:30.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.737 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.737 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1600 2024-09-13T10:12:30.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.737 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.990716+0000", 2024-09-13T10:12:30.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.990716+0000", 2024-09-13T10:12:30.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.990720+0000", 2024-09-13T10:12:30.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9689999999999996e-06 2024-09-13T10:12:30.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991825+0000", 2024-09-13T10:12:30.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001104764 2024-09-13T10:12:30.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991856+0000", 2024-09-13T10:12:30.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1473000000000002e-05 2024-09-13T10:12:30.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991872+0000", 2024-09-13T10:12:30.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6212999999999999e-05 2024-09-13T10:12:30.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991974+0000", 2024-09-13T10:12:30.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010160799999999999 2024-09-13T10:12:30.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:30.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:05.991988+0000", 2024-09-13T10:12:30.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4106e-05 2024-09-13T10:12:30.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403179+0000", 2024-09-13T10:12:30.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41119127100000002 2024-09-13T10:12:30.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403233+0000", 2024-09-13T10:12:30.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3894999999999998e-05 2024-09-13T10:12:30.741 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.741 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.742 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1601 4.10 4.31420bd0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:12:30.742 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.008020+0000", 2024-09-13T10:12:30.742 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 84.724898586999998, 2024-09-13T10:12:30.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39533983, 2024-09-13T10:12:30.742 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.742 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.742 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:12:30.742 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.742 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.742 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.742 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1601 2024-09-13T10:12:30.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.743 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.008020+0000", 2024-09-13T10:12:30.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.008020+0000", 2024-09-13T10:12:30.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.008023+0000", 2024-09-13T10:12:30.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.807e-06 2024-09-13T10:12:30.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009268+0000", 2024-09-13T10:12:30.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001245257 2024-09-13T10:12:30.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009298+0000", 2024-09-13T10:12:30.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0171999999999998e-05 2024-09-13T10:12:30.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009313+0000", 2024-09-13T10:12:30.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.471e-05 2024-09-13T10:12:30.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009469+0000", 2024-09-13T10:12:30.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015595600000000001 2024-09-13T10:12:30.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:30.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009481+0000", 2024-09-13T10:12:30.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2454e-05 2024-09-13T10:12:30.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403334+0000", 2024-09-13T10:12:30.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39385227499999997 2024-09-13T10:12:30.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403360+0000", 2024-09-13T10:12:30.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6199000000000001e-05 2024-09-13T10:12:30.746 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.746 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.746 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.747 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1602 4.10 4.31420bd0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:12:30.747 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.009344+0000", 2024-09-13T10:12:30.747 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 84.723574755000001, 2024-09-13T10:12:30.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39420750999999998, 2024-09-13T10:12:30.747 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.747 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.747 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:12:30.747 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.747 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.747 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.747 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1602 2024-09-13T10:12:30.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.747 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009344+0000", 2024-09-13T10:12:30.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009344+0000", 2024-09-13T10:12:30.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009345+0000", 2024-09-13T10:12:30.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1930000000000001e-06 2024-09-13T10:12:30.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009351+0000", 2024-09-13T10:12:30.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1399999999999997e-06 2024-09-13T10:12:30.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009361+0000", 2024-09-13T10:12:30.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9569999999999997e-06 2024-09-13T10:12:30.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009370+0000", 2024-09-13T10:12:30.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.619e-06 2024-09-13T10:12:30.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009570+0000", 2024-09-13T10:12:30.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019990599999999999 2024-09-13T10:12:30.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:30.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009583+0000", 2024-09-13T10:12:30.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2886e-05 2024-09-13T10:12:30.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403522+0000", 2024-09-13T10:12:30.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39393976000000003 2024-09-13T10:12:30.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403551+0000", 2024-09-13T10:12:30.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9048999999999999e-05 2024-09-13T10:12:30.751 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.751 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.751 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.752 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1603 4.10 4.31420bd0 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e48)", 2024-09-13T10:12:30.752 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:06.009383+0000", 2024-09-13T10:12:30.752 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 84.723535862999995, 2024-09-13T10:12:30.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39425412900000001, 2024-09-13T10:12:30.752 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.752 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.752 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:12:30.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.752 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.752 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1603 2024-09-13T10:12:30.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.753 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009383+0000", 2024-09-13T10:12:30.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009383+0000", 2024-09-13T10:12:30.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009384+0000", 2024-09-13T10:12:30.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8700000000000005e-07 2024-09-13T10:12:30.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009389+0000", 2024-09-13T10:12:30.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1669999999999998e-06 2024-09-13T10:12:30.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009398+0000", 2024-09-13T10:12:30.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6260000000000007e-06 2024-09-13T10:12:30.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009407+0000", 2024-09-13T10:12:30.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1960000000000008e-06 2024-09-13T10:12:30.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009623+0000", 2024-09-13T10:12:30.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000216842 2024-09-13T10:12:30.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:30.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.009632+0000", 2024-09-13T10:12:30.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3529999999999995e-06 2024-09-13T10:12:30.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403609+0000", 2024-09-13T10:12:30.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39397721600000002 2024-09-13T10:12:30.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:06.403637+0000", 2024-09-13T10:12:30.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7942000000000001e-05 2024-09-13T10:12:30.756 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.756 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.756 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.757 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3264 4.10 4:0b4c105e:::smithi19530714-16:head [read 264340~2357585,checksum,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:30.757 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.083141+0000", 2024-09-13T10:12:30.757 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.649778097000002, 2024-09-13T10:12:30.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0093235160000000004, 2024-09-13T10:12:30.757 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.757 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.757 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.757 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.757 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.757 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.757 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3264 2024-09-13T10:12:30.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.758 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083141+0000", 2024-09-13T10:12:30.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083141+0000", 2024-09-13T10:12:30.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083144+0000", 2024-09-13T10:12:30.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1269999999999999e-06 2024-09-13T10:12:30.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083156+0000", 2024-09-13T10:12:30.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1956e-05 2024-09-13T10:12:30.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083181+0000", 2024-09-13T10:12:30.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4856000000000001e-05 2024-09-13T10:12:30.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083196+0000", 2024-09-13T10:12:30.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5593000000000001e-05 2024-09-13T10:12:30.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083340+0000", 2024-09-13T10:12:30.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014363200000000001 2024-09-13T10:12:30.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083595+0000", 2024-09-13T10:12:30.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025489700000000002 2024-09-13T10:12:30.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.092464+0000", 2024-09-13T10:12:30.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008869455 2024-09-13T10:12:30.761 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.761 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.761 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.761 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3265 4.10 4:0b4c105e:::smithi19530714-16:head [read 0~2621925,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:30.761 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.083253+0000", 2024-09-13T10:12:30.761 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.649665626999997, 2024-09-13T10:12:30.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01421477, 2024-09-13T10:12:30.762 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.762 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.762 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.762 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3265 2024-09-13T10:12:30.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.762 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083253+0000", 2024-09-13T10:12:30.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083253+0000", 2024-09-13T10:12:30.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083255+0000", 2024-09-13T10:12:30.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.717e-06 2024-09-13T10:12:30.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083264+0000", 2024-09-13T10:12:30.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3989999999999993e-06 2024-09-13T10:12:30.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083281+0000", 2024-09-13T10:12:30.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6833e-05 2024-09-13T10:12:30.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083300+0000", 2024-09-13T10:12:30.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8972000000000001e-05 2024-09-13T10:12:30.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.091405+0000", 2024-09-13T10:12:30.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0081045509999999998 2024-09-13T10:12:30.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.091609+0000", 2024-09-13T10:12:30.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020448299999999999 2024-09-13T10:12:30.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.097468+0000", 2024-09-13T10:12:30.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058588149999999999 2024-09-13T10:12:30.765 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.766 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.766 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.766 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3266 4.10 4:0b4c105e:::smithi19530714-16:head [sparse-read 0~1476046] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-13T10:12:30.766 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.083328+0000", 2024-09-13T10:12:30.766 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.649590953000001, 2024-09-13T10:12:30.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015205013, 2024-09-13T10:12:30.766 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.766 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.766 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.766 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.766 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.766 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.766 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3266 2024-09-13T10:12:30.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.767 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083328+0000", 2024-09-13T10:12:30.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083328+0000", 2024-09-13T10:12:30.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083329+0000", 2024-09-13T10:12:30.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.313e-06 2024-09-13T10:12:30.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083336+0000", 2024-09-13T10:12:30.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.153e-06 2024-09-13T10:12:30.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083390+0000", 2024-09-13T10:12:30.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3662000000000002e-05 2024-09-13T10:12:30.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.083405+0000", 2024-09-13T10:12:30.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5163e-05 2024-09-13T10:12:30.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.097580+0000", 2024-09-13T10:12:30.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014174755000000001 2024-09-13T10:12:30.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.097822+0000", 2024-09-13T10:12:30.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024186499999999999 2024-09-13T10:12:30.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.098533+0000", 2024-09-13T10:12:30.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00071110200000000002 2024-09-13T10:12:30.770 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.770 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.770 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.770 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:3234 4.12 e81/80 4:4ac406d9:::smithi19530714-26:head v 81'40, mlcod=0'0)", 2024-09-13T10:12:30.771 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.085189+0000", 2024-09-13T10:12:30.771 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.647729593000001, 2024-09-13T10:12:30.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017250062, 2024-09-13T10:12:30.771 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.771 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.771 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:30.771 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.085189+0000", 2024-09-13T10:12:30.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.085189+0000", 2024-09-13T10:12:30.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.085191+0000", 2024-09-13T10:12:30.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0829999999999998e-06 2024-09-13T10:12:30.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.087261+0000", 2024-09-13T10:12:30.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002069417 2024-09-13T10:12:30.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.087291+0000", 2024-09-13T10:12:30.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0222e-05 2024-09-13T10:12:30.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.087317+0000", 2024-09-13T10:12:30.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6003000000000002e-05 2024-09-13T10:12:30.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.087564+0000", 2024-09-13T10:12:30.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024710800000000001 2024-09-13T10:12:30.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.087662+0000", 2024-09-13T10:12:30.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8153999999999996e-05 2024-09-13T10:12:30.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:30.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.101938+0000", 2024-09-13T10:12:30.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.014275738 2024-09-13T10:12:30.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.102439+0000", 2024-09-13T10:12:30.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050133700000000003 2024-09-13T10:12:30.775 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.775 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.775 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.775 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:3235 4.12 e81/80 4:4ac406d9:::smithi19530714-26:head v 81'41, mlcod=0'0)", 2024-09-13T10:12:30.775 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.091873+0000", 2024-09-13T10:12:30.775 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.641045794999997, 2024-09-13T10:12:30.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011472080000000001, 2024-09-13T10:12:30.775 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.775 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.775 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:30.775 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.091873+0000", 2024-09-13T10:12:30.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.091873+0000", 2024-09-13T10:12:30.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.091875+0000", 2024-09-13T10:12:30.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2199999999999999e-06 2024-09-13T10:12:30.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.092906+0000", 2024-09-13T10:12:30.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001031105 2024-09-13T10:12:30.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.092937+0000", 2024-09-13T10:12:30.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0326000000000001e-05 2024-09-13T10:12:30.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.092957+0000", 2024-09-13T10:12:30.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0225999999999999e-05 2024-09-13T10:12:30.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.099823+0000", 2024-09-13T10:12:30.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0068664319999999996 2024-09-13T10:12:30.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.099873+0000", 2024-09-13T10:12:30.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9838e-05 2024-09-13T10:12:30.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:30.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.103240+0000", 2024-09-13T10:12:30.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003367395 2024-09-13T10:12:30.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.103345+0000", 2024-09-13T10:12:30.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010453799999999999 2024-09-13T10:12:30.779 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.779 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.779 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.779 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4607.0:3236 4.12 e81/80 4:4ac406d9:::smithi19530714-26:head v 81'42, mlcod=0'0)", 2024-09-13T10:12:30.779 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:43.096873+0000", 2024-09-13T10:12:30.779 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 47.636046043, 2024-09-13T10:12:30.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072886340000000004, 2024-09-13T10:12:30.780 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.780 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.780 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:30.780 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096873+0000", 2024-09-13T10:12:30.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096873+0000", 2024-09-13T10:12:30.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096875+0000", 2024-09-13T10:12:30.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.193e-06 2024-09-13T10:12:30.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096890+0000", 2024-09-13T10:12:30.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5243e-05 2024-09-13T10:12:30.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096918+0000", 2024-09-13T10:12:30.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7985999999999999e-05 2024-09-13T10:12:30.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.096943+0000", 2024-09-13T10:12:30.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4549000000000002e-05 2024-09-13T10:12:30.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.102522+0000", 2024-09-13T10:12:30.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0055797579999999998 2024-09-13T10:12:30.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.102586+0000", 2024-09-13T10:12:30.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3748e-05 2024-09-13T10:12:30.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:30.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.104035+0000", 2024-09-13T10:12:30.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001448438 2024-09-13T10:12:30.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:43.104161+0000", 2024-09-13T10:12:30.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012671899999999999 2024-09-13T10:12:30.783 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.784 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.784 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.784 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:3732 4.10 4:0b4c105e:::smithi19530714-16:head [assert-version v55,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e90)", 2024-09-13T10:12:30.784 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:53.879799+0000", 2024-09-13T10:12:30.784 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.853119665000001, 2024-09-13T10:12:30.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078265209999999995, 2024-09-13T10:12:30.784 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.784 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.784 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:30.784 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.784 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.784 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.785 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 3732 2024-09-13T10:12:30.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.785 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.879799+0000", 2024-09-13T10:12:30.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.879799+0000", 2024-09-13T10:12:30.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.879803+0000", 2024-09-13T10:12:30.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6799999999999999e-06 2024-09-13T10:12:30.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.879821+0000", 2024-09-13T10:12:30.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8559e-05 2024-09-13T10:12:30.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.879858+0000", 2024-09-13T10:12:30.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6294999999999997e-05 2024-09-13T10:12:30.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.879875+0000", 2024-09-13T10:12:30.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7399999999999999e-05 2024-09-13T10:12:30.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.880046+0000", 2024-09-13T10:12:30.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017084899999999999 2024-09-13T10:12:30.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.880308+0000", 2024-09-13T10:12:30.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026218299999999998 2024-09-13T10:12:30.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3,14", 2024-09-13T10:12:30.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.885684+0000", 2024-09-13T10:12:30.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0053760800000000001 2024-09-13T10:12:30.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:30.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.886927+0000", 2024-09-13T10:12:30.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00124244 2024-09-13T10:12:30.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.887269+0000", 2024-09-13T10:12:30.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034287800000000002 2024-09-13T10:12:30.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.887471+0000", 2024-09-13T10:12:30.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000201864 2024-09-13T10:12:30.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:30.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.887556+0000", 2024-09-13T10:12:30.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4777999999999994e-05 2024-09-13T10:12:30.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:53.887626+0000", 2024-09-13T10:12:30.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9515000000000001e-05 2024-09-13T10:12:30.790 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.790 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.790 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.790 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4114 4.10 4:0b4c105e:::smithi19530714-16:head [write 652204~777572] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T10:12:30.790 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:03.951781+0000", 2024-09-13T10:12:30.790 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.781137301000001, 2024-09-13T10:12:30.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.01227927, 2024-09-13T10:12:30.790 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.791 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.791 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:30.791 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.791 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.791 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.791 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4114 2024-09-13T10:12:30.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.791 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.951781+0000", 2024-09-13T10:12:30.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.951781+0000", 2024-09-13T10:12:30.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.951785+0000", 2024-09-13T10:12:30.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2030000000000001e-06 2024-09-13T10:12:30.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.953481+0000", 2024-09-13T10:12:30.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001696173 2024-09-13T10:12:30.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.953515+0000", 2024-09-13T10:12:30.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3929e-05 2024-09-13T10:12:30.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.953532+0000", 2024-09-13T10:12:30.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7224e-05 2024-09-13T10:12:30.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.953672+0000", 2024-09-13T10:12:30.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014041300000000001 2024-09-13T10:12:30.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-13T10:12:30.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.953684+0000", 2024-09-13T10:12:30.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1337e-05 2024-09-13T10:12:30.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.956654+0000", 2024-09-13T10:12:30.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029703249999999998 2024-09-13T10:12:30.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.956845+0000", 2024-09-13T10:12:30.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000190776 2024-09-13T10:12:30.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3,14", 2024-09-13T10:12:30.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.957082+0000", 2024-09-13T10:12:30.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023739799999999999 2024-09-13T10:12:30.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:30.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.963207+0000", 2024-09-13T10:12:30.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0061252269999999996 2024-09-13T10:12:30.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.963683+0000", 2024-09-13T10:12:30.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00047594599999999999 2024-09-13T10:12:30.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.963783+0000", 2024-09-13T10:12:30.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9281e-05 2024-09-13T10:12:30.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:30.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.963989+0000", 2024-09-13T10:12:30.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020614200000000001 2024-09-13T10:12:30.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964061+0000", 2024-09-13T10:12:30.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1896000000000004e-05 2024-09-13T10:12:30.797 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.797 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.797 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.797 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4115 4.10 4:0b4c105e:::smithi19530714-16:head [write 1952125~300079] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T10:12:30.797 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:03.965581+0000", 2024-09-13T10:12:30.797 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.767338021, 2024-09-13T10:12:30.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0077346350000000001, 2024-09-13T10:12:30.797 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.797 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.798 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:30.798 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.798 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.798 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.798 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4115 2024-09-13T10:12:30.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.798 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.965581+0000", 2024-09-13T10:12:30.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.965581+0000", 2024-09-13T10:12:30.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.965584+0000", 2024-09-13T10:12:30.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0369999999999999e-06 2024-09-13T10:12:30.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966073+0000", 2024-09-13T10:12:30.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00048940500000000005 2024-09-13T10:12:30.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966108+0000", 2024-09-13T10:12:30.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4396000000000001e-05 2024-09-13T10:12:30.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966129+0000", 2024-09-13T10:12:30.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1263e-05 2024-09-13T10:12:30.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966282+0000", 2024-09-13T10:12:30.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015279000000000001 2024-09-13T10:12:30.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966498+0000", 2024-09-13T10:12:30.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021653199999999999 2024-09-13T10:12:30.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3,14", 2024-09-13T10:12:30.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966820+0000", 2024-09-13T10:12:30.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032185299999999998 2024-09-13T10:12:30.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:30.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.970736+0000", 2024-09-13T10:12:30.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039162950000000002 2024-09-13T10:12:30.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.970925+0000", 2024-09-13T10:12:30.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018865200000000001 2024-09-13T10:12:30.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.973125+0000", 2024-09-13T10:12:30.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021998859999999999 2024-09-13T10:12:30.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:30.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.973227+0000", 2024-09-13T10:12:30.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101912 2024-09-13T10:12:30.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.973315+0000", 2024-09-13T10:12:30.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8614000000000002e-05 2024-09-13T10:12:30.803 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.803 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.803 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.803 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4116 4.10 4:0b4c105e:::smithi19530714-16:head [setxattr _header (56) in=7b,truncate 2252204] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-13T10:12:30.803 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:03.966168+0000", 2024-09-13T10:12:30.803 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.766750231, 2024-09-13T10:12:30.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083925149999999997, 2024-09-13T10:12:30.803 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.803 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.803 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:30.804 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.804 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.804 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.804 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4116 2024-09-13T10:12:30.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.804 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966168+0000", 2024-09-13T10:12:30.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966168+0000", 2024-09-13T10:12:30.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966170+0000", 2024-09-13T10:12:30.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.544e-06 2024-09-13T10:12:30.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966177+0000", 2024-09-13T10:12:30.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6699999999999997e-06 2024-09-13T10:12:30.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966195+0000", 2024-09-13T10:12:30.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8529000000000002e-05 2024-09-13T10:12:30.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966205+0000", 2024-09-13T10:12:30.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9499999999999996e-06 2024-09-13T10:12:30.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.968990+0000", 2024-09-13T10:12:30.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027848059999999999 2024-09-13T10:12:30.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.969173+0000", 2024-09-13T10:12:30.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018300599999999999 2024-09-13T10:12:30.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3,14", 2024-09-13T10:12:30.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.969399+0000", 2024-09-13T10:12:30.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022624900000000001 2024-09-13T10:12:30.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:30.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.972501+0000", 2024-09-13T10:12:30.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031013619999999999 2024-09-13T10:12:30.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.973436+0000", 2024-09-13T10:12:30.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00093499299999999996 2024-09-13T10:12:30.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.974390+0000", 2024-09-13T10:12:30.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00095395699999999996 2024-09-13T10:12:30.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:30.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.974467+0000", 2024-09-13T10:12:30.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7594000000000003e-05 2024-09-13T10:12:30.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.974561+0000", 2024-09-13T10:12:30.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3855000000000002e-05 2024-09-13T10:12:30.809 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.809 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.809 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.809 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4117 4.10 4:0b4c105e:::smithi19530714-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e97)", 2024-09-13T10:12:30.809 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:03.966260+0000", 2024-09-13T10:12:30.810 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.766659159, 2024-09-13T10:12:30.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.008682492, 2024-09-13T10:12:30.810 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.810 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.810 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.810 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.810 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4117 2024-09-13T10:12:30.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.810 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966260+0000", 2024-09-13T10:12:30.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966260+0000", 2024-09-13T10:12:30.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966260+0000", 2024-09-13T10:12:30.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9599999999999998e-07 2024-09-13T10:12:30.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966265+0000", 2024-09-13T10:12:30.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8169999999999998e-06 2024-09-13T10:12:30.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966277+0000", 2024-09-13T10:12:30.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1523e-05 2024-09-13T10:12:30.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.966286+0000", 2024-09-13T10:12:30.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9700000000000005e-06 2024-09-13T10:12:30.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.970534+0000", 2024-09-13T10:12:30.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042480770000000003 2024-09-13T10:12:30.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-09-13T10:12:30.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.970706+0000", 2024-09-13T10:12:30.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000172456 2024-09-13T10:12:30.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.974672+0000", 2024-09-13T10:12:30.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039656650000000002 2024-09-13T10:12:30.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.974791+0000", 2024-09-13T10:12:30.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118714 2024-09-13T10:12:30.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.974942+0000", 2024-09-13T10:12:30.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015137399999999999 2024-09-13T10:12:30.814 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.814 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.814 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.815 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4156 4.10 4:0b4c105e:::smithi19530714-16:head [sparse-read 0~1588652,omap-get-vals-by-keys,omap-get-keys,omap-get-vals,omap-get-header,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:30.815 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.151897+0000", 2024-09-13T10:12:30.815 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.581021630999999, 2024-09-13T10:12:30.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0082248189999999995, 2024-09-13T10:12:30.815 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.815 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.815 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.815 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.815 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4156 2024-09-13T10:12:30.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.815 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.151897+0000", 2024-09-13T10:12:30.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.151897+0000", 2024-09-13T10:12:30.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.151901+0000", 2024-09-13T10:12:30.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3699999999999997e-06 2024-09-13T10:12:30.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.151923+0000", 2024-09-13T10:12:30.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1691999999999998e-05 2024-09-13T10:12:30.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.151967+0000", 2024-09-13T10:12:30.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4279000000000001e-05 2024-09-13T10:12:30.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.151992+0000", 2024-09-13T10:12:30.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4267e-05 2024-09-13T10:12:30.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152163+0000", 2024-09-13T10:12:30.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017141300000000001 2024-09-13T10:12:30.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-13T10:12:30.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152180+0000", 2024-09-13T10:12:30.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7376000000000001e-05 2024-09-13T10:12:30.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.156082+0000", 2024-09-13T10:12:30.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0039017190000000001 2024-09-13T10:12:30.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.156298+0000", 2024-09-13T10:12:30.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021607199999999999 2024-09-13T10:12:30.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.160122+0000", 2024-09-13T10:12:30.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038236310000000001 2024-09-13T10:12:30.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.819 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.820 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.820 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4157 4.10 4:0b4c105e:::smithi19530714-16:head [read 0~2252204,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:30.820 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.152052+0000", 2024-09-13T10:12:30.820 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.580866381, 2024-09-13T10:12:30.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010542621, 2024-09-13T10:12:30.820 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.820 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.820 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.820 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.820 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.821 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.821 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4157 2024-09-13T10:12:30.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.821 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152052+0000", 2024-09-13T10:12:30.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152052+0000", 2024-09-13T10:12:30.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152054+0000", 2024-09-13T10:12:30.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7230000000000001e-06 2024-09-13T10:12:30.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152061+0000", 2024-09-13T10:12:30.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1359999999999999e-06 2024-09-13T10:12:30.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152082+0000", 2024-09-13T10:12:30.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0339999999999998e-05 2024-09-13T10:12:30.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152098+0000", 2024-09-13T10:12:30.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6952999999999998e-05 2024-09-13T10:12:30.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152367+0000", 2024-09-13T10:12:30.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026855800000000001 2024-09-13T10:12:30.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-13T10:12:30.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152382+0000", 2024-09-13T10:12:30.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5393e-05 2024-09-13T10:12:30.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.160202+0000", 2024-09-13T10:12:30.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0078198550000000006 2024-09-13T10:12:30.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.160407+0000", 2024-09-13T10:12:30.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000204248 2024-09-13T10:12:30.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.162595+0000", 2024-09-13T10:12:30.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002188415 2024-09-13T10:12:30.825 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.825 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.825 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.825 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4158 4.10 4:0b4c105e:::smithi19530714-16:head [sparse-read 965971~1163353] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:30.826 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.152124+0000", 2024-09-13T10:12:30.826 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.580794742999998, 2024-09-13T10:12:30.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011258710999999999, 2024-09-13T10:12:30.826 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.826 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.826 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:30.826 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.826 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.826 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.826 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4158 2024-09-13T10:12:30.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.826 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152124+0000", 2024-09-13T10:12:30.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152124+0000", 2024-09-13T10:12:30.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152125+0000", 2024-09-13T10:12:30.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2470000000000001e-06 2024-09-13T10:12:30.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152133+0000", 2024-09-13T10:12:30.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3170000000000004e-06 2024-09-13T10:12:30.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152149+0000", 2024-09-13T10:12:30.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6526000000000001e-05 2024-09-13T10:12:30.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152165+0000", 2024-09-13T10:12:30.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5569e-05 2024-09-13T10:12:30.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152495+0000", 2024-09-13T10:12:30.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032994300000000002 2024-09-13T10:12:30.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-09-13T10:12:30.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.152511+0000", 2024-09-13T10:12:30.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6160000000000001e-05 2024-09-13T10:12:30.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.162726+0000", 2024-09-13T10:12:30.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010215307999999999 2024-09-13T10:12:30.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.162896+0000", 2024-09-13T10:12:30.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016949099999999999 2024-09-13T10:12:30.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.163383+0000", 2024-09-13T10:12:30.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00048715 2024-09-13T10:12:30.830 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.830 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.830 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.831 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4199 4.10 4:0b4c105e:::smithi19530714-16:head [assert-version v59,set-chunk] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:30.831 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.711426+0000", 2024-09-13T10:12:30.831 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.021493126999999, 2024-09-13T10:12:30.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0086774999999999994, 2024-09-13T10:12:30.831 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.831 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.831 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:30.831 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.831 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.831 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.831 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4199 2024-09-13T10:12:30.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.831 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.711426+0000", 2024-09-13T10:12:30.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.711426+0000", 2024-09-13T10:12:30.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.711428+0000", 2024-09-13T10:12:30.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6000000000000001e-06 2024-09-13T10:12:30.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.711440+0000", 2024-09-13T10:12:30.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2036e-05 2024-09-13T10:12:30.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.711470+0000", 2024-09-13T10:12:30.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9298999999999998e-05 2024-09-13T10:12:30.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.711486+0000", 2024-09-13T10:12:30.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6336000000000001e-05 2024-09-13T10:12:30.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.711629+0000", 2024-09-13T10:12:30.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000143151 2024-09-13T10:12:30.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.711872+0000", 2024-09-13T10:12:30.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024269499999999999 2024-09-13T10:12:30.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3,14", 2024-09-13T10:12:30.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.717340+0000", 2024-09-13T10:12:30.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054679359999999996 2024-09-13T10:12:30.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:30.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.719241+0000", 2024-09-13T10:12:30.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001901678 2024-09-13T10:12:30.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.719874+0000", 2024-09-13T10:12:30.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00063235699999999995 2024-09-13T10:12:30.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.719952+0000", 2024-09-13T10:12:30.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8634999999999995e-05 2024-09-13T10:12:30.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:30.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.720035+0000", 2024-09-13T10:12:30.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2768000000000004e-05 2024-09-13T10:12:30.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.720103+0000", 2024-09-13T10:12:30.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8009000000000006e-05 2024-09-13T10:12:30.837 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.837 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.837 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.837 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4909 4.10 4:0b4c105e:::smithi19530714-16:head [write 746343~651628] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e115)", 2024-09-13T10:12:30.837 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:24.648042+0000", 2024-09-13T10:12:30.837 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0848762650000001, 2024-09-13T10:12:30.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011798193, 2024-09-13T10:12:30.837 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:30.837 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:30.838 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:30.838 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:30.838 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:30.838 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:30.838 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4909 2024-09-13T10:12:30.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.838 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:30.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:30.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:24.648042+0000", 2024-09-13T10:12:30.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:30.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:24.648042+0000", 2024-09-13T10:12:30.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:30.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:30.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:24.648046+0000", 2024-09-13T10:12:30.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2629999999999999e-06 2024-09-13T10:12:30.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:30.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:24.649105+0000", 2024-09-13T10:12:30.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010593309999999999 2024-09-13T10:12:30.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:30.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:24.649137+0000", 2024-09-13T10:12:30.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2088999999999997e-05 2024-09-13T10:12:30.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:30.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:24.649153+0000", 2024-09-13T10:12:30.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5889999999999999e-05 2024-09-13T10:12:30.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:24.649322+0000", 2024-09-13T10:12:30.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016852499999999999 2024-09-13T10:12:30.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-09-13T10:12:30.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:24.649340+0000", 2024-09-13T10:12:30.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8233e-05 2024-09-13T10:12:30.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:30.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:24.652793+0000", 2024-09-13T10:12:30.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003453448 2024-09-13T10:12:30.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:30.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:24.652944+0000", 2024-09-13T10:12:30.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000150419 2024-09-13T10:12:30.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 3,14", 2024-09-13T10:12:30.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:24.653186+0000", 2024-09-13T10:12:30.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0002421 2024-09-13T10:12:30.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:30.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:24.658111+0000", 2024-09-13T10:12:30.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049256329999999996 2024-09-13T10:12:30.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:24.658551+0000", 2024-09-13T10:12:30.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043917099999999997 2024-09-13T10:12:30.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:30.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:24.659562+0000", 2024-09-13T10:12:30.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001011212 2024-09-13T10:12:30.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:30.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:24.659720+0000", 2024-09-13T10:12:30.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000158252 2024-09-13T10:12:30.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:30.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:30.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:30.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:24.659841+0000", 2024-09-13T10:12:30.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120627 2024-09-13T10:12:30.844 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.844 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.844 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.844 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:30.844 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:30.844 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:30.846 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-09-13T10:12:30.846 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:30.841+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:30.946 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:12:30.941+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:12:30.974 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:30.974 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:12:30.974 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-09-13T10:12:30.974 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:30.990 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-09-13T10:12:31.048 INFO:tasks.ceph.osd.15.smithi195.stderr:2024-09-13T10:12:31.041+0000 7fd5dccec640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 26585) UID: 0 2024-09-13T10:12:31.114 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:31.114 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-09-13T10:12:31.114 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-09-13T10:12:31.114 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-09-13T10:12:31.114 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:31.128 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-09-13T10:12:31.149 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:31.145+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:31.169 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:12:31.169 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":122,"fsid":"c1e91d47-9492-4461-a043-8698fd74ccf3","created":"2024-09-13T10:10:14.019912+0000","modified":"2024-09-13T10:12:30.965569+0000","last_up_change":"2024-09-13T10:10:28.089885+0000","last_in_change":"2024-09-13T10:10:18.489546+0000","flags":"noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5801984,"flags_set":["noscrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T10:10:28.924860+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":27,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":5.2300000190734863,"score_stable":5.2300000190734863,"optimal_score":0.79000002145767212,"raw_score_acting":4.130000114440918,"raw_score_stable":4.130000114440918,"primary_affinity_weighted":0.8372650146484375,"average_primary_affinity":0.95999997854232788,"average_primary_affinity_weighted":0.8399999737739563}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T10:10:28.972519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":14.289999961853027,"score_stable":14.289999961853027,"optimal_score":0.20999999344348907,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.8372650146484375,"average_primary_affinity":0.95999997854232788,"average_primary_affinity_weighted":0.8399999737739563}},{"pool":3,"pool_name":"low_tier","create_time":"2024-09-13T10:10:41.320719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":3.5199999809265137,"score_stable":3.5199999809265137,"optimal_score":0.63999998569488525,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.8372650146484375,"average_primary_affinity":0.95999997854232788,"average_primary_affinity_weighted":0.8399999737739563}},{"pool":4,"pool_name":"unique_pool_0","create_time":"2024-09-13T10:10:42.575395+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":26,"pg_placement_num":18,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":26,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"41","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"dedup_cdc_chunk_size":131072,"dedup_chunk_algorithm":"fastcdc","dedup_tier":3,"fingerprint_algorithm":"sha256"},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3.2400000095367432,"score_stable":3.7799999713897705,"optimal_score":1.1399999856948853,"raw_score_acting":3.690000057220459,"raw_score_stable":4.309999942779541,"primary_affinity_weighted":0.8372650146484375,"average_primary_affinity":0.95999997854232788,"average_primary_affinity_weighted":0.8399999737739563}}],"osds":[{"osd":0,"uuid":"f7d92188-c27c-4744-a330-4b7105e0f2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1977392321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":1977392321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6815","nonce":1977392321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1977392321}]},"public_addr":"172.21.15.73:6812/1977392321","cluster_addr":"172.21.15.73:6813/1977392321","heartbeat_back_addr":"172.21.15.73:6815/1977392321","heartbeat_front_addr":"172.21.15.73:6814/1977392321","state":["exists","up"]},{"osd":1,"uuid":"e900b771-7c96-4b5d-93d7-35a8ea6c026a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1228102864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6809","nonce":1228102864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6811","nonce":1228102864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1228102864}]},"public_addr":"172.21.15.136:6808/1228102864","cluster_addr":"172.21.15.136:6809/1228102864","heartbeat_back_addr":"172.21.15.136:6811/1228102864","heartbeat_front_addr":"172.21.15.136:6810/1228102864","state":["exists","up"]},{"osd":2,"uuid":"28b7aa59-33b2-4e24-8f3d-758f90e25f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3314820936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":3314820936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6803","nonce":3314820936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3314820936}]},"public_addr":"172.21.15.181:6800/3314820936","cluster_addr":"172.21.15.181:6801/3314820936","heartbeat_back_addr":"172.21.15.181:6803/3314820936","heartbeat_front_addr":"172.21.15.181:6802/3314820936","state":["exists","up"]},{"osd":3,"uuid":"a93d2df5-781e-49e1-a507-1718b6f954ed","up":1,"in":1,"weight":0.95001220703125,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6801","nonce":2558565460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2558565460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2558565460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":2558565460}]},"public_addr":"172.21.15.195:6801/2558565460","cluster_addr":"172.21.15.195:6802/2558565460","heartbeat_back_addr":"172.21.15.195:6804/2558565460","heartbeat_front_addr":"172.21.15.195:6803/2558565460","state":["exists","up"]},{"osd":4,"uuid":"e69053fd-f6d9-437e-a9d0-8896de6b927f","up":1,"in":1,"weight":0.95001220703125,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":858330493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6801","nonce":858330493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6803","nonce":858330493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":858330493}]},"public_addr":"172.21.15.73:6800/858330493","cluster_addr":"172.21.15.73:6801/858330493","heartbeat_back_addr":"172.21.15.73:6803/858330493","heartbeat_front_addr":"172.21.15.73:6802/858330493","state":["exists","up"]},{"osd":5,"uuid":"3667ec10-561e-4a23-8197-6357db4af315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":984313481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6805","nonce":984313481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6807","nonce":984313481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":984313481}]},"public_addr":"172.21.15.136:6804/984313481","cluster_addr":"172.21.15.136:6805/984313481","heartbeat_back_addr":"172.21.15.136:6807/984313481","heartbeat_front_addr":"172.21.15.136:6806/984313481","state":["exists","up"]},{"osd":6,"uuid":"30d4d63c-f04f-4423-91a6-816a7cbc0287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1505939978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":1505939978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":1505939978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1505939978}]},"public_addr":"172.21.15.181:6812/1505939978","cluster_addr":"172.21.15.181:6813/1505939978","heartbeat_back_addr":"172.21.15.181:6815/1505939978","heartbeat_front_addr":"172.21.15.181:6814/1505939978","state":["exists","up"]},{"osd":7,"uuid":"48812f05-82f1-4263-8ec5-4861a6e91b61","up":1,"in":1,"weight":1,"primary_affinity":0,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":3679292757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3679292757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3679292757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6811","nonce":3679292757}]},"public_addr":"172.21.15.195:6809/3679292757","cluster_addr":"172.21.15.195:6810/3679292757","heartbeat_back_addr":"172.21.15.195:6812/3679292757","heartbeat_front_addr":"172.21.15.195:6811/3679292757","state":["exists","up"]},{"osd":8,"uuid":"72056e61-d04b-4fd3-94bf-57ee3df4b466","up":1,"in":1,"weight":1,"primary_affinity":0.396240234375,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4052106885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6805","nonce":4052106885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":4052106885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4052106885}]},"public_addr":"172.21.15.73:6804/4052106885","cluster_addr":"172.21.15.73:6805/4052106885","heartbeat_back_addr":"172.21.15.73:6807/4052106885","heartbeat_front_addr":"172.21.15.73:6806/4052106885","state":["exists","up"]},{"osd":9,"uuid":"20e7f118-7148-4c27-9ec4-a33954709515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4266571820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6801","nonce":4266571820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6803","nonce":4266571820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4266571820}]},"public_addr":"172.21.15.136:6800/4266571820","cluster_addr":"172.21.15.136:6801/4266571820","heartbeat_back_addr":"172.21.15.136:6803/4266571820","heartbeat_front_addr":"172.21.15.136:6802/4266571820","state":["exists","up"]},{"osd":10,"uuid":"012409e8-51c2-4584-aba7-95996f0d72ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":958255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":958255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":958255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":958255259}]},"public_addr":"172.21.15.181:6808/958255259","cluster_addr":"172.21.15.181:6809/958255259","heartbeat_back_addr":"172.21.15.181:6811/958255259","heartbeat_front_addr":"172.21.15.181:6810/958255259","state":["exists","up"]},{"osd":11,"uuid":"88b137b1-7d30-45f7-ac5c-ba026704ef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":281884267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":281884267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":281884267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6815","nonce":281884267}]},"public_addr":"172.21.15.195:6813/281884267","cluster_addr":"172.21.15.195:6814/281884267","heartbeat_back_addr":"172.21.15.195:6816/281884267","heartbeat_front_addr":"172.21.15.195:6815/281884267","state":["exists","up"]},{"osd":12,"uuid":"b54732a2-9dff-42e1-b4ef-281cbca3ad7c","up":1,"in":1,"weight":1,"primary_affinity":0,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3532043806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6809","nonce":3532043806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6811","nonce":3532043806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3532043806}]},"public_addr":"172.21.15.73:6808/3532043806","cluster_addr":"172.21.15.73:6809/3532043806","heartbeat_back_addr":"172.21.15.73:6811/3532043806","heartbeat_front_addr":"172.21.15.73:6810/3532043806","state":["exists","up"]},{"osd":13,"uuid":"20b48864-44dc-4732-8dff-d0d21cc08032","up":1,"in":1,"weight":0.95001220703125,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":4083982131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":4083982131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6815","nonce":4083982131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":4083982131}]},"public_addr":"172.21.15.136:6812/4083982131","cluster_addr":"172.21.15.136:6813/4083982131","heartbeat_back_addr":"172.21.15.136:6815/4083982131","heartbeat_front_addr":"172.21.15.136:6814/4083982131","state":["exists","up"]},{"osd":14,"uuid":"9fac3ef4-e9be-4e8b-9a21-0de35b543d65","up":1,"in":1,"weight":0.95001220703125,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1173379175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":1173379175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":1173379175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1173379175}]},"public_addr":"172.21.15.181:6804/1173379175","cluster_addr":"172.21.15.181:6805/1173379175","heartbeat_back_addr":"172.21.15.181:6807/1173379175","heartbeat_front_addr":"172.21.15.181:6806/1173379175","state":["exists","up"]},{"osd":15,"uuid":"16b34f90-3148-4d41-a403-ba0f900991e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6805","nonce":848686366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":848686366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":848686366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":848686366}]},"public_addr":"172.21.15.195:6805/848686366","cluster_addr":"172.21.15.195:6806/848686366","heartbeat_back_addr":"172.21.15.195:6808/848686366","heartbeat_front_addr":"172.21.15.195:6807/848686366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:11:52.724148+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:11:45.986851+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:11:47.777103+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:11:47.179172+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:11:54.012724+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:11:46.147756+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:11:47.809723+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:11:38.606440+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:11:37.606540+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:11:25.726157+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:11:27.329800+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:11:26.722524+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:11:25.954484+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:11:25.900950+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:11:40.864138+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:11:48.003986+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"4.e","osds":[8,3,15]},{"pgid":"4.12","osds":[1,4,13]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"4.6","osds":[1,7,13]},{"pgid":"4.e","osds":[2,8,4]},{"pgid":"4.14","osds":[0,1,11]},{"pgid":"4.15","osds":[6,7,13]},{"pgid":"4.16","osds":[14,5,13]},{"pgid":"4.19","osds":[2,5,3]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T10:12:31.189 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:12:31.250 INFO:tasks.ceph.osd.2.smithi181.stderr:2024-09-13T10:12:31.245+0000 7ff0f7ebe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 26304) UID: 0 2024-09-13T10:12:31.252 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-09-13T10:12:31.252 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-09-13T10:12:31.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-09-13T10:12:31.253 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-09-13T10:12:31.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.253 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:203 3.1 3:b033e801:::smithi19530714-4 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:31.253 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:12:31.253 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 100.258571381, 2024-09-13T10:12:31.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035547764000000003, 2024-09-13T10:12:31.253 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.253 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.253 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:31.253 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:31.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:31.254 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:31.254 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 203 2024-09-13T10:12:31.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.254 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:12:31.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991464+0000", 2024-09-13T10:12:31.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991467+0000", 2024-09-13T10:12:31.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5030000000000002e-06 2024-09-13T10:12:31.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.255 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991483+0000", 2024-09-13T10:12:31.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5503e-05 2024-09-13T10:12:31.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991522+0000", 2024-09-13T10:12:31.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9399e-05 2024-09-13T10:12:31.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991544+0000", 2024-09-13T10:12:31.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2408999999999998e-05 2024-09-13T10:12:31.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991646+0000", 2024-09-13T10:12:31.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101798 2024-09-13T10:12:31.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991818+0000", 2024-09-13T10:12:31.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017204300000000001 2024-09-13T10:12:31.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:31.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.001618+0000", 2024-09-13T10:12:31.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0097993490000000006 2024-09-13T10:12:31.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:31.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026785+0000", 2024-09-13T10:12:31.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025167446 2024-09-13T10:12:31.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026856+0000", 2024-09-13T10:12:31.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0575999999999996e-05 2024-09-13T10:12:31.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026915+0000", 2024-09-13T10:12:31.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9258000000000001e-05 2024-09-13T10:12:31.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.259 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:31.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.026964+0000", 2024-09-13T10:12:31.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9280000000000003e-05 2024-09-13T10:12:31.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027011+0000", 2024-09-13T10:12:31.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7200000000000002e-05 2024-09-13T10:12:31.260 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.260 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.260 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.260 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:204 3.1 3:babcb876:::smithi19530714-5 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:31.260 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:12:31.261 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 100.258446865, 2024-09-13T10:12:31.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.22268644700000001, 2024-09-13T10:12:31.261 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.261 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.261 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:31.261 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:31.261 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:31.261 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:31.261 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 204 2024-09-13T10:12:31.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.261 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:12:31.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991588+0000", 2024-09-13T10:12:31.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991589+0000", 2024-09-13T10:12:31.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3859999999999999e-06 2024-09-13T10:12:31.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991598+0000", 2024-09-13T10:12:31.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7669999999999992e-06 2024-09-13T10:12:31.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991616+0000", 2024-09-13T10:12:31.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7589e-05 2024-09-13T10:12:31.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991631+0000", 2024-09-13T10:12:31.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.47e-05 2024-09-13T10:12:31.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992050+0000", 2024-09-13T10:12:31.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00041913399999999999 2024-09-13T10:12:31.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992229+0000", 2024-09-13T10:12:31.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017956499999999999 2024-09-13T10:12:31.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:31.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.203594+0000", 2024-09-13T10:12:31.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.211364303 2024-09-13T10:12:31.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:31.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.208526+0000", 2024-09-13T10:12:31.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049325020000000001 2024-09-13T10:12:31.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214019+0000", 2024-09-13T10:12:31.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0054933270000000001 2024-09-13T10:12:31.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214108+0000", 2024-09-13T10:12:31.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8517000000000006e-05 2024-09-13T10:12:31.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:31.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214202+0000", 2024-09-13T10:12:31.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3763000000000003e-05 2024-09-13T10:12:31.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.214275+0000", 2024-09-13T10:12:31.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2893999999999997e-05 2024-09-13T10:12:31.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.268 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.268 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.268 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:205 3.1 3:aec11a92:::smithi19530714-6 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:31.268 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:12:31.268 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 100.258371211, 2024-09-13T10:12:31.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.056950259000000003, 2024-09-13T10:12:31.268 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.269 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.269 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:31.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:31.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:31.269 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:31.269 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 205 2024-09-13T10:12:31.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.269 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:12:31.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991664+0000", 2024-09-13T10:12:31.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991665+0000", 2024-09-13T10:12:31.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1400000000000001e-06 2024-09-13T10:12:31.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991673+0000", 2024-09-13T10:12:31.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5730000000000008e-06 2024-09-13T10:12:31.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991689+0000", 2024-09-13T10:12:31.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5956000000000001e-05 2024-09-13T10:12:31.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991705+0000", 2024-09-13T10:12:31.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5539000000000001e-05 2024-09-13T10:12:31.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992393+0000", 2024-09-13T10:12:31.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00068836600000000004 2024-09-13T10:12:31.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992509+0000", 2024-09-13T10:12:31.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000115563 2024-09-13T10:12:31.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:31.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.011894+0000", 2024-09-13T10:12:31.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019385594999999999 2024-09-13T10:12:31.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:31.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027029+0000", 2024-09-13T10:12:31.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015134664000000001 2024-09-13T10:12:31.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.031277+0000", 2024-09-13T10:12:31.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004247804 2024-09-13T10:12:31.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048542+0000", 2024-09-13T10:12:31.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017265401999999999 2024-09-13T10:12:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048579+0000", 2024-09-13T10:12:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6238000000000001e-05 2024-09-13T10:12:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.048614+0000", 2024-09-13T10:12:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5419000000000001e-05 2024-09-13T10:12:31.275 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:209 3.1 3:9798d58d:::smithi19530714-10 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:12:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 100.25824299600001, 2024-09-13T10:12:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.069824676000000002, 2024-09-13T10:12:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:31.276 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 209 2024-09-13T10:12:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:12:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991792+0000", 2024-09-13T10:12:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991793+0000", 2024-09-13T10:12:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.057e-06 2024-09-13T10:12:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991803+0000", 2024-09-13T10:12:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7860000000000008e-06 2024-09-13T10:12:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991817+0000", 2024-09-13T10:12:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4066000000000001e-05 2024-09-13T10:12:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.991828+0000", 2024-09-13T10:12:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1097000000000001e-05 2024-09-13T10:12:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992666+0000", 2024-09-13T10:12:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00083755799999999997 2024-09-13T10:12:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992792+0000", 2024-09-13T10:12:31.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000126334 2024-09-13T10:12:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.027675+0000", 2024-09-13T10:12:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.034883299 2024-09-13T10:12:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.035648+0000", 2024-09-13T10:12:31.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.007972972 2024-09-13T10:12:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.045764+0000", 2024-09-13T10:12:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010115832 2024-09-13T10:12:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061460+0000", 2024-09-13T10:12:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015695563999999999 2024-09-13T10:12:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061544+0000", 2024-09-13T10:12:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4866999999999999e-05 2024-09-13T10:12:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.061617+0000", 2024-09-13T10:12:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2243999999999995e-05 2024-09-13T10:12:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:213 3.1 3:accf4e56:::smithi19530714-14 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:12:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 100.25746905299999, 2024-09-13T10:12:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.060593045999999998, 2024-09-13T10:12:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:31.284 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 213 2024-09-13T10:12:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:12:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992566+0000", 2024-09-13T10:12:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992568+0000", 2024-09-13T10:12:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3640000000000002e-06 2024-09-13T10:12:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992580+0000", 2024-09-13T10:12:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1796e-05 2024-09-13T10:12:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992662+0000", 2024-09-13T10:12:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1814e-05 2024-09-13T10:12:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.992683+0000", 2024-09-13T10:12:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0746000000000001e-05 2024-09-13T10:12:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.993024+0000", 2024-09-13T10:12:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034141299999999999 2024-09-13T10:12:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:50.993182+0000", 2024-09-13T10:12:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015817699999999999 2024-09-13T10:12:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.019852+0000", 2024-09-13T10:12:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.026669470000000001 2024-09-13T10:12:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.032551+0000", 2024-09-13T10:12:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012699729999999999 2024-09-13T10:12:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.034351+0000", 2024-09-13T10:12:31.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017998269999999999 2024-09-13T10:12:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053106+0000", 2024-09-13T10:12:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018755236000000002 2024-09-13T10:12:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053131+0000", 2024-09-13T10:12:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4729e-05 2024-09-13T10:12:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.053159+0000", 2024-09-13T10:12:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7744000000000001e-05 2024-09-13T10:12:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:219 3.1 3:87bfc7b1:::smithi19530714-20 low_tier:head [copy-from ver 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:12:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 100.21701998, 2024-09-13T10:12:31.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.037494069999999997, 2024-09-13T10:12:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 219 2024-09-13T10:12:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:12:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033015+0000", 2024-09-13T10:12:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033019+0000", 2024-09-13T10:12:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.726e-06 2024-09-13T10:12:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033036+0000", 2024-09-13T10:12:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7329999999999998e-05 2024-09-13T10:12:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033060+0000", 2024-09-13T10:12:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4286000000000001e-05 2024-09-13T10:12:31.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033079+0000", 2024-09-13T10:12:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8872000000000002e-05 2024-09-13T10:12:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033155+0000", 2024-09-13T10:12:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6118999999999994e-05 2024-09-13T10:12:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.033297+0000", 2024-09-13T10:12:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014154700000000001 2024-09-13T10:12:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.041666+0000", 2024-09-13T10:12:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0083696180000000005 2024-09-13T10:12:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:31.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046621+0000", 2024-09-13T10:12:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049545759999999996 2024-09-13T10:12:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.052333+0000", 2024-09-13T10:12:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057120160000000003 2024-09-13T10:12:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070340+0000", 2024-09-13T10:12:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.018006859 2024-09-13T10:12:31.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070447+0000", 2024-09-13T10:12:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106637 2024-09-13T10:12:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070509+0000", 2024-09-13T10:12:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2483999999999993e-05 2024-09-13T10:12:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.298 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:222 3.1 3:bdac4e61:::smithi19530714-23 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:12:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 100.203332006, 2024-09-13T10:12:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.041434034000000002, 2024-09-13T10:12:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:31.299 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 222 2024-09-13T10:12:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:12:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046703+0000", 2024-09-13T10:12:31.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046706+0000", 2024-09-13T10:12:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0089999999999999e-06 2024-09-13T10:12:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046719+0000", 2024-09-13T10:12:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2565000000000001e-05 2024-09-13T10:12:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046740+0000", 2024-09-13T10:12:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0979e-05 2024-09-13T10:12:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046752+0000", 2024-09-13T10:12:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2917e-05 2024-09-13T10:12:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046803+0000", 2024-09-13T10:12:31.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0417000000000002e-05 2024-09-13T10:12:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.046908+0000", 2024-09-13T10:12:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105039 2024-09-13T10:12:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.066097+0000", 2024-09-13T10:12:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.019189277000000001 2024-09-13T10:12:31.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070969+0000", 2024-09-13T10:12:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0048721429999999998 2024-09-13T10:12:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.087852+0000", 2024-09-13T10:12:31.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.016882489000000001 2024-09-13T10:12:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.087942+0000", 2024-09-13T10:12:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9738e-05 2024-09-13T10:12:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088076+0000", 2024-09-13T10:12:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013400700000000001 2024-09-13T10:12:31.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088137+0000", 2024-09-13T10:12:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1453999999999998e-05 2024-09-13T10:12:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.306 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:223 3.1 3:8759f4e0:::smithi19530714-24 low_tier:head [copy-from ver 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:12:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 100.19065513699999, 2024-09-13T10:12:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053691092000000003, 2024-09-13T10:12:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 223 2024-09-13T10:12:31.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:12:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059380+0000", 2024-09-13T10:12:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059383+0000", 2024-09-13T10:12:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.847e-06 2024-09-13T10:12:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059399+0000", 2024-09-13T10:12:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6739e-05 2024-09-13T10:12:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059433+0000", 2024-09-13T10:12:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3946000000000003e-05 2024-09-13T10:12:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059456+0000", 2024-09-13T10:12:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2163000000000001e-05 2024-09-13T10:12:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.059621+0000", 2024-09-13T10:12:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016546700000000001 2024-09-13T10:12:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.060003+0000", 2024-09-13T10:12:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000381982 2024-09-13T10:12:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.085339+0000", 2024-09-13T10:12:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.025335610000000001 2024-09-13T10:12:31.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.089896+0000", 2024-09-13T10:12:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045575930000000004 2024-09-13T10:12:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.102515+0000", 2024-09-13T10:12:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.012618664999999999 2024-09-13T10:12:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113009+0000", 2024-09-13T10:12:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010493784000000001 2024-09-13T10:12:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113044+0000", 2024-09-13T10:12:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5543999999999997e-05 2024-09-13T10:12:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.113071+0000", 2024-09-13T10:12:31.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6752000000000001e-05 2024-09-13T10:12:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:234 3.1 3:bdce9caa:::smithi19530714-35 low_tier mmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm:head [copy-from ver 12] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:12:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 100.18093587600001, 2024-09-13T10:12:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.043032263000000001, 2024-09-13T10:12:31.314 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 234 2024-09-13T10:12:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:12:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069099+0000", 2024-09-13T10:12:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069102+0000", 2024-09-13T10:12:31.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.543e-06 2024-09-13T10:12:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069112+0000", 2024-09-13T10:12:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0091e-05 2024-09-13T10:12:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069133+0000", 2024-09-13T10:12:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1188000000000001e-05 2024-09-13T10:12:31.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.069143+0000", 2024-09-13T10:12:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0456e-05 2024-09-13T10:12:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070588+0000", 2024-09-13T10:12:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001445046 2024-09-13T10:12:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.070801+0000", 2024-09-13T10:12:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000212151 2024-09-13T10:12:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.082362+0000", 2024-09-13T10:12:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.011561781 2024-09-13T10:12:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.088161+0000", 2024-09-13T10:12:31.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0057986119999999999 2024-09-13T10:12:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.101972+0000", 2024-09-13T10:12:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013811498 2024-09-13T10:12:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112028+0000", 2024-09-13T10:12:31.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.010055530999999999 2024-09-13T10:12:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112082+0000", 2024-09-13T10:12:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3974999999999997e-05 2024-09-13T10:12:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.112131+0000", 2024-09-13T10:12:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9391000000000002e-05 2024-09-13T10:12:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.321 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:243 3.1 3:a4dd4282:::smithi19530714-44 low_tier:head [copy-from ver 9] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T10:12:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:12:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 100.15272817100001, 2024-09-13T10:12:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.036759694000000002, 2024-09-13T10:12:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.322 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 243 2024-09-13T10:12:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:12:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097307+0000", 2024-09-13T10:12:31.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097308+0000", 2024-09-13T10:12:31.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2099999999999995e-07 2024-09-13T10:12:31.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097314+0000", 2024-09-13T10:12:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9989999999999997e-06 2024-09-13T10:12:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097326+0000", 2024-09-13T10:12:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1787000000000001e-05 2024-09-13T10:12:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097332+0000", 2024-09-13T10:12:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6950000000000003e-06 2024-09-13T10:12:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097378+0000", 2024-09-13T10:12:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5744000000000001e-05 2024-09-13T10:12:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.097489+0000", 2024-09-13T10:12:31.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000111352 2024-09-13T10:12:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,14", 2024-09-13T10:12:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.118571+0000", 2024-09-13T10:12:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.021081249999999999 2024-09-13T10:12:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.128494+0000", 2024-09-13T10:12:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0099237270000000002 2024-09-13T10:12:31.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.128775+0000", 2024-09-13T10:12:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028027299999999999 2024-09-13T10:12:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.133963+0000", 2024-09-13T10:12:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.005188191 2024-09-13T10:12:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134015+0000", 2024-09-13T10:12:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2438000000000002e-05 2024-09-13T10:12:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:51.134066+0000", 2024-09-13T10:12:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1316999999999997e-05 2024-09-13T10:12:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:1272 4.f 4:f7ad6ef6:::smithi19530714-1:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e45)", 2024-09-13T10:12:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:12:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 91.374499995999997, 2024-09-13T10:12:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.050041173000000001, 2024-09-13T10:12:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 1272 2024-09-13T10:12:31.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:12:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875535+0000", 2024-09-13T10:12:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875537+0000", 2024-09-13T10:12:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2689999999999998e-06 2024-09-13T10:12:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875548+0000", 2024-09-13T10:12:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0291e-05 2024-09-13T10:12:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875567+0000", 2024-09-13T10:12:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9284999999999999e-05 2024-09-13T10:12:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875578+0000", 2024-09-13T10:12:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1256e-05 2024-09-13T10:12:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875763+0000", 2024-09-13T10:12:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000185396 2024-09-13T10:12:31.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.875908+0000", 2024-09-13T10:12:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014412299999999999 2024-09-13T10:12:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:10:59.925576+0000", 2024-09-13T10:12:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.049668552999999997 2024-09-13T10:12:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.334 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:2719 4.f 4:f7ad6ef6:::smithi19530714-1:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e69)", 2024-09-13T10:12:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:11:29.386743+0000", 2024-09-13T10:12:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 61.863291912999998, 2024-09-13T10:12:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047467252000000001, 2024-09-13T10:12:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:31.335 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 2719 2024-09-13T10:12:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386743+0000", 2024-09-13T10:12:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386743+0000", 2024-09-13T10:12:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386746+0000", 2024-09-13T10:12:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2729999999999998e-06 2024-09-13T10:12:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386758+0000", 2024-09-13T10:12:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1703e-05 2024-09-13T10:12:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386789+0000", 2024-09-13T10:12:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0815000000000002e-05 2024-09-13T10:12:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386806+0000", 2024-09-13T10:12:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7595999999999999e-05 2024-09-13T10:12:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.386996+0000", 2024-09-13T10:12:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000189521 2024-09-13T10:12:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.387151+0000", 2024-09-13T10:12:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015501600000000001 2024-09-13T10:12:31.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:11:29.434210+0000", 2024-09-13T10:12:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.047059327999999997 2024-09-13T10:12:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.9.21:5 3.1 3.b06a67a1 (undecoded) ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e97)", 2024-09-13T10:12:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:03.964065+0000", 2024-09-13T10:12:31.340 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.285970053, 2024-09-13T10:12:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82246186799999998, 2024-09-13T10:12:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-09-13T10:12:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964065+0000", 2024-09-13T10:12:31.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964065+0000", 2024-09-13T10:12:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964069+0000", 2024-09-13T10:12:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9060000000000004e-06 2024-09-13T10:12:31.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964092+0000", 2024-09-13T10:12:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2892999999999999e-05 2024-09-13T10:12:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964133+0000", 2024-09-13T10:12:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1576000000000003e-05 2024-09-13T10:12:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964152+0000", 2024-09-13T10:12:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9165999999999999e-05 2024-09-13T10:12:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964289+0000", 2024-09-13T10:12:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000136694 2024-09-13T10:12:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:03.964310+0000", 2024-09-13T10:12:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1169e-05 2024-09-13T10:12:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:04.786399+0000", 2024-09-13T10:12:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.82208862999999999 2024-09-13T10:12:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:04.786527+0000", 2024-09-13T10:12:31.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012783400000000001 2024-09-13T10:12:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.9.21:5 3.1 3:85e6560d:::989de9f18158d38e3c2846cd6c5625b2e4406737a04dca59570c2ce866cf967c:head [call cas.chunk_put_ref in=16b] snapc 0=[] RETRY=1 ondisk+retry+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.793509+0000", 2024-09-13T10:12:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.456526338, 2024-09-13T10:12:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.060644814999999998, 2024-09-13T10:12:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.346 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793509+0000", 2024-09-13T10:12:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793509+0000", 2024-09-13T10:12:31.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793511+0000", 2024-09-13T10:12:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2800000000000002e-06 2024-09-13T10:12:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793521+0000", 2024-09-13T10:12:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8090000000000004e-06 2024-09-13T10:12:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793539+0000", 2024-09-13T10:12:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8546000000000002e-05 2024-09-13T10:12:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793562+0000", 2024-09-13T10:12:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2490999999999999e-05 2024-09-13T10:12:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794585+0000", 2024-09-13T10:12:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010233519999999999 2024-09-13T10:12:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794602+0000", 2024-09-13T10:12:31.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6680999999999998e-05 2024-09-13T10:12:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.847842+0000", 2024-09-13T10:12:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.053240290000000003 2024-09-13T10:12:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.848265+0000", 2024-09-13T10:12:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042276500000000002 2024-09-13T10:12:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,10,14", 2024-09-13T10:12:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.848806+0000", 2024-09-13T10:12:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00054073499999999998 2024-09-13T10:12:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.851591+0000", 2024-09-13T10:12:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027856589999999998 2024-09-13T10:12:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.851712+0000", 2024-09-13T10:12:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000120321 2024-09-13T10:12:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.851735+0000", 2024-09-13T10:12:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3825999999999999e-05 2024-09-13T10:12:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.853601+0000", 2024-09-13T10:12:31.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018660879999999999 2024-09-13T10:12:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.853776+0000", 2024-09-13T10:12:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017476300000000001 2024-09-13T10:12:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.854153+0000", 2024-09-13T10:12:31.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037720899999999999 2024-09-13T10:12:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.15.22:44 3.1 3:9ec9087f:::5d8d6b7076a8d108632e41519fcb6bd459cf76343846e25ee0c5e911722fca24:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.793735+0000", 2024-09-13T10:12:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.456299718, 2024-09-13T10:12:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.067146312, 2024-09-13T10:12:31.355 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793735+0000", 2024-09-13T10:12:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793735+0000", 2024-09-13T10:12:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.793737+0000", 2024-09-13T10:12:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.404e-06 2024-09-13T10:12:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794023+0000", 2024-09-13T10:12:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028664599999999999 2024-09-13T10:12:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794045+0000", 2024-09-13T10:12:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1789000000000001e-05 2024-09-13T10:12:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794057+0000", 2024-09-13T10:12:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2061e-05 2024-09-13T10:12:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794659+0000", 2024-09-13T10:12:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00060190099999999996 2024-09-13T10:12:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794670+0000", 2024-09-13T10:12:31.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0761000000000001e-05 2024-09-13T10:12:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.854930+0000", 2024-09-13T10:12:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.060260237000000001 2024-09-13T10:12:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.855350+0000", 2024-09-13T10:12:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042033800000000003 2024-09-13T10:12:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,10,14", 2024-09-13T10:12:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.856082+0000", 2024-09-13T10:12:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073115799999999998 2024-09-13T10:12:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.860193+0000", 2024-09-13T10:12:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0041111079999999996 2024-09-13T10:12:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.860338+0000", 2024-09-13T10:12:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014509700000000001 2024-09-13T10:12:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.860470+0000", 2024-09-13T10:12:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000132304 2024-09-13T10:12:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.860603+0000", 2024-09-13T10:12:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000133137 2024-09-13T10:12:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.860686+0000", 2024-09-13T10:12:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2315e-05 2024-09-13T10:12:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.860882+0000", 2024-09-13T10:12:31.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019605600000000001 2024-09-13T10:12:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.15.22:45 3.1 3:974a72fa:::beb33e09e1c5ba22d3db3e2cd707b299c6694e34eb9bd9ac42f66c64167565c7:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.794078+0000", 2024-09-13T10:12:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.455956820000001, 2024-09-13T10:12:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.075604088999999999, 2024-09-13T10:12:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.364 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794078+0000", 2024-09-13T10:12:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794078+0000", 2024-09-13T10:12:31.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794079+0000", 2024-09-13T10:12:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0189999999999999e-06 2024-09-13T10:12:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794467+0000", 2024-09-13T10:12:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00038798499999999998 2024-09-13T10:12:31.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794484+0000", 2024-09-13T10:12:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6388999999999998e-05 2024-09-13T10:12:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794495+0000", 2024-09-13T10:12:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.118e-05 2024-09-13T10:12:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794708+0000", 2024-09-13T10:12:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000213546 2024-09-13T10:12:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794718+0000", 2024-09-13T10:12:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9620000000000005e-06 2024-09-13T10:12:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.860986+0000", 2024-09-13T10:12:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.066268218000000004 2024-09-13T10:12:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.861385+0000", 2024-09-13T10:12:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00039855799999999997 2024-09-13T10:12:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,10,14", 2024-09-13T10:12:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.862131+0000", 2024-09-13T10:12:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074556799999999995 2024-09-13T10:12:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.864957+0000", 2024-09-13T10:12:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028266950000000002 2024-09-13T10:12:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.865115+0000", 2024-09-13T10:12:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015747299999999999 2024-09-13T10:12:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.866537+0000", 2024-09-13T10:12:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014227409999999999 2024-09-13T10:12:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.869419+0000", 2024-09-13T10:12:31.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0028810770000000001 2024-09-13T10:12:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.869518+0000", 2024-09-13T10:12:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8997000000000001e-05 2024-09-13T10:12:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.869682+0000", 2024-09-13T10:12:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016468099999999999 2024-09-13T10:12:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.372 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.15.22:51 3.1 3:845d12bf:::28848c4777c1cb2908d895c4117b5dc4ef8bdb4529e24915ac4b533e48aa206d:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] RETRY=1 ondisk+retry+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.794512+0000", 2024-09-13T10:12:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.455522943999998, 2024-09-13T10:12:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.079145091000000001, 2024-09-13T10:12:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.373 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794512+0000", 2024-09-13T10:12:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794512+0000", 2024-09-13T10:12:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.374 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794513+0000", 2024-09-13T10:12:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0699999999999996e-07 2024-09-13T10:12:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794755+0000", 2024-09-13T10:12:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000241955 2024-09-13T10:12:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.375 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794769+0000", 2024-09-13T10:12:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3784999999999999e-05 2024-09-13T10:12:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794778+0000", 2024-09-13T10:12:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6970000000000005e-06 2024-09-13T10:12:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.376 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794903+0000", 2024-09-13T10:12:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012476499999999999 2024-09-13T10:12:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794920+0000", 2024-09-13T10:12:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6393999999999999e-05 2024-09-13T10:12:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.377 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.866605+0000", 2024-09-13T10:12:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.071685663999999996 2024-09-13T10:12:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.867044+0000", 2024-09-13T10:12:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043897700000000001 2024-09-13T10:12:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,10,14", 2024-09-13T10:12:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.867828+0000", 2024-09-13T10:12:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00078432999999999997 2024-09-13T10:12:31.378 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.870457+0000", 2024-09-13T10:12:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026287229999999999 2024-09-13T10:12:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.870558+0000", 2024-09-13T10:12:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010079800000000001 2024-09-13T10:12:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.379 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.872235+0000", 2024-09-13T10:12:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016767780000000001 2024-09-13T10:12:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.873474+0000", 2024-09-13T10:12:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001239448 2024-09-13T10:12:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:31.380 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.873539+0000", 2024-09-13T10:12:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4655999999999999e-05 2024-09-13T10:12:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.873657+0000", 2024-09-13T10:12:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118214 2024-09-13T10:12:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.381 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.15.22:59 3.1 3:964c391a:::c9c0cf4a00ae29c03b283d9cbb01cfb175b42402a28680dcd93389b12aeeb768:head [call cas.chunk_create_or_get_ref in=26b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e98)", 2024-09-13T10:12:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.794801+0000", 2024-09-13T10:12:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.455233884999998, 2024-09-13T10:12:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.080623212999999999, 2024-09-13T10:12:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794801+0000", 2024-09-13T10:12:31.382 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794801+0000", 2024-09-13T10:12:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794802+0000", 2024-09-13T10:12:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.029e-06 2024-09-13T10:12:31.383 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794839+0000", 2024-09-13T10:12:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6442000000000001e-05 2024-09-13T10:12:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794907+0000", 2024-09-13T10:12:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8913999999999998e-05 2024-09-13T10:12:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.384 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794923+0000", 2024-09-13T10:12:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5393e-05 2024-09-13T10:12:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.794993+0000", 2024-09-13T10:12:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9777000000000006e-05 2024-09-13T10:12:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:31.385 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.795005+0000", 2024-09-13T10:12:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2536e-05 2024-09-13T10:12:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.870632+0000", 2024-09-13T10:12:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.075626342999999999 2024-09-13T10:12:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.870939+0000", 2024-09-13T10:12:31.386 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000306994 2024-09-13T10:12:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,10,14", 2024-09-13T10:12:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.871368+0000", 2024-09-13T10:12:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00042931399999999998 2024-09-13T10:12:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.873711+0000", 2024-09-13T10:12:31.387 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023427159999999999 2024-09-13T10:12:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.873802+0000", 2024-09-13T10:12:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1730000000000004e-05 2024-09-13T10:12:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.873834+0000", 2024-09-13T10:12:31.388 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2106000000000001e-05 2024-09-13T10:12:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.875254+0000", 2024-09-13T10:12:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014194279999999999 2024-09-13T10:12:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.389 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.875313+0000", 2024-09-13T10:12:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9660000000000001e-05 2024-09-13T10:12:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.875424+0000", 2024-09-13T10:12:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110831 2024-09-13T10:12:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.390 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.2.23:171 3.1 3:ae1afb27:::0b4da3b2f5b021545e17c9fa615b6bae2f6acc3b51ee6b20bac582de1c7125d8:head [call cas.chunk_put_ref in=16b] snapc 0=[] ondisk+write+rwordered+ignore_cache+ignore_overlay+known_if_redirected+supports_pool_eio e99)", 2024-09-13T10:12:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:05.834554+0000", 2024-09-13T10:12:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.415480714000001, 2024-09-13T10:12:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.035802232000000003, 2024-09-13T10:12:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T10:12:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.391 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.834554+0000", 2024-09-13T10:12:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.834554+0000", 2024-09-13T10:12:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.392 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.834555+0000", 2024-09-13T10:12:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4399999999999999e-07 2024-09-13T10:12:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.834560+0000", 2024-09-13T10:12:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7600000000000002e-06 2024-09-13T10:12:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.393 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.834569+0000", 2024-09-13T10:12:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3279999999999995e-06 2024-09-13T10:12:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.834578+0000", 2024-09-13T10:12:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4179999999999997e-06 2024-09-13T10:12:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.394 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.834759+0000", 2024-09-13T10:12:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000181029 2024-09-13T10:12:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for peered", 2024-09-13T10:12:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.834771+0000", 2024-09-13T10:12:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2660000000000001e-05 2024-09-13T10:12:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.395 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.865214+0000", 2024-09-13T10:12:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.030443006000000002 2024-09-13T10:12:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.865441+0000", 2024-09-13T10:12:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022639900000000001 2024-09-13T10:12:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.396 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for subops from 4,10,14", 2024-09-13T10:12:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.865775+0000", 2024-09-13T10:12:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033417000000000001 2024-09-13T10:12:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.869809+0000", 2024-09-13T10:12:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040337819999999996 2024-09-13T10:12:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.397 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.869955+0000", 2024-09-13T10:12:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014625400000000001 2024-09-13T10:12:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op_commit", 2024-09-13T10:12:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.869996+0000", 2024-09-13T10:12:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1219e-05 2024-09-13T10:12:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.398 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_commit_rec", 2024-09-13T10:12:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.870086+0000", 2024-09-13T10:12:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0101999999999999e-05 2024-09-13T10:12:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-09-13T10:12:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.870184+0000", 2024-09-13T10:12:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7861000000000004e-05 2024-09-13T10:12:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.399 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:05.870356+0000", 2024-09-13T10:12:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017239999999999999 2024-09-13T10:12:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4607.0:4552 4.f 4:f7ad6ef6:::smithi19530714-1:head [tier-flush] snapc 0=[] ondisk+read+ignore_cache+known_if_redirected+supports_pool_eio e106)", 2024-09-13T10:12:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-09-13T10:12:14.681527+0000", 2024-09-13T10:12:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.568507975999999, 2024-09-13T10:12:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040766716000000001, 2024-09-13T10:12:31.400 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-09-13T10:12:31.401 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-09-13T10:12:31.401 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-09-13T10:12:31.401 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-09-13T10:12:31.401 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4607", 2024-09-13T10:12:31.401 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.195:0/3963675565", 2024-09-13T10:12:31.401 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 4552 2024-09-13T10:12:31.401 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.401 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-09-13T10:12:31.401 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.401 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-09-13T10:12:31.401 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.681527+0000", 2024-09-13T10:12:31.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-09-13T10:12:31.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.681527+0000", 2024-09-13T10:12:31.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-09-13T10:12:31.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.402 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.402 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-09-13T10:12:31.402 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.681530+0000", 2024-09-13T10:12:31.402 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8289999999999999e-06 2024-09-13T10:12:31.402 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-09-13T10:12:31.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.681543+0000", 2024-09-13T10:12:31.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2747e-05 2024-09-13T10:12:31.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.403 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-09-13T10:12:31.403 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.681570+0000", 2024-09-13T10:12:31.403 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7492e-05 2024-09-13T10:12:31.403 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.403 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-09-13T10:12:31.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.681587+0000", 2024-09-13T10:12:31.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6650000000000002e-05 2024-09-13T10:12:31.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-09-13T10:12:31.404 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.681691+0000", 2024-09-13T10:12:31.404 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000104751 2024-09-13T10:12:31.404 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.404 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.404 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-09-13T10:12:31.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.681791+0000", 2024-09-13T10:12:31.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9263000000000002e-05 2024-09-13T10:12:31.405 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-09-13T10:12:31.405 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-09-13T10:12:31.405 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-09-13T10:12:31.405 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-09-13T10:12:14.722294+0000", 2024-09-13T10:12:31.405 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.040502983999999999 2024-09-13T10:12:31.405 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.405 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.405 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.405 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-09-13T10:12:31.405 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-09-13T10:12:31.406 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-09-13T10:12:31.407 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T10:12:31.408 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:31.405+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:31.439 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T10:12:31.507 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:12:31.507 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:12:31.509 INFO:tasks.ceph.osd.13.smithi136.stderr:2024-09-13T10:12:31.506+0000 7f7cdec34640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 26307) UID: 0 2024-09-13T10:12:31.526 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-13T10:12:30.976656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":248576831,"num_objects":195,"num_object_clones":0,"num_object_copies":585,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":22,"num_objects_unfound":0,"num_objects_dirty":191,"num_whiteouts":0,"num_read":6313,"num_read_kb":4525408,"num_write":3623,"num_write_kb":830305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":63971336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":19,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":117,"acting":117,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":1054196,"kb_used_data":567120,"kb_used_omap":24,"kb_used_meta":486183,"kb_avail":1508895244,"statfs":{"total":1546188226560,"available":1545108729856,"internally_reserved":0,"allocated":580730880,"data_stored":570826056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":497851471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16244713,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":234,"num_read_kb":205605,"num_write":111,"num_write_kb":35100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013516"},"pg_stats":[{"pgid":"4.19","version":"106'70","reported_seq":390,"reported_epoch":120,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:12:28.951146+0000","last_change":"2024-09-13T10:12:13.894581+0000","last_active":"2024-09-13T10:12:28.951146+0000","last_peered":"2024-09-13T10:12:28.951146+0000","last_clean":"2024-09-13T10:12:09.846417+0000","last_became_active":"2024-09-13T10:12:13.882915+0000","last_became_peered":"2024-09-13T10:12:13.882915+0000","last_unstale":"2024-09-13T10:12:28.951146+0000","last_undegraded":"2024-09-13T10:12:28.951146+0000","last_fullsized":"2024-09-13T10:12:28.951146+0000","mapping_epoch":105,"log_start":"106'67","ondisk_log_start":"106'67","created":41,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":5,"last_scrub":"45'28","last_scrub_stamp":"2024-09-13T10:11:00.388132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.388132+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":67,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5294337,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":131646,"num_write":46,"num_write_kb":11392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2376704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,9],"acting":[2,5,3],"avail_no_missing":["2","3","5"],"object_location_counts":[{"shards":"2,3,5","objects":2}],"blocked_by":[],"up_primary":11,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.18","version":"112'35","reported_seq":306,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T10:12:27.955109+0000","last_change":"2024-09-13T10:12:24.965502+0000","last_active":"2024-09-13T10:12:27.955109+0000","last_peered":"2024-09-13T10:12:27.955109+0000","last_clean":"2024-09-13T10:12:27.955109+0000","last_became_active":"2024-09-13T10:12:17.721145+0000","last_became_peered":"2024-09-13T10:12:17.721145+0000","last_unstale":"2024-09-13T10:12:27.955109+0000","last_undegraded":"2024-09-13T10:12:27.955109+0000","last_fullsized":"2024-09-13T10:12:27.955109+0000","mapping_epoch":109,"log_start":"112'31","ondisk_log_start":"112'31","created":41,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":5,"last_scrub":"112'35","last_scrub_stamp":"2024-09-13T10:12:24.965343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:24.965343+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":31,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:41.721906+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3294238,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":103,"num_read_kb":89615,"num_write":33,"num_write_kb":9625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3925645,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,0],"acting":[3,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"116'141","reported_seq":546,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T10:12:26.941401+0000","last_change":"2024-09-13T10:12:19.734795+0000","last_active":"2024-09-13T10:12:26.941401+0000","last_peered":"2024-09-13T10:12:26.941401+0000","last_clean":"2024-09-13T10:12:26.941401+0000","last_became_active":"2024-09-13T10:12:19.734082+0000","last_became_peered":"2024-09-13T10:12:19.734082+0000","last_unstale":"2024-09-13T10:12:26.941401+0000","last_undegraded":"2024-09-13T10:12:26.941401+0000","last_fullsized":"2024-09-13T10:12:26.941401+0000","mapping_epoch":111,"log_start":"112'138","ondisk_log_start":"112'138","created":41,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":5,"last_scrub":"105'131","last_scrub_stamp":"2024-09-13T10:12:14.226050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:14.226050+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":138,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:19.970257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6336371,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":273,"num_read_kb":239047,"num_write":133,"num_write_kb":38525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":7608719,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,10],"acting":[1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"119'89","reported_seq":443,"reported_epoch":119,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:12:28.123511+0000","last_change":"2024-09-13T10:12:18.770392+0000","last_active":"2024-09-13T10:12:28.123511+0000","last_peered":"2024-09-13T10:12:28.123511+0000","last_clean":"2024-09-13T10:12:03.112281+0000","last_became_active":"2024-09-13T10:12:18.754536+0000","last_became_peered":"2024-09-13T10:12:18.754536+0000","last_unstale":"2024-09-13T10:12:28.123511+0000","last_undegraded":"2024-09-13T10:12:28.123511+0000","last_fullsized":"2024-09-13T10:12:28.123511+0000","mapping_epoch":110,"log_start":"116'85","ondisk_log_start":"116'85","created":41,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":5,"last_scrub":"46'44","last_scrub_stamp":"2024-09-13T10:11:00.456457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.456457+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":82,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3764834,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":194,"num_read_kb":225120,"num_write":52,"num_write_kb":13910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":3338234,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,4],"acting":[14,5,13],"avail_no_missing":["14","5","13"],"object_location_counts":[{"shards":"5,13,14","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":14,"purged_snaps":[]},{"pgid":"4.15","version":"112'52","reported_seq":353,"reported_epoch":119,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:12:27.957596+0000","last_change":"2024-09-13T10:12:26.024556+0000","last_active":"2024-09-13T10:12:27.957596+0000","last_peered":"2024-09-13T10:12:27.957596+0000","last_clean":"2024-09-13T10:12:22.968090+0000","last_became_active":"2024-09-13T10:12:26.021929+0000","last_became_peered":"2024-09-13T10:12:26.021929+0000","last_unstale":"2024-09-13T10:12:27.957596+0000","last_undegraded":"2024-09-13T10:12:27.957596+0000","last_fullsized":"2024-09-13T10:12:27.957596+0000","mapping_epoch":116,"log_start":"112'49","ondisk_log_start":"112'49","created":41,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":5,"last_scrub":"107'48","last_scrub_stamp":"2024-09-13T10:12:16.084431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:16.084431+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":47,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3378273,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":45,"num_read_kb":39290,"num_write":15,"num_write_kb":4430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1702851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,0],"acting":[6,7,13],"avail_no_missing":["6","7","13"],"object_location_counts":[{"shards":"6,7,13","objects":1}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"39'40","reported_seq":165,"reported_epoch":119,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:12:28.093342+0000","last_change":"2024-09-13T10:12:28.093084+0000","last_active":"2024-09-13T10:12:28.093342+0000","last_peered":"2024-09-13T10:12:28.093342+0000","last_clean":"2024-09-13T10:10:51.933477+0000","last_became_active":"2024-09-13T10:12:28.089262+0000","last_became_peered":"2024-09-13T10:12:28.089262+0000","last_unstale":"2024-09-13T10:12:28.093342+0000","last_undegraded":"2024-09-13T10:12:28.093342+0000","last_fullsized":"2024-09-13T10:12:28.093342+0000","mapping_epoch":118,"log_start":"39'37","ondisk_log_start":"39'37","created":41,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":5,"last_scrub":"39'40","last_scrub_stamp":"2024-09-13T10:12:00.310748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.310748+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":37,"ondisk_log_size":3,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7831963,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":7,"num_read_kb":112,"num_write":16,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10],"acting":[0,1,11],"avail_no_missing":["0","1","11"],"object_location_counts":[{"shards":"0,1,11","objects":3}],"blocked_by":[],"up_primary":6,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"98'44","reported_seq":330,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-09-13T10:12:21.622176+0000","last_change":"2024-09-13T10:12:12.572121+0000","last_active":"2024-09-13T10:12:21.622176+0000","last_peered":"2024-09-13T10:12:21.622176+0000","last_clean":"2024-09-13T10:12:21.622176+0000","last_became_active":"2024-09-13T10:11:15.757364+0000","last_became_peered":"2024-09-13T10:11:15.757364+0000","last_unstale":"2024-09-13T10:12:21.622176+0000","last_undegraded":"2024-09-13T10:12:21.622176+0000","last_fullsized":"2024-09-13T10:12:21.622176+0000","mapping_epoch":58,"log_start":"97'41","ondisk_log_start":"97'41","created":41,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":5,"last_scrub":"98'44","last_scrub_stamp":"2024-09-13T10:12:12.572014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:12.572014+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":39,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:21.886670+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2336239,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":75,"num_read_kb":61934,"num_write":31,"num_write_kb":7734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1493592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"113'55","reported_seq":394,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T10:12:26.941179+0000","last_change":"2024-09-13T10:11:55.737652+0000","last_active":"2024-09-13T10:12:26.941179+0000","last_peered":"2024-09-13T10:12:26.941179+0000","last_clean":"2024-09-13T10:12:26.941179+0000","last_became_active":"2024-09-13T10:11:55.736402+0000","last_became_peered":"2024-09-13T10:11:55.736402+0000","last_unstale":"2024-09-13T10:12:26.941179+0000","last_undegraded":"2024-09-13T10:12:26.941179+0000","last_fullsized":"2024-09-13T10:12:26.941179+0000","mapping_epoch":91,"log_start":"100'52","ondisk_log_start":"100'52","created":41,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":5,"last_scrub":"42'17","last_scrub_stamp":"2024-09-13T10:11:02.582829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:02.582829+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":52,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:30.381168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2369853,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":178,"num_read_kb":147980,"num_write":36,"num_write_kb":10155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":7321937,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,13],"acting":[1,4,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"112'38","reported_seq":324,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-13T10:12:25.933503+0000","last_change":"2024-09-13T10:12:24.732215+0000","last_active":"2024-09-13T10:12:25.933503+0000","last_peered":"2024-09-13T10:12:25.933503+0000","last_clean":"2024-09-13T10:12:25.933503+0000","last_became_active":"2024-09-13T10:11:40.064608+0000","last_became_peered":"2024-09-13T10:11:40.064608+0000","last_unstale":"2024-09-13T10:12:25.933503+0000","last_undegraded":"2024-09-13T10:12:25.933503+0000","last_fullsized":"2024-09-13T10:12:25.933503+0000","mapping_epoch":77,"log_start":"112'35","ondisk_log_start":"112'35","created":41,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":5,"last_scrub":"112'38","last_scrub_stamp":"2024-09-13T10:12:24.732055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:24.732055+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":32,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:30.903088+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3344659,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":79,"num_read_kb":61838,"num_write":30,"num_write_kb":10139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":1805029,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,12],"acting":[13,4,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.10","version":"115'65","reported_seq":414,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T10:12:26.959915+0000","last_change":"2024-09-13T10:12:13.384690+0000","last_active":"2024-09-13T10:12:26.959915+0000","last_peered":"2024-09-13T10:12:26.959915+0000","last_clean":"2024-09-13T10:12:26.959915+0000","last_became_active":"2024-09-13T10:11:40.063380+0000","last_became_peered":"2024-09-13T10:11:40.063380+0000","last_unstale":"2024-09-13T10:12:26.959915+0000","last_undegraded":"2024-09-13T10:12:26.959915+0000","last_fullsized":"2024-09-13T10:12:26.959915+0000","mapping_epoch":77,"log_start":"115'62","ondisk_log_start":"115'62","created":41,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":5,"last_scrub":"98'60","last_scrub_stamp":"2024-09-13T10:12:13.384591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:13.384591+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":60,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:35.167402+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6638864,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":198,"num_read_kb":161976,"num_write":56,"num_write_kb":15705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":2889991,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,14,3],"acting":[9,14,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":147,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T10:12:27.955529+0000","last_change":"2024-09-13T10:12:03.961952+0000","last_active":"2024-09-13T10:12:27.955529+0000","last_peered":"2024-09-13T10:12:27.955529+0000","last_clean":"2024-09-13T10:12:27.955529+0000","last_became_active":"2024-09-13T10:11:12.744090+0000","last_became_peered":"2024-09-13T10:11:12.744090+0000","last_unstale":"2024-09-13T10:12:27.955529+0000","last_undegraded":"2024-09-13T10:12:27.955529+0000","last_fullsized":"2024-09-13T10:12:27.955529+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:03.961779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:03.961779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:12.481787+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,12],"acting":[3,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"39'12","reported_seq":251,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T10:12:27.956936+0000","last_change":"2024-09-13T10:11:01.537080+0000","last_active":"2024-09-13T10:12:27.956936+0000","last_peered":"2024-09-13T10:12:27.956936+0000","last_clean":"2024-09-13T10:12:27.956936+0000","last_became_active":"2024-09-13T10:10:54.318519+0000","last_became_peered":"2024-09-13T10:10:54.318519+0000","last_unstale":"2024-09-13T10:12:27.956936+0000","last_undegraded":"2024-09-13T10:12:27.956936+0000","last_fullsized":"2024-09-13T10:12:27.956936+0000","mapping_epoch":31,"log_start":"38'9","ondisk_log_start":"38'9","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'12","last_scrub_stamp":"2024-09-13T10:11:01.536963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.536963+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":9,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,14],"acting":[6,8,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":211,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T10:12:27.955256+0000","last_change":"2024-09-13T10:12:12.892985+0000","last_active":"2024-09-13T10:12:27.955256+0000","last_peered":"2024-09-13T10:12:27.955256+0000","last_clean":"2024-09-13T10:12:27.955256+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:12:27.955256+0000","last_undegraded":"2024-09-13T10:12:27.955256+0000","last_fullsized":"2024-09-13T10:12:27.955256+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:12.892836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:12.892836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:20.216623+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"112'102","reported_seq":449,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-13T10:12:28.951594+0000","last_change":"2024-09-13T10:11:01.219554+0000","last_active":"2024-09-13T10:12:28.951594+0000","last_peered":"2024-09-13T10:12:28.951594+0000","last_clean":"2024-09-13T10:12:28.951594+0000","last_became_active":"2024-09-13T10:10:54.314545+0000","last_became_peered":"2024-09-13T10:10:54.314545+0000","last_unstale":"2024-09-13T10:12:28.951594+0000","last_undegraded":"2024-09-13T10:12:28.951594+0000","last_fullsized":"2024-09-13T10:12:28.951594+0000","mapping_epoch":31,"log_start":"102'99","ondisk_log_start":"102'99","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"43'27","last_scrub_stamp":"2024-09-13T10:11:01.219486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:01.219486+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":98,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:30.784020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4571636,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":233,"num_read_kb":189971,"num_write":91,"num_write_kb":24881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,8],"acting":[2,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":209,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T10:12:27.152968+0000","last_change":"2024-09-13T10:12:01.407368+0000","last_active":"2024-09-13T10:12:27.152968+0000","last_peered":"2024-09-13T10:12:27.152968+0000","last_clean":"2024-09-13T10:12:27.152968+0000","last_became_active":"2024-09-13T10:10:30.128658+0000","last_became_peered":"2024-09-13T10:10:30.128658+0000","last_unstale":"2024-09-13T10:12:27.152968+0000","last_undegraded":"2024-09-13T10:12:27.152968+0000","last_fullsized":"2024-09-13T10:12:27.152968+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:01.407274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:01.407274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:11.291191+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.0","version":"111'43","reported_seq":329,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T10:12:26.933690+0000","last_change":"2024-09-13T10:11:05.681812+0000","last_active":"2024-09-13T10:12:26.933690+0000","last_peered":"2024-09-13T10:12:26.933690+0000","last_clean":"2024-09-13T10:12:26.933690+0000","last_became_active":"2024-09-13T10:10:54.320014+0000","last_became_peered":"2024-09-13T10:10:54.320014+0000","last_unstale":"2024-09-13T10:12:26.933690+0000","last_undegraded":"2024-09-13T10:12:26.933690+0000","last_fullsized":"2024-09-13T10:12:26.933690+0000","mapping_epoch":31,"log_start":"111'40","ondisk_log_start":"111'40","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'14","last_scrub_stamp":"2024-09-13T10:11:05.681653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.681653+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":39,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:34.112673+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3243817,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":95,"num_read_kb":92312,"num_write":33,"num_write_kb":8858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":55,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T10:12:27.955470+0000","last_change":"2024-09-13T10:12:05.204517+0000","last_active":"2024-09-13T10:12:27.955470+0000","last_peered":"2024-09-13T10:12:27.955470+0000","last_clean":"2024-09-13T10:12:27.955470+0000","last_became_active":"2024-09-13T10:12:05.203738+0000","last_became_peered":"2024-09-13T10:12:05.203738+0000","last_unstale":"2024-09-13T10:12:27.955470+0000","last_undegraded":"2024-09-13T10:12:27.955470+0000","last_fullsized":"2024-09-13T10:12:27.955470+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:11:41.627456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:41.627456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:49.027751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12,1],"acting":[3,12,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"111'45","reported_seq":325,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-13T10:12:28.951167+0000","last_change":"2024-09-13T10:12:27.388632+0000","last_active":"2024-09-13T10:12:28.951167+0000","last_peered":"2024-09-13T10:12:28.951167+0000","last_clean":"2024-09-13T10:12:28.951167+0000","last_became_active":"2024-09-13T10:10:54.317600+0000","last_became_peered":"2024-09-13T10:10:54.317600+0000","last_unstale":"2024-09-13T10:12:28.951167+0000","last_undegraded":"2024-09-13T10:12:28.951167+0000","last_fullsized":"2024-09-13T10:12:28.951167+0000","mapping_epoch":31,"log_start":"109'42","ondisk_log_start":"109'42","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"111'45","last_scrub_stamp":"2024-09-13T10:12:27.388449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:27.388449+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":42,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:45.761314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":73519,"num_write":32,"num_write_kb":9404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,12],"acting":[2,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"116'325","reported_seq":605,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-13T10:12:25.933312+0000","last_change":"2024-09-13T10:12:07.816766+0000","last_active":"2024-09-13T10:12:25.933312+0000","last_peered":"2024-09-13T10:12:25.933312+0000","last_clean":"2024-09-13T10:12:25.933312+0000","last_became_active":"2024-09-13T10:12:07.813457+0000","last_became_peered":"2024-09-13T10:12:07.813457+0000","last_unstale":"2024-09-13T10:12:25.933312+0000","last_undegraded":"2024-09-13T10:12:25.933312+0000","last_fullsized":"2024-09-13T10:12:25.933312+0000","mapping_epoch":100,"log_start":"115'322","ondisk_log_start":"115'322","created":30,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:10:42.187897+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":337,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:27.759150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27728175,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":300,"num_read_kb":7000,"num_write":422,"num_write_kb":79121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":26621319,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,14,10],"acting":[13,14,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":294,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T10:12:27.955270+0000","last_change":"2024-09-13T10:11:48.946651+0000","last_active":"2024-09-13T10:12:27.955270+0000","last_peered":"2024-09-13T10:12:27.955270+0000","last_clean":"2024-09-13T10:12:27.955270+0000","last_became_active":"2024-09-13T10:11:12.699043+0000","last_became_peered":"2024-09-13T10:11:12.699043+0000","last_unstale":"2024-09-13T10:12:27.955270+0000","last_undegraded":"2024-09-13T10:12:27.955270+0000","last_fullsized":"2024-09-13T10:12:27.955270+0000","mapping_epoch":55,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'112","last_scrub_stamp":"2024-09-13T10:11:48.946478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:48.946478+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:12.399266+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,7],"acting":[3,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":115,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-13T10:12:25.933406+0000","last_change":"2024-09-13T10:11:38.808286+0000","last_active":"2024-09-13T10:12:25.933406+0000","last_peered":"2024-09-13T10:12:25.933406+0000","last_clean":"2024-09-13T10:12:25.933406+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:12:25.933406+0000","last_undegraded":"2024-09-13T10:12:25.933406+0000","last_fullsized":"2024-09-13T10:12:25.933406+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T10:11:38.808119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:38.808119+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:43.158000+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"4.6","version":"105'93","reported_seq":418,"reported_epoch":118,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:12:26.947467+0000","last_change":"2024-09-13T10:12:05.805129+0000","last_active":"2024-09-13T10:12:26.947467+0000","last_peered":"2024-09-13T10:12:26.947467+0000","last_clean":"2024-09-13T10:12:00.406498+0000","last_became_active":"2024-09-13T10:12:05.800127+0000","last_became_peered":"2024-09-13T10:12:05.800127+0000","last_unstale":"2024-09-13T10:12:26.947467+0000","last_undegraded":"2024-09-13T10:12:26.947467+0000","last_fullsized":"2024-09-13T10:12:26.947467+0000","mapping_epoch":98,"log_start":"105'90","ondisk_log_start":"105'90","created":31,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"47'44","last_scrub_stamp":"2024-09-13T10:11:05.396838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:05.396838+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":90,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6538022,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":171,"num_read_kb":158772,"num_write":57,"num_write_kb":16492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,4],"acting":[1,7,13],"avail_no_missing":["1","7","13"],"object_location_counts":[{"shards":"1,7,13","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"118'364","reported_seq":581,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T10:12:27.954874+0000","last_change":"2024-09-13T10:12:00.967692+0000","last_active":"2024-09-13T10:12:27.954874+0000","last_peered":"2024-09-13T10:12:27.954874+0000","last_clean":"2024-09-13T10:12:27.954874+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:12:27.954874+0000","last_undegraded":"2024-09-13T10:12:27.954874+0000","last_fullsized":"2024-09-13T10:12:27.954874+0000","mapping_epoch":30,"log_start":"118'361","ondisk_log_start":"118'361","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'314","last_scrub_stamp":"2024-09-13T10:12:00.967585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.967585+0000","objects_scrubbed":33,"log_size":3,"log_dups_size":373,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:19.050419+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33652726,"num_objects":43,"num_object_clones":0,"num_object_copies":129,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":0,"num_read":333,"num_read_kb":12994,"num_write":481,"num_write_kb":101793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":217,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-13T10:12:28.951195+0000","last_change":"2024-09-13T10:11:30.291380+0000","last_active":"2024-09-13T10:12:28.951195+0000","last_peered":"2024-09-13T10:12:28.951195+0000","last_clean":"2024-09-13T10:12:28.951195+0000","last_became_active":"2024-09-13T10:10:30.127232+0000","last_became_peered":"2024-09-13T10:10:30.127232+0000","last_unstale":"2024-09-13T10:12:28.951195+0000","last_undegraded":"2024-09-13T10:12:28.951195+0000","last_fullsized":"2024-09-13T10:12:28.951195+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-13T10:11:30.291157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:30.291157+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:12:31.417661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0034622780000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"116'112","reported_seq":444,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T10:12:26.941869+0000","last_change":"2024-09-13T10:12:25.618740+0000","last_active":"2024-09-13T10:12:26.941869+0000","last_peered":"2024-09-13T10:12:26.941869+0000","last_clean":"2024-09-13T10:12:26.941869+0000","last_became_active":"2024-09-13T10:10:54.314465+0000","last_became_peered":"2024-09-13T10:10:54.314465+0000","last_unstale":"2024-09-13T10:12:26.941869+0000","last_undegraded":"2024-09-13T10:12:26.941869+0000","last_fullsized":"2024-09-13T10:12:26.941869+0000","mapping_epoch":31,"log_start":"115'109","ondisk_log_start":"115'109","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"116'112","last_scrub_stamp":"2024-09-13T10:12:25.618565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:25.618565+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":106,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:55.463319+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7176721,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":126736,"num_write":85,"num_write_kb":24924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,10],"acting":[1,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"118'346","reported_seq":549,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T10:12:26.994011+0000","last_change":"2024-09-13T10:12:02.384819+0000","last_active":"2024-09-13T10:12:26.994011+0000","last_peered":"2024-09-13T10:12:26.994011+0000","last_clean":"2024-09-13T10:12:26.994011+0000","last_became_active":"2024-09-13T10:10:43.220074+0000","last_became_peered":"2024-09-13T10:10:43.220074+0000","last_unstale":"2024-09-13T10:12:26.994011+0000","last_undegraded":"2024-09-13T10:12:26.994011+0000","last_fullsized":"2024-09-13T10:12:26.994011+0000","mapping_epoch":30,"log_start":"116'343","ondisk_log_start":"116'343","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'292","last_scrub_stamp":"2024-09-13T10:12:02.384617+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:02.384617+0000","objects_scrubbed":28,"log_size":3,"log_dups_size":355,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:26.698118+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27658528,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":310,"num_read_kb":4757,"num_write":455,"num_write_kb":83526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":103,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T10:12:27.152777+0000","last_change":"2024-09-13T10:11:53.335859+0000","last_active":"2024-09-13T10:12:27.152777+0000","last_peered":"2024-09-13T10:12:27.152777+0000","last_clean":"2024-09-13T10:12:27.152777+0000","last_became_active":"2024-09-13T10:11:39.046090+0000","last_became_peered":"2024-09-13T10:11:39.046090+0000","last_unstale":"2024-09-13T10:12:27.152777+0000","last_undegraded":"2024-09-13T10:12:27.152777+0000","last_fullsized":"2024-09-13T10:12:27.152777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:11:53.335756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:53.335756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:17.649356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,7,9],"acting":[15,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.4","version":"118'210","reported_seq":771,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T10:12:28.249279+0000","last_change":"2024-09-13T10:11:03.633738+0000","last_active":"2024-09-13T10:12:28.249279+0000","last_peered":"2024-09-13T10:12:28.249279+0000","last_clean":"2024-09-13T10:12:28.249279+0000","last_became_active":"2024-09-13T10:10:54.316518+0000","last_became_peered":"2024-09-13T10:10:54.316518+0000","last_unstale":"2024-09-13T10:12:28.249279+0000","last_undegraded":"2024-09-13T10:12:28.249279+0000","last_fullsized":"2024-09-13T10:12:28.249279+0000","mapping_epoch":31,"log_start":"115'206","ondisk_log_start":"115'206","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'66","last_scrub_stamp":"2024-09-13T10:11:03.633499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:03.633499+0000","objects_scrubbed":5,"log_size":4,"log_dups_size":206,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15109790,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":679,"num_read_kb":669262,"num_write":162,"num_write_kb":45837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10],"acting":[6,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"119'401","reported_seq":612,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T10:12:28.245714+0000","last_change":"2024-09-13T10:12:00.627361+0000","last_active":"2024-09-13T10:12:28.245714+0000","last_peered":"2024-09-13T10:12:28.245714+0000","last_clean":"2024-09-13T10:12:28.245714+0000","last_became_active":"2024-09-13T10:10:43.229961+0000","last_became_peered":"2024-09-13T10:10:43.229961+0000","last_unstale":"2024-09-13T10:12:28.245714+0000","last_undegraded":"2024-09-13T10:12:28.245714+0000","last_fullsized":"2024-09-13T10:12:28.245714+0000","mapping_epoch":30,"log_start":"118'398","ondisk_log_start":"118'398","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'359","last_scrub_stamp":"2024-09-13T10:12:00.627252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.627252+0000","objects_scrubbed":25,"log_size":3,"log_dups_size":407,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:24.042469+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20269983,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":364,"num_read_kb":6442,"num_write":499,"num_write_kb":98736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":212,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T10:12:27.956627+0000","last_change":"2024-09-13T10:11:51.041068+0000","last_active":"2024-09-13T10:12:27.956627+0000","last_peered":"2024-09-13T10:12:27.956627+0000","last_clean":"2024-09-13T10:12:27.956627+0000","last_became_active":"2024-09-13T10:10:30.272007+0000","last_became_peered":"2024-09-13T10:10:30.272007+0000","last_unstale":"2024-09-13T10:12:27.956627+0000","last_undegraded":"2024-09-13T10:12:27.956627+0000","last_fullsized":"2024-09-13T10:12:27.956627+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:11:51.040975+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:51.040975+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:13.227058+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.000987604,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"116'179","reported_seq":665,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T10:12:27.957357+0000","last_change":"2024-09-13T10:11:00.541038+0000","last_active":"2024-09-13T10:12:27.957357+0000","last_peered":"2024-09-13T10:12:27.957357+0000","last_clean":"2024-09-13T10:12:27.957357+0000","last_became_active":"2024-09-13T10:10:54.315196+0000","last_became_peered":"2024-09-13T10:10:54.315196+0000","last_unstale":"2024-09-13T10:12:27.957357+0000","last_undegraded":"2024-09-13T10:12:27.957357+0000","last_fullsized":"2024-09-13T10:12:27.957357+0000","mapping_epoch":31,"log_start":"114'176","ondisk_log_start":"114'176","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'48","last_scrub_stamp":"2024-09-13T10:11:00.540905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:00.540905+0000","objects_scrubbed":5,"log_size":3,"log_dups_size":175,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15546739,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":482,"num_read_kb":447684,"num_write":152,"num_write_kb":47322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":3,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,0],"acting":[6,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"39'5","reported_seq":242,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T10:12:27.955045+0000","last_change":"2024-09-13T10:12:13.871233+0000","last_active":"2024-09-13T10:12:27.955045+0000","last_peered":"2024-09-13T10:12:27.955045+0000","last_clean":"2024-09-13T10:12:27.955045+0000","last_became_active":"2024-09-13T10:10:54.314935+0000","last_became_peered":"2024-09-13T10:10:54.314935+0000","last_unstale":"2024-09-13T10:12:27.955045+0000","last_undegraded":"2024-09-13T10:12:27.955045+0000","last_fullsized":"2024-09-13T10:12:27.955045+0000","mapping_epoch":31,"log_start":"38'2","ondisk_log_start":"38'2","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'5","last_scrub_stamp":"2024-09-13T10:12:13.871148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:13.871148+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":2,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:35.499147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,0],"acting":[3,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"118'46","reported_seq":349,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T10:12:27.948113+0000","last_change":"2024-09-13T10:12:26.342967+0000","last_active":"2024-09-13T10:12:27.948113+0000","last_peered":"2024-09-13T10:12:27.948113+0000","last_clean":"2024-09-13T10:12:27.948113+0000","last_became_active":"2024-09-13T10:10:54.316057+0000","last_became_peered":"2024-09-13T10:10:54.316057+0000","last_unstale":"2024-09-13T10:12:27.948113+0000","last_undegraded":"2024-09-13T10:12:27.948113+0000","last_fullsized":"2024-09-13T10:12:27.948113+0000","mapping_epoch":31,"log_start":"94'43","ondisk_log_start":"94'43","created":31,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'45","last_scrub_stamp":"2024-09-13T10:12:26.342816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:26.342816+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":40,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:32.081381+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2806802,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":136,"num_read_kb":117500,"num_write":14,"num_write_kb":3748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,9],"acting":[11,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.a","version":"119'37","reported_seq":297,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T10:12:28.082476+0000","last_change":"2024-09-13T10:11:12.708135+0000","last_active":"2024-09-13T10:12:28.082476+0000","last_peered":"2024-09-13T10:12:28.082476+0000","last_clean":"2024-09-13T10:12:28.082476+0000","last_became_active":"2024-09-13T10:11:12.707306+0000","last_became_peered":"2024-09-13T10:11:12.707306+0000","last_unstale":"2024-09-13T10:12:28.082476+0000","last_undegraded":"2024-09-13T10:12:28.082476+0000","last_fullsized":"2024-09-13T10:12:28.082476+0000","mapping_epoch":55,"log_start":"119'34","ondisk_log_start":"119'34","created":31,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":34,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3731220,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":100,"num_read_kb":85434,"num_write":40,"num_write_kb":13241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,8,13],"acting":[6,8,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"119'94","reported_seq":474,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-13T10:12:28.951277+0000","last_change":"2024-09-13T10:12:17.219674+0000","last_active":"2024-09-13T10:12:28.951277+0000","last_peered":"2024-09-13T10:12:28.951277+0000","last_clean":"2024-09-13T10:12:28.951277+0000","last_became_active":"2024-09-13T10:12:07.819670+0000","last_became_peered":"2024-09-13T10:12:07.819670+0000","last_unstale":"2024-09-13T10:12:28.951277+0000","last_undegraded":"2024-09-13T10:12:28.951277+0000","last_fullsized":"2024-09-13T10:12:28.951277+0000","mapping_epoch":100,"log_start":"119'91","ondisk_log_start":"119'91","created":31,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'89","last_scrub_stamp":"2024-09-13T10:12:17.219609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:17.219609+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":91,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:46.457217+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3832062,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":363,"num_read_kb":343508,"num_write":91,"num_write_kb":25119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4887315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"116'156","reported_seq":609,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-13T10:12:27.957971+0000","last_change":"2024-09-13T10:10:59.373229+0000","last_active":"2024-09-13T10:12:27.957971+0000","last_peered":"2024-09-13T10:12:27.957971+0000","last_clean":"2024-09-13T10:12:27.957971+0000","last_became_active":"2024-09-13T10:10:59.366341+0000","last_became_peered":"2024-09-13T10:10:59.366341+0000","last_unstale":"2024-09-13T10:12:27.957971+0000","last_undegraded":"2024-09-13T10:12:27.957971+0000","last_fullsized":"2024-09-13T10:12:27.957971+0000","mapping_epoch":44,"log_start":"116'153","ondisk_log_start":"116'153","created":31,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":153,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12689549,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":586,"num_read_kb":527744,"num_write":166,"num_write_kb":46882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,12,3],"acting":[6,12,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"114'77","reported_seq":366,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-13T10:12:27.153157+0000","last_change":"2024-09-13T10:12:03.448785+0000","last_active":"2024-09-13T10:12:27.153157+0000","last_peered":"2024-09-13T10:12:27.153157+0000","last_clean":"2024-09-13T10:12:27.153157+0000","last_became_active":"2024-09-13T10:10:59.364443+0000","last_became_peered":"2024-09-13T10:10:59.364443+0000","last_unstale":"2024-09-13T10:12:27.153157+0000","last_undegraded":"2024-09-13T10:12:27.153157+0000","last_fullsized":"2024-09-13T10:12:27.153157+0000","mapping_epoch":44,"log_start":"114'74","ondisk_log_start":"114'74","created":31,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"96'67","last_scrub_stamp":"2024-09-13T10:12:03.448609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:12:03.448609+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":74,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:16.920584+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6521248,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":153,"num_read_kb":145968,"num_write":77,"num_write_kb":21898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":1,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,12,7],"acting":[15,12,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"4.e","version":"111'32","reported_seq":246,"reported_epoch":120,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:12:28.951547+0000","last_change":"2024-09-13T10:12:20.880028+0000","last_active":"2024-09-13T10:12:28.951547+0000","last_peered":"2024-09-13T10:12:28.951547+0000","last_clean":"2024-09-13T10:11:03.155929+0000","last_became_active":"2024-09-13T10:11:20.874865+0000","last_became_peered":"2024-09-13T10:11:20.874865+0000","last_unstale":"2024-09-13T10:12:28.951547+0000","last_undegraded":"2024-09-13T10:12:28.951547+0000","last_fullsized":"2024-09-13T10:12:28.951547+0000","mapping_epoch":61,"log_start":"93'29","ondisk_log_start":"93'29","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:10:43.197988+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":29,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3193396,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":67,"num_read_kb":67323,"num_write":34,"num_write_kb":10536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,15],"acting":[2,8,4],"avail_no_missing":["2","4","8"],"object_location_counts":[{"shards":"2,4,8","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"115'126","reported_seq":473,"reported_epoch":117,"state":"active+clean","last_fresh":"2024-09-13T10:12:25.933830+0000","last_change":"2024-09-13T10:11:53.668346+0000","last_active":"2024-09-13T10:12:25.933830+0000","last_peered":"2024-09-13T10:12:25.933830+0000","last_clean":"2024-09-13T10:12:25.933830+0000","last_became_active":"2024-09-13T10:10:44.223700+0000","last_became_peered":"2024-09-13T10:10:44.223700+0000","last_unstale":"2024-09-13T10:12:25.933830+0000","last_undegraded":"2024-09-13T10:12:25.933830+0000","last_fullsized":"2024-09-13T10:12:25.933830+0000","mapping_epoch":31,"log_start":"115'123","ondisk_log_start":"115'123","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"89'87","last_scrub_stamp":"2024-09-13T10:11:53.668179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:43.197988+0000","last_clean_scrub_stamp":"2024-09-13T10:11:53.668179+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":123,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:10.956333+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9126399,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":309,"num_read_kb":289851,"num_write":137,"num_write_kb":38994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":2,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,7],"acting":[13,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":26,"stat_sum":{"num_bytes":138677032,"num_objects":45,"num_object_clones":0,"num_object_copies":135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":22,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":0,"num_read":4920,"num_read_kb":4494142,"num_write":1619,"num_write_kb":464455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":37350017,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":19,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":257019904,"data_stored":256226705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":78,"acting":78,"num_store_stats":16},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":109309412,"num_objects":146,"num_object_clones":0,"num_object_copies":438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":1307,"num_read_kb":31193,"num_write":1857,"num_write_kb":363176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":26621319,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":296718336,"data_stored":295976656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":13}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247835,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":86588,"kb_used_data":55404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94285252,"statfs":{"total":96636764160,"available":96548098048,"internally_reserved":0,"allocated":56733696,"data_stored":56087287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247837,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48440,"kb_used_data":17256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323400,"statfs":{"total":96636764160,"available":96587161600,"internally_reserved":0,"allocated":17670144,"data_stored":17121205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247837,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67372,"kb_used_data":36188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94304468,"statfs":{"total":96636764160,"available":96567775232,"internally_reserved":0,"allocated":37056512,"data_stored":36436141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313244,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34168,"kb_used_data":7144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337672,"statfs":{"total":96636764160,"available":96601776128,"internally_reserved":0,"allocated":7315456,"data_stored":6785007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,5,6,8,10,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313243,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84452,"kb_used_data":53268,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94287388,"statfs":{"total":96636764160,"available":96550285312,"internally_reserved":0,"allocated":54546432,"data_stored":53942849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,6,7,8,9,10,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247835,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58996,"kb_used_data":27804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312844,"statfs":{"total":96636764160,"available":96576352256,"internally_reserved":0,"allocated":28471296,"data_stored":27857853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,7,8,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280539,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62824,"kb_used_data":31640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94309016,"statfs":{"total":96636764160,"available":96572432384,"internally_reserved":0,"allocated":32399360,"data_stored":31729886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280539,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":108676,"kb_used_data":77492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94263164,"statfs":{"total":96636764160,"available":96525479936,"internally_reserved":0,"allocated":79351808,"data_stored":78655772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,8,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280540,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79752,"kb_used_data":48568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94292088,"statfs":{"total":96636764160,"available":96555098112,"internally_reserved":0,"allocated":49733632,"data_stored":49114968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280538,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48668,"kb_used_data":21644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94323172,"statfs":{"total":96636764160,"available":96586928128,"internally_reserved":0,"allocated":22163456,"data_stored":21610981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280539,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82580,"kb_used_data":51396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94289260,"statfs":{"total":96636764160,"available":96552202240,"internally_reserved":0,"allocated":52629504,"data_stored":52000845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280539,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":98192,"kb_used_data":67004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94273648,"statfs":{"total":96636764160,"available":96536215552,"internally_reserved":0,"allocated":68612096,"data_stored":67825499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280539,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38416,"kb_used_data":7232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333424,"statfs":{"total":96636764160,"available":96597426176,"internally_reserved":0,"allocated":7405568,"data_stored":6869355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280539,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":5196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":5320704,"data_stored":4793897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247835,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44012,"kb_used_data":12828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327828,"statfs":{"total":96636764160,"available":96591695872,"internally_reserved":0,"allocated":13135872,"data_stored":12582040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247834,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78840,"kb_used_data":47056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94293000,"statfs":{"total":96636764160,"available":96556032000,"internally_reserved":0,"allocated":48185344,"data_stored":47412471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,5,7,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15941632,"data_stored":15901087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":54214656,"data_stored":54087573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":33726464,"data_stored":33652726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33726464,"data_stored":33652726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":27742208,"data_stored":27658528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":38506496,"data_stored":38392033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":27783168,"data_stored":27728175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":48394240,"data_stored":48280928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":11145216,"data_stored":11095974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14344192,"data_stored":14254956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":13570048,"data_stored":13536700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24072192,"data_stored":24023005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":14635008,"data_stored":14610192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20791296,"data_stored":20758931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":45428736,"data_stored":45203575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":14925824,"data_stored":14894759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":7335936,"data_stored":7326453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":5951488,"data_stored":5941713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":28733440,"data_stored":28581416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":6033408,"data_stored":6017305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":16580608,"data_stored":16557459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":22568960,"data_stored":22536853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":6955008,"data_stored":6945567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":3948544,"data_stored":3941847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:12:31.528 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-13T10:12:31.529 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-13T10:12:31.540 INFO:teuthology.orchestra.run.smithi181.stdout:{ 2024-09-13T10:12:31.540 INFO:teuthology.orchestra.run.smithi181.stdout: "ops": [], 2024-09-13T10:12:31.540 INFO:teuthology.orchestra.run.smithi181.stdout: "num_ops": 0 2024-09-13T10:12:31.540 INFO:teuthology.orchestra.run.smithi181.stdout:} 2024-09-13T10:12:31.554 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T10:12:31.580 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-09-13T10:12:31.580 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T10:12:31.601 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-09-13T10:12:31.601 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T10:12:31.610 INFO:tasks.ceph.osd.10.smithi181.stderr:2024-09-13T10:12:31.605+0000 7f574f55f640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 26308) UID: 0 2024-09-13T10:12:31.620 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-09-13T10:12:31.620 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T10:12:31.641 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-09-13T10:12:31.641 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T10:12:31.657 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-09-13T10:12:31.657 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T10:12:31.677 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:31.677 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:31.680 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-09-13T10:12:31.680 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T10:12:31.691 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T10:12:31.700 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-09-13T10:12:31.700 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T10:12:31.710 INFO:tasks.ceph.osd.11.smithi195.stderr:2024-09-13T10:12:31.705+0000 7f442d45b640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 26581) UID: 0 2024-09-13T10:12:31.720 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-09-13T10:12:31.721 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T10:12:31.736 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-09-13T10:12:31.737 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-09-13T10:12:31.758 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-09-13T10:12:31.758 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-09-13T10:12:31.774 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-09-13T10:12:31.775 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-09-13T10:12:31.795 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-09-13T10:12:31.795 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-09-13T10:12:31.811 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-09-13T10:12:31.811 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-09-13T10:12:31.812 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-09-13T10:12:31.805+0000 7f3e27d00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 26583) UID: 0 2024-09-13T10:12:31.831 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-09-13T10:12:31.831 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-09-13T10:12:31.850 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-09-13T10:12:31.850 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-09-13T10:12:31.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:12:31.850+0000 7f3162d9d640 0 -- 172.21.15.73:0/2494127428 >> v2:172.21.15.195:6805/848686366 conn(0x7f31642259b0 msgr2=0x7f3164227da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2024-09-13T10:12:31.857 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:31.857 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:31.871 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T10:12:31.872 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-09-13T10:12:31.872 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-09-13T10:12:31.912 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-13T10:12:31.910+0000 7f3ed56cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 26446) UID: 0 2024-09-13T10:12:31.958 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-09-13T10:12:31.958 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T10:12:31.958 INFO:tasks.thrashosds.thrasher:check thrash_hosts: in_osds > minin 2024-09-13T10:12:31.958 INFO:tasks.thrashosds.thrasher:Checking the number of in OSDs in host ubuntu@smithi181.front.sepia.ceph.com 2024-09-13T10:12:31.958 INFO:tasks.thrashosds.thrasher:Removing all OSDs in host ubuntu@smithi181.front.sepia.ceph.com 2024-09-13T10:12:31.958 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] 2024-09-13T10:12:31.958 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-13T10:12:31.975 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'unique_pool_0' removed 2024-09-13T10:12:31.992 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T10:12:31.993 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:31.993 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:31.998 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-13T10:12:32.007 INFO:tasks.thrashosds:joining thrashosds 2024-09-13T10:12:32.008 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T10:12:32.134 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:32.135 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:32.151 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T10:12:32.273 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:32.273 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:32.275 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-13T10:12:32.274+0000 7f57be0c0640 -1 mon.a@0(leader).osd e123 definitely_dead 0 2024-09-13T10:12:32.287 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-09-13T10:12:32.406 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:32.406 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:32.420 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-09-13T10:12:32.541 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:32.541 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:32.555 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-09-13T10:12:32.677 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:32.678 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:32.691 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-09-13T10:12:32.718 INFO:teuthology.orchestra.run.smithi073.stderr:marked out osd.2. 2024-09-13T10:12:32.718 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-09-13T10:12:32.745 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [0, 1, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] 2024-09-13T10:12:32.746 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 6 2024-09-13T10:12:32.812 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:32.813 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:32.827 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-09-13T10:12:32.948 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:32.949 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:32.963 DEBUG:teuthology.orchestra.run.smithi181:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-09-13T10:12:33.081 INFO:teuthology.orchestra.run.smithi181.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:33.081 INFO:teuthology.orchestra.run.smithi181.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:33.097 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T10:12:33.109 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-13T10:12:33.106+0000 7f57be0c0640 -1 mon.a@0(leader).osd e124 definitely_dead 0 2024-09-13T10:12:33.228 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:33.229 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:33.243 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T10:12:33.368 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:33.368 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:33.382 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T10:12:33.502 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:33.502 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:33.517 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T10:12:33.634 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:33.634 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:33.648 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T10:12:33.727 INFO:teuthology.orchestra.run.smithi073.stderr:marked out osd.6. 2024-09-13T10:12:33.754 INFO:tasks.thrashosds.thrasher:Removing osd 10, in_osds are: [0, 1, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15] 2024-09-13T10:12:33.754 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 10 2024-09-13T10:12:33.769 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:33.769 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:33.784 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T10:12:33.901 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:33.901 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:33.915 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-09-13T10:12:34.037 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:34.037 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:34.053 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-09-13T10:12:34.084 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-13T10:12:34.078+0000 7f57be0c0640 -1 mon.a@0(leader).osd e125 definitely_dead 0 2024-09-13T10:12:34.175 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:34.176 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:34.190 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-09-13T10:12:34.306 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:34.307 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:34.321 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-09-13T10:12:34.444 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:34.445 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:34.459 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-09-13T10:12:34.580 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:34.580 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:34.595 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-09-13T10:12:34.717 INFO:teuthology.orchestra.run.smithi195.stdout:ERROR: (22) Invalid argument 2024-09-13T10:12:34.718 INFO:teuthology.orchestra.run.smithi195.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T10:12:34.729 INFO:teuthology.orchestra.run.smithi073.stderr:marked out osd.10. 2024-09-13T10:12:34.748 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T10:12:34.757 INFO:tasks.thrashosds.thrasher:Removing osd 14, in_osds are: [0, 1, 3, 4, 5, 7, 8, 9, 11, 12, 13, 14, 15] 2024-09-13T10:12:34.758 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 14 2024-09-13T10:12:35.121 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-13T10:12:35.118+0000 7f57be0c0640 -1 mon.a@0(leader).osd e126 definitely_dead 0 2024-09-13T10:12:35.739 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-09-13T10:12:35.740 INFO:teuthology.orchestra.run.smithi073.stderr:marked out osd.14. 2024-09-13T10:12:35.763 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T10:12:36.742 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-09-13T10:12:40.770 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-13T10:12:40.771 INFO:tasks.thrashosds.thrasher:Adding osd 14 2024-09-13T10:12:40.771 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 14 2024-09-13T10:12:41.091 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-13T10:12:41.089+0000 7f57be0c0640 -1 mon.a@0(leader).osd e130 definitely_dead 0 2024-09-13T10:12:42.031 INFO:teuthology.orchestra.run.smithi073.stderr:marked in osd.14. 2024-09-13T10:12:42.054 INFO:tasks.thrashosds.thrasher:Added osd 14 2024-09-13T10:12:42.054 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-13T10:12:42.055 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-13T10:12:42.055 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-13T10:12:42.362 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-13T10:12:42.357+0000 7f57be0c0640 -1 mon.a@0(leader).osd e131 definitely_dead 0 2024-09-13T10:12:42.722 INFO:teuthology.orchestra.run.smithi073.stderr:marked in osd.2. 2024-09-13T10:12:42.752 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-13T10:12:42.752 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-13T10:12:42.752 INFO:tasks.thrashosds.thrasher:Adding osd 10 2024-09-13T10:12:42.752 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 10 2024-09-13T10:12:43.057 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-13T10:12:43.053+0000 7f57be0c0640 -1 mon.a@0(leader).osd e132 definitely_dead 0 2024-09-13T10:12:43.739 INFO:teuthology.orchestra.run.smithi073.stderr:marked in osd.10. 2024-09-13T10:12:43.768 INFO:tasks.thrashosds.thrasher:Added osd 10 2024-09-13T10:12:43.768 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-13T10:12:43.768 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-09-13T10:12:43.768 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 6 2024-09-13T10:12:44.085 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-13T10:12:44.081+0000 7f57be0c0640 -1 mon.a@0(leader).osd e133 definitely_dead 0 2024-09-13T10:12:44.741 INFO:teuthology.orchestra.run.smithi073.stderr:marked in osd.6. 2024-09-13T10:12:44.766 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-09-13T10:12:44.766 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-13T10:12:44.905 INFO:teuthology.orchestra.run.smithi073.stdout:mon.a: {} 2024-09-13T10:12:44.905 INFO:teuthology.orchestra.run.smithi073.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T10:12:44.919 INFO:teuthology.orchestra.run.smithi073.stdout:mon.b: {} 2024-09-13T10:12:44.919 INFO:teuthology.orchestra.run.smithi073.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T10:12:44.933 INFO:teuthology.orchestra.run.smithi073.stdout:mon.c: {} 2024-09-13T10:12:44.933 INFO:teuthology.orchestra.run.smithi073.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T10:12:44.949 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T10:12:45.750 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T10:12:45.775 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-13T10:12:47.049 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-13T10:12:47.072 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T10:12:48.054 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T10:12:48.081 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-13T10:12:49.063 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-13T10:12:49.090 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-09-13T10:12:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.8 to 1 (10000) 2024-09-13T10:12:50.095 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 8 1 2024-09-13T10:12:51.070 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.8 primary-affinity to 1 (8655362) 2024-09-13T10:12:51.097 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-09-13T10:12:52.087 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.12 to 1 (10000) 2024-09-13T10:12:52.114 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 12 1 2024-09-13T10:12:52.728 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.12 primary-affinity to 1 (8655362) 2024-09-13T10:12:52.755 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T10:12:53.731 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T10:12:53.757 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-13T10:12:54.742 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-13T10:12:54.765 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T10:12:55.752 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T10:12:55.779 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-13T10:12:56.761 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-13T10:12:56.787 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-09-13T10:12:57.763 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.9 to 1 (10000) 2024-09-13T10:12:57.791 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 9 1 2024-09-13T10:12:58.773 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.9 primary-affinity to 1 (8655362) 2024-09-13T10:12:58.797 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-09-13T10:12:59.789 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.13 to 1 (10000) 2024-09-13T10:12:59.814 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 13 1 2024-09-13T10:13:00.792 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.13 primary-affinity to 1 (8655362) 2024-09-13T10:13:00.818 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T10:13:01.810 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T10:13:01.839 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-13T10:13:02.733 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-13T10:13:02.762 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T10:13:03.744 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T10:13:03.768 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-13T10:13:04.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-13T10:13:04.101+0000 7f58a6ffd640 0 --2- 172.21.15.73:0/3268868129 >> v2:172.21.15.73:3300/0 conn(0x7f58a806edb0 0x7f58a80c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T10:13:04.829 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-13T10:13:04.854 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-09-13T10:13:05.837 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.10 to 1 (10000) 2024-09-13T10:13:05.864 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 10 1 2024-09-13T10:13:06.840 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.10 primary-affinity to 1 (8655362) 2024-09-13T10:13:06.866 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-09-13T10:13:07.858 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.14 to 1 (10000) 2024-09-13T10:13:07.882 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 14 1 2024-09-13T10:13:08.868 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.14 primary-affinity to 1 (8655362) 2024-09-13T10:13:08.896 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T10:13:09.878 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T10:13:09.902 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-13T10:13:10.887 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-13T10:13:10.912 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T10:13:11.896 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T10:13:11.920 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-13T10:13:12.738 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-13T10:13:12.761 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-09-13T10:13:13.741 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.11 to 1 (10000) 2024-09-13T10:13:13.765 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 11 1 2024-09-13T10:13:14.917 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.11 primary-affinity to 1 (8655362) 2024-09-13T10:13:14.944 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-09-13T10:13:15.926 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.15 to 1 (10000) 2024-09-13T10:13:15.949 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 15 1 2024-09-13T10:13:16.929 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.15 primary-affinity to 1 (8655362) 2024-09-13T10:13:16.954 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-13T10:13:16.954 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-13T10:13:16.954 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-13T10:13:16.954 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-13T10:13:16.954 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T10:13:16.955 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T10:13:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:13:17.270 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":166,"fsid":"c1e91d47-9492-4461-a043-8698fd74ccf3","created":"2024-09-13T10:10:14.019912+0000","modified":"2024-09-13T10:13:16.925520+0000","last_up_change":"2024-09-13T10:10:28.089885+0000","last_in_change":"2024-09-13T10:12:44.738215+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T10:10:28.924860+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":27,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":0.68999999761581421,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T10:10:28.972519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.789999961853027,"score_stable":15.789999961853027,"optimal_score":0.18999999761581421,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"low_tier","create_time":"2024-09-13T10:10:41.320719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7d92188-c27c-4744-a330-4b7105e0f2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1977392321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":1977392321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6815","nonce":1977392321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1977392321}]},"public_addr":"172.21.15.73:6812/1977392321","cluster_addr":"172.21.15.73:6813/1977392321","heartbeat_back_addr":"172.21.15.73:6815/1977392321","heartbeat_front_addr":"172.21.15.73:6814/1977392321","state":["exists","up"]},{"osd":1,"uuid":"e900b771-7c96-4b5d-93d7-35a8ea6c026a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1228102864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6809","nonce":1228102864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6811","nonce":1228102864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1228102864}]},"public_addr":"172.21.15.136:6808/1228102864","cluster_addr":"172.21.15.136:6809/1228102864","heartbeat_back_addr":"172.21.15.136:6811/1228102864","heartbeat_front_addr":"172.21.15.136:6810/1228102864","state":["exists","up"]},{"osd":2,"uuid":"28b7aa59-33b2-4e24-8f3d-758f90e25f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3314820936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":3314820936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6803","nonce":3314820936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3314820936}]},"public_addr":"172.21.15.181:6800/3314820936","cluster_addr":"172.21.15.181:6801/3314820936","heartbeat_back_addr":"172.21.15.181:6803/3314820936","heartbeat_front_addr":"172.21.15.181:6802/3314820936","state":["exists","up"]},{"osd":3,"uuid":"a93d2df5-781e-49e1-a507-1718b6f954ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6801","nonce":2558565460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2558565460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2558565460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":2558565460}]},"public_addr":"172.21.15.195:6801/2558565460","cluster_addr":"172.21.15.195:6802/2558565460","heartbeat_back_addr":"172.21.15.195:6804/2558565460","heartbeat_front_addr":"172.21.15.195:6803/2558565460","state":["exists","up"]},{"osd":4,"uuid":"e69053fd-f6d9-437e-a9d0-8896de6b927f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":858330493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6801","nonce":858330493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6803","nonce":858330493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":858330493}]},"public_addr":"172.21.15.73:6800/858330493","cluster_addr":"172.21.15.73:6801/858330493","heartbeat_back_addr":"172.21.15.73:6803/858330493","heartbeat_front_addr":"172.21.15.73:6802/858330493","state":["exists","up"]},{"osd":5,"uuid":"3667ec10-561e-4a23-8197-6357db4af315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":984313481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6805","nonce":984313481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6807","nonce":984313481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":984313481}]},"public_addr":"172.21.15.136:6804/984313481","cluster_addr":"172.21.15.136:6805/984313481","heartbeat_back_addr":"172.21.15.136:6807/984313481","heartbeat_front_addr":"172.21.15.136:6806/984313481","state":["exists","up"]},{"osd":6,"uuid":"30d4d63c-f04f-4423-91a6-816a7cbc0287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1505939978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":1505939978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":1505939978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1505939978}]},"public_addr":"172.21.15.181:6812/1505939978","cluster_addr":"172.21.15.181:6813/1505939978","heartbeat_back_addr":"172.21.15.181:6815/1505939978","heartbeat_front_addr":"172.21.15.181:6814/1505939978","state":["exists","up"]},{"osd":7,"uuid":"48812f05-82f1-4263-8ec5-4861a6e91b61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":3679292757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3679292757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3679292757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6811","nonce":3679292757}]},"public_addr":"172.21.15.195:6809/3679292757","cluster_addr":"172.21.15.195:6810/3679292757","heartbeat_back_addr":"172.21.15.195:6812/3679292757","heartbeat_front_addr":"172.21.15.195:6811/3679292757","state":["exists","up"]},{"osd":8,"uuid":"72056e61-d04b-4fd3-94bf-57ee3df4b466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4052106885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6805","nonce":4052106885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":4052106885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4052106885}]},"public_addr":"172.21.15.73:6804/4052106885","cluster_addr":"172.21.15.73:6805/4052106885","heartbeat_back_addr":"172.21.15.73:6807/4052106885","heartbeat_front_addr":"172.21.15.73:6806/4052106885","state":["exists","up"]},{"osd":9,"uuid":"20e7f118-7148-4c27-9ec4-a33954709515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4266571820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6801","nonce":4266571820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6803","nonce":4266571820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4266571820}]},"public_addr":"172.21.15.136:6800/4266571820","cluster_addr":"172.21.15.136:6801/4266571820","heartbeat_back_addr":"172.21.15.136:6803/4266571820","heartbeat_front_addr":"172.21.15.136:6802/4266571820","state":["exists","up"]},{"osd":10,"uuid":"012409e8-51c2-4584-aba7-95996f0d72ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":958255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":958255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":958255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":958255259}]},"public_addr":"172.21.15.181:6808/958255259","cluster_addr":"172.21.15.181:6809/958255259","heartbeat_back_addr":"172.21.15.181:6811/958255259","heartbeat_front_addr":"172.21.15.181:6810/958255259","state":["exists","up"]},{"osd":11,"uuid":"88b137b1-7d30-45f7-ac5c-ba026704ef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":281884267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":281884267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":281884267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6815","nonce":281884267}]},"public_addr":"172.21.15.195:6813/281884267","cluster_addr":"172.21.15.195:6814/281884267","heartbeat_back_addr":"172.21.15.195:6816/281884267","heartbeat_front_addr":"172.21.15.195:6815/281884267","state":["exists","up"]},{"osd":12,"uuid":"b54732a2-9dff-42e1-b4ef-281cbca3ad7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3532043806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6809","nonce":3532043806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6811","nonce":3532043806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3532043806}]},"public_addr":"172.21.15.73:6808/3532043806","cluster_addr":"172.21.15.73:6809/3532043806","heartbeat_back_addr":"172.21.15.73:6811/3532043806","heartbeat_front_addr":"172.21.15.73:6810/3532043806","state":["exists","up"]},{"osd":13,"uuid":"20b48864-44dc-4732-8dff-d0d21cc08032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":4083982131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":4083982131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6815","nonce":4083982131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":4083982131}]},"public_addr":"172.21.15.136:6812/4083982131","cluster_addr":"172.21.15.136:6813/4083982131","heartbeat_back_addr":"172.21.15.136:6815/4083982131","heartbeat_front_addr":"172.21.15.136:6814/4083982131","state":["exists","up"]},{"osd":14,"uuid":"9fac3ef4-e9be-4e8b-9a21-0de35b543d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1173379175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":1173379175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":1173379175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1173379175}]},"public_addr":"172.21.15.181:6804/1173379175","cluster_addr":"172.21.15.181:6805/1173379175","heartbeat_back_addr":"172.21.15.181:6807/1173379175","heartbeat_front_addr":"172.21.15.181:6806/1173379175","state":["exists","up"]},{"osd":15,"uuid":"16b34f90-3148-4d41-a403-ba0f900991e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6805","nonce":848686366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":848686366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":848686366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":848686366}]},"public_addr":"172.21.15.195:6805/848686366","cluster_addr":"172.21.15.195:6806/848686366","heartbeat_back_addr":"172.21.15.195:6808/848686366","heartbeat_front_addr":"172.21.15.195:6807/848686366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:08.968353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:00.403159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:51.688552+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:12.174284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:01.221496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:11.382273+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:59.207758+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:02.780540+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:01.246971+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:47.636366+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:52.175110+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:51.721988+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:51.157861+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:42.954410+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:58.881999+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:54.838146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"3.1","osds":[13,14,10]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T10:13:17.290 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T10:13:17.290 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T10:13:17.592 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:13:17.592 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":166,"fsid":"c1e91d47-9492-4461-a043-8698fd74ccf3","created":"2024-09-13T10:10:14.019912+0000","modified":"2024-09-13T10:13:16.925520+0000","last_up_change":"2024-09-13T10:10:28.089885+0000","last_in_change":"2024-09-13T10:12:44.738215+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T10:10:28.924860+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":27,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":0.68999999761581421,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T10:10:28.972519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.789999961853027,"score_stable":15.789999961853027,"optimal_score":0.18999999761581421,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"low_tier","create_time":"2024-09-13T10:10:41.320719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7d92188-c27c-4744-a330-4b7105e0f2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1977392321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":1977392321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6815","nonce":1977392321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1977392321}]},"public_addr":"172.21.15.73:6812/1977392321","cluster_addr":"172.21.15.73:6813/1977392321","heartbeat_back_addr":"172.21.15.73:6815/1977392321","heartbeat_front_addr":"172.21.15.73:6814/1977392321","state":["exists","up"]},{"osd":1,"uuid":"e900b771-7c96-4b5d-93d7-35a8ea6c026a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1228102864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6809","nonce":1228102864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6811","nonce":1228102864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1228102864}]},"public_addr":"172.21.15.136:6808/1228102864","cluster_addr":"172.21.15.136:6809/1228102864","heartbeat_back_addr":"172.21.15.136:6811/1228102864","heartbeat_front_addr":"172.21.15.136:6810/1228102864","state":["exists","up"]},{"osd":2,"uuid":"28b7aa59-33b2-4e24-8f3d-758f90e25f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3314820936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":3314820936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6803","nonce":3314820936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3314820936}]},"public_addr":"172.21.15.181:6800/3314820936","cluster_addr":"172.21.15.181:6801/3314820936","heartbeat_back_addr":"172.21.15.181:6803/3314820936","heartbeat_front_addr":"172.21.15.181:6802/3314820936","state":["exists","up"]},{"osd":3,"uuid":"a93d2df5-781e-49e1-a507-1718b6f954ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6801","nonce":2558565460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2558565460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2558565460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":2558565460}]},"public_addr":"172.21.15.195:6801/2558565460","cluster_addr":"172.21.15.195:6802/2558565460","heartbeat_back_addr":"172.21.15.195:6804/2558565460","heartbeat_front_addr":"172.21.15.195:6803/2558565460","state":["exists","up"]},{"osd":4,"uuid":"e69053fd-f6d9-437e-a9d0-8896de6b927f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":858330493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6801","nonce":858330493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6803","nonce":858330493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":858330493}]},"public_addr":"172.21.15.73:6800/858330493","cluster_addr":"172.21.15.73:6801/858330493","heartbeat_back_addr":"172.21.15.73:6803/858330493","heartbeat_front_addr":"172.21.15.73:6802/858330493","state":["exists","up"]},{"osd":5,"uuid":"3667ec10-561e-4a23-8197-6357db4af315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":984313481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6805","nonce":984313481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6807","nonce":984313481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":984313481}]},"public_addr":"172.21.15.136:6804/984313481","cluster_addr":"172.21.15.136:6805/984313481","heartbeat_back_addr":"172.21.15.136:6807/984313481","heartbeat_front_addr":"172.21.15.136:6806/984313481","state":["exists","up"]},{"osd":6,"uuid":"30d4d63c-f04f-4423-91a6-816a7cbc0287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1505939978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":1505939978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":1505939978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1505939978}]},"public_addr":"172.21.15.181:6812/1505939978","cluster_addr":"172.21.15.181:6813/1505939978","heartbeat_back_addr":"172.21.15.181:6815/1505939978","heartbeat_front_addr":"172.21.15.181:6814/1505939978","state":["exists","up"]},{"osd":7,"uuid":"48812f05-82f1-4263-8ec5-4861a6e91b61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":3679292757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3679292757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3679292757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6811","nonce":3679292757}]},"public_addr":"172.21.15.195:6809/3679292757","cluster_addr":"172.21.15.195:6810/3679292757","heartbeat_back_addr":"172.21.15.195:6812/3679292757","heartbeat_front_addr":"172.21.15.195:6811/3679292757","state":["exists","up"]},{"osd":8,"uuid":"72056e61-d04b-4fd3-94bf-57ee3df4b466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4052106885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6805","nonce":4052106885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":4052106885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4052106885}]},"public_addr":"172.21.15.73:6804/4052106885","cluster_addr":"172.21.15.73:6805/4052106885","heartbeat_back_addr":"172.21.15.73:6807/4052106885","heartbeat_front_addr":"172.21.15.73:6806/4052106885","state":["exists","up"]},{"osd":9,"uuid":"20e7f118-7148-4c27-9ec4-a33954709515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4266571820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6801","nonce":4266571820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6803","nonce":4266571820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4266571820}]},"public_addr":"172.21.15.136:6800/4266571820","cluster_addr":"172.21.15.136:6801/4266571820","heartbeat_back_addr":"172.21.15.136:6803/4266571820","heartbeat_front_addr":"172.21.15.136:6802/4266571820","state":["exists","up"]},{"osd":10,"uuid":"012409e8-51c2-4584-aba7-95996f0d72ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":958255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":958255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":958255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":958255259}]},"public_addr":"172.21.15.181:6808/958255259","cluster_addr":"172.21.15.181:6809/958255259","heartbeat_back_addr":"172.21.15.181:6811/958255259","heartbeat_front_addr":"172.21.15.181:6810/958255259","state":["exists","up"]},{"osd":11,"uuid":"88b137b1-7d30-45f7-ac5c-ba026704ef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":281884267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":281884267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":281884267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6815","nonce":281884267}]},"public_addr":"172.21.15.195:6813/281884267","cluster_addr":"172.21.15.195:6814/281884267","heartbeat_back_addr":"172.21.15.195:6816/281884267","heartbeat_front_addr":"172.21.15.195:6815/281884267","state":["exists","up"]},{"osd":12,"uuid":"b54732a2-9dff-42e1-b4ef-281cbca3ad7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3532043806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6809","nonce":3532043806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6811","nonce":3532043806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3532043806}]},"public_addr":"172.21.15.73:6808/3532043806","cluster_addr":"172.21.15.73:6809/3532043806","heartbeat_back_addr":"172.21.15.73:6811/3532043806","heartbeat_front_addr":"172.21.15.73:6810/3532043806","state":["exists","up"]},{"osd":13,"uuid":"20b48864-44dc-4732-8dff-d0d21cc08032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":4083982131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":4083982131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6815","nonce":4083982131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":4083982131}]},"public_addr":"172.21.15.136:6812/4083982131","cluster_addr":"172.21.15.136:6813/4083982131","heartbeat_back_addr":"172.21.15.136:6815/4083982131","heartbeat_front_addr":"172.21.15.136:6814/4083982131","state":["exists","up"]},{"osd":14,"uuid":"9fac3ef4-e9be-4e8b-9a21-0de35b543d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1173379175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":1173379175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":1173379175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1173379175}]},"public_addr":"172.21.15.181:6804/1173379175","cluster_addr":"172.21.15.181:6805/1173379175","heartbeat_back_addr":"172.21.15.181:6807/1173379175","heartbeat_front_addr":"172.21.15.181:6806/1173379175","state":["exists","up"]},{"osd":15,"uuid":"16b34f90-3148-4d41-a403-ba0f900991e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6805","nonce":848686366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":848686366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":848686366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":848686366}]},"public_addr":"172.21.15.195:6805/848686366","cluster_addr":"172.21.15.195:6806/848686366","heartbeat_back_addr":"172.21.15.195:6808/848686366","heartbeat_front_addr":"172.21.15.195:6807/848686366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:08.968353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:00.403159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:51.688552+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:12.174284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:01.221496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:11.382273+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:59.207758+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:02.780540+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:01.246971+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:47.636366+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:52.175110+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:51.721988+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:51.157861+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:42.954410+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:58.881999+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:54.838146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"3.1","osds":[13,14,10]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T10:13:17.610 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T10:13:17.611 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T10:13:17.611 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T10:13:17.612 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T10:13:17.612 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T10:13:17.613 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T10:13:17.613 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T10:13:17.613 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T10:13:17.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-13T10:13:17.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-13T10:13:17.615 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-09-13T10:13:17.615 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-09-13T10:13:17.616 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-09-13T10:13:17.616 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-09-13T10:13:17.616 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-09-13T10:13:17.617 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-09-13T10:13:18.038 INFO:teuthology.orchestra.run.smithi073.stdout:98784247844 2024-09-13T10:13:18.038 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T10:13:18.092 INFO:teuthology.orchestra.run.smithi073.stdout:94489280549 2024-09-13T10:13:18.092 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T10:13:18.106 INFO:teuthology.orchestra.run.smithi073.stdout:94489280550 2024-09-13T10:13:18.106 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T10:13:18.107 INFO:teuthology.orchestra.run.smithi073.stdout:98784247845 2024-09-13T10:13:18.108 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-09-13T10:13:18.110 INFO:teuthology.orchestra.run.smithi073.stdout:98784247845 2024-09-13T10:13:18.110 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T10:13:18.128 INFO:teuthology.orchestra.run.smithi073.stdout:94489280550 2024-09-13T10:13:18.128 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-13T10:13:18.134 INFO:teuthology.orchestra.run.smithi073.stdout:94489280550 2024-09-13T10:13:18.135 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-09-13T10:13:18.139 INFO:teuthology.orchestra.run.smithi073.stdout:98784247845 2024-09-13T10:13:18.140 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T10:13:18.147 INFO:teuthology.orchestra.run.smithi073.stdout:94489280550 2024-09-13T10:13:18.148 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T10:13:18.151 INFO:teuthology.orchestra.run.smithi073.stdout:94489280550 2024-09-13T10:13:18.152 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T10:13:18.157 INFO:teuthology.orchestra.run.smithi073.stdout:90194313254 2024-09-13T10:13:18.158 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-09-13T10:13:18.159 INFO:teuthology.orchestra.run.smithi073.stdout:94489280551 2024-09-13T10:13:18.160 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T10:13:18.172 INFO:teuthology.orchestra.run.smithi073.stdout:90194313255 2024-09-13T10:13:18.172 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-13T10:13:18.184 INFO:teuthology.orchestra.run.smithi073.stdout:98784247847 2024-09-13T10:13:18.184 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-09-13T10:13:18.190 INFO:teuthology.orchestra.run.smithi073.stdout:94489280550 2024-09-13T10:13:18.191 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-13T10:13:18.200 INFO:teuthology.orchestra.run.smithi073.stdout:98784247847 2024-09-13T10:13:18.201 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-13T10:13:18.998 INFO:teuthology.orchestra.run.smithi073.stdout:94489280547 2024-09-13T10:13:19.038 INFO:teuthology.orchestra.run.smithi073.stdout:98784247843 2024-09-13T10:13:19.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280549 got 94489280547 for osd.5 2024-09-13T10:13:19.049 INFO:teuthology.orchestra.run.smithi073.stdout:98784247844 2024-09-13T10:13:19.073 INFO:teuthology.orchestra.run.smithi073.stdout:94489280548 2024-09-13T10:13:19.087 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247844 got 98784247843 for osd.6 2024-09-13T10:13:19.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247845 got 98784247844 for osd.14 2024-09-13T10:13:19.127 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280550 got 94489280548 for osd.11 2024-09-13T10:13:19.132 INFO:teuthology.orchestra.run.smithi073.stdout:94489280548 2024-09-13T10:13:19.164 INFO:teuthology.orchestra.run.smithi073.stdout:98784247844 2024-09-13T10:13:19.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280550 got 94489280548 for osd.1 2024-09-13T10:13:19.177 INFO:teuthology.orchestra.run.smithi073.stdout:90194313253 2024-09-13T10:13:19.199 INFO:teuthology.orchestra.run.smithi073.stdout:94489280548 2024-09-13T10:13:19.209 INFO:teuthology.orchestra.run.smithi073.stdout:90194313254 2024-09-13T10:13:19.209 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247845 got 98784247844 for osd.0 2024-09-13T10:13:19.220 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313254 got 90194313253 for osd.13 2024-09-13T10:13:19.239 INFO:teuthology.orchestra.run.smithi073.stdout:98784247846 2024-09-13T10:13:19.242 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313255 got 90194313254 for osd.9 2024-09-13T10:13:19.243 INFO:teuthology.orchestra.run.smithi073.stdout:94489280548 2024-09-13T10:13:19.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280550 got 94489280548 for osd.3 2024-09-13T10:13:19.265 INFO:teuthology.orchestra.run.smithi073.stdout:94489280549 2024-09-13T10:13:19.269 INFO:teuthology.orchestra.run.smithi073.stdout:98784247846 2024-09-13T10:13:19.273 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247847 got 98784247846 for osd.8 2024-09-13T10:13:19.275 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280550 got 94489280548 for osd.7 2024-09-13T10:13:19.275 INFO:teuthology.orchestra.run.smithi073.stdout:94489280548 2024-09-13T10:13:19.292 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247847 got 98784247846 for osd.12 2024-09-13T10:13:19.295 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280551 got 94489280549 for osd.4 2024-09-13T10:13:19.303 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280550 got 94489280548 for osd.15 2024-09-13T10:13:19.305 INFO:teuthology.orchestra.run.smithi073.stdout:98784247844 2024-09-13T10:13:19.308 INFO:teuthology.orchestra.run.smithi073.stdout:94489280548 2024-09-13T10:13:19.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247845 got 98784247844 for osd.2 2024-09-13T10:13:19.327 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280550 got 94489280548 for osd.10 2024-09-13T10:13:20.050 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T10:13:20.088 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T10:13:20.120 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-09-13T10:13:20.128 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-09-13T10:13:20.178 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T10:13:20.210 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T10:13:20.221 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-09-13T10:13:20.243 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-13T10:13:20.266 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T10:13:20.274 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-13T10:13:20.275 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T10:13:20.293 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-09-13T10:13:20.296 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T10:13:20.304 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-09-13T10:13:20.326 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T10:13:20.327 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-09-13T10:13:20.685 INFO:teuthology.orchestra.run.smithi073.stdout:98784247845 2024-09-13T10:13:20.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247844 got 98784247845 for osd.6 2024-09-13T10:13:20.764 DEBUG:teuthology.parallel:result is None 2024-09-13T10:13:20.767 INFO:teuthology.orchestra.run.smithi073.stdout:94489280549 2024-09-13T10:13:20.797 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280549 got 94489280549 for osd.5 2024-09-13T10:13:20.797 DEBUG:teuthology.parallel:result is None 2024-09-13T10:13:20.914 INFO:teuthology.orchestra.run.smithi073.stdout:94489280550 2024-09-13T10:13:20.960 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280550 got 94489280550 for osd.11 2024-09-13T10:13:20.960 DEBUG:teuthology.parallel:result is None 2024-09-13T10:13:20.994 INFO:teuthology.orchestra.run.smithi073.stdout:98784247845 2024-09-13T10:13:21.046 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247845 got 98784247845 for osd.14 2024-09-13T10:13:21.046 DEBUG:teuthology.parallel:result is None 2024-09-13T10:13:21.100 INFO:teuthology.orchestra.run.smithi073.stdout:90194313254 2024-09-13T10:13:21.155 INFO:teuthology.orchestra.run.smithi073.stdout:94489280550 2024-09-13T10:13:21.156 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313254 got 90194313254 for osd.13 2024-09-13T10:13:21.157 DEBUG:teuthology.parallel:result is None 2024-09-13T10:13:21.172 INFO:teuthology.orchestra.run.smithi073.stdout:90194313255 2024-09-13T10:13:21.177 INFO:teuthology.orchestra.run.smithi073.stdout:98784247845 2024-09-13T10:13:21.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280550 got 94489280550 for osd.1 2024-09-13T10:13:21.187 DEBUG:teuthology.parallel:result is None 2024-09-13T10:13:21.209 INFO:teuthology.orchestra.run.smithi073.stdout:94489280550 2024-09-13T10:13:21.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313255 got 90194313255 for osd.9 2024-09-13T10:13:21.214 DEBUG:teuthology.parallel:result is None 2024-09-13T10:13:21.216 INFO:teuthology.orchestra.run.smithi073.stdout:98784247847 2024-09-13T10:13:21.220 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247845 got 98784247845 for osd.0 2024-09-13T10:13:21.220 DEBUG:teuthology.parallel:result is None 2024-09-13T10:13:21.242 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280550 got 94489280550 for osd.7 2024-09-13T10:13:21.242 DEBUG:teuthology.parallel:result is None 2024-09-13T10:13:21.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247847 got 98784247847 for osd.12 2024-09-13T10:13:21.247 DEBUG:teuthology.parallel:result is None 2024-09-13T10:13:21.253 INFO:teuthology.orchestra.run.smithi073.stdout:94489280550 2024-09-13T10:13:21.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280550 got 94489280550 for osd.15 2024-09-13T10:13:21.270 DEBUG:teuthology.parallel:result is None 2024-09-13T10:13:21.280 INFO:teuthology.orchestra.run.smithi073.stdout:94489280550 2024-09-13T10:13:21.285 INFO:teuthology.orchestra.run.smithi073.stdout:98784247845 2024-09-13T10:13:21.304 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247845 got 98784247845 for osd.2 2024-09-13T10:13:21.304 DEBUG:teuthology.parallel:result is None 2024-09-13T10:13:21.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280550 got 94489280550 for osd.3 2024-09-13T10:13:21.307 DEBUG:teuthology.parallel:result is None 2024-09-13T10:13:21.307 INFO:teuthology.orchestra.run.smithi073.stdout:94489280550 2024-09-13T10:13:21.326 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280550 got 94489280550 for osd.10 2024-09-13T10:13:21.327 DEBUG:teuthology.parallel:result is None 2024-09-13T10:13:21.338 INFO:teuthology.orchestra.run.smithi073.stdout:98784247847 2024-09-13T10:13:21.355 INFO:teuthology.orchestra.run.smithi073.stdout:94489280551 2024-09-13T10:13:21.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 98784247847 got 98784247847 for osd.8 2024-09-13T10:13:21.356 DEBUG:teuthology.parallel:result is None 2024-09-13T10:13:21.373 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280551 got 94489280551 for osd.4 2024-09-13T10:13:21.373 DEBUG:teuthology.parallel:result is None 2024-09-13T10:13:21.373 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T10:13:21.373 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:13:21.697 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:13:21.698 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:13:21.713 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-13T10:13:20.982437+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110070969,"num_objects":152,"num_object_clones":0,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":39,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":1451,"num_read_kb":31320,"num_write":2016,"num_write_kb":368306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":87878062,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":39,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":811232,"kb_used_data":324716,"kb_used_omap":24,"kb_used_meta":486183,"kb_avail":1509138208,"statfs":{"total":1546188226560,"available":1545357524992,"internally_reserved":0,"allocated":332509184,"data_stored":325368261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":497851471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-40,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":16622880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.055503"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":137,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.951736+0000","last_change":"2024-09-13T10:12:52.201492+0000","last_active":"2024-09-13T10:13:17.951736+0000","last_peered":"2024-09-13T10:13:17.951736+0000","last_clean":"2024-09-13T10:13:17.951736+0000","last_became_active":"2024-09-13T10:12:52.200952+0000","last_became_peered":"2024-09-13T10:12:52.200952+0000","last_unstale":"2024-09-13T10:13:17.951736+0000","last_undegraded":"2024-09-13T10:13:17.951736+0000","last_fullsized":"2024-09-13T10:13:17.951736+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:03.961779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:03.961779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":305,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.076563+0000","last_change":"2024-09-13T10:12:12.892985+0000","last_active":"2024-09-13T10:13:17.076563+0000","last_peered":"2024-09-13T10:13:17.076563+0000","last_clean":"2024-09-13T10:13:17.076563+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:13:17.076563+0000","last_undegraded":"2024-09-13T10:13:17.076563+0000","last_fullsized":"2024-09-13T10:13:17.076563+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:12.892836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:12.892836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:20.216623+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":318,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.302566+0000","last_change":"2024-09-13T10:12:45.768853+0000","last_active":"2024-09-13T10:13:17.302566+0000","last_peered":"2024-09-13T10:13:17.302566+0000","last_clean":"2024-09-13T10:13:17.302566+0000","last_became_active":"2024-09-13T10:12:45.768237+0000","last_became_peered":"2024-09-13T10:12:45.768237+0000","last_unstale":"2024-09-13T10:13:17.302566+0000","last_undegraded":"2024-09-13T10:13:17.302566+0000","last_fullsized":"2024-09-13T10:13:17.302566+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:01.407274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:01.407274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:23.291168+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":30,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:18.729021+0000","last_change":"2024-09-13T10:13:09.126344+0000","last_active":"2024-09-13T10:13:18.729021+0000","last_peered":"2024-09-13T10:13:18.729021+0000","last_clean":"2024-09-13T10:13:18.729021+0000","last_became_active":"2024-09-13T10:13:09.125529+0000","last_became_peered":"2024-09-13T10:13:09.125529+0000","last_unstale":"2024-09-13T10:13:18.729021+0000","last_undegraded":"2024-09-13T10:13:18.729021+0000","last_fullsized":"2024-09-13T10:13:18.729021+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:50.136244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:50.136244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:14.321249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"119'332","reported_seq":849,"reported_epoch":166,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-13T10:13:17.076461+0000","last_change":"2024-09-13T10:12:50.227345+0000","last_active":"2024-09-13T10:13:17.076461+0000","last_peered":"2024-09-13T10:13:17.076461+0000","last_clean":"2024-09-13T10:12:47.052582+0000","last_became_active":"2024-09-13T10:12:50.225448+0000","last_became_peered":"2024-09-13T10:12:50.225448+0000","last_unstale":"2024-09-13T10:13:17.076461+0000","last_undegraded":"2024-09-13T10:13:17.076461+0000","last_fullsized":"2024-09-13T10:13:17.076461+0000","mapping_epoch":138,"log_start":"118'329","ondisk_log_start":"118'329","created":30,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'332","last_scrub_stamp":"2024-09-13T10:12:28.697464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:28.697464+0000","objects_scrubbed":39,"log_size":3,"log_dups_size":344,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28010945,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":39,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":327,"num_read_kb":7023,"num_write":433,"num_write_kb":81577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":54632264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,14,10],"avail_no_missing":["13","10","14"],"object_location_counts":[{"shards":"10,13,14","objects":39}],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":276,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.951731+0000","last_change":"2024-09-13T10:12:58.899445+0000","last_active":"2024-09-13T10:13:17.951731+0000","last_peered":"2024-09-13T10:13:17.951731+0000","last_clean":"2024-09-13T10:13:17.951731+0000","last_became_active":"2024-09-13T10:12:52.099716+0000","last_became_peered":"2024-09-13T10:12:52.099716+0000","last_unstale":"2024-09-13T10:13:17.951731+0000","last_undegraded":"2024-09-13T10:13:17.951731+0000","last_fullsized":"2024-09-13T10:13:17.951731+0000","mapping_epoch":140,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'112","last_scrub_stamp":"2024-09-13T10:12:58.899291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.899291+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:21.742546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":219,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.076415+0000","last_change":"2024-09-13T10:12:53.845086+0000","last_active":"2024-09-13T10:13:17.076415+0000","last_peered":"2024-09-13T10:13:17.076415+0000","last_clean":"2024-09-13T10:13:17.076415+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:13:17.076415+0000","last_undegraded":"2024-09-13T10:13:17.076415+0000","last_fullsized":"2024-09-13T10:13:17.076415+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T10:12:53.844861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:53.844861+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.984156+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"119'365","reported_seq":695,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.076267+0000","last_change":"2024-09-13T10:12:00.967692+0000","last_active":"2024-09-13T10:13:17.076267+0000","last_peered":"2024-09-13T10:13:17.076267+0000","last_clean":"2024-09-13T10:13:17.076267+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:13:17.076267+0000","last_undegraded":"2024-09-13T10:13:17.076267+0000","last_fullsized":"2024-09-13T10:13:17.076267+0000","mapping_epoch":30,"log_start":"118'362","ondisk_log_start":"118'362","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'314","last_scrub_stamp":"2024-09-13T10:12:00.967585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.967585+0000","objects_scrubbed":33,"log_size":3,"log_dups_size":374,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:19.050419+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33541126,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":353,"num_read_kb":13014,"num_write":482,"num_write_kb":101793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":101,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:18.697325+0000","last_change":"2024-09-13T10:12:58.690112+0000","last_active":"2024-09-13T10:13:18.697325+0000","last_peered":"2024-09-13T10:13:18.697325+0000","last_clean":"2024-09-13T10:13:18.697325+0000","last_became_active":"2024-09-13T10:12:45.765925+0000","last_became_peered":"2024-09-13T10:12:45.765925+0000","last_unstale":"2024-09-13T10:13:18.697325+0000","last_undegraded":"2024-09-13T10:13:18.697325+0000","last_fullsized":"2024-09-13T10:13:18.697325+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-13T10:12:58.689917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.689917+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.031308+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"118'346","reported_seq":826,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.958498+0000","last_change":"2024-09-13T10:13:17.957473+0000","last_active":"2024-09-13T10:13:17.958498+0000","last_peered":"2024-09-13T10:13:17.958498+0000","last_clean":"2024-09-13T10:13:17.958498+0000","last_became_active":"2024-09-13T10:13:17.956444+0000","last_became_peered":"2024-09-13T10:13:17.956444+0000","last_unstale":"2024-09-13T10:13:17.958498+0000","last_undegraded":"2024-09-13T10:13:17.958498+0000","last_fullsized":"2024-09-13T10:13:17.958498+0000","mapping_epoch":166,"log_start":"116'344","ondisk_log_start":"116'344","created":30,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'346","last_scrub_stamp":"2024-09-13T10:13:16.741058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:16.741058+0000","objects_scrubbed":40,"log_size":2,"log_dups_size":356,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.865324+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27658528,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":310,"num_read_kb":4757,"num_write":455,"num_write_kb":83526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":33245760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.076448+0000","last_change":"2024-09-13T10:13:14.496339+0000","last_active":"2024-09-13T10:13:17.076448+0000","last_peered":"2024-09-13T10:13:17.076448+0000","last_clean":"2024-09-13T10:13:17.076448+0000","last_became_active":"2024-09-13T10:13:14.495396+0000","last_became_peered":"2024-09-13T10:13:14.495396+0000","last_unstale":"2024-09-13T10:13:17.076448+0000","last_undegraded":"2024-09-13T10:13:17.076448+0000","last_fullsized":"2024-09-13T10:13:17.076448+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:11:53.335756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:53.335756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:21.857139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"119'401","reported_seq":714,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:18.728989+0000","last_change":"2024-09-13T10:12:43.753889+0000","last_active":"2024-09-13T10:13:18.728989+0000","last_peered":"2024-09-13T10:13:18.728989+0000","last_clean":"2024-09-13T10:13:18.728989+0000","last_became_active":"2024-09-13T10:12:43.752738+0000","last_became_peered":"2024-09-13T10:12:43.752738+0000","last_unstale":"2024-09-13T10:13:18.728989+0000","last_undegraded":"2024-09-13T10:13:18.728989+0000","last_fullsized":"2024-09-13T10:13:18.728989+0000","mapping_epoch":132,"log_start":"118'398","ondisk_log_start":"118'398","created":30,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'359","last_scrub_stamp":"2024-09-13T10:12:00.627252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.627252+0000","objects_scrubbed":25,"log_size":3,"log_dups_size":407,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:18.843887+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20269983,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":375,"num_read_kb":6453,"num_write":499,"num_write_kb":98736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.945038+0000","last_change":"2024-09-13T10:13:12.541253+0000","last_active":"2024-09-13T10:13:17.945038+0000","last_peered":"2024-09-13T10:13:17.945038+0000","last_clean":"2024-09-13T10:13:17.945038+0000","last_became_active":"2024-09-13T10:12:45.763788+0000","last_became_peered":"2024-09-13T10:12:45.763788+0000","last_unstale":"2024-09-13T10:13:17.945038+0000","last_undegraded":"2024-09-13T10:13:17.945038+0000","last_fullsized":"2024-09-13T10:13:17.945038+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:12.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:12.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:19.031321+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":109480582,"num_objects":148,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":39,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":1365,"num_read_kb":31247,"num_write":1869,"num_write_kb":365632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":147,"num_bytes_recovered":87878024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":295931904,"data_stored":295197586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":14}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247846,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79548,"kb_used_data":48364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94292292,"statfs":{"total":96636764160,"available":96555307008,"internally_reserved":0,"allocated":49524736,"data_stored":49016492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247848,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":735564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247848,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59972,"kb_used_data":28788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311868,"statfs":{"total":96636764160,"available":96575352832,"internally_reserved":0,"allocated":29478912,"data_stored":28992128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313255,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1134592,"data_stored":735583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313254,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59700,"kb_used_data":28516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312140,"statfs":{"total":96636764160,"available":96575631360,"internally_reserved":0,"allocated":29200384,"data_stored":28746509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247846,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64160,"kb_used_data":32964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94307680,"statfs":{"total":96636764160,"available":96571064320,"internally_reserved":0,"allocated":33755136,"data_stored":33259531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280550,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48592,"kb_used_data":17396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323248,"statfs":{"total":96636764160,"available":96587005952,"internally_reserved":0,"allocated":17813504,"data_stored":17358444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,6,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280550,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85528,"kb_used_data":54344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94286312,"statfs":{"total":96636764160,"available":96549183488,"internally_reserved":0,"allocated":55648256,"data_stored":55129344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,8,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280551,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65112,"kb_used_data":33928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306728,"statfs":{"total":96636764160,"available":96570089472,"internally_reserved":0,"allocated":34742272,"data_stored":34276690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280549,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1118208,"data_stored":727867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280550,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65692,"kb_used_data":34508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306148,"statfs":{"total":96636764160,"available":96569495552,"internally_reserved":0,"allocated":35336192,"data_stored":34867058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280550,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69352,"kb_used_data":38156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302488,"statfs":{"total":96636764160,"available":96565747712,"internally_reserved":0,"allocated":39071744,"data_stored":38565421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":735564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280550,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":1130496,"data_stored":735564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247846,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1146880,"data_stored":743251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247845,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1146880,"data_stored":743251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15941632,"data_stored":15901087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53936128,"data_stored":53811109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":33611776,"data_stored":33541126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33611776,"data_stored":33541126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":27742208,"data_stored":27658528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":37941248,"data_stored":37829857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":28069888,"data_stored":28010945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":48394240,"data_stored":48280928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:13:21.716 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:13:22.002 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:13:22.002 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:13:22.018 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-09-13T10:13:21.979771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110070969,"num_objects":152,"num_object_clones":0,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":1451,"num_read_kb":31320,"num_write":2016,"num_write_kb":368306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":115889007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":39,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":811244,"kb_used_data":324728,"kb_used_omap":24,"kb_used_meta":486183,"kb_avail":1509138196,"statfs":{"total":1546188226560,"available":1545357512704,"internally_reserved":0,"allocated":332521472,"data_stored":325375929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":497851471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-79,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":44633825,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.050420"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":137,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.951736+0000","last_change":"2024-09-13T10:12:52.201492+0000","last_active":"2024-09-13T10:13:17.951736+0000","last_peered":"2024-09-13T10:13:17.951736+0000","last_clean":"2024-09-13T10:13:17.951736+0000","last_became_active":"2024-09-13T10:12:52.200952+0000","last_became_peered":"2024-09-13T10:12:52.200952+0000","last_unstale":"2024-09-13T10:13:17.951736+0000","last_undegraded":"2024-09-13T10:13:17.951736+0000","last_fullsized":"2024-09-13T10:13:17.951736+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:03.961779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:03.961779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":305,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.076563+0000","last_change":"2024-09-13T10:12:12.892985+0000","last_active":"2024-09-13T10:13:17.076563+0000","last_peered":"2024-09-13T10:13:17.076563+0000","last_clean":"2024-09-13T10:13:17.076563+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:13:17.076563+0000","last_undegraded":"2024-09-13T10:13:17.076563+0000","last_fullsized":"2024-09-13T10:13:17.076563+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:12.892836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:12.892836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:20.216623+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":318,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.302566+0000","last_change":"2024-09-13T10:12:45.768853+0000","last_active":"2024-09-13T10:13:17.302566+0000","last_peered":"2024-09-13T10:13:17.302566+0000","last_clean":"2024-09-13T10:13:17.302566+0000","last_became_active":"2024-09-13T10:12:45.768237+0000","last_became_peered":"2024-09-13T10:12:45.768237+0000","last_unstale":"2024-09-13T10:13:17.302566+0000","last_undegraded":"2024-09-13T10:13:17.302566+0000","last_fullsized":"2024-09-13T10:13:17.302566+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:01.407274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:01.407274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:23.291168+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":30,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:18.729021+0000","last_change":"2024-09-13T10:13:09.126344+0000","last_active":"2024-09-13T10:13:18.729021+0000","last_peered":"2024-09-13T10:13:18.729021+0000","last_clean":"2024-09-13T10:13:18.729021+0000","last_became_active":"2024-09-13T10:13:09.125529+0000","last_became_peered":"2024-09-13T10:13:09.125529+0000","last_unstale":"2024-09-13T10:13:18.729021+0000","last_undegraded":"2024-09-13T10:13:18.729021+0000","last_fullsized":"2024-09-13T10:13:18.729021+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:50.136244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:50.136244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:14.321249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"119'332","reported_seq":937,"reported_epoch":167,"state":"peering","last_fresh":"2024-09-13T10:13:20.530384+0000","last_change":"2024-09-13T10:13:20.978575+0000","last_active":"2024-09-13T10:13:20.978575+0000","last_peered":"2024-09-13T10:13:20.530384+0000","last_clean":"2024-09-13T10:13:20.530384+0000","last_became_active":"2024-09-13T10:12:50.225448+0000","last_became_peered":"2024-09-13T10:12:50.225448+0000","last_unstale":"2024-09-13T10:13:20.530384+0000","last_undegraded":"2024-09-13T10:13:20.530384+0000","last_fullsized":"2024-09-13T10:13:20.530384+0000","mapping_epoch":168,"log_start":"118'329","ondisk_log_start":"118'329","created":30,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'332","last_scrub_stamp":"2024-09-13T10:12:28.697464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:28.697464+0000","objects_scrubbed":39,"log_size":3,"log_dups_size":344,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28010945,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":327,"num_read_kb":7023,"num_write":433,"num_write_kb":81577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":82643209,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,4,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,10,14],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":276,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.951731+0000","last_change":"2024-09-13T10:12:58.899445+0000","last_active":"2024-09-13T10:13:17.951731+0000","last_peered":"2024-09-13T10:13:17.951731+0000","last_clean":"2024-09-13T10:13:17.951731+0000","last_became_active":"2024-09-13T10:12:52.099716+0000","last_became_peered":"2024-09-13T10:12:52.099716+0000","last_unstale":"2024-09-13T10:13:17.951731+0000","last_undegraded":"2024-09-13T10:13:17.951731+0000","last_fullsized":"2024-09-13T10:13:17.951731+0000","mapping_epoch":140,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'112","last_scrub_stamp":"2024-09-13T10:12:58.899291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.899291+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:21.742546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":223,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-13T10:13:20.977615+0000","last_change":"2024-09-13T10:12:53.845086+0000","last_active":"2024-09-13T10:13:20.977615+0000","last_peered":"2024-09-13T10:13:20.977615+0000","last_clean":"2024-09-13T10:13:20.977615+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:13:20.977615+0000","last_undegraded":"2024-09-13T10:13:20.977615+0000","last_fullsized":"2024-09-13T10:13:20.977615+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T10:12:53.844861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:53.844861+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.984156+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"119'365","reported_seq":695,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.076267+0000","last_change":"2024-09-13T10:12:00.967692+0000","last_active":"2024-09-13T10:13:17.076267+0000","last_peered":"2024-09-13T10:13:17.076267+0000","last_clean":"2024-09-13T10:13:17.076267+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:13:17.076267+0000","last_undegraded":"2024-09-13T10:13:17.076267+0000","last_fullsized":"2024-09-13T10:13:17.076267+0000","mapping_epoch":30,"log_start":"118'362","ondisk_log_start":"118'362","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'314","last_scrub_stamp":"2024-09-13T10:12:00.967585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.967585+0000","objects_scrubbed":33,"log_size":3,"log_dups_size":374,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:19.050419+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33541126,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":353,"num_read_kb":13014,"num_write":482,"num_write_kb":101793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":101,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:18.697325+0000","last_change":"2024-09-13T10:12:58.690112+0000","last_active":"2024-09-13T10:13:18.697325+0000","last_peered":"2024-09-13T10:13:18.697325+0000","last_clean":"2024-09-13T10:13:18.697325+0000","last_became_active":"2024-09-13T10:12:45.765925+0000","last_became_peered":"2024-09-13T10:12:45.765925+0000","last_unstale":"2024-09-13T10:13:18.697325+0000","last_undegraded":"2024-09-13T10:13:18.697325+0000","last_fullsized":"2024-09-13T10:13:18.697325+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-13T10:12:58.689917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.689917+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.031308+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"118'346","reported_seq":826,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.958498+0000","last_change":"2024-09-13T10:13:17.957473+0000","last_active":"2024-09-13T10:13:17.958498+0000","last_peered":"2024-09-13T10:13:17.958498+0000","last_clean":"2024-09-13T10:13:17.958498+0000","last_became_active":"2024-09-13T10:13:17.956444+0000","last_became_peered":"2024-09-13T10:13:17.956444+0000","last_unstale":"2024-09-13T10:13:17.958498+0000","last_undegraded":"2024-09-13T10:13:17.958498+0000","last_fullsized":"2024-09-13T10:13:17.958498+0000","mapping_epoch":166,"log_start":"116'344","ondisk_log_start":"116'344","created":30,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'346","last_scrub_stamp":"2024-09-13T10:13:16.741058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:16.741058+0000","objects_scrubbed":40,"log_size":2,"log_dups_size":356,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.865324+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27658528,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":310,"num_read_kb":4757,"num_write":455,"num_write_kb":83526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":33245760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.076448+0000","last_change":"2024-09-13T10:13:14.496339+0000","last_active":"2024-09-13T10:13:17.076448+0000","last_peered":"2024-09-13T10:13:17.076448+0000","last_clean":"2024-09-13T10:13:17.076448+0000","last_became_active":"2024-09-13T10:13:14.495396+0000","last_became_peered":"2024-09-13T10:13:14.495396+0000","last_unstale":"2024-09-13T10:13:17.076448+0000","last_undegraded":"2024-09-13T10:13:17.076448+0000","last_fullsized":"2024-09-13T10:13:17.076448+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:11:53.335756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:53.335756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:21.857139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"119'401","reported_seq":714,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:18.728989+0000","last_change":"2024-09-13T10:12:43.753889+0000","last_active":"2024-09-13T10:13:18.728989+0000","last_peered":"2024-09-13T10:13:18.728989+0000","last_clean":"2024-09-13T10:13:18.728989+0000","last_became_active":"2024-09-13T10:12:43.752738+0000","last_became_peered":"2024-09-13T10:12:43.752738+0000","last_unstale":"2024-09-13T10:13:18.728989+0000","last_undegraded":"2024-09-13T10:13:18.728989+0000","last_fullsized":"2024-09-13T10:13:18.728989+0000","mapping_epoch":132,"log_start":"118'398","ondisk_log_start":"118'398","created":30,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'359","last_scrub_stamp":"2024-09-13T10:12:00.627252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.627252+0000","objects_scrubbed":25,"log_size":3,"log_dups_size":407,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:18.843887+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20269983,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":375,"num_read_kb":6453,"num_write":499,"num_write_kb":98736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.945038+0000","last_change":"2024-09-13T10:13:12.541253+0000","last_active":"2024-09-13T10:13:17.945038+0000","last_peered":"2024-09-13T10:13:17.945038+0000","last_clean":"2024-09-13T10:13:17.945038+0000","last_became_active":"2024-09-13T10:12:45.763788+0000","last_became_peered":"2024-09-13T10:12:45.763788+0000","last_unstale":"2024-09-13T10:13:17.945038+0000","last_undegraded":"2024-09-13T10:13:17.945038+0000","last_fullsized":"2024-09-13T10:13:17.945038+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:12.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:12.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:19.031321+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":109480582,"num_objects":148,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":1365,"num_read_kb":31247,"num_write":1869,"num_write_kb":365632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":115888969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":295931904,"data_stored":295197586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":14}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247846,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79548,"kb_used_data":48364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94292292,"statfs":{"total":96636764160,"available":96555307008,"internally_reserved":0,"allocated":49524736,"data_stored":49016492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247848,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":735564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247848,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59972,"kb_used_data":28788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311868,"statfs":{"total":96636764160,"available":96575352832,"internally_reserved":0,"allocated":29478912,"data_stored":28992128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313255,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1134592,"data_stored":735583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313255,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59712,"kb_used_data":28528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312128,"statfs":{"total":96636764160,"available":96575619072,"internally_reserved":0,"allocated":29212672,"data_stored":28754177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247846,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64160,"kb_used_data":32964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94307680,"statfs":{"total":96636764160,"available":96571064320,"internally_reserved":0,"allocated":33755136,"data_stored":33259531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280550,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48592,"kb_used_data":17396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323248,"statfs":{"total":96636764160,"available":96587005952,"internally_reserved":0,"allocated":17813504,"data_stored":17358444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,6,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280550,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85528,"kb_used_data":54344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94286312,"statfs":{"total":96636764160,"available":96549183488,"internally_reserved":0,"allocated":55648256,"data_stored":55129344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,8,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280551,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65112,"kb_used_data":33928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306728,"statfs":{"total":96636764160,"available":96570089472,"internally_reserved":0,"allocated":34742272,"data_stored":34276690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280549,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1118208,"data_stored":727867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280550,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65692,"kb_used_data":34508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306148,"statfs":{"total":96636764160,"available":96569495552,"internally_reserved":0,"allocated":35336192,"data_stored":34867058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280550,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69352,"kb_used_data":38156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302488,"statfs":{"total":96636764160,"available":96565747712,"internally_reserved":0,"allocated":39071744,"data_stored":38565421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":735564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280550,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":1130496,"data_stored":735564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247846,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1146880,"data_stored":743251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247845,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1146880,"data_stored":743251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15941632,"data_stored":15901087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53936128,"data_stored":53811109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":33611776,"data_stored":33541126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33611776,"data_stored":33541126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":27742208,"data_stored":27658528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":37941248,"data_stored":37829857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":28069888,"data_stored":28010945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":48394240,"data_stored":48280928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:13:22.020 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-13T10:13:22.361 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:13:22.361 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"c1e91d47-9492-4461-a043-8698fd74ccf3","health":{"status":"HEALTH_WARN","checks":{"PG_BACKFILL_FULL":{"severity":"HEALTH_WARN","summary":{"message":"Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull","count":1},"muted":false},"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"1 pool(s) do not have an application enabled","count":1},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":188,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":169,"num_osds":16,"num_up_osds":16,"osd_up_since":1726222228,"num_in_osds":16,"osd_in_since":1726222364,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+remapped+backfill_toofull","count":1}],"num_pgs":13,"num_pools":3,"num_objects":152,"data_bytes":110070969,"bytes_used":830701568,"bytes_avail":1545357524992,"bytes_total":1546188226560,"misplaced_objects":39,"misplaced_total":456,"misplaced_ratio":0.085526315789473686,"recovering_objects_per_sec":7,"recovering_bytes_per_sec":3288076,"recovering_keys_per_sec":0,"num_objects_recovered":40,"num_bytes_recovered":16622880,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-13T10:10:13:971496+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-13T10:11:30.944348+0000","services":{}},"progress_events":{}} 2024-09-13T10:13:22.379 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-13T10:13:22.379 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:13:22.658 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:13:22.658 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:13:22.675 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-09-13T10:13:21.979771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110070969,"num_objects":152,"num_object_clones":0,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":1451,"num_read_kb":31320,"num_write":2016,"num_write_kb":368306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":115889007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":39,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":811244,"kb_used_data":324728,"kb_used_omap":24,"kb_used_meta":486183,"kb_avail":1509138196,"statfs":{"total":1546188226560,"available":1545357512704,"internally_reserved":0,"allocated":332521472,"data_stored":325375929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":497851471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-79,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":44633825,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.050420"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":137,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.951736+0000","last_change":"2024-09-13T10:12:52.201492+0000","last_active":"2024-09-13T10:13:17.951736+0000","last_peered":"2024-09-13T10:13:17.951736+0000","last_clean":"2024-09-13T10:13:17.951736+0000","last_became_active":"2024-09-13T10:12:52.200952+0000","last_became_peered":"2024-09-13T10:12:52.200952+0000","last_unstale":"2024-09-13T10:13:17.951736+0000","last_undegraded":"2024-09-13T10:13:17.951736+0000","last_fullsized":"2024-09-13T10:13:17.951736+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:03.961779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:03.961779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":305,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.076563+0000","last_change":"2024-09-13T10:12:12.892985+0000","last_active":"2024-09-13T10:13:17.076563+0000","last_peered":"2024-09-13T10:13:17.076563+0000","last_clean":"2024-09-13T10:13:17.076563+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:13:17.076563+0000","last_undegraded":"2024-09-13T10:13:17.076563+0000","last_fullsized":"2024-09-13T10:13:17.076563+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:12.892836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:12.892836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:20.216623+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":318,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.302566+0000","last_change":"2024-09-13T10:12:45.768853+0000","last_active":"2024-09-13T10:13:17.302566+0000","last_peered":"2024-09-13T10:13:17.302566+0000","last_clean":"2024-09-13T10:13:17.302566+0000","last_became_active":"2024-09-13T10:12:45.768237+0000","last_became_peered":"2024-09-13T10:12:45.768237+0000","last_unstale":"2024-09-13T10:13:17.302566+0000","last_undegraded":"2024-09-13T10:13:17.302566+0000","last_fullsized":"2024-09-13T10:13:17.302566+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:01.407274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:01.407274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:23.291168+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":30,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:18.729021+0000","last_change":"2024-09-13T10:13:09.126344+0000","last_active":"2024-09-13T10:13:18.729021+0000","last_peered":"2024-09-13T10:13:18.729021+0000","last_clean":"2024-09-13T10:13:18.729021+0000","last_became_active":"2024-09-13T10:13:09.125529+0000","last_became_peered":"2024-09-13T10:13:09.125529+0000","last_unstale":"2024-09-13T10:13:18.729021+0000","last_undegraded":"2024-09-13T10:13:18.729021+0000","last_fullsized":"2024-09-13T10:13:18.729021+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:50.136244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:50.136244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:14.321249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"119'332","reported_seq":937,"reported_epoch":167,"state":"peering","last_fresh":"2024-09-13T10:13:20.530384+0000","last_change":"2024-09-13T10:13:20.978575+0000","last_active":"2024-09-13T10:13:20.978575+0000","last_peered":"2024-09-13T10:13:20.530384+0000","last_clean":"2024-09-13T10:13:20.530384+0000","last_became_active":"2024-09-13T10:12:50.225448+0000","last_became_peered":"2024-09-13T10:12:50.225448+0000","last_unstale":"2024-09-13T10:13:20.530384+0000","last_undegraded":"2024-09-13T10:13:20.530384+0000","last_fullsized":"2024-09-13T10:13:20.530384+0000","mapping_epoch":168,"log_start":"118'329","ondisk_log_start":"118'329","created":30,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'332","last_scrub_stamp":"2024-09-13T10:12:28.697464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:28.697464+0000","objects_scrubbed":39,"log_size":3,"log_dups_size":344,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28010945,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":327,"num_read_kb":7023,"num_write":433,"num_write_kb":81577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":82643209,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,4,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,10,14],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":276,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.951731+0000","last_change":"2024-09-13T10:12:58.899445+0000","last_active":"2024-09-13T10:13:17.951731+0000","last_peered":"2024-09-13T10:13:17.951731+0000","last_clean":"2024-09-13T10:13:17.951731+0000","last_became_active":"2024-09-13T10:12:52.099716+0000","last_became_peered":"2024-09-13T10:12:52.099716+0000","last_unstale":"2024-09-13T10:13:17.951731+0000","last_undegraded":"2024-09-13T10:13:17.951731+0000","last_fullsized":"2024-09-13T10:13:17.951731+0000","mapping_epoch":140,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'112","last_scrub_stamp":"2024-09-13T10:12:58.899291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.899291+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:21.742546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":223,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-13T10:13:20.977615+0000","last_change":"2024-09-13T10:12:53.845086+0000","last_active":"2024-09-13T10:13:20.977615+0000","last_peered":"2024-09-13T10:13:20.977615+0000","last_clean":"2024-09-13T10:13:20.977615+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:13:20.977615+0000","last_undegraded":"2024-09-13T10:13:20.977615+0000","last_fullsized":"2024-09-13T10:13:20.977615+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T10:12:53.844861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:53.844861+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.984156+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"119'365","reported_seq":695,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.076267+0000","last_change":"2024-09-13T10:12:00.967692+0000","last_active":"2024-09-13T10:13:17.076267+0000","last_peered":"2024-09-13T10:13:17.076267+0000","last_clean":"2024-09-13T10:13:17.076267+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:13:17.076267+0000","last_undegraded":"2024-09-13T10:13:17.076267+0000","last_fullsized":"2024-09-13T10:13:17.076267+0000","mapping_epoch":30,"log_start":"118'362","ondisk_log_start":"118'362","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"95'314","last_scrub_stamp":"2024-09-13T10:12:00.967585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.967585+0000","objects_scrubbed":33,"log_size":3,"log_dups_size":374,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:19.050419+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33541126,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":353,"num_read_kb":13014,"num_write":482,"num_write_kb":101793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":101,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:18.697325+0000","last_change":"2024-09-13T10:12:58.690112+0000","last_active":"2024-09-13T10:13:18.697325+0000","last_peered":"2024-09-13T10:13:18.697325+0000","last_clean":"2024-09-13T10:13:18.697325+0000","last_became_active":"2024-09-13T10:12:45.765925+0000","last_became_peered":"2024-09-13T10:12:45.765925+0000","last_unstale":"2024-09-13T10:13:18.697325+0000","last_undegraded":"2024-09-13T10:13:18.697325+0000","last_fullsized":"2024-09-13T10:13:18.697325+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-13T10:12:58.689917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.689917+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.031308+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"118'346","reported_seq":826,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.958498+0000","last_change":"2024-09-13T10:13:17.957473+0000","last_active":"2024-09-13T10:13:17.958498+0000","last_peered":"2024-09-13T10:13:17.958498+0000","last_clean":"2024-09-13T10:13:17.958498+0000","last_became_active":"2024-09-13T10:13:17.956444+0000","last_became_peered":"2024-09-13T10:13:17.956444+0000","last_unstale":"2024-09-13T10:13:17.958498+0000","last_undegraded":"2024-09-13T10:13:17.958498+0000","last_fullsized":"2024-09-13T10:13:17.958498+0000","mapping_epoch":166,"log_start":"116'344","ondisk_log_start":"116'344","created":30,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'346","last_scrub_stamp":"2024-09-13T10:13:16.741058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:16.741058+0000","objects_scrubbed":40,"log_size":2,"log_dups_size":356,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.865324+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27658528,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":310,"num_read_kb":4757,"num_write":455,"num_write_kb":83526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":33245760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":134,"reported_epoch":166,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.076448+0000","last_change":"2024-09-13T10:13:14.496339+0000","last_active":"2024-09-13T10:13:17.076448+0000","last_peered":"2024-09-13T10:13:17.076448+0000","last_clean":"2024-09-13T10:13:17.076448+0000","last_became_active":"2024-09-13T10:13:14.495396+0000","last_became_peered":"2024-09-13T10:13:14.495396+0000","last_unstale":"2024-09-13T10:13:17.076448+0000","last_undegraded":"2024-09-13T10:13:17.076448+0000","last_fullsized":"2024-09-13T10:13:17.076448+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:11:53.335756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:11:53.335756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:21.857139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"119'401","reported_seq":714,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:18.728989+0000","last_change":"2024-09-13T10:12:43.753889+0000","last_active":"2024-09-13T10:13:18.728989+0000","last_peered":"2024-09-13T10:13:18.728989+0000","last_clean":"2024-09-13T10:13:18.728989+0000","last_became_active":"2024-09-13T10:12:43.752738+0000","last_became_peered":"2024-09-13T10:12:43.752738+0000","last_unstale":"2024-09-13T10:13:18.728989+0000","last_undegraded":"2024-09-13T10:13:18.728989+0000","last_fullsized":"2024-09-13T10:13:18.728989+0000","mapping_epoch":132,"log_start":"118'398","ondisk_log_start":"118'398","created":30,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'359","last_scrub_stamp":"2024-09-13T10:12:00.627252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:00.627252+0000","objects_scrubbed":25,"log_size":3,"log_dups_size":407,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:18.843887+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20269983,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":375,"num_read_kb":6453,"num_write":499,"num_write_kb":98736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":167,"state":"active+clean","last_fresh":"2024-09-13T10:13:17.945038+0000","last_change":"2024-09-13T10:13:12.541253+0000","last_active":"2024-09-13T10:13:17.945038+0000","last_peered":"2024-09-13T10:13:17.945038+0000","last_clean":"2024-09-13T10:13:17.945038+0000","last_became_active":"2024-09-13T10:12:45.763788+0000","last_became_peered":"2024-09-13T10:12:45.763788+0000","last_unstale":"2024-09-13T10:13:17.945038+0000","last_undegraded":"2024-09-13T10:13:17.945038+0000","last_fullsized":"2024-09-13T10:13:17.945038+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:12.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:12.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:19.031321+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":109480582,"num_objects":148,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":1365,"num_read_kb":31247,"num_write":1869,"num_write_kb":365632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":115888969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":295931904,"data_stored":295197586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":14}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247846,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79548,"kb_used_data":48364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94292292,"statfs":{"total":96636764160,"available":96555307008,"internally_reserved":0,"allocated":49524736,"data_stored":49016492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247848,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":735564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247848,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59972,"kb_used_data":28788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311868,"statfs":{"total":96636764160,"available":96575352832,"internally_reserved":0,"allocated":29478912,"data_stored":28992128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313255,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28132,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343708,"statfs":{"total":96636764160,"available":96607956992,"internally_reserved":0,"allocated":1134592,"data_stored":735583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313255,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59712,"kb_used_data":28528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312128,"statfs":{"total":96636764160,"available":96575619072,"internally_reserved":0,"allocated":29212672,"data_stored":28754177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247846,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64160,"kb_used_data":32964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94307680,"statfs":{"total":96636764160,"available":96571064320,"internally_reserved":0,"allocated":33755136,"data_stored":33259531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280550,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48592,"kb_used_data":17396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323248,"statfs":{"total":96636764160,"available":96587005952,"internally_reserved":0,"allocated":17813504,"data_stored":17358444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,6,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280550,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85528,"kb_used_data":54344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94286312,"statfs":{"total":96636764160,"available":96549183488,"internally_reserved":0,"allocated":55648256,"data_stored":55129344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,8,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280551,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65112,"kb_used_data":33928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306728,"statfs":{"total":96636764160,"available":96570089472,"internally_reserved":0,"allocated":34742272,"data_stored":34276690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280549,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1118208,"data_stored":727867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280550,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65692,"kb_used_data":34508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306148,"statfs":{"total":96636764160,"available":96569495552,"internally_reserved":0,"allocated":35336192,"data_stored":34867058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280550,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69352,"kb_used_data":38156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94302488,"statfs":{"total":96636764160,"available":96565747712,"internally_reserved":0,"allocated":39071744,"data_stored":38565421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":735564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280550,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":1130496,"data_stored":735564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247846,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1146880,"data_stored":743251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247845,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1146880,"data_stored":743251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15941632,"data_stored":15901087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53936128,"data_stored":53811109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":33611776,"data_stored":33541126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33611776,"data_stored":33541126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":27742208,"data_stored":27658528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":37941248,"data_stored":37829857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":28069888,"data_stored":28010945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":48394240,"data_stored":48280928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:13:25.677 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:13:25.997 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:13:25.997 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:13:26.015 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-13T10:13:24.983776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110070969,"num_objects":152,"num_object_clones":0,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":1451,"num_read_kb":31320,"num_write":2016,"num_write_kb":368306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":115889007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":39,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":807596,"kb_used_data":321104,"kb_used_omap":24,"kb_used_meta":486183,"kb_avail":1509141844,"statfs":{"total":1546188226560,"available":1545361248256,"internally_reserved":0,"allocated":328810496,"data_stored":321493799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":497851471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-39,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":28010945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002129"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":151,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505805+0000","last_change":"2024-09-13T10:13:21.781722+0000","last_active":"2024-09-13T10:13:23.505805+0000","last_peered":"2024-09-13T10:13:23.505805+0000","last_clean":"2024-09-13T10:13:23.505805+0000","last_became_active":"2024-09-13T10:12:52.200952+0000","last_became_peered":"2024-09-13T10:12:52.200952+0000","last_unstale":"2024-09-13T10:13:23.505805+0000","last_undegraded":"2024-09-13T10:13:23.505805+0000","last_fullsized":"2024-09-13T10:13:23.505805+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:21.781648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:21.781648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:37.772623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":313,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.743571+0000","last_change":"2024-09-13T10:12:12.892985+0000","last_active":"2024-09-13T10:13:22.743571+0000","last_peered":"2024-09-13T10:13:22.743571+0000","last_clean":"2024-09-13T10:13:22.743571+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:13:22.743571+0000","last_undegraded":"2024-09-13T10:13:22.743571+0000","last_fullsized":"2024-09-13T10:13:22.743571+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:12.892836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:12.892836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":326,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.942174+0000","last_change":"2024-09-13T10:12:45.768853+0000","last_active":"2024-09-13T10:13:22.942174+0000","last_peered":"2024-09-13T10:13:22.942174+0000","last_clean":"2024-09-13T10:13:22.942174+0000","last_became_active":"2024-09-13T10:12:45.768237+0000","last_became_peered":"2024-09-13T10:12:45.768237+0000","last_unstale":"2024-09-13T10:13:22.942174+0000","last_undegraded":"2024-09-13T10:13:22.942174+0000","last_fullsized":"2024-09-13T10:13:22.942174+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:01.407274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:01.407274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:23.291168+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505514+0000","last_change":"2024-09-13T10:13:09.126344+0000","last_active":"2024-09-13T10:13:23.505514+0000","last_peered":"2024-09-13T10:13:23.505514+0000","last_clean":"2024-09-13T10:13:23.505514+0000","last_became_active":"2024-09-13T10:13:09.125529+0000","last_became_peered":"2024-09-13T10:13:09.125529+0000","last_unstale":"2024-09-13T10:13:23.505514+0000","last_undegraded":"2024-09-13T10:13:23.505514+0000","last_fullsized":"2024-09-13T10:13:23.505514+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:50.136244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:50.136244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:14.321249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"119'332","reported_seq":937,"reported_epoch":167,"state":"peering","last_fresh":"2024-09-13T10:13:20.530384+0000","last_change":"2024-09-13T10:13:20.978575+0000","last_active":"2024-09-13T10:13:20.978575+0000","last_peered":"2024-09-13T10:13:20.530384+0000","last_clean":"2024-09-13T10:13:20.530384+0000","last_became_active":"2024-09-13T10:12:50.225448+0000","last_became_peered":"2024-09-13T10:12:50.225448+0000","last_unstale":"2024-09-13T10:13:20.530384+0000","last_undegraded":"2024-09-13T10:13:20.530384+0000","last_fullsized":"2024-09-13T10:13:20.530384+0000","mapping_epoch":168,"log_start":"118'329","ondisk_log_start":"118'329","created":30,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'332","last_scrub_stamp":"2024-09-13T10:12:28.697464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:28.697464+0000","objects_scrubbed":39,"log_size":3,"log_dups_size":344,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28010945,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":327,"num_read_kb":7023,"num_write":433,"num_write_kb":81577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":82643209,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,4,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,10,14],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":282,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505794+0000","last_change":"2024-09-13T10:12:58.899445+0000","last_active":"2024-09-13T10:13:23.505794+0000","last_peered":"2024-09-13T10:13:23.505794+0000","last_clean":"2024-09-13T10:13:23.505794+0000","last_became_active":"2024-09-13T10:12:52.099716+0000","last_became_peered":"2024-09-13T10:12:52.099716+0000","last_unstale":"2024-09-13T10:13:23.505794+0000","last_undegraded":"2024-09-13T10:13:23.505794+0000","last_fullsized":"2024-09-13T10:13:23.505794+0000","mapping_epoch":140,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'112","last_scrub_stamp":"2024-09-13T10:12:58.899291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.899291+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:21.742546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":223,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-13T10:13:20.977615+0000","last_change":"2024-09-13T10:12:53.845086+0000","last_active":"2024-09-13T10:13:20.977615+0000","last_peered":"2024-09-13T10:13:20.977615+0000","last_clean":"2024-09-13T10:13:20.977615+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:13:20.977615+0000","last_undegraded":"2024-09-13T10:13:20.977615+0000","last_fullsized":"2024-09-13T10:13:20.977615+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T10:12:53.844861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:53.844861+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.984156+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"119'365","reported_seq":711,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.743459+0000","last_change":"2024-09-13T10:13:22.036708+0000","last_active":"2024-09-13T10:13:22.743459+0000","last_peered":"2024-09-13T10:13:22.743459+0000","last_clean":"2024-09-13T10:13:22.743459+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:13:22.743459+0000","last_undegraded":"2024-09-13T10:13:22.743459+0000","last_fullsized":"2024-09-13T10:13:22.743459+0000","mapping_epoch":30,"log_start":"118'362","ondisk_log_start":"118'362","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'365","last_scrub_stamp":"2024-09-13T10:13:22.036498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:22.036498+0000","objects_scrubbed":42,"log_size":3,"log_dups_size":374,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:24.928602+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33541126,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":353,"num_read_kb":13014,"num_write":482,"num_write_kb":101793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":107,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505195+0000","last_change":"2024-09-13T10:12:58.690112+0000","last_active":"2024-09-13T10:13:23.505195+0000","last_peered":"2024-09-13T10:13:23.505195+0000","last_clean":"2024-09-13T10:13:23.505195+0000","last_became_active":"2024-09-13T10:12:45.765925+0000","last_became_peered":"2024-09-13T10:12:45.765925+0000","last_unstale":"2024-09-13T10:13:23.505195+0000","last_undegraded":"2024-09-13T10:13:23.505195+0000","last_fullsized":"2024-09-13T10:13:23.505195+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-13T10:12:58.689917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.689917+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.031308+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"118'346","reported_seq":830,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.067501+0000","last_change":"2024-09-13T10:13:17.957473+0000","last_active":"2024-09-13T10:13:22.067501+0000","last_peered":"2024-09-13T10:13:22.067501+0000","last_clean":"2024-09-13T10:13:22.067501+0000","last_became_active":"2024-09-13T10:13:17.956444+0000","last_became_peered":"2024-09-13T10:13:17.956444+0000","last_unstale":"2024-09-13T10:13:22.067501+0000","last_undegraded":"2024-09-13T10:13:22.067501+0000","last_fullsized":"2024-09-13T10:13:22.067501+0000","mapping_epoch":166,"log_start":"116'344","ondisk_log_start":"116'344","created":30,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'346","last_scrub_stamp":"2024-09-13T10:13:16.741058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:16.741058+0000","objects_scrubbed":40,"log_size":2,"log_dups_size":356,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.865324+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27658528,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":310,"num_read_kb":4757,"num_write":455,"num_write_kb":83526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":33245760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":148,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.027674+0000","last_change":"2024-09-13T10:13:22.027674+0000","last_active":"2024-09-13T10:13:22.027674+0000","last_peered":"2024-09-13T10:13:22.027674+0000","last_clean":"2024-09-13T10:13:22.027674+0000","last_became_active":"2024-09-13T10:13:14.495396+0000","last_became_peered":"2024-09-13T10:13:14.495396+0000","last_unstale":"2024-09-13T10:13:22.027674+0000","last_undegraded":"2024-09-13T10:13:22.027674+0000","last_fullsized":"2024-09-13T10:13:22.027674+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:22.027512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:22.027512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:47.896470+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"119'401","reported_seq":728,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505478+0000","last_change":"2024-09-13T10:13:19.721098+0000","last_active":"2024-09-13T10:13:23.505478+0000","last_peered":"2024-09-13T10:13:23.505478+0000","last_clean":"2024-09-13T10:13:23.505478+0000","last_became_active":"2024-09-13T10:12:43.752738+0000","last_became_peered":"2024-09-13T10:12:43.752738+0000","last_unstale":"2024-09-13T10:13:23.505478+0000","last_undegraded":"2024-09-13T10:13:23.505478+0000","last_fullsized":"2024-09-13T10:13:23.505478+0000","mapping_epoch":132,"log_start":"118'398","ondisk_log_start":"118'398","created":30,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'401","last_scrub_stamp":"2024-09-13T10:13:19.720891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:19.720891+0000","objects_scrubbed":27,"log_size":3,"log_dups_size":407,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:49.615137+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20269983,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":375,"num_read_kb":6453,"num_write":499,"num_write_kb":98736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.506028+0000","last_change":"2024-09-13T10:13:12.541253+0000","last_active":"2024-09-13T10:13:23.506028+0000","last_peered":"2024-09-13T10:13:23.506028+0000","last_clean":"2024-09-13T10:13:23.506028+0000","last_became_active":"2024-09-13T10:12:45.763788+0000","last_became_peered":"2024-09-13T10:12:45.763788+0000","last_unstale":"2024-09-13T10:13:23.506028+0000","last_undegraded":"2024-09-13T10:13:23.506028+0000","last_fullsized":"2024-09-13T10:13:23.506028+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:12.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:12.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:19.031321+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":109480582,"num_objects":148,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":1365,"num_read_kb":31247,"num_write":1869,"num_write_kb":365632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":115888969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":308350976,"data_stored":307593987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":14}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247847,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79596,"kb_used_data":48412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94292244,"statfs":{"total":96636764160,"available":96555257856,"internally_reserved":0,"allocated":49573888,"data_stored":49047078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247849,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247849,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60008,"kb_used_data":28824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311832,"statfs":{"total":96636764160,"available":96575315968,"internally_reserved":0,"allocated":29515776,"data_stored":29015046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313256,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343696,"statfs":{"total":96636764160,"available":96607944704,"internally_reserved":0,"allocated":1146880,"data_stored":743251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313255,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59712,"kb_used_data":28528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312128,"statfs":{"total":96636764160,"available":96575619072,"internally_reserved":0,"allocated":29212672,"data_stored":28754177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247847,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47900,"kb_used_data":16708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323940,"statfs":{"total":96636764160,"available":96587714560,"internally_reserved":0,"allocated":17108992,"data_stored":16659610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280551,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48624,"kb_used_data":17432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323216,"statfs":{"total":96636764160,"available":96586973184,"internally_reserved":0,"allocated":17850368,"data_stored":17381403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,6,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280551,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85564,"kb_used_data":54380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94286276,"statfs":{"total":96636764160,"available":96549146624,"internally_reserved":0,"allocated":55685120,"data_stored":55152365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,8,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280552,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92560,"kb_used_data":61376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94279280,"statfs":{"total":96636764160,"available":96541982720,"internally_reserved":0,"allocated":62849024,"data_stored":62310594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28164,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343676,"statfs":{"total":96636764160,"available":96607924224,"internally_reserved":0,"allocated":1167360,"data_stored":758523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280551,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65728,"kb_used_data":34544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306112,"statfs":{"total":96636764160,"available":96569458688,"internally_reserved":0,"allocated":35373056,"data_stored":34890017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280551,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54096,"kb_used_data":22908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94317744,"statfs":{"total":96636764160,"available":96581369856,"internally_reserved":0,"allocated":23457792,"data_stored":22973836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32312,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1155072,"data_stored":750888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280551,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28164,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343676,"statfs":{"total":96636764160,"available":96607924224,"internally_reserved":0,"allocated":1167360,"data_stored":758523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247847,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247846,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15941632,"data_stored":15901087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53936128,"data_stored":53811109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61681664,"data_stored":61552071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33611776,"data_stored":33541126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":27742208,"data_stored":27658528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":22290432,"data_stored":22215313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":28069888,"data_stored":28010945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":48394240,"data_stored":48280928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:13:26.017 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-13T10:13:26.361 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:13:26.361 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"c1e91d47-9492-4461-a043-8698fd74ccf3","health":{"status":"HEALTH_WARN","checks":{"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"1 pool(s) do not have an application enabled","count":1},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":192,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":170,"num_osds":16,"num_up_osds":16,"osd_up_since":1726222228,"num_in_osds":16,"osd_in_since":1726222364,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"peering","count":1}],"num_pgs":13,"num_pools":3,"num_objects":152,"data_bytes":110070969,"bytes_used":826978304,"bytes_avail":1545361248256,"bytes_total":1546188226560,"inactive_pgs_ratio":0.076923079788684845,"recovering_objects_per_sec":6,"recovering_bytes_per_sec":4666834,"recovering_keys_per_sec":0,"num_objects_recovered":39,"num_bytes_recovered":28010945,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-13T10:10:13:971496+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-13T10:11:30.944348+0000","services":{}},"progress_events":{}} 2024-09-13T10:13:26.378 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-09-13T10:13:26.379 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:13:26.665 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:13:26.665 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:13:26.681 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-13T10:13:24.983776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110070969,"num_objects":152,"num_object_clones":0,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":1451,"num_read_kb":31320,"num_write":2016,"num_write_kb":368306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":115889007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":39,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":807596,"kb_used_data":321104,"kb_used_omap":24,"kb_used_meta":486183,"kb_avail":1509141844,"statfs":{"total":1546188226560,"available":1545361248256,"internally_reserved":0,"allocated":328810496,"data_stored":321493799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":497851471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-39,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":28010945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002129"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":151,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505805+0000","last_change":"2024-09-13T10:13:21.781722+0000","last_active":"2024-09-13T10:13:23.505805+0000","last_peered":"2024-09-13T10:13:23.505805+0000","last_clean":"2024-09-13T10:13:23.505805+0000","last_became_active":"2024-09-13T10:12:52.200952+0000","last_became_peered":"2024-09-13T10:12:52.200952+0000","last_unstale":"2024-09-13T10:13:23.505805+0000","last_undegraded":"2024-09-13T10:13:23.505805+0000","last_fullsized":"2024-09-13T10:13:23.505805+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:21.781648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:21.781648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:37.772623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":313,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.743571+0000","last_change":"2024-09-13T10:12:12.892985+0000","last_active":"2024-09-13T10:13:22.743571+0000","last_peered":"2024-09-13T10:13:22.743571+0000","last_clean":"2024-09-13T10:13:22.743571+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:13:22.743571+0000","last_undegraded":"2024-09-13T10:13:22.743571+0000","last_fullsized":"2024-09-13T10:13:22.743571+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:12.892836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:12.892836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":326,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.942174+0000","last_change":"2024-09-13T10:12:45.768853+0000","last_active":"2024-09-13T10:13:22.942174+0000","last_peered":"2024-09-13T10:13:22.942174+0000","last_clean":"2024-09-13T10:13:22.942174+0000","last_became_active":"2024-09-13T10:12:45.768237+0000","last_became_peered":"2024-09-13T10:12:45.768237+0000","last_unstale":"2024-09-13T10:13:22.942174+0000","last_undegraded":"2024-09-13T10:13:22.942174+0000","last_fullsized":"2024-09-13T10:13:22.942174+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:01.407274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:01.407274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:23.291168+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505514+0000","last_change":"2024-09-13T10:13:09.126344+0000","last_active":"2024-09-13T10:13:23.505514+0000","last_peered":"2024-09-13T10:13:23.505514+0000","last_clean":"2024-09-13T10:13:23.505514+0000","last_became_active":"2024-09-13T10:13:09.125529+0000","last_became_peered":"2024-09-13T10:13:09.125529+0000","last_unstale":"2024-09-13T10:13:23.505514+0000","last_undegraded":"2024-09-13T10:13:23.505514+0000","last_fullsized":"2024-09-13T10:13:23.505514+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:50.136244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:50.136244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:14.321249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"119'332","reported_seq":937,"reported_epoch":167,"state":"peering","last_fresh":"2024-09-13T10:13:20.530384+0000","last_change":"2024-09-13T10:13:20.978575+0000","last_active":"2024-09-13T10:13:20.978575+0000","last_peered":"2024-09-13T10:13:20.530384+0000","last_clean":"2024-09-13T10:13:20.530384+0000","last_became_active":"2024-09-13T10:12:50.225448+0000","last_became_peered":"2024-09-13T10:12:50.225448+0000","last_unstale":"2024-09-13T10:13:20.530384+0000","last_undegraded":"2024-09-13T10:13:20.530384+0000","last_fullsized":"2024-09-13T10:13:20.530384+0000","mapping_epoch":168,"log_start":"118'329","ondisk_log_start":"118'329","created":30,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'332","last_scrub_stamp":"2024-09-13T10:12:28.697464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:28.697464+0000","objects_scrubbed":39,"log_size":3,"log_dups_size":344,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28010945,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":327,"num_read_kb":7023,"num_write":433,"num_write_kb":81577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":82643209,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,4,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,10,14],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":282,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505794+0000","last_change":"2024-09-13T10:12:58.899445+0000","last_active":"2024-09-13T10:13:23.505794+0000","last_peered":"2024-09-13T10:13:23.505794+0000","last_clean":"2024-09-13T10:13:23.505794+0000","last_became_active":"2024-09-13T10:12:52.099716+0000","last_became_peered":"2024-09-13T10:12:52.099716+0000","last_unstale":"2024-09-13T10:13:23.505794+0000","last_undegraded":"2024-09-13T10:13:23.505794+0000","last_fullsized":"2024-09-13T10:13:23.505794+0000","mapping_epoch":140,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'112","last_scrub_stamp":"2024-09-13T10:12:58.899291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.899291+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:21.742546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":223,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-13T10:13:20.977615+0000","last_change":"2024-09-13T10:12:53.845086+0000","last_active":"2024-09-13T10:13:20.977615+0000","last_peered":"2024-09-13T10:13:20.977615+0000","last_clean":"2024-09-13T10:13:20.977615+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:13:20.977615+0000","last_undegraded":"2024-09-13T10:13:20.977615+0000","last_fullsized":"2024-09-13T10:13:20.977615+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T10:12:53.844861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:53.844861+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.984156+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"119'365","reported_seq":711,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.743459+0000","last_change":"2024-09-13T10:13:22.036708+0000","last_active":"2024-09-13T10:13:22.743459+0000","last_peered":"2024-09-13T10:13:22.743459+0000","last_clean":"2024-09-13T10:13:22.743459+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:13:22.743459+0000","last_undegraded":"2024-09-13T10:13:22.743459+0000","last_fullsized":"2024-09-13T10:13:22.743459+0000","mapping_epoch":30,"log_start":"118'362","ondisk_log_start":"118'362","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'365","last_scrub_stamp":"2024-09-13T10:13:22.036498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:22.036498+0000","objects_scrubbed":42,"log_size":3,"log_dups_size":374,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:24.928602+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33541126,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":353,"num_read_kb":13014,"num_write":482,"num_write_kb":101793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":107,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505195+0000","last_change":"2024-09-13T10:12:58.690112+0000","last_active":"2024-09-13T10:13:23.505195+0000","last_peered":"2024-09-13T10:13:23.505195+0000","last_clean":"2024-09-13T10:13:23.505195+0000","last_became_active":"2024-09-13T10:12:45.765925+0000","last_became_peered":"2024-09-13T10:12:45.765925+0000","last_unstale":"2024-09-13T10:13:23.505195+0000","last_undegraded":"2024-09-13T10:13:23.505195+0000","last_fullsized":"2024-09-13T10:13:23.505195+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-13T10:12:58.689917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.689917+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.031308+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"118'346","reported_seq":830,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.067501+0000","last_change":"2024-09-13T10:13:17.957473+0000","last_active":"2024-09-13T10:13:22.067501+0000","last_peered":"2024-09-13T10:13:22.067501+0000","last_clean":"2024-09-13T10:13:22.067501+0000","last_became_active":"2024-09-13T10:13:17.956444+0000","last_became_peered":"2024-09-13T10:13:17.956444+0000","last_unstale":"2024-09-13T10:13:22.067501+0000","last_undegraded":"2024-09-13T10:13:22.067501+0000","last_fullsized":"2024-09-13T10:13:22.067501+0000","mapping_epoch":166,"log_start":"116'344","ondisk_log_start":"116'344","created":30,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'346","last_scrub_stamp":"2024-09-13T10:13:16.741058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:16.741058+0000","objects_scrubbed":40,"log_size":2,"log_dups_size":356,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.865324+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27658528,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":310,"num_read_kb":4757,"num_write":455,"num_write_kb":83526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":33245760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":148,"reported_epoch":169,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.027674+0000","last_change":"2024-09-13T10:13:22.027674+0000","last_active":"2024-09-13T10:13:22.027674+0000","last_peered":"2024-09-13T10:13:22.027674+0000","last_clean":"2024-09-13T10:13:22.027674+0000","last_became_active":"2024-09-13T10:13:14.495396+0000","last_became_peered":"2024-09-13T10:13:14.495396+0000","last_unstale":"2024-09-13T10:13:22.027674+0000","last_undegraded":"2024-09-13T10:13:22.027674+0000","last_fullsized":"2024-09-13T10:13:22.027674+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:22.027512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:22.027512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:47.896470+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"119'401","reported_seq":728,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505478+0000","last_change":"2024-09-13T10:13:19.721098+0000","last_active":"2024-09-13T10:13:23.505478+0000","last_peered":"2024-09-13T10:13:23.505478+0000","last_clean":"2024-09-13T10:13:23.505478+0000","last_became_active":"2024-09-13T10:12:43.752738+0000","last_became_peered":"2024-09-13T10:12:43.752738+0000","last_unstale":"2024-09-13T10:13:23.505478+0000","last_undegraded":"2024-09-13T10:13:23.505478+0000","last_fullsized":"2024-09-13T10:13:23.505478+0000","mapping_epoch":132,"log_start":"118'398","ondisk_log_start":"118'398","created":30,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'401","last_scrub_stamp":"2024-09-13T10:13:19.720891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:19.720891+0000","objects_scrubbed":27,"log_size":3,"log_dups_size":407,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:49.615137+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20269983,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":375,"num_read_kb":6453,"num_write":499,"num_write_kb":98736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.506028+0000","last_change":"2024-09-13T10:13:12.541253+0000","last_active":"2024-09-13T10:13:23.506028+0000","last_peered":"2024-09-13T10:13:23.506028+0000","last_clean":"2024-09-13T10:13:23.506028+0000","last_became_active":"2024-09-13T10:12:45.763788+0000","last_became_peered":"2024-09-13T10:12:45.763788+0000","last_unstale":"2024-09-13T10:13:23.506028+0000","last_undegraded":"2024-09-13T10:13:23.506028+0000","last_fullsized":"2024-09-13T10:13:23.506028+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:12.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:12.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:19.031321+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":109480582,"num_objects":148,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":1365,"num_read_kb":31247,"num_write":1869,"num_write_kb":365632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":115888969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":308350976,"data_stored":307593987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":14}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247847,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79596,"kb_used_data":48412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94292244,"statfs":{"total":96636764160,"available":96555257856,"internally_reserved":0,"allocated":49573888,"data_stored":49047078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247849,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247849,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60008,"kb_used_data":28824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311832,"statfs":{"total":96636764160,"available":96575315968,"internally_reserved":0,"allocated":29515776,"data_stored":29015046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313256,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343696,"statfs":{"total":96636764160,"available":96607944704,"internally_reserved":0,"allocated":1146880,"data_stored":743251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313255,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59712,"kb_used_data":28528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312128,"statfs":{"total":96636764160,"available":96575619072,"internally_reserved":0,"allocated":29212672,"data_stored":28754177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247847,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47900,"kb_used_data":16708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323940,"statfs":{"total":96636764160,"available":96587714560,"internally_reserved":0,"allocated":17108992,"data_stored":16659610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280551,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48624,"kb_used_data":17432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323216,"statfs":{"total":96636764160,"available":96586973184,"internally_reserved":0,"allocated":17850368,"data_stored":17381403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,6,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280551,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85564,"kb_used_data":54380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94286276,"statfs":{"total":96636764160,"available":96549146624,"internally_reserved":0,"allocated":55685120,"data_stored":55152365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,8,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280552,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92560,"kb_used_data":61376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94279280,"statfs":{"total":96636764160,"available":96541982720,"internally_reserved":0,"allocated":62849024,"data_stored":62310594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28164,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343676,"statfs":{"total":96636764160,"available":96607924224,"internally_reserved":0,"allocated":1167360,"data_stored":758523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280551,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65728,"kb_used_data":34544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306112,"statfs":{"total":96636764160,"available":96569458688,"internally_reserved":0,"allocated":35373056,"data_stored":34890017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280551,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54096,"kb_used_data":22908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94317744,"statfs":{"total":96636764160,"available":96581369856,"internally_reserved":0,"allocated":23457792,"data_stored":22973836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32312,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1155072,"data_stored":750888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280551,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28164,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343676,"statfs":{"total":96636764160,"available":96607924224,"internally_reserved":0,"allocated":1167360,"data_stored":758523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247847,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247846,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15941632,"data_stored":15901087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53936128,"data_stored":53811109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61681664,"data_stored":61552071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33611776,"data_stored":33541126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":27742208,"data_stored":27658528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":22290432,"data_stored":22215313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":28069888,"data_stored":28010945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":48394240,"data_stored":48280928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:13:29.684 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:13:29.993 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:13:29.994 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:13:30.011 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-13T10:13:28.985413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110070969,"num_objects":152,"num_object_clones":0,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":1451,"num_read_kb":31320,"num_write":2016,"num_write_kb":368306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":115889007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":39,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":802320,"kb_used_data":315820,"kb_used_omap":24,"kb_used_meta":486183,"kb_avail":1509147120,"statfs":{"total":1546188226560,"available":1545366650880,"internally_reserved":0,"allocated":323399680,"data_stored":316023488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":497851471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-39,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":28010945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002975"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":151,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505805+0000","last_change":"2024-09-13T10:13:21.781722+0000","last_active":"2024-09-13T10:13:23.505805+0000","last_peered":"2024-09-13T10:13:23.505805+0000","last_clean":"2024-09-13T10:13:23.505805+0000","last_became_active":"2024-09-13T10:12:52.200952+0000","last_became_peered":"2024-09-13T10:12:52.200952+0000","last_unstale":"2024-09-13T10:13:23.505805+0000","last_undegraded":"2024-09-13T10:13:23.505805+0000","last_fullsized":"2024-09-13T10:13:23.505805+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:21.781648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:21.781648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:37.772623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":321,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:24.991050+0000","last_change":"2024-09-13T10:13:24.991050+0000","last_active":"2024-09-13T10:13:24.991050+0000","last_peered":"2024-09-13T10:13:24.991050+0000","last_clean":"2024-09-13T10:13:24.991050+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:13:24.991050+0000","last_undegraded":"2024-09-13T10:13:24.991050+0000","last_fullsized":"2024-09-13T10:13:24.991050+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:24.990889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:24.990889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:43.157415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":326,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.942174+0000","last_change":"2024-09-13T10:12:45.768853+0000","last_active":"2024-09-13T10:13:22.942174+0000","last_peered":"2024-09-13T10:13:22.942174+0000","last_clean":"2024-09-13T10:13:22.942174+0000","last_became_active":"2024-09-13T10:12:45.768237+0000","last_became_peered":"2024-09-13T10:12:45.768237+0000","last_unstale":"2024-09-13T10:13:22.942174+0000","last_undegraded":"2024-09-13T10:13:22.942174+0000","last_fullsized":"2024-09-13T10:13:22.942174+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:01.407274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:01.407274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:23.291168+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505514+0000","last_change":"2024-09-13T10:13:09.126344+0000","last_active":"2024-09-13T10:13:23.505514+0000","last_peered":"2024-09-13T10:13:23.505514+0000","last_clean":"2024-09-13T10:13:23.505514+0000","last_became_active":"2024-09-13T10:13:09.125529+0000","last_became_peered":"2024-09-13T10:13:09.125529+0000","last_unstale":"2024-09-13T10:13:23.505514+0000","last_undegraded":"2024-09-13T10:13:23.505514+0000","last_fullsized":"2024-09-13T10:13:23.505514+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:50.136244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:50.136244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:14.321249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"119'332","reported_seq":949,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.547968+0000","last_change":"2024-09-13T10:13:21.995285+0000","last_active":"2024-09-13T10:13:23.547968+0000","last_peered":"2024-09-13T10:13:23.547968+0000","last_clean":"2024-09-13T10:13:23.547968+0000","last_became_active":"2024-09-13T10:13:21.994528+0000","last_became_peered":"2024-09-13T10:13:21.994528+0000","last_unstale":"2024-09-13T10:13:23.547968+0000","last_undegraded":"2024-09-13T10:13:23.547968+0000","last_fullsized":"2024-09-13T10:13:23.547968+0000","mapping_epoch":168,"log_start":"118'329","ondisk_log_start":"118'329","created":30,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'332","last_scrub_stamp":"2024-09-13T10:12:28.697464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:28.697464+0000","objects_scrubbed":39,"log_size":3,"log_dups_size":344,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:40.159242+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28010945,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":327,"num_read_kb":7023,"num_write":433,"num_write_kb":81577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":82643209,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,4,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":282,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505794+0000","last_change":"2024-09-13T10:12:58.899445+0000","last_active":"2024-09-13T10:13:23.505794+0000","last_peered":"2024-09-13T10:13:23.505794+0000","last_clean":"2024-09-13T10:13:23.505794+0000","last_became_active":"2024-09-13T10:12:52.099716+0000","last_became_peered":"2024-09-13T10:12:52.099716+0000","last_unstale":"2024-09-13T10:13:23.505794+0000","last_undegraded":"2024-09-13T10:13:23.505794+0000","last_fullsized":"2024-09-13T10:13:23.505794+0000","mapping_epoch":140,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'112","last_scrub_stamp":"2024-09-13T10:12:58.899291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.899291+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:21.742546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":227,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.548028+0000","last_change":"2024-09-13T10:12:53.845086+0000","last_active":"2024-09-13T10:13:23.548028+0000","last_peered":"2024-09-13T10:13:23.548028+0000","last_clean":"2024-09-13T10:13:23.548028+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:13:23.548028+0000","last_undegraded":"2024-09-13T10:13:23.548028+0000","last_fullsized":"2024-09-13T10:13:23.548028+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T10:12:53.844861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:53.844861+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.984156+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"119'365","reported_seq":711,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.743459+0000","last_change":"2024-09-13T10:13:22.036708+0000","last_active":"2024-09-13T10:13:22.743459+0000","last_peered":"2024-09-13T10:13:22.743459+0000","last_clean":"2024-09-13T10:13:22.743459+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:13:22.743459+0000","last_undegraded":"2024-09-13T10:13:22.743459+0000","last_fullsized":"2024-09-13T10:13:22.743459+0000","mapping_epoch":30,"log_start":"118'362","ondisk_log_start":"118'362","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'365","last_scrub_stamp":"2024-09-13T10:13:22.036498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:22.036498+0000","objects_scrubbed":42,"log_size":3,"log_dups_size":374,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:24.928602+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33541126,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":353,"num_read_kb":13014,"num_write":482,"num_write_kb":101793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":107,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505195+0000","last_change":"2024-09-13T10:12:58.690112+0000","last_active":"2024-09-13T10:13:23.505195+0000","last_peered":"2024-09-13T10:13:23.505195+0000","last_clean":"2024-09-13T10:13:23.505195+0000","last_became_active":"2024-09-13T10:12:45.765925+0000","last_became_peered":"2024-09-13T10:12:45.765925+0000","last_unstale":"2024-09-13T10:13:23.505195+0000","last_undegraded":"2024-09-13T10:13:23.505195+0000","last_fullsized":"2024-09-13T10:13:23.505195+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-13T10:12:58.689917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.689917+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.031308+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"118'346","reported_seq":832,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.492527+0000","last_change":"2024-09-13T10:13:17.957473+0000","last_active":"2024-09-13T10:13:23.492527+0000","last_peered":"2024-09-13T10:13:23.492527+0000","last_clean":"2024-09-13T10:13:23.492527+0000","last_became_active":"2024-09-13T10:13:17.956444+0000","last_became_peered":"2024-09-13T10:13:17.956444+0000","last_unstale":"2024-09-13T10:13:23.492527+0000","last_undegraded":"2024-09-13T10:13:23.492527+0000","last_fullsized":"2024-09-13T10:13:23.492527+0000","mapping_epoch":166,"log_start":"116'344","ondisk_log_start":"116'344","created":30,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'346","last_scrub_stamp":"2024-09-13T10:13:16.741058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:16.741058+0000","objects_scrubbed":40,"log_size":2,"log_dups_size":356,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.865324+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27658528,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":310,"num_read_kb":4757,"num_write":455,"num_write_kb":83526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":33245760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":150,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505388+0000","last_change":"2024-09-13T10:13:22.027674+0000","last_active":"2024-09-13T10:13:23.505388+0000","last_peered":"2024-09-13T10:13:23.505388+0000","last_clean":"2024-09-13T10:13:23.505388+0000","last_became_active":"2024-09-13T10:13:14.495396+0000","last_became_peered":"2024-09-13T10:13:14.495396+0000","last_unstale":"2024-09-13T10:13:23.505388+0000","last_undegraded":"2024-09-13T10:13:23.505388+0000","last_fullsized":"2024-09-13T10:13:23.505388+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:22.027512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:22.027512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:47.896470+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"119'401","reported_seq":728,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505478+0000","last_change":"2024-09-13T10:13:19.721098+0000","last_active":"2024-09-13T10:13:23.505478+0000","last_peered":"2024-09-13T10:13:23.505478+0000","last_clean":"2024-09-13T10:13:23.505478+0000","last_became_active":"2024-09-13T10:12:43.752738+0000","last_became_peered":"2024-09-13T10:12:43.752738+0000","last_unstale":"2024-09-13T10:13:23.505478+0000","last_undegraded":"2024-09-13T10:13:23.505478+0000","last_fullsized":"2024-09-13T10:13:23.505478+0000","mapping_epoch":132,"log_start":"118'398","ondisk_log_start":"118'398","created":30,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'401","last_scrub_stamp":"2024-09-13T10:13:19.720891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:19.720891+0000","objects_scrubbed":27,"log_size":3,"log_dups_size":407,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:49.615137+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20269983,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":375,"num_read_kb":6453,"num_write":499,"num_write_kb":98736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.506028+0000","last_change":"2024-09-13T10:13:12.541253+0000","last_active":"2024-09-13T10:13:23.506028+0000","last_peered":"2024-09-13T10:13:23.506028+0000","last_clean":"2024-09-13T10:13:23.506028+0000","last_became_active":"2024-09-13T10:12:45.763788+0000","last_became_peered":"2024-09-13T10:12:45.763788+0000","last_unstale":"2024-09-13T10:13:23.506028+0000","last_undegraded":"2024-09-13T10:13:23.506028+0000","last_fullsized":"2024-09-13T10:13:23.506028+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:12.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:12.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:19.031321+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":109480582,"num_objects":148,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":1365,"num_read_kb":31247,"num_write":1869,"num_write_kb":365632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":115888969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":302743552,"data_stored":302001554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":14}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247847,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79596,"kb_used_data":48412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94292244,"statfs":{"total":96636764160,"available":96555257856,"internally_reserved":0,"allocated":49573888,"data_stored":49047078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247849,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247849,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60008,"kb_used_data":28824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311832,"statfs":{"total":96636764160,"available":96575315968,"internally_reserved":0,"allocated":29515776,"data_stored":29015046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313257,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28180,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343660,"statfs":{"total":96636764160,"available":96607907840,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313256,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59748,"kb_used_data":28564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312092,"statfs":{"total":96636764160,"available":96575582208,"internally_reserved":0,"allocated":29249536,"data_stored":28777095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247847,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47900,"kb_used_data":16708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323940,"statfs":{"total":96636764160,"available":96587714560,"internally_reserved":0,"allocated":17108992,"data_stored":16659610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280552,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48640,"kb_used_data":17444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323200,"statfs":{"total":96636764160,"available":96586956800,"internally_reserved":0,"allocated":17862656,"data_stored":17389030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,6,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280552,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85588,"kb_used_data":54404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94286252,"statfs":{"total":96636764160,"available":96549122048,"internally_reserved":0,"allocated":55709696,"data_stored":55167627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,8,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280553,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92572,"kb_used_data":61388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94279268,"statfs":{"total":96636764160,"available":96541970432,"internally_reserved":0,"allocated":62861312,"data_stored":62318221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280552,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65740,"kb_used_data":34556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306100,"statfs":{"total":96636764160,"available":96569446400,"internally_reserved":0,"allocated":35385344,"data_stored":34897644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280552,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48636,"kb_used_data":17444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323204,"statfs":{"total":96636764160,"available":96586960896,"internally_reserved":0,"allocated":17862656,"data_stored":17389030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280552,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247847,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247847,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15941632,"data_stored":15901087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53936128,"data_stored":53811109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61681664,"data_stored":61552071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33611776,"data_stored":33541126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":27742208,"data_stored":27658528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":28069888,"data_stored":28010945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":48394240,"data_stored":48280928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:13:30.013 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T10:13:30.013 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T10:13:30.025 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T10:13:30.025 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:13:30.305 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:13:30.306 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:13:30.322 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-13T10:13:28.985413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110070969,"num_objects":152,"num_object_clones":0,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":1451,"num_read_kb":31320,"num_write":2016,"num_write_kb":368306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":115889007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":39,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":802320,"kb_used_data":315820,"kb_used_omap":24,"kb_used_meta":486183,"kb_avail":1509147120,"statfs":{"total":1546188226560,"available":1545366650880,"internally_reserved":0,"allocated":323399680,"data_stored":316023488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":497851471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-39,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":28010945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002975"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":151,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505805+0000","last_change":"2024-09-13T10:13:21.781722+0000","last_active":"2024-09-13T10:13:23.505805+0000","last_peered":"2024-09-13T10:13:23.505805+0000","last_clean":"2024-09-13T10:13:23.505805+0000","last_became_active":"2024-09-13T10:12:52.200952+0000","last_became_peered":"2024-09-13T10:12:52.200952+0000","last_unstale":"2024-09-13T10:13:23.505805+0000","last_undegraded":"2024-09-13T10:13:23.505805+0000","last_fullsized":"2024-09-13T10:13:23.505805+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:21.781648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:21.781648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:37.772623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":321,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:24.991050+0000","last_change":"2024-09-13T10:13:24.991050+0000","last_active":"2024-09-13T10:13:24.991050+0000","last_peered":"2024-09-13T10:13:24.991050+0000","last_clean":"2024-09-13T10:13:24.991050+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:13:24.991050+0000","last_undegraded":"2024-09-13T10:13:24.991050+0000","last_fullsized":"2024-09-13T10:13:24.991050+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:24.990889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:24.990889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:43.157415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":326,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.942174+0000","last_change":"2024-09-13T10:12:45.768853+0000","last_active":"2024-09-13T10:13:22.942174+0000","last_peered":"2024-09-13T10:13:22.942174+0000","last_clean":"2024-09-13T10:13:22.942174+0000","last_became_active":"2024-09-13T10:12:45.768237+0000","last_became_peered":"2024-09-13T10:12:45.768237+0000","last_unstale":"2024-09-13T10:13:22.942174+0000","last_undegraded":"2024-09-13T10:13:22.942174+0000","last_fullsized":"2024-09-13T10:13:22.942174+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:01.407274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:01.407274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:23.291168+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505514+0000","last_change":"2024-09-13T10:13:09.126344+0000","last_active":"2024-09-13T10:13:23.505514+0000","last_peered":"2024-09-13T10:13:23.505514+0000","last_clean":"2024-09-13T10:13:23.505514+0000","last_became_active":"2024-09-13T10:13:09.125529+0000","last_became_peered":"2024-09-13T10:13:09.125529+0000","last_unstale":"2024-09-13T10:13:23.505514+0000","last_undegraded":"2024-09-13T10:13:23.505514+0000","last_fullsized":"2024-09-13T10:13:23.505514+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:50.136244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:50.136244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:14.321249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"119'332","reported_seq":949,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.547968+0000","last_change":"2024-09-13T10:13:21.995285+0000","last_active":"2024-09-13T10:13:23.547968+0000","last_peered":"2024-09-13T10:13:23.547968+0000","last_clean":"2024-09-13T10:13:23.547968+0000","last_became_active":"2024-09-13T10:13:21.994528+0000","last_became_peered":"2024-09-13T10:13:21.994528+0000","last_unstale":"2024-09-13T10:13:23.547968+0000","last_undegraded":"2024-09-13T10:13:23.547968+0000","last_fullsized":"2024-09-13T10:13:23.547968+0000","mapping_epoch":168,"log_start":"118'329","ondisk_log_start":"118'329","created":30,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'332","last_scrub_stamp":"2024-09-13T10:12:28.697464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:28.697464+0000","objects_scrubbed":39,"log_size":3,"log_dups_size":344,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:40.159242+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28010945,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":327,"num_read_kb":7023,"num_write":433,"num_write_kb":81577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":82643209,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,4,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":282,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505794+0000","last_change":"2024-09-13T10:12:58.899445+0000","last_active":"2024-09-13T10:13:23.505794+0000","last_peered":"2024-09-13T10:13:23.505794+0000","last_clean":"2024-09-13T10:13:23.505794+0000","last_became_active":"2024-09-13T10:12:52.099716+0000","last_became_peered":"2024-09-13T10:12:52.099716+0000","last_unstale":"2024-09-13T10:13:23.505794+0000","last_undegraded":"2024-09-13T10:13:23.505794+0000","last_fullsized":"2024-09-13T10:13:23.505794+0000","mapping_epoch":140,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'112","last_scrub_stamp":"2024-09-13T10:12:58.899291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.899291+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:21.742546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":227,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.548028+0000","last_change":"2024-09-13T10:12:53.845086+0000","last_active":"2024-09-13T10:13:23.548028+0000","last_peered":"2024-09-13T10:13:23.548028+0000","last_clean":"2024-09-13T10:13:23.548028+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:13:23.548028+0000","last_undegraded":"2024-09-13T10:13:23.548028+0000","last_fullsized":"2024-09-13T10:13:23.548028+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T10:12:53.844861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:53.844861+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.984156+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"119'365","reported_seq":711,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.743459+0000","last_change":"2024-09-13T10:13:22.036708+0000","last_active":"2024-09-13T10:13:22.743459+0000","last_peered":"2024-09-13T10:13:22.743459+0000","last_clean":"2024-09-13T10:13:22.743459+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:13:22.743459+0000","last_undegraded":"2024-09-13T10:13:22.743459+0000","last_fullsized":"2024-09-13T10:13:22.743459+0000","mapping_epoch":30,"log_start":"118'362","ondisk_log_start":"118'362","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'365","last_scrub_stamp":"2024-09-13T10:13:22.036498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:22.036498+0000","objects_scrubbed":42,"log_size":3,"log_dups_size":374,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:24.928602+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33541126,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":353,"num_read_kb":13014,"num_write":482,"num_write_kb":101793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":107,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505195+0000","last_change":"2024-09-13T10:12:58.690112+0000","last_active":"2024-09-13T10:13:23.505195+0000","last_peered":"2024-09-13T10:13:23.505195+0000","last_clean":"2024-09-13T10:13:23.505195+0000","last_became_active":"2024-09-13T10:12:45.765925+0000","last_became_peered":"2024-09-13T10:12:45.765925+0000","last_unstale":"2024-09-13T10:13:23.505195+0000","last_undegraded":"2024-09-13T10:13:23.505195+0000","last_fullsized":"2024-09-13T10:13:23.505195+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-13T10:12:58.689917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.689917+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.031308+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"118'346","reported_seq":832,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.492527+0000","last_change":"2024-09-13T10:13:17.957473+0000","last_active":"2024-09-13T10:13:23.492527+0000","last_peered":"2024-09-13T10:13:23.492527+0000","last_clean":"2024-09-13T10:13:23.492527+0000","last_became_active":"2024-09-13T10:13:17.956444+0000","last_became_peered":"2024-09-13T10:13:17.956444+0000","last_unstale":"2024-09-13T10:13:23.492527+0000","last_undegraded":"2024-09-13T10:13:23.492527+0000","last_fullsized":"2024-09-13T10:13:23.492527+0000","mapping_epoch":166,"log_start":"116'344","ondisk_log_start":"116'344","created":30,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'346","last_scrub_stamp":"2024-09-13T10:13:16.741058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:16.741058+0000","objects_scrubbed":40,"log_size":2,"log_dups_size":356,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.865324+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27658528,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":310,"num_read_kb":4757,"num_write":455,"num_write_kb":83526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":33245760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":150,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505388+0000","last_change":"2024-09-13T10:13:22.027674+0000","last_active":"2024-09-13T10:13:23.505388+0000","last_peered":"2024-09-13T10:13:23.505388+0000","last_clean":"2024-09-13T10:13:23.505388+0000","last_became_active":"2024-09-13T10:13:14.495396+0000","last_became_peered":"2024-09-13T10:13:14.495396+0000","last_unstale":"2024-09-13T10:13:23.505388+0000","last_undegraded":"2024-09-13T10:13:23.505388+0000","last_fullsized":"2024-09-13T10:13:23.505388+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:22.027512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:22.027512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:47.896470+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"119'401","reported_seq":728,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505478+0000","last_change":"2024-09-13T10:13:19.721098+0000","last_active":"2024-09-13T10:13:23.505478+0000","last_peered":"2024-09-13T10:13:23.505478+0000","last_clean":"2024-09-13T10:13:23.505478+0000","last_became_active":"2024-09-13T10:12:43.752738+0000","last_became_peered":"2024-09-13T10:12:43.752738+0000","last_unstale":"2024-09-13T10:13:23.505478+0000","last_undegraded":"2024-09-13T10:13:23.505478+0000","last_fullsized":"2024-09-13T10:13:23.505478+0000","mapping_epoch":132,"log_start":"118'398","ondisk_log_start":"118'398","created":30,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'401","last_scrub_stamp":"2024-09-13T10:13:19.720891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:19.720891+0000","objects_scrubbed":27,"log_size":3,"log_dups_size":407,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:49.615137+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20269983,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":375,"num_read_kb":6453,"num_write":499,"num_write_kb":98736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.506028+0000","last_change":"2024-09-13T10:13:12.541253+0000","last_active":"2024-09-13T10:13:23.506028+0000","last_peered":"2024-09-13T10:13:23.506028+0000","last_clean":"2024-09-13T10:13:23.506028+0000","last_became_active":"2024-09-13T10:12:45.763788+0000","last_became_peered":"2024-09-13T10:12:45.763788+0000","last_unstale":"2024-09-13T10:13:23.506028+0000","last_undegraded":"2024-09-13T10:13:23.506028+0000","last_fullsized":"2024-09-13T10:13:23.506028+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:12.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:12.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:19.031321+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":109480582,"num_objects":148,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":1365,"num_read_kb":31247,"num_write":1869,"num_write_kb":365632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":115888969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":302743552,"data_stored":302001554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":14}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247847,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79596,"kb_used_data":48412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94292244,"statfs":{"total":96636764160,"available":96555257856,"internally_reserved":0,"allocated":49573888,"data_stored":49047078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247849,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247849,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60008,"kb_used_data":28824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311832,"statfs":{"total":96636764160,"available":96575315968,"internally_reserved":0,"allocated":29515776,"data_stored":29015046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313257,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28180,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343660,"statfs":{"total":96636764160,"available":96607907840,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313256,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59748,"kb_used_data":28564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312092,"statfs":{"total":96636764160,"available":96575582208,"internally_reserved":0,"allocated":29249536,"data_stored":28777095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247847,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47900,"kb_used_data":16708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323940,"statfs":{"total":96636764160,"available":96587714560,"internally_reserved":0,"allocated":17108992,"data_stored":16659610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280552,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48640,"kb_used_data":17444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323200,"statfs":{"total":96636764160,"available":96586956800,"internally_reserved":0,"allocated":17862656,"data_stored":17389030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,6,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280552,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85588,"kb_used_data":54404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94286252,"statfs":{"total":96636764160,"available":96549122048,"internally_reserved":0,"allocated":55709696,"data_stored":55167627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,8,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280553,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92572,"kb_used_data":61388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94279268,"statfs":{"total":96636764160,"available":96541970432,"internally_reserved":0,"allocated":62861312,"data_stored":62318221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280552,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65740,"kb_used_data":34556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306100,"statfs":{"total":96636764160,"available":96569446400,"internally_reserved":0,"allocated":35385344,"data_stored":34897644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280552,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48636,"kb_used_data":17444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323204,"statfs":{"total":96636764160,"available":96586960896,"internally_reserved":0,"allocated":17862656,"data_stored":17389030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280552,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247847,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247847,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15941632,"data_stored":15901087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53936128,"data_stored":53811109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61681664,"data_stored":61552071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33611776,"data_stored":33541126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":27742208,"data_stored":27658528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":28069888,"data_stored":28010945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":48394240,"data_stored":48280928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:13:30.324 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:13:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:13:30.622 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:13:30.639 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-13T10:13:28.985413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110070969,"num_objects":152,"num_object_clones":0,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":1451,"num_read_kb":31320,"num_write":2016,"num_write_kb":368306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":115889007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":39,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":802320,"kb_used_data":315820,"kb_used_omap":24,"kb_used_meta":486183,"kb_avail":1509147120,"statfs":{"total":1546188226560,"available":1545366650880,"internally_reserved":0,"allocated":323399680,"data_stored":316023488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":497851471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-39,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":28010945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002975"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":151,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505805+0000","last_change":"2024-09-13T10:13:21.781722+0000","last_active":"2024-09-13T10:13:23.505805+0000","last_peered":"2024-09-13T10:13:23.505805+0000","last_clean":"2024-09-13T10:13:23.505805+0000","last_became_active":"2024-09-13T10:12:52.200952+0000","last_became_peered":"2024-09-13T10:12:52.200952+0000","last_unstale":"2024-09-13T10:13:23.505805+0000","last_undegraded":"2024-09-13T10:13:23.505805+0000","last_fullsized":"2024-09-13T10:13:23.505805+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:21.781648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:21.781648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:37.772623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":321,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:24.991050+0000","last_change":"2024-09-13T10:13:24.991050+0000","last_active":"2024-09-13T10:13:24.991050+0000","last_peered":"2024-09-13T10:13:24.991050+0000","last_clean":"2024-09-13T10:13:24.991050+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:13:24.991050+0000","last_undegraded":"2024-09-13T10:13:24.991050+0000","last_fullsized":"2024-09-13T10:13:24.991050+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:24.990889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:24.990889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:43.157415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":326,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.942174+0000","last_change":"2024-09-13T10:12:45.768853+0000","last_active":"2024-09-13T10:13:22.942174+0000","last_peered":"2024-09-13T10:13:22.942174+0000","last_clean":"2024-09-13T10:13:22.942174+0000","last_became_active":"2024-09-13T10:12:45.768237+0000","last_became_peered":"2024-09-13T10:12:45.768237+0000","last_unstale":"2024-09-13T10:13:22.942174+0000","last_undegraded":"2024-09-13T10:13:22.942174+0000","last_fullsized":"2024-09-13T10:13:22.942174+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:01.407274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:01.407274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:23.291168+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505514+0000","last_change":"2024-09-13T10:13:09.126344+0000","last_active":"2024-09-13T10:13:23.505514+0000","last_peered":"2024-09-13T10:13:23.505514+0000","last_clean":"2024-09-13T10:13:23.505514+0000","last_became_active":"2024-09-13T10:13:09.125529+0000","last_became_peered":"2024-09-13T10:13:09.125529+0000","last_unstale":"2024-09-13T10:13:23.505514+0000","last_undegraded":"2024-09-13T10:13:23.505514+0000","last_fullsized":"2024-09-13T10:13:23.505514+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:50.136244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:50.136244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:14.321249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"119'332","reported_seq":949,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.547968+0000","last_change":"2024-09-13T10:13:21.995285+0000","last_active":"2024-09-13T10:13:23.547968+0000","last_peered":"2024-09-13T10:13:23.547968+0000","last_clean":"2024-09-13T10:13:23.547968+0000","last_became_active":"2024-09-13T10:13:21.994528+0000","last_became_peered":"2024-09-13T10:13:21.994528+0000","last_unstale":"2024-09-13T10:13:23.547968+0000","last_undegraded":"2024-09-13T10:13:23.547968+0000","last_fullsized":"2024-09-13T10:13:23.547968+0000","mapping_epoch":168,"log_start":"118'329","ondisk_log_start":"118'329","created":30,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'332","last_scrub_stamp":"2024-09-13T10:12:28.697464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:28.697464+0000","objects_scrubbed":39,"log_size":3,"log_dups_size":344,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:40.159242+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28010945,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":327,"num_read_kb":7023,"num_write":433,"num_write_kb":81577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":82643209,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,4,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":282,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505794+0000","last_change":"2024-09-13T10:12:58.899445+0000","last_active":"2024-09-13T10:13:23.505794+0000","last_peered":"2024-09-13T10:13:23.505794+0000","last_clean":"2024-09-13T10:13:23.505794+0000","last_became_active":"2024-09-13T10:12:52.099716+0000","last_became_peered":"2024-09-13T10:12:52.099716+0000","last_unstale":"2024-09-13T10:13:23.505794+0000","last_undegraded":"2024-09-13T10:13:23.505794+0000","last_fullsized":"2024-09-13T10:13:23.505794+0000","mapping_epoch":140,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'112","last_scrub_stamp":"2024-09-13T10:12:58.899291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.899291+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:21.742546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":227,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.548028+0000","last_change":"2024-09-13T10:12:53.845086+0000","last_active":"2024-09-13T10:13:23.548028+0000","last_peered":"2024-09-13T10:13:23.548028+0000","last_clean":"2024-09-13T10:13:23.548028+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:13:23.548028+0000","last_undegraded":"2024-09-13T10:13:23.548028+0000","last_fullsized":"2024-09-13T10:13:23.548028+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T10:12:53.844861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:53.844861+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.984156+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"119'365","reported_seq":711,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.743459+0000","last_change":"2024-09-13T10:13:22.036708+0000","last_active":"2024-09-13T10:13:22.743459+0000","last_peered":"2024-09-13T10:13:22.743459+0000","last_clean":"2024-09-13T10:13:22.743459+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:13:22.743459+0000","last_undegraded":"2024-09-13T10:13:22.743459+0000","last_fullsized":"2024-09-13T10:13:22.743459+0000","mapping_epoch":30,"log_start":"118'362","ondisk_log_start":"118'362","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'365","last_scrub_stamp":"2024-09-13T10:13:22.036498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:22.036498+0000","objects_scrubbed":42,"log_size":3,"log_dups_size":374,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:24.928602+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33541126,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":353,"num_read_kb":13014,"num_write":482,"num_write_kb":101793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":107,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505195+0000","last_change":"2024-09-13T10:12:58.690112+0000","last_active":"2024-09-13T10:13:23.505195+0000","last_peered":"2024-09-13T10:13:23.505195+0000","last_clean":"2024-09-13T10:13:23.505195+0000","last_became_active":"2024-09-13T10:12:45.765925+0000","last_became_peered":"2024-09-13T10:12:45.765925+0000","last_unstale":"2024-09-13T10:13:23.505195+0000","last_undegraded":"2024-09-13T10:13:23.505195+0000","last_fullsized":"2024-09-13T10:13:23.505195+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-13T10:12:58.689917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.689917+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.031308+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"118'346","reported_seq":832,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.492527+0000","last_change":"2024-09-13T10:13:17.957473+0000","last_active":"2024-09-13T10:13:23.492527+0000","last_peered":"2024-09-13T10:13:23.492527+0000","last_clean":"2024-09-13T10:13:23.492527+0000","last_became_active":"2024-09-13T10:13:17.956444+0000","last_became_peered":"2024-09-13T10:13:17.956444+0000","last_unstale":"2024-09-13T10:13:23.492527+0000","last_undegraded":"2024-09-13T10:13:23.492527+0000","last_fullsized":"2024-09-13T10:13:23.492527+0000","mapping_epoch":166,"log_start":"116'344","ondisk_log_start":"116'344","created":30,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'346","last_scrub_stamp":"2024-09-13T10:13:16.741058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:16.741058+0000","objects_scrubbed":40,"log_size":2,"log_dups_size":356,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.865324+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27658528,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":310,"num_read_kb":4757,"num_write":455,"num_write_kb":83526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":33245760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":150,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505388+0000","last_change":"2024-09-13T10:13:22.027674+0000","last_active":"2024-09-13T10:13:23.505388+0000","last_peered":"2024-09-13T10:13:23.505388+0000","last_clean":"2024-09-13T10:13:23.505388+0000","last_became_active":"2024-09-13T10:13:14.495396+0000","last_became_peered":"2024-09-13T10:13:14.495396+0000","last_unstale":"2024-09-13T10:13:23.505388+0000","last_undegraded":"2024-09-13T10:13:23.505388+0000","last_fullsized":"2024-09-13T10:13:23.505388+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:22.027512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:22.027512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:47.896470+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"119'401","reported_seq":728,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505478+0000","last_change":"2024-09-13T10:13:19.721098+0000","last_active":"2024-09-13T10:13:23.505478+0000","last_peered":"2024-09-13T10:13:23.505478+0000","last_clean":"2024-09-13T10:13:23.505478+0000","last_became_active":"2024-09-13T10:12:43.752738+0000","last_became_peered":"2024-09-13T10:12:43.752738+0000","last_unstale":"2024-09-13T10:13:23.505478+0000","last_undegraded":"2024-09-13T10:13:23.505478+0000","last_fullsized":"2024-09-13T10:13:23.505478+0000","mapping_epoch":132,"log_start":"118'398","ondisk_log_start":"118'398","created":30,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'401","last_scrub_stamp":"2024-09-13T10:13:19.720891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:19.720891+0000","objects_scrubbed":27,"log_size":3,"log_dups_size":407,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:49.615137+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20269983,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":375,"num_read_kb":6453,"num_write":499,"num_write_kb":98736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.506028+0000","last_change":"2024-09-13T10:13:12.541253+0000","last_active":"2024-09-13T10:13:23.506028+0000","last_peered":"2024-09-13T10:13:23.506028+0000","last_clean":"2024-09-13T10:13:23.506028+0000","last_became_active":"2024-09-13T10:12:45.763788+0000","last_became_peered":"2024-09-13T10:12:45.763788+0000","last_unstale":"2024-09-13T10:13:23.506028+0000","last_undegraded":"2024-09-13T10:13:23.506028+0000","last_fullsized":"2024-09-13T10:13:23.506028+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:12.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:12.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:19.031321+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":109480582,"num_objects":148,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":1365,"num_read_kb":31247,"num_write":1869,"num_write_kb":365632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":115888969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":302743552,"data_stored":302001554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":14}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247847,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79596,"kb_used_data":48412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94292244,"statfs":{"total":96636764160,"available":96555257856,"internally_reserved":0,"allocated":49573888,"data_stored":49047078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247849,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247849,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60008,"kb_used_data":28824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311832,"statfs":{"total":96636764160,"available":96575315968,"internally_reserved":0,"allocated":29515776,"data_stored":29015046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313257,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28180,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343660,"statfs":{"total":96636764160,"available":96607907840,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313256,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59748,"kb_used_data":28564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312092,"statfs":{"total":96636764160,"available":96575582208,"internally_reserved":0,"allocated":29249536,"data_stored":28777095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247847,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47900,"kb_used_data":16708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323940,"statfs":{"total":96636764160,"available":96587714560,"internally_reserved":0,"allocated":17108992,"data_stored":16659610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280552,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48640,"kb_used_data":17444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323200,"statfs":{"total":96636764160,"available":96586956800,"internally_reserved":0,"allocated":17862656,"data_stored":17389030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,6,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280552,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85588,"kb_used_data":54404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94286252,"statfs":{"total":96636764160,"available":96549122048,"internally_reserved":0,"allocated":55709696,"data_stored":55167627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,8,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280553,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92572,"kb_used_data":61388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94279268,"statfs":{"total":96636764160,"available":96541970432,"internally_reserved":0,"allocated":62861312,"data_stored":62318221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280552,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65740,"kb_used_data":34556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306100,"statfs":{"total":96636764160,"available":96569446400,"internally_reserved":0,"allocated":35385344,"data_stored":34897644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280552,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48636,"kb_used_data":17444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323204,"statfs":{"total":96636764160,"available":96586960896,"internally_reserved":0,"allocated":17862656,"data_stored":17389030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280552,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247847,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247847,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15941632,"data_stored":15901087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53936128,"data_stored":53811109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61681664,"data_stored":61552071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33611776,"data_stored":33541126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":27742208,"data_stored":27658528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":28069888,"data_stored":28010945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":48394240,"data_stored":48280928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:13:30.640 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T10:13:30.641 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:13:30.925 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:13:30.926 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:13:30.942 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-13T10:13:28.985413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110070969,"num_objects":152,"num_object_clones":0,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":1451,"num_read_kb":31320,"num_write":2016,"num_write_kb":368306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":115889007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":39,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":802320,"kb_used_data":315820,"kb_used_omap":24,"kb_used_meta":486183,"kb_avail":1509147120,"statfs":{"total":1546188226560,"available":1545366650880,"internally_reserved":0,"allocated":323399680,"data_stored":316023488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":497851471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-39,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":28010945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002975"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":151,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505805+0000","last_change":"2024-09-13T10:13:21.781722+0000","last_active":"2024-09-13T10:13:23.505805+0000","last_peered":"2024-09-13T10:13:23.505805+0000","last_clean":"2024-09-13T10:13:23.505805+0000","last_became_active":"2024-09-13T10:12:52.200952+0000","last_became_peered":"2024-09-13T10:12:52.200952+0000","last_unstale":"2024-09-13T10:13:23.505805+0000","last_undegraded":"2024-09-13T10:13:23.505805+0000","last_fullsized":"2024-09-13T10:13:23.505805+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:21.781648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:21.781648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:37.772623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":321,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:24.991050+0000","last_change":"2024-09-13T10:13:24.991050+0000","last_active":"2024-09-13T10:13:24.991050+0000","last_peered":"2024-09-13T10:13:24.991050+0000","last_clean":"2024-09-13T10:13:24.991050+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:13:24.991050+0000","last_undegraded":"2024-09-13T10:13:24.991050+0000","last_fullsized":"2024-09-13T10:13:24.991050+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:24.990889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:24.990889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:43.157415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":326,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.942174+0000","last_change":"2024-09-13T10:12:45.768853+0000","last_active":"2024-09-13T10:13:22.942174+0000","last_peered":"2024-09-13T10:13:22.942174+0000","last_clean":"2024-09-13T10:13:22.942174+0000","last_became_active":"2024-09-13T10:12:45.768237+0000","last_became_peered":"2024-09-13T10:12:45.768237+0000","last_unstale":"2024-09-13T10:13:22.942174+0000","last_undegraded":"2024-09-13T10:13:22.942174+0000","last_fullsized":"2024-09-13T10:13:22.942174+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:01.407274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:01.407274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:23.291168+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505514+0000","last_change":"2024-09-13T10:13:09.126344+0000","last_active":"2024-09-13T10:13:23.505514+0000","last_peered":"2024-09-13T10:13:23.505514+0000","last_clean":"2024-09-13T10:13:23.505514+0000","last_became_active":"2024-09-13T10:13:09.125529+0000","last_became_peered":"2024-09-13T10:13:09.125529+0000","last_unstale":"2024-09-13T10:13:23.505514+0000","last_undegraded":"2024-09-13T10:13:23.505514+0000","last_fullsized":"2024-09-13T10:13:23.505514+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:50.136244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:50.136244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:14.321249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"119'332","reported_seq":949,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.547968+0000","last_change":"2024-09-13T10:13:21.995285+0000","last_active":"2024-09-13T10:13:23.547968+0000","last_peered":"2024-09-13T10:13:23.547968+0000","last_clean":"2024-09-13T10:13:23.547968+0000","last_became_active":"2024-09-13T10:13:21.994528+0000","last_became_peered":"2024-09-13T10:13:21.994528+0000","last_unstale":"2024-09-13T10:13:23.547968+0000","last_undegraded":"2024-09-13T10:13:23.547968+0000","last_fullsized":"2024-09-13T10:13:23.547968+0000","mapping_epoch":168,"log_start":"118'329","ondisk_log_start":"118'329","created":30,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'332","last_scrub_stamp":"2024-09-13T10:12:28.697464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:28.697464+0000","objects_scrubbed":39,"log_size":3,"log_dups_size":344,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:40.159242+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28010945,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":327,"num_read_kb":7023,"num_write":433,"num_write_kb":81577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":82643209,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,4,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":282,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505794+0000","last_change":"2024-09-13T10:12:58.899445+0000","last_active":"2024-09-13T10:13:23.505794+0000","last_peered":"2024-09-13T10:13:23.505794+0000","last_clean":"2024-09-13T10:13:23.505794+0000","last_became_active":"2024-09-13T10:12:52.099716+0000","last_became_peered":"2024-09-13T10:12:52.099716+0000","last_unstale":"2024-09-13T10:13:23.505794+0000","last_undegraded":"2024-09-13T10:13:23.505794+0000","last_fullsized":"2024-09-13T10:13:23.505794+0000","mapping_epoch":140,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'112","last_scrub_stamp":"2024-09-13T10:12:58.899291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.899291+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:21.742546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":227,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.548028+0000","last_change":"2024-09-13T10:12:53.845086+0000","last_active":"2024-09-13T10:13:23.548028+0000","last_peered":"2024-09-13T10:13:23.548028+0000","last_clean":"2024-09-13T10:13:23.548028+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:13:23.548028+0000","last_undegraded":"2024-09-13T10:13:23.548028+0000","last_fullsized":"2024-09-13T10:13:23.548028+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T10:12:53.844861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:53.844861+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.984156+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"119'365","reported_seq":711,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.743459+0000","last_change":"2024-09-13T10:13:22.036708+0000","last_active":"2024-09-13T10:13:22.743459+0000","last_peered":"2024-09-13T10:13:22.743459+0000","last_clean":"2024-09-13T10:13:22.743459+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:13:22.743459+0000","last_undegraded":"2024-09-13T10:13:22.743459+0000","last_fullsized":"2024-09-13T10:13:22.743459+0000","mapping_epoch":30,"log_start":"118'362","ondisk_log_start":"118'362","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'365","last_scrub_stamp":"2024-09-13T10:13:22.036498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:22.036498+0000","objects_scrubbed":42,"log_size":3,"log_dups_size":374,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:24.928602+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33541126,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":353,"num_read_kb":13014,"num_write":482,"num_write_kb":101793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":107,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505195+0000","last_change":"2024-09-13T10:12:58.690112+0000","last_active":"2024-09-13T10:13:23.505195+0000","last_peered":"2024-09-13T10:13:23.505195+0000","last_clean":"2024-09-13T10:13:23.505195+0000","last_became_active":"2024-09-13T10:12:45.765925+0000","last_became_peered":"2024-09-13T10:12:45.765925+0000","last_unstale":"2024-09-13T10:13:23.505195+0000","last_undegraded":"2024-09-13T10:13:23.505195+0000","last_fullsized":"2024-09-13T10:13:23.505195+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-13T10:12:58.689917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.689917+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.031308+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"118'346","reported_seq":832,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.492527+0000","last_change":"2024-09-13T10:13:17.957473+0000","last_active":"2024-09-13T10:13:23.492527+0000","last_peered":"2024-09-13T10:13:23.492527+0000","last_clean":"2024-09-13T10:13:23.492527+0000","last_became_active":"2024-09-13T10:13:17.956444+0000","last_became_peered":"2024-09-13T10:13:17.956444+0000","last_unstale":"2024-09-13T10:13:23.492527+0000","last_undegraded":"2024-09-13T10:13:23.492527+0000","last_fullsized":"2024-09-13T10:13:23.492527+0000","mapping_epoch":166,"log_start":"116'344","ondisk_log_start":"116'344","created":30,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"118'346","last_scrub_stamp":"2024-09-13T10:13:16.741058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:16.741058+0000","objects_scrubbed":40,"log_size":2,"log_dups_size":356,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.865324+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27658528,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":310,"num_read_kb":4757,"num_write":455,"num_write_kb":83526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":33245760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":150,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505388+0000","last_change":"2024-09-13T10:13:22.027674+0000","last_active":"2024-09-13T10:13:23.505388+0000","last_peered":"2024-09-13T10:13:23.505388+0000","last_clean":"2024-09-13T10:13:23.505388+0000","last_became_active":"2024-09-13T10:13:14.495396+0000","last_became_peered":"2024-09-13T10:13:14.495396+0000","last_unstale":"2024-09-13T10:13:23.505388+0000","last_undegraded":"2024-09-13T10:13:23.505388+0000","last_fullsized":"2024-09-13T10:13:23.505388+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:22.027512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:22.027512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:47.896470+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"119'401","reported_seq":728,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505478+0000","last_change":"2024-09-13T10:13:19.721098+0000","last_active":"2024-09-13T10:13:23.505478+0000","last_peered":"2024-09-13T10:13:23.505478+0000","last_clean":"2024-09-13T10:13:23.505478+0000","last_became_active":"2024-09-13T10:12:43.752738+0000","last_became_peered":"2024-09-13T10:12:43.752738+0000","last_unstale":"2024-09-13T10:13:23.505478+0000","last_undegraded":"2024-09-13T10:13:23.505478+0000","last_fullsized":"2024-09-13T10:13:23.505478+0000","mapping_epoch":132,"log_start":"118'398","ondisk_log_start":"118'398","created":30,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'401","last_scrub_stamp":"2024-09-13T10:13:19.720891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:19.720891+0000","objects_scrubbed":27,"log_size":3,"log_dups_size":407,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:49.615137+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20269983,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":375,"num_read_kb":6453,"num_write":499,"num_write_kb":98736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.506028+0000","last_change":"2024-09-13T10:13:12.541253+0000","last_active":"2024-09-13T10:13:23.506028+0000","last_peered":"2024-09-13T10:13:23.506028+0000","last_clean":"2024-09-13T10:13:23.506028+0000","last_became_active":"2024-09-13T10:12:45.763788+0000","last_became_peered":"2024-09-13T10:12:45.763788+0000","last_unstale":"2024-09-13T10:13:23.506028+0000","last_undegraded":"2024-09-13T10:13:23.506028+0000","last_fullsized":"2024-09-13T10:13:23.506028+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:12.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:12.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:19.031321+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":109480582,"num_objects":148,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":1365,"num_read_kb":31247,"num_write":1869,"num_write_kb":365632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":115888969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":302743552,"data_stored":302001554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11,"ondisk_log_size":11,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":14}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247847,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79596,"kb_used_data":48412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94292244,"statfs":{"total":96636764160,"available":96555257856,"internally_reserved":0,"allocated":49573888,"data_stored":49047078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247849,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247849,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60008,"kb_used_data":28824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311832,"statfs":{"total":96636764160,"available":96575315968,"internally_reserved":0,"allocated":29515776,"data_stored":29015046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313257,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28180,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343660,"statfs":{"total":96636764160,"available":96607907840,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313256,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59748,"kb_used_data":28564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312092,"statfs":{"total":96636764160,"available":96575582208,"internally_reserved":0,"allocated":29249536,"data_stored":28777095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247847,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47900,"kb_used_data":16708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323940,"statfs":{"total":96636764160,"available":96587714560,"internally_reserved":0,"allocated":17108992,"data_stored":16659610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280552,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48640,"kb_used_data":17444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323200,"statfs":{"total":96636764160,"available":96586956800,"internally_reserved":0,"allocated":17862656,"data_stored":17389030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,6,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280552,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85588,"kb_used_data":54404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94286252,"statfs":{"total":96636764160,"available":96549122048,"internally_reserved":0,"allocated":55709696,"data_stored":55167627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,8,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280553,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92572,"kb_used_data":61388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94279268,"statfs":{"total":96636764160,"available":96541970432,"internally_reserved":0,"allocated":62861312,"data_stored":62318221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280551,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280552,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65740,"kb_used_data":34556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306100,"statfs":{"total":96636764160,"available":96569446400,"internally_reserved":0,"allocated":35385344,"data_stored":34897644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280552,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48636,"kb_used_data":17444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323204,"statfs":{"total":96636764160,"available":96586960896,"internally_reserved":0,"allocated":17862656,"data_stored":17389030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280552,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280552,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247847,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247847,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15941632,"data_stored":15901087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53936128,"data_stored":53811109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61681664,"data_stored":61552071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33611776,"data_stored":33541126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":27742208,"data_stored":27658528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":28069888,"data_stored":28010945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":48394240,"data_stored":48280928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:13:30.943 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T10:13:31.233 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:13:31.234 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":170,"fsid":"c1e91d47-9492-4461-a043-8698fd74ccf3","created":"2024-09-13T10:10:14.019912+0000","modified":"2024-09-13T10:13:22.737727+0000","last_up_change":"2024-09-13T10:10:28.089885+0000","last_in_change":"2024-09-13T10:12:44.738215+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T10:10:28.924860+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":27,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":0.68999999761581421,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T10:10:28.972519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.789999961853027,"score_stable":15.789999961853027,"optimal_score":0.18999999761581421,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"low_tier","create_time":"2024-09-13T10:10:41.320719+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7d92188-c27c-4744-a330-4b7105e0f2d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1977392321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":1977392321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6815","nonce":1977392321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1977392321}]},"public_addr":"172.21.15.73:6812/1977392321","cluster_addr":"172.21.15.73:6813/1977392321","heartbeat_back_addr":"172.21.15.73:6815/1977392321","heartbeat_front_addr":"172.21.15.73:6814/1977392321","state":["exists","up"]},{"osd":1,"uuid":"e900b771-7c96-4b5d-93d7-35a8ea6c026a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":165,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":1228102864}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6809","nonce":1228102864}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6811","nonce":1228102864}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":1228102864}]},"public_addr":"172.21.15.136:6808/1228102864","cluster_addr":"172.21.15.136:6809/1228102864","heartbeat_back_addr":"172.21.15.136:6811/1228102864","heartbeat_front_addr":"172.21.15.136:6810/1228102864","state":["exists","up"]},{"osd":2,"uuid":"28b7aa59-33b2-4e24-8f3d-758f90e25f68","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6800","nonce":3314820936}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6801","nonce":3314820936}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6803","nonce":3314820936}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6802","nonce":3314820936}]},"public_addr":"172.21.15.181:6800/3314820936","cluster_addr":"172.21.15.181:6801/3314820936","heartbeat_back_addr":"172.21.15.181:6803/3314820936","heartbeat_front_addr":"172.21.15.181:6802/3314820936","state":["exists","up"]},{"osd":3,"uuid":"a93d2df5-781e-49e1-a507-1718b6f954ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6801","nonce":2558565460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":2558565460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":2558565460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6803","nonce":2558565460}]},"public_addr":"172.21.15.195:6801/2558565460","cluster_addr":"172.21.15.195:6802/2558565460","heartbeat_back_addr":"172.21.15.195:6804/2558565460","heartbeat_front_addr":"172.21.15.195:6803/2558565460","state":["exists","up"]},{"osd":4,"uuid":"e69053fd-f6d9-437e-a9d0-8896de6b927f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":858330493}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6801","nonce":858330493}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6803","nonce":858330493}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":858330493}]},"public_addr":"172.21.15.73:6800/858330493","cluster_addr":"172.21.15.73:6801/858330493","heartbeat_back_addr":"172.21.15.73:6803/858330493","heartbeat_front_addr":"172.21.15.73:6802/858330493","state":["exists","up"]},{"osd":5,"uuid":"3667ec10-561e-4a23-8197-6357db4af315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":984313481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6805","nonce":984313481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6807","nonce":984313481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":984313481}]},"public_addr":"172.21.15.136:6804/984313481","cluster_addr":"172.21.15.136:6805/984313481","heartbeat_back_addr":"172.21.15.136:6807/984313481","heartbeat_front_addr":"172.21.15.136:6806/984313481","state":["exists","up"]},{"osd":6,"uuid":"30d4d63c-f04f-4423-91a6-816a7cbc0287","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6812","nonce":1505939978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6813","nonce":1505939978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6815","nonce":1505939978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6814","nonce":1505939978}]},"public_addr":"172.21.15.181:6812/1505939978","cluster_addr":"172.21.15.181:6813/1505939978","heartbeat_back_addr":"172.21.15.181:6815/1505939978","heartbeat_front_addr":"172.21.15.181:6814/1505939978","state":["exists","up"]},{"osd":7,"uuid":"48812f05-82f1-4263-8ec5-4861a6e91b61","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":162,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6809","nonce":3679292757}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":3679292757}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3679292757}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6811","nonce":3679292757}]},"public_addr":"172.21.15.195:6809/3679292757","cluster_addr":"172.21.15.195:6810/3679292757","heartbeat_back_addr":"172.21.15.195:6812/3679292757","heartbeat_front_addr":"172.21.15.195:6811/3679292757","state":["exists","up"]},{"osd":8,"uuid":"72056e61-d04b-4fd3-94bf-57ee3df4b466","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":4052106885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6805","nonce":4052106885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":4052106885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":4052106885}]},"public_addr":"172.21.15.73:6804/4052106885","cluster_addr":"172.21.15.73:6805/4052106885","heartbeat_back_addr":"172.21.15.73:6807/4052106885","heartbeat_front_addr":"172.21.15.73:6806/4052106885","state":["exists","up"]},{"osd":9,"uuid":"20e7f118-7148-4c27-9ec4-a33954709515","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":126,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":4266571820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6801","nonce":4266571820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6803","nonce":4266571820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":4266571820}]},"public_addr":"172.21.15.136:6800/4266571820","cluster_addr":"172.21.15.136:6801/4266571820","heartbeat_back_addr":"172.21.15.136:6803/4266571820","heartbeat_front_addr":"172.21.15.136:6802/4266571820","state":["exists","up"]},{"osd":10,"uuid":"012409e8-51c2-4584-aba7-95996f0d72ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6808","nonce":958255259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6809","nonce":958255259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6811","nonce":958255259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6810","nonce":958255259}]},"public_addr":"172.21.15.181:6808/958255259","cluster_addr":"172.21.15.181:6809/958255259","heartbeat_back_addr":"172.21.15.181:6811/958255259","heartbeat_front_addr":"172.21.15.181:6810/958255259","state":["exists","up"]},{"osd":11,"uuid":"88b137b1-7d30-45f7-ac5c-ba026704ef24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":281884267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6814","nonce":281884267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":281884267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6815","nonce":281884267}]},"public_addr":"172.21.15.195:6813/281884267","cluster_addr":"172.21.15.195:6814/281884267","heartbeat_back_addr":"172.21.15.195:6816/281884267","heartbeat_front_addr":"172.21.15.195:6815/281884267","state":["exists","up"]},{"osd":12,"uuid":"b54732a2-9dff-42e1-b4ef-281cbca3ad7c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3532043806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6809","nonce":3532043806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6811","nonce":3532043806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3532043806}]},"public_addr":"172.21.15.73:6808/3532043806","cluster_addr":"172.21.15.73:6809/3532043806","heartbeat_back_addr":"172.21.15.73:6811/3532043806","heartbeat_front_addr":"172.21.15.73:6810/3532043806","state":["exists","up"]},{"osd":13,"uuid":"20b48864-44dc-4732-8dff-d0d21cc08032","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":4083982131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":4083982131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6815","nonce":4083982131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":4083982131}]},"public_addr":"172.21.15.136:6812/4083982131","cluster_addr":"172.21.15.136:6813/4083982131","heartbeat_back_addr":"172.21.15.136:6815/4083982131","heartbeat_front_addr":"172.21.15.136:6814/4083982131","state":["exists","up"]},{"osd":14,"uuid":"9fac3ef4-e9be-4e8b-9a21-0de35b543d65","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6804","nonce":1173379175}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6805","nonce":1173379175}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6807","nonce":1173379175}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.181:6806","nonce":1173379175}]},"public_addr":"172.21.15.181:6804/1173379175","cluster_addr":"172.21.15.181:6805/1173379175","heartbeat_back_addr":"172.21.15.181:6807/1173379175","heartbeat_front_addr":"172.21.15.181:6806/1173379175","state":["exists","up"]},{"osd":15,"uuid":"16b34f90-3148-4d41-a403-ba0f900991e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6805","nonce":848686366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":848686366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6808","nonce":848686366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":848686366}]},"public_addr":"172.21.15.195:6805/848686366","cluster_addr":"172.21.15.195:6806/848686366","heartbeat_back_addr":"172.21.15.195:6808/848686366","heartbeat_front_addr":"172.21.15.195:6807/848686366","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:08.968353+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:00.403159+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:51.688552+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:12.174284+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:01.221496+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:11.382273+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:59.207758+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:02.780540+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:13:01.246971+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:47.636366+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:52.175110+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:51.721988+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:51.157861+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:42.954410+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:58.881999+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T10:12:54.838146+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T10:13:32.252 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T10:13:32.252 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T10:13:32.412 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:13:32.412 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T10:13:32.412 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:13:32.428 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T10:13:32.726 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T10:13:32.745 INFO:tasks.ceph:Scrubbing osd.4 2024-09-13T10:13:32.745 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-13T10:13:32.892 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:13:32.892 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T10:13:32.892 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:13:32.909 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-13T10:13:33.195 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 4 to deep-scrub 2024-09-13T10:13:33.213 INFO:tasks.ceph:Scrubbing osd.8 2024-09-13T10:13:33.214 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2024-09-13T10:13:33.352 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:13:33.352 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T10:13:33.352 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:13:33.367 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2024-09-13T10:13:33.637 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 8 to deep-scrub 2024-09-13T10:13:33.655 INFO:tasks.ceph:Scrubbing osd.12 2024-09-13T10:13:33.655 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 config set osd_debug_deep_scrub_sleep 0 2024-09-13T10:13:33.806 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:13:33.806 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T10:13:33.806 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:13:33.822 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 12 2024-09-13T10:13:34.105 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 12 to deep-scrub 2024-09-13T10:13:34.123 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T10:13:34.123 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T10:13:34.264 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:13:34.264 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T10:13:34.264 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:13:34.279 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T10:13:34.562 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T10:13:34.579 INFO:tasks.ceph:Scrubbing osd.5 2024-09-13T10:13:34.579 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-13T10:13:34.714 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:13:34.714 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T10:13:34.714 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:13:34.729 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-13T10:13:35.017 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 5 to deep-scrub 2024-09-13T10:13:35.036 INFO:tasks.ceph:Scrubbing osd.9 2024-09-13T10:13:35.036 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2024-09-13T10:13:35.184 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:13:35.184 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T10:13:35.184 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:13:35.199 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2024-09-13T10:13:35.482 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 9 to deep-scrub 2024-09-13T10:13:35.500 INFO:tasks.ceph:Scrubbing osd.13 2024-09-13T10:13:35.500 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 config set osd_debug_deep_scrub_sleep 0 2024-09-13T10:13:35.652 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:13:35.652 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T10:13:35.652 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:13:35.669 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 13 2024-09-13T10:13:35.973 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 13 to deep-scrub 2024-09-13T10:13:35.993 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T10:13:35.993 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T10:13:36.148 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:13:36.148 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T10:13:36.148 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:13:36.164 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T10:13:36.482 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T10:13:36.500 INFO:tasks.ceph:Scrubbing osd.6 2024-09-13T10:13:36.500 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-13T10:13:36.651 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:13:36.652 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T10:13:36.652 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:13:36.668 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-13T10:13:36.957 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 6 to deep-scrub 2024-09-13T10:13:36.975 INFO:tasks.ceph:Scrubbing osd.10 2024-09-13T10:13:36.975 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 config set osd_debug_deep_scrub_sleep 0 2024-09-13T10:13:37.123 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:13:37.123 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T10:13:37.124 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:13:37.138 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 10 2024-09-13T10:13:37.419 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 10 to deep-scrub 2024-09-13T10:13:37.437 INFO:tasks.ceph:Scrubbing osd.14 2024-09-13T10:13:37.437 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 config set osd_debug_deep_scrub_sleep 0 2024-09-13T10:13:37.579 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:13:37.579 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T10:13:37.579 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:13:37.595 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 14 2024-09-13T10:13:37.876 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 14 to deep-scrub 2024-09-13T10:13:37.894 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T10:13:37.894 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T10:13:38.051 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:13:38.051 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T10:13:38.051 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:13:38.067 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T10:13:38.351 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T10:13:38.369 INFO:tasks.ceph:Scrubbing osd.7 2024-09-13T10:13:38.369 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-13T10:13:38.518 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:13:38.518 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T10:13:38.518 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:13:38.533 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-13T10:13:38.829 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 7 to deep-scrub 2024-09-13T10:13:38.848 INFO:tasks.ceph:Scrubbing osd.11 2024-09-13T10:13:38.848 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 config set osd_debug_deep_scrub_sleep 0 2024-09-13T10:13:38.990 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:13:38.990 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T10:13:38.991 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:13:39.006 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 11 2024-09-13T10:13:39.297 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 11 to deep-scrub 2024-09-13T10:13:39.317 INFO:tasks.ceph:Scrubbing osd.15 2024-09-13T10:13:39.317 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 config set osd_debug_deep_scrub_sleep 0 2024-09-13T10:13:39.459 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-13T10:13:39.459 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T10:13:39.459 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-13T10:13:39.475 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 15 2024-09-13T10:13:39.765 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 15 to deep-scrub 2024-09-13T10:13:39.784 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:13:40.083 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:13:40.083 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:13:40.099 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2024-09-13T10:13:38.989243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110070969,"num_objects":152,"num_object_clones":0,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":1451,"num_read_kb":31320,"num_write":2016,"num_write_kb":368306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":115889007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":39,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":802328,"kb_used_data":315832,"kb_used_omap":24,"kb_used_meta":486183,"kb_avail":1509147112,"statfs":{"total":1546188226560,"available":1545366642688,"internally_reserved":0,"allocated":323411968,"data_stored":316031115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":497851471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004584"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":159,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:33.730048+0000","last_change":"2024-09-13T10:13:33.730048+0000","last_active":"2024-09-13T10:13:33.730048+0000","last_peered":"2024-09-13T10:13:33.730048+0000","last_clean":"2024-09-13T10:13:33.730048+0000","last_became_active":"2024-09-13T10:12:52.200952+0000","last_became_peered":"2024-09-13T10:12:52.200952+0000","last_unstale":"2024-09-13T10:13:33.730048+0000","last_undegraded":"2024-09-13T10:13:33.730048+0000","last_fullsized":"2024-09-13T10:13:33.730048+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:33.729945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:13:33.729945+0000","last_clean_scrub_stamp":"2024-09-13T10:13:33.729945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.494243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":321,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:24.991050+0000","last_change":"2024-09-13T10:13:24.991050+0000","last_active":"2024-09-13T10:13:24.991050+0000","last_peered":"2024-09-13T10:13:24.991050+0000","last_clean":"2024-09-13T10:13:24.991050+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:13:24.991050+0000","last_undegraded":"2024-09-13T10:13:24.991050+0000","last_fullsized":"2024-09-13T10:13:24.991050+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:24.990889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:24.990889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:43.157415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":334,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:31.388728+0000","last_change":"2024-09-13T10:13:31.388728+0000","last_active":"2024-09-13T10:13:31.388728+0000","last_peered":"2024-09-13T10:13:31.388728+0000","last_clean":"2024-09-13T10:13:31.388728+0000","last_became_active":"2024-09-13T10:12:45.768237+0000","last_became_peered":"2024-09-13T10:12:45.768237+0000","last_unstale":"2024-09-13T10:13:31.388728+0000","last_undegraded":"2024-09-13T10:13:31.388728+0000","last_fullsized":"2024-09-13T10:13:31.388728+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:31.388530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:31.388530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:32.591473+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":36,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505514+0000","last_change":"2024-09-13T10:13:09.126344+0000","last_active":"2024-09-13T10:13:23.505514+0000","last_peered":"2024-09-13T10:13:23.505514+0000","last_clean":"2024-09-13T10:13:23.505514+0000","last_became_active":"2024-09-13T10:13:09.125529+0000","last_became_peered":"2024-09-13T10:13:09.125529+0000","last_unstale":"2024-09-13T10:13:23.505514+0000","last_undegraded":"2024-09-13T10:13:23.505514+0000","last_fullsized":"2024-09-13T10:13:23.505514+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:12:50.136244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:50.136244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:14.321249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"119'332","reported_seq":949,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.547968+0000","last_change":"2024-09-13T10:13:21.995285+0000","last_active":"2024-09-13T10:13:23.547968+0000","last_peered":"2024-09-13T10:13:23.547968+0000","last_clean":"2024-09-13T10:13:23.547968+0000","last_became_active":"2024-09-13T10:13:21.994528+0000","last_became_peered":"2024-09-13T10:13:21.994528+0000","last_unstale":"2024-09-13T10:13:23.547968+0000","last_undegraded":"2024-09-13T10:13:23.547968+0000","last_fullsized":"2024-09-13T10:13:23.547968+0000","mapping_epoch":168,"log_start":"118'329","ondisk_log_start":"118'329","created":30,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'332","last_scrub_stamp":"2024-09-13T10:12:28.697464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:12:28.697464+0000","objects_scrubbed":39,"log_size":3,"log_dups_size":344,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:40.159242+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28010945,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":327,"num_read_kb":7023,"num_write":433,"num_write_kb":81577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":82643209,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,4,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"26'112","reported_seq":282,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505794+0000","last_change":"2024-09-13T10:12:58.899445+0000","last_active":"2024-09-13T10:13:23.505794+0000","last_peered":"2024-09-13T10:13:23.505794+0000","last_clean":"2024-09-13T10:13:23.505794+0000","last_became_active":"2024-09-13T10:12:52.099716+0000","last_became_peered":"2024-09-13T10:12:52.099716+0000","last_unstale":"2024-09-13T10:13:23.505794+0000","last_undegraded":"2024-09-13T10:13:23.505794+0000","last_fullsized":"2024-09-13T10:13:23.505794+0000","mapping_epoch":140,"log_start":"26'109","ondisk_log_start":"26'109","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"26'112","last_scrub_stamp":"2024-09-13T10:12:58.899291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.899291+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":109,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:21.742546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.3","version":"25'1","reported_seq":227,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.548028+0000","last_change":"2024-09-13T10:12:53.845086+0000","last_active":"2024-09-13T10:13:23.548028+0000","last_peered":"2024-09-13T10:13:23.548028+0000","last_clean":"2024-09-13T10:13:23.548028+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:13:23.548028+0000","last_undegraded":"2024-09-13T10:13:23.548028+0000","last_fullsized":"2024-09-13T10:13:23.548028+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1","last_scrub_stamp":"2024-09-13T10:12:53.844861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:53.844861+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.984156+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"119'365","reported_seq":711,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:22.743459+0000","last_change":"2024-09-13T10:13:22.036708+0000","last_active":"2024-09-13T10:13:22.743459+0000","last_peered":"2024-09-13T10:13:22.743459+0000","last_clean":"2024-09-13T10:13:22.743459+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:13:22.743459+0000","last_undegraded":"2024-09-13T10:13:22.743459+0000","last_fullsized":"2024-09-13T10:13:22.743459+0000","mapping_epoch":30,"log_start":"118'362","ondisk_log_start":"118'362","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'365","last_scrub_stamp":"2024-09-13T10:13:22.036498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:22.036498+0000","objects_scrubbed":42,"log_size":3,"log_dups_size":374,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:24.928602+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33541126,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":353,"num_read_kb":13014,"num_write":482,"num_write_kb":101793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"27'2","reported_seq":107,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505195+0000","last_change":"2024-09-13T10:12:58.690112+0000","last_active":"2024-09-13T10:13:23.505195+0000","last_peered":"2024-09-13T10:13:23.505195+0000","last_clean":"2024-09-13T10:13:23.505195+0000","last_became_active":"2024-09-13T10:12:45.765925+0000","last_became_peered":"2024-09-13T10:12:45.765925+0000","last_unstale":"2024-09-13T10:13:23.505195+0000","last_undegraded":"2024-09-13T10:13:23.505195+0000","last_fullsized":"2024-09-13T10:13:23.505195+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-13T10:12:58.689917+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:12:58.689917+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:20.031308+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"170'386","reported_seq":881,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:37.674958+0000","last_change":"2024-09-13T10:13:37.674958+0000","last_active":"2024-09-13T10:13:37.674958+0000","last_peered":"2024-09-13T10:13:37.674958+0000","last_clean":"2024-09-13T10:13:37.674958+0000","last_became_active":"2024-09-13T10:13:17.956444+0000","last_became_peered":"2024-09-13T10:13:17.956444+0000","last_unstale":"2024-09-13T10:13:37.674958+0000","last_undegraded":"2024-09-13T10:13:37.674958+0000","last_fullsized":"2024-09-13T10:13:37.674958+0000","mapping_epoch":166,"log_start":"170'370","ondisk_log_start":"170'370","created":30,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'386","last_scrub_stamp":"2024-09-13T10:13:37.674759+0000","last_deep_scrub":"170'386","last_deep_scrub_stamp":"2024-09-13T10:13:37.674759+0000","last_clean_scrub_stamp":"2024-09-13T10:13:37.674759+0000","objects_scrubbed":40,"log_size":16,"log_dups_size":382,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:05.691889+0000","scrub_duration":111,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27658528,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":310,"num_read_kb":4757,"num_write":455,"num_write_kb":83526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":33245760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":150,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505388+0000","last_change":"2024-09-13T10:13:22.027674+0000","last_active":"2024-09-13T10:13:23.505388+0000","last_peered":"2024-09-13T10:13:23.505388+0000","last_clean":"2024-09-13T10:13:23.505388+0000","last_became_active":"2024-09-13T10:13:14.495396+0000","last_became_peered":"2024-09-13T10:13:14.495396+0000","last_unstale":"2024-09-13T10:13:23.505388+0000","last_undegraded":"2024-09-13T10:13:23.505388+0000","last_fullsized":"2024-09-13T10:13:23.505388+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:22.027512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:29.101645+0000","last_clean_scrub_stamp":"2024-09-13T10:13:22.027512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:47.896470+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"119'401","reported_seq":728,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:23.505478+0000","last_change":"2024-09-13T10:13:19.721098+0000","last_active":"2024-09-13T10:13:23.505478+0000","last_peered":"2024-09-13T10:13:23.505478+0000","last_clean":"2024-09-13T10:13:23.505478+0000","last_became_active":"2024-09-13T10:12:43.752738+0000","last_became_peered":"2024-09-13T10:12:43.752738+0000","last_unstale":"2024-09-13T10:13:23.505478+0000","last_undegraded":"2024-09-13T10:13:23.505478+0000","last_fullsized":"2024-09-13T10:13:23.505478+0000","mapping_epoch":132,"log_start":"118'398","ondisk_log_start":"118'398","created":30,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'401","last_scrub_stamp":"2024-09-13T10:13:19.720891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:10:42.187897+0000","last_clean_scrub_stamp":"2024-09-13T10:13:19.720891+0000","objects_scrubbed":27,"log_size":3,"log_dups_size":407,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:49.615137+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20269983,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":375,"num_read_kb":6453,"num_write":499,"num_write_kb":98736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":94,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:37.458902+0000","last_change":"2024-09-13T10:13:37.458902+0000","last_active":"2024-09-13T10:13:37.458902+0000","last_peered":"2024-09-13T10:13:37.458902+0000","last_clean":"2024-09-13T10:13:37.458902+0000","last_became_active":"2024-09-13T10:12:45.763788+0000","last_became_peered":"2024-09-13T10:12:45.763788+0000","last_unstale":"2024-09-13T10:13:37.458902+0000","last_undegraded":"2024-09-13T10:13:37.458902+0000","last_fullsized":"2024-09-13T10:13:37.458902+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:37.458725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:13:37.458725+0000","last_clean_scrub_stamp":"2024-09-13T10:13:37.458725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:48.883395+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":109480582,"num_objects":148,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":1365,"num_read_kb":31247,"num_write":1869,"num_write_kb":365632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":115888969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":302743552,"data_stored":302001554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":24,"acting":24,"num_store_stats":14}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247849,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79596,"kb_used_data":48412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94292244,"statfs":{"total":96636764160,"available":96555257856,"internally_reserved":0,"allocated":49573888,"data_stored":49047078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60008,"kb_used_data":28824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311832,"statfs":{"total":96636764160,"available":96575315968,"internally_reserved":0,"allocated":29515776,"data_stored":29015046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313259,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28180,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343660,"statfs":{"total":96636764160,"available":96607907840,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313258,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59748,"kb_used_data":28564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312092,"statfs":{"total":96636764160,"available":96575582208,"internally_reserved":0,"allocated":29249536,"data_stored":28777095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247849,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47916,"kb_used_data":16720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323924,"statfs":{"total":96636764160,"available":96587698176,"internally_reserved":0,"allocated":17121280,"data_stored":16667237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280554,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48640,"kb_used_data":17444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323200,"statfs":{"total":96636764160,"available":96586956800,"internally_reserved":0,"allocated":17862656,"data_stored":17389030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,6,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280554,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85588,"kb_used_data":54404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94286252,"statfs":{"total":96636764160,"available":96549122048,"internally_reserved":0,"allocated":55709696,"data_stored":55167627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,8,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280555,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92572,"kb_used_data":61388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94279268,"statfs":{"total":96636764160,"available":96541970432,"internally_reserved":0,"allocated":62861312,"data_stored":62318221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280554,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65740,"kb_used_data":34556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306100,"statfs":{"total":96636764160,"available":96569446400,"internally_reserved":0,"allocated":35385344,"data_stored":34897644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280554,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48636,"kb_used_data":17444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323204,"statfs":{"total":96636764160,"available":96586960896,"internally_reserved":0,"allocated":17862656,"data_stored":17389030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280554,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280554,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247849,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247849,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15941632,"data_stored":15901087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53936128,"data_stored":53811109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61681664,"data_stored":61552071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33611776,"data_stored":33541126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":27742208,"data_stored":27658528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":28069888,"data_stored":28010945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":48394240,"data_stored":48280928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:13:40.104 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-13T10:13:24.990889+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=13, tm_sec=24, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=13, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T10:13:40.104 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-13T10:13:31.388530+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=13, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=13, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T10:13:40.104 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-13T10:12:50.136244+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=12, tm_sec=50, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=13, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T10:13:40.104 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-13T10:12:28.697464+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=12, tm_sec=28, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=13, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T10:13:40.105 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T10:12:58.899291+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=12, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=13, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T10:13:40.105 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-13T10:12:53.844861+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=12, tm_sec=53, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=13, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T10:13:40.105 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T10:13:22.036498+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=13, tm_sec=22, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=13, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T10:13:40.105 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-13T10:12:58.689917+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=12, tm_sec=58, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=13, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T10:13:40.105 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-13T10:13:22.027512+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=13, tm_sec=22, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=13, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T10:13:40.105 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T10:13:19.720891+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=13, tm_sec=19, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=10, tm_min=13, tm_sec=31, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T10:13:40.105 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T10:14:00.106 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T10:14:00.422 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-13T10:14:00.423 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-13T10:14:00.439 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-09-13T10:13:58.996568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":110070969,"num_objects":152,"num_object_clones":0,"num_object_copies":456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":1451,"num_read_kb":31320,"num_write":2016,"num_write_kb":368306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":115889007,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60,"ondisk_log_size":60,"up":39,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":14,"kb":1509949440,"kb_used":802296,"kb_used_data":315832,"kb_used_omap":24,"kb_used_meta":486183,"kb_avail":1509147144,"statfs":{"total":1546188226560,"available":1545366675456,"internally_reserved":0,"allocated":323411968,"data_stored":316031115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25521,"internal_metadata":497851471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004176"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":159,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:33.730048+0000","last_change":"2024-09-13T10:13:33.730048+0000","last_active":"2024-09-13T10:13:33.730048+0000","last_peered":"2024-09-13T10:13:33.730048+0000","last_clean":"2024-09-13T10:13:33.730048+0000","last_became_active":"2024-09-13T10:12:52.200952+0000","last_became_peered":"2024-09-13T10:12:52.200952+0000","last_unstale":"2024-09-13T10:13:33.730048+0000","last_undegraded":"2024-09-13T10:13:33.730048+0000","last_fullsized":"2024-09-13T10:13:33.730048+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:33.729945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:13:33.729945+0000","last_clean_scrub_stamp":"2024-09-13T10:13:33.729945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:36.494243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,12],"acting":[8,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":329,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:38.890397+0000","last_change":"2024-09-13T10:13:38.890397+0000","last_active":"2024-09-13T10:13:38.890397+0000","last_peered":"2024-09-13T10:13:38.890397+0000","last_clean":"2024-09-13T10:13:38.890397+0000","last_became_active":"2024-09-13T10:10:30.129731+0000","last_became_peered":"2024-09-13T10:10:30.129731+0000","last_unstale":"2024-09-13T10:13:38.890397+0000","last_undegraded":"2024-09-13T10:13:38.890397+0000","last_fullsized":"2024-09-13T10:13:38.890397+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:38.890208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:13:38.890208+0000","last_clean_scrub_stamp":"2024-09-13T10:13:38.890208+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:52.467651+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.001109074,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,15,7],"acting":[3,15,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":342,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:40.515055+0000","last_change":"2024-09-13T10:13:40.515055+0000","last_active":"2024-09-13T10:13:40.515055+0000","last_peered":"2024-09-13T10:13:40.515055+0000","last_clean":"2024-09-13T10:13:40.515055+0000","last_became_active":"2024-09-13T10:12:45.768237+0000","last_became_peered":"2024-09-13T10:12:45.768237+0000","last_unstale":"2024-09-13T10:13:40.515055+0000","last_undegraded":"2024-09-13T10:13:40.515055+0000","last_fullsized":"2024-09-13T10:13:40.515055+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:40.514888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:13:40.514888+0000","last_clean_scrub_stamp":"2024-09-13T10:13:40.514888+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:42.396438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001153006,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,9,6],"acting":[15,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":44,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:38.414201+0000","last_change":"2024-09-13T10:13:38.414201+0000","last_active":"2024-09-13T10:13:38.414201+0000","last_peered":"2024-09-13T10:13:38.414201+0000","last_clean":"2024-09-13T10:13:38.414201+0000","last_became_active":"2024-09-13T10:13:09.125529+0000","last_became_peered":"2024-09-13T10:13:09.125529+0000","last_unstale":"2024-09-13T10:13:38.414201+0000","last_undegraded":"2024-09-13T10:13:38.414201+0000","last_fullsized":"2024-09-13T10:13:38.414201+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:38.413928+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:13:38.413928+0000","last_clean_scrub_stamp":"2024-09-13T10:13:38.413928+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:47.692968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,12],"acting":[14,3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.1","version":"170'371","reported_seq":997,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:36.450133+0000","last_change":"2024-09-13T10:13:36.450133+0000","last_active":"2024-09-13T10:13:36.450133+0000","last_peered":"2024-09-13T10:13:36.450133+0000","last_clean":"2024-09-13T10:13:36.450133+0000","last_became_active":"2024-09-13T10:13:21.994528+0000","last_became_peered":"2024-09-13T10:13:21.994528+0000","last_unstale":"2024-09-13T10:13:36.450133+0000","last_undegraded":"2024-09-13T10:13:36.450133+0000","last_fullsized":"2024-09-13T10:13:36.450133+0000","mapping_epoch":168,"log_start":"170'356","ondisk_log_start":"170'356","created":30,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'371","last_scrub_stamp":"2024-09-13T10:13:36.449949+0000","last_deep_scrub":"170'371","last_deep_scrub_stamp":"2024-09-13T10:13:36.449949+0000","last_clean_scrub_stamp":"2024-09-13T10:13:36.449949+0000","objects_scrubbed":39,"log_size":15,"log_dups_size":371,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:58.735234+0000","scrub_duration":131,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28010945,"num_objects":39,"num_object_clones":0,"num_object_copies":117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":0,"num_read":327,"num_read_kb":7023,"num_write":433,"num_write_kb":81577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":106,"num_bytes_recovered":82643209,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,4,14],"acting":[13,4,14],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"170'114","reported_seq":292,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:34.745243+0000","last_change":"2024-09-13T10:13:34.745243+0000","last_active":"2024-09-13T10:13:34.745243+0000","last_peered":"2024-09-13T10:13:34.745243+0000","last_clean":"2024-09-13T10:13:34.745243+0000","last_became_active":"2024-09-13T10:12:52.099716+0000","last_became_peered":"2024-09-13T10:12:52.099716+0000","last_unstale":"2024-09-13T10:13:34.745243+0000","last_undegraded":"2024-09-13T10:13:34.745243+0000","last_fullsized":"2024-09-13T10:13:34.745243+0000","mapping_epoch":140,"log_start":"26'111","ondisk_log_start":"26'111","created":24,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'114","last_scrub_stamp":"2024-09-13T10:13:34.745172+0000","last_deep_scrub":"170'114","last_deep_scrub_stamp":"2024-09-13T10:13:34.745172+0000","last_clean_scrub_stamp":"2024-09-13T10:13:34.745172+0000","objects_scrubbed":2,"log_size":3,"log_dups_size":111,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:59.775565+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,7],"acting":[8,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.3","version":"170'2","reported_seq":237,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:37.353813+0000","last_change":"2024-09-13T10:13:37.353677+0000","last_active":"2024-09-13T10:13:37.353813+0000","last_peered":"2024-09-13T10:13:37.353813+0000","last_clean":"2024-09-13T10:13:37.353813+0000","last_became_active":"2024-09-13T10:11:33.001760+0000","last_became_peered":"2024-09-13T10:11:33.001760+0000","last_unstale":"2024-09-13T10:13:37.353813+0000","last_undegraded":"2024-09-13T10:13:37.353813+0000","last_fullsized":"2024-09-13T10:13:37.353813+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'2","last_scrub_stamp":"2024-09-13T10:13:37.353504+0000","last_deep_scrub":"170'2","last_deep_scrub_stamp":"2024-09-13T10:13:37.353504+0000","last_clean_scrub_stamp":"2024-09-13T10:13:37.353504+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:57.093138+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,8,12],"acting":[13,8,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"170'407","reported_seq":762,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:40.020973+0000","last_change":"2024-09-13T10:13:40.020973+0000","last_active":"2024-09-13T10:13:40.020973+0000","last_peered":"2024-09-13T10:13:40.020973+0000","last_clean":"2024-09-13T10:13:40.020973+0000","last_became_active":"2024-09-13T10:10:43.219350+0000","last_became_peered":"2024-09-13T10:10:43.219350+0000","last_unstale":"2024-09-13T10:13:40.020973+0000","last_undegraded":"2024-09-13T10:13:40.020973+0000","last_fullsized":"2024-09-13T10:13:40.020973+0000","mapping_epoch":30,"log_start":"170'389","ondisk_log_start":"170'389","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'407","last_scrub_stamp":"2024-09-13T10:13:40.020838+0000","last_deep_scrub":"170'407","last_deep_scrub_stamp":"2024-09-13T10:13:40.020838+0000","last_clean_scrub_stamp":"2024-09-13T10:13:40.020838+0000","objects_scrubbed":42,"log_size":18,"log_dups_size":401,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:46.531713+0000","scrub_duration":111,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33541126,"num_objects":42,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":353,"num_read_kb":13014,"num_write":482,"num_write_kb":101793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,4],"acting":[3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"170'3","reported_seq":116,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:36.771844+0000","last_change":"2024-09-13T10:13:36.771844+0000","last_active":"2024-09-13T10:13:36.771844+0000","last_peered":"2024-09-13T10:13:36.771844+0000","last_clean":"2024-09-13T10:13:36.771844+0000","last_became_active":"2024-09-13T10:12:45.765925+0000","last_became_peered":"2024-09-13T10:12:45.765925+0000","last_unstale":"2024-09-13T10:13:36.771844+0000","last_undegraded":"2024-09-13T10:13:36.771844+0000","last_fullsized":"2024-09-13T10:13:36.771844+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'3","last_scrub_stamp":"2024-09-13T10:13:36.771630+0000","last_deep_scrub":"170'3","last_deep_scrub_stamp":"2024-09-13T10:13:36.771630+0000","last_clean_scrub_stamp":"2024-09-13T10:13:36.771630+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:03.959674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"170'386","reported_seq":881,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:37.674958+0000","last_change":"2024-09-13T10:13:37.674958+0000","last_active":"2024-09-13T10:13:37.674958+0000","last_peered":"2024-09-13T10:13:37.674958+0000","last_clean":"2024-09-13T10:13:37.674958+0000","last_became_active":"2024-09-13T10:13:17.956444+0000","last_became_peered":"2024-09-13T10:13:17.956444+0000","last_unstale":"2024-09-13T10:13:37.674958+0000","last_undegraded":"2024-09-13T10:13:37.674958+0000","last_fullsized":"2024-09-13T10:13:37.674958+0000","mapping_epoch":166,"log_start":"170'370","ondisk_log_start":"170'370","created":30,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'386","last_scrub_stamp":"2024-09-13T10:13:37.674759+0000","last_deep_scrub":"170'386","last_deep_scrub_stamp":"2024-09-13T10:13:37.674759+0000","last_clean_scrub_stamp":"2024-09-13T10:13:37.674759+0000","objects_scrubbed":40,"log_size":16,"log_dups_size":382,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:05.691889+0000","scrub_duration":111,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27658528,"num_objects":40,"num_object_clones":0,"num_object_copies":120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":0,"num_read":310,"num_read_kb":4757,"num_write":455,"num_write_kb":83526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":80,"num_bytes_recovered":33245760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,1],"acting":[10,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":158,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:39.083730+0000","last_change":"2024-09-13T10:13:39.083730+0000","last_active":"2024-09-13T10:13:39.083730+0000","last_peered":"2024-09-13T10:13:39.083730+0000","last_clean":"2024-09-13T10:13:39.083730+0000","last_became_active":"2024-09-13T10:13:14.495396+0000","last_became_peered":"2024-09-13T10:13:14.495396+0000","last_unstale":"2024-09-13T10:13:39.083730+0000","last_undegraded":"2024-09-13T10:13:39.083730+0000","last_fullsized":"2024-09-13T10:13:39.083730+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:39.083569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:13:39.083569+0000","last_clean_scrub_stamp":"2024-09-13T10:13:39.083569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:48.782018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,15,9],"acting":[7,15,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"170'428","reported_seq":764,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:39.476188+0000","last_change":"2024-09-13T10:13:39.476188+0000","last_active":"2024-09-13T10:13:39.476188+0000","last_peered":"2024-09-13T10:13:39.476188+0000","last_clean":"2024-09-13T10:13:39.476188+0000","last_became_active":"2024-09-13T10:12:43.752738+0000","last_became_peered":"2024-09-13T10:12:43.752738+0000","last_unstale":"2024-09-13T10:13:39.476188+0000","last_undegraded":"2024-09-13T10:13:39.476188+0000","last_fullsized":"2024-09-13T10:13:39.476188+0000","mapping_epoch":132,"log_start":"170'425","ondisk_log_start":"170'425","created":30,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"170'428","last_scrub_stamp":"2024-09-13T10:13:39.475964+0000","last_deep_scrub":"170'428","last_deep_scrub_stamp":"2024-09-13T10:13:39.475964+0000","last_clean_scrub_stamp":"2024-09-13T10:13:39.475964+0000","objects_scrubbed":27,"log_size":3,"log_dups_size":434,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:15:06.016797+0000","scrub_duration":83,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20269983,"num_objects":27,"num_object_clones":0,"num_object_copies":81,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":375,"num_read_kb":6453,"num_write":499,"num_write_kb":98736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,3,0],"acting":[14,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":94,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-13T10:13:37.458902+0000","last_change":"2024-09-13T10:13:37.458902+0000","last_active":"2024-09-13T10:13:37.458902+0000","last_peered":"2024-09-13T10:13:37.458902+0000","last_clean":"2024-09-13T10:13:37.458902+0000","last_became_active":"2024-09-13T10:12:45.763788+0000","last_became_peered":"2024-09-13T10:12:45.763788+0000","last_unstale":"2024-09-13T10:13:37.458902+0000","last_undegraded":"2024-09-13T10:13:37.458902+0000","last_fullsized":"2024-09-13T10:13:37.458902+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T10:13:37.458725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T10:13:37.458725+0000","last_clean_scrub_stamp":"2024-09-13T10:13:37.458725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:14:48.883395+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,15,0],"acting":[6,15,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":109480582,"num_objects":148,"num_object_clones":0,"num_object_copies":444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":1365,"num_read_kb":31247,"num_write":1869,"num_write_kb":365632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":115888969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":302743552,"data_stored":302001554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":12,"acting":12,"num_store_stats":9},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":145,"num_write_kb":2672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":3,"acting":3,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":76,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":24,"acting":24,"num_store_stats":14}],"osd_stats":[{"osd":14,"up_from":23,"seq":98784247853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79596,"kb_used_data":48412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94292244,"statfs":{"total":96636764160,"available":96555257856,"internally_reserved":0,"allocated":49573888,"data_stored":49047078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,7,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":23,"seq":98784247855,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":23,"seq":98784247855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60008,"kb_used_data":28824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94311832,"statfs":{"total":96636764160,"available":96575315968,"internally_reserved":0,"allocated":29515776,"data_stored":29015046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,6,7,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":21,"seq":90194313263,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28180,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343660,"statfs":{"total":96636764160,"available":96607907840,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[2,3,4,6,8,10,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":21,"seq":90194313262,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59748,"kb_used_data":28564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312092,"statfs":{"total":96636764160,"available":96575582208,"internally_reserved":0,"allocated":29249536,"data_stored":28777095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,6,7,8,9,10,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":23,"seq":98784247853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47904,"kb_used_data":16720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323936,"statfs":{"total":96636764160,"available":96587710464,"internally_reserved":0,"allocated":17121280,"data_stored":16667237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,5,6,7,9,10,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":22,"seq":94489280558,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48628,"kb_used_data":17444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323212,"statfs":{"total":96636764160,"available":96586969088,"internally_reserved":0,"allocated":17862656,"data_stored":17389030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,6,7,8,10,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":22,"seq":94489280558,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85588,"kb_used_data":54404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94286252,"statfs":{"total":96636764160,"available":96549122048,"internally_reserved":0,"allocated":55709696,"data_stored":55167627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,8,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":22,"seq":94489280559,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":92572,"kb_used_data":61388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94279268,"statfs":{"total":96636764160,"available":96541970432,"internally_reserved":0,"allocated":62861312,"data_stored":62318221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,3,5,6,8,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":22,"seq":94489280557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,9,10,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280558,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65740,"kb_used_data":34556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94306100,"statfs":{"total":96636764160,"available":96569446400,"internally_reserved":0,"allocated":35385344,"data_stored":34897644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,6,8,9,10,11,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280558,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48628,"kb_used_data":17444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323212,"statfs":{"total":96636764160,"available":96586969088,"internally_reserved":0,"allocated":17862656,"data_stored":17389030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,5,7,8,9,11,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":22,"seq":94489280558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,5,9,10,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":15,"up_from":22,"seq":94489280558,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1179648,"data_stored":766150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,6,7,8,9,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":23,"seq":98784247853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,9,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":23,"seq":98784247853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1183744,"data_stored":766169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,3,5,7,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":15941632,"data_stored":15901087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":53936128,"data_stored":53811109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61681664,"data_stored":61552071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":33611776,"data_stored":33541126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":27742208,"data_stored":27658528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":16683008,"data_stored":16622880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":28069888,"data_stored":28010945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":48394240,"data_stored":48280928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T10:14:00.441 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T10:14:00.774 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T10:14:00.775 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T10:14:00.775 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T10:14:00.775 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:00.861 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T10:14:00.861 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T10:14:00.861 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:00.941 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T10:14:00.942 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2024-09-13T10:14:00.942 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:01.018 INFO:tasks.ceph.osd.8:Stopped 2024-09-13T10:14:01.018 DEBUG:tasks.ceph.osd.12:waiting for process to exit 2024-09-13T10:14:01.018 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:01.074 INFO:tasks.ceph.osd.12:Stopped 2024-09-13T10:14:01.075 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T10:14:01.075 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:02.872 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-13T10:14:07.177 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T10:14:07.177 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T10:14:07.177 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:07.272 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T10:14:07.273 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2024-09-13T10:14:07.273 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:07.366 INFO:tasks.ceph.osd.9:Stopped 2024-09-13T10:14:07.366 DEBUG:tasks.ceph.osd.13:waiting for process to exit 2024-09-13T10:14:07.366 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:07.449 INFO:tasks.ceph.osd.13:Stopped 2024-09-13T10:14:07.449 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T10:14:07.449 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:07.527 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T10:14:07.527 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-13T10:14:07.527 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:07.610 INFO:tasks.ceph.osd.6:Stopped 2024-09-13T10:14:07.611 DEBUG:tasks.ceph.osd.10:waiting for process to exit 2024-09-13T10:14:07.611 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:07.697 INFO:tasks.ceph.osd.10:Stopped 2024-09-13T10:14:07.698 DEBUG:tasks.ceph.osd.14:waiting for process to exit 2024-09-13T10:14:07.698 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:07.776 INFO:tasks.ceph.osd.14:Stopped 2024-09-13T10:14:07.776 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T10:14:07.776 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:07.861 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T10:14:07.862 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T10:14:07.862 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:07.938 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T10:14:07.939 DEBUG:tasks.ceph.osd.11:waiting for process to exit 2024-09-13T10:14:07.939 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:08.035 INFO:tasks.ceph.osd.11:Stopped 2024-09-13T10:14:08.035 DEBUG:tasks.ceph.osd.15:waiting for process to exit 2024-09-13T10:14:08.035 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:08.131 INFO:tasks.ceph.osd.15:Stopped 2024-09-13T10:14:08.132 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T10:14:08.132 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T10:14:08.132 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:08.161 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T10:14:08.162 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T10:14:08.162 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:08.193 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T10:14:08.193 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T10:14:08.194 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T10:14:08.194 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:08.227 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T10:14:08.227 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T10:14:08.227 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:08.252 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T10:14:08.252 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T10:14:08.253 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T10:14:08.274 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-13T10:14:08.274 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.11 is failed for ~0s 2024-09-13T10:14:08.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.15 is failed for ~0s 2024-09-13T10:14:08.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-09-13T10:14:08.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-13T10:14:08.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-09-13T10:14:08.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-13T10:14:08.275 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T10:14:08.275 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T10:14:08.276 DEBUG:teuthology.orchestra.run.smithi073:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-09-13T10:14:08.295 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-13T10:14:08.295 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T10:14:08.634 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-13T10:14:08.634 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-13T10:14:08.832 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-8 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-13T10:14:08.832 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8 2024-09-13T10:14:08.989 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-12 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-13T10:14:08.989 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-12 2024-09-13T10:14:09.145 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-13T10:14:09.146 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T10:14:09.381 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-13T10:14:09.381 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-13T10:14:09.617 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-9 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-13T10:14:09.618 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-9 2024-09-13T10:14:09.809 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-13 on ubuntu@smithi136.front.sepia.ceph.com 2024-09-13T10:14:09.810 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-13 2024-09-13T10:14:09.984 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi181.front.sepia.ceph.com 2024-09-13T10:14:09.984 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T10:14:10.220 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi181.front.sepia.ceph.com 2024-09-13T10:14:10.221 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-13T10:14:10.479 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-10 on ubuntu@smithi181.front.sepia.ceph.com 2024-09-13T10:14:10.479 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-10 2024-09-13T10:14:10.654 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-14 on ubuntu@smithi181.front.sepia.ceph.com 2024-09-13T10:14:10.654 DEBUG:teuthology.orchestra.run.smithi181:> sync && sudo umount -f /var/lib/ceph/osd/ceph-14 2024-09-13T10:14:10.828 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-13T10:14:10.828 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T10:14:11.125 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-13T10:14:11.126 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-13T10:14:11.291 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-13T10:14:11.291 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11 2024-09-13T10:14:11.434 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-15 on ubuntu@smithi195.front.sepia.ceph.com 2024-09-13T10:14:11.434 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-15 2024-09-13T10:14:11.567 INFO:tasks.ceph:Archiving mon data... 2024-09-13T10:14:11.568 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334/data/mon.a.tgz 2024-09-13T10:14:11.568 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-09-13T10:14:11.574 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.XcRbSzZAJJ 2024-09-13T10:14:11.574 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.XcRbSzZAJJ 2024-09-13T10:14:11.732 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.XcRbSzZAJJ 2024-09-13T10:14:11.794 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.XcRbSzZAJJ is 512KB 2024-09-13T10:14:11.856 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.XcRbSzZAJJ 2024-09-13T10:14:11.862 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334/data/mon.b.tgz 2024-09-13T10:14:11.862 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-09-13T10:14:11.868 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.5mVcf5igLn 2024-09-13T10:14:11.869 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.5mVcf5igLn 2024-09-13T10:14:12.024 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.5mVcf5igLn 2024-09-13T10:14:12.088 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.5mVcf5igLn is 518KB 2024-09-13T10:14:12.152 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.5mVcf5igLn 2024-09-13T10:14:12.157 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334/data/mon.c.tgz 2024-09-13T10:14:12.158 DEBUG:teuthology.orchestra.run.smithi181:> mktemp 2024-09-13T10:14:12.163 INFO:teuthology.orchestra.run.smithi181.stdout:/tmp/tmp.pBq0wmuTAg 2024-09-13T10:14:12.164 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.pBq0wmuTAg 2024-09-13T10:14:12.323 DEBUG:teuthology.orchestra.run.smithi181:> sudo chmod 0666 /tmp/tmp.pBq0wmuTAg 2024-09-13T10:14:12.383 DEBUG:teuthology.orchestra.remote:smithi181:/tmp/tmp.pBq0wmuTAg is 518KB 2024-09-13T10:14:12.446 DEBUG:teuthology.orchestra.run.smithi181:> rm -fr /tmp/tmp.pBq0wmuTAg 2024-09-13T10:14:12.450 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T10:14:12.451 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T10:14:12.453 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T10:14:12.455 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T10:14:12.493 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T10:14:12.515 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-13T10:14:12.521 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-13T10:14:12.552 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-13T10:14:12.557 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-13T10:14:12.557 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T10:14:12.559 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/crash to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334/remote/smithi073/crash 2024-09-13T10:14:12.559 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T10:14:12.572 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/crash to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334/remote/smithi136/crash 2024-09-13T10:14:12.573 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T10:14:12.587 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/lib/ceph/crash to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334/remote/smithi181/crash 2024-09-13T10:14:12.587 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T10:14:12.608 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/crash to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334/remote/smithi195/crash 2024-09-13T10:14:12.609 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T10:14:12.623 INFO:tasks.ceph:Compressing logs... 2024-09-13T10:14:12.623 DEBUG:teuthology.orchestra.run.smithi073:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-13T10:14:12.626 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-13T10:14:12.632 DEBUG:teuthology.orchestra.run.smithi181:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-13T10:14:12.648 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40020.log 2024-09-13T10:14:12.649 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34864.log 2024-09-13T10:14:12.649 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40554.log 2024-09-13T10:14:12.649 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40020.log.gz 2024-09-13T10:14:12.650 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39645.log 2024-09-13T10:14:12.652 INFO:teuthology.orchestra.run.smithi073.stderr: 19.1% -- replaced with /var/log/ceph/ceph-client.admin.34864.log.gz 2024-09-13T10:14:12.652 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39067.log 2024-09-13T10:14:12.653 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40554.log.gz 2024-09-13T10:14:12.653 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40057.log 2024-09-13T10:14:12.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39645.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39645.log.gz 2024-09-13T10:14:12.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36103.log 2024-09-13T10:14:12.653 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39067.log.gz 2024-09-13T10:14:12.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40057.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40057.log.gz 2024-09-13T10:14:12.653 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34123.log 2024-09-13T10:14:12.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39405.log 2024-09-13T10:14:12.653 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36103.log.gz 2024-09-13T10:14:12.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32812.log 2024-09-13T10:14:12.653 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34123.log.gz 2024-09-13T10:14:12.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32139.log 2024-09-13T10:14:12.653 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39405.log.gz 2024-09-13T10:14:12.654 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34099.log 2024-09-13T10:14:12.654 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32812.log.gz 2024-09-13T10:14:12.660 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42032.log 2024-09-13T10:14:12.660 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32139.log.gz 2024-09-13T10:14:12.660 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34099.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-13T10:14:12.660 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34099.log.gz 2024-09-13T10:14:12.660 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35606.log 2024-09-13T10:14:12.661 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42032.log.gz 2024-09-13T10:14:12.661 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34026.log 2024-09-13T10:14:12.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.35606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41823.log 2024-09-13T10:14:12.661 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35606.log.gz 2024-09-13T10:14:12.661 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38385.log 2024-09-13T10:14:12.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34026.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34026.log.gz 2024-09-13T10:14:12.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40052.log 2024-09-13T10:14:12.661 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41823.log.gz 2024-09-13T10:14:12.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39145.log 2024-09-13T10:14:12.661 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38385.log.gz 2024-09-13T10:14:12.661 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39509.log 2024-09-13T10:14:12.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40052.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40052.log.gz 2024-09-13T10:14:12.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36679.log 2024-09-13T10:14:12.662 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39145.log.gz 2024-09-13T10:14:12.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40991.log 2024-09-13T10:14:12.662 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39509.log.gz 2024-09-13T10:14:12.662 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33511.log 2024-09-13T10:14:12.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36679.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36679.log.gz 2024-09-13T10:14:12.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40071.log 2024-09-13T10:14:12.662 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40991.log.gz 2024-09-13T10:14:12.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39609.log 2024-09-13T10:14:12.662 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33511.log.gz 2024-09-13T10:14:12.662 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39642.log 2024-09-13T10:14:12.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40071.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40071.log.gz 2024-09-13T10:14:12.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40423.log 2024-09-13T10:14:12.662 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39609.log.gz 2024-09-13T10:14:12.663 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39642.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.39642.log.gz -5 2024-09-13T10:14:12.663 INFO:teuthology.orchestra.run.smithi073.stderr: --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-13T10:14:12.663 INFO:teuthology.orchestra.run.smithi073.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-13T10:14:12.663 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41121.log 2024-09-13T10:14:12.663 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40423.log.gz 2024-09-13T10:14:12.663 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40483.log 2024-09-13T10:14:12.663 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34173.log 2024-09-13T10:14:12.663 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41121.log.gz 2024-09-13T10:14:12.663 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41407.log 2024-09-13T10:14:12.663 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40483.log.gz 2024-09-13T10:14:12.665 DEBUG:teuthology.orchestra.run.smithi195:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-13T10:14:12.668 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-13T10:14:12.668 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-13T10:14:12.668 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-13T10:14:12.668 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-13T10:14:12.668 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.9.log 2024-09-13T10:14:12.668 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.13.log 2024-09-13T10:14:12.668 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-13T10:14:12.668 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.13.log: /var/log/ceph/ceph-mon.b.log: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-13T10:14:12.668 INFO:teuthology.orchestra.run.smithi136.stderr: 86.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-13T10:14:12.680 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-13T10:14:12.680 INFO:teuthology.orchestra.run.smithi181.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-13T10:14:12.680 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.14.log 2024-09-13T10:14:12.681 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-13T10:14:12.681 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-osd.14.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-13T10:14:12.681 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-13T10:14:12.681 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.10.log 2024-09-13T10:14:12.683 INFO:teuthology.orchestra.run.smithi181.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-osd.10.log: 86.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-13T10:14:12.683 INFO:teuthology.orchestra.run.smithi181.stderr: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-13T10:14:12.712 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26552.log 2024-09-13T10:14:12.712 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26256.log 2024-09-13T10:14:12.712 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26552.log.gz 2024-09-13T10:14:12.712 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-13T10:14:12.712 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26479.log 2024-09-13T10:14:12.712 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26256.log.gz 2024-09-13T10:14:12.712 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26208.log 2024-09-13T10:14:12.712 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26504.log 2024-09-13T10:14:12.712 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26479.log.gz 2024-09-13T10:14:12.712 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26383.log 2024-09-13T10:14:12.712 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26208.log.gz 2024-09-13T10:14:12.713 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26504.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-13T10:14:12.713 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26504.log.gz 2024-09-13T10:14:12.713 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26383.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.11.log 2024-09-13T10:14:12.713 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26383.log.gz 2024-09-13T10:14:12.713 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26184.log 2024-09-13T10:14:12.713 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26431.log 2024-09-13T10:14:12.713 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26184.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.15.log 2024-09-13T10:14:12.713 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26184.log.gz 2024-09-13T10:14:12.713 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26431.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-13T10:14:12.713 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26431.log.gz 2024-09-13T10:14:12.713 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.15.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.30714.log 2024-09-13T10:14:12.713 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26232.log 2024-09-13T10:14:12.713 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.0.30714.log: gzip -5 --verbose 82.2% -- replaced with /var/log/ceph/ceph-client.0.30714.log.gz 2024-09-13T10:14:12.714 INFO:teuthology.orchestra.run.smithi195.stderr: -- /var/log/ceph/ceph-client.admin.26455.log 2024-09-13T10:14:12.714 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26232.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26232.log.gz 2024-09-13T10:14:12.714 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26304.log 2024-09-13T10:14:12.714 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30691.log 2024-09-13T10:14:12.714 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26455.log.gz 2024-09-13T10:14:12.714 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26528.log 2024-09-13T10:14:12.714 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26304.log.gz 2024-09-13T10:14:12.714 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26280.log 2024-09-13T10:14:12.714 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.30691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30691.log.gz 2024-09-13T10:14:12.714 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26328.log 2024-09-13T10:14:12.714 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26528.log.gz 2024-09-13T10:14:12.714 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26407.log 2024-09-13T10:14:12.714 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26280.log: /var/log/ceph/ceph-client.admin.26328.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26280.log.gz 2024-09-13T10:14:12.715 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26352.log 2024-09-13T10:14:12.715 INFO:teuthology.orchestra.run.smithi195.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26328.log.gz 2024-09-13T10:14:12.715 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26407.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26407.log.gz 2024-09-13T10:14:12.715 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.26352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26352.log.gz 2024-09-13T10:14:12.759 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41199.log 2024-09-13T10:14:12.759 INFO:teuthology.orchestra.run.smithi073.stderr: 19.7% -- replaced with /var/log/ceph/ceph-client.admin.34173.log.gz 2024-09-13T10:14:12.759 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37230.log 2024-09-13T10:14:12.760 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41407.log.gz 2024-09-13T10:14:12.760 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39171.log 2024-09-13T10:14:12.760 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41199.log.gz 2024-09-13T10:14:12.760 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40455.log 2024-09-13T10:14:12.760 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37230.log.gz 2024-09-13T10:14:12.760 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38277.log 2024-09-13T10:14:12.760 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39171.log.gz 2024-09-13T10:14:12.760 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40939.log 2024-09-13T10:14:12.760 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40455.log.gz 2024-09-13T10:14:12.760 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34270.log 2024-09-13T10:14:12.760 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38277.log.gz 2024-09-13T10:14:12.761 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41589.log 2024-09-13T10:14:12.761 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40939.log.gz 2024-09-13T10:14:12.761 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41797.log 2024-09-13T10:14:12.761 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34270.log.gz 2024-09-13T10:14:12.761 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38833.log 2024-09-13T10:14:12.761 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41589.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41589.log.gz 2024-09-13T10:14:12.761 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34625.log 2024-09-13T10:14:12.761 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41797.log.gz 2024-09-13T10:14:12.761 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36962.log 2024-09-13T10:14:12.761 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38833.log.gz 2024-09-13T10:14:12.761 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41043.log 2024-09-13T10:14:12.761 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34625.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.34625.log.gz 2024-09-13T10:14:12.761 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38490.log 2024-09-13T10:14:12.762 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36962.log.gz 2024-09-13T10:14:12.762 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38464.log 2024-09-13T10:14:12.762 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41043.log.gz 2024-09-13T10:14:12.762 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32912.log 2024-09-13T10:14:12.762 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38490.log.gz 2024-09-13T10:14:12.762 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38494.log 2024-09-13T10:14:12.762 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38464.log.gz 2024-09-13T10:14:12.762 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39249.log 2024-09-13T10:14:12.762 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32912.log.gz 2024-09-13T10:14:12.762 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41979.log 2024-09-13T10:14:12.762 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38494.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38494.log.gz 2024-09-13T10:14:12.762 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39249.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.39249.log.gz 2024-09-13T10:14:12.763 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36034.log 2024-09-13T10:14:12.763 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35253.log 2024-09-13T10:14:12.763 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41979.log.gz 2024-09-13T10:14:12.763 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39483.log 2024-09-13T10:14:12.763 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36034.log.gz 2024-09-13T10:14:12.763 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35253.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.35253.log.gz 2024-09-13T10:14:12.763 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35458.log 2024-09-13T10:14:12.763 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38885.log 2024-09-13T10:14:12.763 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39483.log.gz 2024-09-13T10:14:12.763 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40887.log 2024-09-13T10:14:12.763 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35458.log: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.35458.log.gz 2024-09-13T10:14:12.763 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38885.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.38885.log.gz 2024-09-13T10:14:12.763 INFO:teuthology.orchestra.run.smithi073.stderr: /var/log/ceph/ceph-client.admin.33703.log 2024-09-13T10:14:12.764 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41355.log 2024-09-13T10:14:12.764 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40887.log.gz 2024-09-13T10:14:12.764 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33703.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.8.log 2024-09-13T10:14:12.764 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33703.log.gz 2024-09-13T10:14:12.764 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36918.log 2024-09-13T10:14:12.764 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41355.log.gz 2024-09-13T10:14:12.764 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34392.log 2024-09-13T10:14:12.764 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33227.log 2024-09-13T10:14:12.764 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36918.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36918.log.gz 2024-09-13T10:14:12.764 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34392.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34392.log.gz 2024-09-13T10:14:12.764 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32172.log 2024-09-13T10:14:12.764 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37365.log 2024-09-13T10:14:12.764 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33227.log.gz 2024-09-13T10:14:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32172.log.gz 2024-09-13T10:14:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32553.log 2024-09-13T10:14:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.37365.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39015.log 2024-09-13T10:14:12.765 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37365.log.gz 2024-09-13T10:14:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32553.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32553.log.gz 2024-09-13T10:14:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36340.log 2024-09-13T10:14:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32144.log 2024-09-13T10:14:12.765 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39015.log.gz 2024-09-13T10:14:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36340.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36340.log.gz 2024-09-13T10:14:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37605.log 2024-09-13T10:14:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36989.log 2024-09-13T10:14:12.765 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32144.log.gz 2024-09-13T10:14:12.765 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37605.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37605.log.gz 2024-09-13T10:14:12.766 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-13T10:14:12.766 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41953.log 2024-09-13T10:14:12.766 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36989.log.gz 2024-09-13T10:14:12.766 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40056.log 2024-09-13T10:14:12.766 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41953.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41953.log.gz 2024-09-13T10:14:12.766 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38438.log 2024-09-13T10:14:12.766 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36199.log 2024-09-13T10:14:12.766 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40056.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40056.log.gz 2024-09-13T10:14:12.766 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32185.log 2024-09-13T10:14:12.766 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38438.log.gz 2024-09-13T10:14:12.766 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38568.log 2024-09-13T10:14:12.766 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36199.log.gz 2024-09-13T10:14:12.766 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36400.log 2024-09-13T10:14:12.767 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32185.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32185.log.gz 2024-09-13T10:14:12.767 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34001.log 2024-09-13T10:14:12.767 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38568.log.gz 2024-09-13T10:14:12.767 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40016.log 2024-09-13T10:14:12.767 INFO:teuthology.orchestra.run.smithi073.stderr: 19.2% -- replaced with /var/log/ceph/ceph-client.admin.36400.log.gz 2024-09-13T10:14:12.767 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39197.log 2024-09-13T10:14:12.767 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34001.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34001.log.gz 2024-09-13T10:14:12.767 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5/var/log/ceph/ceph-client.admin.40016.log: --verbose -- /var/log/ceph/ceph-client.admin.36746.log 2024-09-13T10:14:12.767 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40016.log.gz 2024-09-13T10:14:12.767 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34734.log 2024-09-13T10:14:12.767 INFO:teuthology.orchestra.run.smithi073.stderr: 26.4% -- replaced with /var/log/ceph/ceph-client.admin.39197.log.gz 2024-09-13T10:14:12.767 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40419.log 2024-09-13T10:14:12.767 INFO:teuthology.orchestra.run.smithi073.stderr: 18.7% -- replaced with /var/log/ceph/ceph-client.admin.36746.log.gz 2024-09-13T10:14:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40568.log 2024-09-13T10:14:12.768 INFO:teuthology.orchestra.run.smithi073.stderr: 18.6% -- replaced with /var/log/ceph/ceph-client.admin.34734.log.gz 2024-09-13T10:14:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39301.log 2024-09-13T10:14:12.768 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40419.log.gz 2024-09-13T10:14:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39641.log 2024-09-13T10:14:12.768 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40568.log.gz 2024-09-13T10:14:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41225.log 2024-09-13T10:14:12.768 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39301.log.gz 2024-09-13T10:14:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39641.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39641.log.gz 2024-09-13T10:14:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41563.log 2024-09-13T10:14:12.768 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37394.log 2024-09-13T10:14:12.768 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41225.log.gz 2024-09-13T10:14:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40042.log 2024-09-13T10:14:12.769 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41563.log.gz 2024-09-13T10:14:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37798.log 2024-09-13T10:14:12.769 INFO:teuthology.orchestra.run.smithi073.stderr: 17.6% -- replaced with /var/log/ceph/ceph-client.admin.37394.log.gz 2024-09-13T10:14:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39535.log 2024-09-13T10:14:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40042.log.gz 2024-09-13T10:14:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32179.log 2024-09-13T10:14:12.769 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37798.log.gz 2024-09-13T10:14:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33297.log 2024-09-13T10:14:12.769 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39535.log.gz 2024-09-13T10:14:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40553.log 2024-09-13T10:14:12.769 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32179.log.gz 2024-09-13T10:14:12.769 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40033.log 2024-09-13T10:14:12.770 INFO:teuthology.orchestra.run.smithi073.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.33297.log.gz 2024-09-13T10:14:12.770 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35698.log 2024-09-13T10:14:12.770 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40553.log.gz 2024-09-13T10:14:12.770 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32968.log 2024-09-13T10:14:12.770 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40033.log.gz 2024-09-13T10:14:12.770 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31757.log 2024-09-13T10:14:12.770 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35698.log.gz 2024-09-13T10:14:12.770 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31695.log 2024-09-13T10:14:12.770 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32968.log.gz 2024-09-13T10:14:12.770 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36504.log 2024-09-13T10:14:12.770 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31757.log.gz 2024-09-13T10:14:12.770 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37928.log 2024-09-13T10:14:12.771 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31695.log.gz 2024-09-13T10:14:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40026.log 2024-09-13T10:14:12.771 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36504.log.gz 2024-09-13T10:14:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34599.log 2024-09-13T10:14:12.771 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37928.log.gz 2024-09-13T10:14:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34982.log 2024-09-13T10:14:12.771 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40026.log.gz 2024-09-13T10:14:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.34599.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32613.log 2024-09-13T10:14:12.771 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34599.log.gz 2024-09-13T10:14:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39625.log 2024-09-13T10:14:12.771 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34982.log.gz 2024-09-13T10:14:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36584.log 2024-09-13T10:14:12.771 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32613.log.gz 2024-09-13T10:14:12.771 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31780.log 2024-09-13T10:14:12.772 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39625.log.gz 2024-09-13T10:14:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34977.log 2024-09-13T10:14:12.772 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36584.log.gz 2024-09-13T10:14:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39093.log 2024-09-13T10:14:12.772 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31780.log.gz 2024-09-13T10:14:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35116.log 2024-09-13T10:14:12.772 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34977.log.gz 2024-09-13T10:14:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33121.log 2024-09-13T10:14:12.772 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39093.log.gz 2024-09-13T10:14:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34978.log 2024-09-13T10:14:12.772 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35116.log.gz 2024-09-13T10:14:12.772 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39457.log 2024-09-13T10:14:12.772 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33121.log.gz 2024-09-13T10:14:12.773 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31540.log 2024-09-13T10:14:12.773 INFO:teuthology.orchestra.run.smithi073.stderr: 20.1% -- replaced with /var/log/ceph/ceph-client.admin.34978.log.gz 2024-09-13T10:14:12.773 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34364.log 2024-09-13T10:14:12.773 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39457.log.gz 2024-09-13T10:14:12.773 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40555.log 2024-09-13T10:14:12.773 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31540.log.gz 2024-09-13T10:14:12.773 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40965.log 2024-09-13T10:14:12.773 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34364.log.gz 2024-09-13T10:14:12.773 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41875.log 2024-09-13T10:14:12.773 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40555.log.gz 2024-09-13T10:14:12.773 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39613.log 2024-09-13T10:14:12.773 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40965.log.gz 2024-09-13T10:14:12.773 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40861.log 2024-09-13T10:14:12.774 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41875.log.gz 2024-09-13T10:14:12.774 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36396.log 2024-09-13T10:14:12.774 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39613.log.gz 2024-09-13T10:14:12.774 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34306.log 2024-09-13T10:14:12.774 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40861.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40861.log.gz 2024-09-13T10:14:12.774 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41095.log 2024-09-13T10:14:12.774 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36396.log.gz 2024-09-13T10:14:12.774 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39637.log 2024-09-13T10:14:12.774 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34306.log.gz 2024-09-13T10:14:12.774 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.41095.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40913.log 2024-09-13T10:14:12.774 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41095.log.gz 2024-09-13T10:14:12.774 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38937.log 2024-09-13T10:14:12.774 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39637.log.gz 2024-09-13T10:14:12.775 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36156.log 2024-09-13T10:14:12.775 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40913.log.gz 2024-09-13T10:14:12.775 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.38937.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37760.log 2024-09-13T10:14:12.775 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38937.log.gz 2024-09-13T10:14:12.775 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41329.log 2024-09-13T10:14:12.775 INFO:teuthology.orchestra.run.smithi073.stderr: 19.1% -- replaced with /var/log/ceph/ceph-client.admin.36156.log.gz 2024-09-13T10:14:12.775 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39624.log 2024-09-13T10:14:12.775 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37760.log.gz 2024-09-13T10:14:12.775 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32627.log 2024-09-13T10:14:12.775 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41329.log.gz 2024-09-13T10:14:12.775 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33408.log 2024-09-13T10:14:12.775 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39624.log.gz 2024-09-13T10:14:12.775 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26460.log 2024-09-13T10:14:12.776 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32627.log.gz 2024-09-13T10:14:12.776 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38963.log 2024-09-13T10:14:12.776 INFO:teuthology.orchestra.run.smithi073.stderr: 19.9% -- replaced with /var/log/ceph/ceph-client.admin.33408.log.gz 2024-09-13T10:14:12.776 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.26460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33271.log 2024-09-13T10:14:12.776 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26460.log.gz 2024-09-13T10:14:12.776 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.38963.log: -- /var/log/ceph/ceph-client.admin.33541.log 2024-09-13T10:14:12.776 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38963.log.gz 2024-09-13T10:14:12.776 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35640.log 2024-09-13T10:14:12.776 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33271.log.gz 2024-09-13T10:14:12.776 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.33541.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38620.log 18.7% -- replaced with /var/log/ceph/ceph-client.admin.33541.log.gz 2024-09-13T10:14:12.776 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-09-13T10:14:12.776 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35640.log: 18.3% -- replaced with /var/log/ceph/ceph-client.admin.35640.log.gz 2024-09-13T10:14:12.776 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32250.log 2024-09-13T10:14:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38807.log 2024-09-13T10:14:12.777 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38620.log.gz 2024-09-13T10:14:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38542.log 2024-09-13T10:14:12.777 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32250.log.gz 2024-09-13T10:14:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34444.log 2024-09-13T10:14:12.777 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38807.log.gz 2024-09-13T10:14:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32208.log 2024-09-13T10:14:12.777 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38542.log.gz 2024-09-13T10:14:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34542.log 2024-09-13T10:14:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34444.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34444.log.gz 2024-09-13T10:14:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33187.log 2024-09-13T10:14:12.777 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32208.log.gz 2024-09-13T10:14:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39647.log 2024-09-13T10:14:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34542.log: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.34542.log.gz 2024-09-13T10:14:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31643.log 2024-09-13T10:14:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33187.log: 18.3% -- replaced with /var/log/ceph/ceph-client.admin.33187.log.gz 2024-09-13T10:14:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40055.log 2024-09-13T10:14:12.778 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39647.log.gz 2024-09-13T10:14:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38307.log 2024-09-13T10:14:12.778 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31643.log.gz 2024-09-13T10:14:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31748.log 2024-09-13T10:14:12.778 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40055.log.gz 2024-09-13T10:14:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38859.log 2024-09-13T10:14:12.778 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38307.log.gz 2024-09-13T10:14:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37985.log 2024-09-13T10:14:12.779 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31748.log.gz 2024-09-13T10:14:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34922.log 2024-09-13T10:14:12.779 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38859.log.gz 2024-09-13T10:14:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37579.log 2024-09-13T10:14:12.779 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37985.log.gz 2024-09-13T10:14:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33865.log 2024-09-13T10:14:12.779 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34922.log.gz 2024-09-13T10:14:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31759.log 2024-09-13T10:14:12.779 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37579.log.gz 2024-09-13T10:14:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32938.log 2024-09-13T10:14:12.779 INFO:teuthology.orchestra.run.smithi073.stderr: 17.4% -- replaced with /var/log/ceph/ceph-client.admin.33865.log.gz 2024-09-13T10:14:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37049.log 2024-09-13T10:14:12.779 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31759.log.gz 2024-09-13T10:14:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37496.log 2024-09-13T10:14:12.780 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32938.log.gz 2024-09-13T10:14:12.780 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38989.log 2024-09-13T10:14:12.780 INFO:teuthology.orchestra.run.smithi073.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.37049.log.gz 2024-09-13T10:14:12.780 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34336.log 2024-09-13T10:14:12.780 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37496.log.gz 2024-09-13T10:14:12.780 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31778.log 2024-09-13T10:14:12.780 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38989.log.gz 2024-09-13T10:14:12.780 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36886.log 2024-09-13T10:14:12.780 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34336.log.gz 2024-09-13T10:14:12.780 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34839.log 2024-09-13T10:14:12.780 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31778.log.gz 2024-09-13T10:14:12.780 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39646.log 2024-09-13T10:14:12.780 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36886.log.gz 2024-09-13T10:14:12.781 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41147.log 2024-09-13T10:14:12.781 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34839.log.gz 2024-09-13T10:14:12.781 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32787.log 2024-09-13T10:14:12.781 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39646.log.gz 2024-09-13T10:14:12.781 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39561.log 2024-09-13T10:14:12.781 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41147.log.gz 2024-09-13T10:14:12.781 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38160.log 2024-09-13T10:14:12.781 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32787.log.gz 2024-09-13T10:14:12.781 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41849.log 2024-09-13T10:14:12.781 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39561.log.gz 2024-09-13T10:14:12.781 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.38160.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39626.log 2024-09-13T10:14:12.781 INFO:teuthology.orchestra.run.smithi073.stderr: 19.0% -- replaced with /var/log/ceph/ceph-client.admin.38160.log.gz 2024-09-13T10:14:12.782 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33162.log 2024-09-13T10:14:12.782 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41849.log.gz 2024-09-13T10:14:12.782 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33791.log 2024-09-13T10:14:12.782 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39626.log.gz 2024-09-13T10:14:12.782 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26574.log 2024-09-13T10:14:12.782 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33162.log.gz 2024-09-13T10:14:12.782 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35322.log 2024-09-13T10:14:12.782 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33791.log.gz 2024-09-13T10:14:12.782 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.26574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40023.log 2024-09-13T10:14:12.782 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26574.log.gz 2024-09-13T10:14:12.782 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41069.log 2024-09-13T10:14:12.782 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35322.log.gz 2024-09-13T10:14:12.782 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40023.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.40023.log.gz -- 2024-09-13T10:14:12.783 INFO:teuthology.orchestra.run.smithi073.stderr: /var/log/ceph/ceph-client.admin.40430.log 2024-09-13T10:14:12.783 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41069.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41069.log.gz 2024-09-13T10:14:12.783 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41667.log 2024-09-13T10:14:12.783 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37902.log 2024-09-13T10:14:12.783 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40430.log.gz 2024-09-13T10:14:12.783 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-13T10:14:12.783 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41667.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41667.log.gz 2024-09-13T10:14:12.783 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38594.log 2024-09-13T10:14:12.783 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37902.log.gz 2024-09-13T10:14:12.783 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41277.log 2024-09-13T10:14:12.783 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41173.log 2024-09-13T10:14:12.783 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38594.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38594.log.gz 2024-09-13T10:14:12.783 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40526.log 2024-09-13T10:14:12.784 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41277.log.gz 2024-09-13T10:14:12.784 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32201.log 2024-09-13T10:14:12.784 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41173.log.gz 2024-09-13T10:14:12.784 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41693.log 2024-09-13T10:14:12.784 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40526.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40526.log.gz 2024-09-13T10:14:12.784 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41017.log 2024-09-13T10:14:12.784 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32201.log.gz 2024-09-13T10:14:12.784 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33008.log 2024-09-13T10:14:12.784 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41693.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41693.log.gz 2024-09-13T10:14:12.784 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39639.log 2024-09-13T10:14:12.784 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41017.log.gz 2024-09-13T10:14:12.784 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38301.log 2024-09-13T10:14:12.784 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33008.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33008.log.gz 2024-09-13T10:14:12.785 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33007.log 2024-09-13T10:14:12.785 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39639.log.gz 2024-09-13T10:14:12.785 INFO:teuthology.orchestra.run.smithi073.stderr: 86.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-13T10:14:12.785 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39640.log 2024-09-13T10:14:12.785 INFO:teuthology.orchestra.run.smithi073.stderr: 19.1% -- replaced with /var/log/ceph/ceph-client.admin.38301.log.gz 2024-09-13T10:14:12.785 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36465.log 2024-09-13T10:14:12.785 INFO:teuthology.orchestra.run.smithi073.stderr: 17.5% -- replaced with /var/log/ceph/ceph-client.admin.33007.log.gz 2024-09-13T10:14:12.785 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32147.log 2024-09-13T10:14:12.785 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39640.log.gz 2024-09-13T10:14:12.785 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32132.log 2024-09-13T10:14:12.785 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36465.log.gz 2024-09-13T10:14:12.785 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36021.log 2024-09-13T10:14:12.786 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32147.log.gz 2024-09-13T10:14:12.786 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39327.log 2024-09-13T10:14:12.786 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32132.log.gz 2024-09-13T10:14:12.786 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31772.log 2024-09-13T10:14:12.786 INFO:teuthology.orchestra.run.smithi073.stderr: 18.3% -- replaced with /var/log/ceph/ceph-client.admin.36021.log.gz 2024-09-13T10:14:12.786 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37139.log 2024-09-13T10:14:12.786 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39327.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39327.log.gz 2024-09-13T10:14:12.786 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38182.log 2024-09-13T10:14:12.786 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31772.log.gz 2024-09-13T10:14:12.786 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35428.log 2024-09-13T10:14:12.786 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37139.log.gz 2024-09-13T10:14:12.786 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31781.log 2024-09-13T10:14:12.786 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38182.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38182.log.gz 2024-09-13T10:14:12.787 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39623.log 2024-09-13T10:14:12.787 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35428.log.gz 2024-09-13T10:14:12.787 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39041.log 2024-09-13T10:14:12.787 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31781.log.gz 2024-09-13T10:14:12.787 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42008.log 2024-09-13T10:14:12.787 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39623.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39623.log.gz 2024-09-13T10:14:12.787 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41745.log 2024-09-13T10:14:12.787 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39041.log.gz 2024-09-13T10:14:12.787 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31751.log 2024-09-13T10:14:12.787 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42008.log.gz 2024-09-13T10:14:12.787 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37500.log 2024-09-13T10:14:12.787 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41745.log.gz 2024-09-13T10:14:12.787 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38251.log 2024-09-13T10:14:12.788 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31751.log.gz 2024-09-13T10:14:12.788 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32583.log 2024-09-13T10:14:12.788 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37500.log.gz 2024-09-13T10:14:12.788 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40035.log 2024-09-13T10:14:12.788 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38251.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38251.log.gz 2024-09-13T10:14:12.788 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38729.log 2024-09-13T10:14:12.788 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32583.log.gz 2024-09-13T10:14:12.788 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34147.log 2024-09-13T10:14:12.788 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40035.log.gz 2024-09-13T10:14:12.788 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39379.log 2024-09-13T10:14:12.788 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38729.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38729.log.gz 2024-09-13T10:14:12.788 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34175.log 2024-09-13T10:14:12.788 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34147.log.gz 2024-09-13T10:14:12.788 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41615.log 2024-09-13T10:14:12.789 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39379.log.gz 2024-09-13T10:14:12.789 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41459.log 2024-09-13T10:14:12.789 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34175.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34175.log.gz 2024-09-13T10:14:12.789 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33732.log 2024-09-13T10:14:12.789 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41615.log.gz 2024-09-13T10:14:12.789 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34473.log 2024-09-13T10:14:12.789 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41459.log.gz 2024-09-13T10:14:12.789 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36912.log 2024-09-13T10:14:12.789 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33732.log: 18.7% -- replaced with /var/log/ceph/ceph-client.admin.33732.log.gz 2024-09-13T10:14:12.789 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32608.log 2024-09-13T10:14:12.789 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34473.log.gz 2024-09-13T10:14:12.789 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35917.log 2024-09-13T10:14:12.789 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36912.log.gz 2024-09-13T10:14:12.790 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41641.log 2024-09-13T10:14:12.790 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32608.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32608.log.gz 2024-09-13T10:14:12.790 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34224.log 2024-09-13T10:14:12.790 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35917.log.gz 2024-09-13T10:14:12.790 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36130.log 2024-09-13T10:14:12.790 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41641.log.gz 2024-09-13T10:14:12.790 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35811.log 2024-09-13T10:14:12.790 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34224.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34224.log.gz 2024-09-13T10:14:12.790 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32202.log 2024-09-13T10:14:12.790 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36130.log.gz 2024-09-13T10:14:12.790 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40527.log 2024-09-13T10:14:12.790 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35811.log.gz 2024-09-13T10:14:12.791 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33944.log 2024-09-13T10:14:12.791 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32202.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32202.log.gz 2024-09-13T10:14:12.791 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40527.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-13T10:14:12.791 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40527.log.gz 2024-09-13T10:14:12.791 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31617.log 2024-09-13T10:14:12.791 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33944.log.gz 2024-09-13T10:14:12.791 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35513.log 2024-09-13T10:14:12.791 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.31617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41485.log 2024-09-13T10:14:12.791 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31617.log.gz 2024-09-13T10:14:12.791 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31752.log 2024-09-13T10:14:12.791 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35513.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35513.log.gz 2024-09-13T10:14:12.791 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5/var/log/ceph/ceph-client.admin.41485.log: --verbose -- /var/log/ceph/ceph-client.admin.37656.log 2024-09-13T10:14:12.791 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41485.log.gz 2024-09-13T10:14:12.791 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38390.log 2024-09-13T10:14:12.792 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31752.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31752.log.gz 2024-09-13T10:14:12.792 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32735.log 2024-09-13T10:14:12.792 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37656.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37656.log.gz 2024-09-13T10:14:12.792 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37016.log 2024-09-13T10:14:12.792 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38390.log.gz 2024-09-13T10:14:12.792 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38703.log 2024-09-13T10:14:12.792 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32735.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32735.log.gz 2024-09-13T10:14:12.792 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37173.log 2024-09-13T10:14:12.792 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37016.log: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-13T10:14:12.792 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37016.log.gz 2024-09-13T10:14:12.792 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39431.log 2024-09-13T10:14:12.792 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38703.log.gz 2024-09-13T10:14:12.793 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34682.log 2024-09-13T10:14:12.793 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37173.log: 18.9% -- replaced with /var/log/ceph/ceph-client.admin.37173.log.gz 2024-09-13T10:14:12.793 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32178.log 2024-09-13T10:14:12.793 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39431.log.gz 2024-09-13T10:14:12.793 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35261.log 2024-09-13T10:14:12.793 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34682.log.gz 2024-09-13T10:14:12.793 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41303.log 2024-09-13T10:14:12.793 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32178.log.gz 2024-09-13T10:14:12.793 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35261.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.35261.log.gz -- 2024-09-13T10:14:12.793 INFO:teuthology.orchestra.run.smithi073.stderr: /var/log/ceph/ceph-client.admin.34051.log 2024-09-13T10:14:12.793 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39223.log 2024-09-13T10:14:12.793 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41303.log.gz 2024-09-13T10:14:12.793 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32174.log 2024-09-13T10:14:12.793 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34051.log.gz 2024-09-13T10:14:12.794 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35568.log 2024-09-13T10:14:12.794 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39223.log.gz 2024-09-13T10:14:12.794 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31775.log 2024-09-13T10:14:12.794 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32174.log.gz 2024-09-13T10:14:12.794 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32860.log 2024-09-13T10:14:12.794 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35568.log.gz 2024-09-13T10:14:12.794 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35348.log 2024-09-13T10:14:12.794 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31775.log.gz 2024-09-13T10:14:12.794 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32860.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.12.log 2024-09-13T10:14:12.794 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32860.log.gz 2024-09-13T10:14:12.794 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35348.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35348.log.gzgzip 2024-09-13T10:14:12.794 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34708.log 2024-09-13T10:14:12.794 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.12.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38015.log 2024-09-13T10:14:12.795 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35724.log 2024-09-13T10:14:12.795 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34708.log.gz 2024-09-13T10:14:12.795 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31747.log 2024-09-13T10:14:12.795 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38015.log.gz 2024-09-13T10:14:12.795 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38911.log 2024-09-13T10:14:12.795 INFO:teuthology.orchestra.run.smithi073.stderr: 18.9% -- replaced with /var/log/ceph/ceph-client.admin.35724.log.gz 2024-09-13T10:14:12.795 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31747.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.31747.log.gz /var/log/ceph/ceph-client.admin.37845.log 2024-09-13T10:14:12.795 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-09-13T10:14:12.795 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41901.log 2024-09-13T10:14:12.795 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38911.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38911.log.gz 2024-09-13T10:14:12.795 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31669.log 2024-09-13T10:14:12.795 INFO:teuthology.orchestra.run.smithi073.stderr: 18.4% -- replaced with /var/log/ceph/ceph-client.admin.37845.log.gz 2024-09-13T10:14:12.795 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41901.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41901.log.gz -5 2024-09-13T10:14:12.796 INFO:teuthology.orchestra.run.smithi073.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41719.log 2024-09-13T10:14:12.796 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.31669.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31771.log 2024-09-13T10:14:12.796 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31669.log.gz 2024-09-13T10:14:12.796 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41719.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41719.log.gz 2024-09-13T10:14:12.796 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34948.log 2024-09-13T10:14:12.796 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37255.log 2024-09-13T10:14:12.796 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31771.log.gz 2024-09-13T10:14:12.796 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39353.log 2024-09-13T10:14:12.796 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34948.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34948.log.gz 2024-09-13T10:14:12.796 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31567.log 2024-09-13T10:14:12.796 INFO:teuthology.orchestra.run.smithi073.stderr: 18.4% -- replaced with /var/log/ceph/ceph-client.admin.37255.log.gz 2024-09-13T10:14:12.796 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39353.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40431.log 2024-09-13T10:14:12.796 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39353.log.gz 2024-09-13T10:14:12.797 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40835.log 2024-09-13T10:14:12.797 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31567.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31567.log.gz 2024-09-13T10:14:12.797 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37688.log 2024-09-13T10:14:12.797 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40431.log.gz 2024-09-13T10:14:12.797 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32154.log 2024-09-13T10:14:12.797 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40835.log.gz 2024-09-13T10:14:12.797 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35781.log 2024-09-13T10:14:12.797 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37688.log.gz 2024-09-13T10:14:12.797 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38781.log 2024-09-13T10:14:12.797 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32154.log.gz 2024-09-13T10:14:12.797 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35781.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35781.log.gz 2024-09-13T10:14:12.797 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41433.log 2024-09-13T10:14:12.797 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32157.log 2024-09-13T10:14:12.797 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38781.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38781.log.gz 2024-09-13T10:14:12.798 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32886.log 2024-09-13T10:14:12.798 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41433.log.gz 2024-09-13T10:14:12.798 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33646.log 2024-09-13T10:14:12.798 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32157.log.gz 2024-09-13T10:14:12.798 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31776.log 2024-09-13T10:14:12.798 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32886.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32886.log.gz 2024-09-13T10:14:12.798 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35083.log 2024-09-13T10:14:12.798 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33646.log.gz 2024-09-13T10:14:12.798 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38672.log 2024-09-13T10:14:12.798 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31776.log.gz 2024-09-13T10:14:12.798 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34075.log 2024-09-13T10:14:12.798 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35083.log: 18.3% -- replaced with /var/log/ceph/ceph-client.admin.35083.log.gz 2024-09-13T10:14:12.799 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31515.log 2024-09-13T10:14:12.799 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38672.log.gz 2024-09-13T10:14:12.799 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40051.log 2024-09-13T10:14:12.799 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34075.log.gz 2024-09-13T10:14:12.799 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41511.log 2024-09-13T10:14:12.799 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31515.log.gz 2024-09-13T10:14:12.799 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38755.log 2024-09-13T10:14:12.799 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40051.log.gz 2024-09-13T10:14:12.799 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33620.log 2024-09-13T10:14:12.799 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41511.log.gz 2024-09-13T10:14:12.799 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36531.log 2024-09-13T10:14:12.799 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38755.log.gz 2024-09-13T10:14:12.799 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33620.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21553.log 2024-09-13T10:14:12.799 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33620.log.gz 2024-09-13T10:14:12.800 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36531.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-13T10:14:12.800 INFO:teuthology.orchestra.run.smithi073.stderr: 18.3% -- replaced with /var/log/ceph/ceph-client.admin.36531.log.gz 2024-09-13T10:14:12.801 INFO:teuthology.orchestra.run.smithi136.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-13T10:14:12.813 INFO:teuthology.orchestra.run.smithi195.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-13T10:14:12.816 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph.tmp-client.admin.21553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32609.log 2024-09-13T10:14:12.816 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21553.log.gz 2024-09-13T10:14:12.816 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26340.log 2024-09-13T10:14:12.816 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32836.log 2024-09-13T10:14:12.816 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32609.log.gz 2024-09-13T10:14:12.816 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.26340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33861.log 2024-09-13T10:14:12.817 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.26340.log.gz 2024-09-13T10:14:12.817 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38646.log 2024-09-13T10:14:12.817 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32836.log.gz 2024-09-13T10:14:12.817 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33861.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33861.log.gz 2024-09-13T10:14:12.817 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39990.log 2024-09-13T10:14:12.817 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41381.log 2024-09-13T10:14:12.817 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38646.log.gz 2024-09-13T10:14:12.817 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37337.log 2024-09-13T10:14:12.817 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39990.log.gz 2024-09-13T10:14:12.817 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41251.log 2024-09-13T10:14:12.817 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41381.log: /var/log/ceph/ceph-client.admin.37337.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41381.log.gz 2024-09-13T10:14:12.817 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37337.log.gz 2024-09-13T10:14:12.817 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39275.log 2024-09-13T10:14:12.818 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40539.log 2024-09-13T10:14:12.818 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41251.log.gz 2024-09-13T10:14:12.818 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35951.log 2024-09-13T10:14:12.818 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39275.log.gz 2024-09-13T10:14:12.818 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40539.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40539.log.gz 2024-09-13T10:14:12.818 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33350.log 2024-09-13T10:14:12.818 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38009.log 2024-09-13T10:14:12.818 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35951.log.gz 2024-09-13T10:14:12.818 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36370.log 2024-09-13T10:14:12.818 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33350.log.gz 2024-09-13T10:14:12.818 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38009.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38009.log.gz 2024-09-13T10:14:12.818 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40580.log 2024-09-13T10:14:12.818 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40064.log 2024-09-13T10:14:12.819 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36370.log.gz 2024-09-13T10:14:12.819 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41771.log 2024-09-13T10:14:12.819 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40580.log.gz 2024-09-13T10:14:12.819 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39119.log 2024-09-13T10:14:12.819 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40064.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40064.log.gz 2024-09-13T10:14:12.819 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31768.log 2024-09-13T10:14:12.819 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41771.log.gz 2024-09-13T10:14:12.819 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39612.log 2024-09-13T10:14:12.819 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39119.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39119.log.gz 2024-09-13T10:14:12.819 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40525.log 2024-09-13T10:14:12.819 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31768.log.gz 2024-09-13T10:14:12.819 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35214.log 2024-09-13T10:14:12.819 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39612.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39612.log.gz 2024-09-13T10:14:12.819 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32557.log 2024-09-13T10:14:12.820 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40525.log.gz 2024-09-13T10:14:12.820 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33537.log 2024-09-13T10:14:12.820 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35214.log: 18.7% -- replaced with /var/log/ceph/ceph-client.admin.35214.log.gz/var/log/ceph/ceph-client.admin.32557.log: 2024-09-13T10:14:12.820 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41927.log 2024-09-13T10:14:12.820 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32557.log.gz 2024-09-13T10:14:12.820 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38060.log 2024-09-13T10:14:12.820 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33537.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33537.log.gz 2024-09-13T10:14:12.820 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32761.log 2024-09-13T10:14:12.820 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41927.log.gz 2024-09-13T10:14:12.820 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38060.log.gz 2024-09-13T10:14:12.820 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35375.log 2024-09-13T10:14:12.820 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32160.log 2024-09-13T10:14:12.820 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32761.log.gz 2024-09-13T10:14:12.821 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31758.log 2024-09-13T10:14:12.821 INFO:teuthology.orchestra.run.smithi073.stderr: 19.6% -- replaced with /var/log/ceph/ceph-client.admin.35375.log.gz 2024-09-13T10:14:12.821 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40482.log 2024-09-13T10:14:12.821 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32160.log.gz 2024-09-13T10:14:12.821 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39619.log 2024-09-13T10:14:12.821 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31758.log.gz 2024-09-13T10:14:12.821 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31766.log 2024-09-13T10:14:12.821 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40482.log: /var/log/ceph/ceph-client.admin.39619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36630.log 2024-09-13T10:14:12.821 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39619.log.gz 2024-09-13T10:14:12.821 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40482.log.gz 2024-09-13T10:14:12.821 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31766.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31766.log.gz 2024-09-13T10:14:12.821 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36829.log 2024-09-13T10:14:12.821 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40030.log 2024-09-13T10:14:12.822 INFO:teuthology.orchestra.run.smithi073.stderr: 18.7% -- replaced with /var/log/ceph/ceph-client.admin.36630.log.gz 2024-09-13T10:14:12.822 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33485.log 2024-09-13T10:14:12.822 INFO:teuthology.orchestra.run.smithi073.stderr: 19.5% -- replaced with /var/log/ceph/ceph-client.admin.36829.log.gz 2024-09-13T10:14:12.822 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40030.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40030.log.gz 2024-09-13T10:14:12.822 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36803.log 2024-09-13T10:14:12.822 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34396.log 2024-09-13T10:14:12.822 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33485.log.gz 2024-09-13T10:14:12.822 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41537.log 2024-09-13T10:14:12.822 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36803.log.gz 2024-09-13T10:14:12.822 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37298.log 2024-09-13T10:14:12.822 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34396.log: /var/log/ceph/ceph-client.admin.41537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34501.log 2024-09-13T10:14:12.822 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41537.log.gz 2024-09-13T10:14:12.822 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34396.log.gz 2024-09-13T10:14:12.823 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37298.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37298.log.gz 2024-09-13T10:14:12.823 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35806.log 2024-09-13T10:14:12.823 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36263.log 2024-09-13T10:14:12.823 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34501.log.gz 2024-09-13T10:14:12.823 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35806.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35806.log.gz 2024-09-13T10:14:12.823 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36263.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36263.log.gz 2024-09-13T10:14:12.861 INFO:teuthology.orchestra.run.smithi181.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-13T10:14:12.998 INFO:teuthology.orchestra.run.smithi136.stderr: 90.7% -- replaced with /var/log/ceph/ceph-osd.9.log.gz 2024-09-13T10:14:13.024 INFO:teuthology.orchestra.run.smithi195.stderr: 90.8% -- replaced with /var/log/ceph/ceph-osd.15.log.gz 2024-09-13T10:14:13.113 INFO:teuthology.orchestra.run.smithi073.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-13T10:14:13.145 INFO:teuthology.orchestra.run.smithi136.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-13T10:14:13.228 INFO:teuthology.orchestra.run.smithi195.stderr: 91.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-13T10:14:13.276 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-13T10:14:13.276 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.11 has been restored 2024-09-13T10:14:13.276 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.15 has been restored 2024-09-13T10:14:13.276 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-09-13T10:14:13.276 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-13T10:14:13.276 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-09-13T10:14:13.405 INFO:teuthology.orchestra.run.smithi195.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.11.log.gz 2024-09-13T10:14:13.453 INFO:teuthology.orchestra.run.smithi073.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.12.log.gz 2024-09-13T10:14:13.456 INFO:teuthology.orchestra.run.smithi136.stderr: 91.4% -- replaced with /var/log/ceph/ceph-osd.13.log.gz 2024-09-13T10:14:13.465 INFO:teuthology.orchestra.run.smithi073.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-13T10:14:13.490 INFO:teuthology.orchestra.run.smithi181.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.10.log.gz 2024-09-13T10:14:13.537 INFO:teuthology.orchestra.run.smithi195.stderr: 91.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-13T10:14:13.538 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-09-13T10:14:13.539 INFO:teuthology.orchestra.run.smithi195.stderr:real 0m0.865s 2024-09-13T10:14:13.539 INFO:teuthology.orchestra.run.smithi195.stderr:user 0m2.474s 2024-09-13T10:14:13.539 INFO:teuthology.orchestra.run.smithi195.stderr:sys 0m0.090s 2024-09-13T10:14:13.551 INFO:teuthology.orchestra.run.smithi073.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.8.log.gz 2024-09-13T10:14:13.558 INFO:teuthology.orchestra.run.smithi181.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-13T10:14:13.627 INFO:teuthology.orchestra.run.smithi073.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-13T10:14:13.629 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-09-13T10:14:13.629 INFO:teuthology.orchestra.run.smithi073.stderr:real 0m1.001s 2024-09-13T10:14:13.629 INFO:teuthology.orchestra.run.smithi073.stderr:user 0m3.827s 2024-09-13T10:14:13.629 INFO:teuthology.orchestra.run.smithi073.stderr:sys 0m0.235s 2024-09-13T10:14:13.825 INFO:teuthology.orchestra.run.smithi136.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-13T10:14:13.826 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-09-13T10:14:13.826 INFO:teuthology.orchestra.run.smithi136.stderr:real 0m1.193s 2024-09-13T10:14:13.826 INFO:teuthology.orchestra.run.smithi136.stderr:user 0m2.837s 2024-09-13T10:14:13.826 INFO:teuthology.orchestra.run.smithi136.stderr:sys 0m0.095s 2024-09-13T10:14:13.828 INFO:teuthology.orchestra.run.smithi181.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.14.log.gz 2024-09-13T10:14:14.355 INFO:teuthology.orchestra.run.smithi181.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-13T10:14:14.356 INFO:teuthology.orchestra.run.smithi181.stderr: 2024-09-13T10:14:14.357 INFO:teuthology.orchestra.run.smithi181.stderr:real 0m1.699s 2024-09-13T10:14:14.357 INFO:teuthology.orchestra.run.smithi181.stderr:user 0m4.538s 2024-09-13T10:14:14.357 INFO:teuthology.orchestra.run.smithi181.stderr:sys 0m0.193s 2024-09-13T10:14:14.357 INFO:tasks.ceph:Archiving logs... 2024-09-13T10:14:14.357 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/log/ceph to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334/remote/smithi073/log 2024-09-13T10:14:14.358 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T10:14:15.086 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334/remote/smithi136/log 2024-09-13T10:14:15.087 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T10:14:15.481 DEBUG:teuthology.misc:Transferring archived files from smithi181:/var/log/ceph to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334/remote/smithi181/log 2024-09-13T10:14:15.482 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T10:14:16.137 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334/remote/smithi195/log 2024-09-13T10:14:16.138 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T10:14:16.509 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T10:14:16.518 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T10:14:16.519 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T10:14:16.521 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T10:14:16.524 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T10:14:16.526 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T10:14:16.639 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T10:14:16.648 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T10:14:16.649 DEBUG:teuthology.orchestra.run.smithi073:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T10:14:16.651 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T10:14:16.654 DEBUG:teuthology.orchestra.run.smithi181:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T10:14:16.656 DEBUG:teuthology.orchestra.run.smithi195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T10:14:16.662 INFO:teuthology.orchestra.run.smithi073.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T10:14:16.662 INFO:teuthology.orchestra.run.smithi073.stdout:============================================================================== 2024-09-13T10:14:16.662 INFO:teuthology.orchestra.run.smithi073.stdout:+hv01.front.sepi 67.205.162.81 3 u 49 64 177 0.115 -0.206 1.462 2024-09-13T10:14:16.662 INFO:teuthology.orchestra.run.smithi073.stdout:*hv02.front.sepi 96.245.170.99 3 u 51 64 177 0.087 -0.717 1.461 2024-09-13T10:14:16.662 INFO:teuthology.orchestra.run.smithi073.stdout:+hv03.front.sepi 74.6.168.72 3 u 48 64 177 0.098 -0.439 1.436 2024-09-13T10:14:16.662 INFO:teuthology.orchestra.run.smithi073.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:14:16.665 INFO:teuthology.orchestra.run.smithi181.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T10:14:16.665 INFO:teuthology.orchestra.run.smithi181.stdout:============================================================================== 2024-09-13T10:14:16.665 INFO:teuthology.orchestra.run.smithi181.stdout:*hv01.front.sepi 67.205.162.81 3 u 55 64 177 0.116 -0.271 0.972 2024-09-13T10:14:16.666 INFO:teuthology.orchestra.run.smithi181.stdout:+hv02.front.sepi 96.245.170.99 3 u 53 64 177 0.072 -1.023 0.790 2024-09-13T10:14:16.666 INFO:teuthology.orchestra.run.smithi181.stdout:+hv03.front.sepi 74.6.168.72 3 u 51 64 177 0.078 -0.444 1.013 2024-09-13T10:14:16.666 INFO:teuthology.orchestra.run.smithi181.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:14:16.666 INFO:teuthology.orchestra.run.smithi136.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T10:14:16.666 INFO:teuthology.orchestra.run.smithi136.stdout:============================================================================== 2024-09-13T10:14:16.666 INFO:teuthology.orchestra.run.smithi136.stdout:+hv01.front.sepi 67.205.162.81 3 u 45 64 177 0.097 +0.164 1.389 2024-09-13T10:14:16.666 INFO:teuthology.orchestra.run.smithi136.stdout:+hv02.front.sepi 96.245.170.99 3 u 49 64 177 0.088 -1.399 0.846 2024-09-13T10:14:16.666 INFO:teuthology.orchestra.run.smithi136.stdout:*hv03.front.sepi 74.6.168.72 3 u 51 64 177 0.116 -0.635 1.119 2024-09-13T10:14:16.666 INFO:teuthology.orchestra.run.smithi136.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:14:16.669 INFO:teuthology.orchestra.run.smithi195.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T10:14:16.669 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================== 2024-09-13T10:14:16.669 INFO:teuthology.orchestra.run.smithi195.stdout:*hv01.front.sepi 67.205.162.81 3 u 50 64 177 0.077 -0.540 0.768 2024-09-13T10:14:16.670 INFO:teuthology.orchestra.run.smithi195.stdout:+hv02.front.sepi 96.245.170.99 3 u 51 64 177 0.074 -1.042 0.792 2024-09-13T10:14:16.670 INFO:teuthology.orchestra.run.smithi195.stdout:+hv03.front.sepi 74.6.168.72 3 u 47 64 177 0.082 -0.272 1.068 2024-09-13T10:14:16.670 INFO:teuthology.orchestra.run.smithi195.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T10:14:16.670 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T10:14:16.680 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T10:14:16.680 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T10:14:16.688 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T10:14:16.696 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T10:14:16.705 INFO:teuthology.task.internal:Duration was 905.707493 seconds 2024-09-13T10:14:16.706 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T10:14:16.789 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T10:14:16.789 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T10:14:16.792 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T10:14:16.795 DEBUG:teuthology.orchestra.run.smithi181:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T10:14:16.797 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T10:14:16.835 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T10:14:16.835 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi073.front.sepia.ceph.com 2024-09-13T10:14:16.835 DEBUG:teuthology.orchestra.run.smithi073:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T10:14:16.883 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-09-13T10:14:16.884 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T10:14:16.894 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi181.front.sepia.ceph.com 2024-09-13T10:14:16.894 DEBUG:teuthology.orchestra.run.smithi181:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T10:14:16.904 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi195.front.sepia.ceph.com 2024-09-13T10:14:16.904 DEBUG:teuthology.orchestra.run.smithi195:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T10:14:16.915 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T10:14:16.915 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T10:14:16.928 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T10:14:16.944 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T10:14:16.949 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T10:14:16.985 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T10:14:16.986 DEBUG:teuthology.orchestra.run.smithi073:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T10:14:17.001 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T10:14:17.013 DEBUG:teuthology.orchestra.run.smithi181:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T10:14:17.023 DEBUG:teuthology.orchestra.run.smithi195:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T10:14:17.096 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T10:14:17.228 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T10:14:17.228 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T10:14:17.242 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T10:14:17.256 DEBUG:teuthology.orchestra.run.smithi181:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T10:14:17.270 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T10:14:17.285 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T10:14:17.295 DEBUG:teuthology.orchestra.run.smithi073:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T10:14:17.298 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T10:14:17.300 DEBUG:teuthology.orchestra.run.smithi181:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T10:14:17.308 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = core 2024-09-13T10:14:17.310 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-09-13T10:14:17.313 DEBUG:teuthology.orchestra.run.smithi195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T10:14:17.323 INFO:teuthology.orchestra.run.smithi181.stdout:kernel.core_pattern = core 2024-09-13T10:14:17.338 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = core 2024-09-13T10:14:17.406 DEBUG:teuthology.orchestra.run.smithi073:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T10:14:17.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:14:17.451 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T10:14:17.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:14:17.457 DEBUG:teuthology.orchestra.run.smithi181:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T10:14:17.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:14:17.462 DEBUG:teuthology.orchestra.run.smithi195:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T10:14:17.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T10:14:17.468 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T10:14:17.478 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T10:14:17.479 DEBUG:teuthology.misc:Transferring archived files from smithi073:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334/remote/smithi073 2024-09-13T10:14:17.479 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T10:14:17.515 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334/remote/smithi136 2024-09-13T10:14:17.516 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T10:14:17.538 DEBUG:teuthology.misc:Transferring archived files from smithi181:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334/remote/smithi181 2024-09-13T10:14:17.538 DEBUG:teuthology.orchestra.run.smithi181:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T10:14:17.560 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/akupczyk-2024-09-13_09:37:56-rados-wip-aclamk-cbt-combined-distro-default-smithi/7903334/remote/smithi195 2024-09-13T10:14:17.561 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T10:14:17.583 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T10:14:17.583 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T10:14:17.586 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T10:14:17.588 DEBUG:teuthology.orchestra.run.smithi181:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T10:14:17.601 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T10:14:17.628 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T10:14:17.638 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T10:14:17.638 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T10:14:17.652 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T10:14:17.652 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T10:14:17.654 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T10:14:17.657 DEBUG:teuthology.orchestra.run.smithi181:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T10:14:17.658 INFO:teuthology.orchestra.run.smithi073.stdout: 658466 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 10:14 /home/ubuntu/cephtest 2024-09-13T10:14:17.659 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T10:14:17.661 INFO:teuthology.orchestra.run.smithi136.stdout: 658637 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 10:14 /home/ubuntu/cephtest 2024-09-13T10:14:17.662 INFO:teuthology.orchestra.run.smithi181.stdout: 658462 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 10:14 /home/ubuntu/cephtest 2024-09-13T10:14:17.676 INFO:teuthology.orchestra.run.smithi195.stdout: 658640 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 10:14 /home/ubuntu/cephtest 2024-09-13T10:14:17.678 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T10:14:17.688 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T10:14:17.762 INFO:teuthology.run:Summary data: description: rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-5} backoff/peering_and_degraded ceph clusters/{fixed-4 openstack} crc-failures/default d-balancer/crush-compat mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-low-osd-mem-target} rados supported-random-distro$/{ubuntu_latest} thrashers/careful_host thrashosds-health workloads/dedup-io-mixed} duration: 905.7074933052063 flavor: default owner: scheduled_akupczyk@teuthology success: true 2024-09-13T10:14:17.762 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T10:14:17.836 INFO:teuthology.run:pass